2024-04-22T19:42:47.628 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-22T19:42:47.628 DEBUG:teuthology.run:Teuthology command: teuthology --name cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi --archive /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525 --description rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} --owner scheduled_cbodley@teuthology --verbose -- /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/orig.config.yaml 2024-04-22T19:42:47.676 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T19:42:47.771 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525 branch: wip-65551 description: rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} email: null first_in_suite: false job_id: '7668525' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-65551 ceph: conf: client: debug rgw: 20 rgw bucket counters cache: true rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw s3 auth use sts: true rgw sts key: abcdefghijklmnop rgw torrent flag: true rgw user counters cache: true setgroup: ceph setuser: ceph global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 rgw override bucket index max shards: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled - \(OSD_SLOW_PING_TIME sha1: 9ed8263edff7e87907bf24b054e5c7144c194dbe ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9ed8263edff7e87907bf24b054e5c7144c194dbe openssl_keys: rgw.client.0: ca: root client: client.0 embed-key: true root: client: client.0 cn: teuthology install: - client.0 key-type: rsa:4096 rgw: client.0: ssl certificate: rgw.client.0 valgrind: - --tool=memcheck - --max-threads=1024 compression type: random ec-data-pool: true frontend: beast inline data: false storage classes: LUKEWARM, FROZEN s3tests: accounts: iam alt root: RGW99999999999999999 iam root: RGW88888888888888888 s3 tenant: RGW33333333333333333 force-branch: ceph-master slow_backend: true storage classes: LUKEWARM, FROZEN selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-65551 sha1: 9ed8263edff7e87907bf24b054e5c7144c194dbe owner: scheduled_cbodley@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 7738 sha1: 9ed8263edff7e87907bf24b054e5c7144c194dbe sleep_before_teardown: 0 suite: rgw suite_branch: wip-65551 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9ed8263edff7e87907bf24b054e5c7144c194dbe targets: smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJSCkFNzF6ZvVnumrKIjSS6/2qOd60SruNj/CDdUaDztzMmFgqo2+kgONtxBG4u9h14txovdnHTbzSRMJ1F78DU= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1p9ccY9F1b48i+zlg0avenm21Ad/5kMNXPeMFgLexvKge3oAIUxDNWtttOMQZ0FQmAJsbIkeSjsllTxCSr6vI= tasks: - install: extra_system_packages: deb: - s3cmd - maven rpm: - s3cmd - maven - ceph: null - openssl_keys: null - rgw: client.0: null - tox: - client.0 - workunit: clients: client.0: - rgw/run-bucket-check.sh - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - cls/test_cls_rgw_gc.sh - cls/test_cls_rgw_stats.sh - cls/test_cls_cmpomap.sh - cls/test_cls_2pc_queue.sh - cls/test_cls_user.sh - rgw/test_rgw_gc_log.sh - rgw/test_rgw_obj.sh - rgw/test_librgw_file.sh - rgw/test_awssdkv4_sig.sh - workunit: clients: client.0: - rgw/test_rgw_s3_mp_reupload.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - workunit: clients: client.0: - rgw/run-reshard.sh - s3tests-java: client.0: force-branch: ceph-master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: rgw_server: client.0 - workunit: clients: client.0: - rgw/run-versioning.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-22_19:31:26 tube: smithi user: cbodley verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-22T19:42:47.771 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa; will attempt to use it 2024-04-22T19:42:47.772 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks 2024-04-22T19:42:47.772 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-22T19:42:47.774 INFO:teuthology.task.internal:Checking packages... 2024-04-22T19:42:47.798 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9ed8263edff7e87907bf24b054e5c7144c194dbe' 2024-04-22T19:42:47.798 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-22T19:42:47.798 INFO:teuthology.packaging:ref: None 2024-04-22T19:42:47.798 INFO:teuthology.packaging:tag: None 2024-04-22T19:42:47.799 INFO:teuthology.packaging:branch: wip-65551 2024-04-22T19:42:47.799 INFO:teuthology.packaging:sha1: 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:42:47.799 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-65551 2024-04-22T19:42:47.946 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3152.g9ed8263e 2024-04-22T19:42:47.948 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-22T19:42:47.955 INFO:teuthology.task.internal:no buildpackages task found 2024-04-22T19:42:47.955 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-22T19:42:47.992 INFO:teuthology.task.internal:Saving configuration 2024-04-22T19:42:48.013 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-22T19:42:48.019 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T19:42:48.045 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 19:32:49.616780', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJSCkFNzF6ZvVnumrKIjSS6/2qOd60SruNj/CDdUaDztzMmFgqo2+kgONtxBG4u9h14txovdnHTbzSRMJ1F78DU='} 2024-04-22T19:42:48.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 19:32:49.615539', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1p9ccY9F1b48i+zlg0avenm21Ad/5kMNXPeMFgLexvKge3oAIUxDNWtttOMQZ0FQmAJsbIkeSjsllTxCSr6vI='} 2024-04-22T19:42:48.069 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-22T19:42:48.075 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-22T19:42:48.075 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-22T19:42:48.075 INFO:teuthology.run_tasks:Running task console_log... 2024-04-22T19:42:48.167 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdfe4d11280>, signals=[15]) 2024-04-22T19:42:48.168 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-22T19:42:48.175 INFO:teuthology.task.internal:Opening connections... 2024-04-22T19:42:48.175 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T19:42:48.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T19:42:48.254 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T19:42:48.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T19:42:48.331 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-22T19:42:48.337 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-04-22T19:42:48.355 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-04-22T19:42:48.355 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-04-22T19:42:48.412 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-04-22T19:42:48.412 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-04-22T19:42:48.412 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-04-22T19:42:48.412 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T19:42:48.413 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T19:42:48.414 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-04-22T19:42:48.439 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-04-22T19:42:48.458 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-04-22T19:42:48.458 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-04-22T19:42:48.514 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-04-22T19:42:48.514 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-22T19:42:48.515 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-22T19:42:48.516 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-04-22T19:42:48.539 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-22T19:42:48.549 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-22T19:42:48.575 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-22T19:42:48.576 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-04-22T19:42:48.578 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-04-22T19:42:48.595 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-22T19:42:48.600 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-22T19:42:48.600 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-04-22T19:42:48.635 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-04-22T19:42:48.757 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-22T19:42:48.814 INFO:teuthology.run_tasks:Running task kernel... 2024-04-22T19:42:48.831 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-22T19:42:48.831 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-22T19:42:48.831 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T19:42:48.832 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-22T19:42:48.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T19:42:48.848 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-04-22T19:42:48.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T19:42:48.849 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-04-22T19:42:48.905 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-437.el9.x86_64 2024-04-22T19:42:48.906 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.14.0-437.el9.x86_64 2024-04-22T19:42:48.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum install -y kernel 2024-04-22T19:42:48.907 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-437.el9.x86_64 2024-04-22T19:42:48.907 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-437.el9.x86_64 2024-04-22T19:42:48.907 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-04-22T19:42:51.349 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-04-22T19:42:53.358 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-22T19:42:57.617 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-22T19:42:59.151 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 16 kB 00:00 2024-04-22T19:42:59.877 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 896 kB/s | 8.0 MB 00:09 2024-04-22T19:43:01.828 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-22T19:43:03.685 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 21 MB 00:04 2024-04-22T19:43:05.820 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.2 MB 00:00 2024-04-22T19:43:07.404 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-04-22T19:43:08.743 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-22T19:43:09.216 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-22T19:43:10.616 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T19:43:10.616 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T19:43:10.700 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-22T19:43:10.701 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2024-04-22T19:43:10.701 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-22T19:43:10.823 DEBUG:teuthology.orchestra.run.smithi164:> echo no | sudo yum reinstall kernel || true 2024-04-22T19:43:11.328 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 07:43:08 PM UTC. 2024-04-22T19:43:11.449 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-22T19:43:11.465 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-22T19:43:11.465 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-04-22T19:43:11.465 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-22T19:43:11.465 INFO:teuthology.orchestra.run.smithi164.stdout:Reinstalling: 2024-04-22T19:43:11.465 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T19:43:11.465 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:43:11.466 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-22T19:43:11.466 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-22T19:43:11.466 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:43:11.466 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 255 k 2024-04-22T19:43:11.466 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 0 2024-04-22T19:43:11.466 INFO:teuthology.orchestra.run.smithi164.stderr:Operation aborted. 2024-04-22T19:43:11.522 INFO:teuthology.orchestra.run.smithi164.stdout:Is this ok [y/N]: 2024-04-22T19:43:11.523 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum reinstall -y kernel || true 2024-04-22T19:43:12.022 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:04 ago on Mon 22 Apr 2024 07:43:08 PM UTC. 2024-04-22T19:43:12.140 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout:Reinstalling: 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-04-22T19:43:12.141 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:43:12.142 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 255 k 2024-04-22T19:43:12.142 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 0 2024-04-22T19:43:12.142 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-22T19:43:14.020 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-437.el9.x86_64.rpm 159 kB/s | 255 kB 00:01 2024-04-22T19:43:14.020 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-22T19:43:14.020 INFO:teuthology.orchestra.run.smithi164.stdout:Total 136 kB/s | 255 kB 00:01 2024-04-22T19:43:14.043 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-22T19:43:14.231 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-22T19:43:14.232 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-22T19:43:14.237 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-22T19:43:14.237 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-22T19:43:14.255 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-22T19:43:14.370 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-22T19:43:14.491 INFO:teuthology.orchestra.run.smithi164.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T19:43:14.492 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T19:43:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T19:43:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T19:43:15.897 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T19:43:15.897 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:43:15.898 INFO:teuthology.orchestra.run.smithi164.stdout:Reinstalled: 2024-04-22T19:43:15.898 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T19:43:15.898 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:43:15.898 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-22T19:43:16.085 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T19:43:16.103 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-22T19:43:16.103 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-22T19:43:16.131 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T19:43:16.131 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T19:43:16.131 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T19:43:16.131 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-22T19:43:16.132 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T19:43:16.159 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-04-22T19:43:16.183 DEBUG:teuthology.parallel:result is None 2024-04-22T19:43:16.197 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-22T19:43:16.198 INFO:teuthology.orchestra.run.smithi136.stdout:Nothing to do. 2024-04-22T19:43:16.198 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-22T19:43:16.321 DEBUG:teuthology.orchestra.run.smithi136:> echo no | sudo yum reinstall kernel || true 2024-04-22T19:43:16.816 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:02 ago on Mon 22 Apr 2024 07:43:14 PM UTC. 2024-04-22T19:43:16.934 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-22T19:43:16.946 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T19:43:16.946 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout:Reinstalling: 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 255 k 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 0 2024-04-22T19:43:16.947 INFO:teuthology.orchestra.run.smithi136.stderr:Operation aborted. 2024-04-22T19:43:17.011 INFO:teuthology.orchestra.run.smithi136.stdout:Is this ok [y/N]: 2024-04-22T19:43:17.011 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum reinstall -y kernel || true 2024-04-22T19:43:17.503 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:00:03 ago on Mon 22 Apr 2024 07:43:14 PM UTC. 2024-04-22T19:43:17.618 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-22T19:43:17.618 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T19:43:17.618 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Reinstalling: 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 255 k 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 0 2024-04-22T19:43:17.619 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-22T19:43:18.274 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-437.el9.x86_64.rpm 497 kB/s | 255 kB 00:00 2024-04-22T19:43:18.274 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-22T19:43:18.274 INFO:teuthology.orchestra.run.smithi136.stdout:Total 389 kB/s | 255 kB 00:00 2024-04-22T19:43:18.289 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-22T19:43:18.475 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-22T19:43:18.475 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-22T19:43:18.480 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-22T19:43:18.480 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-22T19:43:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-22T19:43:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T19:43:18.711 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T19:43:19.428 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T19:43:19.429 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-22T19:43:20.117 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-22T19:43:20.117 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:43:20.117 INFO:teuthology.orchestra.run.smithi136.stdout:Reinstalled: 2024-04-22T19:43:20.117 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-22T19:43:20.118 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:43:20.118 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-22T19:43:20.315 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-04-22T19:43:20.362 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-22T19:43:20.362 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-22T19:43:20.362 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-22T19:43:20.363 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-22T19:43:20.363 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-22T19:43:20.396 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-04-22T19:43:20.414 DEBUG:teuthology.parallel:result is None 2024-04-22T19:43:20.415 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-22T19:43:20.422 INFO:teuthology.task.internal:Creating test directory... 2024-04-22T19:43:20.422 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T19:43:20.457 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-22T19:43:20.475 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-22T19:43:20.482 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-22T19:43:20.516 INFO:teuthology.task.internal:Creating archive directory... 2024-04-22T19:43:20.516 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T19:43:20.519 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-22T19:43:20.596 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-22T19:43:20.607 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-22T19:43:20.607 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T19:43:20.611 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-22T19:43:20.641 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T19:43:20.650 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T19:43:20.659 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T19:43:20.670 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-22T19:43:20.671 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-22T19:43:20.679 INFO:teuthology.task.internal:Configuring sudo... 2024-04-22T19:43:20.679 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T19:43:20.695 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-22T19:43:20.739 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-22T19:43:20.749 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-22T19:43:20.749 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T19:43:20.758 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-22T19:43:20.797 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T19:43:20.855 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T19:43:20.904 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T19:43:20.985 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T19:43:21.023 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:43:21.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T19:43:21.087 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T19:43:21.130 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-22T19:43:21.187 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T19:43:21.271 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-22T19:43:21.312 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:43:21.312 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-22T19:43:21.378 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-04-22T19:43:21.381 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-04-22T19:43:21.413 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T19:43:21.461 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T19:43:21.813 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-22T19:43:21.822 INFO:teuthology.task.internal:Starting timer... 2024-04-22T19:43:21.822 INFO:teuthology.run_tasks:Running task pcp... 2024-04-22T19:43:21.869 INFO:teuthology.run_tasks:Running task selinux... 2024-04-22T19:43:21.892 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-22T19:43:21.892 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-04-22T19:43:21.943 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-04-22T19:43:21.945 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-04-22T19:43:22.015 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-04-22T19:43:22.016 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-22T19:43:22.017 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-04-22T19:43:22.034 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-04-22T19:43:22.034 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-04-22T19:43:22.063 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-04-22T19:43:22.063 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi136.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-04-22T19:43:22.063 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T19:43:22.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T19:43:22.098 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T19:43:22.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T19:43:22.130 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-22T19:43:22.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-04-22T19:43:22.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-04-22T19:43:22.232 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-22T19:43:22.244 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-22T19:43:22.262 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-22T19:43:22.264 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi136.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-22T19:47:10.984 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi136.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-04-22T19:47:10.985 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-22T19:47:10.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T19:47:11.126 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-04-22T19:47:11.169 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-22T19:47:11.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-22T19:47:11.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-22T19:47:11.285 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-04-22T19:47:11.363 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-04-22T19:47:11.363 INFO:teuthology.run_tasks:Running task clock... 2024-04-22T19:47:11.373 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-22T19:47:11.374 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T19:47:11.375 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T19:47:11.378 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-22T19:47:11.378 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T19:47:11.406 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T19:47:11.420 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T19:47:11.434 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-22T19:47:11.443 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-04-22T19:47:11.447 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-22T19:47:11.458 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-04-22T19:47:11.467 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T19:47:11.474 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-04-22T19:47:11.480 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T19:47:11.481 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-04-22T19:47:11.495 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-22T19:47:11.511 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-22T19:47:11.534 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-04-22T19:47:11.537 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T19:47:11.537 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-04-22T19:47:11.538 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T19:47:11.538 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T19:47:11.538 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T19:47:11.538 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T19:47:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-22T19:47:11.575 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T19:47:11.575 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-22T19:47:11.575 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T19:47:11.575 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T19:47:11.575 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T19:47:11.575 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T19:47:11.575 INFO:teuthology.run_tasks:Running task install... 2024-04-22T19:47:11.584 DEBUG:teuthology.task.install:project ceph 2024-04-22T19:47:11.584 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe'}} 2024-04-22T19:47:11.584 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe'} 2024-04-22T19:47:11.584 INFO:teuthology.task.install:Using flavor: default 2024-04-22T19:47:11.591 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-22T19:47:11.591 INFO:teuthology.task.install:extra packages: [] 2024-04-22T19:47:11.591 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe', 'tag': None, 'wait_for_package': False} 2024-04-22T19:47:11.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:47:11.594 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe', 'tag': None, 'wait_for_package': False} 2024-04-22T19:47:11.594 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:47:11.754 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-65551/9ed8263edff7e87907bf24b054e5c7144c194dbe/centos/9/flavors/default/ 2024-04-22T19:47:11.754 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3152.g9ed8263e 2024-04-22T19:47:11.758 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-65551/9ed8263edff7e87907bf24b054e5c7144c194dbe/centos/9/flavors/default/ 2024-04-22T19:47:11.759 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3152.g9ed8263e 2024-04-22T19:47:11.868 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-65551/9ed8263edff7e87907bf24b054e5c7144c194dbe/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-65551/9ed8263edff7e87907bf24b054e5c7144c194dbe/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-65551/9ed8263edff7e87907bf24b054e5c7144c194dbe/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T19:47:11.868 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:47:11.868 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T19:47:11.893 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, s3cmd, maven, s3cmd, maven, s3cmd, maven on remote rpm x86_64 2024-04-22T19:47:11.894 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9ed8263edff7e87907bf24b054e5c7144c194dbe/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T19:47:11.942 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-65551/9ed8263edff7e87907bf24b054e5c7144c194dbe/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-65551/9ed8263edff7e87907bf24b054e5c7144c194dbe/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-65551/9ed8263edff7e87907bf24b054e5c7144c194dbe/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-22T19:47:11.942 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:47:11.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-22T19:47:11.959 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T19:47:11.971 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, s3cmd, maven, s3cmd, maven, s3cmd, maven, s3cmd, maven on remote rpm x86_64 2024-04-22T19:47:11.971 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9ed8263edff7e87907bf24b054e5c7144c194dbe/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-22T19:47:12.032 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T19:47:12.036 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-22T19:47:12.094 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-04-22T19:47:12.096 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-04-22T19:47:12.111 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-22T19:47:12.174 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-04-22T19:47:12.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-04-22T19:47:12.317 INFO:teuthology.orchestra.run.smithi136.stdout:48 files removed 2024-04-22T19:47:12.345 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd s3cmd maven s3cmd maven s3cmd maven s3cmd maven 2024-04-22T19:47:12.406 INFO:teuthology.orchestra.run.smithi164.stdout:48 files removed 2024-04-22T19:47:12.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd s3cmd maven s3cmd maven s3cmd maven s3cmd maven 2024-04-22T19:47:13.016 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-04-22T19:47:13.017 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 256 kB/s | 83 kB 00:00 2024-04-22T19:47:13.375 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-22T19:47:13.434 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-22T19:47:13.751 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-22T19:47:13.818 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 5.4 kB/s | 1.9 kB 00:00 2024-04-22T19:47:14.587 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-22T19:47:22.710 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 923 kB/s | 8.0 MB 00:08 2024-04-22T19:47:32.001 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-04-22T19:47:36.395 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-22T19:47:38.202 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-22T19:47:39.814 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-22T19:47:44.870 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-22T19:47:46.382 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 862 kB/s | 19 MB 00:22 2024-04-22T19:47:46.979 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T19:47:46.980 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T19:47:47.071 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-04-22T19:47:47.087 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================================== 2024-04-22T19:47:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-04-22T19:47:47.087 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================================== 2024-04-22T19:47:47.087 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 6.6 k 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 5.4 M 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 890 k 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 145 k 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 1.5 M 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 142 k 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 3.5 M 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 7.4 M 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 49 k 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 11 M 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 49 M 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 273 k 2024-04-22T19:47:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 762 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 32 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 746 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 127 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: maven noarch 1:3.6.3-15.el9 appstream 18 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 161 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 321 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 298 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 99 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 85 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 3.1 M 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 171 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: s3cmd noarch 2.4.0-1.el9 epel 206 k 2024-04-22T19:47:47.089 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 3.4 M 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 3.1 M 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: ModemManager-glib x86_64 1.20.2-1.el9 baseos 335 k 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: adwaita-cursor-theme noarch 40.1.1-3.el9 appstream 650 k 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: adwaita-icon-theme noarch 40.1.1-3.el9 appstream 12 M 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: alsa-lib x86_64 1.2.10-2.el9 appstream 523 k 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: apache-commons-cli noarch 1.4-17.el9 appstream 71 k 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: apache-commons-codec noarch 1.15-7.el9 appstream 302 k 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: apache-commons-io noarch 1:2.8.0-8.el9 appstream 283 k 2024-04-22T19:47:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: apache-commons-lang3 noarch 3.12.0-6.el9 appstream 559 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: at-spi2-atk x86_64 2.38.0-4.el9 appstream 87 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: at-spi2-core x86_64 2.40.3-1.el9 appstream 195 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: atinject noarch 1.0.3-6.el9 appstream 23 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: atk x86_64 2.36.0-5.el9 appstream 292 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: avahi-glib x86_64 0.8-20.el9 appstream 15 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: bluez-libs x86_64 5.56-6.el9 baseos 85 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: bubblewrap x86_64 0.4.1-7.el9 baseos 49 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: cdi-api noarch 2.0.2-6.el9 appstream 53 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 21 M 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 34 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 2.4 M 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 249 k 2024-04-22T19:47:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 4.7 M 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 17 M 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 16 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 25 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: colord-libs x86_64 1.4.5-4.el9 appstream 229 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: copy-jdk-configs noarch 4.0-3.el9 appstream 28 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: cups-libs x86_64 1:2.3.3op2-24.el9 baseos 262 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: exempi x86_64 2.6.0-0.2.20211007gite23c213.el9 appstream 525 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: exiv2-libs x86_64 0.27.5-2.el9 appstream 781 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: fdk-aac-free x86_64 2.0.0-8.el9 appstream 325 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: flac-libs x86_64 1.3.3-12.el9 appstream 218 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: flatpak-selinux noarch 1.12.8-1.el9 appstream 21 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: flatpak-session-helper x86_64 1.12.8-1.el9 appstream 74 k 2024-04-22T19:47:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T19:47:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T19:47:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T19:47:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T19:47:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: gdk-pixbuf2-modules x86_64 2.42.6-3.el9 appstream 88 k 2024-04-22T19:47:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: geoclue2 x86_64 2.6.0-7.el9 appstream 124 k 2024-04-22T19:47:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: giflib x86_64 5.2.1-9.el9 appstream 49 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: google-guice noarch 4.2.3-9.el9 appstream 473 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: graphene x86_64 1.10.6-2.el9 appstream 64 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: gsm x86_64 1.0.19-6.el9 appstream 34 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: gstreamer1 x86_64 1.22.1-2.el9 appstream 1.4 M 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: gstreamer1-plugins-base x86_64 1.22.1-2.el9 appstream 2.2 M 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: gtk-update-icon-cache x86_64 3.24.31-2.el9 appstream 35 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: guava noarch 30.1-6.el9 appstream 2.3 M 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: hicolor-icon-theme noarch 0.17-13.el9 appstream 219 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: httpcomponents-client noarch 4.5.13-3.el9 appstream 657 k 2024-04-22T19:47:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: httpcomponents-core noarch 4.4.13-7.el9 appstream 632 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: iso-codes noarch 4.6.0-3.el9 appstream 3.5 M 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: jakarta-annotations noarch 1.3.5-13.el9 appstream 45 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: jansi x86_64 2.3.3-6.el9 appstream 110 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: java-11-openjdk x86_64 1:11.0.18.0.10-3.el9 appstream 440 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: java-11-openjdk-headless x86_64 1:11.0.18.0.10-3.el9 appstream 40 M 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: javapackages-filesystem noarch 6.0.0-4.el9 appstream 13 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: jbigkit-libs x86_64 2.1-23.el9 appstream 53 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: jcl-over-slf4j noarch 1.7.30-13.el9 appstream 24 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: jsoup noarch 1.13.1-10.el9 appstream 377 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: jsr-305 noarch 3.0.2-6.el9 appstream 35 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: lcms2 x86_64 2.12-3.el9 appstream 168 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: libX11-xcb x86_64 1.7.0-9.el9 appstream 11 k 2024-04-22T19:47:47.095 INFO:teuthology.orchestra.run.smithi164.stdout: libXcomposite x86_64 0.4.5-7.el9 appstream 24 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXcursor x86_64 1.2.0-7.el9 appstream 31 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXdamage x86_64 1.1.5-7.el9 appstream 23 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXfixes x86_64 5.0.3-16.el9 appstream 20 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXi x86_64 1.7.10-8.el9 appstream 40 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXinerama x86_64 1.1.4-10.el9 appstream 15 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXrandr x86_64 1.5.2-8.el9 appstream 28 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXtst x86_64 1.2.3-16.el9 appstream 21 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXv x86_64 1.0.11-16.el9 appstream 19 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libXxf86vm x86_64 1.1.4-18.el9 appstream 19 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libasyncns x86_64 0.8-22.el9 appstream 30 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libcanberra x86_64 0.30-27.el9 appstream 86 k 2024-04-22T19:47:47.096 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 163 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libdrm x86_64 2.4.117-1.el9 appstream 161 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libepoxy x86_64 1.5.5-4.el9 appstream 241 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libexif x86_64 0.6.22-6.el9 appstream 435 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libfontenc x86_64 1.1.3-17.el9 appstream 31 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libgexiv2 x86_64 0.12.3-1.el9 appstream 83 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libglvnd x86_64 1:1.3.4-1.el9 appstream 135 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libglvnd-egl x86_64 1:1.3.4-1.el9 appstream 37 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libglvnd-glx x86_64 1:1.3.4-1.el9 appstream 142 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libgsf x86_64 1.14.47-5.el9 appstream 256 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libgxps x86_64 0.3.2-3.el9 appstream 79 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libiptcdata x86_64 1.0.5-9.el9 appstream 62 k 2024-04-22T19:47:47.097 INFO:teuthology.orchestra.run.smithi164.stdout: libldac x86_64 2.0.2.3-10.el9 appstream 41 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: libnotify x86_64 0.7.9-8.el9 appstream 44 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: libogg x86_64 2:1.3.4-6.el9 appstream 34 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: libosinfo x86_64 1.10.0-1.el9 appstream 319 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: libpciaccess x86_64 0.16-7.el9 baseos 27 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 500 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T19:47:47.098 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 5.4 M 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libsbc x86_64 1.4-9.el9 appstream 45 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libsndfile x86_64 1.0.31-8.el9 appstream 206 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libtheora x86_64 1:1.1.1-31.el9 appstream 164 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libtiff x86_64 4.4.0-12.el9 appstream 198 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-ltdl x86_64 2.4.6-45.el9 appstream 37 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libtracker-sparql x86_64 3.1.2-3.el9 appstream 324 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libvisual x86_64 1:0.4.0-34.el9 appstream 144 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libvorbis x86_64 1:1.3.7-5.el9 appstream 193 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libwayland-client x86_64 1.21.0-1.el9 appstream 33 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libwayland-cursor x86_64 1.21.0-1.el9 appstream 19 k 2024-04-22T19:47:47.099 INFO:teuthology.orchestra.run.smithi164.stdout: libwayland-egl x86_64 1.21.0-1.el9 appstream 13 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: libwayland-server x86_64 1.21.0-1.el9 appstream 42 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: libwebp x86_64 1.2.0-8.el9 appstream 277 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: libxkbcommon x86_64 1.0.3-4.el9 appstream 133 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: libxshmfence x86_64 1.3-10.el9 appstream 13 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: low-memory-monitor x86_64 2.1-4.el9 appstream 36 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: lua-posix x86_64 35.0-8.el9 appstream 151 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: maven-lib noarch 1:3.6.3-15.el9 appstream 1.5 M 2024-04-22T19:47:47.100 INFO:teuthology.orchestra.run.smithi164.stdout: maven-openjdk11 noarch 1:3.6.3-15.el9 appstream 9.0 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: maven-resolver noarch 1:1.6.1-10.el9 appstream 517 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: maven-shared-utils noarch 3.3.4-3.el9 appstream 151 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: maven-wagon noarch 3.4.2-7.el9 appstream 112 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: mesa-libEGL x86_64 23.3.3-1.el9 appstream 127 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: mesa-libGL x86_64 23.3.3-1.el9 appstream 168 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: mesa-libgbm x86_64 23.3.3-1.el9 appstream 38 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: mesa-libglapi x86_64 23.3.3-1.el9 appstream 44 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: mkfontscale x86_64 1.2.1-3.el9 appstream 32 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: openjpeg2 x86_64 2.4.0-7.el9 appstream 163 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: opus x86_64 1.3.1-10.el9 appstream 200 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: orc x86_64 0.4.31-6.el9 appstream 184 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: osinfo-db noarch 20231215-1.el9 appstream 521 k 2024-04-22T19:47:47.101 INFO:teuthology.orchestra.run.smithi164.stdout: osinfo-db-tools x86_64 1.10.0-1.el9 appstream 74 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: ostree-libs x86_64 2024.5-2.el9 appstream 488 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-jack-audio-connection-kit-libs x86_64 1.0.1-1.el9 appstream 135 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-libs x86_64 1.0.1-1.el9 appstream 1.9 M 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-cipher noarch 1.7-27.el9 appstream 26 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-classworlds noarch 2.6.0-11.el9 appstream 60 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-containers-component-annotations noarch 2.1.0-10.el9 appstream 18 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-interpolation noarch 1.26-11.el9 appstream 80 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-sec-dispatcher noarch 1.4-37.el9 appstream 34 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-utils noarch 3.3.0-10.el9 appstream 254 k 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: poppler x86_64 21.01.0-19.el9 appstream 1.1 M 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: poppler-data noarch 0.4.9-9.el9 appstream 1.9 M 2024-04-22T19:47:47.102 INFO:teuthology.orchestra.run.smithi164.stdout: poppler-glib x86_64 21.01.0-19.el9 appstream 152 k 2024-04-22T19:47:47.103 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T19:47:47.103 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T19:47:47.103 INFO:teuthology.orchestra.run.smithi164.stdout: publicsuffix-list noarch 20210518-3.el9 appstream 83 k 2024-04-22T19:47:47.103 INFO:teuthology.orchestra.run.smithi164.stdout: pulseaudio-libs x86_64 15.0-2.el9 appstream 679 k 2024-04-22T19:47:47.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T19:47:47.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T19:47:47.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T19:47:47.103 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 45 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 135 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T19:47:47.104 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T19:47:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T19:47:47.106 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T19:47:47.107 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: rtkit x86_64 0.11-28.el9 appstream 56 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: sisu noarch 1:0.3.4-10.el9 appstream 510 k 2024-04-22T19:47:47.108 INFO:teuthology.orchestra.run.smithi164.stdout: slf4j noarch 1.7.30-13.el9 appstream 69 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: sound-theme-freedesktop noarch 0.8-17.el9 appstream 383 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: totem-pl-parser x86_64 3.26.6-2.el9 appstream 152 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: tracker x86_64 3.1.2-3.el9 appstream 555 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: ttmkfdir x86_64 3.0.9-65.el9 appstream 53 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: tzdata-java noarch 2024a-2.el9 appstream 230 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: upower x86_64 0.99.11-11.el9 appstream 173 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: webrtc-audio-processing x86_64 0.3.1-8.el9 appstream 306 k 2024-04-22T19:47:47.109 INFO:teuthology.orchestra.run.smithi164.stdout: wireplumber x86_64 0.4.14-1.el9 appstream 96 k 2024-04-22T19:47:47.110 INFO:teuthology.orchestra.run.smithi164.stdout: wireplumber-libs x86_64 0.4.14-1.el9 appstream 358 k 2024-04-22T19:47:47.110 INFO:teuthology.orchestra.run.smithi164.stdout: xdg-dbus-proxy x86_64 0.1.3-1.el9 appstream 42 k 2024-04-22T19:47:47.110 INFO:teuthology.orchestra.run.smithi164.stdout: xdg-desktop-portal x86_64 1.12.6-1.el9 appstream 385 k 2024-04-22T19:47:47.110 INFO:teuthology.orchestra.run.smithi164.stdout: xkeyboard-config noarch 2.33-2.el9 appstream 859 k 2024-04-22T19:47:47.110 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T19:47:47.110 INFO:teuthology.orchestra.run.smithi164.stdout: xorg-x11-fonts-Type1 noarch 7.5-33.el9 appstream 505 k 2024-04-22T19:47:47.111 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-04-22T19:47:47.111 INFO:teuthology.orchestra.run.smithi164.stdout: dconf x86_64 0.40.0-6.el9 appstream 113 k 2024-04-22T19:47:47.111 INFO:teuthology.orchestra.run.smithi164.stdout: exiv2 x86_64 0.27.5-2.el9 appstream 980 k 2024-04-22T19:47:47.111 INFO:teuthology.orchestra.run.smithi164.stdout: flatpak x86_64 1.12.8-1.el9 appstream 1.7 M 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: gtk3 x86_64 3.24.31-2.el9 appstream 4.9 M 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: java-11-openjdk-devel x86_64 1:11.0.18.0.10-3.el9 appstream 3.3 M 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: libcanberra-gtk3 x86_64 0.30-27.el9 appstream 32 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: p11-kit-server x86_64 0.25.3-2.el9 appstream 246 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire x86_64 1.0.1-1.el9 appstream 107 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-alsa x86_64 1.0.1-1.el9 appstream 57 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-jack-audio-connection-kit x86_64 1.0.1-1.el9 appstream 9.1 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-pulseaudio x86_64 1.0.1-1.el9 appstream 192 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: tracker-miners x86_64 3.1.2-4.el9 appstream 944 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: xdg-desktop-portal-gtk x86_64 1.12.0-3.el9 appstream 134 k 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:47:47.112 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-04-22T19:47:47.113 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================================== 2024-04-22T19:47:47.113 INFO:teuthology.orchestra.run.smithi164.stdout:Install 280 Packages 2024-04-22T19:47:47.113 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 2 Packages 2024-04-22T19:47:47.113 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:47:47.113 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 305 M 2024-04-22T19:47:47.114 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-04-22T19:47:48.063 INFO:teuthology.orchestra.run.smithi164.stdout:(1/282): ceph-19.0.0-3152.g9ed8263e.el7.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-22T19:47:48.713 INFO:teuthology.orchestra.run.smithi164.stdout:(2/282): ceph-fuse-19.0.0-3152.g9ed8263e.el7.x8 1.3 MB/s | 890 kB 00:00 2024-04-22T19:47:48.855 INFO:teuthology.orchestra.run.smithi164.stdout:(3/282): ceph-immutable-object-cache-19.0.0-315 1.0 MB/s | 145 kB 00:00 2024-04-22T19:47:49.055 INFO:teuthology.orchestra.run.smithi164.stdout:(4/282): ceph-base-19.0.0-3152.g9ed8263e.el7.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-22T19:47:49.322 INFO:teuthology.orchestra.run.smithi164.stdout:(5/282): ceph-mgr-19.0.0-3152.g9ed8263e.el7.x86 5.6 MB/s | 1.5 MB 00:00 2024-04-22T19:47:49.589 INFO:teuthology.orchestra.run.smithi164.stdout:(6/282): ceph-mds-19.0.0-3152.g9ed8263e.el7.x86 3.2 MB/s | 2.4 MB 00:00 2024-04-22T19:47:49.897 INFO:teuthology.orchestra.run.smithi164.stdout:(7/282): ceph-mon-19.0.0-3152.g9ed8263e.el7.x86 8.2 MB/s | 4.7 MB 00:00 2024-04-22T19:47:51.181 INFO:teuthology.orchestra.run.smithi164.stdout:(8/282): ceph-radosgw-19.0.0-3152.g9ed8263e.el7 8.3 MB/s | 11 MB 00:01 2024-04-22T19:47:51.281 INFO:teuthology.orchestra.run.smithi164.stdout:(9/282): ceph-selinux-19.0.0-3152.g9ed8263e.el7 252 kB/s | 25 kB 00:00 2024-04-22T19:47:54.323 INFO:teuthology.orchestra.run.smithi164.stdout:(10/282): ceph-common-19.0.0-3152.g9ed8263e.el7 3.2 MB/s | 21 MB 00:06 2024-04-22T19:47:54.461 INFO:teuthology.orchestra.run.smithi164.stdout:(11/282): libcephfs-devel-19.0.0-3152.g9ed8263e 238 kB/s | 32 kB 00:00 2024-04-22T19:47:56.257 INFO:teuthology.orchestra.run.smithi164.stdout:(12/282): libcephfs2-19.0.0-3152.g9ed8263e.el7. 415 kB/s | 746 kB 00:01 2024-04-22T19:47:56.690 INFO:teuthology.orchestra.run.smithi164.stdout:(13/282): libcephsqlite-19.0.0-3152.g9ed8263e.e 377 kB/s | 163 kB 00:00 2024-04-22T19:47:57.008 INFO:teuthology.orchestra.run.smithi164.stdout:(14/282): librados-devel-19.0.0-3152.g9ed8263e. 400 kB/s | 127 kB 00:00 2024-04-22T19:47:57.940 INFO:teuthology.orchestra.run.smithi164.stdout:(15/282): libradosstriper1-19.0.0-3152.g9ed8263 536 kB/s | 500 kB 00:00 2024-04-22T19:47:58.806 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 716 kB/s | 6.2 MB 00:08 2024-04-22T19:48:00.435 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-22T19:48:02.965 INFO:teuthology.orchestra.run.smithi164.stdout:(16/282): ceph-osd-19.0.0-3152.g9ed8263e.el7.x8 1.3 MB/s | 17 MB 00:13 2024-04-22T19:48:03.065 INFO:teuthology.orchestra.run.smithi164.stdout:(17/282): python3-ceph-argparse-19.0.0-3152.g9e 453 kB/s | 45 kB 00:00 2024-04-22T19:48:03.274 INFO:teuthology.orchestra.run.smithi164.stdout:(18/282): python3-ceph-common-19.0.0-3152.g9ed8 646 kB/s | 135 kB 00:00 2024-04-22T19:48:03.474 INFO:teuthology.orchestra.run.smithi164.stdout:(19/282): python3-cephfs-19.0.0-3152.g9ed8263e. 808 kB/s | 161 kB 00:00 2024-04-22T19:48:03.849 INFO:teuthology.orchestra.run.smithi164.stdout:(20/282): python3-rados-19.0.0-3152.g9ed8263e.e 855 kB/s | 321 kB 00:00 2024-04-22T19:48:04.166 INFO:teuthology.orchestra.run.smithi164.stdout:(21/282): python3-rbd-19.0.0-3152.g9ed8263e.el7 943 kB/s | 298 kB 00:00 2024-04-22T19:48:04.274 INFO:teuthology.orchestra.run.smithi164.stdout:(22/282): python3-rgw-19.0.0-3152.g9ed8263e.el7 920 kB/s | 99 kB 00:00 2024-04-22T19:48:04.385 INFO:teuthology.orchestra.run.smithi164.stdout:(23/282): rbd-fuse-19.0.0-3152.g9ed8263e.el7.x8 772 kB/s | 85 kB 00:00 2024-04-22T19:48:04.785 INFO:teuthology.orchestra.run.smithi164.stdout:(24/282): librgw2-19.0.0-3152.g9ed8263e.el7.x86 804 kB/s | 5.4 MB 00:06 2024-04-22T19:48:04.980 INFO:teuthology.orchestra.run.smithi164.stdout:(25/282): rbd-nbd-19.0.0-3152.g9ed8263e.el7.x86 894 kB/s | 171 kB 00:00 2024-04-22T19:48:05.086 INFO:teuthology.orchestra.run.smithi164.stdout:(26/282): ceph-grafana-dashboards-19.0.0-3152.g 315 kB/s | 34 kB 00:00 2024-04-22T19:48:05.278 INFO:teuthology.orchestra.run.smithi164.stdout:(27/282): ceph-mgr-cephadm-19.0.0-3152.g9ed8263 742 kB/s | 142 kB 00:00 2024-04-22T19:48:06.195 INFO:teuthology.orchestra.run.smithi164.stdout:(28/282): rbd-mirror-19.0.0-3152.g9ed8263e.el7. 1.7 MB/s | 3.1 MB 00:01 2024-04-22T19:48:08.661 INFO:teuthology.orchestra.run.smithi164.stdout:(29/282): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-04-22T19:48:08.812 INFO:teuthology.orchestra.run.smithi164.stdout:(30/282): ceph-mgr-dashboard-19.0.0-3152.g9ed82 1.0 MB/s | 3.5 MB 00:03 2024-04-22T19:48:08.987 INFO:teuthology.orchestra.run.smithi164.stdout:(31/282): ceph-mgr-rook-19.0.0-3152.g9ed8263e.e 282 kB/s | 49 kB 00:00 2024-04-22T19:48:09.130 INFO:teuthology.orchestra.run.smithi164.stdout:(32/282): ceph-prometheus-alerts-19.0.0-3152.g9 116 kB/s | 16 kB 00:00 2024-04-22T19:48:09.412 INFO:teuthology.orchestra.run.smithi164.stdout:(33/282): ceph-volume-19.0.0-3152.g9ed8263e.el7 964 kB/s | 273 kB 00:00 2024-04-22T19:48:09.504 INFO:teuthology.orchestra.run.smithi164.stdout:(34/282): ceph-mgr-modules-core-19.0.0-3152.g9e 295 kB/s | 249 kB 00:00 2024-04-22T19:48:09.679 INFO:teuthology.orchestra.run.smithi164.stdout:(35/282): ModemManager-glib-1.20.2-1.el9.x86_64 1.9 MB/s | 335 kB 00:00 2024-04-22T19:48:09.749 INFO:teuthology.orchestra.run.smithi164.stdout:(36/282): bluez-libs-5.56-6.el9.x86_64.rpm 1.2 MB/s | 85 kB 00:00 2024-04-22T19:48:09.822 INFO:teuthology.orchestra.run.smithi164.stdout:(37/282): bubblewrap-0.4.1-7.el9.x86_64.rpm 658 kB/s | 49 kB 00:00 2024-04-22T19:48:09.930 INFO:teuthology.orchestra.run.smithi164.stdout:(38/282): cups-libs-2.3.3op2-24.el9.x86_64.rpm 2.4 MB/s | 262 kB 00:00 2024-04-22T19:48:10.006 INFO:teuthology.orchestra.run.smithi164.stdout:(39/282): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-04-22T19:48:10.198 INFO:teuthology.orchestra.run.smithi164.stdout:(40/282): cephadm-19.0.0-3152.g9ed8263e.el7.noa 971 kB/s | 762 kB 00:00 2024-04-22T19:48:10.248 INFO:teuthology.orchestra.run.smithi164.stdout:(41/282): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-04-22T19:48:10.265 INFO:teuthology.orchestra.run.smithi164.stdout:(42/282): libpciaccess-0.16-7.el9.x86_64.rpm 404 kB/s | 27 kB 00:00 2024-04-22T19:48:10.348 INFO:teuthology.orchestra.run.smithi164.stdout:(43/282): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-22T19:48:10.374 INFO:teuthology.orchestra.run.smithi164.stdout:(44/282): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-22T19:48:10.466 INFO:teuthology.orchestra.run.smithi164.stdout:(45/282): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-22T19:48:10.533 INFO:teuthology.orchestra.run.smithi164.stdout:(46/282): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-22T19:48:10.616 INFO:teuthology.orchestra.run.smithi164.stdout:(47/282): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-22T19:48:10.683 INFO:teuthology.orchestra.run.smithi164.stdout:(48/282): python3-cryptography-36.0.1-4.el9.x86 4.0 MB/s | 1.2 MB 00:00 2024-04-22T19:48:10.725 INFO:teuthology.orchestra.run.smithi164.stdout:(49/282): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-22T19:48:11.184 INFO:teuthology.orchestra.run.smithi164.stdout:(50/282): ceph-test-19.0.0-3152.g9ed8263e.el7.x 2.5 MB/s | 49 MB 00:19 2024-04-22T19:48:11.218 INFO:teuthology.orchestra.run.smithi164.stdout:(51/282): python3-urllib3-1.26.5-5.el9.noarch.r 402 kB/s | 215 kB 00:00 2024-04-22T19:48:11.335 INFO:teuthology.orchestra.run.smithi164.stdout:(52/282): adwaita-cursor-theme-40.1.1-3.el9.noa 1.0 MB/s | 650 kB 00:00 2024-04-22T19:48:11.393 INFO:teuthology.orchestra.run.smithi164.stdout:(53/282): apache-commons-cli-1.4-17.el9.noarch. 1.2 MB/s | 71 kB 00:00 2024-04-22T19:48:11.418 INFO:teuthology.orchestra.run.smithi164.stdout:(54/282): alsa-lib-1.2.10-2.el9.x86_64.rpm 2.5 MB/s | 523 kB 00:00 2024-04-22T19:48:11.460 INFO:teuthology.orchestra.run.smithi164.stdout:(55/282): apache-commons-codec-1.15-7.el9.noarc 4.4 MB/s | 302 kB 00:00 2024-04-22T19:48:11.528 INFO:teuthology.orchestra.run.smithi164.stdout:(56/282): apache-commons-io-2.8.0-8.el9.noarch. 2.5 MB/s | 283 kB 00:00 2024-04-22T19:48:11.670 INFO:teuthology.orchestra.run.smithi164.stdout:(57/282): apache-commons-lang3-3.12.0-6.el9.noa 2.6 MB/s | 559 kB 00:00 2024-04-22T19:48:11.703 INFO:teuthology.orchestra.run.smithi164.stdout:(58/282): at-spi2-atk-2.38.0-4.el9.x86_64.rpm 494 kB/s | 87 kB 00:00 2024-04-22T19:48:11.739 INFO:teuthology.orchestra.run.smithi164.stdout:(59/282): at-spi2-core-2.40.3-1.el9.x86_64.rpm 2.8 MB/s | 195 kB 00:00 2024-04-22T19:48:11.762 INFO:teuthology.orchestra.run.smithi164.stdout:(60/282): atinject-1.0.3-6.el9.noarch.rpm 397 kB/s | 23 kB 00:00 2024-04-22T19:48:11.796 INFO:teuthology.orchestra.run.smithi164.stdout:(61/282): atk-2.36.0-5.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-22T19:48:11.838 INFO:teuthology.orchestra.run.smithi164.stdout:(62/282): avahi-glib-0.8-20.el9.x86_64.rpm 198 kB/s | 15 kB 00:00 2024-04-22T19:48:11.863 INFO:teuthology.orchestra.run.smithi164.stdout:(63/282): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-22T19:48:11.924 INFO:teuthology.orchestra.run.smithi164.stdout:(64/282): cdi-api-2.0.2-6.el9.noarch.rpm 639 kB/s | 53 kB 00:00 2024-04-22T19:48:12.013 INFO:teuthology.orchestra.run.smithi164.stdout:(65/282): colord-libs-1.4.5-4.el9.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2024-04-22T19:48:12.039 INFO:teuthology.orchestra.run.smithi164.stdout:(66/282): copy-jdk-configs-4.0-3.el9.noarch.rpm 241 kB/s | 28 kB 00:00 2024-04-22T19:48:12.064 INFO:teuthology.orchestra.run.smithi164.stdout:(67/282): dconf-0.40.0-6.el9.x86_64.rpm 2.2 MB/s | 113 kB 00:00 2024-04-22T19:48:12.182 INFO:teuthology.orchestra.run.smithi164.stdout:(68/282): adwaita-icon-theme-40.1.1-3.el9.noarc 12 MB/s | 12 MB 00:00 2024-04-22T19:48:12.206 INFO:teuthology.orchestra.run.smithi164.stdout:(69/282): exempi-2.6.0-0.2.20211007gite23c213.e 3.1 MB/s | 525 kB 00:00 2024-04-22T19:48:12.240 INFO:teuthology.orchestra.run.smithi164.stdout:(70/282): exiv2-libs-0.27.5-2.el9.x86_64.rpm 13 MB/s | 781 kB 00:00 2024-04-22T19:48:12.265 INFO:teuthology.orchestra.run.smithi164.stdout:(71/282): exiv2-0.27.5-2.el9.x86_64.rpm 4.8 MB/s | 980 kB 00:00 2024-04-22T19:48:12.290 INFO:teuthology.orchestra.run.smithi164.stdout:(72/282): flac-libs-1.3.3-12.el9.x86_64.rpm 4.2 MB/s | 218 kB 00:00 2024-04-22T19:48:12.319 INFO:teuthology.orchestra.run.smithi164.stdout:(73/282): fdk-aac-free-2.0.0-8.el9.x86_64.rpm 2.9 MB/s | 325 kB 00:00 2024-04-22T19:48:12.399 INFO:teuthology.orchestra.run.smithi164.stdout:(74/282): flatpak-selinux-1.12.8-1.el9.noarch.r 196 kB/s | 21 kB 00:00 2024-04-22T19:48:12.424 INFO:teuthology.orchestra.run.smithi164.stdout:(75/282): flatpak-session-helper-1.12.8-1.el9.x 681 kB/s | 74 kB 00:00 2024-04-22T19:48:12.466 INFO:teuthology.orchestra.run.smithi164.stdout:(76/282): flatpak-1.12.8-1.el9.x86_64.rpm 8.5 MB/s | 1.7 MB 00:00 2024-04-22T19:48:12.487 INFO:teuthology.orchestra.run.smithi164.stdout:(77/282): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-04-22T19:48:12.508 INFO:teuthology.orchestra.run.smithi164.stdout:(78/282): flexiblas-openblas-openmp-3.0.4-8.el9 436 kB/s | 18 kB 00:00 2024-04-22T19:48:12.534 INFO:teuthology.orchestra.run.smithi164.stdout:(79/282): gdk-pixbuf2-modules-2.42.6-3.el9.x86_ 1.7 MB/s | 88 kB 00:00 2024-04-22T19:48:12.584 INFO:teuthology.orchestra.run.smithi164.stdout:(80/282): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-22T19:48:12.601 INFO:teuthology.orchestra.run.smithi164.stdout:(81/282): giflib-5.2.1-9.el9.x86_64.rpm 725 kB/s | 49 kB 00:00 2024-04-22T19:48:12.626 INFO:teuthology.orchestra.run.smithi164.stdout:(82/282): geoclue2-2.6.0-7.el9.x86_64.rpm 1.0 MB/s | 124 kB 00:00 2024-04-22T19:48:12.676 INFO:teuthology.orchestra.run.smithi164.stdout:(83/282): google-guice-4.2.3-9.el9.noarch.rpm 5.0 MB/s | 473 kB 00:00 2024-04-22T19:48:12.702 INFO:teuthology.orchestra.run.smithi164.stdout:(84/282): gsm-1.0.19-6.el9.x86_64.rpm 452 kB/s | 34 kB 00:00 2024-04-22T19:48:12.738 INFO:teuthology.orchestra.run.smithi164.stdout:(85/282): graphene-1.10.6-2.el9.x86_64.rpm 480 kB/s | 64 kB 00:00 2024-04-22T19:48:12.794 INFO:teuthology.orchestra.run.smithi164.stdout:(86/282): gstreamer1-1.22.1-2.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-04-22T19:48:12.820 INFO:teuthology.orchestra.run.smithi164.stdout:(87/282): gtk-update-icon-cache-3.24.31-2.el9.x 422 kB/s | 35 kB 00:00 2024-04-22T19:48:12.870 INFO:teuthology.orchestra.run.smithi164.stdout:(88/282): gstreamer1-plugins-base-1.22.1-2.el9. 13 MB/s | 2.2 MB 00:00 2024-04-22T19:48:12.920 INFO:teuthology.orchestra.run.smithi164.stdout:(89/282): hicolor-icon-theme-0.17-13.el9.noarch 4.3 MB/s | 219 kB 00:00 2024-04-22T19:48:12.987 INFO:teuthology.orchestra.run.smithi164.stdout:(90/282): guava-30.1-6.el9.noarch.rpm 14 MB/s | 2.3 MB 00:00 2024-04-22T19:48:13.029 INFO:teuthology.orchestra.run.smithi164.stdout:(91/282): httpcomponents-client-4.5.13-3.el9.no 5.9 MB/s | 657 kB 00:00 2024-04-22T19:48:13.063 INFO:teuthology.orchestra.run.smithi164.stdout:(92/282): httpcomponents-core-4.4.13-7.el9.noar 8.2 MB/s | 632 kB 00:00 2024-04-22T19:48:13.129 INFO:teuthology.orchestra.run.smithi164.stdout:(93/282): gtk3-3.24.31-2.el9.x86_64.rpm 15 MB/s | 4.9 MB 00:00 2024-04-22T19:48:13.180 INFO:teuthology.orchestra.run.smithi164.stdout:(94/282): jakarta-annotations-1.3.5-13.el9.noar 385 kB/s | 45 kB 00:00 2024-04-22T19:48:13.305 INFO:teuthology.orchestra.run.smithi164.stdout:(95/282): jansi-2.3.3-6.el9.x86_64.rpm 626 kB/s | 110 kB 00:00 2024-04-22T19:48:13.347 INFO:teuthology.orchestra.run.smithi164.stdout:(96/282): java-11-openjdk-11.0.18.0.10-3.el9.x8 2.6 MB/s | 440 kB 00:00 2024-04-22T19:48:13.581 INFO:teuthology.orchestra.run.smithi164.stdout:(97/282): iso-codes-4.6.0-3.el9.noarch.rpm 6.4 MB/s | 3.5 MB 00:00 2024-04-22T19:48:14.039 INFO:teuthology.orchestra.run.smithi164.stdout:(98/282): javapackages-filesystem-6.0.0-4.el9.n 28 kB/s | 13 kB 00:00 2024-04-22T19:48:14.089 INFO:teuthology.orchestra.run.smithi164.stdout:(99/282): java-11-openjdk-devel-11.0.18.0.10-3. 4.2 MB/s | 3.3 MB 00:00 2024-04-22T19:48:14.172 INFO:teuthology.orchestra.run.smithi164.stdout:(100/282): jbigkit-libs-2.1-23.el9.x86_64.rpm 398 kB/s | 53 kB 00:00 2024-04-22T19:48:14.198 INFO:teuthology.orchestra.run.smithi164.stdout:(101/282): jcl-over-slf4j-1.7.30-13.el9.noarch. 221 kB/s | 24 kB 00:00 2024-04-22T19:48:14.308 INFO:teuthology.orchestra.run.smithi164.stdout:(102/282): jsoup-1.13.1-10.el9.noarch.rpm 2.8 MB/s | 377 kB 00:00 2024-04-22T19:48:14.373 INFO:teuthology.orchestra.run.smithi164.stdout:(103/282): jsr-305-3.0.2-6.el9.noarch.rpm 203 kB/s | 35 kB 00:00 2024-04-22T19:48:14.399 INFO:teuthology.orchestra.run.smithi164.stdout:(104/282): lcms2-2.12-3.el9.x86_64.rpm 1.8 MB/s | 168 kB 00:00 2024-04-22T19:48:14.432 INFO:teuthology.orchestra.run.smithi164.stdout:(105/282): libX11-xcb-1.7.0-9.el9.x86_64.rpm 189 kB/s | 11 kB 00:00 2024-04-22T19:48:14.549 INFO:teuthology.orchestra.run.smithi164.stdout:(106/282): libXcomposite-0.4.5-7.el9.x86_64.rpm 161 kB/s | 24 kB 00:00 2024-04-22T19:48:14.574 INFO:teuthology.orchestra.run.smithi164.stdout:(107/282): libXcursor-1.2.0-7.el9.x86_64.rpm 217 kB/s | 31 kB 00:00 2024-04-22T19:48:14.600 INFO:teuthology.orchestra.run.smithi164.stdout:(108/282): libXdamage-1.1.5-7.el9.x86_64.rpm 455 kB/s | 23 kB 00:00 2024-04-22T19:48:14.625 INFO:teuthology.orchestra.run.smithi164.stdout:(109/282): libXfixes-5.0.3-16.el9.x86_64.rpm 394 kB/s | 20 kB 00:00 2024-04-22T19:48:14.650 INFO:teuthology.orchestra.run.smithi164.stdout:(110/282): libXi-1.7.10-8.el9.x86_64.rpm 788 kB/s | 40 kB 00:00 2024-04-22T19:48:14.684 INFO:teuthology.orchestra.run.smithi164.stdout:(111/282): libXinerama-1.1.4-10.el9.x86_64.rpm 257 kB/s | 15 kB 00:00 2024-04-22T19:48:14.734 INFO:teuthology.orchestra.run.smithi164.stdout:(112/282): libXrandr-1.5.2-8.el9.x86_64.rpm 338 kB/s | 28 kB 00:00 2024-04-22T19:48:14.755 INFO:teuthology.orchestra.run.smithi164.stdout:(113/282): libXtst-1.2.3-16.el9.x86_64.rpm 321 kB/s | 21 kB 00:00 2024-04-22T19:48:14.801 INFO:teuthology.orchestra.run.smithi164.stdout:(114/282): libXv-1.0.11-16.el9.x86_64.rpm 286 kB/s | 19 kB 00:00 2024-04-22T19:48:14.819 INFO:teuthology.orchestra.run.smithi164.stdout:(115/282): libXxf86vm-1.1.4-18.el9.x86_64.rpm 280 kB/s | 19 kB 00:00 2024-04-22T19:48:14.854 INFO:teuthology.orchestra.run.smithi164.stdout:(116/282): libasyncns-0.8-22.el9.x86_64.rpm 607 kB/s | 30 kB 00:00 2024-04-22T19:48:14.886 INFO:teuthology.orchestra.run.smithi164.stdout:(117/282): libcanberra-0.30-27.el9.x86_64.rpm 1.3 MB/s | 86 kB 00:00 2024-04-22T19:48:14.903 INFO:teuthology.orchestra.run.smithi164.stdout:(118/282): libcanberra-gtk3-0.30-27.el9.x86_64. 633 kB/s | 32 kB 00:00 2024-04-22T19:48:14.945 INFO:teuthology.orchestra.run.smithi164.stdout:(119/282): libdrm-2.4.117-1.el9.x86_64.rpm 2.7 MB/s | 161 kB 00:00 2024-04-22T19:48:14.978 INFO:teuthology.orchestra.run.smithi164.stdout:(120/282): libepoxy-1.5.5-4.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-22T19:48:15.037 INFO:teuthology.orchestra.run.smithi164.stdout:(121/282): libexif-0.6.22-6.el9.x86_64.rpm 4.6 MB/s | 435 kB 00:00 2024-04-22T19:48:15.062 INFO:teuthology.orchestra.run.smithi164.stdout:(122/282): libfontenc-1.1.3-17.el9.x86_64.rpm 370 kB/s | 31 kB 00:00 2024-04-22T19:48:15.131 INFO:teuthology.orchestra.run.smithi164.stdout:(123/282): libgexiv2-0.12.3-1.el9.x86_64.rpm 900 kB/s | 83 kB 00:00 2024-04-22T19:48:15.165 INFO:teuthology.orchestra.run.smithi164.stdout:(124/282): libglvnd-1.3.4-1.el9.x86_64.rpm 1.3 MB/s | 135 kB 00:00 2024-04-22T19:48:15.180 INFO:teuthology.orchestra.run.smithi164.stdout:(125/282): libglvnd-egl-1.3.4-1.el9.x86_64.rpm 732 kB/s | 37 kB 00:00 2024-04-22T19:48:15.232 INFO:teuthology.orchestra.run.smithi164.stdout:(126/282): libglvnd-glx-1.3.4-1.el9.x86_64.rpm 2.1 MB/s | 142 kB 00:00 2024-04-22T19:48:15.272 INFO:teuthology.orchestra.run.smithi164.stdout:(127/282): libgsf-1.14.47-5.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-04-22T19:48:15.330 INFO:teuthology.orchestra.run.smithi164.stdout:(128/282): libgxps-0.3.2-3.el9.x86_64.rpm 788 kB/s | 79 kB 00:00 2024-04-22T19:48:15.356 INFO:teuthology.orchestra.run.smithi164.stdout:(129/282): libiptcdata-1.0.5-9.el9.x86_64.rpm 743 kB/s | 62 kB 00:00 2024-04-22T19:48:15.381 INFO:teuthology.orchestra.run.smithi164.stdout:(130/282): libldac-2.0.2.3-10.el9.x86_64.rpm 821 kB/s | 41 kB 00:00 2024-04-22T19:48:15.415 INFO:teuthology.orchestra.run.smithi164.stdout:(131/282): libnotify-0.7.9-8.el9.x86_64.rpm 743 kB/s | 44 kB 00:00 2024-04-22T19:48:15.431 INFO:teuthology.orchestra.run.smithi164.stdout:(132/282): libogg-1.3.4-6.el9.x86_64.rpm 670 kB/s | 34 kB 00:00 2024-04-22T19:48:15.500 INFO:teuthology.orchestra.run.smithi164.stdout:(133/282): libosinfo-1.10.0-1.el9.x86_64.rpm 3.7 MB/s | 319 kB 00:00 2024-04-22T19:48:15.524 INFO:teuthology.orchestra.run.smithi164.stdout:(134/282): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-22T19:48:15.916 INFO:teuthology.orchestra.run.smithi164.stdout:(135/282): librabbitmq-0.11.0-7.el9.x86_64.rpm 108 kB/s | 45 kB 00:00 2024-04-22T19:48:16.258 INFO:teuthology.orchestra.run.smithi164.stdout:(136/282): java-11-openjdk-headless-11.0.18.0.1 14 MB/s | 40 MB 00:02 2024-04-22T19:48:16.275 INFO:teuthology.orchestra.run.smithi164.stdout:(137/282): libsbc-1.4-9.el9.x86_64.rpm 125 kB/s | 45 kB 00:00 2024-04-22T19:48:16.300 INFO:teuthology.orchestra.run.smithi164.stdout:(138/282): librdkafka-1.6.1-102.el9.x86_64.rpm 854 kB/s | 662 kB 00:00 2024-04-22T19:48:16.325 INFO:teuthology.orchestra.run.smithi164.stdout:(139/282): libtheora-1.1.1-31.el9.x86_64.rpm 3.2 MB/s | 164 kB 00:00 2024-04-22T19:48:16.352 INFO:teuthology.orchestra.run.smithi164.stdout:(140/282): libsndfile-1.0.31-8.el9.x86_64.rpm 2.2 MB/s | 206 kB 00:00 2024-04-22T19:48:16.377 INFO:teuthology.orchestra.run.smithi164.stdout:(141/282): libtool-ltdl-2.4.6-45.el9.x86_64.rpm 735 kB/s | 37 kB 00:00 2024-04-22T19:48:16.401 INFO:teuthology.orchestra.run.smithi164.stdout:(142/282): libtiff-4.4.0-12.el9.x86_64.rpm 1.9 MB/s | 198 kB 00:00 2024-04-22T19:48:16.426 INFO:teuthology.orchestra.run.smithi164.stdout:(143/282): libvisual-0.4.0-34.el9.x86_64.rpm 2.8 MB/s | 144 kB 00:00 2024-04-22T19:48:16.455 INFO:teuthology.orchestra.run.smithi164.stdout:(144/282): libtracker-sparql-3.1.2-3.el9.x86_64 3.1 MB/s | 324 kB 00:00 2024-04-22T19:48:16.477 INFO:teuthology.orchestra.run.smithi164.stdout:(145/282): libvorbis-1.3.7-5.el9.x86_64.rpm 2.5 MB/s | 193 kB 00:00 2024-04-22T19:48:16.502 INFO:teuthology.orchestra.run.smithi164.stdout:(146/282): libwayland-client-1.21.0-1.el9.x86_6 442 kB/s | 33 kB 00:00 2024-04-22T19:48:16.527 INFO:teuthology.orchestra.run.smithi164.stdout:(147/282): libwayland-cursor-1.21.0-1.el9.x86_6 257 kB/s | 19 kB 00:00 2024-04-22T19:48:16.553 INFO:teuthology.orchestra.run.smithi164.stdout:(148/282): libwayland-egl-1.21.0-1.el9.x86_64.r 171 kB/s | 13 kB 00:00 2024-04-22T19:48:16.578 INFO:teuthology.orchestra.run.smithi164.stdout:(149/282): libwayland-server-1.21.0-1.el9.x86_6 555 kB/s | 42 kB 00:00 2024-04-22T19:48:16.603 INFO:teuthology.orchestra.run.smithi164.stdout:(150/282): libwebp-1.2.0-8.el9.x86_64.rpm 5.4 MB/s | 277 kB 00:00 2024-04-22T19:48:16.629 INFO:teuthology.orchestra.run.smithi164.stdout:(151/282): libxkbcommon-1.0.3-4.el9.x86_64.rpm 1.7 MB/s | 133 kB 00:00 2024-04-22T19:48:16.645 INFO:teuthology.orchestra.run.smithi164.stdout:(152/282): libxshmfence-1.3-10.el9.x86_64.rpm 188 kB/s | 13 kB 00:00 2024-04-22T19:48:16.662 INFO:teuthology.orchestra.run.smithi164.stdout:(153/282): low-memory-monitor-2.1-4.el9.x86_64. 612 kB/s | 36 kB 00:00 2024-04-22T19:48:16.863 INFO:teuthology.orchestra.run.smithi164.stdout:(154/282): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-04-22T19:48:16.889 INFO:teuthology.orchestra.run.smithi164.stdout:(155/282): lua-posix-35.0-8.el9.x86_64.rpm 670 kB/s | 151 kB 00:00 2024-04-22T19:48:16.918 INFO:teuthology.orchestra.run.smithi164.stdout:(156/282): lua-5.4.4-4.el9.x86_64.rpm 704 kB/s | 188 kB 00:00 2024-04-22T19:48:16.942 INFO:teuthology.orchestra.run.smithi164.stdout:(157/282): maven-3.6.3-15.el9.noarch.rpm 235 kB/s | 18 kB 00:00 2024-04-22T19:48:16.997 INFO:teuthology.orchestra.run.smithi164.stdout:(158/282): maven-openjdk11-3.6.3-15.el9.noarch. 108 kB/s | 9.0 kB 00:00 2024-04-22T19:48:17.047 INFO:teuthology.orchestra.run.smithi164.stdout:(159/282): maven-resolver-1.6.1-10.el9.noarch.r 4.6 MB/s | 517 kB 00:00 2024-04-22T19:48:17.089 INFO:teuthology.orchestra.run.smithi164.stdout:(160/282): maven-lib-3.6.3-15.el9.noarch.rpm 7.4 MB/s | 1.5 MB 00:00 2024-04-22T19:48:17.115 INFO:teuthology.orchestra.run.smithi164.stdout:(161/282): maven-wagon-3.4.2-7.el9.noarch.rpm 1.6 MB/s | 112 kB 00:00 2024-04-22T19:48:17.146 INFO:teuthology.orchestra.run.smithi164.stdout:(162/282): maven-shared-utils-3.3.4-3.el9.noarc 1.0 MB/s | 151 kB 00:00 2024-04-22T19:48:17.165 INFO:teuthology.orchestra.run.smithi164.stdout:(163/282): mesa-libEGL-23.3.3-1.el9.x86_64.rpm 1.6 MB/s | 127 kB 00:00 2024-04-22T19:48:17.190 INFO:teuthology.orchestra.run.smithi164.stdout:(164/282): mesa-libGL-23.3.3-1.el9.x86_64.rpm 2.2 MB/s | 168 kB 00:00 2024-04-22T19:48:17.207 INFO:teuthology.orchestra.run.smithi164.stdout:(165/282): mesa-libgbm-23.3.3-1.el9.x86_64.rpm 563 kB/s | 38 kB 00:00 2024-04-22T19:48:17.224 INFO:teuthology.orchestra.run.smithi164.stdout:(166/282): mesa-libglapi-23.3.3-1.el9.x86_64.rp 753 kB/s | 44 kB 00:00 2024-04-22T19:48:17.274 INFO:teuthology.orchestra.run.smithi164.stdout:(167/282): mkfontscale-1.2.1-3.el9.x86_64.rpm 483 kB/s | 32 kB 00:00 2024-04-22T19:48:17.300 INFO:teuthology.orchestra.run.smithi164.stdout:(168/282): openblas-0.3.21-2.el9.x86_64.rpm 383 kB/s | 35 kB 00:00 2024-04-22T19:48:17.350 INFO:teuthology.orchestra.run.smithi164.stdout:(169/282): openjpeg2-2.4.0-7.el9.x86_64.rpm 2.1 MB/s | 163 kB 00:00 2024-04-22T19:48:17.394 INFO:teuthology.orchestra.run.smithi164.stdout:(170/282): opus-1.3.1-10.el9.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2024-04-22T19:48:17.517 INFO:teuthology.orchestra.run.smithi164.stdout:(171/282): orc-0.4.31-6.el9.x86_64.rpm 1.1 MB/s | 184 kB 00:00 2024-04-22T19:48:17.592 INFO:teuthology.orchestra.run.smithi164.stdout:(172/282): openblas-openmp-0.3.21-2.el9.x86_64. 13 MB/s | 4.7 MB 00:00 2024-04-22T19:48:17.626 INFO:teuthology.orchestra.run.smithi164.stdout:(173/282): osinfo-db-tools-1.10.0-1.el9.x86_64. 678 kB/s | 74 kB 00:00 2024-04-22T19:48:17.651 INFO:teuthology.orchestra.run.smithi164.stdout:(174/282): osinfo-db-20231215-1.el9.noarch.rpm 2.0 MB/s | 521 kB 00:00 2024-04-22T19:48:17.677 INFO:teuthology.orchestra.run.smithi164.stdout:(175/282): ostree-libs-2024.5-2.el9.x86_64.rpm 5.7 MB/s | 488 kB 00:00 2024-04-22T19:48:17.702 INFO:teuthology.orchestra.run.smithi164.stdout:(176/282): p11-kit-server-0.25.3-2.el9.x86_64.r 3.2 MB/s | 246 kB 00:00 2024-04-22T19:48:17.744 INFO:teuthology.orchestra.run.smithi164.stdout:(177/282): pipewire-1.0.1-1.el9.x86_64.rpm 1.1 MB/s | 107 kB 00:00 2024-04-22T19:48:17.769 INFO:teuthology.orchestra.run.smithi164.stdout:(178/282): pipewire-jack-audio-connection-kit-1 137 kB/s | 9.1 kB 00:00 2024-04-22T19:48:17.795 INFO:teuthology.orchestra.run.smithi164.stdout:(179/282): pipewire-alsa-1.0.1-1.el9.x86_64.rpm 488 kB/s | 57 kB 00:00 2024-04-22T19:48:17.837 INFO:teuthology.orchestra.run.smithi164.stdout:(180/282): pipewire-jack-audio-connection-kit-l 1.4 MB/s | 135 kB 00:00 2024-04-22T19:48:17.920 INFO:teuthology.orchestra.run.smithi164.stdout:(181/282): pipewire-pulseaudio-1.0.1-1.el9.x86_ 1.5 MB/s | 192 kB 00:00 2024-04-22T19:48:17.979 INFO:teuthology.orchestra.run.smithi164.stdout:(182/282): pipewire-libs-1.0.1-1.el9.x86_64.rpm 9.2 MB/s | 1.9 MB 00:00 2024-04-22T19:48:18.004 INFO:teuthology.orchestra.run.smithi164.stdout:(183/282): plexus-cipher-1.7-27.el9.noarch.rpm 154 kB/s | 26 kB 00:00 2024-04-22T19:48:18.029 INFO:teuthology.orchestra.run.smithi164.stdout:(184/282): plexus-classworlds-2.6.0-11.el9.noar 551 kB/s | 60 kB 00:00 2024-04-22T19:48:18.055 INFO:teuthology.orchestra.run.smithi164.stdout:(185/282): plexus-containers-component-annotati 244 kB/s | 18 kB 00:00 2024-04-22T19:48:18.080 INFO:teuthology.orchestra.run.smithi164.stdout:(186/282): plexus-interpolation-1.26-11.el9.noa 1.0 MB/s | 80 kB 00:00 2024-04-22T19:48:18.105 INFO:teuthology.orchestra.run.smithi164.stdout:(187/282): plexus-sec-dispatcher-1.4-37.el9.noa 446 kB/s | 34 kB 00:00 2024-04-22T19:48:18.130 INFO:teuthology.orchestra.run.smithi164.stdout:(188/282): plexus-utils-3.3.0-10.el9.noarch.rpm 3.3 MB/s | 254 kB 00:00 2024-04-22T19:48:18.330 INFO:teuthology.orchestra.run.smithi164.stdout:(189/282): poppler-glib-21.01.0-19.el9.x86_64.r 762 kB/s | 152 kB 00:00 2024-04-22T19:48:18.367 INFO:teuthology.orchestra.run.smithi164.stdout:(190/282): poppler-21.01.0-19.el9.x86_64.rpm 3.7 MB/s | 1.1 MB 00:00 2024-04-22T19:48:18.410 INFO:teuthology.orchestra.run.smithi164.stdout:(191/282): poppler-data-0.4.9-9.el9.noarch.rpm 6.2 MB/s | 1.9 MB 00:00 2024-04-22T19:48:18.431 INFO:teuthology.orchestra.run.smithi164.stdout:(192/282): publicsuffix-list-20210518-3.el9.noa 1.2 MB/s | 83 kB 00:00 2024-04-22T19:48:18.465 INFO:teuthology.orchestra.run.smithi164.stdout:(193/282): protobuf-3.14.0-13.el9.x86_64.rpm 7.5 MB/s | 1.0 MB 00:00 2024-04-22T19:48:18.504 INFO:teuthology.orchestra.run.smithi164.stdout:(194/282): pulseaudio-libs-15.0-2.el9.x86_64.rp 7.2 MB/s | 679 kB 00:00 2024-04-22T19:48:18.525 INFO:teuthology.orchestra.run.smithi164.stdout:(195/282): python3-devel-3.9.18-3.el9.x86_64.rp 4.1 MB/s | 244 kB 00:00 2024-04-22T19:48:18.549 INFO:teuthology.orchestra.run.smithi164.stdout:(196/282): python3-jinja2-2.11.3-5.el9.noarch.r 4.8 MB/s | 248 kB 00:00 2024-04-22T19:48:18.566 INFO:teuthology.orchestra.run.smithi164.stdout:(197/282): python3-jsonpatch-1.21-16.el9.noarch 609 kB/s | 26 kB 00:00 2024-04-22T19:48:18.591 INFO:teuthology.orchestra.run.smithi164.stdout:(198/282): python3-jsonpointer-2.0-4.el9.noarch 457 kB/s | 19 kB 00:00 2024-04-22T19:48:18.675 INFO:teuthology.orchestra.run.smithi164.stdout:(199/282): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-04-22T19:48:18.691 INFO:teuthology.orchestra.run.smithi164.stdout:(200/282): python3-markupsafe-1.1.1-12.el9.x86_ 347 kB/s | 35 kB 00:00 2024-04-22T19:48:18.717 INFO:teuthology.orchestra.run.smithi164.stdout:(201/282): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-22T19:48:18.750 INFO:teuthology.orchestra.run.smithi164.stdout:(202/282): python3-numpy-f2py-1.20.1-5.el9.x86_ 4.2 MB/s | 250 kB 00:00 2024-04-22T19:48:18.775 INFO:teuthology.orchestra.run.smithi164.stdout:(203/282): python3-oauthlib-3.1.1-5.el9.noarch. 3.7 MB/s | 222 kB 00:00 2024-04-22T19:48:18.834 INFO:teuthology.orchestra.run.smithi164.stdout:(204/282): python3-packaging-20.9-5.el9.noarch. 920 kB/s | 77 kB 00:00 2024-04-22T19:48:18.862 INFO:teuthology.orchestra.run.smithi164.stdout:(205/282): python3-prettytable-0.7.2-27.el9.noa 504 kB/s | 42 kB 00:00 2024-04-22T19:48:18.887 INFO:teuthology.orchestra.run.smithi164.stdout:(206/282): python3-protobuf-3.14.0-13.el9.noarc 5.2 MB/s | 269 kB 00:00 2024-04-22T19:48:18.911 INFO:teuthology.orchestra.run.smithi164.stdout:(207/282): python3-pyasn1-0.4.8-6.el9.noarch.rp 3.1 MB/s | 159 kB 00:00 2024-04-22T19:48:18.935 INFO:teuthology.orchestra.run.smithi164.stdout:(208/282): python3-pyasn1-modules-0.4.8-6.el9.n 5.4 MB/s | 279 kB 00:00 2024-04-22T19:48:19.002 INFO:teuthology.orchestra.run.smithi164.stdout:(209/282): python3-numpy-1.20.1-5.el9.x86_64.rp 16 MB/s | 5.1 MB 00:00 2024-04-22T19:48:19.028 INFO:teuthology.orchestra.run.smithi164.stdout:(210/282): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-04-22T19:48:19.053 INFO:teuthology.orchestra.run.smithi164.stdout:(211/282): python3-requests-oauthlib-1.3.0-12.e 458 kB/s | 54 kB 00:00 2024-04-22T19:48:19.120 INFO:teuthology.orchestra.run.smithi164.stdout:(212/282): python3-toml-0.10.2-6.el9.noarch.rpm 627 kB/s | 42 kB 00:00 2024-04-22T19:48:19.215 INFO:teuthology.orchestra.run.smithi164.stdout:(213/282): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-04-22T19:48:19.280 INFO:teuthology.orchestra.run.smithi164.stdout:(214/282): qatlib-service-24.02.0-1.el9.x86_64. 224 kB/s | 36 kB 00:00 2024-04-22T19:48:19.312 INFO:teuthology.orchestra.run.smithi164.stdout:(215/282): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-04-22T19:48:19.346 INFO:teuthology.orchestra.run.smithi164.stdout:(216/282): rtkit-0.11-28.el9.x86_64.rpm 853 kB/s | 56 kB 00:00 2024-04-22T19:48:19.415 INFO:teuthology.orchestra.run.smithi164.stdout:(217/282): sisu-0.3.4-10.el9.noarch.rpm 5.0 MB/s | 510 kB 00:00 2024-04-22T19:48:19.438 INFO:teuthology.orchestra.run.smithi164.stdout:(218/282): slf4j-1.7.30-13.el9.noarch.rpm 751 kB/s | 69 kB 00:00 2024-04-22T19:48:19.472 INFO:teuthology.orchestra.run.smithi164.stdout:(219/282): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-22T19:48:19.505 INFO:teuthology.orchestra.run.smithi164.stdout:(220/282): sound-theme-freedesktop-0.8-17.el9.n 5.6 MB/s | 383 kB 00:00 2024-04-22T19:48:19.530 INFO:teuthology.orchestra.run.smithi164.stdout:(221/282): totem-pl-parser-3.26.6-2.el9.x86_64. 2.5 MB/s | 152 kB 00:00 2024-04-22T19:48:19.564 INFO:teuthology.orchestra.run.smithi164.stdout:(222/282): tracker-3.1.2-3.el9.x86_64.rpm 9.3 MB/s | 555 kB 00:00 2024-04-22T19:48:19.723 INFO:teuthology.orchestra.run.smithi164.stdout:(223/282): tracker-miners-3.1.2-4.el9.x86_64.rp 4.8 MB/s | 944 kB 00:00 2024-04-22T19:48:19.748 INFO:teuthology.orchestra.run.smithi164.stdout:(224/282): ttmkfdir-3.0.9-65.el9.x86_64.rpm 290 kB/s | 53 kB 00:00 2024-04-22T19:48:19.790 INFO:teuthology.orchestra.run.smithi164.stdout:(225/282): tzdata-java-2024a-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-22T19:48:19.815 INFO:teuthology.orchestra.run.smithi164.stdout:(226/282): upower-0.99.11-11.el9.x86_64.rpm 2.5 MB/s | 173 kB 00:00 2024-04-22T19:48:19.974 INFO:teuthology.orchestra.run.smithi164.stdout:(227/282): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-22T19:48:20.001 INFO:teuthology.orchestra.run.smithi164.stdout:(228/282): wireplumber-0.4.14-1.el9.x86_64.rpm 525 kB/s | 96 kB 00:00 2024-04-22T19:48:20.025 INFO:teuthology.orchestra.run.smithi164.stdout:(229/282): webrtc-audio-processing-0.3.1-8.el9. 1.3 MB/s | 306 kB 00:00 2024-04-22T19:48:20.083 INFO:teuthology.orchestra.run.smithi164.stdout:(230/282): wireplumber-libs-0.4.14-1.el9.x86_64 3.2 MB/s | 358 kB 00:00 2024-04-22T19:48:20.111 INFO:teuthology.orchestra.run.smithi164.stdout:(231/282): xdg-dbus-proxy-0.1.3-1.el9.x86_64.rp 390 kB/s | 42 kB 00:00 2024-04-22T19:48:20.200 INFO:teuthology.orchestra.run.smithi164.stdout:(232/282): xdg-desktop-portal-1.12.6-1.el9.x86_ 2.1 MB/s | 385 kB 00:00 2024-04-22T19:48:20.225 INFO:teuthology.orchestra.run.smithi164.stdout:(233/282): xdg-desktop-portal-gtk-1.12.0-3.el9. 946 kB/s | 134 kB 00:00 2024-04-22T19:48:20.260 INFO:teuthology.orchestra.run.smithi164.stdout:(234/282): xkeyboard-config-2.33-2.el9.noarch.r 5.6 MB/s | 859 kB 00:00 2024-04-22T19:48:20.289 INFO:teuthology.orchestra.run.smithi164.stdout:(235/282): xmlstarlet-1.6.1-20.el9.x86_64.rpm 758 kB/s | 64 kB 00:00 2024-04-22T19:48:20.292 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-22T19:48:20.318 INFO:teuthology.orchestra.run.smithi164.stdout:(236/282): xorg-x11-fonts-Type1-7.5-33.el9.noar 5.3 MB/s | 505 kB 00:00 2024-04-22T19:48:20.343 INFO:teuthology.orchestra.run.smithi164.stdout:(237/282): lua-devel-5.4.4-4.el9.x86_64.rpm 265 kB/s | 22 kB 00:00 2024-04-22T19:48:20.535 INFO:teuthology.orchestra.run.smithi164.stdout:(238/282): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-04-22T19:48:20.602 INFO:teuthology.orchestra.run.smithi164.stdout:(239/282): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-22T19:48:20.627 INFO:teuthology.orchestra.run.smithi164.stdout:(240/282): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-22T19:48:20.653 INFO:teuthology.orchestra.run.smithi164.stdout:(241/282): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-04-22T19:48:20.670 INFO:teuthology.orchestra.run.smithi164.stdout:(242/282): libarrow-doc-9.0.0-10.el9.noarch.rpm 621 kB/s | 26 kB 00:00 2024-04-22T19:48:20.728 INFO:teuthology.orchestra.run.smithi164.stdout:(243/282): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-22T19:48:20.753 INFO:teuthology.orchestra.run.smithi164.stdout:(244/282): liboath-2.6.7-2.el9.x86_64.rpm 483 kB/s | 49 kB 00:00 2024-04-22T19:48:20.787 INFO:teuthology.orchestra.run.smithi164.stdout:(245/282): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-22T19:48:20.854 INFO:teuthology.orchestra.run.smithi164.stdout:(246/282): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-22T19:48:20.888 INFO:teuthology.orchestra.run.smithi164.stdout:(247/282): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-22T19:48:20.913 INFO:teuthology.orchestra.run.smithi164.stdout:(248/282): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-04-22T19:48:20.955 INFO:teuthology.orchestra.run.smithi164.stdout:(249/282): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-04-22T19:48:20.980 INFO:teuthology.orchestra.run.smithi164.stdout:(250/282): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-22T19:48:21.019 INFO:teuthology.orchestra.run.smithi164.stdout:(251/282): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-04-22T19:48:21.039 INFO:teuthology.orchestra.run.smithi164.stdout:(252/282): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-22T19:48:21.064 INFO:teuthology.orchestra.run.smithi164.stdout:(253/282): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-22T19:48:21.098 INFO:teuthology.orchestra.run.smithi164.stdout:(254/282): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-22T19:48:21.131 INFO:teuthology.orchestra.run.smithi164.stdout:(255/282): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-22T19:48:21.159 INFO:teuthology.orchestra.run.smithi164.stdout:(256/282): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-04-22T19:48:21.207 INFO:teuthology.orchestra.run.smithi164.stdout:(257/282): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-22T19:48:21.249 INFO:teuthology.orchestra.run.smithi164.stdout:(258/282): python3-jaraco-classes-3.2.1-5.el9.n 150 kB/s | 18 kB 00:00 2024-04-22T19:48:21.266 INFO:teuthology.orchestra.run.smithi164.stdout:(259/282): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-22T19:48:21.283 INFO:teuthology.orchestra.run.smithi164.stdout:(260/282): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-22T19:48:21.308 INFO:teuthology.orchestra.run.smithi164.stdout:(261/282): python3-jaraco-text-3.2.0-6.el9.noar 474 kB/s | 20 kB 00:00 2024-04-22T19:48:21.325 INFO:teuthology.orchestra.run.smithi164.stdout:(262/282): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-22T19:48:21.368 INFO:teuthology.orchestra.run.smithi164.stdout:(263/282): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-22T19:48:21.400 INFO:teuthology.orchestra.run.smithi164.stdout:(264/282): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-04-22T19:48:21.417 INFO:teuthology.orchestra.run.smithi164.stdout:(265/282): python3-natsort-7.1.1-5.el9.noarch.r 625 kB/s | 58 kB 00:00 2024-04-22T19:48:21.442 INFO:teuthology.orchestra.run.smithi164.stdout:(266/282): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-22T19:48:21.468 INFO:teuthology.orchestra.run.smithi164.stdout:(267/282): python3-portend-3.1.0-2.el9.noarch.r 243 kB/s | 16 kB 00:00 2024-04-22T19:48:21.493 INFO:teuthology.orchestra.run.smithi164.stdout:(268/282): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-22T19:48:21.518 INFO:teuthology.orchestra.run.smithi164.stdout:(269/282): python3-repoze-lru-0.7-16.el9.noarch 405 kB/s | 31 kB 00:00 2024-04-22T19:48:21.544 INFO:teuthology.orchestra.run.smithi164.stdout:(270/282): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-22T19:48:21.571 INFO:teuthology.orchestra.run.smithi164.stdout:(271/282): python3-rsa-4.9-2.el9.noarch.rpm 764 kB/s | 59 kB 00:00 2024-04-22T19:48:21.594 INFO:teuthology.orchestra.run.smithi164.stdout:(272/282): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-22T19:48:21.620 INFO:teuthology.orchestra.run.smithi164.stdout:(273/282): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-22T19:48:21.645 INFO:teuthology.orchestra.run.smithi164.stdout:(274/282): python3-webob-1.8.7-6.el9.noarch.rpm 3.1 MB/s | 230 kB 00:00 2024-04-22T19:48:21.670 INFO:teuthology.orchestra.run.smithi164.stdout:(275/282): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-22T19:48:21.705 INFO:teuthology.orchestra.run.smithi164.stdout:(276/282): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-22T19:48:21.729 INFO:teuthology.orchestra.run.smithi164.stdout:(277/282): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-22T19:48:21.755 INFO:teuthology.orchestra.run.smithi164.stdout:(278/282): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-22T19:48:21.781 INFO:teuthology.orchestra.run.smithi164.stdout:(279/282): s3cmd-2.4.0-1.el9.noarch.rpm 2.7 MB/s | 206 kB 00:00 2024-04-22T19:48:21.821 INFO:teuthology.orchestra.run.smithi164.stdout:(280/282): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-22T19:48:22.521 INFO:teuthology.orchestra.run.smithi164.stdout:(281/282): librbd1-19.0.0-3152.g9ed8263e.el7.x8 4.2 MB/s | 3.1 MB 00:00 2024-04-22T19:48:23.113 INFO:teuthology.orchestra.run.smithi164.stdout:(282/282): librados2-19.0.0-3152.g9ed8263e.el7. 2.5 MB/s | 3.4 MB 00:01 2024-04-22T19:48:23.117 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-04-22T19:48:23.117 INFO:teuthology.orchestra.run.smithi164.stdout:Total 8.5 MB/s | 305 MB 00:36 2024-04-22T19:48:25.338 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-22T19:48:26.049 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-04-22T19:48:26.235 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-04-22T19:48:26.236 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-04-22T19:48:27.441 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T19:48:27.441 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-22T19:48:27.529 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-22T19:48:27.545 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================================== 2024-04-22T19:48:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-22T19:48:27.545 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================================== 2024-04-22T19:48:27.545 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-22T19:48:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 6.6 k 2024-04-22T19:48:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 5.4 M 2024-04-22T19:48:27.545 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 890 k 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 145 k 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 1.5 M 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 142 k 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 3.5 M 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 7.4 M 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 49 k 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 11 M 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 49 M 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 273 k 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 762 k 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 32 k 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 746 k 2024-04-22T19:48:27.546 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 127 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: maven noarch 1:3.6.3-15.el9 appstream 18 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 161 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 321 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 298 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 99 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 85 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 3.1 M 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 171 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: s3cmd noarch 2.4.0-1.el9 epel 206 k 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 3.4 M 2024-04-22T19:48:27.547 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 3.1 M 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: ModemManager-glib x86_64 1.20.2-1.el9 baseos 335 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: adwaita-cursor-theme noarch 40.1.1-3.el9 appstream 650 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: adwaita-icon-theme noarch 40.1.1-3.el9 appstream 12 M 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: alsa-lib x86_64 1.2.10-2.el9 appstream 523 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: apache-commons-cli noarch 1.4-17.el9 appstream 71 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: apache-commons-codec noarch 1.15-7.el9 appstream 302 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: apache-commons-io noarch 1:2.8.0-8.el9 appstream 283 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: apache-commons-lang3 noarch 3.12.0-6.el9 appstream 559 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: at-spi2-atk x86_64 2.38.0-4.el9 appstream 87 k 2024-04-22T19:48:27.548 INFO:teuthology.orchestra.run.smithi136.stdout: at-spi2-core x86_64 2.40.3-1.el9 appstream 195 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: atinject noarch 1.0.3-6.el9 appstream 23 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: atk x86_64 2.36.0-5.el9 appstream 292 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: avahi-glib x86_64 0.8-20.el9 appstream 15 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: bluez-libs x86_64 5.56-6.el9 baseos 85 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: bubblewrap x86_64 0.4.1-7.el9 baseos 49 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: cdi-api noarch 2.0.2-6.el9 appstream 53 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 21 M 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 34 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 2.4 M 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 249 k 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 4.7 M 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 17 M 2024-04-22T19:48:27.549 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3152.g9ed8263e.el7 ceph-noarch 16 k 2024-04-22T19:48:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 25 k 2024-04-22T19:48:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: colord-libs x86_64 1.4.5-4.el9 appstream 229 k 2024-04-22T19:48:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: copy-jdk-configs noarch 4.0-3.el9 appstream 28 k 2024-04-22T19:48:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: cups-libs x86_64 1:2.3.3op2-24.el9 baseos 262 k 2024-04-22T19:48:27.550 INFO:teuthology.orchestra.run.smithi136.stdout: exempi x86_64 2.6.0-0.2.20211007gite23c213.el9 appstream 525 k 2024-04-22T19:48:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: exiv2-libs x86_64 0.27.5-2.el9 appstream 781 k 2024-04-22T19:48:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: fdk-aac-free x86_64 2.0.0-8.el9 appstream 325 k 2024-04-22T19:48:27.551 INFO:teuthology.orchestra.run.smithi136.stdout: flac-libs x86_64 1.3.3-12.el9 appstream 218 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: flatpak-selinux noarch 1.12.8-1.el9 appstream 21 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: flatpak-session-helper x86_64 1.12.8-1.el9 appstream 74 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: gdk-pixbuf2-modules x86_64 2.42.6-3.el9 appstream 88 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: geoclue2 x86_64 2.6.0-7.el9 appstream 124 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: giflib x86_64 5.2.1-9.el9 appstream 49 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: google-guice noarch 4.2.3-9.el9 appstream 473 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-22T19:48:27.552 INFO:teuthology.orchestra.run.smithi136.stdout: graphene x86_64 1.10.6-2.el9 appstream 64 k 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: gsm x86_64 1.0.19-6.el9 appstream 34 k 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: gstreamer1 x86_64 1.22.1-2.el9 appstream 1.4 M 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: gstreamer1-plugins-base x86_64 1.22.1-2.el9 appstream 2.2 M 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: gtk-update-icon-cache x86_64 3.24.31-2.el9 appstream 35 k 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: guava noarch 30.1-6.el9 appstream 2.3 M 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: hicolor-icon-theme noarch 0.17-13.el9 appstream 219 k 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: httpcomponents-client noarch 4.5.13-3.el9 appstream 657 k 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: httpcomponents-core noarch 4.4.13-7.el9 appstream 632 k 2024-04-22T19:48:27.553 INFO:teuthology.orchestra.run.smithi136.stdout: iso-codes noarch 4.6.0-3.el9 appstream 3.5 M 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: jakarta-annotations noarch 1.3.5-13.el9 appstream 45 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: jansi x86_64 2.3.3-6.el9 appstream 110 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: java-11-openjdk x86_64 1:11.0.18.0.10-3.el9 appstream 440 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: java-11-openjdk-headless x86_64 1:11.0.18.0.10-3.el9 appstream 40 M 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: javapackages-filesystem noarch 6.0.0-4.el9 appstream 13 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: jbigkit-libs x86_64 2.1-23.el9 appstream 53 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: jcl-over-slf4j noarch 1.7.30-13.el9 appstream 24 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: jsoup noarch 1.13.1-10.el9 appstream 377 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: jsr-305 noarch 3.0.2-6.el9 appstream 35 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: lcms2 x86_64 2.12-3.el9 appstream 168 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: libX11-xcb x86_64 1.7.0-9.el9 appstream 11 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: libXcomposite x86_64 0.4.5-7.el9 appstream 24 k 2024-04-22T19:48:27.554 INFO:teuthology.orchestra.run.smithi136.stdout: libXcursor x86_64 1.2.0-7.el9 appstream 31 k 2024-04-22T19:48:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: libXdamage x86_64 1.1.5-7.el9 appstream 23 k 2024-04-22T19:48:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: libXfixes x86_64 5.0.3-16.el9 appstream 20 k 2024-04-22T19:48:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: libXi x86_64 1.7.10-8.el9 appstream 40 k 2024-04-22T19:48:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: libXinerama x86_64 1.1.4-10.el9 appstream 15 k 2024-04-22T19:48:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: libXrandr x86_64 1.5.2-8.el9 appstream 28 k 2024-04-22T19:48:27.555 INFO:teuthology.orchestra.run.smithi136.stdout: libXtst x86_64 1.2.3-16.el9 appstream 21 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libXv x86_64 1.0.11-16.el9 appstream 19 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libXxf86vm x86_64 1.1.4-18.el9 appstream 19 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libasyncns x86_64 0.8-22.el9 appstream 30 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libcanberra x86_64 0.30-27.el9 appstream 86 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 163 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libdrm x86_64 2.4.117-1.el9 appstream 161 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libepoxy x86_64 1.5.5-4.el9 appstream 241 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libexif x86_64 0.6.22-6.el9 appstream 435 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libfontenc x86_64 1.1.3-17.el9 appstream 31 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libgexiv2 x86_64 0.12.3-1.el9 appstream 83 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-22T19:48:27.556 INFO:teuthology.orchestra.run.smithi136.stdout: libglvnd x86_64 1:1.3.4-1.el9 appstream 135 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libglvnd-egl x86_64 1:1.3.4-1.el9 appstream 37 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libglvnd-glx x86_64 1:1.3.4-1.el9 appstream 142 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libgsf x86_64 1.14.47-5.el9 appstream 256 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libgxps x86_64 0.3.2-3.el9 appstream 79 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libiptcdata x86_64 1.0.5-9.el9 appstream 62 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libldac x86_64 2.0.2.3-10.el9 appstream 41 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libnotify x86_64 0.7.9-8.el9 appstream 44 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libogg x86_64 2:1.3.4-6.el9 appstream 34 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libosinfo x86_64 1.10.0-1.el9 appstream 319 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libpciaccess x86_64 0.16-7.el9 baseos 27 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-22T19:48:27.557 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 500 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 5.4 M 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libsbc x86_64 1.4-9.el9 appstream 45 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libsndfile x86_64 1.0.31-8.el9 appstream 206 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libtheora x86_64 1:1.1.1-31.el9 appstream 164 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libtiff x86_64 4.4.0-12.el9 appstream 198 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-ltdl x86_64 2.4.6-45.el9 appstream 37 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libtracker-sparql x86_64 3.1.2-3.el9 appstream 324 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libvisual x86_64 1:0.4.0-34.el9 appstream 144 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libvorbis x86_64 1:1.3.7-5.el9 appstream 193 k 2024-04-22T19:48:27.558 INFO:teuthology.orchestra.run.smithi136.stdout: libwayland-client x86_64 1.21.0-1.el9 appstream 33 k 2024-04-22T19:48:27.559 INFO:teuthology.orchestra.run.smithi136.stdout: libwayland-cursor x86_64 1.21.0-1.el9 appstream 19 k 2024-04-22T19:48:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: libwayland-egl x86_64 1.21.0-1.el9 appstream 13 k 2024-04-22T19:48:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: libwayland-server x86_64 1.21.0-1.el9 appstream 42 k 2024-04-22T19:48:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: libwebp x86_64 1.2.0-8.el9 appstream 277 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: libxkbcommon x86_64 1.0.3-4.el9 appstream 133 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: libxshmfence x86_64 1.3-10.el9 appstream 13 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: low-memory-monitor x86_64 2.1-4.el9 appstream 36 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: lua-posix x86_64 35.0-8.el9 appstream 151 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: maven-lib noarch 1:3.6.3-15.el9 appstream 1.5 M 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: maven-openjdk11 noarch 1:3.6.3-15.el9 appstream 9.0 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: maven-resolver noarch 1:1.6.1-10.el9 appstream 517 k 2024-04-22T19:48:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: maven-shared-utils noarch 3.3.4-3.el9 appstream 151 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: maven-wagon noarch 3.4.2-7.el9 appstream 112 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: mesa-libEGL x86_64 23.3.3-1.el9 appstream 127 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: mesa-libGL x86_64 23.3.3-1.el9 appstream 168 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: mesa-libgbm x86_64 23.3.3-1.el9 appstream 38 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: mesa-libglapi x86_64 23.3.3-1.el9 appstream 44 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: mkfontscale x86_64 1.2.1-3.el9 appstream 32 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: openjpeg2 x86_64 2.4.0-7.el9 appstream 163 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: opus x86_64 1.3.1-10.el9 appstream 200 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: orc x86_64 0.4.31-6.el9 appstream 184 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: osinfo-db noarch 20231215-1.el9 appstream 521 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: osinfo-db-tools x86_64 1.10.0-1.el9 appstream 74 k 2024-04-22T19:48:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: ostree-libs x86_64 2024.5-2.el9 appstream 488 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-jack-audio-connection-kit-libs x86_64 1.0.1-1.el9 appstream 135 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-libs x86_64 1.0.1-1.el9 appstream 1.9 M 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-cipher noarch 1.7-27.el9 appstream 26 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-classworlds noarch 2.6.0-11.el9 appstream 60 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-containers-component-annotations noarch 2.1.0-10.el9 appstream 18 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-interpolation noarch 1.26-11.el9 appstream 80 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-sec-dispatcher noarch 1.4-37.el9 appstream 34 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-utils noarch 3.3.0-10.el9 appstream 254 k 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: poppler x86_64 21.01.0-19.el9 appstream 1.1 M 2024-04-22T19:48:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: poppler-data noarch 0.4.9-9.el9 appstream 1.9 M 2024-04-22T19:48:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: poppler-glib x86_64 21.01.0-19.el9 appstream 152 k 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: publicsuffix-list noarch 20210518-3.el9 appstream 83 k 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: pulseaudio-libs x86_64 15.0-2.el9 appstream 679 k 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-22T19:48:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 45 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.0.0-3152.g9ed8263e.el7 ceph 135 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-22T19:48:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-22T19:48:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-22T19:48:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-22T19:48:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-22T19:48:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-22T19:48:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-22T19:48:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-22T19:48:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: rtkit x86_64 0.11-28.el9 appstream 56 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: sisu noarch 1:0.3.4-10.el9 appstream 510 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: slf4j noarch 1.7.30-13.el9 appstream 69 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: sound-theme-freedesktop noarch 0.8-17.el9 appstream 383 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: totem-pl-parser x86_64 3.26.6-2.el9 appstream 152 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: tracker x86_64 3.1.2-3.el9 appstream 555 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: ttmkfdir x86_64 3.0.9-65.el9 appstream 53 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: tzdata-java noarch 2024a-2.el9 appstream 230 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: upower x86_64 0.99.11-11.el9 appstream 173 k 2024-04-22T19:48:27.581 INFO:teuthology.orchestra.run.smithi136.stdout: webrtc-audio-processing x86_64 0.3.1-8.el9 appstream 306 k 2024-04-22T19:48:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: wireplumber x86_64 0.4.14-1.el9 appstream 96 k 2024-04-22T19:48:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: wireplumber-libs x86_64 0.4.14-1.el9 appstream 358 k 2024-04-22T19:48:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: xdg-dbus-proxy x86_64 0.1.3-1.el9 appstream 42 k 2024-04-22T19:48:27.582 INFO:teuthology.orchestra.run.smithi136.stdout: xdg-desktop-portal x86_64 1.12.6-1.el9 appstream 385 k 2024-04-22T19:48:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: xkeyboard-config noarch 2.33-2.el9 appstream 859 k 2024-04-22T19:48:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-22T19:48:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: xorg-x11-fonts-Type1 noarch 7.5-33.el9 appstream 505 k 2024-04-22T19:48:27.587 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-04-22T19:48:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: dconf x86_64 0.40.0-6.el9 appstream 113 k 2024-04-22T19:48:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: exiv2 x86_64 0.27.5-2.el9 appstream 980 k 2024-04-22T19:48:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: flatpak x86_64 1.12.8-1.el9 appstream 1.7 M 2024-04-22T19:48:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: gtk3 x86_64 3.24.31-2.el9 appstream 4.9 M 2024-04-22T19:48:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: java-11-openjdk-devel x86_64 1:11.0.18.0.10-3.el9 appstream 3.3 M 2024-04-22T19:48:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: libcanberra-gtk3 x86_64 0.30-27.el9 appstream 32 k 2024-04-22T19:48:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: p11-kit-server x86_64 0.25.3-2.el9 appstream 246 k 2024-04-22T19:48:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire x86_64 1.0.1-1.el9 appstream 107 k 2024-04-22T19:48:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-alsa x86_64 1.0.1-1.el9 appstream 57 k 2024-04-22T19:48:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-jack-audio-connection-kit x86_64 1.0.1-1.el9 appstream 9.1 k 2024-04-22T19:48:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-pulseaudio x86_64 1.0.1-1.el9 appstream 192 k 2024-04-22T19:48:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-22T19:48:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: tracker-miners x86_64 3.1.2-4.el9 appstream 944 k 2024-04-22T19:48:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: xdg-desktop-portal-gtk x86_64 1.12.0-3.el9 appstream 134 k 2024-04-22T19:48:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:48:27.589 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-22T19:48:27.589 INFO:teuthology.orchestra.run.smithi136.stdout:====================================================================================================== 2024-04-22T19:48:27.589 INFO:teuthology.orchestra.run.smithi136.stdout:Install 280 Packages 2024-04-22T19:48:27.589 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 2 Packages 2024-04-22T19:48:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:48:27.590 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 305 M 2024-04-22T19:48:27.590 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-22T19:48:28.355 INFO:teuthology.orchestra.run.smithi136.stdout:(1/282): ceph-19.0.0-3152.g9ed8263e.el7.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-22T19:48:28.747 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-04-22T19:48:28.747 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-04-22T19:48:29.099 INFO:teuthology.orchestra.run.smithi136.stdout:(2/282): ceph-fuse-19.0.0-3152.g9ed8263e.el7.x8 1.2 MB/s | 890 kB 00:00 2024-04-22T19:48:29.398 INFO:teuthology.orchestra.run.smithi136.stdout:(3/282): ceph-common-19.0.0-3152.g9ed8263e.el7. 17 MB/s | 21 MB 00:01 2024-04-22T19:48:29.423 INFO:teuthology.orchestra.run.smithi136.stdout:(4/282): ceph-immutable-object-cache-19.0.0-315 445 kB/s | 145 kB 00:00 2024-04-22T19:48:29.565 INFO:teuthology.orchestra.run.smithi136.stdout:(5/282): ceph-mds-19.0.0-3152.g9ed8263e.el7.x86 14 MB/s | 2.4 MB 00:00 2024-04-22T19:48:29.857 INFO:teuthology.orchestra.run.smithi136.stdout:(6/282): ceph-mon-19.0.0-3152.g9ed8263e.el7.x86 16 MB/s | 4.7 MB 00:00 2024-04-22T19:48:30.232 INFO:teuthology.orchestra.run.smithi136.stdout:(7/282): ceph-mgr-19.0.0-3152.g9ed8263e.el7.x86 1.8 MB/s | 1.5 MB 00:00 2024-04-22T19:48:30.415 INFO:teuthology.orchestra.run.smithi136.stdout:(8/282): ceph-base-19.0.0-3152.g9ed8263e.el7.x8 2.3 MB/s | 5.4 MB 00:02 2024-04-22T19:48:30.492 INFO:teuthology.orchestra.run.smithi136.stdout:(9/282): ceph-selinux-19.0.0-3152.g9ed8263e.el7 335 kB/s | 25 kB 00:00 2024-04-22T19:48:30.775 INFO:teuthology.orchestra.run.smithi136.stdout:(10/282): ceph-osd-19.0.0-3152.g9ed8263e.el7.x8 18 MB/s | 17 MB 00:00 2024-04-22T19:48:30.816 INFO:teuthology.orchestra.run.smithi136.stdout:(11/282): libcephfs-devel-19.0.0-3152.g9ed8263e 767 kB/s | 32 kB 00:00 2024-04-22T19:48:30.900 INFO:teuthology.orchestra.run.smithi136.stdout:(12/282): libcephfs2-19.0.0-3152.g9ed8263e.el7. 8.8 MB/s | 746 kB 00:00 2024-04-22T19:48:30.958 INFO:teuthology.orchestra.run.smithi136.stdout:(13/282): libcephsqlite-19.0.0-3152.g9ed8263e.e 2.7 MB/s | 163 kB 00:00 2024-04-22T19:48:31.017 INFO:teuthology.orchestra.run.smithi136.stdout:(14/282): librados-devel-19.0.0-3152.g9ed8263e. 2.1 MB/s | 127 kB 00:00 2024-04-22T19:48:31.151 INFO:teuthology.orchestra.run.smithi136.stdout:(15/282): libradosstriper1-19.0.0-3152.g9ed8263 3.7 MB/s | 500 kB 00:00 2024-04-22T19:48:31.526 INFO:teuthology.orchestra.run.smithi136.stdout:(16/282): librgw2-19.0.0-3152.g9ed8263e.el7.x86 14 MB/s | 5.4 MB 00:00 2024-04-22T19:48:31.560 INFO:teuthology.orchestra.run.smithi136.stdout:(17/282): python3-ceph-argparse-19.0.0-3152.g9e 1.3 MB/s | 45 kB 00:00 2024-04-22T19:48:31.610 INFO:teuthology.orchestra.run.smithi136.stdout:(18/282): python3-ceph-common-19.0.0-3152.g9ed8 2.6 MB/s | 135 kB 00:00 2024-04-22T19:48:31.660 INFO:teuthology.orchestra.run.smithi136.stdout:(19/282): python3-cephfs-19.0.0-3152.g9ed8263e. 3.2 MB/s | 161 kB 00:00 2024-04-22T19:48:31.727 INFO:teuthology.orchestra.run.smithi136.stdout:(20/282): python3-rados-19.0.0-3152.g9ed8263e.e 4.7 MB/s | 321 kB 00:00 2024-04-22T19:48:31.786 INFO:teuthology.orchestra.run.smithi136.stdout:(21/282): python3-rbd-19.0.0-3152.g9ed8263e.el7 5.0 MB/s | 298 kB 00:00 2024-04-22T19:48:31.836 INFO:teuthology.orchestra.run.smithi136.stdout:(22/282): python3-rgw-19.0.0-3152.g9ed8263e.el7 1.9 MB/s | 99 kB 00:00 2024-04-22T19:48:31.879 INFO:teuthology.orchestra.run.smithi136.stdout:(23/282): rbd-fuse-19.0.0-3152.g9ed8263e.el7.x8 2.0 MB/s | 85 kB 00:00 2024-04-22T19:48:32.121 INFO:teuthology.orchestra.run.smithi136.stdout:(24/282): rbd-mirror-19.0.0-3152.g9ed8263e.el7. 13 MB/s | 3.1 MB 00:00 2024-04-22T19:48:32.188 INFO:teuthology.orchestra.run.smithi136.stdout:(25/282): rbd-nbd-19.0.0-3152.g9ed8263e.el7.x86 2.5 MB/s | 171 kB 00:00 2024-04-22T19:48:32.255 INFO:teuthology.orchestra.run.smithi136.stdout:(26/282): ceph-grafana-dashboards-19.0.0-3152.g 515 kB/s | 34 kB 00:00 2024-04-22T19:48:32.322 INFO:teuthology.orchestra.run.smithi136.stdout:(27/282): ceph-mgr-cephadm-19.0.0-3152.g9ed8263 2.1 MB/s | 142 kB 00:00 2024-04-22T19:48:32.672 INFO:teuthology.orchestra.run.smithi136.stdout:(28/282): ceph-mgr-dashboard-19.0.0-3152.g9ed82 10 MB/s | 3.5 MB 00:00 2024-04-22T19:48:33.280 INFO:teuthology.orchestra.run.smithi136.stdout:(29/282): ceph-mgr-diskprediction-local-19.0.0- 12 MB/s | 7.4 MB 00:00 2024-04-22T19:48:33.347 INFO:teuthology.orchestra.run.smithi136.stdout:(30/282): ceph-mgr-modules-core-19.0.0-3152.g9e 3.7 MB/s | 249 kB 00:00 2024-04-22T19:48:33.390 INFO:teuthology.orchestra.run.smithi136.stdout:(31/282): ceph-mgr-rook-19.0.0-3152.g9ed8263e.e 1.2 MB/s | 49 kB 00:00 2024-04-22T19:48:33.448 INFO:teuthology.orchestra.run.smithi136.stdout:(32/282): ceph-prometheus-alerts-19.0.0-3152.g9 282 kB/s | 16 kB 00:00 2024-04-22T19:48:33.532 INFO:teuthology.orchestra.run.smithi136.stdout:(33/282): ceph-volume-19.0.0-3152.g9ed8263e.el7 3.2 MB/s | 273 kB 00:00 2024-04-22T19:48:33.615 INFO:teuthology.orchestra.run.smithi136.stdout:(34/282): cephadm-19.0.0-3152.g9ed8263e.el7.noa 9.0 MB/s | 762 kB 00:00 2024-04-22T19:48:34.082 INFO:teuthology.orchestra.run.smithi136.stdout:(35/282): ModemManager-glib-1.20.2-1.el9.x86_64 718 kB/s | 335 kB 00:00 2024-04-22T19:48:34.182 INFO:teuthology.orchestra.run.smithi136.stdout:(36/282): bluez-libs-5.56-6.el9.x86_64.rpm 847 kB/s | 85 kB 00:00 2024-04-22T19:48:34.609 INFO:teuthology.orchestra.run.smithi136.stdout:(37/282): ceph-test-19.0.0-3152.g9ed8263e.el7.x 12 MB/s | 49 MB 00:04 2024-04-22T19:48:34.675 INFO:teuthology.orchestra.run.smithi136.stdout:(38/282): bubblewrap-0.4.1-7.el9.x86_64.rpm 100 kB/s | 49 kB 00:00 2024-04-22T19:48:34.783 INFO:teuthology.orchestra.run.smithi136.stdout:(39/282): fuse-2.9.9-15.el9.x86_64.rpm 988 kB/s | 80 kB 00:00 2024-04-22T19:48:34.859 INFO:teuthology.orchestra.run.smithi136.stdout:(40/282): cups-libs-2.3.3op2-24.el9.x86_64.rpm 1.0 MB/s | 262 kB 00:00 2024-04-22T19:48:34.917 INFO:teuthology.orchestra.run.smithi136.stdout:(41/282): libpciaccess-0.16-7.el9.x86_64.rpm 463 kB/s | 27 kB 00:00 2024-04-22T19:48:35.034 INFO:teuthology.orchestra.run.smithi136.stdout:(42/282): ceph-radosgw-19.0.0-3152.g9ed8263e.el 2.2 MB/s | 11 MB 00:04 2024-04-22T19:48:35.078 INFO:teuthology.orchestra.run.smithi136.stdout:(43/282): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-04-22T19:48:35.101 INFO:teuthology.orchestra.run.smithi136.stdout:(44/282): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-22T19:48:35.135 INFO:teuthology.orchestra.run.smithi136.stdout:(45/282): lksctp-tools-1.0.19-2.el9.x86_64.rpm 934 kB/s | 94 kB 00:00 2024-04-22T19:48:35.160 INFO:teuthology.orchestra.run.smithi136.stdout:(46/282): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 253 kB 00:00 2024-04-22T19:48:35.219 INFO:teuthology.orchestra.run.smithi136.stdout:(47/282): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-22T19:48:35.311 INFO:teuthology.orchestra.run.smithi136.stdout:(48/282): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-04-22T19:48:35.378 INFO:teuthology.orchestra.run.smithi136.stdout:(49/282): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-04-22T19:48:35.421 INFO:teuthology.orchestra.run.smithi136.stdout:(50/282): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-04-22T19:48:35.445 INFO:teuthology.orchestra.run.smithi136.stdout:(51/282): python3-pycparser-2.20-6.el9.noarch.r 475 kB/s | 135 kB 00:00 2024-04-22T19:48:35.604 INFO:teuthology.orchestra.run.smithi136.stdout:(52/282): adwaita-cursor-theme-40.1.1-3.el9.noa 2.8 MB/s | 650 kB 00:00 2024-04-22T19:48:35.629 INFO:teuthology.orchestra.run.smithi136.stdout:(53/282): alsa-lib-1.2.10-2.el9.x86_64.rpm 2.8 MB/s | 523 kB 00:00 2024-04-22T19:48:35.654 INFO:teuthology.orchestra.run.smithi136.stdout:(54/282): apache-commons-cli-1.4-17.el9.noarch. 1.4 MB/s | 71 kB 00:00 2024-04-22T19:48:35.680 INFO:teuthology.orchestra.run.smithi136.stdout:(55/282): apache-commons-codec-1.15-7.el9.noarc 5.9 MB/s | 302 kB 00:00 2024-04-22T19:48:35.705 INFO:teuthology.orchestra.run.smithi136.stdout:(56/282): apache-commons-io-2.8.0-8.el9.noarch. 5.5 MB/s | 283 kB 00:00 2024-04-22T19:48:35.739 INFO:teuthology.orchestra.run.smithi136.stdout:(57/282): apache-commons-lang3-3.12.0-6.el9.noa 9.3 MB/s | 559 kB 00:00 2024-04-22T19:48:35.764 INFO:teuthology.orchestra.run.smithi136.stdout:(58/282): at-spi2-atk-2.38.0-4.el9.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2024-04-22T19:48:35.781 INFO:teuthology.orchestra.run.smithi136.stdout:(59/282): at-spi2-core-2.40.3-1.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-22T19:48:35.814 INFO:teuthology.orchestra.run.smithi136.stdout:(60/282): atinject-1.0.3-6.el9.noarch.rpm 464 kB/s | 23 kB 00:00 2024-04-22T19:48:35.840 INFO:teuthology.orchestra.run.smithi136.stdout:(61/282): atk-2.36.0-5.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-22T19:48:35.865 INFO:teuthology.orchestra.run.smithi136.stdout:(62/282): avahi-glib-0.8-20.el9.x86_64.rpm 297 kB/s | 15 kB 00:00 2024-04-22T19:48:35.907 INFO:teuthology.orchestra.run.smithi136.stdout:(63/282): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-22T19:48:35.932 INFO:teuthology.orchestra.run.smithi136.stdout:(64/282): cdi-api-2.0.2-6.el9.noarch.rpm 798 kB/s | 53 kB 00:00 2024-04-22T19:48:35.957 INFO:teuthology.orchestra.run.smithi136.stdout:(65/282): colord-libs-1.4.5-4.el9.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2024-04-22T19:48:35.983 INFO:teuthology.orchestra.run.smithi136.stdout:(66/282): copy-jdk-configs-4.0-3.el9.noarch.rpm 562 kB/s | 28 kB 00:00 2024-04-22T19:48:36.008 INFO:teuthology.orchestra.run.smithi136.stdout:(67/282): dconf-0.40.0-6.el9.x86_64.rpm 2.2 MB/s | 113 kB 00:00 2024-04-22T19:48:36.033 INFO:teuthology.orchestra.run.smithi136.stdout:(68/282): exempi-2.6.0-0.2.20211007gite23c213.e 10 MB/s | 525 kB 00:00 2024-04-22T19:48:36.167 INFO:teuthology.orchestra.run.smithi136.stdout:(69/282): adwaita-icon-theme-40.1.1-3.el9.noarc 16 MB/s | 12 MB 00:00 2024-04-22T19:48:36.209 INFO:teuthology.orchestra.run.smithi136.stdout:(70/282): exiv2-0.27.5-2.el9.x86_64.rpm 4.8 MB/s | 980 kB 00:00 2024-04-22T19:48:36.234 INFO:teuthology.orchestra.run.smithi136.stdout:(71/282): fdk-aac-free-2.0.0-8.el9.x86_64.rpm 4.8 MB/s | 325 kB 00:00 2024-04-22T19:48:36.268 INFO:teuthology.orchestra.run.smithi136.stdout:(72/282): exiv2-libs-0.27.5-2.el9.x86_64.rpm 3.3 MB/s | 781 kB 00:00 2024-04-22T19:48:36.293 INFO:teuthology.orchestra.run.smithi136.stdout:(73/282): flac-libs-1.3.3-12.el9.x86_64.rpm 2.5 MB/s | 218 kB 00:00 2024-04-22T19:48:36.310 INFO:teuthology.orchestra.run.smithi136.stdout:(74/282): flatpak-selinux-1.12.8-1.el9.noarch.r 508 kB/s | 21 kB 00:00 2024-04-22T19:48:36.335 INFO:teuthology.orchestra.run.smithi136.stdout:(75/282): flatpak-session-helper-1.12.8-1.el9.x 1.7 MB/s | 74 kB 00:00 2024-04-22T19:48:36.352 INFO:teuthology.orchestra.run.smithi136.stdout:(76/282): flexiblas-3.0.4-8.el9.x86_64.rpm 781 kB/s | 33 kB 00:00 2024-04-22T19:48:36.395 INFO:teuthology.orchestra.run.smithi136.stdout:(77/282): flatpak-1.12.8-1.el9.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2024-04-22T19:48:36.412 INFO:teuthology.orchestra.run.smithi136.stdout:(78/282): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-04-22T19:48:36.439 INFO:teuthology.orchestra.run.smithi136.stdout:(79/282): gdk-pixbuf2-modules-2.42.6-3.el9.x86_ 2.1 MB/s | 88 kB 00:00 2024-04-22T19:48:36.462 INFO:teuthology.orchestra.run.smithi136.stdout:(80/282): geoclue2-2.6.0-7.el9.x86_64.rpm 2.4 MB/s | 124 kB 00:00 2024-04-22T19:48:36.488 INFO:teuthology.orchestra.run.smithi136.stdout:(81/282): giflib-5.2.1-9.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-04-22T19:48:36.513 INFO:teuthology.orchestra.run.smithi136.stdout:(82/282): google-guice-4.2.3-9.el9.noarch.rpm 9.2 MB/s | 473 kB 00:00 2024-04-22T19:48:36.538 INFO:teuthology.orchestra.run.smithi136.stdout:(83/282): graphene-1.10.6-2.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-22T19:48:36.563 INFO:teuthology.orchestra.run.smithi136.stdout:(84/282): gsm-1.0.19-6.el9.x86_64.rpm 677 kB/s | 34 kB 00:00 2024-04-22T19:48:36.614 INFO:teuthology.orchestra.run.smithi136.stdout:(85/282): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-22T19:48:36.655 INFO:teuthology.orchestra.run.smithi136.stdout:(86/282): gstreamer1-1.22.1-2.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-04-22T19:48:36.681 INFO:teuthology.orchestra.run.smithi136.stdout:(87/282): gtk-update-icon-cache-3.24.31-2.el9.x 531 kB/s | 35 kB 00:00 2024-04-22T19:48:36.739 INFO:teuthology.orchestra.run.smithi136.stdout:(88/282): gstreamer1-plugins-base-1.22.1-2.el9. 13 MB/s | 2.2 MB 00:00 2024-04-22T19:48:36.823 INFO:teuthology.orchestra.run.smithi136.stdout:(89/282): gtk3-3.24.31-2.el9.x86_64.rpm 29 MB/s | 4.9 MB 00:00 2024-04-22T19:48:36.848 INFO:teuthology.orchestra.run.smithi136.stdout:(90/282): hicolor-icon-theme-0.17-13.el9.noarch 2.0 MB/s | 219 kB 00:00 2024-04-22T19:48:36.890 INFO:teuthology.orchestra.run.smithi136.stdout:(91/282): httpcomponents-client-4.5.13-3.el9.no 9.6 MB/s | 657 kB 00:00 2024-04-22T19:48:36.940 INFO:teuthology.orchestra.run.smithi136.stdout:(92/282): guava-30.1-6.el9.noarch.rpm 8.9 MB/s | 2.3 MB 00:00 2024-04-22T19:48:36.974 INFO:teuthology.orchestra.run.smithi136.stdout:(93/282): httpcomponents-core-4.4.13-7.el9.noar 4.9 MB/s | 632 kB 00:00 2024-04-22T19:48:37.002 INFO:teuthology.orchestra.run.smithi136.stdout:(94/282): jakarta-annotations-1.3.5-13.el9.noar 771 kB/s | 45 kB 00:00 2024-04-22T19:48:37.025 INFO:teuthology.orchestra.run.smithi136.stdout:(95/282): jansi-2.3.3-6.el9.x86_64.rpm 2.1 MB/s | 110 kB 00:00 2024-04-22T19:48:37.050 INFO:teuthology.orchestra.run.smithi136.stdout:(96/282): java-11-openjdk-11.0.18.0.10-3.el9.x8 8.5 MB/s | 440 kB 00:00 2024-04-22T19:48:37.117 INFO:teuthology.orchestra.run.smithi136.stdout:(97/282): iso-codes-4.6.0-3.el9.noarch.rpm 16 MB/s | 3.5 MB 00:00 2024-04-22T19:48:37.184 INFO:teuthology.orchestra.run.smithi136.stdout:(98/282): java-11-openjdk-devel-11.0.18.0.10-3. 21 MB/s | 3.3 MB 00:00 2024-04-22T19:48:37.209 INFO:teuthology.orchestra.run.smithi136.stdout:(99/282): javapackages-filesystem-6.0.0-4.el9.n 142 kB/s | 13 kB 00:00 2024-04-22T19:48:37.234 INFO:teuthology.orchestra.run.smithi136.stdout:(100/282): jbigkit-libs-2.1-23.el9.x86_64.rpm 1.0 MB/s | 53 kB 00:00 2024-04-22T19:48:37.260 INFO:teuthology.orchestra.run.smithi136.stdout:(101/282): jcl-over-slf4j-1.7.30-13.el9.noarch. 478 kB/s | 24 kB 00:00 2024-04-22T19:48:37.298 INFO:teuthology.orchestra.run.smithi136.stdout:(102/282): jsoup-1.13.1-10.el9.noarch.rpm 6.3 MB/s | 377 kB 00:00 2024-04-22T19:48:37.326 INFO:teuthology.orchestra.run.smithi136.stdout:(103/282): jsr-305-3.0.2-6.el9.noarch.rpm 606 kB/s | 35 kB 00:00 2024-04-22T19:48:37.344 INFO:teuthology.orchestra.run.smithi136.stdout:(104/282): lcms2-2.12-3.el9.x86_64.rpm 3.3 MB/s | 168 kB 00:00 2024-04-22T19:48:37.370 INFO:teuthology.orchestra.run.smithi136.stdout:(105/282): libX11-xcb-1.7.0-9.el9.x86_64.rpm 220 kB/s | 11 kB 00:00 2024-04-22T19:48:37.395 INFO:teuthology.orchestra.run.smithi136.stdout:(106/282): libXcomposite-0.4.5-7.el9.x86_64.rpm 481 kB/s | 24 kB 00:00 2024-04-22T19:48:37.412 INFO:teuthology.orchestra.run.smithi136.stdout:(107/282): libXcursor-1.2.0-7.el9.x86_64.rpm 735 kB/s | 31 kB 00:00 2024-04-22T19:48:37.437 INFO:teuthology.orchestra.run.smithi136.stdout:(108/282): libXdamage-1.1.5-7.el9.x86_64.rpm 545 kB/s | 23 kB 00:00 2024-04-22T19:48:37.466 INFO:teuthology.orchestra.run.smithi136.stdout:(109/282): libXfixes-5.0.3-16.el9.x86_64.rpm 395 kB/s | 20 kB 00:00 2024-04-22T19:48:37.488 INFO:teuthology.orchestra.run.smithi136.stdout:(110/282): libXi-1.7.10-8.el9.x86_64.rpm 789 kB/s | 40 kB 00:00 2024-04-22T19:48:37.513 INFO:teuthology.orchestra.run.smithi136.stdout:(111/282): libXinerama-1.1.4-10.el9.x86_64.rpm 299 kB/s | 15 kB 00:00 2024-04-22T19:48:37.531 INFO:teuthology.orchestra.run.smithi136.stdout:(112/282): libXrandr-1.5.2-8.el9.x86_64.rpm 679 kB/s | 28 kB 00:00 2024-04-22T19:48:37.547 INFO:teuthology.orchestra.run.smithi136.stdout:(113/282): libXtst-1.2.3-16.el9.x86_64.rpm 637 kB/s | 21 kB 00:00 2024-04-22T19:48:37.573 INFO:teuthology.orchestra.run.smithi136.stdout:(114/282): libXv-1.0.11-16.el9.x86_64.rpm 456 kB/s | 19 kB 00:00 2024-04-22T19:48:37.589 INFO:teuthology.orchestra.run.smithi136.stdout:(115/282): libXxf86vm-1.1.4-18.el9.x86_64.rpm 449 kB/s | 19 kB 00:00 2024-04-22T19:48:37.606 INFO:teuthology.orchestra.run.smithi136.stdout:(116/282): libasyncns-0.8-22.el9.x86_64.rpm 916 kB/s | 30 kB 00:00 2024-04-22T19:48:37.632 INFO:teuthology.orchestra.run.smithi136.stdout:(117/282): libcanberra-0.30-27.el9.x86_64.rpm 2.0 MB/s | 86 kB 00:00 2024-04-22T19:48:37.649 INFO:teuthology.orchestra.run.smithi136.stdout:(118/282): libcanberra-gtk3-0.30-27.el9.x86_64. 755 kB/s | 32 kB 00:00 2024-04-22T19:48:37.673 INFO:teuthology.orchestra.run.smithi136.stdout:(119/282): libdrm-2.4.117-1.el9.x86_64.rpm 3.8 MB/s | 161 kB 00:00 2024-04-22T19:48:37.699 INFO:teuthology.orchestra.run.smithi136.stdout:(120/282): libepoxy-1.5.5-4.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-22T19:48:37.725 INFO:teuthology.orchestra.run.smithi136.stdout:(121/282): libexif-0.6.22-6.el9.x86_64.rpm 8.5 MB/s | 435 kB 00:00 2024-04-22T19:48:37.750 INFO:teuthology.orchestra.run.smithi136.stdout:(122/282): libfontenc-1.1.3-17.el9.x86_64.rpm 616 kB/s | 31 kB 00:00 2024-04-22T19:48:37.775 INFO:teuthology.orchestra.run.smithi136.stdout:(123/282): libgexiv2-0.12.3-1.el9.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2024-04-22T19:48:37.800 INFO:teuthology.orchestra.run.smithi136.stdout:(124/282): libglvnd-1.3.4-1.el9.x86_64.rpm 2.6 MB/s | 135 kB 00:00 2024-04-22T19:48:37.825 INFO:teuthology.orchestra.run.smithi136.stdout:(125/282): libglvnd-egl-1.3.4-1.el9.x86_64.rpm 730 kB/s | 37 kB 00:00 2024-04-22T19:48:37.851 INFO:teuthology.orchestra.run.smithi136.stdout:(126/282): libglvnd-glx-1.3.4-1.el9.x86_64.rpm 2.8 MB/s | 142 kB 00:00 2024-04-22T19:48:37.884 INFO:teuthology.orchestra.run.smithi136.stdout:(127/282): libgsf-1.14.47-5.el9.x86_64.rpm 4.3 MB/s | 256 kB 00:00 2024-04-22T19:48:37.910 INFO:teuthology.orchestra.run.smithi136.stdout:(128/282): libgxps-0.3.2-3.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-04-22T19:48:37.939 INFO:teuthology.orchestra.run.smithi136.stdout:(129/282): libiptcdata-1.0.5-9.el9.x86_64.rpm 1.2 MB/s | 62 kB 00:00 2024-04-22T19:48:37.960 INFO:teuthology.orchestra.run.smithi136.stdout:(130/282): libldac-2.0.2.3-10.el9.x86_64.rpm 820 kB/s | 41 kB 00:00 2024-04-22T19:48:37.986 INFO:teuthology.orchestra.run.smithi136.stdout:(131/282): libnotify-0.7.9-8.el9.x86_64.rpm 864 kB/s | 44 kB 00:00 2024-04-22T19:48:38.011 INFO:teuthology.orchestra.run.smithi136.stdout:(132/282): libogg-1.3.4-6.el9.x86_64.rpm 667 kB/s | 34 kB 00:00 2024-04-22T19:48:38.036 INFO:teuthology.orchestra.run.smithi136.stdout:(133/282): libosinfo-1.10.0-1.el9.x86_64.rpm 6.2 MB/s | 319 kB 00:00 2024-04-22T19:48:38.062 INFO:teuthology.orchestra.run.smithi136.stdout:(134/282): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-22T19:48:38.087 INFO:teuthology.orchestra.run.smithi136.stdout:(135/282): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-04-22T19:48:38.121 INFO:teuthology.orchestra.run.smithi136.stdout:(136/282): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-22T19:48:38.146 INFO:teuthology.orchestra.run.smithi136.stdout:(137/282): libsbc-1.4-9.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-04-22T19:48:38.171 INFO:teuthology.orchestra.run.smithi136.stdout:(138/282): libsndfile-1.0.31-8.el9.x86_64.rpm 4.0 MB/s | 206 kB 00:00 2024-04-22T19:48:38.196 INFO:teuthology.orchestra.run.smithi136.stdout:(139/282): libtheora-1.1.1-31.el9.x86_64.rpm 3.2 MB/s | 164 kB 00:00 2024-04-22T19:48:38.224 INFO:teuthology.orchestra.run.smithi136.stdout:(140/282): libtiff-4.4.0-12.el9.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2024-04-22T19:48:38.247 INFO:teuthology.orchestra.run.smithi136.stdout:(141/282): libtool-ltdl-2.4.6-45.el9.x86_64.rpm 737 kB/s | 37 kB 00:00 2024-04-22T19:48:38.273 INFO:teuthology.orchestra.run.smithi136.stdout:(142/282): libtracker-sparql-3.1.2-3.el9.x86_64 6.3 MB/s | 324 kB 00:00 2024-04-22T19:48:38.298 INFO:teuthology.orchestra.run.smithi136.stdout:(143/282): libvisual-0.4.0-34.el9.x86_64.rpm 2.8 MB/s | 144 kB 00:00 2024-04-22T19:48:38.346 INFO:teuthology.orchestra.run.smithi136.stdout:(144/282): libvorbis-1.3.7-5.el9.x86_64.rpm 2.6 MB/s | 193 kB 00:00 2024-04-22T19:48:38.373 INFO:teuthology.orchestra.run.smithi136.stdout:(145/282): libwayland-client-1.21.0-1.el9.x86_6 443 kB/s | 33 kB 00:00 2024-04-22T19:48:38.390 INFO:teuthology.orchestra.run.smithi136.stdout:(146/282): libwayland-cursor-1.21.0-1.el9.x86_6 429 kB/s | 19 kB 00:00 2024-04-22T19:48:38.416 INFO:teuthology.orchestra.run.smithi136.stdout:(147/282): libwayland-egl-1.21.0-1.el9.x86_64.r 306 kB/s | 13 kB 00:00 2024-04-22T19:48:38.441 INFO:teuthology.orchestra.run.smithi136.stdout:(148/282): libwayland-server-1.21.0-1.el9.x86_6 829 kB/s | 42 kB 00:00 2024-04-22T19:48:38.463 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: copy-jdk-configs-4.0-3.el9.noarch 1/1 2024-04-22T19:48:38.466 INFO:teuthology.orchestra.run.smithi136.stdout:(149/282): libwebp-1.2.0-8.el9.x86_64.rpm 5.4 MB/s | 277 kB 00:00 2024-04-22T19:48:38.493 INFO:teuthology.orchestra.run.smithi136.stdout:(150/282): libxkbcommon-1.0.3-4.el9.x86_64.rpm 2.6 MB/s | 133 kB 00:00 2024-04-22T19:48:38.509 INFO:teuthology.orchestra.run.smithi136.stdout:(151/282): libxshmfence-1.3-10.el9.x86_64.rpm 300 kB/s | 13 kB 00:00 2024-04-22T19:48:38.534 INFO:teuthology.orchestra.run.smithi136.stdout:(152/282): low-memory-monitor-2.1-4.el9.x86_64. 861 kB/s | 36 kB 00:00 2024-04-22T19:48:38.559 INFO:teuthology.orchestra.run.smithi136.stdout:(153/282): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-22T19:48:38.585 INFO:teuthology.orchestra.run.smithi136.stdout:(154/282): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-22T19:48:38.610 INFO:teuthology.orchestra.run.smithi136.stdout:(155/282): lua-posix-35.0-8.el9.x86_64.rpm 2.9 MB/s | 151 kB 00:00 2024-04-22T19:48:38.614 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x86_64 1/1 2024-04-22T19:48:38.635 INFO:teuthology.orchestra.run.smithi136.stdout:(156/282): maven-3.6.3-15.el9.noarch.rpm 352 kB/s | 18 kB 00:00 2024-04-22T19:48:38.686 INFO:teuthology.orchestra.run.smithi136.stdout:(157/282): maven-openjdk11-3.6.3-15.el9.noarch. 179 kB/s | 9.0 kB 00:00 2024-04-22T19:48:38.728 INFO:teuthology.orchestra.run.smithi136.stdout:(158/282): maven-lib-3.6.3-15.el9.noarch.rpm 13 MB/s | 1.5 MB 00:00 2024-04-22T19:48:38.761 INFO:teuthology.orchestra.run.smithi136.stdout:(159/282): maven-resolver-1.6.1-10.el9.noarch.r 6.7 MB/s | 517 kB 00:00 2024-04-22T19:48:38.762 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-04-22T19:48:38.787 INFO:teuthology.orchestra.run.smithi136.stdout:(160/282): maven-shared-utils-3.3.4-3.el9.noarc 2.5 MB/s | 151 kB 00:00 2024-04-22T19:48:38.806 INFO:teuthology.orchestra.run.smithi136.stdout:(161/282): maven-wagon-3.4.2-7.el9.noarch.rpm 2.6 MB/s | 112 kB 00:00 2024-04-22T19:48:38.829 INFO:teuthology.orchestra.run.smithi136.stdout:(162/282): mesa-libEGL-23.3.3-1.el9.x86_64.rpm 3.0 MB/s | 127 kB 00:00 2024-04-22T19:48:38.839 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : javapackages-filesystem-6.0.0-4.el9.noarch 1/284 2024-04-22T19:48:38.854 INFO:teuthology.orchestra.run.smithi136.stdout:(163/282): mesa-libGL-23.3.3-1.el9.x86_64.rpm 3.3 MB/s | 168 kB 00:00 2024-04-22T19:48:38.866 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libogg-2:1.3.4-6.el9.x86_64 2/284 2024-04-22T19:48:38.883 INFO:teuthology.orchestra.run.smithi136.stdout:(164/282): mesa-libgbm-23.3.3-1.el9.x86_64.rpm 750 kB/s | 38 kB 00:00 2024-04-22T19:48:38.905 INFO:teuthology.orchestra.run.smithi136.stdout:(165/282): mesa-libglapi-23.3.3-1.el9.x86_64.rp 882 kB/s | 44 kB 00:00 2024-04-22T19:48:38.931 INFO:teuthology.orchestra.run.smithi136.stdout:(166/282): mkfontscale-1.2.1-3.el9.x86_64.rpm 643 kB/s | 32 kB 00:00 2024-04-22T19:48:38.933 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXi-1.7.10-8.el9.x86_64 3/284 2024-04-22T19:48:38.955 INFO:teuthology.orchestra.run.smithi136.stdout:(167/282): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-22T19:48:38.982 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : alsa-lib-1.2.10-2.el9.x86_64 4/284 2024-04-22T19:48:39.007 INFO:teuthology.orchestra.run.smithi136.stdout:(168/282): openjpeg2-2.4.0-7.el9.x86_64.rpm 3.2 MB/s | 163 kB 00:00 2024-04-22T19:48:39.034 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libvorbis-1:1.3.7-5.el9.x86_64 5/284 2024-04-22T19:48:39.056 INFO:teuthology.orchestra.run.smithi136.stdout:(169/282): opus-1.3.1-10.el9.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2024-04-22T19:48:39.069 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/284 2024-04-22T19:48:39.094 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/284 2024-04-22T19:48:39.131 INFO:teuthology.orchestra.run.smithi136.stdout:(170/282): openblas-openmp-0.3.21-2.el9.x86_64. 24 MB/s | 4.7 MB 00:00 2024-04-22T19:48:39.138 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/284 2024-04-22T19:48:39.156 INFO:teuthology.orchestra.run.smithi136.stdout:(171/282): orc-0.4.31-6.el9.x86_64.rpm 1.8 MB/s | 184 kB 00:00 2024-04-22T19:48:39.190 INFO:teuthology.orchestra.run.smithi136.stdout:(172/282): osinfo-db-20231215-1.el9.noarch.rpm 8.7 MB/s | 521 kB 00:00 2024-04-22T19:48:39.216 INFO:teuthology.orchestra.run.smithi136.stdout:(173/282): osinfo-db-tools-1.10.0-1.el9.x86_64. 1.2 MB/s | 74 kB 00:00 2024-04-22T19:48:39.241 INFO:teuthology.orchestra.run.smithi136.stdout:(174/282): ostree-libs-2024.5-2.el9.x86_64.rpm 9.5 MB/s | 488 kB 00:00 2024-04-22T19:48:39.266 INFO:teuthology.orchestra.run.smithi136.stdout:(175/282): p11-kit-server-0.25.3-2.el9.x86_64.r 4.8 MB/s | 246 kB 00:00 2024-04-22T19:48:39.291 INFO:teuthology.orchestra.run.smithi136.stdout:(176/282): pipewire-1.0.1-1.el9.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2024-04-22T19:48:39.317 INFO:teuthology.orchestra.run.smithi136.stdout:(177/282): pipewire-alsa-1.0.1-1.el9.x86_64.rpm 1.1 MB/s | 57 kB 00:00 2024-04-22T19:48:39.342 INFO:teuthology.orchestra.run.smithi136.stdout:(178/282): pipewire-jack-audio-connection-kit-1 182 kB/s | 9.1 kB 00:00 2024-04-22T19:48:39.369 INFO:teuthology.orchestra.run.smithi136.stdout:(179/282): pipewire-jack-audio-connection-kit-l 2.6 MB/s | 135 kB 00:00 2024-04-22T19:48:39.409 INFO:teuthology.orchestra.run.smithi136.stdout:(180/282): pipewire-pulseaudio-1.0.1-1.el9.x86_ 4.5 MB/s | 192 kB 00:00 2024-04-22T19:48:39.454 INFO:teuthology.orchestra.run.smithi136.stdout:(181/282): pipewire-libs-1.0.1-1.el9.x86_64.rpm 18 MB/s | 1.9 MB 00:00 2024-04-22T19:48:39.461 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/284 2024-04-22T19:48:39.477 INFO:teuthology.orchestra.run.smithi136.stdout:(182/282): plexus-cipher-1.7-27.el9.noarch.rpm 386 kB/s | 26 kB 00:00 2024-04-22T19:48:39.502 INFO:teuthology.orchestra.run.smithi136.stdout:(183/282): plexus-classworlds-2.6.0-11.el9.noar 1.2 MB/s | 60 kB 00:00 2024-04-22T19:48:39.536 INFO:teuthology.orchestra.run.smithi136.stdout:(184/282): plexus-containers-component-annotati 312 kB/s | 18 kB 00:00 2024-04-22T19:48:39.561 INFO:teuthology.orchestra.run.smithi136.stdout:(185/282): plexus-interpolation-1.26-11.el9.noa 1.3 MB/s | 80 kB 00:00 2024-04-22T19:48:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.0.0-3152.g9ed8263e.el7.x86_64 10/284 2024-04-22T19:48:39.586 INFO:teuthology.orchestra.run.smithi136.stdout:(186/282): plexus-sec-dispatcher-1.4-37.el9.noa 670 kB/s | 34 kB 00:00 2024-04-22T19:48:39.612 INFO:teuthology.orchestra.run.smithi136.stdout:(187/282): plexus-utils-3.3.0-10.el9.noarch.rpm 5.0 MB/s | 254 kB 00:00 2024-04-22T19:48:39.645 INFO:teuthology.orchestra.run.smithi136.stdout:(188/282): poppler-21.01.0-19.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-04-22T19:48:39.691 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.0.0-3152.g9ed8263e.el7.x86_64 10/284 2024-04-22T19:48:39.721 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.0.0-3152.g9ed8263e.el7.x86_64 11/284 2024-04-22T19:48:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.0.0-3152.g9ed8263e.el7.x86_64 11/284 2024-04-22T19:48:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.0.0-3152.g9ed8263e.el7.x86_64 12/284 2024-04-22T19:48:39.880 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libwayland-client-1.21.0-1.el9.x86_64 13/284 2024-04-22T19:48:39.908 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 14/284 2024-04-22T19:48:39.937 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 15/284 2024-04-22T19:48:39.960 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXfixes-5.0.3-16.el9.x86_64 16/284 2024-04-22T19:48:39.987 INFO:teuthology.orchestra.run.smithi136.stdout:(189/282): java-11-openjdk-headless-11.0.18.0.1 14 MB/s | 40 MB 00:02 2024-04-22T19:48:40.000 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : opus-1.3.1-10.el9.x86_64 17/284 2024-04-22T19:48:40.013 INFO:teuthology.orchestra.run.smithi136.stdout:(190/282): poppler-glib-21.01.0-19.el9.x86_64.r 415 kB/s | 152 kB 00:00 2024-04-22T19:48:40.026 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 18/284 2024-04-22T19:48:40.054 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libX11-xcb-1.7.0-9.el9.x86_64 19/284 2024-04-22T19:48:40.071 INFO:teuthology.orchestra.run.smithi136.stdout:(191/282): poppler-data-0.4.9-9.el9.noarch.rpm 4.1 MB/s | 1.9 MB 00:00 2024-04-22T19:48:40.102 INFO:teuthology.orchestra.run.smithi136.stdout:(192/282): publicsuffix-list-20210518-3.el9.noa 993 kB/s | 83 kB 00:00 2024-04-22T19:48:40.143 INFO:teuthology.orchestra.run.smithi136.stdout:(193/282): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-22T19:48:40.145 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lcms2-2.12-3.el9.x86_64 20/284 2024-04-22T19:48:40.164 INFO:teuthology.orchestra.run.smithi136.stdout:(194/282): pulseaudio-libs-15.0-2.el9.x86_64.rp 7.2 MB/s | 679 kB 00:00 2024-04-22T19:48:40.189 INFO:teuthology.orchestra.run.smithi136.stdout:(195/282): python3-devel-3.9.18-3.el9.x86_64.rp 4.8 MB/s | 244 kB 00:00 2024-04-22T19:48:40.214 INFO:teuthology.orchestra.run.smithi136.stdout:(196/282): python3-jinja2-2.11.3-5.el9.noarch.r 4.8 MB/s | 248 kB 00:00 2024-04-22T19:48:40.240 INFO:teuthology.orchestra.run.smithi136.stdout:(197/282): python3-jsonpatch-1.21-16.el9.noarch 512 kB/s | 26 kB 00:00 2024-04-22T19:48:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gstreamer1-1.22.1-2.el9.x86_64 21/284 2024-04-22T19:48:40.265 INFO:teuthology.orchestra.run.smithi136.stdout:(198/282): python3-jsonpointer-2.0-4.el9.noarch 382 kB/s | 19 kB 00:00 2024-04-22T19:48:40.290 INFO:teuthology.orchestra.run.smithi136.stdout:(199/282): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-22T19:48:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : atk-2.36.0-5.el9.x86_64 22/284 2024-04-22T19:48:40.315 INFO:teuthology.orchestra.run.smithi136.stdout:(200/282): python3-markupsafe-1.1.1-12.el9.x86_ 695 kB/s | 35 kB 00:00 2024-04-22T19:48:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libwayland-cursor-1.21.0-1.el9.x86_64 23/284 2024-04-22T19:48:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.0.0-3152.g9ed8263e.el7.x86_64 24/284 2024-04-22T19:48:40.365 INFO:teuthology.orchestra.run.smithi136.stdout:(201/282): python3-numpy-f2py-1.20.1-5.el9.x86_ 4.9 MB/s | 250 kB 00:00 2024-04-22T19:48:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3152.g9ed8263e.el7.x86_64 24/284 2024-04-22T19:48:40.458 INFO:teuthology.orchestra.run.smithi136.stdout:(202/282): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-22T19:48:40.476 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.0.0-3152.g9ed8263e.el7.x86 25/284 2024-04-22T19:48:40.483 INFO:teuthology.orchestra.run.smithi136.stdout:(203/282): python3-oauthlib-3.1.1-5.el9.noarch. 1.9 MB/s | 222 kB 00:00 2024-04-22T19:48:40.508 INFO:teuthology.orchestra.run.smithi136.stdout:(204/282): python3-packaging-20.9-5.el9.noarch. 1.5 MB/s | 77 kB 00:00 2024-04-22T19:48:40.523 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3152.g9ed8263e.el7.x86 25/284 2024-04-22T19:48:40.533 INFO:teuthology.orchestra.run.smithi136.stdout:(205/282): python3-prettytable-0.7.2-27.el9.noa 846 kB/s | 42 kB 00:00 2024-04-22T19:48:40.546 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXtst-1.2.3-16.el9.x86_64 26/284 2024-04-22T19:48:40.559 INFO:teuthology.orchestra.run.smithi136.stdout:(206/282): python3-protobuf-3.14.0-13.el9.noarc 5.2 MB/s | 269 kB 00:00 2024-04-22T19:48:40.569 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 27/284 2024-04-22T19:48:40.584 INFO:teuthology.orchestra.run.smithi136.stdout:(207/282): python3-pyasn1-0.4.8-6.el9.noarch.rp 3.1 MB/s | 159 kB 00:00 2024-04-22T19:48:40.591 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 27/284 2024-04-22T19:48:40.613 INFO:teuthology.orchestra.run.smithi136.stdout:(208/282): python3-pyasn1-modules-0.4.8-6.el9.n 5.4 MB/s | 279 kB 00:00 2024-04-22T19:48:40.634 INFO:teuthology.orchestra.run.smithi136.stdout:(209/282): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-22T19:48:40.660 INFO:teuthology.orchestra.run.smithi136.stdout:(210/282): python3-requests-oauthlib-1.3.0-12.e 1.0 MB/s | 54 kB 00:00 2024-04-22T19:48:40.727 INFO:teuthology.orchestra.run.smithi136.stdout:(211/282): python3-numpy-1.20.1-5.el9.x86_64.rp 12 MB/s | 5.1 MB 00:00 2024-04-22T19:48:40.746 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 28/284 2024-04-22T19:48:40.761 INFO:teuthology.orchestra.run.smithi136.stdout:(212/282): python3-toml-0.10.2-6.el9.noarch.rpm 414 kB/s | 42 kB 00:00 2024-04-22T19:48:40.778 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 28/284 2024-04-22T19:48:40.786 INFO:teuthology.orchestra.run.smithi136.stdout:(213/282): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-22T19:48:40.811 INFO:teuthology.orchestra.run.smithi136.stdout:(214/282): qatlib-service-24.02.0-1.el9.x86_64. 717 kB/s | 36 kB 00:00 2024-04-22T19:48:40.836 INFO:teuthology.orchestra.run.smithi136.stdout:(215/282): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-22T19:48:40.863 INFO:teuthology.orchestra.run.smithi136.stdout:(216/282): rtkit-0.11-28.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-22T19:48:40.886 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-3.el9.x86_64 29/284 2024-04-22T19:48:40.895 INFO:teuthology.orchestra.run.smithi136.stdout:(217/282): sisu-0.3.4-10.el9.noarch.rpm 8.5 MB/s | 510 kB 00:00 2024-04-22T19:48:40.921 INFO:teuthology.orchestra.run.smithi136.stdout:(218/282): slf4j-1.7.30-13.el9.noarch.rpm 1.2 MB/s | 69 kB 00:00 2024-04-22T19:48:40.946 INFO:teuthology.orchestra.run.smithi136.stdout:(219/282): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-22T19:48:40.971 INFO:teuthology.orchestra.run.smithi136.stdout:(220/282): sound-theme-freedesktop-0.8-17.el9.n 7.4 MB/s | 383 kB 00:00 2024-04-22T19:48:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 30/284 2024-04-22T19:48:40.996 INFO:teuthology.orchestra.run.smithi136.stdout:(221/282): totem-pl-parser-3.26.6-2.el9.x86_64. 3.0 MB/s | 152 kB 00:00 2024-04-22T19:48:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 31/284 2024-04-22T19:48:41.041 INFO:teuthology.orchestra.run.smithi136.stdout:(222/282): tracker-3.1.2-3.el9.x86_64.rpm 8.1 MB/s | 555 kB 00:00 2024-04-22T19:48:41.053 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 32/284 2024-04-22T19:48:41.080 INFO:teuthology.orchestra.run.smithi136.stdout:(223/282): tracker-miners-3.1.2-4.el9.x86_64.rp 11 MB/s | 944 kB 00:00 2024-04-22T19:48:41.106 INFO:teuthology.orchestra.run.smithi136.stdout:(224/282): ttmkfdir-3.0.9-65.el9.x86_64.rpm 793 kB/s | 53 kB 00:00 2024-04-22T19:48:41.128 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/284 2024-04-22T19:48:41.131 INFO:teuthology.orchestra.run.smithi136.stdout:(225/282): tzdata-java-2024a-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-22T19:48:41.156 INFO:teuthology.orchestra.run.smithi136.stdout:(226/282): upower-0.99.11-11.el9.x86_64.rpm 3.4 MB/s | 173 kB 00:00 2024-04-22T19:48:41.169 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/284 2024-04-22T19:48:41.182 INFO:teuthology.orchestra.run.smithi136.stdout:(227/282): webrtc-audio-processing-0.3.1-8.el9. 6.0 MB/s | 306 kB 00:00 2024-04-22T19:48:41.199 INFO:teuthology.orchestra.run.smithi136.stdout:(228/282): wireplumber-0.4.14-1.el9.x86_64.rpm 2.3 MB/s | 96 kB 00:00 2024-04-22T19:48:41.213 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/284 2024-04-22T19:48:41.232 INFO:teuthology.orchestra.run.smithi136.stdout:(229/282): wireplumber-libs-0.4.14-1.el9.x86_64 6.9 MB/s | 358 kB 00:00 2024-04-22T19:48:41.238 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/284 2024-04-22T19:48:41.249 INFO:teuthology.orchestra.run.smithi136.stdout:(230/282): xdg-dbus-proxy-0.1.3-1.el9.x86_64.rp 846 kB/s | 42 kB 00:00 2024-04-22T19:48:41.262 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : mesa-libglapi-23.3.3-1.el9.x86_64 37/284 2024-04-22T19:48:41.289 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libxshmfence-1.3-10.el9.x86_64 38/284 2024-04-22T19:48:41.291 INFO:teuthology.orchestra.run.smithi136.stdout:(231/282): xdg-desktop-portal-1.12.6-1.el9.x86_ 6.4 MB/s | 385 kB 00:00 2024-04-22T19:48:41.317 INFO:teuthology.orchestra.run.smithi136.stdout:(232/282): xdg-desktop-portal-gtk-1.12.0-3.el9. 2.1 MB/s | 134 kB 00:00 2024-04-22T19:48:41.330 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libwayland-server-1.21.0-1.el9.x86_64 39/284 2024-04-22T19:48:41.356 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libwayland-egl-1.21.0-1.el9.x86_64 40/284 2024-04-22T19:48:41.358 INFO:teuthology.orchestra.run.smithi136.stdout:(233/282): xkeyboard-config-2.33-2.el9.noarch.r 13 MB/s | 859 kB 00:00 2024-04-22T19:48:41.384 INFO:teuthology.orchestra.run.smithi136.stdout:(234/282): xmlstarlet-1.6.1-20.el9.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-04-22T19:48:41.384 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-ltdl-2.4.6-45.el9.x86_64 41/284 2024-04-22T19:48:41.411 INFO:teuthology.orchestra.run.smithi136.stdout:(235/282): xorg-x11-fonts-Type1-7.5-33.el9.noar 9.8 MB/s | 505 kB 00:00 2024-04-22T19:48:41.417 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libglvnd-1:1.3.4-1.el9.x86_64 42/284 2024-04-22T19:48:41.442 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXcomposite-0.4.5-7.el9.x86_64 43/284 2024-04-22T19:48:41.473 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 44/284 2024-04-22T19:48:41.506 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : dconf-0.40.0-6.el9.x86_64 45/284 2024-04-22T19:48:41.577 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: dconf-0.40.0-6.el9.x86_64 45/284 2024-04-22T19:48:41.618 INFO:teuthology.orchestra.run.smithi136.stdout:(236/282): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-04-22T19:48:41.633 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 46/284 2024-04-22T19:48:41.662 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 47/284 2024-04-22T19:48:41.707 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 48/284 2024-04-22T19:48:41.742 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 49/284 2024-04-22T19:48:41.775 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-15.el9.x86_64 50/284 2024-04-22T19:48:41.855 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cups-libs-1:2.3.3op2-24.el9.x86_64 51/284 2024-04-22T19:48:41.885 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.0.0-3152.g9ed8263e.el7. 52/284 2024-04-22T19:48:41.928 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.0.0-3152.g9ed8263e.el 53/284 2024-04-22T19:48:41.970 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.0.0-3152.g9ed8263e.el7.x86_6 54/284 2024-04-22T19:48:42.047 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 55/284 2024-04-22T19:48:42.126 INFO:teuthology.orchestra.run.smithi136.stdout:(237/282): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-22T19:48:42.154 INFO:teuthology.orchestra.run.smithi136.stdout:(238/282): lua-devel-5.4.4-4.el9.x86_64.rpm 29 kB/s | 22 kB 00:00 2024-04-22T19:48:42.183 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 56/284 2024-04-22T19:48:42.227 INFO:teuthology.orchestra.run.smithi136.stdout:(239/282): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-04-22T19:48:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 57/284 2024-04-22T19:48:42.427 INFO:teuthology.orchestra.run.smithi136.stdout:(240/282): grpc-data-1.46.7-10.el9.noarch.rpm 70 kB/s | 19 kB 00:00 2024-04-22T19:48:42.537 INFO:teuthology.orchestra.run.smithi136.stdout:(241/282): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-22T19:48:42.603 INFO:teuthology.orchestra.run.smithi136.stdout:(242/282): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-22T19:48:42.653 INFO:teuthology.orchestra.run.smithi136.stdout:(243/282): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-04-22T19:48:42.679 INFO:teuthology.orchestra.run.smithi136.stdout:(244/282): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2024-04-22T19:48:42.771 INFO:teuthology.orchestra.run.smithi136.stdout:(245/282): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-22T19:48:42.781 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 58/284 2024-04-22T19:48:42.842 INFO:teuthology.orchestra.run.smithi136.stdout:(246/282): luarocks-3.9.2-1.el9.noarch.rpm 821 kB/s | 151 kB 00:00 2024-04-22T19:48:42.869 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 59/284 2024-04-22T19:48:42.871 INFO:teuthology.orchestra.run.smithi136.stdout:(247/282): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-22T19:48:42.909 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 60/284 2024-04-22T19:48:42.921 INFO:teuthology.orchestra.run.smithi136.stdout:(248/282): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-04-22T19:48:42.947 INFO:teuthology.orchestra.run.smithi136.stdout:(249/282): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-22T19:48:42.974 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 61/284 2024-04-22T19:48:43.005 INFO:teuthology.orchestra.run.smithi136.stdout:(250/282): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-22T19:48:43.010 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : at-spi2-core-2.40.3-1.el9.x86_64 62/284 2024-04-22T19:48:43.039 INFO:teuthology.orchestra.run.smithi136.stdout:(251/282): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-22T19:48:43.047 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : at-spi2-atk-2.38.0-4.el9.x86_64 63/284 2024-04-22T19:48:43.076 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : colord-libs-1.4.5-4.el9.x86_64 64/284 2024-04-22T19:48:43.123 INFO:teuthology.orchestra.run.smithi136.stdout:(252/282): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-22T19:48:43.146 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 65/284 2024-04-22T19:48:43.178 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-1.el9.noarch 66/284 2024-04-22T19:48:43.203 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXcursor-1.2.0-7.el9.x86_64 67/284 2024-04-22T19:48:43.214 INFO:teuthology.orchestra.run.smithi136.stdout:(253/282): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-04-22T19:48:43.255 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXdamage-1.1.5-7.el9.x86_64 68/284 2024-04-22T19:48:43.282 INFO:teuthology.orchestra.run.smithi136.stdout:(254/282): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-22T19:48:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.0.0-3152.g9ed8263e.el7.x86_6 69/284 2024-04-22T19:48:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 70/284 2024-04-22T19:48:43.332 INFO:teuthology.orchestra.run.smithi136.stdout:(255/282): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-22T19:48:43.357 INFO:teuthology.orchestra.run.smithi136.stdout:(256/282): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-22T19:48:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flac-libs-1.3.3-12.el9.x86_64 71/284 2024-04-22T19:48:43.399 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtheora-1:1.1.1-31.el9.x86_64 72/284 2024-04-22T19:48:43.407 INFO:teuthology.orchestra.run.smithi136.stdout:(257/282): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-22T19:48:43.457 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : exiv2-libs-0.27.5-2.el9.x86_64 73/284 2024-04-22T19:48:43.458 INFO:teuthology.orchestra.run.smithi136.stdout:(258/282): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-22T19:48:43.491 INFO:teuthology.orchestra.run.smithi136.stdout:(259/282): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-22T19:48:43.492 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : exiv2-0.27.5-2.el9.x86_64 74/284 2024-04-22T19:48:43.529 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgexiv2-0.12.3-1.el9.x86_64 75/284 2024-04-22T19:48:43.533 INFO:teuthology.orchestra.run.smithi136.stdout:(260/282): protobuf-compiler-3.14.0-13.el9.x86_ 406 kB/s | 863 kB 00:02 2024-04-22T19:48:43.558 INFO:teuthology.orchestra.run.smithi136.stdout:(261/282): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-04-22T19:48:43.585 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 76/284 2024-04-22T19:48:43.617 INFO:teuthology.orchestra.run.smithi136.stdout:(262/282): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-22T19:48:43.658 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 77/284 2024-04-22T19:48:43.671 INFO:teuthology.orchestra.run.smithi136.stdout:(263/282): python3-more-itertools-8.12.0-2.el9. 726 kB/s | 79 kB 00:00 2024-04-22T19:48:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 78/284 2024-04-22T19:48:43.726 INFO:teuthology.orchestra.run.smithi136.stdout:(264/282): python3-natsort-7.1.1-5.el9.noarch.r 706 kB/s | 58 kB 00:00 2024-04-22T19:48:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 79/284 2024-04-22T19:48:43.779 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 80/284 2024-04-22T19:48:43.793 INFO:teuthology.orchestra.run.smithi136.stdout:(265/282): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-22T19:48:43.810 INFO:teuthology.orchestra.run.smithi136.stdout:(266/282): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-22T19:48:43.825 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 81/284 2024-04-22T19:48:43.867 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 82/284 2024-04-22T19:48:43.894 INFO:teuthology.orchestra.run.smithi136.stdout:(267/282): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-04-22T19:48:43.919 INFO:teuthology.orchestra.run.smithi136.stdout:(268/282): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-04-22T19:48:43.944 INFO:teuthology.orchestra.run.smithi136.stdout:(269/282): python3-logutils-0.3.5-21.el9.noarch 113 kB/s | 46 kB 00:00 2024-04-22T19:48:43.995 INFO:teuthology.orchestra.run.smithi136.stdout:(270/282): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-22T19:48:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 83/284 2024-04-22T19:48:44.020 INFO:teuthology.orchestra.run.smithi136.stdout:(271/282): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-22T19:48:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 84/284 2024-04-22T19:48:44.045 INFO:teuthology.orchestra.run.smithi136.stdout:(272/282): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-04-22T19:48:44.085 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 85/284 2024-04-22T19:48:44.087 INFO:teuthology.orchestra.run.smithi136.stdout:(273/282): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-04-22T19:48:44.121 INFO:teuthology.orchestra.run.smithi136.stdout:(274/282): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-22T19:48:44.181 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 86/284 2024-04-22T19:48:44.213 INFO:teuthology.orchestra.run.smithi136.stdout:(275/282): python3-websocket-client-1.2.3-2.el9 537 kB/s | 90 kB 00:00 2024-04-22T19:48:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 87/284 2024-04-22T19:48:44.239 INFO:teuthology.orchestra.run.smithi136.stdout:(276/282): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-22T19:48:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 88/284 2024-04-22T19:48:44.264 INFO:teuthology.orchestra.run.smithi136.stdout:(277/282): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-04-22T19:48:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 89/284 2024-04-22T19:48:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 90/284 2024-04-22T19:48:44.347 INFO:teuthology.orchestra.run.smithi136.stdout:(278/282): s3cmd-2.4.0-1.el9.noarch.rpm 1.9 MB/s | 206 kB 00:00 2024-04-22T19:48:44.384 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 91/284 2024-04-22T19:48:44.389 INFO:teuthology.orchestra.run.smithi136.stdout:(279/282): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-22T19:48:44.512 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 92/284 2024-04-22T19:48:44.543 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 93/284 2024-04-22T19:48:44.623 INFO:teuthology.orchestra.run.smithi136.stdout:(280/282): thrift-0.15.0-2.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-04-22T19:48:44.737 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/284 2024-04-22T19:48:44.777 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xkeyboard-config-2.33-2.el9.noarch 95/284 2024-04-22T19:48:44.809 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libxkbcommon-1.0.3-4.el9.x86_64 96/284 2024-04-22T19:48:44.839 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xdg-dbus-proxy-0.1.3-1.el9.x86_64 97/284 2024-04-22T19:48:44.881 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : webrtc-audio-processing-0.3.1-8.el9.x86_64 98/284 2024-04-22T19:48:44.910 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : upower-0.99.11-11.el9.x86_64 99/284 2024-04-22T19:48:44.922 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: upower-0.99.11-11.el9.x86_64 99/284 2024-04-22T19:48:44.922 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/graphical.target.wants/upower.service → /usr/lib/systemd/system/upower.service. 2024-04-22T19:48:44.923 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:48:45.105 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : tzdata-java-2024a-2.el9.noarch 100/284 2024-04-22T19:48:45.174 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ttmkfdir-3.0.9-65.el9.x86_64 101/284 2024-04-22T19:48:45.224 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : totem-pl-parser-3.26.6-2.el9.x86_64 102/284 2024-04-22T19:48:45.249 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : sound-theme-freedesktop-0.8-17.el9.noarch 103/284 2024-04-22T19:48:45.273 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: sound-theme-freedesktop-0.8-17.el9.noarch 103/284 2024-04-22T19:48:45.301 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 104/284 2024-04-22T19:48:45.415 INFO:teuthology.orchestra.run.smithi136.stdout:(281/282): librbd1-19.0.0-3152.g9ed8263e.el7.x8 3.1 MB/s | 3.1 MB 00:01 2024-04-22T19:48:45.671 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rtkit-0.11-28.el9.x86_64 105/284 2024-04-22T19:48:45.704 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rtkit-0.11-28.el9.x86_64 105/284 2024-04-22T19:48:45.716 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rtkit-0.11-28.el9.x86_64 105/284 2024-04-22T19:48:45.716 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/graphical.target.wants/rtkit-daemon.service → /usr/lib/systemd/system/rtkit-daemon.service. 2024-04-22T19:48:45.716 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:48:45.760 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 106/284 2024-04-22T19:48:45.795 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 107/284 2024-04-22T19:48:45.828 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 108/284 2024-04-22T19:48:45.873 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 109/284 2024-04-22T19:48:45.904 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 110/284 2024-04-22T19:48:45.991 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 111/284 2024-04-22T19:48:46.051 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 112/284 2024-04-22T19:48:46.082 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 113/284 2024-04-22T19:48:46.129 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 114/284 2024-04-22T19:48:46.228 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 115/284 2024-04-22T19:48:46.292 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 116/284 2024-04-22T19:48:46.964 INFO:teuthology.orchestra.run.smithi136.stdout:(282/282): librados2-19.0.0-3152.g9ed8263e.el7. 1.3 MB/s | 3.4 MB 00:02 2024-04-22T19:48:46.971 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-22T19:48:46.971 INFO:teuthology.orchestra.run.smithi136.stdout:Total 16 MB/s | 305 MB 00:19 2024-04-22T19:48:47.030 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 117/284 2024-04-22T19:48:47.072 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 118/284 2024-04-22T19:48:47.105 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 119/284 2024-04-22T19:48:47.206 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 120/284 2024-04-22T19:48:47.238 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 121/284 2024-04-22T19:48:47.412 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : publicsuffix-list-20210518-3.el9.noarch 122/284 2024-04-22T19:48:47.445 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : poppler-data-0.4.9-9.el9.noarch 123/284 2024-04-22T19:48:47.477 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : p11-kit-server-0.25.3-2.el9.x86_64 124/284 2024-04-22T19:48:47.518 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ostree-libs-2024.5-2.el9.x86_64 125/284 2024-04-22T19:48:48.009 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : osinfo-db-tools-1.10.0-1.el9.x86_64 126/284 2024-04-22T19:48:48.090 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : osinfo-db-20231215-1.el9.noarch 127/284 2024-04-22T19:48:48.116 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libosinfo-1.10.0-1.el9.x86_64 128/284 2024-04-22T19:48:48.150 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : orc-0.4.31-6.el9.x86_64 129/284 2024-04-22T19:48:48.183 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openjpeg2-2.4.0-7.el9.x86_64 130/284 2024-04-22T19:48:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.21-2.el9.x86_64 131/284 2024-04-22T19:48:48.397 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 132/284 2024-04-22T19:48:48.475 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 133/284 2024-04-22T19:48:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 134/284 2024-04-22T19:48:49.255 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 135/284 2024-04-22T19:48:49.934 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-22T19:48:50.116 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-22T19:48:50.116 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-22T19:48:50.519 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 136/284 2024-04-22T19:48:50.627 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 137/284 2024-04-22T19:48:50.659 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-posix-35.0-8.el9.x86_64 138/284 2024-04-22T19:48:50.686 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : copy-jdk-configs-4.0-3.el9.noarch 139/284 2024-04-22T19:48:50.716 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : low-memory-monitor-2.1-4.el9.x86_64 140/284 2024-04-22T19:48:50.727 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: low-memory-monitor-2.1-4.el9.x86_64 140/284 2024-04-22T19:48:50.728 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/basic.target.wants/low-memory-monitor.service → /usr/lib/systemd/system/low-memory-monitor.service. 2024-04-22T19:48:50.728 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:48:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libwebp-1.2.0-8.el9.x86_64 141/284 2024-04-22T19:48:50.803 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libvisual-1:0.4.0-34.el9.x86_64 142/284 2024-04-22T19:48:50.836 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libsbc-1.4-9.el9.x86_64 143/284 2024-04-22T19:48:51.148 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 144/284 2024-04-22T19:48:51.178 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.0.0-3152.g9ed8263e.el7.x86_64 145/284 2024-04-22T19:48:51.260 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.0.0-3152.g9ed8263e.el7.x86_64 145/284 2024-04-22T19:48:51.296 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 146/284 2024-04-22T19:48:51.326 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libnotify-0.7.9-8.el9.x86_64 147/284 2024-04-22T19:48:51.354 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libldac-2.0.2.3-10.el9.x86_64 148/284 2024-04-22T19:48:51.408 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libiptcdata-1.0.5-9.el9.x86_64 149/284 2024-04-22T19:48:51.447 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgsf-1.14.47-5.el9.x86_64 150/284 2024-04-22T19:48:51.479 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libfontenc-1.1.3-17.el9.x86_64 151/284 2024-04-22T19:48:51.532 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : mkfontscale-1.2.1-3.el9.x86_64 152/284 2024-04-22T19:48:51.560 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xorg-x11-fonts-Type1-7.5-33.el9.noarch 153/284 2024-04-22T19:48:51.787 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: xorg-x11-fonts-Type1-7.5-33.el9.noarch 153/284 2024-04-22T19:48:51.829 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libexif-0.6.22-6.el9.x86_64 154/284 2024-04-22T19:48:51.858 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libepoxy-1.5.5-4.el9.x86_64 155/284 2024-04-22T19:48:51.882 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libasyncns-0.8-22.el9.x86_64 156/284 2024-04-22T19:48:51.907 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXxf86vm-1.1.4-18.el9.x86_64 157/284 2024-04-22T19:48:51.934 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXv-1.0.11-16.el9.x86_64 158/284 2024-04-22T19:48:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXrandr-1.5.2-8.el9.x86_64 159/284 2024-04-22T19:48:52.003 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libXinerama-1.1.4-10.el9.x86_64 160/284 2024-04-22T19:48:52.039 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jbigkit-libs-2.1-23.el9.x86_64 161/284 2024-04-22T19:48:52.075 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtiff-4.4.0-12.el9.x86_64 162/284 2024-04-22T19:48:52.106 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gdk-pixbuf2-modules-2.42.6-3.el9.x86_64 163/284 2024-04-22T19:48:52.178 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgxps-0.3.2-3.el9.x86_64 164/284 2024-04-22T19:48:52.214 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : poppler-21.01.0-19.el9.x86_64 165/284 2024-04-22T19:48:52.610 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-22T19:48:52.610 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-22T19:48:52.631 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : poppler-glib-21.01.0-19.el9.x86_64 166/284 2024-04-22T19:48:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : iso-codes-4.6.0-3.el9.noarch 167/284 2024-04-22T19:48:52.811 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : hicolor-icon-theme-0.17-13.el9.noarch 168/284 2024-04-22T19:48:52.837 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gtk-update-icon-cache-3.24.31-2.el9.x86_64 169/284 2024-04-22T19:48:52.865 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gsm-1.0.19-6.el9.x86_64 170/284 2024-04-22T19:48:52.932 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libsndfile-1.0.31-8.el9.x86_64 171/284 2024-04-22T19:48:52.971 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pulseaudio-libs-15.0-2.el9.x86_64 172/284 2024-04-22T19:48:53.005 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcanberra-0.30-27.el9.x86_64 173/284 2024-04-22T19:48:53.024 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcanberra-0.30-27.el9.x86_64 173/284 2024-04-22T19:48:53.051 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : graphene-1.10.6-2.el9.x86_64 174/284 2024-04-22T19:48:53.076 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : giflib-5.2.1-9.el9.x86_64 175/284 2024-04-22T19:48:53.108 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flatpak-session-helper-1.12.8-1.el9.x86_64 176/284 2024-04-22T19:48:53.132 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flatpak-selinux-1.12.8-1.el9.noarch 177/284 2024-04-22T19:49:02.635 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: copy-jdk-configs-4.0-3.el9.noarch 1/1 2024-04-22T19:49:02.778 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x86_64 1/1 2024-04-22T19:49:02.924 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-22T19:49:02.993 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : javapackages-filesystem-6.0.0-4.el9.noarch 1/284 2024-04-22T19:49:03.025 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libogg-2:1.3.4-6.el9.x86_64 2/284 2024-04-22T19:49:03.099 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXi-1.7.10-8.el9.x86_64 3/284 2024-04-22T19:49:03.140 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : alsa-lib-1.2.10-2.el9.x86_64 4/284 2024-04-22T19:49:03.197 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libvorbis-1:1.3.7-5.el9.x86_64 5/284 2024-04-22T19:49:03.228 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/284 2024-04-22T19:49:03.253 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/284 2024-04-22T19:49:03.295 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/284 2024-04-22T19:49:03.623 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/284 2024-04-22T19:49:03.729 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.0.0-3152.g9ed8263e.el7.x86_64 10/284 2024-04-22T19:49:03.851 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.0.0-3152.g9ed8263e.el7.x86_64 10/284 2024-04-22T19:49:03.880 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.0.0-3152.g9ed8263e.el7.x86_64 11/284 2024-04-22T19:49:03.971 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.0.0-3152.g9ed8263e.el7.x86_64 11/284 2024-04-22T19:49:04.000 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.0.0-3152.g9ed8263e.el7.x86_64 12/284 2024-04-22T19:49:04.039 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libwayland-client-1.21.0-1.el9.x86_64 13/284 2024-04-22T19:49:04.068 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 14/284 2024-04-22T19:49:04.094 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 15/284 2024-04-22T19:49:04.123 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXfixes-5.0.3-16.el9.x86_64 16/284 2024-04-22T19:49:04.152 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : opus-1.3.1-10.el9.x86_64 17/284 2024-04-22T19:49:04.186 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 18/284 2024-04-22T19:49:04.215 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libX11-xcb-1.7.0-9.el9.x86_64 19/284 2024-04-22T19:49:04.304 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lcms2-2.12-3.el9.x86_64 20/284 2024-04-22T19:49:04.388 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gstreamer1-1.22.1-2.el9.x86_64 21/284 2024-04-22T19:49:04.398 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: flatpak-selinux-1.12.8-1.el9.noarch 177/284 2024-04-22T19:49:04.421 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : atk-2.36.0-5.el9.x86_64 22/284 2024-04-22T19:49:04.433 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fdk-aac-free-2.0.0-8.el9.x86_64 178/284 2024-04-22T19:49:04.457 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libwayland-cursor-1.21.0-1.el9.x86_64 23/284 2024-04-22T19:49:04.459 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : exempi-2.6.0-0.2.20211007gite23c213.el9.x86_64 179/284 2024-04-22T19:49:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.0.0-3152.g9ed8263e.el7.x86_64 24/284 2024-04-22T19:49:04.495 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 180/284 2024-04-22T19:49:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3152.g9ed8263e.el7.x86_64 24/284 2024-04-22T19:49:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.0.0-3152.g9ed8263e.el7.x86 25/284 2024-04-22T19:49:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3152.g9ed8263e.el7.x86 25/284 2024-04-22T19:49:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXtst-1.2.3-16.el9.x86_64 26/284 2024-04-22T19:49:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 27/284 2024-04-22T19:49:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 27/284 2024-04-22T19:49:04.914 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 28/284 2024-04-22T19:49:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 28/284 2024-04-22T19:49:04.989 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 181/284 2024-04-22T19:49:05.017 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.0.0-3152.g9ed8263e.el7.x86_64 182/284 2024-04-22T19:49:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-3.el9.x86_64 29/284 2024-04-22T19:49:05.105 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.0.0-3152.g9ed8263e.el7.x86_64 182/284 2024-04-22T19:49:05.135 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 183/284 2024-04-22T19:49:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 30/284 2024-04-22T19:49:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 31/284 2024-04-22T19:49:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 32/284 2024-04-22T19:49:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/284 2024-04-22T19:49:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/284 2024-04-22T19:49:05.375 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/284 2024-04-22T19:49:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/284 2024-04-22T19:49:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : mesa-libglapi-23.3.3-1.el9.x86_64 37/284 2024-04-22T19:49:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libxshmfence-1.3-10.el9.x86_64 38/284 2024-04-22T19:49:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libwayland-server-1.21.0-1.el9.x86_64 39/284 2024-04-22T19:49:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libwayland-egl-1.21.0-1.el9.x86_64 40/284 2024-04-22T19:49:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-ltdl-2.4.6-45.el9.x86_64 41/284 2024-04-22T19:49:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libglvnd-1:1.3.4-1.el9.x86_64 42/284 2024-04-22T19:49:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXcomposite-0.4.5-7.el9.x86_64 43/284 2024-04-22T19:49:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 44/284 2024-04-22T19:49:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : dconf-0.40.0-6.el9.x86_64 45/284 2024-04-22T19:49:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: dconf-0.40.0-6.el9.x86_64 45/284 2024-04-22T19:49:05.772 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 46/284 2024-04-22T19:49:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 47/284 2024-04-22T19:49:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 48/284 2024-04-22T19:49:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 49/284 2024-04-22T19:49:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-15.el9.x86_64 50/284 2024-04-22T19:49:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cups-libs-1:2.3.3op2-24.el9.x86_64 51/284 2024-04-22T19:49:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.0.0-3152.g9ed8263e.el7. 52/284 2024-04-22T19:49:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.0.0-3152.g9ed8263e.el 53/284 2024-04-22T19:49:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.0.0-3152.g9ed8263e.el7.x86_6 54/284 2024-04-22T19:49:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 55/284 2024-04-22T19:49:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 56/284 2024-04-22T19:49:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 57/284 2024-04-22T19:49:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 58/284 2024-04-22T19:49:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 59/284 2024-04-22T19:49:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 60/284 2024-04-22T19:49:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 61/284 2024-04-22T19:49:07.134 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : at-spi2-core-2.40.3-1.el9.x86_64 62/284 2024-04-22T19:49:07.170 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : at-spi2-atk-2.38.0-4.el9.x86_64 63/284 2024-04-22T19:49:07.208 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : colord-libs-1.4.5-4.el9.x86_64 64/284 2024-04-22T19:49:07.279 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 65/284 2024-04-22T19:49:07.310 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-1.el9.noarch 66/284 2024-04-22T19:49:07.335 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXcursor-1.2.0-7.el9.x86_64 67/284 2024-04-22T19:49:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXdamage-1.1.5-7.el9.x86_64 68/284 2024-04-22T19:49:07.417 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.0.0-3152.g9ed8263e.el7.x86_6 69/284 2024-04-22T19:49:07.449 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 70/284 2024-04-22T19:49:07.460 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 184/284 2024-04-22T19:49:07.481 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flac-libs-1.3.3-12.el9.x86_64 71/284 2024-04-22T19:49:07.509 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 184/284 2024-04-22T19:49:07.523 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtheora-1:1.1.1-31.el9.x86_64 72/284 2024-04-22T19:49:07.581 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : exiv2-libs-0.27.5-2.el9.x86_64 73/284 2024-04-22T19:49:07.618 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : exiv2-0.27.5-2.el9.x86_64 74/284 2024-04-22T19:49:07.654 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgexiv2-0.12.3-1.el9.x86_64 75/284 2024-04-22T19:49:07.710 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 76/284 2024-04-22T19:49:07.783 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 77/284 2024-04-22T19:49:07.823 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 78/284 2024-04-22T19:49:07.857 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 79/284 2024-04-22T19:49:07.906 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 80/284 2024-04-22T19:49:07.950 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 81/284 2024-04-22T19:49:07.993 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 82/284 2024-04-22T19:49:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 184/284 2024-04-22T19:49:08.095 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.0.0-3152.g9ed8263e.el7.x86_64 185/284 2024-04-22T19:49:08.141 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 83/284 2024-04-22T19:49:08.177 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 84/284 2024-04-22T19:49:08.189 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.0.0-3152.g9ed8263e.el7.x86_64 185/284 2024-04-22T19:49:08.189 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T19:49:08.189 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T19:49:08.189 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:08.219 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 85/284 2024-04-22T19:49:08.221 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.0.0-3152.g9ed8263e.el7.x86_64 186/284 2024-04-22T19:49:08.315 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 86/284 2024-04-22T19:49:08.352 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 87/284 2024-04-22T19:49:08.392 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 88/284 2024-04-22T19:49:08.418 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 89/284 2024-04-22T19:49:08.442 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 90/284 2024-04-22T19:49:08.553 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 91/284 2024-04-22T19:49:08.873 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 92/284 2024-04-22T19:49:08.911 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 93/284 2024-04-22T19:49:09.107 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/284 2024-04-22T19:49:09.146 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xkeyboard-config-2.33-2.el9.noarch 95/284 2024-04-22T19:49:09.178 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libxkbcommon-1.0.3-4.el9.x86_64 96/284 2024-04-22T19:49:09.208 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xdg-dbus-proxy-0.1.3-1.el9.x86_64 97/284 2024-04-22T19:49:09.250 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : webrtc-audio-processing-0.3.1-8.el9.x86_64 98/284 2024-04-22T19:49:09.286 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : upower-0.99.11-11.el9.x86_64 99/284 2024-04-22T19:49:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: upower-0.99.11-11.el9.x86_64 99/284 2024-04-22T19:49:09.300 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/graphical.target.wants/upower.service → /usr/lib/systemd/system/upower.service. 2024-04-22T19:49:09.300 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:09.482 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : tzdata-java-2024a-2.el9.noarch 100/284 2024-04-22T19:49:09.550 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ttmkfdir-3.0.9-65.el9.x86_64 101/284 2024-04-22T19:49:09.610 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : totem-pl-parser-3.26.6-2.el9.x86_64 102/284 2024-04-22T19:49:09.634 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : sound-theme-freedesktop-0.8-17.el9.noarch 103/284 2024-04-22T19:49:09.659 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: sound-theme-freedesktop-0.8-17.el9.noarch 103/284 2024-04-22T19:49:09.706 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 104/284 2024-04-22T19:49:10.107 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rtkit-0.11-28.el9.x86_64 105/284 2024-04-22T19:49:10.148 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rtkit-0.11-28.el9.x86_64 105/284 2024-04-22T19:49:10.161 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rtkit-0.11-28.el9.x86_64 105/284 2024-04-22T19:49:10.161 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/graphical.target.wants/rtkit-daemon.service → /usr/lib/systemd/system/rtkit-daemon.service. 2024-04-22T19:49:10.161 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:10.213 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 106/284 2024-04-22T19:49:10.247 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 107/284 2024-04-22T19:49:10.281 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 108/284 2024-04-22T19:49:10.323 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 109/284 2024-04-22T19:49:10.357 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 110/284 2024-04-22T19:49:10.446 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 111/284 2024-04-22T19:49:10.487 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 112/284 2024-04-22T19:49:10.527 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 113/284 2024-04-22T19:49:10.574 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 114/284 2024-04-22T19:49:10.675 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 115/284 2024-04-22T19:49:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 116/284 2024-04-22T19:49:11.481 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 117/284 2024-04-22T19:49:11.526 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 118/284 2024-04-22T19:49:11.560 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 119/284 2024-04-22T19:49:11.670 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 120/284 2024-04-22T19:49:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 121/284 2024-04-22T19:49:11.897 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : publicsuffix-list-20210518-3.el9.noarch 122/284 2024-04-22T19:49:11.932 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : poppler-data-0.4.9-9.el9.noarch 123/284 2024-04-22T19:49:11.965 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : p11-kit-server-0.25.3-2.el9.x86_64 124/284 2024-04-22T19:49:12.006 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ostree-libs-2024.5-2.el9.x86_64 125/284 2024-04-22T19:49:12.504 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : osinfo-db-tools-1.10.0-1.el9.x86_64 126/284 2024-04-22T19:49:12.567 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : osinfo-db-20231215-1.el9.noarch 127/284 2024-04-22T19:49:12.605 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libosinfo-1.10.0-1.el9.x86_64 128/284 2024-04-22T19:49:12.639 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : orc-0.4.31-6.el9.x86_64 129/284 2024-04-22T19:49:12.672 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openjpeg2-2.4.0-7.el9.x86_64 130/284 2024-04-22T19:49:12.852 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.21-2.el9.x86_64 131/284 2024-04-22T19:49:12.886 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 132/284 2024-04-22T19:49:12.965 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 133/284 2024-04-22T19:49:13.627 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 134/284 2024-04-22T19:49:13.746 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 135/284 2024-04-22T19:49:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 136/284 2024-04-22T19:49:15.114 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 137/284 2024-04-22T19:49:15.158 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-posix-35.0-8.el9.x86_64 138/284 2024-04-22T19:49:15.191 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : copy-jdk-configs-4.0-3.el9.noarch 139/284 2024-04-22T19:49:15.215 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : low-memory-monitor-2.1-4.el9.x86_64 140/284 2024-04-22T19:49:15.227 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: low-memory-monitor-2.1-4.el9.x86_64 140/284 2024-04-22T19:49:15.227 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/basic.target.wants/low-memory-monitor.service → /usr/lib/systemd/system/low-memory-monitor.service. 2024-04-22T19:49:15.227 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:15.273 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libwebp-1.2.0-8.el9.x86_64 141/284 2024-04-22T19:49:15.303 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libvisual-1:0.4.0-34.el9.x86_64 142/284 2024-04-22T19:49:15.331 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libsbc-1.4-9.el9.x86_64 143/284 2024-04-22T19:49:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 144/284 2024-04-22T19:49:15.686 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.0.0-3152.g9ed8263e.el7.x86_64 145/284 2024-04-22T19:49:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.0.0-3152.g9ed8263e.el7.x86_64 145/284 2024-04-22T19:49:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 146/284 2024-04-22T19:49:15.858 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libnotify-0.7.9-8.el9.x86_64 147/284 2024-04-22T19:49:15.889 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libldac-2.0.2.3-10.el9.x86_64 148/284 2024-04-22T19:49:15.943 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libiptcdata-1.0.5-9.el9.x86_64 149/284 2024-04-22T19:49:16.006 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgsf-1.14.47-5.el9.x86_64 150/284 2024-04-22T19:49:16.030 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libfontenc-1.1.3-17.el9.x86_64 151/284 2024-04-22T19:49:16.066 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : mkfontscale-1.2.1-3.el9.x86_64 152/284 2024-04-22T19:49:16.094 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xorg-x11-fonts-Type1-7.5-33.el9.noarch 153/284 2024-04-22T19:49:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: xorg-x11-fonts-Type1-7.5-33.el9.noarch 153/284 2024-04-22T19:49:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libexif-0.6.22-6.el9.x86_64 154/284 2024-04-22T19:49:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libepoxy-1.5.5-4.el9.x86_64 155/284 2024-04-22T19:49:16.400 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libasyncns-0.8-22.el9.x86_64 156/284 2024-04-22T19:49:16.426 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXxf86vm-1.1.4-18.el9.x86_64 157/284 2024-04-22T19:49:16.451 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXv-1.0.11-16.el9.x86_64 158/284 2024-04-22T19:49:16.476 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXrandr-1.5.2-8.el9.x86_64 159/284 2024-04-22T19:49:16.505 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libXinerama-1.1.4-10.el9.x86_64 160/284 2024-04-22T19:49:16.540 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jbigkit-libs-2.1-23.el9.x86_64 161/284 2024-04-22T19:49:16.585 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtiff-4.4.0-12.el9.x86_64 162/284 2024-04-22T19:49:16.624 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gdk-pixbuf2-modules-2.42.6-3.el9.x86_64 163/284 2024-04-22T19:49:16.673 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgxps-0.3.2-3.el9.x86_64 164/284 2024-04-22T19:49:16.707 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : poppler-21.01.0-19.el9.x86_64 165/284 2024-04-22T19:49:17.021 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3152.g9ed8263e.el7.x86_64 186/284 2024-04-22T19:49:17.021 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-04-22T19:49:17.021 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-04-22T19:49:17.021 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-04-22T19:49:17.021 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-04-22T19:49:17.021 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-04-22T19:49:17.021 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-04-22T19:49:17.022 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-04-22T19:49:17.022 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:17.130 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : poppler-glib-21.01.0-19.el9.x86_64 166/284 2024-04-22T19:49:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.0.0-3152.g9ed8263e.el7.noarch 187/284 2024-04-22T19:49:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.0.0-3152.g9ed8263e.el7.noarch 187/284 2024-04-22T19:49:17.202 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:17.202 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T19:49:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:17.284 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : iso-codes-4.6.0-3.el9.noarch 167/284 2024-04-22T19:49:17.332 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : hicolor-icon-theme-0.17-13.el9.noarch 168/284 2024-04-22T19:49:17.361 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gtk-update-icon-cache-3.24.31-2.el9.x86_64 169/284 2024-04-22T19:49:17.385 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gsm-1.0.19-6.el9.x86_64 170/284 2024-04-22T19:49:17.454 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libsndfile-1.0.31-8.el9.x86_64 171/284 2024-04-22T19:49:17.500 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pulseaudio-libs-15.0-2.el9.x86_64 172/284 2024-04-22T19:49:17.525 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcanberra-0.30-27.el9.x86_64 173/284 2024-04-22T19:49:17.545 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcanberra-0.30-27.el9.x86_64 173/284 2024-04-22T19:49:17.570 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : graphene-1.10.6-2.el9.x86_64 174/284 2024-04-22T19:49:17.596 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : giflib-5.2.1-9.el9.x86_64 175/284 2024-04-22T19:49:17.628 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flatpak-session-helper-1.12.8-1.el9.x86_64 176/284 2024-04-22T19:49:17.651 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flatpak-selinux-1.12.8-1.el9.noarch 177/284 2024-04-22T19:49:18.637 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.0.0-3152.g9ed8263e.el7.x86_64 188/284 2024-04-22T19:49:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.0.0-3152.g9ed8263e.el7.x86_64 188/284 2024-04-22T19:49:18.663 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:18.663 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T19:49:18.663 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T19:49:18.663 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T19:49:18.663 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:18.888 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.0.0-3152.g9ed8263e.el7.x86_64 189/284 2024-04-22T19:49:18.910 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.0.0-3152.g9ed8263e.el7.x86_64 189/284 2024-04-22T19:49:18.910 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:18.910 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T19:49:18.911 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T19:49:18.911 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T19:49:18.911 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:19.364 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.0.0-3152.g9ed8263e.el7.x86_64 190/284 2024-04-22T19:49:19.386 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.0.0-3152.g9ed8263e.el7.x86_64 190/284 2024-04-22T19:49:19.386 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:19.387 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T19:49:19.387 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T19:49:19.387 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T19:49:19.387 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:19.493 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : avahi-glib-0.8-20.el9.x86_64 191/284 2024-04-22T19:49:22.350 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : adwaita-cursor-theme-40.1.1-3.el9.noarch 192/284 2024-04-22T19:49:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : adwaita-icon-theme-40.1.1-3.el9.noarch 193/284 2024-04-22T19:49:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 194/284 2024-04-22T19:49:22.576 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 195/284 2024-04-22T19:49:22.775 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 196/284 2024-04-22T19:49:22.834 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 197/284 2024-04-22T19:49:22.920 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 198/284 2024-04-22T19:49:23.018 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 199/284 2024-04-22T19:49:23.158 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 200/284 2024-04-22T19:49:23.200 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 201/284 2024-04-22T19:49:23.233 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 202/284 2024-04-22T19:49:24.142 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 203/284 2024-04-22T19:49:24.182 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x8 204/284 2024-04-22T19:49:24.540 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x8 204/284 2024-04-22T19:49:24.589 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : plexus-utils-3.3.0-10.el9.noarch 205/284 2024-04-22T19:49:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : slf4j-1.7.30-13.el9.noarch 206/284 2024-04-22T19:49:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : atinject-1.0.3-6.el9.noarch 207/284 2024-04-22T19:49:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : apache-commons-io-1:2.8.0-8.el9.noarch 208/284 2024-04-22T19:49:24.719 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : apache-commons-lang3-3.12.0-6.el9.noarch 209/284 2024-04-22T19:49:24.746 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : httpcomponents-core-4.4.13-7.el9.noarch 210/284 2024-04-22T19:49:24.772 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : plexus-cipher-1.7-27.el9.noarch 211/284 2024-04-22T19:49:24.798 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : plexus-classworlds-2.6.0-11.el9.noarch 212/284 2024-04-22T19:49:24.823 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : plexus-containers-component-annotations-2.1.0-10 213/284 2024-04-22T19:49:24.849 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : plexus-sec-dispatcher-1.4-37.el9.noarch 214/284 2024-04-22T19:49:24.875 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : maven-shared-utils-3.3.4-3.el9.noarch 215/284 2024-04-22T19:49:24.901 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cdi-api-2.0.2-6.el9.noarch 216/284 2024-04-22T19:49:24.940 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : sisu-1:0.3.4-10.el9.noarch 217/284 2024-04-22T19:49:24.968 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jcl-over-slf4j-1.7.30-13.el9.noarch 218/284 2024-04-22T19:49:25.003 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : apache-commons-cli-1.4-17.el9.noarch 219/284 2024-04-22T19:49:25.039 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : apache-commons-codec-1.15-7.el9.noarch 220/284 2024-04-22T19:49:25.070 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : httpcomponents-client-4.5.13-3.el9.noarch 221/284 2024-04-22T19:49:25.104 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jakarta-annotations-1.3.5-13.el9.noarch 222/284 2024-04-22T19:49:25.138 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jansi-2.3.3-6.el9.x86_64 223/284 2024-04-22T19:49:25.175 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jsoup-1.13.1-10.el9.noarch 224/284 2024-04-22T19:49:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : maven-wagon-3.4.2-7.el9.noarch 225/284 2024-04-22T19:49:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : maven-resolver-1:1.6.1-10.el9.noarch 226/284 2024-04-22T19:49:25.317 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jsr-305-3.0.2-6.el9.noarch 227/284 2024-04-22T19:49:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : guava-30.1-6.el9.noarch 228/284 2024-04-22T19:49:25.372 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : google-guice-4.2.3-9.el9.noarch 229/284 2024-04-22T19:49:25.426 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : plexus-interpolation-1.26-11.el9.noarch 230/284 2024-04-22T19:49:25.466 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : maven-lib-1:3.6.3-15.el9.noarch 231/284 2024-04-22T19:49:25.496 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpciaccess-0.16-7.el9.x86_64 232/284 2024-04-22T19:49:25.524 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libdrm-2.4.117-1.el9.x86_64 233/284 2024-04-22T19:49:25.549 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : mesa-libgbm-23.3.3-1.el9.x86_64 234/284 2024-04-22T19:49:25.576 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libglvnd-egl-1:1.3.4-1.el9.x86_64 235/284 2024-04-22T19:49:25.603 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : mesa-libEGL-23.3.3-1.el9.x86_64 236/284 2024-04-22T19:49:25.645 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libglvnd-glx-1:1.3.4-1.el9.x86_64 237/284 2024-04-22T19:49:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : mesa-libGL-23.3.3-1.el9.x86_64 238/284 2024-04-22T19:49:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gstreamer1-plugins-base-1.22.1-2.el9.x86_64 239/284 2024-04-22T19:49:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtracker-sparql-3.1.2-3.el9.x86_64 240/284 2024-04-22T19:49:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : tracker-3.1.2-3.el9.x86_64 241/284 2024-04-22T19:49:26.064 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: tracker-3.1.2-3.el9.x86_64 241/284 2024-04-22T19:49:26.105 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : tracker-miners-3.1.2-4.el9.x86_64 242/284 2024-04-22T19:49:26.120 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: tracker-miners-3.1.2-4.el9.x86_64 242/284 2024-04-22T19:49:26.151 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bubblewrap-0.4.1-7.el9.x86_64 243/284 2024-04-22T19:49:26.173 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bluez-libs-5.56-6.el9.x86_64 244/284 2024-04-22T19:49:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: pipewire-1.0.1-1.el9.x86_64 245/284 2024-04-22T19:49:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pipewire-1.0.1-1.el9.x86_64 245/284 2024-04-22T19:49:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: pipewire-1.0.1-1.el9.x86_64 245/284 2024-04-22T19:49:26.659 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/user/sockets.target.wants/pipewire.socket → /usr/lib/systemd/user/pipewire.socket. 2024-04-22T19:49:26.659 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:26.896 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pipewire-libs-1.0.1-1.el9.x86_64 246/284 2024-04-22T19:49:26.966 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : wireplumber-0.4.14-1.el9.x86_64 247/284 2024-04-22T19:49:27.003 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : wireplumber-libs-0.4.14-1.el9.x86_64 248/284 2024-04-22T19:49:27.048 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pipewire-jack-audio-connection-kit-libs-1.0.1-1. 249/284 2024-04-22T19:49:27.082 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ModemManager-glib-1.20.2-1.el9.x86_64 250/284 2024-04-22T19:49:27.513 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: geoclue2-2.6.0-7.el9.x86_64 251/284 2024-04-22T19:49:27.544 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : geoclue2-2.6.0-7.el9.x86_64 251/284 2024-04-22T19:49:27.559 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: geoclue2-2.6.0-7.el9.x86_64 251/284 2024-04-22T19:49:28.104 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: flatpak-1.12.8-1.el9.x86_64 252/284 2024-04-22T19:49:28.190 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flatpak-1.12.8-1.el9.x86_64 252/284 2024-04-22T19:49:28.227 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xdg-desktop-portal-1.12.6-1.el9.x86_64 253/284 2024-04-22T19:49:28.254 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: xdg-desktop-portal-1.12.6-1.el9.x86_64 253/284 2024-04-22T19:49:28.431 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcanberra-gtk3-0.30-27.el9.x86_64 254/284 2024-04-22T19:49:28.496 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gtk3-3.24.31-2.el9.x86_64 255/284 2024-04-22T19:49:28.519 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xdg-desktop-portal-gtk-1.12.0-3.el9.x86_64 256/284 2024-04-22T19:49:28.541 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: xdg-desktop-portal-gtk-1.12.0-3.el9.x86_64 256/284 2024-04-22T19:49:28.570 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 257/284 2024-04-22T19:49:28.640 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 257/284 2024-04-22T19:49:28.673 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_6 258/284 2024-04-22T19:49:28.684 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_6 258/284 2024-04-22T19:49:28.715 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : maven-1:3.6.3-15.el9.noarch 259/284 2024-04-22T19:49:28.731 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: maven-1:3.6.3-15.el9.noarch 259/284 2024-04-22T19:49:28.766 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : maven-openjdk11-1:3.6.3-15.el9.noarch 260/284 2024-04-22T19:49:29.013 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: flatpak-selinux-1.12.8-1.el9.noarch 177/284 2024-04-22T19:49:29.057 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fdk-aac-free-2.0.0-8.el9.x86_64 178/284 2024-04-22T19:49:29.083 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : exempi-2.6.0-0.2.20211007gite23c213.el9.x86_64 179/284 2024-04-22T19:49:29.119 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 180/284 2024-04-22T19:49:29.159 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 261/284 2024-04-22T19:49:29.190 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 261/284 2024-04-22T19:49:29.222 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3152.g9ed8263e.e 262/284 2024-04-22T19:49:29.283 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3152.g9ed8263e. 263/284 2024-04-22T19:49:29.313 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3152.g9ed8263e.el7.noa 264/284 2024-04-22T19:49:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 181/284 2024-04-22T19:49:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.0.0-3152.g9ed8263e.el7.x86_64 182/284 2024-04-22T19:49:29.746 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.0.0-3152.g9ed8263e.el7.x86_64 182/284 2024-04-22T19:49:29.776 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 183/284 2024-04-22T19:49:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3152.g9ed8263e.el7.noa 264/284 2024-04-22T19:49:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3152.g9ed8263e.el7.n 265/284 2024-04-22T19:49:31.353 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3152.g9ed8263e.el7.n 265/284 2024-04-22T19:49:31.381 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3152.g9ed 266/284 2024-04-22T19:49:31.484 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3152.g9ed 266/284 2024-04-22T19:49:31.644 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3152.g9ed8263e.el 267/284 2024-04-22T19:49:31.675 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.0.0-3152.g9ed8263e.el7.x86_64 268/284 2024-04-22T19:49:31.698 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3152.g9ed8263e.el7.x86_64 268/284 2024-04-22T19:49:31.699 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:31.699 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T19:49:31.699 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T19:49:31.699 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T19:49:31.699 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:31.743 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.0.0-3152.g9ed8263e.el7.noarch 269/284 2024-04-22T19:49:31.753 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3152.g9ed8263e.el7.noarch 269/284 2024-04-22T19:49:31.778 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.0.0-3152.g9ed8263e.el7.x86_64 270/284 2024-04-22T19:49:31.806 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pipewire-jack-audio-connection-kit-1.0.1-1.el9.x 271/284 2024-04-22T19:49:31.855 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pipewire-alsa-1.0.1-1.el9.x86_64 272/284 2024-04-22T19:49:31.887 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pipewire-pulseaudio-1.0.1-1.el9.x86_64 273/284 2024-04-22T19:49:31.910 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: pipewire-pulseaudio-1.0.1-1.el9.x86_64 273/284 2024-04-22T19:49:31.910 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/user/sockets.target.wants/pipewire-pulse.socket → /usr/lib/systemd/user/pipewire-pulse.socket. 2024-04-22T19:49:31.911 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:32.054 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 184/284 2024-04-22T19:49:32.107 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 184/284 2024-04-22T19:49:32.649 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 184/284 2024-04-22T19:49:32.685 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.0.0-3152.g9ed8263e.el7.x86_64 185/284 2024-04-22T19:49:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.0.0-3152.g9ed8263e.el7.x86_64 185/284 2024-04-22T19:49:32.774 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-22T19:49:32.774 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-22T19:49:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.0.0-3152.g9ed8263e.el7.x86_64 186/284 2024-04-22T19:49:32.830 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 274/284 2024-04-22T19:49:32.852 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 274/284 2024-04-22T19:49:32.852 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:32.853 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T19:49:32.853 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T19:49:32.853 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T19:49:32.853 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3152.g9ed82 275/284 2024-04-22T19:49:32.919 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3152.g9ed82 275/284 2024-04-22T19:49:32.919 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:32.919 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T19:49:32.919 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:33.251 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.0.0-3152.g9ed8263e.el7.x86_64 276/284 2024-04-22T19:49:33.273 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3152.g9ed8263e.el7.x86_64 276/284 2024-04-22T19:49:33.273 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:33.273 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T19:49:33.274 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T19:49:33.274 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T19:49:33.274 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:37.826 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.0.0-3152.g9ed8263e.el7.x86_64 277/284 2024-04-22T19:49:37.867 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 278/284 2024-04-22T19:49:37.899 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 279/284 2024-04-22T19:49:38.002 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.0.0-3152.g9ed8263e.el7.x86_ 280/284 2024-04-22T19:49:38.068 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 281/284 2024-04-22T19:49:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : s3cmd-2.4.0-1.el9.noarch 282/284 2024-04-22T19:49:38.107 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 283/284 2024-04-22T19:49:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 283/284 2024-04-22T19:49:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 284/284 2024-04-22T19:49:38.299 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 284/284 2024-04-22T19:49:38.315 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: dconf-0.40.0-6.el9.x86_64 284/284 2024-04-22T19:49:38.322 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: copy-jdk-configs-4.0-3.el9.noarch 284/284 2024-04-22T19:49:38.331 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x8 284/284 2024-04-22T19:49:38.339 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: wireplumber-0.4.14-1.el9.x86_64 284/284 2024-04-22T19:49:38.339 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/user/pipewire-session-manager.service → /usr/lib/systemd/user/wireplumber.service. 2024-04-22T19:49:38.339 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/user/pipewire.service.wants/wireplumber.service → /usr/lib/systemd/user/wireplumber.service. 2024-04-22T19:49:38.340 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:38.386 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 284/284 2024-04-22T19:49:38.451 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_6 284/284 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3152.g9ed8263e.el7.x86_64 186/284 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-04-22T19:49:41.734 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:41.900 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.0.0-3152.g9ed8263e.el7.noarch 187/284 2024-04-22T19:49:41.915 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.0.0-3152.g9ed8263e.el7.noarch 187/284 2024-04-22T19:49:41.915 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:41.915 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-22T19:49:41.916 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:43.359 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.0.0-3152.g9ed8263e.el7.x86_64 188/284 2024-04-22T19:49:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.0.0-3152.g9ed8263e.el7.x86_64 188/284 2024-04-22T19:49:43.385 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:43.385 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-22T19:49:43.385 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T19:49:43.385 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-22T19:49:43.385 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:43.610 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.0.0-3152.g9ed8263e.el7.x86_64 189/284 2024-04-22T19:49:43.632 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.0.0-3152.g9ed8263e.el7.x86_64 189/284 2024-04-22T19:49:43.632 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:43.632 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-22T19:49:43.632 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T19:49:43.633 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-22T19:49:43.633 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:44.076 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.0.0-3152.g9ed8263e.el7.x86_64 190/284 2024-04-22T19:49:44.099 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.0.0-3152.g9ed8263e.el7.x86_64 190/284 2024-04-22T19:49:44.099 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:44.100 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-22T19:49:44.100 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T19:49:44.100 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-22T19:49:44.100 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:44.211 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : avahi-glib-0.8-20.el9.x86_64 191/284 2024-04-22T19:49:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 284/284 2024-04-22T19:49:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.0.0-3152.g9ed8263e.el7.x86_64 1/284 2024-04-22T19:49:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.0.0-3152.g9ed8263e.el7.x86_64 2/284 2024-04-22T19:49:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 3/284 2024-04-22T19:49:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 4/284 2024-04-22T19:49:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3152.g9ed82 5/284 2024-04-22T19:49:47.048 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.0.0-3152.g9ed8263e.el7.x86_64 6/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.0.0-3152.g9ed8263e.el7.x86_64 7/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.0.0-3152.g9ed8263e.el7.x86_64 8/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.0.0-3152.g9ed8263e.el7.x86_64 9/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 10/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.0.0-3152.g9ed8263e.el7.x86_64 11/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.0.0-3152.g9ed8263e.el7.x86_64 12/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.0.0-3152.g9ed8263e.el7.x86_ 13/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.0.0-3152.g9ed8263e.el7.x86_64 14/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.0.0-3152.g9ed8263e.el7.x86_64 15/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.0.0-3152.g9ed8263e.el7.x86_6 16/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.0.0-3152.g9ed8263e.el7.x86 17/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.0.0-3152.g9ed8263e.el7.x86_64 18/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3152.g9ed8263e.el 19/284 2024-04-22T19:49:47.049 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.0.0-3152.g9ed8263e.el7. 20/284 2024-04-22T19:49:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.0.0-3152.g9ed8263e.el7.x86_6 21/284 2024-04-22T19:49:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.0.0-3152.g9ed8263e.el7.x86_64 22/284 2024-04-22T19:49:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 23/284 2024-04-22T19:49:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 24/284 2024-04-22T19:49:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 25/284 2024-04-22T19:49:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.0.0-3152.g9ed8263e.el7.x86_64 26/284 2024-04-22T19:49:47.050 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 27/284 2024-04-22T19:49:47.065 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : adwaita-cursor-theme-40.1.1-3.el9.noarch 192/284 2024-04-22T19:49:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3152.g9ed8263e. 28/284 2024-04-22T19:49:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3152.g9ed8263e.el7.noa 29/284 2024-04-22T19:49:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3152.g9ed8263e.el7.n 30/284 2024-04-22T19:49:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3152.g9ed 31/284 2024-04-22T19:49:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3152.g9ed8263e.el 32/284 2024-04-22T19:49:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3152.g9ed8263e.el7.noarch 33/284 2024-04-22T19:49:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3152.g9ed8263e.e 34/284 2024-04-22T19:49:47.075 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.0.0-3152.g9ed8263e.el7.noarch 35/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 36/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ModemManager-glib-1.20.2-1.el9.x86_64 37/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bluez-libs-5.56-6.el9.x86_64 38/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bubblewrap-0.4.1-7.el9.x86_64 39/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cups-libs-1:2.3.3op2-24.el9.x86_64 40/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 42/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpciaccess-0.16-7.el9.x86_64 43/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 45/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/284 2024-04-22T19:49:47.076 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : adwaita-cursor-theme-40.1.1-3.el9.noarch 52/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : adwaita-icon-theme-40.1.1-3.el9.noarch 53/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : alsa-lib-1.2.10-2.el9.x86_64 54/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : apache-commons-cli-1.4-17.el9.noarch 55/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : apache-commons-codec-1.15-7.el9.noarch 56/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : apache-commons-io-1:2.8.0-8.el9.noarch 57/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : apache-commons-lang3-3.12.0-6.el9.noarch 58/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : at-spi2-atk-2.38.0-4.el9.x86_64 59/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : at-spi2-core-2.40.3-1.el9.x86_64 60/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : atinject-1.0.3-6.el9.noarch 61/284 2024-04-22T19:49:47.077 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : atk-2.36.0-5.el9.x86_64 62/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : avahi-glib-0.8-20.el9.x86_64 63/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 64/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cdi-api-2.0.2-6.el9.noarch 65/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : colord-libs-1.4.5-4.el9.x86_64 66/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : copy-jdk-configs-4.0-3.el9.noarch 67/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : dconf-0.40.0-6.el9.x86_64 68/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : exempi-2.6.0-0.2.20211007gite23c213.el9.x86_64 69/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : exiv2-0.27.5-2.el9.x86_64 70/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : exiv2-libs-0.27.5-2.el9.x86_64 71/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fdk-aac-free-2.0.0-8.el9.x86_64 72/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flac-libs-1.3.3-12.el9.x86_64 73/284 2024-04-22T19:49:47.078 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flatpak-1.12.8-1.el9.x86_64 74/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flatpak-selinux-1.12.8-1.el9.noarch 75/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flatpak-session-helper-1.12.8-1.el9.x86_64 76/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 77/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 78/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 79/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gdk-pixbuf2-modules-2.42.6-3.el9.x86_64 80/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : geoclue2-2.6.0-7.el9.x86_64 81/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : giflib-5.2.1-9.el9.x86_64 82/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : google-guice-4.2.3-9.el9.noarch 83/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : graphene-1.10.6-2.el9.x86_64 84/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gsm-1.0.19-6.el9.x86_64 85/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gstreamer1-1.22.1-2.el9.x86_64 86/284 2024-04-22T19:49:47.079 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gstreamer1-plugins-base-1.22.1-2.el9.x86_64 87/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gtk-update-icon-cache-3.24.31-2.el9.x86_64 88/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gtk3-3.24.31-2.el9.x86_64 89/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : guava-30.1-6.el9.noarch 90/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : hicolor-icon-theme-0.17-13.el9.noarch 91/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : httpcomponents-client-4.5.13-3.el9.noarch 92/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : httpcomponents-core-4.4.13-7.el9.noarch 93/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : iso-codes-4.6.0-3.el9.noarch 94/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jakarta-annotations-1.3.5-13.el9.noarch 95/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jansi-2.3.3-6.el9.x86_64 96/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 97/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_6 98/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x8 99/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : javapackages-filesystem-6.0.0-4.el9.noarch 100/284 2024-04-22T19:49:47.080 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jbigkit-libs-2.1-23.el9.x86_64 101/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jcl-over-slf4j-1.7.30-13.el9.noarch 102/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jsoup-1.13.1-10.el9.noarch 103/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jsr-305-3.0.2-6.el9.noarch 104/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lcms2-2.12-3.el9.x86_64 105/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libX11-xcb-1.7.0-9.el9.x86_64 106/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXcomposite-0.4.5-7.el9.x86_64 107/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXcursor-1.2.0-7.el9.x86_64 108/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXdamage-1.1.5-7.el9.x86_64 109/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXfixes-5.0.3-16.el9.x86_64 110/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXi-1.7.10-8.el9.x86_64 111/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXinerama-1.1.4-10.el9.x86_64 112/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXrandr-1.5.2-8.el9.x86_64 113/284 2024-04-22T19:49:47.081 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXtst-1.2.3-16.el9.x86_64 114/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXv-1.0.11-16.el9.x86_64 115/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libXxf86vm-1.1.4-18.el9.x86_64 116/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libasyncns-0.8-22.el9.x86_64 117/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcanberra-0.30-27.el9.x86_64 118/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcanberra-gtk3-0.30-27.el9.x86_64 119/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libdrm-2.4.117-1.el9.x86_64 120/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libepoxy-1.5.5-4.el9.x86_64 121/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libexif-0.6.22-6.el9.x86_64 122/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libfontenc-1.1.3-17.el9.x86_64 123/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgexiv2-0.12.3-1.el9.x86_64 124/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libglvnd-1:1.3.4-1.el9.x86_64 125/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libglvnd-egl-1:1.3.4-1.el9.x86_64 126/284 2024-04-22T19:49:47.082 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libglvnd-glx-1:1.3.4-1.el9.x86_64 127/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgsf-1.14.47-5.el9.x86_64 128/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgxps-0.3.2-3.el9.x86_64 129/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libiptcdata-1.0.5-9.el9.x86_64 130/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libldac-2.0.2.3-10.el9.x86_64 131/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libnotify-0.7.9-8.el9.x86_64 132/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libogg-2:1.3.4-6.el9.x86_64 133/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libosinfo-1.10.0-1.el9.x86_64 134/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 135/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 136/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 137/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libsbc-1.4-9.el9.x86_64 138/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libsndfile-1.0.31-8.el9.x86_64 139/284 2024-04-22T19:49:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtheora-1:1.1.1-31.el9.x86_64 140/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtiff-4.4.0-12.el9.x86_64 141/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-ltdl-2.4.6-45.el9.x86_64 142/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtracker-sparql-3.1.2-3.el9.x86_64 143/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libvisual-1:0.4.0-34.el9.x86_64 144/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libvorbis-1:1.3.7-5.el9.x86_64 145/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libwayland-client-1.21.0-1.el9.x86_64 146/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libwayland-cursor-1.21.0-1.el9.x86_64 147/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libwayland-egl-1.21.0-1.el9.x86_64 148/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libwayland-server-1.21.0-1.el9.x86_64 149/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libwebp-1.2.0-8.el9.x86_64 150/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libxkbcommon-1.0.3-4.el9.x86_64 151/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libxshmfence-1.3-10.el9.x86_64 152/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : low-memory-monitor-2.1-4.el9.x86_64 153/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 154/284 2024-04-22T19:49:47.084 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 155/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-posix-35.0-8.el9.x86_64 156/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : maven-1:3.6.3-15.el9.noarch 157/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : maven-lib-1:3.6.3-15.el9.noarch 158/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : maven-openjdk11-1:3.6.3-15.el9.noarch 159/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : maven-resolver-1:1.6.1-10.el9.noarch 160/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : maven-shared-utils-3.3.4-3.el9.noarch 161/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : maven-wagon-3.4.2-7.el9.noarch 162/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : mesa-libEGL-23.3.3-1.el9.x86_64 163/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : mesa-libGL-23.3.3-1.el9.x86_64 164/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : mesa-libgbm-23.3.3-1.el9.x86_64 165/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : mesa-libglapi-23.3.3-1.el9.x86_64 166/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : mkfontscale-1.2.1-3.el9.x86_64 167/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 168/284 2024-04-22T19:49:47.085 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 169/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openjpeg2-2.4.0-7.el9.x86_64 170/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : opus-1.3.1-10.el9.x86_64 171/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : orc-0.4.31-6.el9.x86_64 172/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : osinfo-db-20231215-1.el9.noarch 173/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : osinfo-db-tools-1.10.0-1.el9.x86_64 174/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ostree-libs-2024.5-2.el9.x86_64 175/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : p11-kit-server-0.25.3-2.el9.x86_64 176/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pipewire-1.0.1-1.el9.x86_64 177/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pipewire-alsa-1.0.1-1.el9.x86_64 178/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pipewire-jack-audio-connection-kit-1.0.1-1.el9.x 179/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pipewire-jack-audio-connection-kit-libs-1.0.1-1. 180/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pipewire-libs-1.0.1-1.el9.x86_64 181/284 2024-04-22T19:49:47.086 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pipewire-pulseaudio-1.0.1-1.el9.x86_64 182/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : plexus-cipher-1.7-27.el9.noarch 183/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : plexus-classworlds-2.6.0-11.el9.noarch 184/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : plexus-containers-component-annotations-2.1.0-10 185/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : plexus-interpolation-1.26-11.el9.noarch 186/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : plexus-sec-dispatcher-1.4-37.el9.noarch 187/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : plexus-utils-3.3.0-10.el9.noarch 188/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : poppler-21.01.0-19.el9.x86_64 189/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : poppler-data-0.4.9-9.el9.noarch 190/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : poppler-glib-21.01.0-19.el9.x86_64 191/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 192/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : publicsuffix-list-20210518-3.el9.noarch 193/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pulseaudio-libs-15.0-2.el9.x86_64 194/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 195/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 196/284 2024-04-22T19:49:47.087 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 197/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 198/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 199/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 200/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 201/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 202/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 203/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 204/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 205/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 206/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 207/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 208/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 209/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 210/284 2024-04-22T19:49:47.088 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 211/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 212/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 213/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 214/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 215/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 216/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rtkit-0.11-28.el9.x86_64 217/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : sisu-1:0.3.4-10.el9.noarch 218/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : slf4j-1.7.30-13.el9.noarch 219/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 220/284 2024-04-22T19:49:47.089 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : sound-theme-freedesktop-0.8-17.el9.noarch 221/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : totem-pl-parser-3.26.6-2.el9.x86_64 222/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : tracker-3.1.2-3.el9.x86_64 223/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : tracker-miners-3.1.2-4.el9.x86_64 224/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ttmkfdir-3.0.9-65.el9.x86_64 225/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : tzdata-java-2024a-2.el9.noarch 226/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : upower-0.99.11-11.el9.x86_64 227/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : webrtc-audio-processing-0.3.1-8.el9.x86_64 228/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : wireplumber-0.4.14-1.el9.x86_64 229/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : wireplumber-libs-0.4.14-1.el9.x86_64 230/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xdg-dbus-proxy-0.1.3-1.el9.x86_64 231/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xdg-desktop-portal-1.12.6-1.el9.x86_64 232/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xdg-desktop-portal-gtk-1.12.0-3.el9.x86_64 233/284 2024-04-22T19:49:47.090 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xkeyboard-config-2.33-2.el9.noarch 234/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 235/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xorg-x11-fonts-Type1-7.5-33.el9.noarch 236/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 237/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 238/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 239/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 240/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 241/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 242/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 243/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 244/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 245/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 246/284 2024-04-22T19:49:47.091 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 247/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 248/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 249/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 250/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 251/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 252/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 253/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 254/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 255/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 256/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 257/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 258/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 259/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 260/284 2024-04-22T19:49:47.092 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 261/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 262/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 263/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 264/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 265/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 266/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 267/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 268/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 269/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 270/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 271/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 272/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 273/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 274/284 2024-04-22T19:49:47.093 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 275/284 2024-04-22T19:49:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 276/284 2024-04-22T19:49:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 277/284 2024-04-22T19:49:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-3.el9.x86_64 278/284 2024-04-22T19:49:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : s3cmd-2.4.0-1.el9.noarch 279/284 2024-04-22T19:49:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 280/284 2024-04-22T19:49:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.0.0-3152.g9ed8263e.el7.x86_64 281/284 2024-04-22T19:49:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 282/284 2024-04-22T19:49:47.094 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.0.0-3152.g9ed8263e.el7.x86_64 283/284 2024-04-22T19:49:47.169 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : adwaita-icon-theme-40.1.1-3.el9.noarch 193/284 2024-04-22T19:49:47.224 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 194/284 2024-04-22T19:49:47.288 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 195/284 2024-04-22T19:49:47.476 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 196/284 2024-04-22T19:49:47.522 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 197/284 2024-04-22T19:49:47.599 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 198/284 2024-04-22T19:49:47.687 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 199/284 2024-04-22T19:49:47.837 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 200/284 2024-04-22T19:49:47.880 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 201/284 2024-04-22T19:49:47.921 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 202/284 2024-04-22T19:49:48.815 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 203/284 2024-04-22T19:49:48.852 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x8 204/284 2024-04-22T19:49:49.214 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x8 204/284 2024-04-22T19:49:49.259 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : plexus-utils-3.3.0-10.el9.noarch 205/284 2024-04-22T19:49:49.290 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : slf4j-1.7.30-13.el9.noarch 206/284 2024-04-22T19:49:49.317 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : atinject-1.0.3-6.el9.noarch 207/284 2024-04-22T19:49:49.352 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : apache-commons-io-1:2.8.0-8.el9.noarch 208/284 2024-04-22T19:49:49.389 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : apache-commons-lang3-3.12.0-6.el9.noarch 209/284 2024-04-22T19:49:49.417 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : httpcomponents-core-4.4.13-7.el9.noarch 210/284 2024-04-22T19:49:49.442 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : plexus-cipher-1.7-27.el9.noarch 211/284 2024-04-22T19:49:49.468 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : plexus-classworlds-2.6.0-11.el9.noarch 212/284 2024-04-22T19:49:49.493 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : plexus-containers-component-annotations-2.1.0-10 213/284 2024-04-22T19:49:49.536 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : plexus-sec-dispatcher-1.4-37.el9.noarch 214/284 2024-04-22T19:49:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : maven-shared-utils-3.3.4-3.el9.noarch 215/284 2024-04-22T19:49:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cdi-api-2.0.2-6.el9.noarch 216/284 2024-04-22T19:49:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : sisu-1:0.3.4-10.el9.noarch 217/284 2024-04-22T19:49:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jcl-over-slf4j-1.7.30-13.el9.noarch 218/284 2024-04-22T19:49:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : apache-commons-cli-1.4-17.el9.noarch 219/284 2024-04-22T19:49:49.730 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : apache-commons-codec-1.15-7.el9.noarch 220/284 2024-04-22T19:49:49.757 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : httpcomponents-client-4.5.13-3.el9.noarch 221/284 2024-04-22T19:49:49.791 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jakarta-annotations-1.3.5-13.el9.noarch 222/284 2024-04-22T19:49:49.825 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jansi-2.3.3-6.el9.x86_64 223/284 2024-04-22T19:49:49.862 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jsoup-1.13.1-10.el9.noarch 224/284 2024-04-22T19:49:49.900 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : maven-wagon-3.4.2-7.el9.noarch 225/284 2024-04-22T19:49:49.943 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : maven-resolver-1:1.6.1-10.el9.noarch 226/284 2024-04-22T19:49:49.987 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : jsr-305-3.0.2-6.el9.noarch 227/284 2024-04-22T19:49:50.019 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : guava-30.1-6.el9.noarch 228/284 2024-04-22T19:49:50.042 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : google-guice-4.2.3-9.el9.noarch 229/284 2024-04-22T19:49:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : plexus-interpolation-1.26-11.el9.noarch 230/284 2024-04-22T19:49:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : maven-lib-1:3.6.3-15.el9.noarch 231/284 2024-04-22T19:49:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpciaccess-0.16-7.el9.x86_64 232/284 2024-04-22T19:49:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libdrm-2.4.117-1.el9.x86_64 233/284 2024-04-22T19:49:50.217 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : mesa-libgbm-23.3.3-1.el9.x86_64 234/284 2024-04-22T19:49:50.243 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libglvnd-egl-1:1.3.4-1.el9.x86_64 235/284 2024-04-22T19:49:50.275 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : mesa-libEGL-23.3.3-1.el9.x86_64 236/284 2024-04-22T19:49:50.307 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libglvnd-glx-1:1.3.4-1.el9.x86_64 237/284 2024-04-22T19:49:50.415 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : mesa-libGL-23.3.3-1.el9.x86_64 238/284 2024-04-22T19:49:50.474 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gstreamer1-plugins-base-1.22.1-2.el9.x86_64 239/284 2024-04-22T19:49:50.543 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtracker-sparql-3.1.2-3.el9.x86_64 240/284 2024-04-22T19:49:50.591 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : tracker-3.1.2-3.el9.x86_64 241/284 2024-04-22T19:49:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: tracker-3.1.2-3.el9.x86_64 241/284 2024-04-22T19:49:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : tracker-miners-3.1.2-4.el9.x86_64 242/284 2024-04-22T19:49:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: tracker-miners-3.1.2-4.el9.x86_64 242/284 2024-04-22T19:49:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 284/284 2024-04-22T19:49:50.769 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: ModemManager-glib-1.20.2-1.el9.x86_64 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: adwaita-cursor-theme-40.1.1-3.el9.noarch 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: adwaita-icon-theme-40.1.1-3.el9.noarch 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: alsa-lib-1.2.10-2.el9.x86_64 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: apache-commons-cli-1.4-17.el9.noarch 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: apache-commons-codec-1.15-7.el9.noarch 2024-04-22T19:49:50.770 INFO:teuthology.orchestra.run.smithi164.stdout: apache-commons-io-1:2.8.0-8.el9.noarch 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: apache-commons-lang3-3.12.0-6.el9.noarch 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: at-spi2-atk-2.38.0-4.el9.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: at-spi2-core-2.40.3-1.el9.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: atinject-1.0.3-6.el9.noarch 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: atk-2.36.0-5.el9.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: avahi-glib-0.8-20.el9.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: bluez-libs-5.56-6.el9.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: bubblewrap-0.4.1-7.el9.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: cdi-api-2.0.2-6.el9.noarch 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.771 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.772 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: colord-libs-1.4.5-4.el9.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: copy-jdk-configs-4.0-3.el9.noarch 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: cups-libs-1:2.3.3op2-24.el9.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: dconf-0.40.0-6.el9.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: exempi-2.6.0-0.2.20211007gite23c213.el9.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: exiv2-0.27.5-2.el9.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: exiv2-libs-0.27.5-2.el9.x86_64 2024-04-22T19:49:50.773 INFO:teuthology.orchestra.run.smithi164.stdout: fdk-aac-free-2.0.0-8.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: flac-libs-1.3.3-12.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: flatpak-1.12.8-1.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: flatpak-selinux-1.12.8-1.el9.noarch 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: flatpak-session-helper-1.12.8-1.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: gdk-pixbuf2-modules-2.42.6-3.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: geoclue2-2.6.0-7.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: giflib-5.2.1-9.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: google-guice-4.2.3-9.el9.noarch 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T19:49:50.774 INFO:teuthology.orchestra.run.smithi164.stdout: graphene-1.10.6-2.el9.x86_64 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: gsm-1.0.19-6.el9.x86_64 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: gstreamer1-1.22.1-2.el9.x86_64 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: gstreamer1-plugins-base-1.22.1-2.el9.x86_64 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: gtk-update-icon-cache-3.24.31-2.el9.x86_64 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: gtk3-3.24.31-2.el9.x86_64 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: guava-30.1-6.el9.noarch 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: hicolor-icon-theme-0.17-13.el9.noarch 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: httpcomponents-client-4.5.13-3.el9.noarch 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: httpcomponents-core-4.4.13-7.el9.noarch 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: iso-codes-4.6.0-3.el9.noarch 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: jakarta-annotations-1.3.5-13.el9.noarch 2024-04-22T19:49:50.775 INFO:teuthology.orchestra.run.smithi164.stdout: jansi-2.3.3-6.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: javapackages-filesystem-6.0.0-4.el9.noarch 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: jbigkit-libs-2.1-23.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: jcl-over-slf4j-1.7.30-13.el9.noarch 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: jsoup-1.13.1-10.el9.noarch 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: jsr-305-3.0.2-6.el9.noarch 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: lcms2-2.12-3.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: libX11-xcb-1.7.0-9.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: libXcomposite-0.4.5-7.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: libXcursor-1.2.0-7.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: libXdamage-1.1.5-7.el9.x86_64 2024-04-22T19:49:50.776 INFO:teuthology.orchestra.run.smithi164.stdout: libXfixes-5.0.3-16.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libXi-1.7.10-8.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libXinerama-1.1.4-10.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libXrandr-1.5.2-8.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libXtst-1.2.3-16.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libXv-1.0.11-16.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libXxf86vm-1.1.4-18.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libasyncns-0.8-22.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libcanberra-0.30-27.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libcanberra-gtk3-0.30-27.el9.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.777 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libdrm-2.4.117-1.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libepoxy-1.5.5-4.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libexif-0.6.22-6.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libfontenc-1.1.3-17.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libgexiv2-0.12.3-1.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libglvnd-1:1.3.4-1.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libglvnd-egl-1:1.3.4-1.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libglvnd-glx-1:1.3.4-1.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libgsf-1.14.47-5.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libgxps-0.3.2-3.el9.x86_64 2024-04-22T19:49:50.778 INFO:teuthology.orchestra.run.smithi164.stdout: libiptcdata-1.0.5-9.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libldac-2.0.2.3-10.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libnotify-0.7.9-8.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libogg-2:1.3.4-6.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libosinfo-1.10.0-1.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libpciaccess-0.16-7.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.779 INFO:teuthology.orchestra.run.smithi164.stdout: libsbc-1.4-9.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libsndfile-1.0.31-8.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libtheora-1:1.1.1-31.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libtiff-4.4.0-12.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-ltdl-2.4.6-45.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libtracker-sparql-3.1.2-3.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libvisual-1:0.4.0-34.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libvorbis-1:1.3.7-5.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libwayland-client-1.21.0-1.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libwayland-cursor-1.21.0-1.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libwayland-egl-1.21.0-1.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libwayland-server-1.21.0-1.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libwebp-1.2.0-8.el9.x86_64 2024-04-22T19:49:50.780 INFO:teuthology.orchestra.run.smithi164.stdout: libxkbcommon-1.0.3-4.el9.x86_64 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: libxshmfence-1.3-10.el9.x86_64 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: low-memory-monitor-2.1-4.el9.x86_64 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: lua-posix-35.0-8.el9.x86_64 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: maven-1:3.6.3-15.el9.noarch 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: maven-lib-1:3.6.3-15.el9.noarch 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: maven-openjdk11-1:3.6.3-15.el9.noarch 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: maven-resolver-1:1.6.1-10.el9.noarch 2024-04-22T19:49:50.781 INFO:teuthology.orchestra.run.smithi164.stdout: maven-shared-utils-3.3.4-3.el9.noarch 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: maven-wagon-3.4.2-7.el9.noarch 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: mesa-libEGL-23.3.3-1.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: mesa-libGL-23.3.3-1.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: mesa-libgbm-23.3.3-1.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: mesa-libglapi-23.3.3-1.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: mkfontscale-1.2.1-3.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: openjpeg2-2.4.0-7.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: opus-1.3.1-10.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: orc-0.4.31-6.el9.x86_64 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: osinfo-db-20231215-1.el9.noarch 2024-04-22T19:49:50.782 INFO:teuthology.orchestra.run.smithi164.stdout: osinfo-db-tools-1.10.0-1.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: ostree-libs-2024.5-2.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: p11-kit-server-0.25.3-2.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-1.0.1-1.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-alsa-1.0.1-1.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-jack-audio-connection-kit-1.0.1-1.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-jack-audio-connection-kit-libs-1.0.1-1.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-libs-1.0.1-1.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: pipewire-pulseaudio-1.0.1-1.el9.x86_64 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-cipher-1.7-27.el9.noarch 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-classworlds-2.6.0-11.el9.noarch 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-containers-component-annotations-2.1.0-10.el9.noarch 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-interpolation-1.26-11.el9.noarch 2024-04-22T19:49:50.783 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-sec-dispatcher-1.4-37.el9.noarch 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: plexus-utils-3.3.0-10.el9.noarch 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: poppler-21.01.0-19.el9.x86_64 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: poppler-data-0.4.9-9.el9.noarch 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: poppler-glib-21.01.0-19.el9.x86_64 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: publicsuffix-list-20210518-3.el9.noarch 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: pulseaudio-libs-15.0-2.el9.x86_64 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T19:49:50.784 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T19:49:50.785 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T19:49:50.786 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.787 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T19:49:50.788 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:49:50.789 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: rtkit-0.11-28.el9.x86_64 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: s3cmd-2.4.0-1.el9.noarch 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: sisu-1:0.3.4-10.el9.noarch 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: slf4j-1.7.30-13.el9.noarch 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: sound-theme-freedesktop-0.8-17.el9.noarch 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: totem-pl-parser-3.26.6-2.el9.x86_64 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: tracker-3.1.2-3.el9.x86_64 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: tracker-miners-3.1.2-4.el9.x86_64 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: ttmkfdir-3.0.9-65.el9.x86_64 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: tzdata-java-2024a-2.el9.noarch 2024-04-22T19:49:50.790 INFO:teuthology.orchestra.run.smithi164.stdout: upower-0.99.11-11.el9.x86_64 2024-04-22T19:49:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: webrtc-audio-processing-0.3.1-8.el9.x86_64 2024-04-22T19:49:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: wireplumber-0.4.14-1.el9.x86_64 2024-04-22T19:49:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: wireplumber-libs-0.4.14-1.el9.x86_64 2024-04-22T19:49:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: xdg-dbus-proxy-0.1.3-1.el9.x86_64 2024-04-22T19:49:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: xdg-desktop-portal-1.12.6-1.el9.x86_64 2024-04-22T19:49:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: xdg-desktop-portal-gtk-1.12.0-3.el9.x86_64 2024-04-22T19:49:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: xkeyboard-config-2.33-2.el9.noarch 2024-04-22T19:49:50.791 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T19:49:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bubblewrap-0.4.1-7.el9.x86_64 243/284 2024-04-22T19:49:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: xorg-x11-fonts-Type1-7.5-33.el9.noarch 2024-04-22T19:49:50.792 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:49:50.792 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-04-22T19:49:50.811 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bluez-libs-5.56-6.el9.x86_64 244/284 2024-04-22T19:49:51.198 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: pipewire-1.0.1-1.el9.x86_64 245/284 2024-04-22T19:49:51.217 DEBUG:teuthology.parallel:result is None 2024-04-22T19:49:51.224 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pipewire-1.0.1-1.el9.x86_64 245/284 2024-04-22T19:49:51.239 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: pipewire-1.0.1-1.el9.x86_64 245/284 2024-04-22T19:49:51.239 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/user/sockets.target.wants/pipewire.socket → /usr/lib/systemd/user/pipewire.socket. 2024-04-22T19:49:51.239 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pipewire-libs-1.0.1-1.el9.x86_64 246/284 2024-04-22T19:49:51.537 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : wireplumber-0.4.14-1.el9.x86_64 247/284 2024-04-22T19:49:51.575 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : wireplumber-libs-0.4.14-1.el9.x86_64 248/284 2024-04-22T19:49:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pipewire-jack-audio-connection-kit-libs-1.0.1-1. 249/284 2024-04-22T19:49:51.637 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ModemManager-glib-1.20.2-1.el9.x86_64 250/284 2024-04-22T19:49:51.999 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: geoclue2-2.6.0-7.el9.x86_64 251/284 2024-04-22T19:49:52.033 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : geoclue2-2.6.0-7.el9.x86_64 251/284 2024-04-22T19:49:52.048 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: geoclue2-2.6.0-7.el9.x86_64 251/284 2024-04-22T19:49:52.565 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: flatpak-1.12.8-1.el9.x86_64 252/284 2024-04-22T19:49:52.653 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flatpak-1.12.8-1.el9.x86_64 252/284 2024-04-22T19:49:52.691 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xdg-desktop-portal-1.12.6-1.el9.x86_64 253/284 2024-04-22T19:49:52.717 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: xdg-desktop-portal-1.12.6-1.el9.x86_64 253/284 2024-04-22T19:49:52.894 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcanberra-gtk3-0.30-27.el9.x86_64 254/284 2024-04-22T19:49:52.957 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gtk3-3.24.31-2.el9.x86_64 255/284 2024-04-22T19:49:52.983 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xdg-desktop-portal-gtk-1.12.0-3.el9.x86_64 256/284 2024-04-22T19:49:53.005 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: xdg-desktop-portal-gtk-1.12.0-3.el9.x86_64 256/284 2024-04-22T19:49:53.034 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 257/284 2024-04-22T19:49:53.104 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 257/284 2024-04-22T19:49:53.137 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_6 258/284 2024-04-22T19:49:53.148 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_6 258/284 2024-04-22T19:49:53.179 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : maven-1:3.6.3-15.el9.noarch 259/284 2024-04-22T19:49:53.192 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: maven-1:3.6.3-15.el9.noarch 259/284 2024-04-22T19:49:53.213 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : maven-openjdk11-1:3.6.3-15.el9.noarch 260/284 2024-04-22T19:49:53.581 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 261/284 2024-04-22T19:49:53.611 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 261/284 2024-04-22T19:49:53.644 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3152.g9ed8263e.e 262/284 2024-04-22T19:49:53.706 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3152.g9ed8263e. 263/284 2024-04-22T19:49:53.743 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3152.g9ed8263e.el7.noa 264/284 2024-04-22T19:49:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3152.g9ed8263e.el7.noa 264/284 2024-04-22T19:49:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3152.g9ed8263e.el7.n 265/284 2024-04-22T19:49:55.795 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3152.g9ed8263e.el7.n 265/284 2024-04-22T19:49:55.829 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3152.g9ed 266/284 2024-04-22T19:49:55.932 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3152.g9ed 266/284 2024-04-22T19:49:56.092 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3152.g9ed8263e.el 267/284 2024-04-22T19:49:56.123 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.0.0-3152.g9ed8263e.el7.x86_64 268/284 2024-04-22T19:49:56.146 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3152.g9ed8263e.el7.x86_64 268/284 2024-04-22T19:49:56.146 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:56.146 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-22T19:49:56.146 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T19:49:56.146 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-22T19:49:56.146 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:56.190 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.0.0-3152.g9ed8263e.el7.noarch 269/284 2024-04-22T19:49:56.201 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3152.g9ed8263e.el7.noarch 269/284 2024-04-22T19:49:56.225 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.0.0-3152.g9ed8263e.el7.x86_64 270/284 2024-04-22T19:49:56.254 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pipewire-jack-audio-connection-kit-1.0.1-1.el9.x 271/284 2024-04-22T19:49:56.302 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pipewire-alsa-1.0.1-1.el9.x86_64 272/284 2024-04-22T19:49:56.326 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : pipewire-pulseaudio-1.0.1-1.el9.x86_64 273/284 2024-04-22T19:49:56.341 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: pipewire-pulseaudio-1.0.1-1.el9.x86_64 273/284 2024-04-22T19:49:56.341 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/user/sockets.target.wants/pipewire-pulse.socket → /usr/lib/systemd/user/pipewire-pulse.socket. 2024-04-22T19:49:56.341 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:57.269 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 274/284 2024-04-22T19:49:57.292 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 274/284 2024-04-22T19:49:57.292 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:57.292 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-22T19:49:57.292 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T19:49:57.292 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-22T19:49:57.292 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:57.336 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3152.g9ed82 275/284 2024-04-22T19:49:57.358 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3152.g9ed82 275/284 2024-04-22T19:49:57.358 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:57.358 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-22T19:49:57.358 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:49:57.661 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.0.0-3152.g9ed8263e.el7.x86_64 276/284 2024-04-22T19:49:57.684 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3152.g9ed8263e.el7.x86_64 276/284 2024-04-22T19:49:57.684 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-22T19:49:57.684 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-22T19:49:57.684 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T19:49:57.684 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-22T19:49:57.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:02.265 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.0.0-3152.g9ed8263e.el7.x86_64 277/284 2024-04-22T19:50:02.306 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 278/284 2024-04-22T19:50:02.338 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 279/284 2024-04-22T19:50:02.441 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.0.0-3152.g9ed8263e.el7.x86_ 280/284 2024-04-22T19:50:02.506 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 281/284 2024-04-22T19:50:02.538 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : s3cmd-2.4.0-1.el9.noarch 282/284 2024-04-22T19:50:02.538 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 283/284 2024-04-22T19:50:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 283/284 2024-04-22T19:50:02.630 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 284/284 2024-04-22T19:50:02.713 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 284/284 2024-04-22T19:50:02.730 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: dconf-0.40.0-6.el9.x86_64 284/284 2024-04-22T19:50:02.738 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: copy-jdk-configs-4.0-3.el9.noarch 284/284 2024-04-22T19:50:02.745 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x8 284/284 2024-04-22T19:50:02.754 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: wireplumber-0.4.14-1.el9.x86_64 284/284 2024-04-22T19:50:02.754 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/user/pipewire-session-manager.service → /usr/lib/systemd/user/wireplumber.service. 2024-04-22T19:50:02.754 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/user/pipewire.service.wants/wireplumber.service → /usr/lib/systemd/user/wireplumber.service. 2024-04-22T19:50:02.755 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:02.805 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 284/284 2024-04-22T19:50:02.872 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_6 284/284 2024-04-22T19:50:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 284/284 2024-04-22T19:50:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.0.0-3152.g9ed8263e.el7.x86_64 1/284 2024-04-22T19:50:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.0.0-3152.g9ed8263e.el7.x86_64 2/284 2024-04-22T19:50:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 3/284 2024-04-22T19:50:11.676 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 4/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3152.g9ed82 5/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.0.0-3152.g9ed8263e.el7.x86_64 6/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.0.0-3152.g9ed8263e.el7.x86_64 7/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.0.0-3152.g9ed8263e.el7.x86_64 8/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.0.0-3152.g9ed8263e.el7.x86_64 9/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 10/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.0.0-3152.g9ed8263e.el7.x86_64 11/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.0.0-3152.g9ed8263e.el7.x86_64 12/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.0.0-3152.g9ed8263e.el7.x86_ 13/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.0.0-3152.g9ed8263e.el7.x86_64 14/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.0.0-3152.g9ed8263e.el7.x86_64 15/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.0.0-3152.g9ed8263e.el7.x86_6 16/284 2024-04-22T19:50:11.677 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.0.0-3152.g9ed8263e.el7.x86 17/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.0.0-3152.g9ed8263e.el7.x86_64 18/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3152.g9ed8263e.el 19/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.0.0-3152.g9ed8263e.el7. 20/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.0.0-3152.g9ed8263e.el7.x86_6 21/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.0.0-3152.g9ed8263e.el7.x86_64 22/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 23/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 24/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 25/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.0.0-3152.g9ed8263e.el7.x86_64 26/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 27/284 2024-04-22T19:50:11.678 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3152.g9ed8263e. 28/284 2024-04-22T19:50:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3152.g9ed8263e.el7.noa 29/284 2024-04-22T19:50:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3152.g9ed8263e.el7.n 30/284 2024-04-22T19:50:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3152.g9ed 31/284 2024-04-22T19:50:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3152.g9ed8263e.el 32/284 2024-04-22T19:50:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3152.g9ed8263e.el7.noarch 33/284 2024-04-22T19:50:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3152.g9ed8263e.e 34/284 2024-04-22T19:50:11.704 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.0.0-3152.g9ed8263e.el7.noarch 35/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 36/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ModemManager-glib-1.20.2-1.el9.x86_64 37/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bluez-libs-5.56-6.el9.x86_64 38/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bubblewrap-0.4.1-7.el9.x86_64 39/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cups-libs-1:2.3.3op2-24.el9.x86_64 40/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 42/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpciaccess-0.16-7.el9.x86_64 43/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 45/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/284 2024-04-22T19:50:11.705 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : adwaita-cursor-theme-40.1.1-3.el9.noarch 52/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : adwaita-icon-theme-40.1.1-3.el9.noarch 53/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : alsa-lib-1.2.10-2.el9.x86_64 54/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : apache-commons-cli-1.4-17.el9.noarch 55/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : apache-commons-codec-1.15-7.el9.noarch 56/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : apache-commons-io-1:2.8.0-8.el9.noarch 57/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : apache-commons-lang3-3.12.0-6.el9.noarch 58/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : at-spi2-atk-2.38.0-4.el9.x86_64 59/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : at-spi2-core-2.40.3-1.el9.x86_64 60/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : atinject-1.0.3-6.el9.noarch 61/284 2024-04-22T19:50:11.706 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : atk-2.36.0-5.el9.x86_64 62/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : avahi-glib-0.8-20.el9.x86_64 63/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 64/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cdi-api-2.0.2-6.el9.noarch 65/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : colord-libs-1.4.5-4.el9.x86_64 66/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : copy-jdk-configs-4.0-3.el9.noarch 67/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : dconf-0.40.0-6.el9.x86_64 68/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : exempi-2.6.0-0.2.20211007gite23c213.el9.x86_64 69/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : exiv2-0.27.5-2.el9.x86_64 70/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : exiv2-libs-0.27.5-2.el9.x86_64 71/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fdk-aac-free-2.0.0-8.el9.x86_64 72/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flac-libs-1.3.3-12.el9.x86_64 73/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flatpak-1.12.8-1.el9.x86_64 74/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flatpak-selinux-1.12.8-1.el9.noarch 75/284 2024-04-22T19:50:11.707 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flatpak-session-helper-1.12.8-1.el9.x86_64 76/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 77/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 78/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 79/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gdk-pixbuf2-modules-2.42.6-3.el9.x86_64 80/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : geoclue2-2.6.0-7.el9.x86_64 81/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : giflib-5.2.1-9.el9.x86_64 82/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : google-guice-4.2.3-9.el9.noarch 83/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : graphene-1.10.6-2.el9.x86_64 84/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gsm-1.0.19-6.el9.x86_64 85/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gstreamer1-1.22.1-2.el9.x86_64 86/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gstreamer1-plugins-base-1.22.1-2.el9.x86_64 87/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gtk-update-icon-cache-3.24.31-2.el9.x86_64 88/284 2024-04-22T19:50:11.708 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gtk3-3.24.31-2.el9.x86_64 89/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : guava-30.1-6.el9.noarch 90/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : hicolor-icon-theme-0.17-13.el9.noarch 91/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : httpcomponents-client-4.5.13-3.el9.noarch 92/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : httpcomponents-core-4.4.13-7.el9.noarch 93/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : iso-codes-4.6.0-3.el9.noarch 94/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jakarta-annotations-1.3.5-13.el9.noarch 95/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jansi-2.3.3-6.el9.x86_64 96/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 97/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_6 98/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x8 99/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : javapackages-filesystem-6.0.0-4.el9.noarch 100/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jbigkit-libs-2.1-23.el9.x86_64 101/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jcl-over-slf4j-1.7.30-13.el9.noarch 102/284 2024-04-22T19:50:11.709 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jsoup-1.13.1-10.el9.noarch 103/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : jsr-305-3.0.2-6.el9.noarch 104/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lcms2-2.12-3.el9.x86_64 105/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libX11-xcb-1.7.0-9.el9.x86_64 106/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXcomposite-0.4.5-7.el9.x86_64 107/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXcursor-1.2.0-7.el9.x86_64 108/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXdamage-1.1.5-7.el9.x86_64 109/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXfixes-5.0.3-16.el9.x86_64 110/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXi-1.7.10-8.el9.x86_64 111/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXinerama-1.1.4-10.el9.x86_64 112/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXrandr-1.5.2-8.el9.x86_64 113/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXtst-1.2.3-16.el9.x86_64 114/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXv-1.0.11-16.el9.x86_64 115/284 2024-04-22T19:50:11.710 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libXxf86vm-1.1.4-18.el9.x86_64 116/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libasyncns-0.8-22.el9.x86_64 117/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcanberra-0.30-27.el9.x86_64 118/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcanberra-gtk3-0.30-27.el9.x86_64 119/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libdrm-2.4.117-1.el9.x86_64 120/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libepoxy-1.5.5-4.el9.x86_64 121/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libexif-0.6.22-6.el9.x86_64 122/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libfontenc-1.1.3-17.el9.x86_64 123/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgexiv2-0.12.3-1.el9.x86_64 124/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libglvnd-1:1.3.4-1.el9.x86_64 125/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libglvnd-egl-1:1.3.4-1.el9.x86_64 126/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libglvnd-glx-1:1.3.4-1.el9.x86_64 127/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgsf-1.14.47-5.el9.x86_64 128/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgxps-0.3.2-3.el9.x86_64 129/284 2024-04-22T19:50:11.711 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libiptcdata-1.0.5-9.el9.x86_64 130/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libldac-2.0.2.3-10.el9.x86_64 131/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libnotify-0.7.9-8.el9.x86_64 132/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libogg-2:1.3.4-6.el9.x86_64 133/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libosinfo-1.10.0-1.el9.x86_64 134/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 135/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 136/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 137/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libsbc-1.4-9.el9.x86_64 138/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libsndfile-1.0.31-8.el9.x86_64 139/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtheora-1:1.1.1-31.el9.x86_64 140/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtiff-4.4.0-12.el9.x86_64 141/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-ltdl-2.4.6-45.el9.x86_64 142/284 2024-04-22T19:50:11.712 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtracker-sparql-3.1.2-3.el9.x86_64 143/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libvisual-1:0.4.0-34.el9.x86_64 144/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libvorbis-1:1.3.7-5.el9.x86_64 145/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libwayland-client-1.21.0-1.el9.x86_64 146/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libwayland-cursor-1.21.0-1.el9.x86_64 147/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libwayland-egl-1.21.0-1.el9.x86_64 148/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libwayland-server-1.21.0-1.el9.x86_64 149/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libwebp-1.2.0-8.el9.x86_64 150/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libxkbcommon-1.0.3-4.el9.x86_64 151/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libxshmfence-1.3-10.el9.x86_64 152/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : low-memory-monitor-2.1-4.el9.x86_64 153/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 154/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 155/284 2024-04-22T19:50:11.713 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-posix-35.0-8.el9.x86_64 156/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : maven-1:3.6.3-15.el9.noarch 157/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : maven-lib-1:3.6.3-15.el9.noarch 158/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : maven-openjdk11-1:3.6.3-15.el9.noarch 159/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : maven-resolver-1:1.6.1-10.el9.noarch 160/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : maven-shared-utils-3.3.4-3.el9.noarch 161/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : maven-wagon-3.4.2-7.el9.noarch 162/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : mesa-libEGL-23.3.3-1.el9.x86_64 163/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : mesa-libGL-23.3.3-1.el9.x86_64 164/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : mesa-libgbm-23.3.3-1.el9.x86_64 165/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : mesa-libglapi-23.3.3-1.el9.x86_64 166/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : mkfontscale-1.2.1-3.el9.x86_64 167/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 168/284 2024-04-22T19:50:11.714 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 169/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openjpeg2-2.4.0-7.el9.x86_64 170/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : opus-1.3.1-10.el9.x86_64 171/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : orc-0.4.31-6.el9.x86_64 172/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : osinfo-db-20231215-1.el9.noarch 173/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : osinfo-db-tools-1.10.0-1.el9.x86_64 174/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ostree-libs-2024.5-2.el9.x86_64 175/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : p11-kit-server-0.25.3-2.el9.x86_64 176/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pipewire-1.0.1-1.el9.x86_64 177/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pipewire-alsa-1.0.1-1.el9.x86_64 178/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pipewire-jack-audio-connection-kit-1.0.1-1.el9.x 179/284 2024-04-22T19:50:11.715 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pipewire-jack-audio-connection-kit-libs-1.0.1-1. 180/284 2024-04-22T19:50:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pipewire-libs-1.0.1-1.el9.x86_64 181/284 2024-04-22T19:50:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pipewire-pulseaudio-1.0.1-1.el9.x86_64 182/284 2024-04-22T19:50:11.716 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : plexus-cipher-1.7-27.el9.noarch 183/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : plexus-classworlds-2.6.0-11.el9.noarch 184/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : plexus-containers-component-annotations-2.1.0-10 185/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : plexus-interpolation-1.26-11.el9.noarch 186/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : plexus-sec-dispatcher-1.4-37.el9.noarch 187/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : plexus-utils-3.3.0-10.el9.noarch 188/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : poppler-21.01.0-19.el9.x86_64 189/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : poppler-data-0.4.9-9.el9.noarch 190/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : poppler-glib-21.01.0-19.el9.x86_64 191/284 2024-04-22T19:50:11.717 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 192/284 2024-04-22T19:50:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : publicsuffix-list-20210518-3.el9.noarch 193/284 2024-04-22T19:50:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : pulseaudio-libs-15.0-2.el9.x86_64 194/284 2024-04-22T19:50:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 195/284 2024-04-22T19:50:11.718 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 196/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 197/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 198/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 199/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 200/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 201/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 202/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 203/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 204/284 2024-04-22T19:50:11.719 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 205/284 2024-04-22T19:50:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 206/284 2024-04-22T19:50:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 207/284 2024-04-22T19:50:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 208/284 2024-04-22T19:50:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 209/284 2024-04-22T19:50:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 210/284 2024-04-22T19:50:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 211/284 2024-04-22T19:50:11.720 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 212/284 2024-04-22T19:50:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 213/284 2024-04-22T19:50:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 214/284 2024-04-22T19:50:11.721 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 215/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 216/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rtkit-0.11-28.el9.x86_64 217/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : sisu-1:0.3.4-10.el9.noarch 218/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : slf4j-1.7.30-13.el9.noarch 219/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 220/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : sound-theme-freedesktop-0.8-17.el9.noarch 221/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : totem-pl-parser-3.26.6-2.el9.x86_64 222/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : tracker-3.1.2-3.el9.x86_64 223/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : tracker-miners-3.1.2-4.el9.x86_64 224/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ttmkfdir-3.0.9-65.el9.x86_64 225/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : tzdata-java-2024a-2.el9.noarch 226/284 2024-04-22T19:50:11.722 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : upower-0.99.11-11.el9.x86_64 227/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : webrtc-audio-processing-0.3.1-8.el9.x86_64 228/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : wireplumber-0.4.14-1.el9.x86_64 229/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : wireplumber-libs-0.4.14-1.el9.x86_64 230/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xdg-dbus-proxy-0.1.3-1.el9.x86_64 231/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xdg-desktop-portal-1.12.6-1.el9.x86_64 232/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xdg-desktop-portal-gtk-1.12.0-3.el9.x86_64 233/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xkeyboard-config-2.33-2.el9.noarch 234/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 235/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xorg-x11-fonts-Type1-7.5-33.el9.noarch 236/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 237/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 238/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 239/284 2024-04-22T19:50:11.723 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 240/284 2024-04-22T19:50:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 241/284 2024-04-22T19:50:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 242/284 2024-04-22T19:50:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 243/284 2024-04-22T19:50:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 244/284 2024-04-22T19:50:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 245/284 2024-04-22T19:50:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 246/284 2024-04-22T19:50:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 247/284 2024-04-22T19:50:11.724 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 248/284 2024-04-22T19:50:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 249/284 2024-04-22T19:50:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 250/284 2024-04-22T19:50:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 251/284 2024-04-22T19:50:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 252/284 2024-04-22T19:50:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 253/284 2024-04-22T19:50:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 254/284 2024-04-22T19:50:11.725 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 255/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 256/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 257/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 258/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 259/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 260/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 261/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 262/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 263/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 264/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 265/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 266/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 267/284 2024-04-22T19:50:11.726 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 268/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 269/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 270/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 271/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 272/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 273/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 274/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 275/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 276/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 277/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-3.el9.x86_64 278/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : s3cmd-2.4.0-1.el9.noarch 279/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 280/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.0.0-3152.g9ed8263e.el7.x86_64 281/284 2024-04-22T19:50:11.727 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 282/284 2024-04-22T19:50:11.728 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.0.0-3152.g9ed8263e.el7.x86_64 283/284 2024-04-22T19:50:15.639 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 284/284 2024-04-22T19:50:15.639 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:15.639 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-04-22T19:50:15.639 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.639 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: ModemManager-glib-1.20.2-1.el9.x86_64 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: adwaita-cursor-theme-40.1.1-3.el9.noarch 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: adwaita-icon-theme-40.1.1-3.el9.noarch 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: alsa-lib-1.2.10-2.el9.x86_64 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: apache-commons-cli-1.4-17.el9.noarch 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: apache-commons-codec-1.15-7.el9.noarch 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: apache-commons-io-1:2.8.0-8.el9.noarch 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: apache-commons-lang3-3.12.0-6.el9.noarch 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: at-spi2-atk-2.38.0-4.el9.x86_64 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: at-spi2-core-2.40.3-1.el9.x86_64 2024-04-22T19:50:15.640 INFO:teuthology.orchestra.run.smithi136.stdout: atinject-1.0.3-6.el9.noarch 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: atk-2.36.0-5.el9.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: avahi-glib-0.8-20.el9.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: bluez-libs-5.56-6.el9.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: bubblewrap-0.4.1-7.el9.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: cdi-api-2.0.2-6.el9.noarch 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.641 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.642 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.0.0-3152.g9ed8263e.el7.noarch 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: colord-libs-1.4.5-4.el9.x86_64 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: copy-jdk-configs-4.0-3.el9.noarch 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: cups-libs-1:2.3.3op2-24.el9.x86_64 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: dconf-0.40.0-6.el9.x86_64 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: exempi-2.6.0-0.2.20211007gite23c213.el9.x86_64 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: exiv2-0.27.5-2.el9.x86_64 2024-04-22T19:50:15.643 INFO:teuthology.orchestra.run.smithi136.stdout: exiv2-libs-0.27.5-2.el9.x86_64 2024-04-22T19:50:15.644 INFO:teuthology.orchestra.run.smithi136.stdout: fdk-aac-free-2.0.0-8.el9.x86_64 2024-04-22T19:50:15.644 INFO:teuthology.orchestra.run.smithi136.stdout: flac-libs-1.3.3-12.el9.x86_64 2024-04-22T19:50:15.644 INFO:teuthology.orchestra.run.smithi136.stdout: flatpak-1.12.8-1.el9.x86_64 2024-04-22T19:50:15.644 INFO:teuthology.orchestra.run.smithi136.stdout: flatpak-selinux-1.12.8-1.el9.noarch 2024-04-22T19:50:15.644 INFO:teuthology.orchestra.run.smithi136.stdout: flatpak-session-helper-1.12.8-1.el9.x86_64 2024-04-22T19:50:15.644 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-22T19:50:15.644 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: gdk-pixbuf2-modules-2.42.6-3.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: geoclue2-2.6.0-7.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: giflib-5.2.1-9.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: google-guice-4.2.3-9.el9.noarch 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: graphene-1.10.6-2.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: gsm-1.0.19-6.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: gstreamer1-1.22.1-2.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: gstreamer1-plugins-base-1.22.1-2.el9.x86_64 2024-04-22T19:50:15.645 INFO:teuthology.orchestra.run.smithi136.stdout: gtk-update-icon-cache-3.24.31-2.el9.x86_64 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: gtk3-3.24.31-2.el9.x86_64 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: guava-30.1-6.el9.noarch 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: hicolor-icon-theme-0.17-13.el9.noarch 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: httpcomponents-client-4.5.13-3.el9.noarch 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: httpcomponents-core-4.4.13-7.el9.noarch 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: iso-codes-4.6.0-3.el9.noarch 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: jakarta-annotations-1.3.5-13.el9.noarch 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: jansi-2.3.3-6.el9.x86_64 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: java-11-openjdk-1:11.0.18.0.10-3.el9.x86_64 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: java-11-openjdk-devel-1:11.0.18.0.10-3.el9.x86_64 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: java-11-openjdk-headless-1:11.0.18.0.10-3.el9.x86_64 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: javapackages-filesystem-6.0.0-4.el9.noarch 2024-04-22T19:50:15.646 INFO:teuthology.orchestra.run.smithi136.stdout: jbigkit-libs-2.1-23.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: jcl-over-slf4j-1.7.30-13.el9.noarch 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: jsoup-1.13.1-10.el9.noarch 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: jsr-305-3.0.2-6.el9.noarch 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: lcms2-2.12-3.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libX11-xcb-1.7.0-9.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXcomposite-0.4.5-7.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXcursor-1.2.0-7.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXdamage-1.1.5-7.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXfixes-5.0.3-16.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXi-1.7.10-8.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXinerama-1.1.4-10.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXrandr-1.5.2-8.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXtst-1.2.3-16.el9.x86_64 2024-04-22T19:50:15.647 INFO:teuthology.orchestra.run.smithi136.stdout: libXv-1.0.11-16.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libXxf86vm-1.1.4-18.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libasyncns-0.8-22.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libcanberra-0.30-27.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libcanberra-gtk3-0.30-27.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libdrm-2.4.117-1.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libepoxy-1.5.5-4.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libexif-0.6.22-6.el9.x86_64 2024-04-22T19:50:15.648 INFO:teuthology.orchestra.run.smithi136.stdout: libfontenc-1.1.3-17.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libgexiv2-0.12.3-1.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libglvnd-1:1.3.4-1.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libglvnd-egl-1:1.3.4-1.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libglvnd-glx-1:1.3.4-1.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libgsf-1.14.47-5.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libgxps-0.3.2-3.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libiptcdata-1.0.5-9.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libldac-2.0.2.3-10.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libnotify-0.7.9-8.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libogg-2:1.3.4-6.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libosinfo-1.10.0-1.el9.x86_64 2024-04-22T19:50:15.649 INFO:teuthology.orchestra.run.smithi136.stdout: libpciaccess-0.16-7.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libsbc-1.4-9.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libsndfile-1.0.31-8.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libtheora-1:1.1.1-31.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libtiff-4.4.0-12.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-ltdl-2.4.6-45.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libtracker-sparql-3.1.2-3.el9.x86_64 2024-04-22T19:50:15.650 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libvisual-1:0.4.0-34.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libvorbis-1:1.3.7-5.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libwayland-client-1.21.0-1.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libwayland-cursor-1.21.0-1.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libwayland-egl-1.21.0-1.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libwayland-server-1.21.0-1.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libwebp-1.2.0-8.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libxkbcommon-1.0.3-4.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: libxshmfence-1.3-10.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: low-memory-monitor-2.1-4.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-04-22T19:50:15.651 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: lua-posix-35.0-8.el9.x86_64 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: maven-1:3.6.3-15.el9.noarch 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: maven-lib-1:3.6.3-15.el9.noarch 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: maven-openjdk11-1:3.6.3-15.el9.noarch 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: maven-resolver-1:1.6.1-10.el9.noarch 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: maven-shared-utils-3.3.4-3.el9.noarch 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: maven-wagon-3.4.2-7.el9.noarch 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: mesa-libEGL-23.3.3-1.el9.x86_64 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: mesa-libGL-23.3.3-1.el9.x86_64 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: mesa-libgbm-23.3.3-1.el9.x86_64 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: mesa-libglapi-23.3.3-1.el9.x86_64 2024-04-22T19:50:15.652 INFO:teuthology.orchestra.run.smithi136.stdout: mkfontscale-1.2.1-3.el9.x86_64 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: openjpeg2-2.4.0-7.el9.x86_64 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: opus-1.3.1-10.el9.x86_64 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: orc-0.4.31-6.el9.x86_64 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: osinfo-db-20231215-1.el9.noarch 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: osinfo-db-tools-1.10.0-1.el9.x86_64 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: ostree-libs-2024.5-2.el9.x86_64 2024-04-22T19:50:15.653 INFO:teuthology.orchestra.run.smithi136.stdout: p11-kit-server-0.25.3-2.el9.x86_64 2024-04-22T19:50:15.654 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-22T19:50:15.654 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-1.0.1-1.el9.x86_64 2024-04-22T19:50:15.654 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-alsa-1.0.1-1.el9.x86_64 2024-04-22T19:50:15.654 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-jack-audio-connection-kit-1.0.1-1.el9.x86_64 2024-04-22T19:50:15.654 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-jack-audio-connection-kit-libs-1.0.1-1.el9.x86_64 2024-04-22T19:50:15.654 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-libs-1.0.1-1.el9.x86_64 2024-04-22T19:50:15.654 INFO:teuthology.orchestra.run.smithi136.stdout: pipewire-pulseaudio-1.0.1-1.el9.x86_64 2024-04-22T19:50:15.655 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-cipher-1.7-27.el9.noarch 2024-04-22T19:50:15.655 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-classworlds-2.6.0-11.el9.noarch 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-containers-component-annotations-2.1.0-10.el9.noarch 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-interpolation-1.26-11.el9.noarch 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-sec-dispatcher-1.4-37.el9.noarch 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: plexus-utils-3.3.0-10.el9.noarch 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: poppler-21.01.0-19.el9.x86_64 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: poppler-data-0.4.9-9.el9.noarch 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: poppler-glib-21.01.0-19.el9.x86_64 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: publicsuffix-list-20210518-3.el9.noarch 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: pulseaudio-libs-15.0-2.el9.x86_64 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-22T19:50:15.656 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-22T19:50:15.657 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-22T19:50:15.658 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-22T19:50:15.659 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-22T19:50:15.660 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.661 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-22T19:50:15.661 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-22T19:50:15.662 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-22T19:50:15.662 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-22T19:50:15.662 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-22T19:50:15.662 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-22T19:50:15.662 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.0.0-3152.g9ed8263e.el7.x86_64 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-3.el9.x86_64 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: rtkit-0.11-28.el9.x86_64 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: s3cmd-2.4.0-1.el9.noarch 2024-04-22T19:50:15.663 INFO:teuthology.orchestra.run.smithi136.stdout: sisu-1:0.3.4-10.el9.noarch 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: slf4j-1.7.30-13.el9.noarch 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: sound-theme-freedesktop-0.8-17.el9.noarch 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: totem-pl-parser-3.26.6-2.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: tracker-3.1.2-3.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: tracker-miners-3.1.2-4.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: ttmkfdir-3.0.9-65.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: tzdata-java-2024a-2.el9.noarch 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: upower-0.99.11-11.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: webrtc-audio-processing-0.3.1-8.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: wireplumber-0.4.14-1.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: wireplumber-libs-0.4.14-1.el9.x86_64 2024-04-22T19:50:15.664 INFO:teuthology.orchestra.run.smithi136.stdout: xdg-dbus-proxy-0.1.3-1.el9.x86_64 2024-04-22T19:50:15.665 INFO:teuthology.orchestra.run.smithi136.stdout: xdg-desktop-portal-1.12.6-1.el9.x86_64 2024-04-22T19:50:15.665 INFO:teuthology.orchestra.run.smithi136.stdout: xdg-desktop-portal-gtk-1.12.0-3.el9.x86_64 2024-04-22T19:50:15.665 INFO:teuthology.orchestra.run.smithi136.stdout: xkeyboard-config-2.33-2.el9.noarch 2024-04-22T19:50:15.665 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-22T19:50:15.665 INFO:teuthology.orchestra.run.smithi136.stdout: xorg-x11-fonts-Type1-7.5-33.el9.noarch 2024-04-22T19:50:15.665 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:15.665 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-22T19:50:16.089 DEBUG:teuthology.parallel:result is None 2024-04-22T19:50:16.090 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:50:16.311 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T19:50:16.369 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-3152.g9ed8263e.el7 2024-04-22T19:50:16.370 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3152.g9ed8263e.el7 2024-04-22T19:50:16.370 INFO:teuthology.task.install:The correct ceph version 19.0.0-3152.g9ed8263e is installed. 2024-04-22T19:50:16.372 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:50:16.520 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-22T19:50:16.576 INFO:teuthology.orchestra.run.smithi164.stdout:19.0.0-3152.g9ed8263e.el7 2024-04-22T19:50:16.576 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3152.g9ed8263e.el7 2024-04-22T19:50:16.576 INFO:teuthology.task.install:The correct ceph version 19.0.0-3152.g9ed8263e is installed. 2024-04-22T19:50:16.578 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-22T19:50:16.578 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:16.578 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T19:50:16.635 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:16.635 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-22T19:50:16.680 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-22T19:50:16.680 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:16.680 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T19:50:16.717 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T19:50:16.793 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:16.794 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-04-22T19:50:16.834 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-22T19:50:16.913 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-22T19:50:16.923 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:16.923 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T19:50:16.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T19:50:17.037 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:17.037 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-22T19:50:17.074 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-22T19:50:17.152 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-22T19:50:17.152 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:17.152 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T19:50:17.189 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T19:50:17.265 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:17.265 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-04-22T19:50:17.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-22T19:50:17.380 INFO:teuthology.run_tasks:Running task ceph... 2024-04-22T19:50:17.479 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-22T19:50:17.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2024-04-22T19:50:17.481 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-04-22T19:50:17.517 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-22T19:50:17.517 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T19:50:17.557 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-22T19:50:17.593 INFO:tasks.ceph:Creating extra log directories... 2024-04-22T19:50:17.593 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T19:50:17.634 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-22T19:50:17.686 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-22T19:50:17.686 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-22T19:50:17.686 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525', 'branch': 'wip-65551', 'description': 'rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7668525', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-65551'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe'}}, 'openssl_keys': {'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}, 'root': {'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0'], 'key-type': 'rsa:4096'}}, 'rgw': {'client.0': {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'compression type': 'random', 'ec-data-pool': True, 'frontend': 'beast', 'inline data': False, 'storage classes': 'LUKEWARM, FROZEN'}, 's3tests': {'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 tenant': 'RGW33333333333333333'}, 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-65551', 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe'}}, 'owner': 'scheduled_cbodley@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 7738, 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'wip-65551', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe', 'targets': {'smithi136.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJSCkFNzF6ZvVnumrKIjSS6/2qOd60SruNj/CDdUaDztzMmFgqo2+kgONtxBG4u9h14txovdnHTbzSRMJ1F78DU=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1p9ccY9F1b48i+zlg0avenm21Ad/5kMNXPeMFgLexvKge3oAIUxDNWtttOMQZ0FQmAJsbIkeSjsllTxCSr6vI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '9ed8263edff7e87907bf24b054e5c7144c194dbe'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'tox': ['client.0']}, {'workunit': {'clients': {'client.0': ['rgw/run-bucket-check.sh']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/test_rgw_s3_mp_reupload.sh']}}}, {'ragweed': {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-reshard.sh']}}}, {'s3tests-java': {'client.0': {'force-branch': 'ceph-master', 'force-repo': 'https://github.com/ceph/java_s3tests.git'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-versioning.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-22_19:31:26', 'tube': 'smithi', 'user': 'cbodley', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-22T19:50:17.690 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T19:50:17.710 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-22T19:50:17.739 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T19:50:17.778 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-04-22T19:50:17.816 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:17.816 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T19:50:17.879 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T19:50:17.880 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-04-22T19:50:17.947 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T19:50:17.947 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T19:50:17.947 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-22T19:50:17.947 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T19:50:17.947 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T19:50:17.947 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-22 19:50:03.659155510 +0000 2024-04-22T19:50:17.947 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-22 19:46:48.251164081 +0000 2024-04-22T19:50:17.947 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-22 19:46:48.251164081 +0000 2024-04-22T19:50:17.948 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-22 19:46:48.251164081 +0000 2024-04-22T19:50:17.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T19:50:18.022 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-22T19:50:18.022 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-22T19:50:18.022 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000194118 s, 2.6 MB/s 2024-04-22T19:50:18.023 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T19:50:18.091 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-04-22T19:50:18.158 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T19:50:18.158 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T19:50:18.158 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-22T19:50:18.158 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T19:50:18.158 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T19:50:18.159 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-22 19:50:03.660155487 +0000 2024-04-22T19:50:18.159 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-22 19:46:48.597156050 +0000 2024-04-22T19:50:18.159 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-22 19:46:48.597156050 +0000 2024-04-22T19:50:18.159 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-22 19:46:48.597156050 +0000 2024-04-22T19:50:18.159 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T19:50:18.232 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-22T19:50:18.233 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-22T19:50:18.233 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000169389 s, 3.0 MB/s 2024-04-22T19:50:18.234 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T19:50:18.303 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-04-22T19:50:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T19:50:18.370 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T19:50:18.370 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-22T19:50:18.371 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T19:50:18.371 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T19:50:18.371 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-22 19:50:03.660155487 +0000 2024-04-22T19:50:18.371 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-22 19:46:48.960147625 +0000 2024-04-22T19:50:18.371 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-22 19:46:48.960147625 +0000 2024-04-22T19:50:18.371 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-22 19:46:48.960147625 +0000 2024-04-22T19:50:18.371 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T19:50:18.446 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-22T19:50:18.446 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-22T19:50:18.446 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000229981 s, 2.2 MB/s 2024-04-22T19:50:18.447 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T19:50:18.515 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-04-22T19:50:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T19:50:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T19:50:18.583 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-22T19:50:18.583 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T19:50:18.583 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T19:50:18.583 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-22 19:50:03.660155487 +0000 2024-04-22T19:50:18.584 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-22 19:46:49.320139269 +0000 2024-04-22T19:50:18.584 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-22 19:46:49.320139269 +0000 2024-04-22T19:50:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-22 19:46:49.320139269 +0000 2024-04-22T19:50:18.584 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T19:50:18.657 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-22T19:50:18.658 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-22T19:50:18.658 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000195196 s, 2.6 MB/s 2024-04-22T19:50:18.659 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T19:50:18.728 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:18.728 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:18.728 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-04-22T19:50:18.761 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-22T19:50:18.761 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-04-22T19:50:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-22T19:50:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T19:50:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-22T19:50:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T19:50:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T19:50:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-22 19:49:39.305222051 +0000 2024-04-22T19:50:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-22 19:46:48.224997333 +0000 2024-04-22T19:50:18.835 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-22 19:46:48.224997333 +0000 2024-04-22T19:50:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-22 19:46:48.224997333 +0000 2024-04-22T19:50:18.835 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-22T19:50:18.915 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-22T19:50:18.916 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-22T19:50:18.916 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000221311 s, 2.3 MB/s 2024-04-22T19:50:18.917 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-22T19:50:18.989 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-04-22T19:50:19.059 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-22T19:50:19.059 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T19:50:19.059 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-22T19:50:19.059 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T19:50:19.059 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T19:50:19.059 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-22 19:49:39.306222035 +0000 2024-04-22T19:50:19.060 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-22 19:46:48.596991011 +0000 2024-04-22T19:50:19.060 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-22 19:46:48.596991011 +0000 2024-04-22T19:50:19.060 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-22 19:46:48.596991011 +0000 2024-04-22T19:50:19.060 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-22T19:50:19.138 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-22T19:50:19.138 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-22T19:50:19.138 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000214476 s, 2.4 MB/s 2024-04-22T19:50:19.139 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-22T19:50:19.207 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-22 19:49:39.306222035 +0000 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-22 19:46:48.963984773 +0000 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-22 19:46:48.963984773 +0000 2024-04-22T19:50:19.276 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-22 19:46:48.963984773 +0000 2024-04-22T19:50:19.277 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-22T19:50:19.351 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-22T19:50:19.351 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-22T19:50:19.351 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000183593 s, 2.8 MB/s 2024-04-22T19:50:19.352 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-22T19:50:19.421 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-04-22T19:50:19.490 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-22T19:50:19.490 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-22T19:50:19.490 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-22T19:50:19.490 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-22T19:50:19.490 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-04-22T19:50:19.491 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-04-22 19:49:39.306222035 +0000 2024-04-22T19:50:19.491 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-04-22 19:46:49.332978502 +0000 2024-04-22T19:50:19.491 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-04-22 19:46:49.332978502 +0000 2024-04-22T19:50:19.491 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-04-22 19:46:49.332978502 +0000 2024-04-22T19:50:19.491 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-22T19:50:19.565 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-04-22T19:50:19.566 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-04-22T19:50:19.566 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000197581 s, 2.6 MB/s 2024-04-22T19:50:19.567 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-22T19:50:19.636 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:19.636 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T19:50:19.636 INFO:tasks.ceph:Generating config... 2024-04-22T19:50:19.637 INFO:tasks.ceph:[client] debug rgw = 20 2024-04-22T19:50:19.637 INFO:tasks.ceph:[client] rgw bucket counters cache = True 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] rgw s3 auth use sts = True 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] rgw sts key = abcdefghijklmnop 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] rgw torrent flag = True 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] rgw user counters cache = True 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] setgroup = ceph 2024-04-22T19:50:19.638 INFO:tasks.ceph:[client] setuser = ceph 2024-04-22T19:50:19.638 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-22T19:50:19.638 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-22T19:50:19.638 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2024-04-22T19:50:19.638 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2024-04-22T19:50:19.638 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2024-04-22T19:50:19.638 INFO:tasks.ceph:[global] rgw override bucket index max shards = 1 2024-04-22T19:50:19.639 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-22T19:50:19.639 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-22T19:50:19.639 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-22T19:50:19.639 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-22T19:50:19.639 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-22T19:50:19.639 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-22T19:50:19.639 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-22T19:50:19.640 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-22T19:50:19.640 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-22T19:50:19.640 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-22T19:50:19.640 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-22T19:50:19.640 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-22T19:50:19.640 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-04-22T19:50:19.640 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-22T19:50:19.640 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-22T19:50:19.640 INFO:tasks.ceph:Setting up mon.a... 2024-04-22T19:50:19.640 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-22T19:50:19.707 INFO:teuthology.orchestra.run.smithi136.stdout:creating /etc/ceph/ceph.keyring 2024-04-22T19:50:19.710 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-22T19:50:19.766 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T19:50:19.803 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.136'), ('mon.c', '[v2:172.21.15.136:3301,v1:172.21.15.136:6790]'), ('mon.b', '172.21.15.164')] 2024-04-22T19:50:19.803 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790],172.21.15.164', 'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-04-22T19:50:19.804 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:19.804 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T19:50:19.871 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.136 --addv c '[v2:172.21.15.136:3301,v1:172.21.15.136:6790]' --add b 172.21.15.164 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:50:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:50:19.968 INFO:teuthology.orchestra.run.smithi136.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:19.968 INFO:teuthology.orchestra.run.smithi136.stdout:monmaptool: generated fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T19:50:19.968 INFO:teuthology.orchestra.run.smithi136.stdout:setting min_mon_release = quincy 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:epoch 0 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:created 2024-04-22T19:50:19.969705+0000 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:min_mon_release 17 (quincy) 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:election_strategy: 1 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T19:50:19.969 INFO:teuthology.orchestra.run.smithi136.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-22T19:50:19.970 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-22T19:50:19.997 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b2f95cfa-7b26-4a63-b13e-7105b011d9c4... 2024-04-22T19:50:19.998 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T19:50:20.041 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.092 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2024-04-22T19:50:20.093 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon_allow_pool_size_one = true 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.094 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790],172.21.15.164" 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: mon client directed command retry = 5 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject socket failures = 5000 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: osd heartbeat grace = 40 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: osd_max_pg_log_entries = 10 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: osd_min_pg_log_entries = 10 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: rgw override bucket index max shards = 1 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.095 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 600 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock profile = high_recovery_ops 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.096 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug trim objects = true 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2024-04-22T19:50:20.097 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: bdev async discard = True 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: bdev enable discard = True 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore allocator = bitmap 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 1/20 2024-04-22T19:50:20.098 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 1/20 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 4/10 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: osd fast shutdown = False 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: osd shutdown pgref assert = True 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2024-04-22T19:50:20.099 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-04-22T19:50:20.100 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2024-04-22T19:50:20.100 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-04-22T19:50:20.100 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-04-22T19:50:20.100 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-04-22T19:50:20.100 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-04-22T19:50:20.100 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2024-04-22T19:50:20.100 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.100 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2024-04-22T19:50:20.101 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-04-22T19:50:20.101 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-04-22T19:50:20.101 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2024-04-22T19:50:20.101 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-04-22T19:50:20.101 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2024-04-22T19:50:20.101 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2024-04-22T19:50:20.101 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: # 1m isn't quite enough 2024-04-22T19:50:20.102 INFO:teuthology.orchestra.run.smithi136.stdout: mon_down_mkfs_grace = 2m 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_filestore_osds = false 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd crush smoke test = False 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: debug rgw = 20 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: rgw bucket counters cache = True 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: rgw crypt require ssl = False 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: rgw crypt s3 kms backend = testing 2024-04-22T19:50:20.103 INFO:teuthology.orchestra.run.smithi136.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout: rgw s3 auth use sts = True 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout: rgw sts key = abcdefghijklmnop 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout: rgw torrent flag = True 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout: rgw user counters cache = True 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout: setgroup = ceph 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout: setuser = ceph 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2024-04-22T19:50:20.104 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-04-22T19:50:20.106 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-04-22T19:50:20.107 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-04-22T19:50:20.108 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790],172.21.15.164" 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: mon client directed command retry = 5 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: ms inject socket failures = 5000 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: osd heartbeat grace = 40 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: osd_max_pg_log_entries = 10 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: osd_min_pg_log_entries = 10 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: rgw override bucket index max shards = 1 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-04-22T19:50:20.109 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-04-22T19:50:20.110 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-04-22T19:50:20.110 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.110 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-04-22T19:50:20.110 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-04-22T19:50:20.110 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.110 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-04-22T19:50:20.110 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.111 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: bdev async discard = True 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: bdev enable discard = True 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore allocator = bitmap 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-04-22T19:50:20.112 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 1/20 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 1/20 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 4/10 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: osd fast shutdown = False 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: osd shutdown pgref assert = True 2024-04-22T19:50:20.113 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-04-22T19:50:20.114 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-22T19:50:20.115 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd crush smoke test = False 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: debug rgw = 20 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: rgw bucket counters cache = True 2024-04-22T19:50:20.116 INFO:teuthology.orchestra.run.smithi164.stdout: rgw crypt require ssl = False 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: rgw crypt s3 kms backend = testing 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: rgw s3 auth use sts = True 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: rgw sts key = abcdefghijklmnop 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: rgw torrent flag = True 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: rgw user counters cache = True 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: setgroup = ceph 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout: setuser = ceph 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-04-22T19:50:20.117 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-04-22T19:50:20.118 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-22T19:50:20.118 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-22T19:50:20.211 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-22T19:50:20.211 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:20.211 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-22T19:50:20.238 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:20.238 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-22T19:50:20.307 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T19:50:20.307 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:20.308 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T19:50:20.308 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T19:50:20.393 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:20.393 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:20.460 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T19:50:20.460 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:20.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-22T19:50:20.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-22T19:50:20.507 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:20.507 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:20.575 INFO:tasks.ceph:Setting up mon nodes... 2024-04-22T19:50:20.575 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-22T19:50:20.575 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T19:50:20.637 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-22T19:50:20.639 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T19:50:20.713 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-22T19:50:20.715 INFO:tasks.ceph:Setting up mds nodes... 2024-04-22T19:50:20.715 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-22T19:50:20.716 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-22T19:50:20.772 INFO:teuthology.orchestra.run.smithi136.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-22T19:50:20.780 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-22T19:50:20.832 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-22T19:50:20.845 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-22T19:50:20.845 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-22T19:50:20.845 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-22T19:50:20.881 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:20.881 INFO:tasks.ceph:role: osd.0 2024-04-22T19:50:20.881 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T19:50:20.881 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T19:50:20.965 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T19:50:21.487 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-04-22T19:50:21.489 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-04-22T19:50:21.489 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-22T19:50:21.598 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-22T19:50:21.639 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-22T19:50:21.716 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:21.716 INFO:tasks.ceph:role: osd.1 2024-04-22T19:50:21.716 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T19:50:21.716 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T19:50:21.800 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T19:50:21.800 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T19:50:21.800 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T19:50:21.800 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T19:50:21.801 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T19:50:21.801 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-04-22T19:50:21.801 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T19:50:21.801 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T19:50:21.801 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T19:50:21.801 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T19:50:22.303 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-04-22T19:50:22.305 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-04-22T19:50:22.305 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-22T19:50:22.430 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-22T19:50:22.475 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-22T19:50:22.510 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:22.510 INFO:tasks.ceph:role: osd.2 2024-04-22T19:50:22.510 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T19:50:22.510 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T19:50:22.595 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T19:50:23.101 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-04-22T19:50:23.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-04-22T19:50:23.103 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-22T19:50:23.167 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-22T19:50:23.211 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-22T19:50:23.288 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:23.288 INFO:tasks.ceph:role: osd.3 2024-04-22T19:50:23.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T19:50:23.289 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T19:50:23.374 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T19:50:23.375 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T19:50:23.904 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-04-22T19:50:23.907 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-04-22T19:50:23.907 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-22T19:50:23.997 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-22T19:50:24.039 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:24.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:24.140+0000 7fc1b3013640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-22T19:50:24.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:24.144+0000 7fc1b3013640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-22T19:50:24.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:24.144+0000 7fc1b3013640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T19:50:24.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:24.144+0000 7fc1b3013640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T19:50:24.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:24.144+0000 7fc1b3013640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-22T19:50:24.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:24.151+0000 7fc1b3013640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-22T19:50:27.003 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-22T19:50:27.045 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:27.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:27.147+0000 7f09ea414640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-22T19:50:27.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:27.147+0000 7f09ea414640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-22T19:50:27.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:27.147+0000 7f09ea414640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T19:50:27.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:27.147+0000 7f09ea414640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T19:50:27.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:27.147+0000 7f09ea414640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-22T19:50:27.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:27.147+0000 7f09ea414640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-22T19:50:29.995 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-22T19:50:30.037 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:30.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:30.138+0000 7f06c621c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-22T19:50:30.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:30.138+0000 7f06c621c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-22T19:50:30.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:30.138+0000 7f06c621c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T19:50:30.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:30.138+0000 7f06c621c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T19:50:30.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:30.138+0000 7f06c621c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-22T19:50:30.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:30.139+0000 7f06c621c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-22T19:50:32.993 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-22T19:50:33.032 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:33.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:33.129+0000 7f213ee1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-22T19:50:33.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:33.130+0000 7f213ee1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-22T19:50:33.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:33.130+0000 7f213ee1b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T19:50:33.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:33.130+0000 7f213ee1b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T19:50:33.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:33.130+0000 7f213ee1b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-22T19:50:33.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:50:33.130+0000 7f213ee1b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-22T19:50:35.976 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-22T19:50:36.014 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-22T19:50:36.058 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:36.058 INFO:tasks.ceph:role: osd.4 2024-04-22T19:50:36.058 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T19:50:36.059 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-22T19:50:36.146 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T19:50:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T19:50:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T19:50:36.146 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T19:50:36.147 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T19:50:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-22T19:50:36.147 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T19:50:36.147 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T19:50:36.147 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T19:50:36.147 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T19:50:36.629 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-22T19:50:36.631 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-22T19:50:36.632 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-22T19:50:36.750 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-22T19:50:36.797 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-22T19:50:36.836 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:36.836 INFO:tasks.ceph:role: osd.5 2024-04-22T19:50:36.836 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T19:50:36.836 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-22T19:50:36.922 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T19:50:36.922 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T19:50:36.922 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T19:50:36.922 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T19:50:36.923 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T19:50:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-22T19:50:36.923 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T19:50:36.923 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T19:50:36.923 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T19:50:36.923 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T19:50:37.371 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-22T19:50:37.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-22T19:50:37.374 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-22T19:50:37.419 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-22T19:50:37.460 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-22T19:50:37.537 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:37.537 INFO:tasks.ceph:role: osd.6 2024-04-22T19:50:37.537 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T19:50:37.537 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-22T19:50:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T19:50:37.620 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T19:50:37.620 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T19:50:37.620 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T19:50:37.620 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T19:50:37.620 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-22T19:50:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T19:50:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T19:50:37.621 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T19:50:37.621 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T19:50:38.104 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-22T19:50:38.106 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-22T19:50:38.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-22T19:50:38.220 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-22T19:50:38.263 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-22T19:50:38.339 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-22T19:50:38.339 INFO:tasks.ceph:role: osd.7 2024-04-22T19:50:38.339 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T19:50:38.339 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-22T19:50:38.423 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-22T19:50:38.423 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-22T19:50:38.423 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-22T19:50:38.423 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-22T19:50:38.423 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-22T19:50:38.423 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-04-22T19:50:38.423 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-22T19:50:38.424 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-22T19:50:38.424 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-22T19:50:38.424 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-22T19:50:38.939 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-04-22T19:50:38.941 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-04-22T19:50:38.941 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-22T19:50:39.072 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-22T19:50:39.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:39.220 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:39.221+0000 7fdb359ef640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-22T19:50:39.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:39.224+0000 7fdb359ef640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-22T19:50:39.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:39.224+0000 7fdb359ef640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T19:50:39.223 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:39.224+0000 7fdb359ef640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T19:50:39.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:39.224+0000 7fdb359ef640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-22T19:50:39.224 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:39.225+0000 7fdb359ef640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-22T19:50:42.074 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-22T19:50:42.116 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:42.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:42.218+0000 7fe281a30640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-22T19:50:42.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:42.219+0000 7fe281a30640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-22T19:50:42.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:42.219+0000 7fe281a30640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T19:50:42.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:42.219+0000 7fe281a30640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T19:50:42.218 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:42.219+0000 7fe281a30640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-22T19:50:42.219 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:42.219+0000 7fe281a30640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-22T19:50:45.067 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-22T19:50:45.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:45.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:45.210+0000 7fd194718640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-22T19:50:45.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:45.210+0000 7fd194718640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-22T19:50:45.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:45.210+0000 7fd194718640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T19:50:45.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:45.210+0000 7fd194718640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T19:50:45.210 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:45.210+0000 7fd194718640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-22T19:50:45.211 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:45.211+0000 7fd194718640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-22T19:50:48.071 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-22T19:50:48.113 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:48.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:48.215+0000 7f65dd9b8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-22T19:50:48.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:48.215+0000 7f65dd9b8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-22T19:50:48.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:48.215+0000 7f65dd9b8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T19:50:48.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:48.215+0000 7f65dd9b8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T19:50:48.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:48.215+0000 7f65dd9b8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-22T19:50:48.215 INFO:teuthology.orchestra.run.smithi164.stderr:2024-04-22T19:50:48.216+0000 7f65dd9b8640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-22T19:50:51.079 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-22T19:50:51.121 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-22T19:50:51.121 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:51.121 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-22T19:50:51.166 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:51.166 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-22T19:50:51.247 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:51.247 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-22T19:50:51.325 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:51.325 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-22T19:50:51.402 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:51.402 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-22T19:50:51.479 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:51.479 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-22T19:50:51.522 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:51.522 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-22T19:50:51.601 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:51.601 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-22T19:50:51.679 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:51.679 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-22T19:50:51.756 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:51.756 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-22T19:50:51.833 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:51.833 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-22T19:50:51.861 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:51.861 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-22T19:50:51.900 INFO:tasks.ceph:Adding keys to all mons... 2024-04-22T19:50:51.900 DEBUG:teuthology.orchestra.run.smithi136:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T19:50:51.903 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-22T19:50:51.943 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.y] 2024-04-22T19:50:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQB8vyZmxeoHJhAAi9/mhC8aaG/rwqRzy5YdEw== 2024-04-22T19:50:51.943 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.0] 2024-04-22T19:50:51.943 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQCAvyZm8J95CBAAFjcWenxQxtQ4T3t2Jb2lTw== 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.1] 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQCDvyZmzCHZCBAAaWD5ALwnasLyhXHvFlpduA== 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.2] 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQCGvyZmHSBTCBAA0BLTssv1HA7Qsm+0asqZUw== 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.3] 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQCJvyZmhOLRBxAAMRI66SXYeV4X1I8fPIiH1g== 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.x] 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQB8vyZmbVqUKhAAZM3ylDJexnDof+Z+h2/oyg== 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.4] 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQCPvyZmYf46DRAA5u97eq/MeR/8jBHBwqTayA== 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.5] 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQCSvyZmuD4SDRAATGz271MGOiD3b/fQdkq9cQ== 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.6] 2024-04-22T19:50:51.944 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQCVvyZmCLSTDBAADoj65t5eLUjXuPexlNdj4w== 2024-04-22T19:50:51.945 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.7] 2024-04-22T19:50:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQCYvyZmI4LbDBAA7Exec6UF2PZ2mUw7M7e9Gg== 2024-04-22T19:50:51.945 INFO:teuthology.orchestra.run.smithi136.stdout:[client.0] 2024-04-22T19:50:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQB8vyZmcxzdLRAAjcyQavtE1gchAEHMcFSDjA== 2024-04-22T19:50:51.945 INFO:teuthology.orchestra.run.smithi136.stdout:[client.1] 2024-04-22T19:50:51.945 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQB8vyZmcI+0MRAAJq6aItZgJNwL5TVIxnjyzQ== 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQB8vyZmxeoHJhAAi9/mhC8aaG/rwqRzy5YdEw== 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCAvyZm8J95CBAAFjcWenxQxtQ4T3t2Jb2lTw== 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCDvyZmzCHZCBAAaWD5ALwnasLyhXHvFlpduA== 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCGvyZmHSBTCBAA0BLTssv1HA7Qsm+0asqZUw== 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCJvyZmhOLRBxAAMRI66SXYeV4X1I8fPIiH1g== 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2024-04-22T19:50:51.975 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQB8vyZmbVqUKhAAZM3ylDJexnDof+Z+h2/oyg== 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCPvyZmYf46DRAA5u97eq/MeR/8jBHBwqTayA== 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCSvyZmuD4SDRAATGz271MGOiD3b/fQdkq9cQ== 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCVvyZmCLSTDBAADoj65t5eLUjXuPexlNdj4w== 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCYvyZmI4LbDBAA7Exec6UF2PZ2mUw7M7e9Gg== 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQB8vyZmcxzdLRAAjcyQavtE1gchAEHMcFSDjA== 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout:[client.1] 2024-04-22T19:50:51.976 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQB8vyZmcI+0MRAAJq6aItZgJNwL5TVIxnjyzQ== 2024-04-22T19:50:51.977 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T19:50:51.988 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T19:50:52.080 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.084 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.145 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.147 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.209 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.211 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.273 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.275 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.336 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T19:50:52.339 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-22T19:50:52.399 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.401 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.463 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.465 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.526 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.528 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.591 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-22T19:50:52.653 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T19:50:52.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T19:50:52.720 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T19:50:52.723 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-22T19:50:52.787 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-22T19:50:52.787 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-22T19:50:52.822 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T19:50:52.928 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-22T19:50:52.964 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-22T19:50:53.041 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T19:50:53.150 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-22T19:50:53.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-22T19:50:53.223 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-22T19:50:53.330 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-22T19:50:53.366 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:53.369 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-22T19:50:53.433 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-22T19:50:53.433 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-22T19:50:53.433 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-22T19:50:53.437 INFO:tasks.ceph.mon.a:Started 2024-04-22T19:50:53.437 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-22T19:50:53.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-22T19:50:53.439 INFO:tasks.ceph.mon.c:Started 2024-04-22T19:50:53.439 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-22T19:50:53.439 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-22T19:50:53.476 INFO:tasks.ceph.mon.b:Started 2024-04-22T19:50:53.476 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-22T19:50:53.476 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-22T19:50:53.476 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-22T19:50:53.478 INFO:tasks.ceph.mgr.y:Started 2024-04-22T19:50:53.478 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-22T19:50:53.479 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-22T19:50:53.481 INFO:tasks.ceph.mgr.x:Started 2024-04-22T19:50:53.481 INFO:tasks.ceph:Setting crush tunables to default 2024-04-22T19:50:53.481 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd crush tunables default 2024-04-22T19:50:58.840 INFO:teuthology.orchestra.run.smithi136.stderr:adjusted tunables profile to default 2024-04-22T19:50:58.852 INFO:tasks.ceph:check_enable_crimson: False 2024-04-22T19:50:58.852 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-22T19:50:58.853 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:58.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-22T19:50:58.880 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:50:58.880+0000 7f0d32e21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T19:50:58.890 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:58.890 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-22T19:50:58.966 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:58.966 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-22T19:50:59.029 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:50:59.029+0000 7f0d32e21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T19:50:59.043 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:50:59.043 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-22T19:50:59.082 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:50:59.081+0000 7f0d32e21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T19:50:59.119 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:59.119 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-22T19:50:59.156 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:59.156 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-22T19:50:59.233 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:59.233 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-22T19:50:59.313 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-04-22T19:50:59.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-22T19:50:59.362 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:50:59.362+0000 7f0d32e21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T19:50:59.390 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 8c300310-767d-4fee-84ea-37d38622ac19 0 2024-04-22T19:50:59.410 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:50:59.410+0000 7f0d32e21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T19:50:59.608 INFO:teuthology.orchestra.run.smithi164.stdout:0 2024-04-22T19:50:59.620 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new d15e0912-05cd-4fea-bae1-79d478c1ec69 1 2024-04-22T19:50:59.667 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:50:59.666+0000 7f3fba54f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-22T19:50:59.813 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:50:59.813+0000 7f3fba54f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-22T19:50:59.820 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-04-22T19:50:59.831 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 4b191e79-cb0c-4d90-a66d-7bd1e50b7d18 2 2024-04-22T19:50:59.867 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:50:59.866+0000 7f3fba54f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-22T19:51:00.045 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-04-22T19:51:00.057 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 1062e5ff-d3f9-4d3a-9d13-37c8f2f9a90c 3 2024-04-22T19:51:00.128 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:00.128+0000 7f3fba54f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-22T19:51:00.147 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:00.146+0000 7f0d32e21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T19:51:00.173 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:00.173+0000 7f3fba54f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-22T19:51:00.239 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:00.239+0000 7f0d32e21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T19:51:00.257 INFO:teuthology.orchestra.run.smithi164.stdout:3 2024-04-22T19:51:00.268 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 85b19339-9132-4b74-af11-bf2838de4cba 4 2024-04-22T19:51:00.285 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:00.285+0000 7f0d32e21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T19:51:00.460 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:00.459+0000 7f0d32e21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T19:51:00.466 INFO:teuthology.orchestra.run.smithi164.stdout:4 2024-04-22T19:51:00.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new ca31b1f5-b528-4f5e-85e0-2c38fa53f7ba 5 2024-04-22T19:51:00.515 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:00.514+0000 7f0d32e21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T19:51:00.571 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:00.570+0000 7f0d32e21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T19:51:00.689 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:00.689+0000 7f0d32e21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T19:51:00.713 INFO:teuthology.orchestra.run.smithi164.stdout:5 2024-04-22T19:51:00.725 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new d2a9730f-bc66-45f1-8b7f-5e982423c8d5 6 2024-04-22T19:51:00.867 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:00.867+0000 7f3fba54f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-22T19:51:00.933 INFO:teuthology.orchestra.run.smithi164.stdout:6 2024-04-22T19:51:00.945 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 845c3a0b-1d2c-4750-9cc2-cabb32ded81e 7 2024-04-22T19:51:00.954 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:00.953+0000 7f3fba54f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-22T19:51:00.995 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:00.995+0000 7f3fba54f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-22T19:51:01.104 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:01.103+0000 7f0d32e21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T19:51:01.142 INFO:teuthology.orchestra.run.smithi164.stdout:7 2024-04-22T19:51:01.153 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-22T19:51:01.153 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-22T19:51:01.156 INFO:tasks.ceph.osd.0:Started 2024-04-22T19:51:01.156 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-22T19:51:01.156 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-22T19:51:01.158 INFO:tasks.ceph.osd.1:Started 2024-04-22T19:51:01.158 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-22T19:51:01.158 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-22T19:51:01.160 INFO:tasks.ceph.osd.2:Started 2024-04-22T19:51:01.160 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-22T19:51:01.161 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-22T19:51:01.161 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:01.160+0000 7f3fba54f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-22T19:51:01.163 INFO:tasks.ceph.osd.3:Started 2024-04-22T19:51:01.163 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-22T19:51:01.163 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-22T19:51:01.165 INFO:tasks.ceph.osd.4:Started 2024-04-22T19:51:01.166 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-22T19:51:01.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-22T19:51:01.168 INFO:tasks.ceph.osd.5:Started 2024-04-22T19:51:01.168 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-22T19:51:01.168 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-22T19:51:01.171 INFO:tasks.ceph.osd.6:Started 2024-04-22T19:51:01.171 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-22T19:51:01.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-22T19:51:01.173 INFO:tasks.ceph.osd.7:Started 2024-04-22T19:51:01.173 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T19:51:01.237 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:01.236+0000 7f3fba54f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-22T19:51:01.303 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:01.303+0000 7f3fba54f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-22T19:51:01.386 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:01.386 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":10,"fsid":"b2f95cfa-7b26-4a63-b13e-7105b011d9c4","created":"2024-04-22T19:50:58.750498+0000","modified":"2024-04-22T19:51:01.138291+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T19:51:01.138291+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c300310-767d-4fee-84ea-37d38622ac19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d15e0912-05cd-4fea-bae1-79d478c1ec69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4b191e79-cb0c-4d90-a66d-7bd1e50b7d18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1062e5ff-d3f9-4d3a-9d13-37c8f2f9a90c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85b19339-9132-4b74-af11-bf2838de4cba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ca31b1f5-b528-4f5e-85e0-2c38fa53f7ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d2a9730f-bc66-45f1-8b7f-5e982423c8d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"845c3a0b-1d2c-4750-9cc2-cabb32ded81e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T19:51:01.394 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:01.394+0000 7f0d32e21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T19:51:01.398 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-22T19:51:01.398 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-22T19:51:01.422 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:01.422+0000 7f3fba54f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-22T19:51:01.445 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:01.445+0000 7f0d32e21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T19:51:01.563 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:01.563+0000 7f0d32e21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T19:51:01.773 INFO:tasks.ceph.osd.0.smithi136.stderr:2024-04-22T19:51:01.773+0000 7fa21572f640 -1 Falling back to public interface 2024-04-22T19:51:01.785 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-04-22T19:51:01.784+0000 7f9de8817640 -1 Falling back to public interface 2024-04-22T19:51:01.787 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-22T19:51:01.787+0000 7fb9a5e63640 -1 Falling back to public interface 2024-04-22T19:51:01.792 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-04-22T19:51:01.792+0000 7f7d65211640 -1 Falling back to public interface 2024-04-22T19:51:01.794 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-04-22T19:51:01.793+0000 7f5dc38b1640 -1 Falling back to public interface 2024-04-22T19:51:01.805 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:01.805+0000 7f3fba54f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-22T19:51:01.807 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-04-22T19:51:01.807+0000 7faa6589a640 -1 Falling back to public interface 2024-04-22T19:51:01.808 INFO:tasks.ceph.osd.3.smithi136.stderr:2024-04-22T19:51:01.808+0000 7f5482e1c640 -1 Falling back to public interface 2024-04-22T19:51:01.814 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-04-22T19:51:01.813+0000 7f0151518640 -1 Falling back to public interface 2024-04-22T19:51:01.847 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:01.847+0000 7f0d32e21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T19:51:01.894 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:01.894+0000 7f0d32e21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T19:51:02.050 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:02.049+0000 7f0d32e21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T19:51:02.061 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.061+0000 7f3fba54f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-22T19:51:02.115 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.115+0000 7f3fba54f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-22T19:51:02.186 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:02.186+0000 7f0d32e21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T19:51:02.204 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T19:51:02.231 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.231+0000 7f3fba54f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-22T19:51:02.234 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:02.234+0000 7f0d32e21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T19:51:02.303 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:02.302+0000 7f0d32e21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T19:51:02.351 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:02.349+0000 7f0d32e21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T19:51:02.377 INFO:teuthology.misc.health.smithi136.stdout: 2024-04-22T19:51:02.377 INFO:teuthology.misc.health.smithi136.stdout:{"epoch":10,"fsid":"b2f95cfa-7b26-4a63-b13e-7105b011d9c4","created":"2024-04-22T19:50:58.750498+0000","modified":"2024-04-22T19:51:01.138291+0000","last_up_change":"0.000000","last_in_change":"2024-04-22T19:51:01.138291+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c300310-767d-4fee-84ea-37d38622ac19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d15e0912-05cd-4fea-bae1-79d478c1ec69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4b191e79-cb0c-4d90-a66d-7bd1e50b7d18","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1062e5ff-d3f9-4d3a-9d13-37c8f2f9a90c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85b19339-9132-4b74-af11-bf2838de4cba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ca31b1f5-b528-4f5e-85e0-2c38fa53f7ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d2a9730f-bc66-45f1-8b7f-5e982423c8d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"845c3a0b-1d2c-4750-9cc2-cabb32ded81e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T19:51:02.387 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-22T19:51:02.516 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.516+0000 7f3fba54f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-22T19:51:02.563 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.563+0000 7f3fba54f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-22T19:51:02.721 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.721+0000 7f3fba54f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-22T19:51:02.824 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:02.823+0000 7f0d32e21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T19:51:02.857 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.857+0000 7f3fba54f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-22T19:51:02.907 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.907+0000 7f3fba54f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-22T19:51:02.941 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:02.940+0000 7f0d32e21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T19:51:02.983 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:02.982+0000 7f3fba54f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-22T19:51:03.028 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:03.027+0000 7f3fba54f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-22T19:51:03.501 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:03.501+0000 7f3fba54f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-22T19:51:03.617 INFO:tasks.ceph.mgr.y.smithi136.stderr:2024-04-22T19:51:03.617+0000 7f3fba54f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-22T19:51:03.958 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:03.957+0000 7f0cd3781640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-22T19:51:03.958 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-04-22T19:51:03.958+0000 7f0cd3781640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-22T19:51:05.242 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-04-22T19:51:05.242+0000 7f9de8817640 -1 osd.7 0 log_to_monitors true 2024-04-22T19:51:05.285 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-04-22T19:51:05.284+0000 7f5dc38b1640 -1 osd.4 0 log_to_monitors true 2024-04-22T19:51:05.299 INFO:tasks.ceph.osd.0.smithi136.stderr:2024-04-22T19:51:05.299+0000 7fa21572f640 -1 osd.0 0 log_to_monitors true 2024-04-22T19:51:05.308 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-04-22T19:51:05.308+0000 7faa6589a640 -1 osd.6 0 log_to_monitors true 2024-04-22T19:51:05.311 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-04-22T19:51:05.311+0000 7f0151518640 -1 osd.5 0 log_to_monitors true 2024-04-22T19:51:05.351 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-22T19:51:05.351+0000 7fb9a5e63640 -1 osd.2 0 log_to_monitors true 2024-04-22T19:51:05.358 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-04-22T19:51:05.358+0000 7f7d65211640 -1 osd.1 0 log_to_monitors true 2024-04-22T19:51:05.371 INFO:tasks.ceph.osd.3.smithi136.stderr:2024-04-22T19:51:05.371+0000 7f5482e1c640 -1 osd.3 0 log_to_monitors true 2024-04-22T19:51:08.413 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-04-22T19:51:08.412+0000 7f7d6138a640 -1 osd.1 0 waiting for initial osdmap 2024-04-22T19:51:08.420 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-04-22T19:51:08.420+0000 7f7d58996640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T19:51:08.458 INFO:tasks.ceph.osd.0.smithi136.stderr:2024-04-22T19:51:08.458+0000 7fa2116c3640 -1 osd.0 0 waiting for initial osdmap 2024-04-22T19:51:08.473 INFO:tasks.ceph.osd.0.smithi136.stderr:2024-04-22T19:51:08.473+0000 7fa2094e3640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T19:51:08.474 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-22T19:51:08.474+0000 7fb9a1df7640 -1 osd.2 0 waiting for initial osdmap 2024-04-22T19:51:08.486 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-22T19:51:08.486+0000 7fb999c17640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T19:51:08.493 INFO:tasks.ceph.osd.3.smithi136.stderr:2024-04-22T19:51:08.493+0000 7f547e75c640 -1 osd.3 0 waiting for initial osdmap 2024-04-22T19:51:08.513 INFO:tasks.ceph.osd.3.smithi136.stderr:2024-04-22T19:51:08.513+0000 7f547657c640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T19:51:08.949 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-04-22T19:51:08.949+0000 7faa6182e640 -1 osd.6 0 waiting for initial osdmap 2024-04-22T19:51:08.962 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-04-22T19:51:08.962+0000 7faa5964e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T19:51:08.978 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-04-22T19:51:08.977+0000 7f9de40e8640 -1 osd.7 0 waiting for initial osdmap 2024-04-22T19:51:08.992 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-04-22T19:51:08.992+0000 7f9ddbf08640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T19:51:09.002 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-04-22T19:51:09.002+0000 7f014d4ac640 -1 osd.5 0 waiting for initial osdmap 2024-04-22T19:51:09.011 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-04-22T19:51:09.011+0000 7f01452cc640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T19:51:09.069 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-04-22T19:51:09.069+0000 7f5dbf845640 -1 osd.4 0 waiting for initial osdmap 2024-04-22T19:51:09.079 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-04-22T19:51:09.079+0000 7f5db7665640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-22T19:51:09.194 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-22T19:51:09.493 INFO:teuthology.misc.health.smithi136.stdout: 2024-04-22T19:51:09.493 INFO:teuthology.misc.health.smithi136.stdout:{"epoch":13,"fsid":"b2f95cfa-7b26-4a63-b13e-7105b011d9c4","created":"2024-04-22T19:50:58.750498+0000","modified":"2024-04-22T19:51:09.418727+0000","last_up_change":"2024-04-22T19:51:09.418727+0000","last_in_change":"2024-04-22T19:51:01.138291+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8c300310-767d-4fee-84ea-37d38622ac19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6801","nonce":2747506017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6803","nonce":2747506017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6807","nonce":2747506017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6805","nonce":2747506017}]},"public_addr":"172.21.15.136:6801/2747506017","cluster_addr":"172.21.15.136:6803/2747506017","heartbeat_back_addr":"172.21.15.136:6807/2747506017","heartbeat_front_addr":"172.21.15.136:6805/2747506017","state":["exists","up"]},{"osd":1,"uuid":"d15e0912-05cd-4fea-bae1-79d478c1ec69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6817","nonce":3821859466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6819","nonce":3821859466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6823","nonce":3821859466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6821","nonce":3821859466}]},"public_addr":"172.21.15.136:6817/3821859466","cluster_addr":"172.21.15.136:6819/3821859466","heartbeat_back_addr":"172.21.15.136:6823/3821859466","heartbeat_front_addr":"172.21.15.136:6821/3821859466","state":["exists","up"]},{"osd":2,"uuid":"4b191e79-cb0c-4d90-a66d-7bd1e50b7d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6809","nonce":253893352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6811","nonce":253893352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6815","nonce":253893352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6813","nonce":253893352}]},"public_addr":"172.21.15.136:6809/253893352","cluster_addr":"172.21.15.136:6811/253893352","heartbeat_back_addr":"172.21.15.136:6815/253893352","heartbeat_front_addr":"172.21.15.136:6813/253893352","state":["exists","up"]},{"osd":3,"uuid":"1062e5ff-d3f9-4d3a-9d13-37c8f2f9a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6825","nonce":3414382223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6827","nonce":3414382223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6831","nonce":3414382223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6829","nonce":3414382223}]},"public_addr":"172.21.15.136:6825/3414382223","cluster_addr":"172.21.15.136:6827/3414382223","heartbeat_back_addr":"172.21.15.136:6831/3414382223","heartbeat_front_addr":"172.21.15.136:6829/3414382223","state":["exists","up"]},{"osd":4,"uuid":"85b19339-9132-4b74-af11-bf2838de4cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6809","nonce":3291801237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6811","nonce":3291801237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6815","nonce":3291801237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6813","nonce":3291801237}]},"public_addr":"172.21.15.164:6809/3291801237","cluster_addr":"172.21.15.164:6811/3291801237","heartbeat_back_addr":"172.21.15.164:6815/3291801237","heartbeat_front_addr":"172.21.15.164:6813/3291801237","state":["exists","up"]},{"osd":5,"uuid":"ca31b1f5-b528-4f5e-85e0-2c38fa53f7ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6825","nonce":100873265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6827","nonce":100873265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6831","nonce":100873265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6829","nonce":100873265}]},"public_addr":"172.21.15.164:6825/100873265","cluster_addr":"172.21.15.164:6827/100873265","heartbeat_back_addr":"172.21.15.164:6831/100873265","heartbeat_front_addr":"172.21.15.164:6829/100873265","state":["exists","up"]},{"osd":6,"uuid":"d2a9730f-bc66-45f1-8b7f-5e982423c8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6817","nonce":2410758924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6819","nonce":2410758924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6823","nonce":2410758924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6821","nonce":2410758924}]},"public_addr":"172.21.15.164:6817/2410758924","cluster_addr":"172.21.15.164:6819/2410758924","heartbeat_back_addr":"172.21.15.164:6823/2410758924","heartbeat_front_addr":"172.21.15.164:6821/2410758924","state":["exists","up"]},{"osd":7,"uuid":"845c3a0b-1d2c-4750-9cc2-cabb32ded81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6801","nonce":4018161608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6803","nonce":4018161608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6807","nonce":4018161608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6805","nonce":4018161608}]},"public_addr":"172.21.15.164:6801/4018161608","cluster_addr":"172.21.15.164:6803/4018161608","heartbeat_back_addr":"172.21.15.164:6807/4018161608","heartbeat_front_addr":"172.21.15.164:6805/4018161608","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T19:51:09.504 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-22T19:51:09.505 INFO:tasks.ceph:Creating RBD pool 2024-04-22T19:51:09.505 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-22T19:51:10.422 INFO:teuthology.orchestra.run.smithi136.stderr:pool 'rbd' created 2024-04-22T19:51:10.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-22T19:51:11.432 INFO:teuthology.orchestra.run.smithi136.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-22T19:51:11.447 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-22T19:51:11.448 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-22T19:51:11.448 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-22T19:51:11.448 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-22T19:51:11.449 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-22T19:51:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:11.810 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":4,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":3034844420},{"type":"v1","addr":"172.21.15.164:6833","nonce":3034844420}]},"active_addr":"172.21.15.164:6833/3034844420","active_change":"2024-04-22T19:51:02.943952+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":3814620392}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":2806813323}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.164:0","nonce":1447927684}]}]} 2024-04-22T19:51:11.814 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-22T19:51:11.814 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-22T19:51:11.814 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T19:51:12.109 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:12.110 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":15,"fsid":"b2f95cfa-7b26-4a63-b13e-7105b011d9c4","created":"2024-04-22T19:50:58.750498+0000","modified":"2024-04-22T19:51:11.423705+0000","last_up_change":"2024-04-22T19:51:09.418727+0000","last_in_change":"2024-04-22T19:51:01.138291+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-22T19:51:09.800954+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-22T19:51:10.966181+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c300310-767d-4fee-84ea-37d38622ac19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6801","nonce":2747506017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6803","nonce":2747506017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6807","nonce":2747506017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6805","nonce":2747506017}]},"public_addr":"172.21.15.136:6801/2747506017","cluster_addr":"172.21.15.136:6803/2747506017","heartbeat_back_addr":"172.21.15.136:6807/2747506017","heartbeat_front_addr":"172.21.15.136:6805/2747506017","state":["exists","up"]},{"osd":1,"uuid":"d15e0912-05cd-4fea-bae1-79d478c1ec69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6817","nonce":3821859466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6819","nonce":3821859466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6823","nonce":3821859466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6821","nonce":3821859466}]},"public_addr":"172.21.15.136:6817/3821859466","cluster_addr":"172.21.15.136:6819/3821859466","heartbeat_back_addr":"172.21.15.136:6823/3821859466","heartbeat_front_addr":"172.21.15.136:6821/3821859466","state":["exists","up"]},{"osd":2,"uuid":"4b191e79-cb0c-4d90-a66d-7bd1e50b7d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6809","nonce":253893352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6811","nonce":253893352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6815","nonce":253893352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6813","nonce":253893352}]},"public_addr":"172.21.15.136:6809/253893352","cluster_addr":"172.21.15.136:6811/253893352","heartbeat_back_addr":"172.21.15.136:6815/253893352","heartbeat_front_addr":"172.21.15.136:6813/253893352","state":["exists","up"]},{"osd":3,"uuid":"1062e5ff-d3f9-4d3a-9d13-37c8f2f9a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6825","nonce":3414382223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6827","nonce":3414382223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6831","nonce":3414382223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6829","nonce":3414382223}]},"public_addr":"172.21.15.136:6825/3414382223","cluster_addr":"172.21.15.136:6827/3414382223","heartbeat_back_addr":"172.21.15.136:6831/3414382223","heartbeat_front_addr":"172.21.15.136:6829/3414382223","state":["exists","up"]},{"osd":4,"uuid":"85b19339-9132-4b74-af11-bf2838de4cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6809","nonce":3291801237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6811","nonce":3291801237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6815","nonce":3291801237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6813","nonce":3291801237}]},"public_addr":"172.21.15.164:6809/3291801237","cluster_addr":"172.21.15.164:6811/3291801237","heartbeat_back_addr":"172.21.15.164:6815/3291801237","heartbeat_front_addr":"172.21.15.164:6813/3291801237","state":["exists","up"]},{"osd":5,"uuid":"ca31b1f5-b528-4f5e-85e0-2c38fa53f7ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6825","nonce":100873265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6827","nonce":100873265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6831","nonce":100873265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6829","nonce":100873265}]},"public_addr":"172.21.15.164:6825/100873265","cluster_addr":"172.21.15.164:6827/100873265","heartbeat_back_addr":"172.21.15.164:6831/100873265","heartbeat_front_addr":"172.21.15.164:6829/100873265","state":["exists","up"]},{"osd":6,"uuid":"d2a9730f-bc66-45f1-8b7f-5e982423c8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6817","nonce":2410758924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6819","nonce":2410758924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6823","nonce":2410758924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6821","nonce":2410758924}]},"public_addr":"172.21.15.164:6817/2410758924","cluster_addr":"172.21.15.164:6819/2410758924","heartbeat_back_addr":"172.21.15.164:6823/2410758924","heartbeat_front_addr":"172.21.15.164:6821/2410758924","state":["exists","up"]},{"osd":7,"uuid":"845c3a0b-1d2c-4750-9cc2-cabb32ded81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6801","nonce":4018161608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6803","nonce":4018161608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6807","nonce":4018161608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6805","nonce":4018161608}]},"public_addr":"172.21.15.164:6801/4018161608","cluster_addr":"172.21.15.164:6803/4018161608","heartbeat_back_addr":"172.21.15.164:6807/4018161608","heartbeat_front_addr":"172.21.15.164:6805/4018161608","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.275552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.318182+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.342147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.342562+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.315740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.359563+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.278972+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.255612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T19:51:12.122 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-22T19:51:12.122 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T19:51:12.421 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:12.422 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":15,"fsid":"b2f95cfa-7b26-4a63-b13e-7105b011d9c4","created":"2024-04-22T19:50:58.750498+0000","modified":"2024-04-22T19:51:11.423705+0000","last_up_change":"2024-04-22T19:51:09.418727+0000","last_in_change":"2024-04-22T19:51:01.138291+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-22T19:51:09.800954+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-22T19:51:10.966181+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c300310-767d-4fee-84ea-37d38622ac19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6801","nonce":2747506017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6803","nonce":2747506017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6807","nonce":2747506017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6805","nonce":2747506017}]},"public_addr":"172.21.15.136:6801/2747506017","cluster_addr":"172.21.15.136:6803/2747506017","heartbeat_back_addr":"172.21.15.136:6807/2747506017","heartbeat_front_addr":"172.21.15.136:6805/2747506017","state":["exists","up"]},{"osd":1,"uuid":"d15e0912-05cd-4fea-bae1-79d478c1ec69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6817","nonce":3821859466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6819","nonce":3821859466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6823","nonce":3821859466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6821","nonce":3821859466}]},"public_addr":"172.21.15.136:6817/3821859466","cluster_addr":"172.21.15.136:6819/3821859466","heartbeat_back_addr":"172.21.15.136:6823/3821859466","heartbeat_front_addr":"172.21.15.136:6821/3821859466","state":["exists","up"]},{"osd":2,"uuid":"4b191e79-cb0c-4d90-a66d-7bd1e50b7d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6809","nonce":253893352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6811","nonce":253893352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6815","nonce":253893352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6813","nonce":253893352}]},"public_addr":"172.21.15.136:6809/253893352","cluster_addr":"172.21.15.136:6811/253893352","heartbeat_back_addr":"172.21.15.136:6815/253893352","heartbeat_front_addr":"172.21.15.136:6813/253893352","state":["exists","up"]},{"osd":3,"uuid":"1062e5ff-d3f9-4d3a-9d13-37c8f2f9a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6825","nonce":3414382223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6827","nonce":3414382223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6831","nonce":3414382223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6829","nonce":3414382223}]},"public_addr":"172.21.15.136:6825/3414382223","cluster_addr":"172.21.15.136:6827/3414382223","heartbeat_back_addr":"172.21.15.136:6831/3414382223","heartbeat_front_addr":"172.21.15.136:6829/3414382223","state":["exists","up"]},{"osd":4,"uuid":"85b19339-9132-4b74-af11-bf2838de4cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6809","nonce":3291801237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6811","nonce":3291801237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6815","nonce":3291801237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6813","nonce":3291801237}]},"public_addr":"172.21.15.164:6809/3291801237","cluster_addr":"172.21.15.164:6811/3291801237","heartbeat_back_addr":"172.21.15.164:6815/3291801237","heartbeat_front_addr":"172.21.15.164:6813/3291801237","state":["exists","up"]},{"osd":5,"uuid":"ca31b1f5-b528-4f5e-85e0-2c38fa53f7ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6825","nonce":100873265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6827","nonce":100873265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6831","nonce":100873265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6829","nonce":100873265}]},"public_addr":"172.21.15.164:6825/100873265","cluster_addr":"172.21.15.164:6827/100873265","heartbeat_back_addr":"172.21.15.164:6831/100873265","heartbeat_front_addr":"172.21.15.164:6829/100873265","state":["exists","up"]},{"osd":6,"uuid":"d2a9730f-bc66-45f1-8b7f-5e982423c8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6817","nonce":2410758924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6819","nonce":2410758924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6823","nonce":2410758924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6821","nonce":2410758924}]},"public_addr":"172.21.15.164:6817/2410758924","cluster_addr":"172.21.15.164:6819/2410758924","heartbeat_back_addr":"172.21.15.164:6823/2410758924","heartbeat_front_addr":"172.21.15.164:6821/2410758924","state":["exists","up"]},{"osd":7,"uuid":"845c3a0b-1d2c-4750-9cc2-cabb32ded81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6801","nonce":4018161608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6803","nonce":4018161608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6807","nonce":4018161608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6805","nonce":4018161608}]},"public_addr":"172.21.15.164:6801/4018161608","cluster_addr":"172.21.15.164:6803/4018161608","heartbeat_back_addr":"172.21.15.164:6807/4018161608","heartbeat_front_addr":"172.21.15.164:6805/4018161608","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.275552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.318182+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.342147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.342562+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.315740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.359563+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.278972+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.255612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T19:51:12.436 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-22T19:51:12.437 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-22T19:51:12.437 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-22T19:51:12.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-22T19:51:12.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-22T19:51:12.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-22T19:51:12.439 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-22T19:51:12.439 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-22T19:51:12.702 INFO:teuthology.orchestra.run.smithi136.stdout:55834574852 2024-04-22T19:51:12.702 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T19:51:12.704 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:12.704 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:12.705 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T19:51:12.705 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T19:51:12.706 INFO:teuthology.orchestra.run.smithi136.stdout:55834574853 2024-04-22T19:51:12.707 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T19:51:12.708 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:12.708 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:12.709 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T19:51:12.709 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T19:51:12.710 INFO:teuthology.orchestra.run.smithi136.stdout:55834574852 2024-04-22T19:51:12.711 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T19:51:12.712 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:12.712 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T19:51:13.299 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:13.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2024-04-22T19:51:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:55834574852 2024-04-22T19:51:13.406 INFO:teuthology.orchestra.run.smithi136.stdout:55834574850 2024-04-22T19:51:13.407 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:13.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-04-22T19:51:13.414 INFO:teuthology.orchestra.run.smithi136.stdout:55834574850 2024-04-22T19:51:13.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-04-22T19:51:13.428 INFO:teuthology.orchestra.run.smithi136.stdout:55834574850 2024-04-22T19:51:13.430 INFO:teuthology.orchestra.run.smithi136.stdout:55834574850 2024-04-22T19:51:13.431 INFO:teuthology.orchestra.run.smithi136.stdout:55834574850 2024-04-22T19:51:13.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2024-04-22T19:51:13.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2024-04-22T19:51:13.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-04-22T19:51:13.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-04-22T19:51:13.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-04-22T19:51:14.334 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-22T19:51:14.413 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-22T19:51:14.427 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-22T19:51:14.434 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-22T19:51:14.436 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-22T19:51:14.452 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-22T19:51:14.452 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-22T19:51:14.453 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-22T19:51:14.843 INFO:teuthology.orchestra.run.smithi136.stdout:55834574852 2024-04-22T19:51:14.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-04-22T19:51:14.865 DEBUG:teuthology.parallel:result is None 2024-04-22T19:51:14.928 INFO:teuthology.orchestra.run.smithi136.stdout:55834574853 2024-04-22T19:51:14.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-04-22T19:51:14.942 DEBUG:teuthology.parallel:result is None 2024-04-22T19:51:15.051 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:15.058 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:15.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2024-04-22T19:51:15.068 INFO:teuthology.orchestra.run.smithi136.stdout:55834574852 2024-04-22T19:51:15.068 DEBUG:teuthology.parallel:result is None 2024-04-22T19:51:15.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-04-22T19:51:15.075 DEBUG:teuthology.parallel:result is None 2024-04-22T19:51:15.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-04-22T19:51:15.082 DEBUG:teuthology.parallel:result is None 2024-04-22T19:51:15.085 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:15.093 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:15.093 INFO:teuthology.orchestra.run.smithi136.stdout:55834574851 2024-04-22T19:51:15.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2024-04-22T19:51:15.099 DEBUG:teuthology.parallel:result is None 2024-04-22T19:51:15.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2024-04-22T19:51:15.108 DEBUG:teuthology.parallel:result is None 2024-04-22T19:51:15.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2024-04-22T19:51:15.108 DEBUG:teuthology.parallel:result is None 2024-04-22T19:51:15.108 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T19:51:15.109 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T19:51:15.443 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:15.444 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T19:51:15.458 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-22T19:51:14.953114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217296,"kb_used_data":1120,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757424,"statfs":{"total":773094113280,"available":772871602176,"internally_reserved":0,"allocated":1146880,"data_stored":328152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:11.438788+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T19:51:11.438788+0000","last_peered":"2024-04-22T19:51:11.438788+0000","last_clean":"2024-04-22T19:51:11.438788+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T19:51:11.438788+0000","last_undegraded":"2024-04-22T19:51:11.438788+0000","last_fullsized":"2024-04-22T19:51:11.438788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:09:17.045448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.252097+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T19:51:12.252097+0000","last_peered":"2024-04-22T19:51:12.252097+0000","last_clean":"2024-04-22T19:51:12.252097+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T19:51:12.252097+0000","last_undegraded":"2024-04-22T19:51:12.252097+0000","last_fullsized":"2024-04-22T19:51:12.252097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:06:09.873678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.436656+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T19:51:12.436656+0000","last_peered":"2024-04-22T19:51:12.436656+0000","last_clean":"2024-04-22T19:51:12.436656+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T19:51:12.436656+0000","last_undegraded":"2024-04-22T19:51:12.436656+0000","last_fullsized":"2024-04-22T19:51:12.436656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438145+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T19:51:12.438145+0000","last_peered":"2024-04-22T19:51:12.438145+0000","last_clean":"2024-04-22T19:51:12.438145+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T19:51:12.438145+0000","last_undegraded":"2024-04-22T19:51:12.438145+0000","last_fullsized":"2024-04-22T19:51:12.438145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3,"reported_epoch":16,"state":"creating+activating","last_fresh":"2024-04-22T19:51:12.438563+0000","last_change":"2024-04-22T19:51:12.438563+0000","last_active":"2024-04-22T19:51:11.423705+0000","last_peered":"2024-04-22T19:51:11.423705+0000","last_clean":"2024-04-22T19:51:11.423705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-22T19:51:12.438563+0000","last_undegraded":"2024-04-22T19:51:12.438563+0000","last_fullsized":"2024-04-22T19:51:12.438563+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:11.439758+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T19:51:11.439758+0000","last_peered":"2024-04-22T19:51:11.439758+0000","last_clean":"2024-04-22T19:51:11.439758+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T19:51:11.439758+0000","last_undegraded":"2024-04-22T19:51:11.439758+0000","last_fullsized":"2024-04-22T19:51:11.439758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:26:33.396119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438404+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T19:51:12.438404+0000","last_peered":"2024-04-22T19:51:12.438404+0000","last_clean":"2024-04-22T19:51:12.438404+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T19:51:12.438404+0000","last_undegraded":"2024-04-22T19:51:12.438404+0000","last_fullsized":"2024-04-22T19:51:12.438404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.251855+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T19:51:12.251855+0000","last_peered":"2024-04-22T19:51:12.251855+0000","last_clean":"2024-04-22T19:51:12.251855+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T19:51:12.251855+0000","last_undegraded":"2024-04-22T19:51:12.251855+0000","last_fullsized":"2024-04-22T19:51:12.251855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:09.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438410+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T19:51:12.438410+0000","last_peered":"2024-04-22T19:51:12.438410+0000","last_clean":"2024-04-22T19:51:12.438410+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T19:51:12.438410+0000","last_undegraded":"2024-04-22T19:51:12.438410+0000","last_fullsized":"2024-04-22T19:51:12.438410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T19:51:15.459 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T19:51:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:15.756 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T19:51:15.769 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-22T19:51:14.953114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217296,"kb_used_data":1120,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757424,"statfs":{"total":773094113280,"available":772871602176,"internally_reserved":0,"allocated":1146880,"data_stored":328152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:11.438788+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T19:51:11.438788+0000","last_peered":"2024-04-22T19:51:11.438788+0000","last_clean":"2024-04-22T19:51:11.438788+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T19:51:11.438788+0000","last_undegraded":"2024-04-22T19:51:11.438788+0000","last_fullsized":"2024-04-22T19:51:11.438788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:09:17.045448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.252097+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T19:51:12.252097+0000","last_peered":"2024-04-22T19:51:12.252097+0000","last_clean":"2024-04-22T19:51:12.252097+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T19:51:12.252097+0000","last_undegraded":"2024-04-22T19:51:12.252097+0000","last_fullsized":"2024-04-22T19:51:12.252097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:06:09.873678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.436656+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T19:51:12.436656+0000","last_peered":"2024-04-22T19:51:12.436656+0000","last_clean":"2024-04-22T19:51:12.436656+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T19:51:12.436656+0000","last_undegraded":"2024-04-22T19:51:12.436656+0000","last_fullsized":"2024-04-22T19:51:12.436656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438145+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T19:51:12.438145+0000","last_peered":"2024-04-22T19:51:12.438145+0000","last_clean":"2024-04-22T19:51:12.438145+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T19:51:12.438145+0000","last_undegraded":"2024-04-22T19:51:12.438145+0000","last_fullsized":"2024-04-22T19:51:12.438145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3,"reported_epoch":16,"state":"creating+activating","last_fresh":"2024-04-22T19:51:12.438563+0000","last_change":"2024-04-22T19:51:12.438563+0000","last_active":"2024-04-22T19:51:11.423705+0000","last_peered":"2024-04-22T19:51:11.423705+0000","last_clean":"2024-04-22T19:51:11.423705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-22T19:51:12.438563+0000","last_undegraded":"2024-04-22T19:51:12.438563+0000","last_fullsized":"2024-04-22T19:51:12.438563+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:11.439758+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T19:51:11.439758+0000","last_peered":"2024-04-22T19:51:11.439758+0000","last_clean":"2024-04-22T19:51:11.439758+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T19:51:11.439758+0000","last_undegraded":"2024-04-22T19:51:11.439758+0000","last_fullsized":"2024-04-22T19:51:11.439758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:26:33.396119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438404+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T19:51:12.438404+0000","last_peered":"2024-04-22T19:51:12.438404+0000","last_clean":"2024-04-22T19:51:12.438404+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T19:51:12.438404+0000","last_undegraded":"2024-04-22T19:51:12.438404+0000","last_fullsized":"2024-04-22T19:51:12.438404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.251855+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T19:51:12.251855+0000","last_peered":"2024-04-22T19:51:12.251855+0000","last_clean":"2024-04-22T19:51:12.251855+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T19:51:12.251855+0000","last_undegraded":"2024-04-22T19:51:12.251855+0000","last_fullsized":"2024-04-22T19:51:12.251855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:09.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438410+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T19:51:12.438410+0000","last_peered":"2024-04-22T19:51:12.438410+0000","last_clean":"2024-04-22T19:51:12.438410+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T19:51:12.438410+0000","last_undegraded":"2024-04-22T19:51:12.438410+0000","last_fullsized":"2024-04-22T19:51:12.438410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T19:51:15.770 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T19:51:16.064 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:16.064 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T19:51:16.076 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-22T19:51:14.953114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217296,"kb_used_data":1120,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757424,"statfs":{"total":773094113280,"available":772871602176,"internally_reserved":0,"allocated":1146880,"data_stored":328152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:11.438788+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T19:51:11.438788+0000","last_peered":"2024-04-22T19:51:11.438788+0000","last_clean":"2024-04-22T19:51:11.438788+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T19:51:11.438788+0000","last_undegraded":"2024-04-22T19:51:11.438788+0000","last_fullsized":"2024-04-22T19:51:11.438788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:09:17.045448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.252097+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T19:51:12.252097+0000","last_peered":"2024-04-22T19:51:12.252097+0000","last_clean":"2024-04-22T19:51:12.252097+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T19:51:12.252097+0000","last_undegraded":"2024-04-22T19:51:12.252097+0000","last_fullsized":"2024-04-22T19:51:12.252097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:06:09.873678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.436656+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T19:51:12.436656+0000","last_peered":"2024-04-22T19:51:12.436656+0000","last_clean":"2024-04-22T19:51:12.436656+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T19:51:12.436656+0000","last_undegraded":"2024-04-22T19:51:12.436656+0000","last_fullsized":"2024-04-22T19:51:12.436656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438145+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T19:51:12.438145+0000","last_peered":"2024-04-22T19:51:12.438145+0000","last_clean":"2024-04-22T19:51:12.438145+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T19:51:12.438145+0000","last_undegraded":"2024-04-22T19:51:12.438145+0000","last_fullsized":"2024-04-22T19:51:12.438145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3,"reported_epoch":16,"state":"creating+activating","last_fresh":"2024-04-22T19:51:12.438563+0000","last_change":"2024-04-22T19:51:12.438563+0000","last_active":"2024-04-22T19:51:11.423705+0000","last_peered":"2024-04-22T19:51:11.423705+0000","last_clean":"2024-04-22T19:51:11.423705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-22T19:51:12.438563+0000","last_undegraded":"2024-04-22T19:51:12.438563+0000","last_fullsized":"2024-04-22T19:51:12.438563+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:11.439758+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T19:51:11.439758+0000","last_peered":"2024-04-22T19:51:11.439758+0000","last_clean":"2024-04-22T19:51:11.439758+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T19:51:11.439758+0000","last_undegraded":"2024-04-22T19:51:11.439758+0000","last_fullsized":"2024-04-22T19:51:11.439758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:26:33.396119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438404+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T19:51:12.438404+0000","last_peered":"2024-04-22T19:51:12.438404+0000","last_clean":"2024-04-22T19:51:12.438404+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T19:51:12.438404+0000","last_undegraded":"2024-04-22T19:51:12.438404+0000","last_fullsized":"2024-04-22T19:51:12.438404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.251855+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T19:51:12.251855+0000","last_peered":"2024-04-22T19:51:12.251855+0000","last_clean":"2024-04-22T19:51:12.251855+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T19:51:12.251855+0000","last_undegraded":"2024-04-22T19:51:12.251855+0000","last_fullsized":"2024-04-22T19:51:12.251855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:09.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438410+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T19:51:12.438410+0000","last_peered":"2024-04-22T19:51:12.438410+0000","last_clean":"2024-04-22T19:51:12.438410+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T19:51:12.438410+0000","last_undegraded":"2024-04-22T19:51:12.438410+0000","last_fullsized":"2024-04-22T19:51:12.438410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T19:51:16.077 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-04-22T19:51:16.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 16, 'state': 'creating+activating', 'last_fresh': '2024-04-22T19:51:12.438563+0000', 'last_change': '2024-04-22T19:51:12.438563+0000', 'last_active': '2024-04-22T19:51:11.423705+0000', 'last_peered': '2024-04-22T19:51:11.423705+0000', 'last_clean': '2024-04-22T19:51:11.423705+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-22T19:51:12.438563+0000', 'last_undegraded': '2024-04-22T19:51:12.438563+0000', 'last_fullsized': '2024-04-22T19:51:12.438563+0000', 'mapping_epoch': 15, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 15, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-22T19:51:11.423705+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-22T19:51:11.423705+0000', 'last_clean_scrub_stamp': '2024-04-22T19:51:11.423705+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': ['7', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-22T19:51:16.078 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-22T19:51:16.437 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:16.437 INFO:teuthology.orchestra.run.smithi136.stdout:{"fsid":"b2f95cfa-7b26-4a63-b13e-7105b011d9c4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":17,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":17,"num_osds":8,"num_up_osds":8,"osd_up_since":1713815469,"num_in_osds":8,"osd_in_since":1713815461,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":8},{"state_name":"creating+activating","count":1}],"num_pgs":9,"num_pools":2,"num_objects":0,"data_bytes":0,"bytes_used":222511104,"bytes_avail":772871602176,"bytes_total":773094113280,"inactive_pgs_ratio":0.1111111119389534},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-04-22T19:50:58.751819+0000","services":{}},"progress_events":{}} 2024-04-22T19:51:16.450 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-04-22T19:51:16.451 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T19:51:16.749 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T19:51:16.762 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-22T19:51:14.953114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217296,"kb_used_data":1120,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757424,"statfs":{"total":773094113280,"available":772871602176,"internally_reserved":0,"allocated":1146880,"data_stored":328152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:11.438788+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T19:51:11.438788+0000","last_peered":"2024-04-22T19:51:11.438788+0000","last_clean":"2024-04-22T19:51:11.438788+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T19:51:11.438788+0000","last_undegraded":"2024-04-22T19:51:11.438788+0000","last_fullsized":"2024-04-22T19:51:11.438788+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:09:17.045448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.252097+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T19:51:12.252097+0000","last_peered":"2024-04-22T19:51:12.252097+0000","last_clean":"2024-04-22T19:51:12.252097+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T19:51:12.252097+0000","last_undegraded":"2024-04-22T19:51:12.252097+0000","last_fullsized":"2024-04-22T19:51:12.252097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:06:09.873678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.436656+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T19:51:12.436656+0000","last_peered":"2024-04-22T19:51:12.436656+0000","last_clean":"2024-04-22T19:51:12.436656+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T19:51:12.436656+0000","last_undegraded":"2024-04-22T19:51:12.436656+0000","last_fullsized":"2024-04-22T19:51:12.436656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438145+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T19:51:12.438145+0000","last_peered":"2024-04-22T19:51:12.438145+0000","last_clean":"2024-04-22T19:51:12.438145+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T19:51:12.438145+0000","last_undegraded":"2024-04-22T19:51:12.438145+0000","last_fullsized":"2024-04-22T19:51:12.438145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":3,"reported_epoch":16,"state":"creating+activating","last_fresh":"2024-04-22T19:51:12.438563+0000","last_change":"2024-04-22T19:51:12.438563+0000","last_active":"2024-04-22T19:51:11.423705+0000","last_peered":"2024-04-22T19:51:11.423705+0000","last_clean":"2024-04-22T19:51:11.423705+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-22T19:51:12.438563+0000","last_undegraded":"2024-04-22T19:51:12.438563+0000","last_fullsized":"2024-04-22T19:51:12.438563+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:11.439758+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T19:51:11.439758+0000","last_peered":"2024-04-22T19:51:11.439758+0000","last_clean":"2024-04-22T19:51:11.439758+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T19:51:11.439758+0000","last_undegraded":"2024-04-22T19:51:11.439758+0000","last_fullsized":"2024-04-22T19:51:11.439758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:26:33.396119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438404+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T19:51:12.438404+0000","last_peered":"2024-04-22T19:51:12.438404+0000","last_clean":"2024-04-22T19:51:12.438404+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T19:51:12.438404+0000","last_undegraded":"2024-04-22T19:51:12.438404+0000","last_fullsized":"2024-04-22T19:51:12.438404+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.251855+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T19:51:12.251855+0000","last_peered":"2024-04-22T19:51:12.251855+0000","last_clean":"2024-04-22T19:51:12.251855+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T19:51:12.251855+0000","last_undegraded":"2024-04-22T19:51:12.251855+0000","last_fullsized":"2024-04-22T19:51:12.251855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:09.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-22T19:51:12.438410+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T19:51:12.438410+0000","last_peered":"2024-04-22T19:51:12.438410+0000","last_clean":"2024-04-22T19:51:12.438410+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T19:51:12.438410+0000","last_undegraded":"2024-04-22T19:51:12.438410+0000","last_fullsized":"2024-04-22T19:51:12.438410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T19:51:19.763 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T19:51:20.071 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:20.071 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T19:51:20.083 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-04-22T19:51:18.953946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218768,"kb_used_data":2460,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755952,"statfs":{"total":773094113280,"available":772870094848,"internally_reserved":0,"allocated":2519040,"data_stored":1598356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000423"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:13.513464+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T19:51:13.513464+0000","last_peered":"2024-04-22T19:51:13.513464+0000","last_clean":"2024-04-22T19:51:13.513464+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T19:51:13.513464+0000","last_undegraded":"2024-04-22T19:51:13.513464+0000","last_fullsized":"2024-04-22T19:51:13.513464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:23.800559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:13.513457+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T19:51:13.513457+0000","last_peered":"2024-04-22T19:51:13.513457+0000","last_clean":"2024-04-22T19:51:13.513457+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T19:51:13.513457+0000","last_undegraded":"2024-04-22T19:51:13.513457+0000","last_fullsized":"2024-04-22T19:51:13.513457+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:01.430732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:13.442908+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T19:51:13.442908+0000","last_peered":"2024-04-22T19:51:13.442908+0000","last_clean":"2024-04-22T19:51:13.442908+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T19:51:13.442908+0000","last_undegraded":"2024-04-22T19:51:13.442908+0000","last_fullsized":"2024-04-22T19:51:13.442908+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:13.446797+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T19:51:13.446797+0000","last_peered":"2024-04-22T19:51:13.446797+0000","last_clean":"2024-04-22T19:51:13.446797+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T19:51:13.446797+0000","last_undegraded":"2024-04-22T19:51:13.446797+0000","last_fullsized":"2024-04-22T19:51:13.446797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:14.176592+0000","last_change":"2024-04-22T19:51:12.892628+0000","last_active":"2024-04-22T19:51:14.176592+0000","last_peered":"2024-04-22T19:51:14.176592+0000","last_clean":"2024-04-22T19:51:14.176592+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T19:51:14.176592+0000","last_undegraded":"2024-04-22T19:51:14.176592+0000","last_fullsized":"2024-04-22T19:51:14.176592+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:34.018008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:14.162871+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T19:51:14.162871+0000","last_peered":"2024-04-22T19:51:14.162871+0000","last_clean":"2024-04-22T19:51:14.162871+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T19:51:14.162871+0000","last_undegraded":"2024-04-22T19:51:14.162871+0000","last_fullsized":"2024-04-22T19:51:14.162871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:22.264706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:14.176565+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T19:51:14.176565+0000","last_peered":"2024-04-22T19:51:14.176565+0000","last_clean":"2024-04-22T19:51:14.176565+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T19:51:14.176565+0000","last_undegraded":"2024-04-22T19:51:14.176565+0000","last_fullsized":"2024-04-22T19:51:14.176565+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:14.115882+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T19:51:14.115882+0000","last_peered":"2024-04-22T19:51:14.115882+0000","last_clean":"2024-04-22T19:51:14.115882+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T19:51:14.115882+0000","last_undegraded":"2024-04-22T19:51:14.115882+0000","last_fullsized":"2024-04-22T19:51:14.115882+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:20:10.880037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-22T19:51:14.176553+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T19:51:14.176553+0000","last_peered":"2024-04-22T19:51:14.176553+0000","last_clean":"2024-04-22T19:51:14.176553+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T19:51:14.176553+0000","last_undegraded":"2024-04-22T19:51:14.176553+0000","last_fullsized":"2024-04-22T19:51:14.176553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":749568,"data_stored":637513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":52925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":52925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":52925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":52925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":52925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":761856,"data_stored":643293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":52925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T19:51:20.084 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-22T19:51:20.084 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-22T19:51:20.084 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-22T19:51:20.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-22T19:51:20.420 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T19:51:20.420 INFO:teuthology.orchestra.run.smithi136.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-22T19:51:20.433 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-22T19:51:20.433 INFO:teuthology.run_tasks:Running task openssl_keys... 2024-04-22T19:51:20.443 DEBUG:teuthology.task:Applying overrides for task openssl_keys: {'rgw.client.0': {'ca': 'root', 'client': 'client.0', 'embed-key': True}, 'root': {'client': 'client.0', 'cn': 'teuthology', 'install': ['client.0'], 'key-type': 'rsa:4096'}} 2024-04-22T19:51:20.444 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p /home/ubuntu/cephtest/ca 2024-04-22T19:51:20.471 DEBUG:teuthology.orchestra.run.smithi136:> openssl req -x509 -nodes -newkey rsa:4096 -keyout /home/ubuntu/cephtest/ca/root.key -days 365 -out /home/ubuntu/cephtest/ca/root.crt -subj /CN=teuthology 2024-04-22T19:51:21.375 INFO:teuthology.orchestra.run.smithi136.stderr:..+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+..+.+........+.+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*............+.........+...+............................+...........+......+.........+..................+.+....................+.........+......+.........+.+.....+.......+..+..................+.............+.....+................+...+.....+.......+......+..+...+....+.........+........................+..+................+........+.........+............+....+.....+....+...+........+......+...+.........................+...+..+......+...................+.....+.+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-04-22T19:51:21.546 INFO:teuthology.orchestra.run.smithi136.stderr:.+.+..+......+.......+............+..+.+.....+...+.......+.....+.........+...+.+..+.............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+..........+..+......+.+....................+.........+......+....+.....+.......+.........+........+.+......+......+..+.+..+............+......+.+...............+......+........+.+...+...+.....+...+................+...+.....+.......+...+......+..+...+.+...+...........+......+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..........................+...................+..+................+.................+............+.+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-04-22T19:51:21.561 INFO:teuthology.orchestra.run.smithi136.stderr:----- 2024-04-22T19:51:21.568 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /home/ubuntu/cephtest/ca/root.crt /usr/share/pki/ca-trust-source/anchors/root.crt 2024-04-22T19:51:21.605 DEBUG:teuthology.orchestra.run.smithi136:> sudo update-ca-trust 2024-04-22T19:51:22.458 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p /home/ubuntu/cephtest/ca 2024-04-22T19:51:22.485 DEBUG:teuthology.orchestra.run.smithi136:> openssl req -nodes -newkey rsa:2048 -keyout /home/ubuntu/cephtest/ca/rgw.client.0.key -out /home/ubuntu/cephtest/ca/rgw.client.0.csr -subj /CN=smithi136.front.sepia.ceph.com 2024-04-22T19:51:22.625 INFO:teuthology.orchestra.run.smithi136.stderr:......+..........+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+...............+..+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-04-22T19:51:22.709 INFO:teuthology.orchestra.run.smithi136.stderr:.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+.....+...+.........+....+.....+....+.....+...+.............+............+..+.+...............+............+...+..+..........+..+......+.+...+..+......+.............+..+....+...+......+..+...+.+........+.+..+...+.+.....+...+............+..........+......+..+............+.+...............+.....+............+...+...+.........+.......+........................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+.....+......+...+.+..+.......+......+..+....+........+.+..+...+......+.+...+.....+.......+..+.+............+..+...+....+.....+.+............+.....+............+.............+.....+...+.......+............+...........+....+.....+.........+......+....+............+...+......+.........+..+...............+............+.........................+..+.+..+...+..........+.....+...+.........+............+....+..+.......+...........+.+...+...........+......+....+......+...........+...+......+.+............+..+...+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2024-04-22T19:51:22.712 INFO:teuthology.orchestra.run.smithi136.stderr:----- 2024-04-22T19:51:22.715 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:51:22.715 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-04-22T19:51:22.783 DEBUG:teuthology.orchestra.run.smithi136:> openssl x509 -req -in /home/ubuntu/cephtest/ca/rgw.client.0.csr -CA /home/ubuntu/cephtest/ca/root.crt -CAkey /home/ubuntu/cephtest/ca/root.key -CAcreateserial -out /home/ubuntu/cephtest/ca/rgw.client.0.crt -days 365 -sha256 -extfile /home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-04-22T19:51:22.873 INFO:teuthology.orchestra.run.smithi136.stderr:Certificate request self-signature ok 2024-04-22T19:51:22.873 INFO:teuthology.orchestra.run.smithi136.stderr:subject=CN = smithi136.front.sepia.ceph.com 2024-04-22T19:51:22.882 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/ca/rgw.client.0.csr /home/ubuntu/cephtest/ca/root.srl /home/ubuntu/cephtest/ca/rgw.client.0.ext 2024-04-22T19:51:22.910 DEBUG:teuthology.orchestra.run.smithi136:> openssl verify -CAfile /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/rgw.client.0.crt 2024-04-22T19:51:22.979 INFO:teuthology.orchestra.run.smithi136.stdout:/home/ubuntu/cephtest/ca/rgw.client.0.crt: OK 2024-04-22T19:51:22.980 DEBUG:teuthology.orchestra.run.smithi136:> cat /home/ubuntu/cephtest/ca/rgw.client.0.key >> /home/ubuntu/cephtest/ca/rgw.client.0.crt 2024-04-22T19:51:23.048 INFO:teuthology.run_tasks:Running task rgw... 2024-04-22T19:51:23.063 DEBUG:tasks.rgw:config is {'client.0': {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2024-04-22T19:51:23.063 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-04-22T19:51:23.063 INFO:tasks.rgw:Creating data pools 2024-04-22T19:51:23.063 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-04-22T19:51:23.063 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph osd erasure-code-profile set client.0 k=2 m=1 crush-failure-domain=osd --cluster ceph 2024-04-22T19:51:23.478 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-04-22T19:51:25.471 INFO:teuthology.orchestra.run.smithi136.stderr:pool 'default.rgw.buckets.data' created 2024-04-22T19:51:25.495 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-04-22T19:51:26.469 INFO:teuthology.orchestra.run.smithi136.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-22T19:51:26.499 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-04-22T19:51:27.494 INFO:teuthology.orchestra.run.smithi136.stderr:pool 'default.rgw.buckets.index' created 2024-04-22T19:51:27.514 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-04-22T19:51:28.492 INFO:teuthology.orchestra.run.smithi136.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-22T19:51:28.524 DEBUG:tasks.rgw:Pools created 2024-04-22T19:51:28.524 INFO:tasks.rgw:Configuring compression type = random 2024-04-22T19:51:28.524 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-04-22T19:51:28.524 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-04-22T19:51:28.524 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-04-22T19:51:28.608 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:28.608 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:30.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:30.962+0000 7f8863a4c640 0 -- 172.21.15.136:0/2181296023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55e885058710 msgr2=0x55e885058b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:51:30.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:30.966+0000 7f885b24b640 0 -- 172.21.15.136:0/3917416402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55e885064260 msgr2=0x55e88506ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2024-04-22T19:51:31.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.167+0000 7f885b24b640 0 --2- 172.21.15.136:0/3917416402 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55e885064260 0x55e88506ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-04-22T19:51:31.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.172+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.174+0000 7f8865f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:31.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.174+0000 7f8865f46800 20 realm 2024-04-22T19:51:31.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.174+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.174+0000 7f8865f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:31.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.174+0000 7f8865f46800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:51:31.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.174+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.174+0000 7f8865f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:31.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.174+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.175+0000 7f8865f46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:51:31.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.175+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.176+0000 7f8865f46800 20 rados_obj.operate() r=0 bl.length=993 2024-04-22T19:51:31.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.176+0000 7f8865f46800 20 searching for the correct realm 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.190+0000 7f8865f46800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.190+0000 7f8865f46800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.190+0000 7f8865f46800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.190+0000 7f8865f46800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.190+0000 7f8865f46800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.190+0000 7f8865f46800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.190+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.191+0000 7f8865f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:31.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.191+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.191+0000 7f8865f46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.191+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 20 rados_obj.operate() r=0 bl.length=436 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 20 zone default found 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 4 Realm: () 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 20 zonegroup default 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:31.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:31.192+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:32.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:32.535+0000 7f8865f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:32.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:32.535+0000 7f8865f46800 20 rados->read ofs=0 len=0 2024-04-22T19:51:32.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:32.539+0000 7f8865f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:32.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:32.539+0000 7f8865f46800 20 started sync module instance, tier type = 2024-04-22T19:51:32.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:32.539+0000 7f8865f46800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:51:34.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.582+0000 7f8865f46800 20 add_watcher() i=0 2024-04-22T19:51:34.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.582+0000 7f8865f46800 20 add_watcher() i=1 2024-04-22T19:51:34.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.582+0000 7f8865f46800 20 add_watcher() i=2 2024-04-22T19:51:34.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.582+0000 7f8865f46800 20 add_watcher() i=3 2024-04-22T19:51:34.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.582+0000 7f8865f46800 20 add_watcher() i=4 2024-04-22T19:51:34.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.582+0000 7f8865f46800 20 add_watcher() i=5 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 20 add_watcher() i=6 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 20 add_watcher() i=7 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 2 all 8 watchers are set, enabling cache 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.584+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-04-22T19:51:34.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-04-22T19:51:34.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-04-22T19:51:34.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-04-22T19:51:34.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.585+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.586+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.586+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.586+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.586+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.586+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.586+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2024-04-22T19:51:34.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2024-04-22T19:51:34.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.587+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.588+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2024-04-22T19:51:34.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2024-04-22T19:51:34.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2024-04-22T19:51:34.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2024-04-22T19:51:34.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.589+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2024-04-22T19:51:34.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.590+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2024-04-22T19:51:34.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.591+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2024-04-22T19:51:34.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2024-04-22T19:51:34.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2024-04-22T19:51:34.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2024-04-22T19:51:34.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.592+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2024-04-22T19:51:34.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2024-04-22T19:51:34.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2024-04-22T19:51:34.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2024-04-22T19:51:34.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2024-04-22T19:51:34.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2024-04-22T19:51:34.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.593+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2024-04-22T19:51:34.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.594+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.595+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2024-04-22T19:51:34.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.596+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2024-04-22T19:51:34.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.597+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2024-04-22T19:51:34.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.598+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2024-04-22T19:51:34.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2024-04-22T19:51:34.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2024-04-22T19:51:34.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2024-04-22T19:51:34.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.599+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2024-04-22T19:51:34.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.600+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.601+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2024-04-22T19:51:34.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.602+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2024-04-22T19:51:34.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.603+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2024-04-22T19:51:34.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.604+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2024-04-22T19:51:34.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.605+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.606+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2024-04-22T19:51:34.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.607+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2024-04-22T19:51:34.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.608+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2024-04-22T19:51:34.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.609+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.610+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.611+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2024-04-22T19:51:34.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.612+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2024-04-22T19:51:34.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.613+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2024-04-22T19:51:34.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.614+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.615+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2024-04-22T19:51:34.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.616+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-04-22T19:51:34.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.618+0000 7f8865f46800 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-04-22T19:51:34.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.624+0000 7f8711fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:51:34.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.624+0000 7f8865f46800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:34.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:34.624+0000 7f8865f46800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:37.557 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.558+0000 7f8865f46800 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-04-22T19:51:37.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.558+0000 7f8865f46800 5 note: GC not initialized 2024-04-22T19:51:37.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.558+0000 7f8865f46800 20 init_complete bucket index max shards: 1 2024-04-22T19:51:37.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.558+0000 7f8710fe1640 20 reqs_thread_entry: start 2024-04-22T19:51:37.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.558+0000 7f8865f46800 20 Filter name: none 2024-04-22T19:51:37.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.559+0000 7f867a7fc640 20 reqs_thread_entry: start 2024-04-22T19:51:37.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.585+0000 7f8711fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:51:37.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.588+0000 7f8865f46800 20 remove_watcher() i=0 2024-04-22T19:51:37.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.588+0000 7f8865f46800 2 removed watcher, disabling cache 2024-04-22T19:51:37.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.589+0000 7f8865f46800 20 remove_watcher() i=1 2024-04-22T19:51:37.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.592+0000 7f8865f46800 20 remove_watcher() i=2 2024-04-22T19:51:37.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.596+0000 7f8865f46800 20 remove_watcher() i=3 2024-04-22T19:51:37.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.597+0000 7f8865f46800 20 remove_watcher() i=4 2024-04-22T19:51:37.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.598+0000 7f8865f46800 20 remove_watcher() i=5 2024-04-22T19:51:37.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.599+0000 7f8865f46800 20 remove_watcher() i=6 2024-04-22T19:51:37.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.601+0000 7f8865f46800 20 remove_watcher() i=7 2024-04-22T19:51:37.626 INFO:teuthology.orchestra.run.smithi136.stdout:[] 2024-04-22T19:51:37.626 DEBUG:tasks.util.rgw: json result: [] 2024-04-22T19:51:37.626 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-22T19:51:37.626 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-04-22T19:51:37.627 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-04-22T19:51:37.688 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:37.689 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:37.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.713+0000 7f806840f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:37.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.713+0000 7f806840f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:37.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:37.716+0000 7f7ec1ffb640 20 reqs_thread_entry: start 2024-04-22T19:51:38.134 INFO:teuthology.orchestra.run.smithi136.stdout:{"id":"19263407-9cd1-4c73-8de4-71c0779d4a94","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-22T19:51:38.134 DEBUG:tasks.util.rgw: json result: {'id': '19263407-9cd1-4c73-8de4-71c0779d4a94', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-22T19:51:38.134 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-04-22T19:51:38.134 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-04-22T19:51:38.134 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-04-22T19:51:38.188 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:38.189 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:38.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.219+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.221+0000 7fdf8d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.221+0000 7fdf8d11f800 20 realm 2024-04-22T19:51:38.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.221+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.221+0000 7fdf8d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.221+0000 7fdf8d11f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:51:38.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.221+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.221+0000 7fdf8d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.221+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.222+0000 7fdf8d11f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:51:38.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.222+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.223+0000 7fdf8d11f800 20 rados_obj.operate() r=0 bl.length=1009 2024-04-22T19:51:38.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.223+0000 7fdf8d11f800 20 searching for the correct realm 2024-04-22T19:51:38.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.225+0000 7fdf895ef640 0 -- 172.21.15.136:0/1162769663 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fdf8407b070 msgr2=0x7fdf840d8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:51:38.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.431+0000 7fdf83fff640 0 -- 172.21.15.136:0/1162769663 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fdf840c5c10 msgr2=0x7fdf84070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:51:38.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:51:38.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:51:38.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:51:38.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:51:38.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:51:38.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.638+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 20 zone default found 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 4 Realm: () 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 20 zonegroup default 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.639+0000 7fdf8d11f800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.640+0000 7fdf8d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.640+0000 7fdf8d11f800 20 started sync module instance, tier type = 2024-04-22T19:51:38.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.640+0000 7fdf8d11f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:51:38.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.654+0000 7fdf8d11f800 20 add_watcher() i=0 2024-04-22T19:51:38.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.654+0000 7fdf8d11f800 20 add_watcher() i=1 2024-04-22T19:51:38.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.654+0000 7fdf8d11f800 20 add_watcher() i=2 2024-04-22T19:51:38.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.654+0000 7fdf8d11f800 20 add_watcher() i=3 2024-04-22T19:51:38.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.654+0000 7fdf8d11f800 20 add_watcher() i=4 2024-04-22T19:51:38.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.654+0000 7fdf8d11f800 20 add_watcher() i=5 2024-04-22T19:51:38.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.657+0000 7fdf8d11f800 20 add_watcher() i=6 2024-04-22T19:51:38.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.657+0000 7fdf8d11f800 20 add_watcher() i=7 2024-04-22T19:51:38.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.657+0000 7fdf8d11f800 2 all 8 watchers are set, enabling cache 2024-04-22T19:51:38.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.659+0000 7fde3efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:51:38.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.659+0000 7fdf8d11f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:38.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.659+0000 7fdf8d11f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:38.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.659+0000 7fdf8d11f800 5 note: GC not initialized 2024-04-22T19:51:38.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.659+0000 7fde3dfe3640 20 reqs_thread_entry: start 2024-04-22T19:51:38.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.659+0000 7fdf8d11f800 20 init_complete bucket index max shards: 1 2024-04-22T19:51:38.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.659+0000 7fdf8d11f800 20 Filter name: none 2024-04-22T19:51:38.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.659+0000 7fddaf7fe640 20 reqs_thread_entry: start 2024-04-22T19:51:38.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.670+0000 7fde3efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:51:38.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.672+0000 7fdf8d11f800 20 remove_watcher() i=0 2024-04-22T19:51:38.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.672+0000 7fdf8d11f800 2 removed watcher, disabling cache 2024-04-22T19:51:38.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.674+0000 7fdf8d11f800 20 remove_watcher() i=1 2024-04-22T19:51:38.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.675+0000 7fdf8d11f800 20 remove_watcher() i=2 2024-04-22T19:51:38.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.676+0000 7fdf8d11f800 20 remove_watcher() i=3 2024-04-22T19:51:38.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.677+0000 7fdf8d11f800 20 remove_watcher() i=4 2024-04-22T19:51:38.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.678+0000 7fdf8d11f800 20 remove_watcher() i=5 2024-04-22T19:51:38.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.679+0000 7fdf8d11f800 20 remove_watcher() i=6 2024-04-22T19:51:38.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.680+0000 7fdf8d11f800 20 remove_watcher() i=7 2024-04-22T19:51:38.689 INFO:teuthology.orchestra.run.smithi136.stdout:[] 2024-04-22T19:51:38.689 DEBUG:tasks.util.rgw: json result: [] 2024-04-22T19:51:38.689 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--placement-inline-data', 'false'] 2024-04-22T19:51:38.689 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--placement-inline-data', 'false'] 2024-04-22T19:51:38.690 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --placement-inline-data false 2024-04-22T19:51:38.749 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:38.749 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:38.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.772+0000 7f9c7ec3d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:38.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.772+0000 7f9c7ec3d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:38.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.776+0000 7f9ac27fc640 20 reqs_thread_entry: start 2024-04-22T19:51:38.790 INFO:teuthology.orchestra.run.smithi136.stdout:{"id":"19263407-9cd1-4c73-8de4-71c0779d4a94","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-22T19:51:38.790 DEBUG:tasks.util.rgw: json result: {'id': '19263407-9cd1-4c73-8de4-71c0779d4a94', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-22T19:51:38.790 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-04-22T19:51:38.790 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-04-22T19:51:38.790 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-04-22T19:51:38.843 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:38.843 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:38.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.874+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.875+0000 7f947e52d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.875+0000 7f947e52d800 20 realm 2024-04-22T19:51:38.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.875+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.876+0000 7f947e52d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.876+0000 7f947e52d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:51:38.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.876+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.876+0000 7f947e52d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:38.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.876+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.877+0000 7f947e52d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:51:38.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.877+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:38.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.877+0000 7f947e52d800 20 rados_obj.operate() r=0 bl.length=1009 2024-04-22T19:51:38.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.877+0000 7f947e52d800 20 searching for the correct realm 2024-04-22T19:51:38.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:38.879+0000 7f947a980640 0 -- 172.21.15.136:0/3409991564 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f9474018bb0 msgr2=0x7f947403eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:51:39.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.287+0000 7f947a980640 0 -- 172.21.15.136:0/3409991564 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f9474018bb0 msgr2=0x7f947403eac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.292+0000 7f947e52d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.292+0000 7f947e52d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.292+0000 7f947e52d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.292+0000 7f947e52d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.292+0000 7f947e52d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.292+0000 7f947e52d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.292+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:39.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:39.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 20 zone default found 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 4 Realm: () 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 20 zonegroup default 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.293+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.294+0000 7f947e52d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.294+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:39.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.294+0000 7f947e52d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:39.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.294+0000 7f947e52d800 20 rados->read ofs=0 len=0 2024-04-22T19:51:39.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.294+0000 7f947e52d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:51:39.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.294+0000 7f947e52d800 20 started sync module instance, tier type = 2024-04-22T19:51:39.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.294+0000 7f947e52d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:51:39.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.309+0000 7f947e52d800 20 add_watcher() i=0 2024-04-22T19:51:39.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.309+0000 7f947e52d800 20 add_watcher() i=1 2024-04-22T19:51:39.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.309+0000 7f947e52d800 20 add_watcher() i=2 2024-04-22T19:51:39.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.309+0000 7f947e52d800 20 add_watcher() i=3 2024-04-22T19:51:39.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.309+0000 7f947e52d800 20 add_watcher() i=4 2024-04-22T19:51:39.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.309+0000 7f947e52d800 20 add_watcher() i=5 2024-04-22T19:51:39.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.309+0000 7f947e52d800 20 add_watcher() i=6 2024-04-22T19:51:39.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.310+0000 7f947e52d800 20 add_watcher() i=7 2024-04-22T19:51:39.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.310+0000 7f947e52d800 2 all 8 watchers are set, enabling cache 2024-04-22T19:51:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.311+0000 7f9330fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:51:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.311+0000 7f947e52d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.311+0000 7f947e52d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.312+0000 7f947e52d800 5 note: GC not initialized 2024-04-22T19:51:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.312+0000 7f92ab7fe640 20 reqs_thread_entry: start 2024-04-22T19:51:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.312+0000 7f947e52d800 20 init_complete bucket index max shards: 1 2024-04-22T19:51:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.312+0000 7f947e52d800 20 Filter name: none 2024-04-22T19:51:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.312+0000 7f92a97fa640 20 reqs_thread_entry: start 2024-04-22T19:51:39.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.322+0000 7f9330fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:51:39.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.324+0000 7f947e52d800 20 remove_watcher() i=0 2024-04-22T19:51:39.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.324+0000 7f947e52d800 2 removed watcher, disabling cache 2024-04-22T19:51:39.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.326+0000 7f947e52d800 20 remove_watcher() i=1 2024-04-22T19:51:39.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.327+0000 7f947e52d800 20 remove_watcher() i=2 2024-04-22T19:51:39.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.328+0000 7f947e52d800 20 remove_watcher() i=3 2024-04-22T19:51:39.328 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.329+0000 7f947e52d800 20 remove_watcher() i=4 2024-04-22T19:51:39.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.330+0000 7f947e52d800 20 remove_watcher() i=5 2024-04-22T19:51:39.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.331+0000 7f947e52d800 20 remove_watcher() i=6 2024-04-22T19:51:39.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.332+0000 7f947e52d800 20 remove_watcher() i=7 2024-04-22T19:51:39.341 INFO:teuthology.orchestra.run.smithi136.stdout:[] 2024-04-22T19:51:39.341 DEBUG:tasks.util.rgw: json result: [] 2024-04-22T19:51:39.341 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2024-04-22T19:51:39.341 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-04-22T19:51:39.341 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-04-22T19:51:39.341 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2024-04-22T19:51:39.401 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:39.401 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:39.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.425+0000 7fcfe4213800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:39.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.425+0000 7fcfe4213800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.428+0000 7fce25ffb640 20 reqs_thread_entry: start 2024-04-22T19:51:39.441 INFO:teuthology.orchestra.run.smithi136.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2024-04-22T19:51:39.441 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2024-04-22T19:51:39.441 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-04-22T19:51:39.441 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-04-22T19:51:39.442 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2024-04-22T19:51:39.505 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:39.505 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:39.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.528+0000 7fb430b2c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:39.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.528+0000 7fb430b2c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:39.530 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.531+0000 7fb272ffd640 20 reqs_thread_entry: start 2024-04-22T19:51:39.544 INFO:teuthology.orchestra.run.smithi136.stdout:{"id":"19263407-9cd1-4c73-8de4-71c0779d4a94","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-22T19:51:39.545 DEBUG:tasks.util.rgw: json result: {'id': '19263407-9cd1-4c73-8de4-71c0779d4a94', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-22T19:51:39.545 INFO:tasks.rgw:Configuring storage class type = FROZEN 2024-04-22T19:51:39.545 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-04-22T19:51:39.545 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-04-22T19:51:39.545 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2024-04-22T19:51:39.605 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:39.605 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:39.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.628+0000 7fbc23898800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:39.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.628+0000 7fbc23898800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:39.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.632+0000 7fbae47e0640 20 reqs_thread_entry: start 2024-04-22T19:51:39.645 INFO:teuthology.orchestra.run.smithi136.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2024-04-22T19:51:39.646 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2024-04-22T19:51:39.646 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-04-22T19:51:39.646 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-04-22T19:51:39.646 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2024-04-22T19:51:39.708 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:51:39.709 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:51:39.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.731+0000 7f1c2923f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:51:39.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.731+0000 7f1c2923f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:51:39.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T19:51:39.734+0000 7f1ae97e2640 20 reqs_thread_entry: start 2024-04-22T19:51:39.749 INFO:teuthology.orchestra.run.smithi136.stdout:{"id":"19263407-9cd1-4c73-8de4-71c0779d4a94","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-22T19:51:39.749 DEBUG:tasks.util.rgw: json result: {'id': '19263407-9cd1-4c73-8de4-71c0779d4a94', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-22T19:51:39.749 INFO:tasks.rgw:Starting rgw... 2024-04-22T19:51:39.749 INFO:tasks.rgw:rgw client.0 config is {'ssl certificate': 'rgw.client.0', 'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2024-04-22T19:51:39.749 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-22T19:51:39.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo echo -n https://smithi136.front.sepia.ceph.com:443 | sudo tee /home/ubuntu/cephtest/url_file 2024-04-22T19:51:39.787 INFO:teuthology.orchestra.run.smithi136.stdout:https://smithi136.front.sepia.ceph.com:443 2024-04-22T19:51:39.787 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-04-22T19:51:39.864 DEBUG:teuthology.orchestra.run.smithi136:> sudo keytool -import -alias smithi136.front.sepia.ceph.com -keystore $(readlink -e $(dirname $(readlink -e $(which keytool)))/../lib/security/cacerts) -file /home/ubuntu/cephtest/ca/rgw.client.0.crt -storepass changeit 2024-04-22T19:51:40.045 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: use -cacerts option to access cacerts keystore 2024-04-22T19:51:40.184 INFO:teuthology.orchestra.run.smithi136.stderr:Certificate was added to keystore 2024-04-22T19:51:40.205 DEBUG:tasks.ceph_manager:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2024-04-22T19:51:40.206 INFO:tasks.rgw.client.0:Restarting daemon 2024-04-22T19:51:40.206 DEBUG:teuthology.orchestra.run.smithi136:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast ssl_certificate=/home/ubuntu/cephtest/ca/rgw.client.0.crt ssl_port=443' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-04-22T19:51:40.208 INFO:tasks.rgw.client.0:Started 2024-04-22T19:51:40.208 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on https://smithi136.front.sepia.ceph.com:443/ 2024-04-22T19:51:40.208 DEBUG:teuthology.orchestra.run.smithi136:> curl https://smithi136.front.sepia.ceph.com:443/ 2024-04-22T19:51:40.241 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-22T19:51:40.241 INFO:teuthology.orchestra.run.smithi136.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-22T19:51:40.241 INFO:teuthology.orchestra.run.smithi136.stderr: Dload Upload Total Spent Left Speed 2024-04-22T19:51:40.242 INFO:teuthology.orchestra.run.smithi136.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-22T19:51:40.242 INFO:teuthology.orchestra.run.smithi136.stderr:curl: (7) Failed to connect to smithi136.front.sepia.ceph.com port 443: Connection refused 2024-04-22T19:51:41.243 DEBUG:teuthology.orchestra.run.smithi136:> curl https://smithi136.front.sepia.ceph.com:443/ 2024-04-22T19:51:41.276 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-22T19:51:41.277 INFO:teuthology.orchestra.run.smithi136.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-22T19:51:41.277 INFO:teuthology.orchestra.run.smithi136.stderr: Dload Upload Total Spent Left Speed 2024-04-22T19:51:41.277 INFO:teuthology.orchestra.run.smithi136.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-22T19:51:41.277 INFO:teuthology.orchestra.run.smithi136.stderr:curl: (7) Failed to connect to smithi136.front.sepia.ceph.com port 443: Connection refused 2024-04-22T19:51:43.278 DEBUG:teuthology.orchestra.run.smithi136:> curl https://smithi136.front.sepia.ceph.com:443/ 2024-04-22T19:51:43.310 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-22T19:51:43.310 INFO:teuthology.orchestra.run.smithi136.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-22T19:51:43.310 INFO:teuthology.orchestra.run.smithi136.stderr: Dload Upload Total Spent Left Speed 2024-04-22T19:51:43.311 INFO:teuthology.orchestra.run.smithi136.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-22T19:51:43.311 INFO:teuthology.orchestra.run.smithi136.stderr:curl: (7) Failed to connect to smithi136.front.sepia.ceph.com port 443: Connection refused 2024-04-22T19:51:47.312 DEBUG:teuthology.orchestra.run.smithi136:> curl https://smithi136.front.sepia.ceph.com:443/ 2024-04-22T19:51:47.343 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-22T19:51:47.343 INFO:teuthology.orchestra.run.smithi136.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-22T19:51:47.343 INFO:teuthology.orchestra.run.smithi136.stderr: Dload Upload Total Spent Left Speed 2024-04-22T19:51:47.343 INFO:teuthology.orchestra.run.smithi136.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-22T19:51:47.344 INFO:teuthology.orchestra.run.smithi136.stderr:curl: (7) Failed to connect to smithi136.front.sepia.ceph.com port 443: Connection refused 2024-04-22T19:51:55.345 DEBUG:teuthology.orchestra.run.smithi136:> curl https://smithi136.front.sepia.ceph.com:443/ 2024-04-22T19:51:55.376 INFO:teuthology.orchestra.run.smithi136.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-22T19:51:55.376 INFO:teuthology.orchestra.run.smithi136.stderr: Dload Upload Total Spent Left Speed 2024-04-22T19:51:55.376 INFO:teuthology.orchestra.run.smithi136.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-22T19:51:55.376 INFO:teuthology.orchestra.run.smithi136.stderr:curl: (7) Failed to connect to smithi136.front.sepia.ceph.com port 443: Connection refused 2024-04-22T19:51:55.377 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-22T19:52:01.664 INFO:tasks.rgw.client.0.smithi136.stdout:2024-04-22T19:52:01.576+0000 a052100 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T19:52:02.576 INFO:tasks.rgw.client.0.smithi136.stdout:2024-04-22T19:52:02.280+0000 a052100 -1 ssl_private_key was not found: rgw/cert//default.key 2024-04-22T19:52:11.378 DEBUG:teuthology.orchestra.run.smithi136:> curl https://smithi136.front.sepia.ceph.com:443/ 2024-04-22T19:52:11.415 INFO:teuthology.orchestra.run.smithi136.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-22T19:52:11.415 INFO:teuthology.orchestra.run.smithi136.stderr: Dload Upload Total Spent Left Speed 2024-04-22T19:52:12.375 INFO:teuthology.orchestra.run.smithi136.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 194 0 --:--:-- --:--:-- --:--:-- 194 2024-04-22T19:52:12.377 INFO:teuthology.orchestra.run.smithi136.stdout:anonymous 2024-04-22T19:52:12.377 INFO:teuthology.run_tasks:Running task tox... 2024-04-22T19:52:12.389 INFO:tasks.tox:Deploying tox from pip... 2024-04-22T19:52:12.389 DEBUG:teuthology.orchestra.run.smithi136:> python3 -m venv /home/ubuntu/cephtest/tox-venv 2024-04-22T19:52:14.928 DEBUG:teuthology.orchestra.run.smithi136:> source /home/ubuntu/cephtest/tox-venv/bin/activate && pip install tox==3.15.0 2024-04-22T19:52:15.491 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting tox==3.15.0 2024-04-22T19:52:15.568 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading tox-3.15.0-py2.py3-none-any.whl (82 kB) 2024-04-22T19:52:15.708 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting toml>=0.9.4 2024-04-22T19:52:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-04-22T19:52:15.812 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting pluggy<1,>=0.12.0 2024-04-22T19:52:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) 2024-04-22T19:52:16.101 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 2024-04-22T19:52:16.120 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading virtualenv-20.25.3-py3-none-any.whl (3.9 MB) 2024-04-22T19:52:16.704 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting packaging>=14 2024-04-22T19:52:16.735 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading packaging-24.0-py3-none-any.whl (53 kB) 2024-04-22T19:52:16.847 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting py<2,>=1.4.17 2024-04-22T19:52:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading py-1.11.0-py2.py3-none-any.whl (98 kB) 2024-04-22T19:52:16.963 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting six<2,>=1.14.0 2024-04-22T19:52:16.980 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-22T19:52:17.087 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting filelock<4,>=3.0.0 2024-04-22T19:52:17.105 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading filelock-3.13.4-py3-none-any.whl (11 kB) 2024-04-22T19:52:17.235 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting distlib<1,>=0.3.7 2024-04-22T19:52:17.254 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB) 2024-04-22T19:52:17.390 INFO:teuthology.orchestra.run.smithi136.stdout:Collecting platformdirs<5,>=3.9.1 2024-04-22T19:52:17.414 INFO:teuthology.orchestra.run.smithi136.stdout: Downloading platformdirs-4.2.0-py3-none-any.whl (17 kB) 2024-04-22T19:52:17.525 INFO:teuthology.orchestra.run.smithi136.stdout:Installing collected packages: platformdirs, filelock, distlib, virtualenv, toml, six, py, pluggy, packaging, tox 2024-04-22T19:52:18.335 INFO:teuthology.orchestra.run.smithi136.stdout:Successfully installed distlib-0.3.8 filelock-3.13.4 packaging-24.0 platformdirs-4.2.0 pluggy-0.13.1 py-1.11.0 six-1.16.0 toml-0.10.2 tox-3.15.0 virtualenv-20.25.3 2024-04-22T19:52:18.566 INFO:teuthology.orchestra.run.smithi136.stderr:WARNING: You are using pip version 21.3.1; however, version 24.0 is available. 2024-04-22T19:52:18.566 INFO:teuthology.orchestra.run.smithi136.stderr:You should consider upgrading via the '/home/ubuntu/cephtest/tox-venv/bin/python3 -m pip install --upgrade pip' command. 2024-04-22T19:52:18.632 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T19:52:18.645 INFO:tasks.workunit:Pulling workunits from ref 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:52:18.645 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T19:52:18.645 DEBUG:teuthology.orchestra.run.smithi136:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T19:52:18.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T19:52:18.673 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-22T19:52:18.673 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T19:52:18.742 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-22T19:52:18.742 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-22T19:52:18.809 INFO:tasks.workunit:timeout=3h 2024-04-22T19:52:18.810 INFO:tasks.workunit:cleanup=True 2024-04-22T19:52:18.810 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:52:18.877 INFO:tasks.workunit.client.0.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T19:53:43.761 INFO:tasks.workunit.client.0.smithi136.stderr:Note: switching to '9ed8263edff7e87907bf24b054e5c7144c194dbe'. 2024-04-22T19:53:43.761 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr: git switch -c 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr:Or undo this operation with: 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:43.762 INFO:tasks.workunit.client.0.smithi136.stderr: git switch - 2024-04-22T19:53:43.763 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:43.763 INFO:tasks.workunit.client.0.smithi136.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T19:53:43.763 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:43.763 INFO:tasks.workunit.client.0.smithi136.stderr:HEAD is now at 9ed8263edff rgw: apply default quota config on account creation 2024-04-22T19:53:43.767 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T19:53:43.885 INFO:tasks.workunit.client.0.smithi136.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T19:53:43.886 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T19:53:43.887 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T19:53:44.423 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T19:53:44.480 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T19:53:44.519 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T19:53:44.521 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T19:53:44.521 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T19:53:44.559 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T19:53:44.562 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:53:44.562 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T19:53:44.590 INFO:tasks.workunit:Running workunits matching rgw/run-bucket-check.sh on client.0... 2024-04-22T19:53:44.592 INFO:tasks.workunit:Running workunit rgw/run-bucket-check.sh... 2024-04-22T19:53:44.592 DEBUG:teuthology.orchestra.run.smithi136:workunit test rgw/run-bucket-check.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-04-22T19:53:44.662 INFO:tasks.workunit.client.0.smithi136.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-04-22T19:53:44.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-22T19:53:44.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-22T19:53:46.927 INFO:tasks.workunit.client.0.smithi136.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ deactivate nondestructive 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n '' ']' 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n '' ']' 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ hash -r 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n '' ']' 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ unset VIRTUAL_ENV 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ export VIRTUAL_ENV 2024-04-22T19:53:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:++ _OLD_VIRTUAL_PATH=/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T19:53:46.929 INFO:tasks.workunit.client.0.smithi136.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T19:53:46.929 INFO:tasks.workunit.client.0.smithi136.stderr:++ export PATH 2024-04-22T19:53:46.929 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n '' ']' 2024-04-22T19:53:46.929 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -z '' ']' 2024-04-22T19:53:46.929 INFO:tasks.workunit.client.0.smithi136.stderr:++ _OLD_VIRTUAL_PS1= 2024-04-22T19:53:46.930 INFO:tasks.workunit.client.0.smithi136.stderr:++ PS1='(rgw) ' 2024-04-22T19:53:46.930 INFO:tasks.workunit.client.0.smithi136.stderr:++ export PS1 2024-04-22T19:53:46.930 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-22T19:53:46.930 INFO:tasks.workunit.client.0.smithi136.stderr:++ hash -r 2024-04-22T19:53:46.930 INFO:tasks.workunit.client.0.smithi136.stderr:+ pip install pip --upgrade 2024-04-22T19:53:47.158 INFO:tasks.workunit.client.0.smithi136.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.9/site-packages (21.3.1) 2024-04-22T19:53:47.373 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting pip 2024-04-22T19:53:47.449 INFO:tasks.workunit.client.0.smithi136.stdout: Downloading pip-24.0-py3-none-any.whl (2.1 MB) 2024-04-22T19:53:47.640 INFO:tasks.workunit.client.0.smithi136.stdout:Installing collected packages: pip 2024-04-22T19:53:47.641 INFO:tasks.workunit.client.0.smithi136.stdout: Attempting uninstall: pip 2024-04-22T19:53:47.641 INFO:tasks.workunit.client.0.smithi136.stdout: Found existing installation: pip 21.3.1 2024-04-22T19:53:47.794 INFO:tasks.workunit.client.0.smithi136.stdout: Uninstalling pip-21.3.1: 2024-04-22T19:53:47.800 INFO:tasks.workunit.client.0.smithi136.stdout: Successfully uninstalled pip-21.3.1 2024-04-22T19:53:48.758 INFO:tasks.workunit.client.0.smithi136.stdout:Successfully installed pip-24.0 2024-04-22T19:53:49.033 INFO:tasks.workunit.client.0.smithi136.stderr:+ pip install boto3 2024-04-22T19:53:49.940 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting boto3 2024-04-22T19:53:50.042 INFO:tasks.workunit.client.0.smithi136.stdout: Downloading boto3-1.34.89-py3-none-any.whl.metadata (6.6 kB) 2024-04-22T19:53:50.720 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting botocore<1.35.0,>=1.34.89 (from boto3) 2024-04-22T19:53:50.737 INFO:tasks.workunit.client.0.smithi136.stdout: Downloading botocore-1.34.89-py3-none-any.whl.metadata (5.7 kB) 2024-04-22T19:53:50.867 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-04-22T19:53:50.886 INFO:tasks.workunit.client.0.smithi136.stdout: Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-04-22T19:53:51.024 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-04-22T19:53:51.045 INFO:tasks.workunit.client.0.smithi136.stdout: Downloading s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-04-22T19:53:51.184 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T19:53:51.209 INFO:tasks.workunit.client.0.smithi136.stdout: Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-04-22T19:53:51.351 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting urllib3<1.27,>=1.25.4 (from botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T19:53:51.368 INFO:tasks.workunit.client.0.smithi136.stdout: Downloading urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB) 2024-04-22T19:53:51.438 INFO:tasks.workunit.client.0.smithi136.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 587.7 kB/s eta 0:00:00 2024-04-22T19:53:51.674 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T19:53:51.692 INFO:tasks.workunit.client.0.smithi136.stdout: Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-04-22T19:53:51.777 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading boto3-1.34.89-py3-none-any.whl (139 kB) 2024-04-22T19:53:51.848 INFO:tasks.workunit.client.0.smithi136.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.3/139.3 kB 1.9 MB/s eta 0:00:00 2024-04-22T19:53:51.867 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading botocore-1.34.89-py3-none-any.whl (12.2 MB) 2024-04-22T19:53:52.158 INFO:tasks.workunit.client.0.smithi136.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.2/12.2 MB 42.4 MB/s eta 0:00:00 2024-04-22T19:53:52.176 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-04-22T19:53:52.240 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-04-22T19:53:52.291 INFO:tasks.workunit.client.0.smithi136.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 82.2/82.2 kB 1.4 MB/s eta 0:00:00 2024-04-22T19:53:52.309 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-04-22T19:53:52.359 INFO:tasks.workunit.client.0.smithi136.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 229.9/229.9 kB 4.7 MB/s eta 0:00:00 2024-04-22T19:53:52.377 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB) 2024-04-22T19:53:52.418 INFO:tasks.workunit.client.0.smithi136.stdout: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.8/143.8 kB 3.4 MB/s eta 0:00:00 2024-04-22T19:53:52.435 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-22T19:53:52.589 INFO:tasks.workunit.client.0.smithi136.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-04-22T19:53:53.656 INFO:tasks.workunit.client.0.smithi136.stdout:Successfully installed boto3-1.34.89 botocore-1.34.89 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-1.26.18 2024-04-22T19:53:53.808 INFO:tasks.workunit.client.0.smithi136.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_bucket_check.py 2024-04-22T19:53:53.939 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin user create --uid check-tester --display-name "Check Testing" --access-key OJODXSLNX4LUNHQG99PA --secret 3l6ffld34qaymfomuh832j94738aie2x4p2o8h6n 2024-04-22T19:53:53.974 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:53:53.974 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:53:54.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.006+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.007+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.007+0000 7f2aea830800 20 realm 2024-04-22T19:53:54.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.007+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.007+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.007+0000 7f2aea830800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:53:54.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.007+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.008+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.008+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.009+0000 7f2aea830800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:54.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.009+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.009+0000 7f2aea830800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:53:54.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.009+0000 7f2aea830800 20 searching for the correct realm 2024-04-22T19:53:54.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.011+0000 7f2ae6293640 0 -- 172.21.15.136:0/4013672727 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f2ae001dbd0 msgr2=0x7f2ae0080a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:53:54.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.217+0000 7f2adffff640 0 -- 172.21.15.136:0/4013672727 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f2ae001d730 msgr2=0x7f2ae007e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:53:54.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.424+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:53:54.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 20 zone default found 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 4 Realm: () 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 20 zonegroup default 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.425+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.426+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.426+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.426+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.426+0000 7f2aea830800 20 started sync module instance, tier type = 2024-04-22T19:53:54.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.426+0000 7f2aea830800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:53:54.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.441+0000 7f2aea830800 20 add_watcher() i=0 2024-04-22T19:53:54.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.441+0000 7f2aea830800 20 add_watcher() i=1 2024-04-22T19:53:54.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.441+0000 7f2aea830800 20 add_watcher() i=2 2024-04-22T19:53:54.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.441+0000 7f2aea830800 20 add_watcher() i=3 2024-04-22T19:53:54.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.441+0000 7f2aea830800 20 add_watcher() i=4 2024-04-22T19:53:54.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.441+0000 7f2aea830800 20 add_watcher() i=5 2024-04-22T19:53:54.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.441+0000 7f2aea830800 20 add_watcher() i=6 2024-04-22T19:53:54.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.442+0000 7f2aea830800 20 add_watcher() i=7 2024-04-22T19:53:54.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.442+0000 7f2aea830800 2 all 8 watchers are set, enabling cache 2024-04-22T19:53:54.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.444+0000 7f29977fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.444+0000 7f2aea830800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:53:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.444+0000 7f2aea830800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:53:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 5 note: GC not initialized 2024-04-22T19:53:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f29967fc640 20 reqs_thread_entry: start 2024-04-22T19:53:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 20 init_complete bucket index max shards: 1 2024-04-22T19:53:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 20 Filter name: none 2024-04-22T19:53:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2907fff640 20 reqs_thread_entry: start 2024-04-22T19:53:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 10 cache get: name=default.rgw.meta+users.uid+check-tester : miss 2024-04-22T19:53:54.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x0 2024-04-22T19:53:54.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 10 adding default.rgw.meta+users.uid+check-tester to cache LRU end 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : miss 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.445+0000 7f2aea830800 20 rados->read ofs=0 len=0 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.446+0000 7f2aea830800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.446+0000 7f2aea830800 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x0 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.446+0000 7f2aea830800 10 adding default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.446+0000 7f2aea830800 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.446+0000 7f2aea830800 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.447+0000 7f2aea830800 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.447+0000 7f2aea830800 10 moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-04-22T19:53:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.447+0000 7f2aea830800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:check-tester, ofs0, ns] 2024-04-22T19:53:54.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.448+0000 7f29c97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 154618822656 cookie 94839079987904 notifier 4577 bl.length()=593 2024-04-22T19:53:54.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.448+0000 7f29c97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-04-22T19:53:54.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.448+0000 7f29c97fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-04-22T19:53:54.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.473+0000 7f2aea830800 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-04-22T19:53:54.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.473+0000 7f2aea830800 10 moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-04-22T19:53:54.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.473+0000 7f2aea830800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:OJODXSLNX4LUNHQG99PA, ofs0, ns] 2024-04-22T19:53:54.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.474+0000 7f29c97fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 154618822656 cookie 94839079981184 notifier 4577 bl.length()=186 2024-04-22T19:53:54.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.474+0000 7f29c97fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-04-22T19:53:54.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.474+0000 7f29c97fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-04-22T19:53:54.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.476+0000 7f29977fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:54.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.479+0000 7f2aea830800 20 remove_watcher() i=0 2024-04-22T19:53:54.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.479+0000 7f2aea830800 2 removed watcher, disabling cache 2024-04-22T19:53:54.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.480+0000 7f2aea830800 20 remove_watcher() i=1 2024-04-22T19:53:54.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.481+0000 7f2aea830800 20 remove_watcher() i=2 2024-04-22T19:53:54.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.483+0000 7f2aea830800 20 remove_watcher() i=3 2024-04-22T19:53:54.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.484+0000 7f2aea830800 20 remove_watcher() i=4 2024-04-22T19:53:54.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.485+0000 7f2aea830800 20 remove_watcher() i=5 2024-04-22T19:53:54.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.486+0000 7f2aea830800 20 remove_watcher() i=6 2024-04-22T19:53:54.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:54.487+0000 7f2aea830800 20 remove_watcher() i=7 2024-04-22T19:53:54.698 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:54.699 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:54.867 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:54.968 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:54.968 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:55.192 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:55.192 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:55.576 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:55.577 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:55.676 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:55.676 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:55.768 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:55.768 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:55.798 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:55.798 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:55.896 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:55.896 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:56.004 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:56.005 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:56.037 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:56.037 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:56.102 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:56.102 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:56.127 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:56.127 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:56.179 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:56.179 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:56.203 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:56.204 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:56.269 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:56.269 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:56.292 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: recalculated bucket check stats are correct 2024-04-22T19:53:56.292 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:56.294 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-04-22T19:53:56.324 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:53:56.324 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:53:56.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.358+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.359+0000 7f81d4f2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.359+0000 7f81d4f2b800 20 realm 2024-04-22T19:53:56.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.359+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.359+0000 7f81d4f2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.359+0000 7f81d4f2b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:53:56.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.359+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.359+0000 7f81d4f2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.359+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.360+0000 7f81d4f2b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:56.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.360+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.361+0000 7f81d4f2b800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:53:56.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.361+0000 7f81d4f2b800 20 searching for the correct realm 2024-04-22T19:53:56.363 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.363+0000 7f81d1353640 0 -- 172.21.15.136:0/2504079372 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f81cc017020 msgr2=0x7f81cc02d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:53:56.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.568+0000 7f81cbfff640 0 -- 172.21.15.136:0/2504079372 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f81cc06b460 msgr2=0x7f81cc06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.775+0000 7f81d4f2b800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.775+0000 7f81d4f2b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.775+0000 7f81d4f2b800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.775+0000 7f81d4f2b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.775+0000 7f81d4f2b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.775+0000 7f81d4f2b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.775+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:56.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 20 zone default found 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 4 Realm: () 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.776+0000 7f81d4f2b800 20 zonegroup default 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.777+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.777+0000 7f81d4f2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.777+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.777+0000 7f81d4f2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.777+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.777+0000 7f81d4f2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.777+0000 7f81d4f2b800 20 started sync module instance, tier type = 2024-04-22T19:53:56.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.777+0000 7f81d4f2b800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:53:56.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.792+0000 7f81d4f2b800 20 add_watcher() i=0 2024-04-22T19:53:56.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.792+0000 7f81d4f2b800 20 add_watcher() i=1 2024-04-22T19:53:56.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.792+0000 7f81d4f2b800 20 add_watcher() i=2 2024-04-22T19:53:56.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.792+0000 7f81d4f2b800 20 add_watcher() i=3 2024-04-22T19:53:56.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.792+0000 7f81d4f2b800 20 add_watcher() i=4 2024-04-22T19:53:56.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.792+0000 7f81d4f2b800 20 add_watcher() i=5 2024-04-22T19:53:56.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.792+0000 7f81d4f2b800 20 add_watcher() i=6 2024-04-22T19:53:56.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.793+0000 7f81d4f2b800 20 add_watcher() i=7 2024-04-22T19:53:56.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.793+0000 7f81d4f2b800 2 all 8 watchers are set, enabling cache 2024-04-22T19:53:56.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.795+0000 7f8086fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:56.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f81d4f2b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:53:56.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f81d4f2b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:53:56.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f81d4f2b800 5 note: GC not initialized 2024-04-22T19:53:56.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f8085fe3640 20 reqs_thread_entry: start 2024-04-22T19:53:56.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f81d4f2b800 20 init_complete bucket index max shards: 1 2024-04-22T19:53:56.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f81d4f2b800 20 Filter name: none 2024-04-22T19:53:56.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f7ff77fe640 20 reqs_thread_entry: start 2024-04-22T19:53:56.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f81d4f2b800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:53:56.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.796+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 20 list_objects_ordered: starting attempt 1 2024-04-22T19:53:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.797+0000 7f81d4f2b800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T19:53:56.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.798+0000 7f81d4f2b800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T19:53:56.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.798+0000 7f81d4f2b800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T19:53:56.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.798+0000 7f81d4f2b800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T19:53:56.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.798+0000 7f81d4f2b800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T19:53:56.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.801+0000 7f8086fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:56.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.804+0000 7f81d4f2b800 20 remove_watcher() i=0 2024-04-22T19:53:56.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.804+0000 7f81d4f2b800 2 removed watcher, disabling cache 2024-04-22T19:53:56.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.805+0000 7f81d4f2b800 20 remove_watcher() i=1 2024-04-22T19:53:56.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.807+0000 7f81d4f2b800 20 remove_watcher() i=2 2024-04-22T19:53:56.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.808+0000 7f81d4f2b800 20 remove_watcher() i=3 2024-04-22T19:53:56.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.809+0000 7f81d4f2b800 20 remove_watcher() i=4 2024-04-22T19:53:56.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.810+0000 7f81d4f2b800 20 remove_watcher() i=5 2024-04-22T19:53:56.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.811+0000 7f81d4f2b800 20 remove_watcher() i=6 2024-04-22T19:53:56.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.812+0000 7f81d4f2b800 20 remove_watcher() i=7 2024-04-22T19:53:56.823 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-04-22T19:53:56.851 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:53:56.851 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:53:56.884 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.884+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.886+0000 7f73c8a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.886+0000 7f73c8a10800 20 realm 2024-04-22T19:53:56.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.886+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.886+0000 7f73c8a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.886+0000 7f73c8a10800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:53:56.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.886+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.886+0000 7f73c8a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:56.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.886+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.887+0000 7f73c8a10800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:56.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.887+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:56.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.887+0000 7f73c8a10800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:53:56.888 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.887+0000 7f73c8a10800 20 searching for the correct realm 2024-04-22T19:53:56.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:56.889+0000 7f73c48fe640 0 -- 172.21.15.136:0/745327106 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f73bc06fc20 msgr2=0x7f73bc0760d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:53:57.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.095+0000 7f73c38fc640 0 -- 172.21.15.136:0/745327106 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f73bc017020 msgr2=0x7f73bc098910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.301+0000 7f73c8a10800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.301+0000 7f73c8a10800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.301+0000 7f73c8a10800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.301+0000 7f73c8a10800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.301+0000 7f73c8a10800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.301+0000 7f73c8a10800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.301+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 20 zone default found 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 4 Realm: () 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 20 zonegroup default 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.302+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.303+0000 7f73c8a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.303+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.303+0000 7f73c8a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.303+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.303+0000 7f73c8a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.303+0000 7f73c8a10800 20 started sync module instance, tier type = 2024-04-22T19:53:57.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.303+0000 7f73c8a10800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:53:57.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.315+0000 7f73c8a10800 20 add_watcher() i=0 2024-04-22T19:53:57.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.315+0000 7f73c8a10800 20 add_watcher() i=1 2024-04-22T19:53:57.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.315+0000 7f73c8a10800 20 add_watcher() i=2 2024-04-22T19:53:57.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.315+0000 7f73c8a10800 20 add_watcher() i=3 2024-04-22T19:53:57.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.315+0000 7f73c8a10800 20 add_watcher() i=4 2024-04-22T19:53:57.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.315+0000 7f73c8a10800 20 add_watcher() i=5 2024-04-22T19:53:57.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.315+0000 7f73c8a10800 20 add_watcher() i=6 2024-04-22T19:53:57.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.318+0000 7f73c8a10800 20 add_watcher() i=7 2024-04-22T19:53:57.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.318+0000 7f73c8a10800 2 all 8 watchers are set, enabling cache 2024-04-22T19:53:57.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.320+0000 7f727afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:57.321 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.321+0000 7f73c8a10800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:53:57.321 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.321+0000 7f73c8a10800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:53:57.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.321+0000 7f73c8a10800 5 note: GC not initialized 2024-04-22T19:53:57.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.321+0000 7f7279fdb640 20 reqs_thread_entry: start 2024-04-22T19:53:57.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.321+0000 7f73c8a10800 20 init_complete bucket index max shards: 1 2024-04-22T19:53:57.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.321+0000 7f73c8a10800 20 Filter name: none 2024-04-22T19:53:57.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.321+0000 7f71eb7fe640 20 reqs_thread_entry: start 2024-04-22T19:53:57.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.321+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.322+0000 7f73c8a10800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.322+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.322+0000 7f73c8a10800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.322+0000 7f73c8a10800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.322+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.322+0000 7f73c8a10800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.323+0000 7f73c8a10800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.323+0000 7f73c8a10800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:53:57.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.323+0000 7f73c8a10800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:53:57.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.325+0000 7f727afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:57.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.327+0000 7f73c8a10800 20 remove_watcher() i=0 2024-04-22T19:53:57.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.327+0000 7f73c8a10800 2 removed watcher, disabling cache 2024-04-22T19:53:57.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.328+0000 7f73c8a10800 20 remove_watcher() i=1 2024-04-22T19:53:57.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.331+0000 7f73c8a10800 20 remove_watcher() i=2 2024-04-22T19:53:57.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.332+0000 7f73c8a10800 20 remove_watcher() i=3 2024-04-22T19:53:57.334 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.334+0000 7f73c8a10800 20 remove_watcher() i=4 2024-04-22T19:53:57.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.335+0000 7f73c8a10800 20 remove_watcher() i=5 2024-04-22T19:53:57.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.336+0000 7f73c8a10800 20 remove_watcher() i=6 2024-04-22T19:53:57.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.337+0000 7f73c8a10800 20 remove_watcher() i=7 2024-04-22T19:53:57.347 INFO:tasks.workunit.client.0.smithi136.stderr:{'rgw.main': {'size': 54, 'size_actual': 24576, 'size_utilized': 95, 'size_kb': 1, 'size_kb_actual': 24, 'size_kb_utilized': 1, 'num_objects': 6}} 2024-04-22T19:53:57.347 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check unlinked does not report normal entries 2024-04-22T19:53:57.347 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:53:57.348 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-04-22T19:53:57.380 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:53:57.380 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:53:57.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.414+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.415+0000 7fde09333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.415+0000 7fde09333800 20 realm 2024-04-22T19:53:57.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.415+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.416+0000 7fde09333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.416+0000 7fde09333800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:53:57.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.416+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.416+0000 7fde09333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.416+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.417+0000 7fde09333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:57.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.417+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.417+0000 7fde09333800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:53:57.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.417+0000 7fde09333800 20 searching for the correct realm 2024-04-22T19:53:57.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.419+0000 7fde057d2640 0 -- 172.21.15.136:0/3812646174 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x557d93da6d50 msgr2=0x557d93da9160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:53:57.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.422+0000 7fde057d2640 0 -- 172.21.15.136:0/3812646174 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7fdcd0078ec0 msgr2=0x7fdcd007b2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:53:57.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:53:57.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:53:57.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:53:57.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:53:57.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:53:57.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:53:57.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:57.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.428+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:53:57.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 20 zone default found 2024-04-22T19:53:57.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 4 Realm: () 2024-04-22T19:53:57.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:53:57.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:53:57.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 20 zonegroup default 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.429+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.430+0000 7fde09333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.430+0000 7fde09333800 20 started sync module instance, tier type = 2024-04-22T19:53:57.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.430+0000 7fde09333800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:53:57.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.443+0000 7fde09333800 20 add_watcher() i=0 2024-04-22T19:53:57.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.443+0000 7fde09333800 20 add_watcher() i=1 2024-04-22T19:53:57.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.443+0000 7fde09333800 20 add_watcher() i=2 2024-04-22T19:53:57.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.443+0000 7fde09333800 20 add_watcher() i=3 2024-04-22T19:53:57.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.443+0000 7fde09333800 20 add_watcher() i=4 2024-04-22T19:53:57.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.443+0000 7fde09333800 20 add_watcher() i=5 2024-04-22T19:53:57.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.443+0000 7fde09333800 20 add_watcher() i=6 2024-04-22T19:53:57.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.446+0000 7fde09333800 20 add_watcher() i=7 2024-04-22T19:53:57.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.446+0000 7fde09333800 2 all 8 watchers are set, enabling cache 2024-04-22T19:53:57.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fdcbafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:57.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fde09333800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:53:57.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fde09333800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:53:57.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fde09333800 5 note: GC not initialized 2024-04-22T19:53:57.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fdcb9fe3640 20 reqs_thread_entry: start 2024-04-22T19:53:57.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fde09333800 20 init_complete bucket index max shards: 1 2024-04-22T19:53:57.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fde09333800 20 Filter name: none 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fdc2b7fe640 20 reqs_thread_entry: start 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fde09333800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.448+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 20 rados->read ofs=0 len=0 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:53:57.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:53:57.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.449+0000 7fde09333800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:53:57.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.450+0000 7fde09333800 1 NOTICE: finished shard 0 (0 entries found) 2024-04-22T19:53:57.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.450+0000 7fde09333800 1 NOTICE: finished all shards (0 entries found) 2024-04-22T19:53:57.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.451+0000 7fdcbafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:57.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.453+0000 7fde09333800 20 remove_watcher() i=0 2024-04-22T19:53:57.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.453+0000 7fde09333800 2 removed watcher, disabling cache 2024-04-22T19:53:57.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.454+0000 7fde09333800 20 remove_watcher() i=1 2024-04-22T19:53:57.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.455+0000 7fde09333800 20 remove_watcher() i=2 2024-04-22T19:53:57.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.456+0000 7fde09333800 20 remove_watcher() i=3 2024-04-22T19:53:57.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.457+0000 7fde09333800 20 remove_watcher() i=4 2024-04-22T19:53:57.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.458+0000 7fde09333800 20 remove_watcher() i=5 2024-04-22T19:53:57.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.459+0000 7fde09333800 20 remove_watcher() i=6 2024-04-22T19:53:57.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:57.460+0000 7fde09333800 20 remove_watcher() i=7 2024-04-22T19:53:57.474 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-04-22T19:53:57.872 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-04-22T19:53:59.265 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:59.265 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:59.366 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T19:53:59.368 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-04-22T19:53:59.398 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:53:59.399 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:53:59.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.434+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.436+0000 7f0fa6898800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.436+0000 7f0fa6898800 20 realm 2024-04-22T19:53:59.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.436+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.436+0000 7f0fa6898800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.436+0000 7f0fa6898800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:53:59.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.436+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.436+0000 7f0fa6898800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.436+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.437+0000 7f0fa6898800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:59.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.437+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.438+0000 7f0fa6898800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:53:59.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.438+0000 7f0fa6898800 20 searching for the correct realm 2024-04-22T19:53:59.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.439+0000 7f0fa4b98640 0 -- 172.21.15.136:0/606696074 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x56314bb071b0 msgr2=0x56314bb095a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:53:59.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.443+0000 7f0f9bfff640 0 -- 172.21.15.136:0/606696074 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f0f9408c060 msgr2=0x7f0f94068040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:53:59.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:53:59.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:53:59.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:53:59.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:53:59.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:53:59.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:53:59.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.449+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 zone default found 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 4 Realm: () 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 zonegroup default 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.450+0000 7f0fa6898800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.451+0000 7f0fa6898800 20 started sync module instance, tier type = 2024-04-22T19:53:59.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.451+0000 7f0fa6898800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:53:59.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.463+0000 7f0fa6898800 20 add_watcher() i=0 2024-04-22T19:53:59.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.463+0000 7f0fa6898800 20 add_watcher() i=1 2024-04-22T19:53:59.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.463+0000 7f0fa6898800 20 add_watcher() i=2 2024-04-22T19:53:59.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.463+0000 7f0fa6898800 20 add_watcher() i=3 2024-04-22T19:53:59.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.463+0000 7f0fa6898800 20 add_watcher() i=4 2024-04-22T19:53:59.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.463+0000 7f0fa6898800 20 add_watcher() i=5 2024-04-22T19:53:59.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.463+0000 7f0fa6898800 20 add_watcher() i=6 2024-04-22T19:53:59.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.466+0000 7f0fa6898800 20 add_watcher() i=7 2024-04-22T19:53:59.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.466+0000 7f0fa6898800 2 all 8 watchers are set, enabling cache 2024-04-22T19:53:59.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.467+0000 7f0e5afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.468+0000 7f0fa6898800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.468+0000 7f0fa6898800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.468+0000 7f0fa6898800 5 note: GC not initialized 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.468+0000 7f0e59fdb640 20 reqs_thread_entry: start 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.468+0000 7f0fa6898800 20 init_complete bucket index max shards: 1 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.468+0000 7f0fa6898800 20 Filter name: none 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.468+0000 7f0dcb7fe640 20 reqs_thread_entry: start 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.468+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.469+0000 7f0fa6898800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:53:59.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.469+0000 7f0fa6898800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.469+0000 7f0fa6898800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:53:59.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.469+0000 7f0fa6898800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:53:59.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.469+0000 7f0fa6898800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:53:59.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.469+0000 7f0fa6898800 20 INFO: main: starting shard=0 2024-04-22T19:53:59.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.469+0000 7f0fa6898800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:53:59.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.470+0000 7f0fa6898800 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-04-22T19:53:59.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.470+0000 7f0fa6898800 20 INFO: main: done 2024-04-22T19:53:59.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.470+0000 7f0e5afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:53:59.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.473+0000 7f0fa6898800 20 remove_watcher() i=0 2024-04-22T19:53:59.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.473+0000 7f0fa6898800 2 removed watcher, disabling cache 2024-04-22T19:53:59.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.474+0000 7f0fa6898800 20 remove_watcher() i=1 2024-04-22T19:53:59.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.475+0000 7f0fa6898800 20 remove_watcher() i=2 2024-04-22T19:53:59.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.476+0000 7f0fa6898800 20 remove_watcher() i=3 2024-04-22T19:53:59.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.478+0000 7f0fa6898800 20 remove_watcher() i=4 2024-04-22T19:53:59.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.479+0000 7f0fa6898800 20 remove_watcher() i=5 2024-04-22T19:53:59.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.483+0000 7f0fa6898800 20 remove_watcher() i=6 2024-04-22T19:53:59.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.484+0000 7f0fa6898800 20 remove_watcher() i=7 2024-04-22T19:53:59.496 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:59.496 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:59.521 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:53:59.522 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:53:59.608 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T19:53:59.610 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-04-22T19:53:59.639 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:53:59.639 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:53:59.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.672+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.674+0000 7f839fa1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.674+0000 7f839fa1d800 20 realm 2024-04-22T19:53:59.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.674+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.674+0000 7f839fa1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.674+0000 7f839fa1d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:53:59.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.674+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.674+0000 7f839fa1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:53:59.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.674+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.675+0000 7f839fa1d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:53:59.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.675+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:53:59.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.676+0000 7f839fa1d800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:53:59.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.676+0000 7f839fa1d800 20 searching for the correct realm 2024-04-22T19:53:59.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.678+0000 7f839dd1d640 0 -- 172.21.15.136:0/3816799824 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f8398017030 msgr2=0x7f83980798d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:53:59.884 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:53:59.884+0000 7f8397fff640 0 -- 172.21.15.136:0/3816799824 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f839806a6d0 msgr2=0x7f8398060030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.091+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:00.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 20 zone default found 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 4 Realm: () 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 20 zonegroup default 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.092+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.093+0000 7f839fa1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.093+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.093+0000 7f839fa1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.093+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.093+0000 7f839fa1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.093+0000 7f839fa1d800 20 started sync module instance, tier type = 2024-04-22T19:54:00.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.093+0000 7f839fa1d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:00.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.113+0000 7f839fa1d800 20 add_watcher() i=0 2024-04-22T19:54:00.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.113+0000 7f839fa1d800 20 add_watcher() i=1 2024-04-22T19:54:00.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.113+0000 7f839fa1d800 20 add_watcher() i=2 2024-04-22T19:54:00.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.113+0000 7f839fa1d800 20 add_watcher() i=3 2024-04-22T19:54:00.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.113+0000 7f839fa1d800 20 add_watcher() i=4 2024-04-22T19:54:00.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.113+0000 7f839fa1d800 20 add_watcher() i=5 2024-04-22T19:54:00.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.113+0000 7f839fa1d800 20 add_watcher() i=6 2024-04-22T19:54:00.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.114+0000 7f839fa1d800 20 add_watcher() i=7 2024-04-22T19:54:00.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.114+0000 7f839fa1d800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:00.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.117+0000 7f82537e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:00.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.117+0000 7f839fa1d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:00.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.117+0000 7f839fa1d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:00.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.117+0000 7f839fa1d800 5 note: GC not initialized 2024-04-22T19:54:00.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.117+0000 7f82527e4640 20 reqs_thread_entry: start 2024-04-22T19:54:00.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.117+0000 7f839fa1d800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:00.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.117+0000 7f839fa1d800 20 Filter name: none 2024-04-22T19:54:00.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.117+0000 7f81c3fff640 20 reqs_thread_entry: start 2024-04-22T19:54:00.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.118+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.118+0000 7f839fa1d800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:00.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.118+0000 7f839fa1d800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.118+0000 7f839fa1d800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:00.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.118+0000 7f839fa1d800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:00.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.118+0000 7f839fa1d800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:00.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.118+0000 7f839fa1d800 20 INFO: main: starting shard=0 2024-04-22T19:54:00.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.118+0000 7f839fa1d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:00.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.119+0000 7f839fa1d800 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-04-22T19:54:00.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.119+0000 7f839fa1d800 20 INFO: main: done 2024-04-22T19:54:00.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.120+0000 7f82537e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:00.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.122+0000 7f839fa1d800 20 remove_watcher() i=0 2024-04-22T19:54:00.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.122+0000 7f839fa1d800 2 removed watcher, disabling cache 2024-04-22T19:54:00.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.124+0000 7f839fa1d800 20 remove_watcher() i=1 2024-04-22T19:54:00.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.125+0000 7f839fa1d800 20 remove_watcher() i=2 2024-04-22T19:54:00.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.126+0000 7f839fa1d800 20 remove_watcher() i=3 2024-04-22T19:54:00.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.127+0000 7f839fa1d800 20 remove_watcher() i=4 2024-04-22T19:54:00.128 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.129+0000 7f839fa1d800 20 remove_watcher() i=5 2024-04-22T19:54:00.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.131+0000 7f839fa1d800 20 remove_watcher() i=6 2024-04-22T19:54:00.132 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.132+0000 7f839fa1d800 20 remove_watcher() i=7 2024-04-22T19:54:00.144 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:00.144 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:00.168 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:00.168 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:00.192 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:00.192 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:00.306 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T19:54:00.308 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-04-22T19:54:00.338 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:00.338 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:00.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.371+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.372+0000 7f322563f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.372+0000 7f322563f800 20 realm 2024-04-22T19:54:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.372+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.373+0000 7f322563f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.373+0000 7f322563f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.373+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.373+0000 7f322563f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.373+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.373+0000 7f322563f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:00.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.373+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.374+0000 7f322563f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:00.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.374+0000 7f322563f800 20 searching for the correct realm 2024-04-22T19:54:00.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.376+0000 7f322393f640 0 -- 172.21.15.136:0/3006929288 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f321c05fff0 msgr2=0x7f321c05ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:00.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.581+0000 7f321bfff640 0 -- 172.21.15.136:0/3006929288 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f321c07fb90 msgr2=0x7f321c0235f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:00.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:00.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.788+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:00.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 20 zone default found 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 4 Realm: () 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 20 zonegroup default 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.789+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.790+0000 7f322563f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.790+0000 7f322563f800 20 started sync module instance, tier type = 2024-04-22T19:54:00.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.790+0000 7f322563f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:00.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.809+0000 7f322563f800 20 add_watcher() i=0 2024-04-22T19:54:00.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.809+0000 7f322563f800 20 add_watcher() i=1 2024-04-22T19:54:00.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.809+0000 7f322563f800 20 add_watcher() i=2 2024-04-22T19:54:00.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.809+0000 7f322563f800 20 add_watcher() i=3 2024-04-22T19:54:00.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.809+0000 7f322563f800 20 add_watcher() i=4 2024-04-22T19:54:00.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.809+0000 7f322563f800 20 add_watcher() i=5 2024-04-22T19:54:00.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.809+0000 7f322563f800 20 add_watcher() i=6 2024-04-22T19:54:00.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.812+0000 7f322563f800 20 add_watcher() i=7 2024-04-22T19:54:00.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.812+0000 7f322563f800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:00.813 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f30d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f322563f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f322563f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f322563f800 5 note: GC not initialized 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f30d8fe1640 20 reqs_thread_entry: start 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f322563f800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f322563f800 20 Filter name: none 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f30527fc640 20 reqs_thread_entry: start 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.814+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.815+0000 7f322563f800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:00.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.815+0000 7f322563f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:00.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.815+0000 7f322563f800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:00.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.815+0000 7f322563f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:00.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.815+0000 7f322563f800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:00.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.815+0000 7f322563f800 20 INFO: main: starting shard=0 2024-04-22T19:54:00.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.815+0000 7f322563f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:00.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.816+0000 7f322563f800 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-22T19:54:00.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.816+0000 7f322563f800 20 INFO: main: done 2024-04-22T19:54:00.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.817+0000 7f30d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:00.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.819+0000 7f322563f800 20 remove_watcher() i=0 2024-04-22T19:54:00.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.819+0000 7f322563f800 2 removed watcher, disabling cache 2024-04-22T19:54:00.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.821+0000 7f322563f800 20 remove_watcher() i=1 2024-04-22T19:54:00.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.822+0000 7f322563f800 20 remove_watcher() i=2 2024-04-22T19:54:00.823 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.823+0000 7f322563f800 20 remove_watcher() i=3 2024-04-22T19:54:00.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.824+0000 7f322563f800 20 remove_watcher() i=4 2024-04-22T19:54:00.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.825+0000 7f322563f800 20 remove_watcher() i=5 2024-04-22T19:54:00.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.827+0000 7f322563f800 20 remove_watcher() i=6 2024-04-22T19:54:00.828 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:00.828+0000 7f322563f800 20 remove_watcher() i=7 2024-04-22T19:54:00.839 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:00.840 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:00.864 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:00.864 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:00.947 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T19:54:00.949 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-04-22T19:54:00.978 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:00.978 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:01.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.012+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.013+0000 7f3ec3e30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:01.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.013+0000 7f3ec3e30800 20 realm 2024-04-22T19:54:01.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.013+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.013+0000 7f3ec3e30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:01.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.013+0000 7f3ec3e30800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:01.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.013+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.013+0000 7f3ec3e30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:01.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.013+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.014+0000 7f3ec3e30800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:01.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.014+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.015+0000 7f3ec3e30800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:01.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.015+0000 7f3ec3e30800 20 searching for the correct realm 2024-04-22T19:54:01.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.017+0000 7f3ebf92f640 0 -- 172.21.15.136:0/3643020426 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f3eb808b450 msgr2=0x7f3eb8084400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:01.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.223+0000 7f3eb692d640 0 -- 172.21.15.136:0/3643020426 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f3eb8011b30 msgr2=0x7f3eb801a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:01.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:01.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:01.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:01.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:01.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:01.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:01.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:01.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:01.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.430+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:01.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 20 zone default found 2024-04-22T19:54:01.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 4 Realm: () 2024-04-22T19:54:01.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:01.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:01.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 20 zonegroup default 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.431+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.432+0000 7f3ec3e30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.432+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.432+0000 7f3ec3e30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.432+0000 7f3ec3e30800 20 started sync module instance, tier type = 2024-04-22T19:54:01.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.432+0000 7f3ec3e30800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:01.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.446+0000 7f3ec3e30800 20 add_watcher() i=0 2024-04-22T19:54:01.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.446+0000 7f3ec3e30800 20 add_watcher() i=1 2024-04-22T19:54:01.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.446+0000 7f3ec3e30800 20 add_watcher() i=2 2024-04-22T19:54:01.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.446+0000 7f3ec3e30800 20 add_watcher() i=3 2024-04-22T19:54:01.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.446+0000 7f3ec3e30800 20 add_watcher() i=4 2024-04-22T19:54:01.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.446+0000 7f3ec3e30800 20 add_watcher() i=5 2024-04-22T19:54:01.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.448+0000 7f3ec3e30800 20 add_watcher() i=6 2024-04-22T19:54:01.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.448+0000 7f3ec3e30800 20 add_watcher() i=7 2024-04-22T19:54:01.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.448+0000 7f3ec3e30800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:01.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.451+0000 7f3d75fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:01.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.451+0000 7f3ec3e30800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:01.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.451+0000 7f3ec3e30800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:01.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.451+0000 7f3ec3e30800 5 note: GC not initialized 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.451+0000 7f3d74fe1640 20 reqs_thread_entry: start 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 Filter name: none 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3cee7fc640 20 reqs_thread_entry: start 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 rados->read ofs=0 len=0 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 INFO: main: starting shard=0 2024-04-22T19:54:01.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.452+0000 7f3ec3e30800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:01.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.453+0000 7f3ec3e30800 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-22T19:54:01.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.453+0000 7f3ec3e30800 20 INFO: main: done 2024-04-22T19:54:01.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.454+0000 7f3d75fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:01.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.456+0000 7f3ec3e30800 20 remove_watcher() i=0 2024-04-22T19:54:01.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.456+0000 7f3ec3e30800 2 removed watcher, disabling cache 2024-04-22T19:54:01.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.458+0000 7f3ec3e30800 20 remove_watcher() i=1 2024-04-22T19:54:01.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.459+0000 7f3ec3e30800 20 remove_watcher() i=2 2024-04-22T19:54:01.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.460+0000 7f3ec3e30800 20 remove_watcher() i=3 2024-04-22T19:54:01.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.461+0000 7f3ec3e30800 20 remove_watcher() i=4 2024-04-22T19:54:01.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.462+0000 7f3ec3e30800 20 remove_watcher() i=5 2024-04-22T19:54:01.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.463+0000 7f3ec3e30800 20 remove_watcher() i=6 2024-04-22T19:54:01.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:01.465+0000 7f3ec3e30800 20 remove_watcher() i=7 2024-04-22T19:54:01.478 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:01.478 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:01.501 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-04-22T19:54:01.904 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-04-22T19:54:02.303 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check unlinked finds unlistable entries 2024-04-22T19:54:02.303 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:02.304 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-04-22T19:54:02.335 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:02.335 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:02.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.369+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.370+0000 7fe211a40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:02.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.370+0000 7fe211a40800 20 realm 2024-04-22T19:54:02.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.370+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.371+0000 7fe211a40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:02.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.371+0000 7fe211a40800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:02.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.371+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.371+0000 7fe211a40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:02.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.371+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.372+0000 7fe211a40800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:02.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.372+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.372+0000 7fe211a40800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:02.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.372+0000 7fe211a40800 20 searching for the correct realm 2024-04-22T19:54:02.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.374+0000 7fe20cedd640 0 -- 172.21.15.136:0/3892132416 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fe2040b7030 msgr2=0x7fe2040a0910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:02.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.581+0000 7fe20bedb640 0 -- 172.21.15.136:0/3892132416 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fe204003fd0 msgr2=0x7fe2040cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:02.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:02.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:02.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:02.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:02.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:02.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:02.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:02.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.788+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 zone default found 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 4 Realm: () 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:02.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 zonegroup default 2024-04-22T19:54:02.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:02.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:02.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:02.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.789+0000 7fe211a40800 20 started sync module instance, tier type = 2024-04-22T19:54:02.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.790+0000 7fe211a40800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:02.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.805+0000 7fe211a40800 20 add_watcher() i=0 2024-04-22T19:54:02.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.805+0000 7fe211a40800 20 add_watcher() i=1 2024-04-22T19:54:02.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.805+0000 7fe211a40800 20 add_watcher() i=2 2024-04-22T19:54:02.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.805+0000 7fe211a40800 20 add_watcher() i=3 2024-04-22T19:54:02.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.805+0000 7fe211a40800 20 add_watcher() i=4 2024-04-22T19:54:02.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.805+0000 7fe211a40800 20 add_watcher() i=5 2024-04-22T19:54:02.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.805+0000 7fe211a40800 20 add_watcher() i=6 2024-04-22T19:54:02.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.806+0000 7fe211a40800 20 add_watcher() i=7 2024-04-22T19:54:02.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.806+0000 7fe211a40800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:02.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.808+0000 7fe0c2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.808+0000 7fe211a40800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.808+0000 7fe211a40800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.808+0000 7fe211a40800 5 note: GC not initialized 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.808+0000 7fe0c1fdb640 20 reqs_thread_entry: start 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.808+0000 7fe211a40800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 20 Filter name: none 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe0337fe640 20 reqs_thread_entry: start 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:54:02.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:02.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:54:02.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:54:02.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:54:02.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 20 rados->read ofs=0 len=0 2024-04-22T19:54:02.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:02.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:54:02.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:54:02.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:54:02.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:54:02.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:54:02.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.809+0000 7fe211a40800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:02.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.812+0000 7fe211a40800 1 NOTICE: finished shard 0 (4 entries found) 2024-04-22T19:54:02.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.812+0000 7fe211a40800 1 NOTICE: finished all shards (4 entries found) 2024-04-22T19:54:02.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.813+0000 7fe0c2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:02.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.815+0000 7fe211a40800 20 remove_watcher() i=0 2024-04-22T19:54:02.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.815+0000 7fe211a40800 2 removed watcher, disabling cache 2024-04-22T19:54:02.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.816+0000 7fe211a40800 20 remove_watcher() i=1 2024-04-22T19:54:02.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.817+0000 7fe211a40800 20 remove_watcher() i=2 2024-04-22T19:54:02.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.818+0000 7fe211a40800 20 remove_watcher() i=3 2024-04-22T19:54:02.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.820+0000 7fe211a40800 20 remove_watcher() i=4 2024-04-22T19:54:02.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.820+0000 7fe211a40800 20 remove_watcher() i=5 2024-04-22T19:54:02.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.822+0000 7fe211a40800 20 remove_watcher() i=6 2024-04-22T19:54:02.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:02.823+0000 7fe211a40800 20 remove_watcher() i=7 2024-04-22T19:54:02.832 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: unlinked entries are not listable 2024-04-22T19:54:02.832 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:02.835 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:02.835 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:02.940 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: GET returns 404 for unlinked entry keys that have no other versions 2024-04-22T19:54:02.940 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:02.943 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:02.943 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.014 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.014 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.074 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check unlinked fixes unlistable entries 2024-04-22T19:54:03.074 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:03.076 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-04-22T19:54:03.109 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:03.110 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:03.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.143+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.144+0000 7f85a5a87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:03.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.144+0000 7f85a5a87800 20 realm 2024-04-22T19:54:03.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.144+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.144+0000 7f85a5a87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:03.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.144+0000 7f85a5a87800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:03.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.144+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.145+0000 7f85a5a87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:03.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.145+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.146+0000 7f85a5a87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:03.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.146+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.146+0000 7f85a5a87800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:03.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.146+0000 7f85a5a87800 20 searching for the correct realm 2024-04-22T19:54:03.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.148+0000 7f85a3d87640 0 -- 172.21.15.136:0/4293398446 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f859c034040 msgr2=0x7f859c034420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:03.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.354+0000 7f859ad85640 0 -- 172.21.15.136:0/4293398446 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f859c018040 msgr2=0x7f859c031900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:03.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:03.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:03.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:03.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:03.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:03.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:03.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:03.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.560+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:03.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 20 zone default found 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 4 Realm: () 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 20 zonegroup default 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.561+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.562+0000 7f85a5a87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:03.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.562+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.562+0000 7f85a5a87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:03.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.562+0000 7f85a5a87800 20 started sync module instance, tier type = 2024-04-22T19:54:03.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.562+0000 7f85a5a87800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:03.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.579+0000 7f85a5a87800 20 add_watcher() i=0 2024-04-22T19:54:03.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.579+0000 7f85a5a87800 20 add_watcher() i=1 2024-04-22T19:54:03.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.579+0000 7f85a5a87800 20 add_watcher() i=2 2024-04-22T19:54:03.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.579+0000 7f85a5a87800 20 add_watcher() i=3 2024-04-22T19:54:03.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.579+0000 7f85a5a87800 20 add_watcher() i=4 2024-04-22T19:54:03.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.579+0000 7f85a5a87800 20 add_watcher() i=5 2024-04-22T19:54:03.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.580+0000 7f85a5a87800 20 add_watcher() i=6 2024-04-22T19:54:03.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.580+0000 7f85a5a87800 20 add_watcher() i=7 2024-04-22T19:54:03.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.580+0000 7f85a5a87800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:03.582 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.582+0000 7f8459fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f85a5a87800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f85a5a87800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f85a5a87800 5 note: GC not initialized 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f8458fe1640 20 reqs_thread_entry: start 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f85a5a87800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f85a5a87800 20 Filter name: none 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f83d27fc640 20 reqs_thread_entry: start 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f85a5a87800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.583+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:54:03.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:54:03.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 20 rados->read ofs=0 len=0 2024-04-22T19:54:03.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:03.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:54:03.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:54:03.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:54:03.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:54:03.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:54:03.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.584+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.585+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.586+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:c state=0x55875458e548 s->prefetch_data=0 2024-04-22T19:54:03.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.586+0000 7f85a5a87800 20 get_obj_state: setting s->obj_tag to u7ck9gvwv1vnw56wsjdbgb52q6rsylor 2024-04-22T19:54:03.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.586+0000 7f85a5a87800 20 get_obj_state_impl: setting s->olh_tag to jykfeb4f7b8md6mx04tew56zsoa6oip6 2024-04-22T19:54:03.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.586+0000 7f85a5a87800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=jykfeb4f7b8md6mx04tew56zsoa6oip6 2024-04-22T19:54:03.588 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.588+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.592+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.592+0000 7f85a5a87800 20 olh_log_entry: epoch=3 op=3 key=c[1UHbqnsmpa78U36MfC58.oL5gCupbcO] 2024-04-22T19:54:03.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.592+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:_:1UHbqnsmpa78U36MfC58.oL5gCupbcO_c state=0x5587545d17b8 s->prefetch_data=0 2024-04-22T19:54:03.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.592+0000 7f85a5a87800 10 manifest: total_size = 22 2024-04-22T19:54:03.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.592+0000 7f85a5a87800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10454028735483610166 2024-04-22T19:54:03.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.592+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:_:1UHbqnsmpa78U36MfC58.oL5gCupbcO_c state=0x5587545d17b8 s->prefetch_data=0 2024-04-22T19:54:03.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.592+0000 7f85a5a87800 20 prepare_atomic_modification: state is not atomic. state=0x5587545d17b8 2024-04-22T19:54:03.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.592+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.595+0000 7f85a5a87800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T19:54:03.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.595+0000 7f85a5a87800 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-04-22T19:54:03.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.595+0000 7f85a5a87800 0 deleting objects inline since gc isn't initialized 2024-04-22T19:54:03.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.595+0000 7f85a5a87800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1__shadow:1UHbqnsmpa78U36MfC58.oL5gCupbcO_.2FEDzkEZvEWvQtku_99K0n9BT2HeV6z_0 2024-04-22T19:54:03.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.597+0000 7f8486ffd640 20 handle_completion(): completion ok for obj=c[1UHbqnsmpa78U36MfC58.oL5gCupbcO] 2024-04-22T19:54:03.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.599+0000 7f85a5a87800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T19:54:03.600 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.600+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.600 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.600+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.606+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.606+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.607+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:d state=0x5587545d4558 s->prefetch_data=0 2024-04-22T19:54:03.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.607+0000 7f85a5a87800 20 get_obj_state: setting s->obj_tag to b24w12guppb8sluschylqndud4697oa7 2024-04-22T19:54:03.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.607+0000 7f85a5a87800 20 get_obj_state_impl: setting s->olh_tag to nb04vfo8n0zf8jakkgojunh2olyya2xi 2024-04-22T19:54:03.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.607+0000 7f85a5a87800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=nb04vfo8n0zf8jakkgojunh2olyya2xi 2024-04-22T19:54:03.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.608+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.610+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.610+0000 7f85a5a87800 20 olh_log_entry: epoch=3 op=3 key=d[oPIPEm4QTBHDSEteLBm7VtuJj3Ajy90] 2024-04-22T19:54:03.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.610+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:_:oPIPEm4QTBHDSEteLBm7VtuJj3Ajy90_d state=0x5587545ce788 s->prefetch_data=0 2024-04-22T19:54:03.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.611+0000 7f85a5a87800 10 manifest: total_size = 11 2024-04-22T19:54:03.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.611+0000 7f85a5a87800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5948744533309867967 2024-04-22T19:54:03.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.611+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:_:oPIPEm4QTBHDSEteLBm7VtuJj3Ajy90_d state=0x5587545ce788 s->prefetch_data=0 2024-04-22T19:54:03.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.611+0000 7f85a5a87800 20 prepare_atomic_modification: state is not atomic. state=0x5587545ce788 2024-04-22T19:54:03.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.611+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.614+0000 7f85a5a87800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T19:54:03.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.614+0000 7f85a5a87800 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2024-04-22T19:54:03.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.614+0000 7f85a5a87800 0 deleting objects inline since gc isn't initialized 2024-04-22T19:54:03.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.614+0000 7f85a5a87800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1__shadow:oPIPEm4QTBHDSEteLBm7VtuJj3Ajy90_.BrEB3ELLuNmd_l8oZCM1rElMqgc3Zbm_0 2024-04-22T19:54:03.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.616+0000 7f84867fc640 20 handle_completion(): completion ok for obj=d[oPIPEm4QTBHDSEteLBm7VtuJj3Ajy90] 2024-04-22T19:54:03.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.616+0000 7f85a5a87800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T19:54:03.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.621+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.621+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.622+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.623+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.623+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:e state=0x5587545d7de8 s->prefetch_data=0 2024-04-22T19:54:03.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.624+0000 7f85a5a87800 20 get_obj_state: setting s->obj_tag to clbipo09fk28tva182oewkr4ed31ctoa 2024-04-22T19:54:03.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.624+0000 7f85a5a87800 20 get_obj_state_impl: setting s->olh_tag to 7976zlh8aa2pxyi3htbnhkkvqccp0or3 2024-04-22T19:54:03.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.624+0000 7f85a5a87800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7976zlh8aa2pxyi3htbnhkkvqccp0or3 2024-04-22T19:54:03.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.625+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.626+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.627+0000 7f85a5a87800 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-04-22T19:54:03.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.627+0000 7f85a5a87800 20 olh_log_entry: epoch=2 op=3 key=e[bg8zysORbgpKSuE0uFmUnmmhLKWIAFb] 2024-04-22T19:54:03.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.627+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:_:bg8zysORbgpKSuE0uFmUnmmhLKWIAFb_e state=0x5587545cf798 s->prefetch_data=0 2024-04-22T19:54:03.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.627+0000 7f85a5a87800 10 manifest: total_size = 12 2024-04-22T19:54:03.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.627+0000 7f85a5a87800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6744681641460913867 2024-04-22T19:54:03.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.627+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:_:bg8zysORbgpKSuE0uFmUnmmhLKWIAFb_e state=0x5587545cf798 s->prefetch_data=0 2024-04-22T19:54:03.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.627+0000 7f85a5a87800 20 prepare_atomic_modification: state is not atomic. state=0x5587545cf798 2024-04-22T19:54:03.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.627+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.632+0000 7f85a5a87800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T19:54:03.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.632+0000 7f85a5a87800 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2024-04-22T19:54:03.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.632+0000 7f85a5a87800 0 deleting objects inline since gc isn't initialized 2024-04-22T19:54:03.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.632+0000 7f85a5a87800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1__shadow:bg8zysORbgpKSuE0uFmUnmmhLKWIAFb_.03kOMu5bu5nOKTubA15lU-o-Ius7LpU_0 2024-04-22T19:54:03.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.635+0000 7f8486ffd640 20 handle_completion(): completion ok for obj=e[bg8zysORbgpKSuE0uFmUnmmhLKWIAFb] 2024-04-22T19:54:03.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.635+0000 7f85a5a87800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T19:54:03.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.637+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:e state=0x5587545d7de8 s->prefetch_data=0 2024-04-22T19:54:03.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.637+0000 7f85a5a87800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=7976zlh8aa2pxyi3htbnhkkvqccp0or3 2024-04-22T19:54:03.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.640+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.642 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.642+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.642+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.643+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:f state=0x5587545d7de8 s->prefetch_data=0 2024-04-22T19:54:03.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.643+0000 7f85a5a87800 20 get_obj_state: setting s->obj_tag to y58bramz4mt0xeycttl1t873kichaa2n 2024-04-22T19:54:03.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.643+0000 7f85a5a87800 20 get_obj_state_impl: setting s->olh_tag to 38v6kw3wub8hg1n96orjdnnb9c5m8mbu 2024-04-22T19:54:03.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.643+0000 7f85a5a87800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=38v6kw3wub8hg1n96orjdnnb9c5m8mbu 2024-04-22T19:54:03.645 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.645+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-04-22T19:54:03.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 20 olh_log_entry: epoch=2 op=3 key=f[7OJTM3b9h1gEoyHwzUXd1h9WrVL73ak] 2024-04-22T19:54:03.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:_:7OJTM3b9h1gEoyHwzUXd1h9WrVL73ak_f state=0x5587545cf798 s->prefetch_data=0 2024-04-22T19:54:03.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 10 manifest: total_size = 22 2024-04-22T19:54:03.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11389206435799992746 2024-04-22T19:54:03.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:_:7OJTM3b9h1gEoyHwzUXd1h9WrVL73ak_f state=0x5587545cf798 s->prefetch_data=0 2024-04-22T19:54:03.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 20 prepare_atomic_modification: state is not atomic. state=0x5587545cf798 2024-04-22T19:54:03.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.651+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.656+0000 7f85a5a87800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T19:54:03.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.656+0000 7f85a5a87800 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-04-22T19:54:03.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.656+0000 7f85a5a87800 0 deleting objects inline since gc isn't initialized 2024-04-22T19:54:03.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.656+0000 7f85a5a87800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1__shadow:7OJTM3b9h1gEoyHwzUXd1h9WrVL73ak_.uLHusyLDVwJETnOC7CmvrzQRHz3B_If_0 2024-04-22T19:54:03.665 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.664+0000 7f85a5a87800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T19:54:03.665 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.664+0000 7f84867fc640 20 handle_completion(): completion ok for obj=f[7OJTM3b9h1gEoyHwzUXd1h9WrVL73ak] 2024-04-22T19:54:03.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.666+0000 7f85a5a87800 20 get_obj_state: octx=0x5587542c1570 obj=check-bucket:f state=0x5587545d7de8 s->prefetch_data=0 2024-04-22T19:54:03.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.666+0000 7f85a5a87800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=38v6kw3wub8hg1n96orjdnnb9c5m8mbu 2024-04-22T19:54:03.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.671+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.673+0000 7f85a5a87800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:03.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.673+0000 7f85a5a87800 1 NOTICE: finished shard 0 (4 entries removed) 2024-04-22T19:54:03.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.673+0000 7f85a5a87800 1 NOTICE: finished all shards (4 entries removed) 2024-04-22T19:54:03.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.674+0000 7f8459fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:03.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.676+0000 7f85a5a87800 20 remove_watcher() i=0 2024-04-22T19:54:03.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.676+0000 7f85a5a87800 2 removed watcher, disabling cache 2024-04-22T19:54:03.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.677+0000 7f85a5a87800 20 remove_watcher() i=1 2024-04-22T19:54:03.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.678+0000 7f85a5a87800 20 remove_watcher() i=2 2024-04-22T19:54:03.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.680+0000 7f85a5a87800 20 remove_watcher() i=3 2024-04-22T19:54:03.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.681+0000 7f85a5a87800 20 remove_watcher() i=4 2024-04-22T19:54:03.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.682+0000 7f85a5a87800 20 remove_watcher() i=5 2024-04-22T19:54:03.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.683+0000 7f85a5a87800 20 remove_watcher() i=6 2024-04-22T19:54:03.684 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:03.684+0000 7f85a5a87800 20 remove_watcher() i=7 2024-04-22T19:54:03.696 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.696 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.715 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.715 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.735 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.736 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.756 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.756 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.777 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check unlinked does not affect normal entries 2024-04-22T19:54:03.777 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:03.778 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.778 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.845 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.845 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.867 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.868 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.891 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.891 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.914 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:03.914 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:03.936 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check unlinked does not find new unlistable entries after fix 2024-04-22T19:54:03.936 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:03.938 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-04-22T19:54:03.969 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:03.969 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:04.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.003+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.004+0000 7f38f5f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.004+0000 7f38f5f33800 20 realm 2024-04-22T19:54:04.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.004+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.004+0000 7f38f5f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.004+0000 7f38f5f33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:04.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.004+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.004+0000 7f38f5f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.004+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.005+0000 7f38f5f33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:04.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.005+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.006+0000 7f38f5f33800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:04.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.006+0000 7f38f5f33800 20 searching for the correct realm 2024-04-22T19:54:04.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.007+0000 7f38f23fd640 0 -- 172.21.15.136:0/4089759635 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55a73bb694e0 msgr2=0x55a73bb6b8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:04.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.010+0000 7f38f23fd640 0 -- 172.21.15.136:0/4089759635 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f37bc078f40 msgr2=0x7f37bc07b370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:04.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:04.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:04.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:04.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:04.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:04.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:04.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:04.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.016+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:04.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 20 zone default found 2024-04-22T19:54:04.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 4 Realm: () 2024-04-22T19:54:04.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 20 zonegroup default 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.017+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.018+0000 7f38f5f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.018+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.018+0000 7f38f5f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.018+0000 7f38f5f33800 20 started sync module instance, tier type = 2024-04-22T19:54:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.018+0000 7f38f5f33800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:04.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.027+0000 7f38f5f33800 20 add_watcher() i=0 2024-04-22T19:54:04.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.027+0000 7f38f5f33800 20 add_watcher() i=1 2024-04-22T19:54:04.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.027+0000 7f38f5f33800 20 add_watcher() i=2 2024-04-22T19:54:04.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.027+0000 7f38f5f33800 20 add_watcher() i=3 2024-04-22T19:54:04.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.027+0000 7f38f5f33800 20 add_watcher() i=4 2024-04-22T19:54:04.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.027+0000 7f38f5f33800 20 add_watcher() i=5 2024-04-22T19:54:04.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.027+0000 7f38f5f33800 20 add_watcher() i=6 2024-04-22T19:54:04.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.028+0000 7f38f5f33800 20 add_watcher() i=7 2024-04-22T19:54:04.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.028+0000 7f38f5f33800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:04.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.029+0000 7f37a37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:04.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f38f5f33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f38f5f33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f38f5f33800 5 note: GC not initialized 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f37a27fc640 20 reqs_thread_entry: start 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f38f5f33800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f38f5f33800 20 Filter name: none 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f3713fff640 20 reqs_thread_entry: start 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f38f5f33800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.030+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:54:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:04.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:54:04.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:54:04.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:54:04.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:54:04.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:54:04.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.031+0000 7f38f5f33800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:04.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.032+0000 7f38f5f33800 1 NOTICE: finished shard 0 (0 entries found) 2024-04-22T19:54:04.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.032+0000 7f38f5f33800 1 NOTICE: finished all shards (0 entries found) 2024-04-22T19:54:04.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.032+0000 7f37a37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.034+0000 7f38f5f33800 20 remove_watcher() i=0 2024-04-22T19:54:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.034+0000 7f38f5f33800 2 removed watcher, disabling cache 2024-04-22T19:54:04.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.036+0000 7f38f5f33800 20 remove_watcher() i=1 2024-04-22T19:54:04.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.037+0000 7f38f5f33800 20 remove_watcher() i=2 2024-04-22T19:54:04.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.038+0000 7f38f5f33800 20 remove_watcher() i=3 2024-04-22T19:54:04.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.039+0000 7f38f5f33800 20 remove_watcher() i=4 2024-04-22T19:54:04.040 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.040+0000 7f38f5f33800 20 remove_watcher() i=5 2024-04-22T19:54:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.041+0000 7f38f5f33800 20 remove_watcher() i=6 2024-04-22T19:54:04.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.042+0000 7f38f5f33800 20 remove_watcher() i=7 2024-04-22T19:54:04.054 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:04.054 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:04.194 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:04.195 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:04.245 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-04-22T19:54:04.274 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:04.274 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:04.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.307+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.309+0000 7f1814b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.309+0000 7f1814b25800 20 realm 2024-04-22T19:54:04.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.309+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.309+0000 7f1814b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.309+0000 7f1814b25800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:04.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.309+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.309+0000 7f1814b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.309+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.310+0000 7f1814b25800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:04.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.310+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.310+0000 7f1814b25800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:04.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.310+0000 7f1814b25800 20 searching for the correct realm 2024-04-22T19:54:04.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.312+0000 7f18110b8640 0 -- 172.21.15.136:0/2604640994 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f180805a740 msgr2=0x7f1808057a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:04.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.517+0000 7f18100b6640 0 -- 172.21.15.136:0/2604640994 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f18080896f0 msgr2=0x7f1808078290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.723+0000 7f1814b25800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.723+0000 7f1814b25800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.723+0000 7f1814b25800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.723+0000 7f1814b25800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.723+0000 7f1814b25800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.723+0000 7f1814b25800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.723+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:04.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 20 zone default found 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 4 Realm: () 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 20 zonegroup default 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.724+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.725+0000 7f1814b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.725+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.725+0000 7f1814b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.725+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.725+0000 7f1814b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.725+0000 7f1814b25800 20 started sync module instance, tier type = 2024-04-22T19:54:04.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.725+0000 7f1814b25800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:04.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.736+0000 7f1814b25800 20 add_watcher() i=0 2024-04-22T19:54:04.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.736+0000 7f1814b25800 20 add_watcher() i=1 2024-04-22T19:54:04.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.736+0000 7f1814b25800 20 add_watcher() i=2 2024-04-22T19:54:04.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.736+0000 7f1814b25800 20 add_watcher() i=3 2024-04-22T19:54:04.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.736+0000 7f1814b25800 20 add_watcher() i=4 2024-04-22T19:54:04.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.736+0000 7f1814b25800 20 add_watcher() i=5 2024-04-22T19:54:04.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.736+0000 7f1814b25800 20 add_watcher() i=6 2024-04-22T19:54:04.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.737+0000 7f1814b25800 20 add_watcher() i=7 2024-04-22T19:54:04.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.737+0000 7f1814b25800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:04.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.739+0000 7f16c6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:04.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.740+0000 7f1814b25800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:04.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.740+0000 7f1814b25800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:04.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.740+0000 7f1814b25800 5 note: GC not initialized 2024-04-22T19:54:04.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.740+0000 7f16c5fe3640 20 reqs_thread_entry: start 2024-04-22T19:54:04.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.740+0000 7f1814b25800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:04.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.740+0000 7f1814b25800 20 Filter name: none 2024-04-22T19:54:04.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.740+0000 7f16377fe640 20 reqs_thread_entry: start 2024-04-22T19:54:04.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.740+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.741+0000 7f1814b25800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.741+0000 7f1814b25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.741+0000 7f1814b25800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.741+0000 7f1814b25800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.741+0000 7f1814b25800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.741+0000 7f1814b25800 20 INFO: main: starting shard=0 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.741+0000 7f1814b25800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.742+0000 7f1814b25800 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-04-22T19:54:04.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.742+0000 7f1814b25800 20 INFO: main: done 2024-04-22T19:54:04.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.742+0000 7f16c6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:04.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.745+0000 7f1814b25800 20 remove_watcher() i=0 2024-04-22T19:54:04.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.745+0000 7f1814b25800 2 removed watcher, disabling cache 2024-04-22T19:54:04.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.746+0000 7f1814b25800 20 remove_watcher() i=1 2024-04-22T19:54:04.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.747+0000 7f1814b25800 20 remove_watcher() i=2 2024-04-22T19:54:04.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.748+0000 7f1814b25800 20 remove_watcher() i=3 2024-04-22T19:54:04.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.749+0000 7f1814b25800 20 remove_watcher() i=4 2024-04-22T19:54:04.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.750+0000 7f1814b25800 20 remove_watcher() i=5 2024-04-22T19:54:04.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.751+0000 7f1814b25800 20 remove_watcher() i=6 2024-04-22T19:54:04.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.752+0000 7f1814b25800 20 remove_watcher() i=7 2024-04-22T19:54:04.763 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-04-22T19:54:04.794 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:04.794 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:04.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.827+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.829+0000 7f33dd23a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.829+0000 7f33dd23a800 20 realm 2024-04-22T19:54:04.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.829+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.829+0000 7f33dd23a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.829+0000 7f33dd23a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:04.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.829+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.829+0000 7f33dd23a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:04.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.829+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.830+0000 7f33dd23a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:04.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.830+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:04.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.830+0000 7f33dd23a800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:04.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.830+0000 7f33dd23a800 20 searching for the correct realm 2024-04-22T19:54:04.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:04.832+0000 7f33d86d6640 0 -- 172.21.15.136:0/1024397643 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f33d005e940 msgr2=0x7f33d0069410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:05.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.038+0000 7f33cffff640 0 -- 172.21.15.136:0/1024397643 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f33d0083950 msgr2=0x7f33d00733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.244+0000 7f33dd23a800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.244+0000 7f33dd23a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.244+0000 7f33dd23a800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.244+0000 7f33dd23a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.244+0000 7f33dd23a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.244+0000 7f33dd23a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.244+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:05.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:05.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 20 zone default found 2024-04-22T19:54:05.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 4 Realm: () 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 20 zonegroup default 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.245+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.246+0000 7f33dd23a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.246+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.246+0000 7f33dd23a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.246+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.246+0000 7f33dd23a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.246+0000 7f33dd23a800 20 started sync module instance, tier type = 2024-04-22T19:54:05.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.246+0000 7f33dd23a800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:05.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.260+0000 7f33dd23a800 20 add_watcher() i=0 2024-04-22T19:54:05.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.260+0000 7f33dd23a800 20 add_watcher() i=1 2024-04-22T19:54:05.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.260+0000 7f33dd23a800 20 add_watcher() i=2 2024-04-22T19:54:05.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.260+0000 7f33dd23a800 20 add_watcher() i=3 2024-04-22T19:54:05.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.260+0000 7f33dd23a800 20 add_watcher() i=4 2024-04-22T19:54:05.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.260+0000 7f33dd23a800 20 add_watcher() i=5 2024-04-22T19:54:05.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.260+0000 7f33dd23a800 20 add_watcher() i=6 2024-04-22T19:54:05.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.263+0000 7f33dd23a800 20 add_watcher() i=7 2024-04-22T19:54:05.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.263+0000 7f33dd23a800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:05.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.264+0000 7f328dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.265+0000 7f33dd23a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.265+0000 7f33dd23a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.265+0000 7f33dd23a800 5 note: GC not initialized 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.265+0000 7f328cfe1640 20 reqs_thread_entry: start 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.265+0000 7f33dd23a800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.265+0000 7f33dd23a800 20 Filter name: none 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.265+0000 7f32067fc640 20 reqs_thread_entry: start 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.265+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.266+0000 7f33dd23a800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:05.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.266+0000 7f33dd23a800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.266+0000 7f33dd23a800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:05.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.266+0000 7f33dd23a800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:05.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.266+0000 7f33dd23a800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:05.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.266+0000 7f33dd23a800 20 INFO: main: starting shard=0 2024-04-22T19:54:05.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.266+0000 7f33dd23a800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:05.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.267+0000 7f33dd23a800 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-04-22T19:54:05.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.267+0000 7f33dd23a800 20 INFO: main: done 2024-04-22T19:54:05.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.268+0000 7f328dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:05.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.270+0000 7f33dd23a800 20 remove_watcher() i=0 2024-04-22T19:54:05.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.270+0000 7f33dd23a800 2 removed watcher, disabling cache 2024-04-22T19:54:05.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.272+0000 7f33dd23a800 20 remove_watcher() i=1 2024-04-22T19:54:05.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.273+0000 7f33dd23a800 20 remove_watcher() i=2 2024-04-22T19:54:05.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.274+0000 7f33dd23a800 20 remove_watcher() i=3 2024-04-22T19:54:05.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.275+0000 7f33dd23a800 20 remove_watcher() i=4 2024-04-22T19:54:05.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.276+0000 7f33dd23a800 20 remove_watcher() i=5 2024-04-22T19:54:05.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.277+0000 7f33dd23a800 20 remove_watcher() i=6 2024-04-22T19:54:05.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.278+0000 7f33dd23a800 20 remove_watcher() i=7 2024-04-22T19:54:05.288 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check olh finds leftover OLH entries 2024-04-22T19:54:05.288 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:05.289 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-04-22T19:54:05.319 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:05.319 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:05.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.353+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.354+0000 7f70d4062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.354+0000 7f70d4062800 20 realm 2024-04-22T19:54:05.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.354+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.354+0000 7f70d4062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.354+0000 7f70d4062800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:05.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.354+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.354+0000 7f70d4062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.354+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.355+0000 7f70d4062800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:05.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.355+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.356+0000 7f70d4062800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:05.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.356+0000 7f70d4062800 20 searching for the correct realm 2024-04-22T19:54:05.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.358+0000 7f70d2362640 0 -- 172.21.15.136:0/3475898805 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f70cc05e070 msgr2=0x7f70cc009ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:05.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.564+0000 7f70d1360640 0 -- 172.21.15.136:0/3475898805 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f70cc0abfc0 msgr2=0x7f70cc067040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:05.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.770+0000 7f70d4062800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:05.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.770+0000 7f70d4062800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:05.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.770+0000 7f70d4062800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:05.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.770+0000 7f70d4062800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:05.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.770+0000 7f70d4062800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:05.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.770+0000 7f70d4062800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:05.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.771+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.771+0000 7f70d4062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.771+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.771+0000 7f70d4062800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:05.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.771+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 zone default found 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 4 Realm: () 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 zonegroup default 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 started sync module instance, tier type = 2024-04-22T19:54:05.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.772+0000 7f70d4062800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:05.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.784+0000 7f70d4062800 20 add_watcher() i=0 2024-04-22T19:54:05.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.784+0000 7f70d4062800 20 add_watcher() i=1 2024-04-22T19:54:05.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.784+0000 7f70d4062800 20 add_watcher() i=2 2024-04-22T19:54:05.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.784+0000 7f70d4062800 20 add_watcher() i=3 2024-04-22T19:54:05.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.784+0000 7f70d4062800 20 add_watcher() i=4 2024-04-22T19:54:05.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.784+0000 7f70d4062800 20 add_watcher() i=5 2024-04-22T19:54:05.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.784+0000 7f70d4062800 20 add_watcher() i=6 2024-04-22T19:54:05.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.785+0000 7f70d4062800 20 add_watcher() i=7 2024-04-22T19:54:05.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.785+0000 7f70d4062800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:05.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f6f837fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f70d4062800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f70d4062800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f70d4062800 5 note: GC not initialized 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f6f827fc640 20 reqs_thread_entry: start 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f70d4062800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f70d4062800 20 Filter name: none 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f6ef3fff640 20 reqs_thread_entry: start 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f70d4062800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.788+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:05.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:54:05.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:54:05.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:54:05.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.789+0000 7f70d4062800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.790+0000 7f70d4062800 1 NOTICE: finished shard 0 (2 entries found) 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.790+0000 7f70d4062800 1 NOTICE: finished all shards (2 entries found) 2024-04-22T19:54:05.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.791+0000 7f6f837fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:05.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.793+0000 7f70d4062800 20 remove_watcher() i=0 2024-04-22T19:54:05.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.793+0000 7f70d4062800 2 removed watcher, disabling cache 2024-04-22T19:54:05.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.794+0000 7f70d4062800 20 remove_watcher() i=1 2024-04-22T19:54:05.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.796+0000 7f70d4062800 20 remove_watcher() i=2 2024-04-22T19:54:05.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.797+0000 7f70d4062800 20 remove_watcher() i=3 2024-04-22T19:54:05.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.798+0000 7f70d4062800 20 remove_watcher() i=4 2024-04-22T19:54:05.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.799+0000 7f70d4062800 20 remove_watcher() i=5 2024-04-22T19:54:05.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.800+0000 7f70d4062800 20 remove_watcher() i=6 2024-04-22T19:54:05.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.801+0000 7f70d4062800 20 remove_watcher() i=7 2024-04-22T19:54:05.810 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check olh fixes leftover OLH entries 2024-04-22T19:54:05.810 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:05.812 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check olh --bucket check-bucket --fix --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-04-22T19:54:05.844 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:05.844 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:05.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.877+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.879+0000 7f57e2d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.879+0000 7f57e2d42800 20 realm 2024-04-22T19:54:05.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.879+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.879+0000 7f57e2d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.879+0000 7f57e2d42800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:05.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.879+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.879+0000 7f57e2d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:05.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.879+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.880+0000 7f57e2d42800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:05.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.880+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:05.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.880+0000 7f57e2d42800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:05.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.880+0000 7f57e2d42800 20 searching for the correct realm 2024-04-22T19:54:05.884 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.884+0000 7f57e0040640 0 -- 172.21.15.136:0/3907747753 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f57d0028040 msgr2=0x7f57d00791c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:05.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:05.887+0000 7f57e0040640 0 -- 172.21.15.136:0/3907747753 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f57d8061970 msgr2=0x7f57d8062f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:06.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.093+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:06.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:06.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 20 zone default found 2024-04-22T19:54:06.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 4 Realm: () 2024-04-22T19:54:06.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 20 zonegroup default 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.094+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.095+0000 7f57e2d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.095+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.095+0000 7f57e2d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.095+0000 7f57e2d42800 20 started sync module instance, tier type = 2024-04-22T19:54:06.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.095+0000 7f57e2d42800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:06.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.113+0000 7f57e2d42800 20 add_watcher() i=0 2024-04-22T19:54:06.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.113+0000 7f57e2d42800 20 add_watcher() i=1 2024-04-22T19:54:06.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.113+0000 7f57e2d42800 20 add_watcher() i=2 2024-04-22T19:54:06.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.113+0000 7f57e2d42800 20 add_watcher() i=3 2024-04-22T19:54:06.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.113+0000 7f57e2d42800 20 add_watcher() i=4 2024-04-22T19:54:06.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.113+0000 7f57e2d42800 20 add_watcher() i=5 2024-04-22T19:54:06.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.113+0000 7f57e2d42800 20 add_watcher() i=6 2024-04-22T19:54:06.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.116+0000 7f57e2d42800 20 add_watcher() i=7 2024-04-22T19:54:06.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.116+0000 7f57e2d42800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:06.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.119+0000 7f5696fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.119+0000 7f57e2d42800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.119+0000 7f57e2d42800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.119+0000 7f57e2d42800 5 note: GC not initialized 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.119+0000 7f5695fe3640 20 reqs_thread_entry: start 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.119+0000 7f57e2d42800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.119+0000 7f57e2d42800 20 Filter name: none 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f56077fe640 20 reqs_thread_entry: start 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:54:06.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:06.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:54:06.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:54:06.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:54:06.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:06.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:54:06.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:54:06.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:54:06.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:54:06.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:54:06.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.120+0000 7f57e2d42800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:06.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.121+0000 7f57e2d42800 20 get_obj_state: octx=0x563bc3ff1580 obj=check-bucket:c state=0x563bc42be4d8 s->prefetch_data=0 2024-04-22T19:54:06.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.122+0000 7f57e2d42800 20 get_obj_state: setting s->obj_tag to u7ck9gvwv1vnw56wsjdbgb52q6rsylor 2024-04-22T19:54:06.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.122+0000 7f57e2d42800 20 get_obj_state_impl: setting s->olh_tag to jykfeb4f7b8md6mx04tew56zsoa6oip6 2024-04-22T19:54:06.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.122+0000 7f57e2d42800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:06.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.122+0000 7f57e2d42800 20 olh_log_entry: epoch=4 op=2 key=c[] 2024-04-22T19:54:06.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.122+0000 7f57e2d42800 20 olh_log_entry: epoch=4 op=3 key=c[sGE6kU5uzgi65NuRm8Gs1tgKVtTiEUQ] 2024-04-22T19:54:06.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.122+0000 7f57e2d42800 20 get_obj_state: octx=0x563bc42fd2b0 obj=check-bucket:_:sGE6kU5uzgi65NuRm8Gs1tgKVtTiEUQ_c state=0x563bc4301b48 s->prefetch_data=0 2024-04-22T19:54:06.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.124+0000 7f57e2d42800 20 get_obj_state: octx=0x563bc42fd2b0 obj=check-bucket:c state=0x563bc42bfd78 s->prefetch_data=0 2024-04-22T19:54:06.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.125+0000 7f57e2d42800 20 get_obj_state: setting s->obj_tag to u7ck9gvwv1vnw56wsjdbgb52q6rsylor 2024-04-22T19:54:06.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.125+0000 7f57e2d42800 20 get_obj_state_impl: setting s->olh_tag to jykfeb4f7b8md6mx04tew56zsoa6oip6 2024-04-22T19:54:06.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.125+0000 7f57e2d42800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=jykfeb4f7b8md6mx04tew56zsoa6oip6 2024-04-22T19:54:06.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.134+0000 7f57e2d42800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:06.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.135+0000 7f57e2d42800 20 get_obj_state: octx=0x563bc3ff1580 obj=check-bucket:d state=0x563bc42be4d8 s->prefetch_data=0 2024-04-22T19:54:06.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.135+0000 7f57e2d42800 20 get_obj_state: setting s->obj_tag to b24w12guppb8sluschylqndud4697oa7 2024-04-22T19:54:06.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.135+0000 7f57e2d42800 20 get_obj_state_impl: setting s->olh_tag to nb04vfo8n0zf8jakkgojunh2olyya2xi 2024-04-22T19:54:06.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.135+0000 7f57e2d42800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:06.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.136+0000 7f57e2d42800 20 olh_log_entry: epoch=4 op=2 key=d[] 2024-04-22T19:54:06.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.136+0000 7f57e2d42800 20 olh_log_entry: epoch=4 op=3 key=d[2vTs0M6AfTLOy.3csj5EcMtAE4L5dNj] 2024-04-22T19:54:06.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.136+0000 7f57e2d42800 20 get_obj_state: octx=0x563bc42fd2b0 obj=check-bucket:_:2vTs0M6AfTLOy.3csj5EcMtAE4L5dNj_d state=0x563bc4302b58 s->prefetch_data=0 2024-04-22T19:54:06.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.138+0000 7f57e2d42800 20 get_obj_state: octx=0x563bc42fd2b0 obj=check-bucket:d state=0x563bc42fdb08 s->prefetch_data=0 2024-04-22T19:54:06.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.138+0000 7f57e2d42800 20 get_obj_state: setting s->obj_tag to b24w12guppb8sluschylqndud4697oa7 2024-04-22T19:54:06.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.138+0000 7f57e2d42800 20 get_obj_state_impl: setting s->olh_tag to nb04vfo8n0zf8jakkgojunh2olyya2xi 2024-04-22T19:54:06.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.138+0000 7f57e2d42800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=nb04vfo8n0zf8jakkgojunh2olyya2xi 2024-04-22T19:54:06.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.148+0000 7f57e2d42800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:06.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.149+0000 7f57e2d42800 1 NOTICE: finished shard 0 (2 entries removed) 2024-04-22T19:54:06.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.149+0000 7f57e2d42800 1 NOTICE: finished all shards (2 entries removed) 2024-04-22T19:54:06.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.150+0000 7f5696fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:06.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.153+0000 7f57e2d42800 20 remove_watcher() i=0 2024-04-22T19:54:06.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.153+0000 7f57e2d42800 2 removed watcher, disabling cache 2024-04-22T19:54:06.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.154+0000 7f57e2d42800 20 remove_watcher() i=1 2024-04-22T19:54:06.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.156+0000 7f57e2d42800 20 remove_watcher() i=2 2024-04-22T19:54:06.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.157+0000 7f57e2d42800 20 remove_watcher() i=3 2024-04-22T19:54:06.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.158+0000 7f57e2d42800 20 remove_watcher() i=4 2024-04-22T19:54:06.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.159+0000 7f57e2d42800 20 remove_watcher() i=5 2024-04-22T19:54:06.161 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.161+0000 7f57e2d42800 20 remove_watcher() i=6 2024-04-22T19:54:06.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.162+0000 7f57e2d42800 20 remove_watcher() i=7 2024-04-22T19:54:06.173 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-04-22T19:54:06.202 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:06.203 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:06.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.423+0000 7fb97e386640 0 --2- 172.21.15.136:0/2823663524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x564d0b85ef80 0x564d0b9d67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-04-22T19:54:06.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.437+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.439+0000 7fb980887800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.439+0000 7fb980887800 20 realm 2024-04-22T19:54:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.439+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.439+0000 7fb980887800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.439+0000 7fb980887800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.439+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.439+0000 7fb980887800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.439+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.440+0000 7fb980887800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.440+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.441+0000 7fb980887800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:06.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.441+0000 7fb980887800 20 searching for the correct realm 2024-04-22T19:54:06.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.441+0000 7fb97e386640 0 -- 172.21.15.136:0/2113795456 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x564d0ba9d4a0 msgr2=0x564d0ba9f960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:06.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.444+0000 7fb97e386640 0 -- 172.21.15.136:0/2113795456 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x7fb9780cc0b0 msgr2=0x7fb978080610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.653+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 20 zone default found 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 4 Realm: () 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 20 zonegroup default 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.654+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.655+0000 7fb980887800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.655+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.655+0000 7fb980887800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.655+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.655+0000 7fb980887800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.655+0000 7fb980887800 20 started sync module instance, tier type = 2024-04-22T19:54:06.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.655+0000 7fb980887800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:06.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.672+0000 7fb980887800 20 add_watcher() i=0 2024-04-22T19:54:06.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.672+0000 7fb980887800 20 add_watcher() i=1 2024-04-22T19:54:06.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.672+0000 7fb980887800 20 add_watcher() i=2 2024-04-22T19:54:06.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.672+0000 7fb980887800 20 add_watcher() i=3 2024-04-22T19:54:06.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.672+0000 7fb980887800 20 add_watcher() i=4 2024-04-22T19:54:06.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.672+0000 7fb980887800 20 add_watcher() i=5 2024-04-22T19:54:06.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.672+0000 7fb980887800 20 add_watcher() i=6 2024-04-22T19:54:06.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.673+0000 7fb980887800 20 add_watcher() i=7 2024-04-22T19:54:06.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.673+0000 7fb980887800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:06.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.675+0000 7fb834fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:06.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.675+0000 7fb980887800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:06.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.675+0000 7fb980887800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:06.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.675+0000 7fb980887800 5 note: GC not initialized 2024-04-22T19:54:06.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.676+0000 7fb7af7fe640 20 reqs_thread_entry: start 2024-04-22T19:54:06.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.676+0000 7fb980887800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:06.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.676+0000 7fb980887800 20 Filter name: none 2024-04-22T19:54:06.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.676+0000 7fb7ad7fa640 20 reqs_thread_entry: start 2024-04-22T19:54:06.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.676+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.676+0000 7fb980887800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:06.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.676+0000 7fb980887800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.677+0000 7fb980887800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:06.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.677+0000 7fb980887800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:06.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.677+0000 7fb980887800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:06.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.677+0000 7fb980887800 20 INFO: main: starting shard=0 2024-04-22T19:54:06.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.677+0000 7fb980887800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:06.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.677+0000 7fb980887800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T19:54:06.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.677+0000 7fb980887800 20 INFO: main: done 2024-04-22T19:54:06.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.678+0000 7fb834fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:06.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.681+0000 7fb980887800 20 remove_watcher() i=0 2024-04-22T19:54:06.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.681+0000 7fb980887800 2 removed watcher, disabling cache 2024-04-22T19:54:06.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.682+0000 7fb980887800 20 remove_watcher() i=1 2024-04-22T19:54:06.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.683+0000 7fb980887800 20 remove_watcher() i=2 2024-04-22T19:54:06.684 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.684+0000 7fb980887800 20 remove_watcher() i=3 2024-04-22T19:54:06.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.685+0000 7fb980887800 20 remove_watcher() i=4 2024-04-22T19:54:06.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.686+0000 7fb980887800 20 remove_watcher() i=5 2024-04-22T19:54:06.688 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.688+0000 7fb980887800 20 remove_watcher() i=6 2024-04-22T19:54:06.689 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.689+0000 7fb980887800 20 remove_watcher() i=7 2024-04-22T19:54:06.701 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-04-22T19:54:06.730 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:06.730 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:06.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.764+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.766+0000 7fdbfbc62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.766+0000 7fdbfbc62800 20 realm 2024-04-22T19:54:06.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.766+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.766+0000 7fdbfbc62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.766+0000 7fdbfbc62800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:06.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.766+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.766+0000 7fdbfbc62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:06.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.766+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.767+0000 7fdbfbc62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:06.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.767+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:06.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.767+0000 7fdbfbc62800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:06.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.767+0000 7fdbfbc62800 20 searching for the correct realm 2024-04-22T19:54:06.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.769+0000 7fdbf9f62640 0 -- 172.21.15.136:0/3223385300 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fdbf4089020 msgr2=0x7fdbf4054030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:06.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:06.975+0000 7fdbf0f60640 0 -- 172.21.15.136:0/3223385300 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fdbf4003f60 msgr2=0x7fdbf40547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:07.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:07.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:07.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:07.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:07.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:07.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:07.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.181+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:07.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:07.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 20 zone default found 2024-04-22T19:54:07.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 4 Realm: () 2024-04-22T19:54:07.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:07.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:07.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:07.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 20 zonegroup default 2024-04-22T19:54:07.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.182+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.183+0000 7fdbfbc62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.183+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.183+0000 7fdbfbc62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.183+0000 7fdbfbc62800 20 started sync module instance, tier type = 2024-04-22T19:54:07.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.183+0000 7fdbfbc62800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:07.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.197+0000 7fdbfbc62800 20 add_watcher() i=0 2024-04-22T19:54:07.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.197+0000 7fdbfbc62800 20 add_watcher() i=1 2024-04-22T19:54:07.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.197+0000 7fdbfbc62800 20 add_watcher() i=2 2024-04-22T19:54:07.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.197+0000 7fdbfbc62800 20 add_watcher() i=3 2024-04-22T19:54:07.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.197+0000 7fdbfbc62800 20 add_watcher() i=4 2024-04-22T19:54:07.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.197+0000 7fdbfbc62800 20 add_watcher() i=5 2024-04-22T19:54:07.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.198+0000 7fdbfbc62800 20 add_watcher() i=6 2024-04-22T19:54:07.200 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.200+0000 7fdbfbc62800 20 add_watcher() i=7 2024-04-22T19:54:07.200 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.200+0000 7fdbfbc62800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:07.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.202+0000 7fdaaf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:07.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.202+0000 7fdbfbc62800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:07.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.202+0000 7fdbfbc62800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:07.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.202+0000 7fdbfbc62800 5 note: GC not initialized 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.203+0000 7fdaae7e4640 20 reqs_thread_entry: start 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.203+0000 7fdbfbc62800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.203+0000 7fdbfbc62800 20 Filter name: none 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.203+0000 7fda1ffff640 20 reqs_thread_entry: start 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.203+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.203+0000 7fdbfbc62800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.203+0000 7fdbfbc62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.204+0000 7fdbfbc62800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:07.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.204+0000 7fdbfbc62800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:07.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.204+0000 7fdbfbc62800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:07.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.204+0000 7fdbfbc62800 20 INFO: main: starting shard=0 2024-04-22T19:54:07.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.204+0000 7fdbfbc62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:07.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.204+0000 7fdbfbc62800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T19:54:07.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.204+0000 7fdbfbc62800 20 INFO: main: done 2024-04-22T19:54:07.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.205+0000 7fdaaf7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:07.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.207+0000 7fdbfbc62800 20 remove_watcher() i=0 2024-04-22T19:54:07.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.207+0000 7fdbfbc62800 2 removed watcher, disabling cache 2024-04-22T19:54:07.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.209+0000 7fdbfbc62800 20 remove_watcher() i=1 2024-04-22T19:54:07.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.210+0000 7fdbfbc62800 20 remove_watcher() i=2 2024-04-22T19:54:07.211 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.211+0000 7fdbfbc62800 20 remove_watcher() i=3 2024-04-22T19:54:07.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.212+0000 7fdbfbc62800 20 remove_watcher() i=4 2024-04-22T19:54:07.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.214+0000 7fdbfbc62800 20 remove_watcher() i=5 2024-04-22T19:54:07.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.215+0000 7fdbfbc62800 20 remove_watcher() i=6 2024-04-22T19:54:07.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.216+0000 7fdbfbc62800 20 remove_watcher() i=7 2024-04-22T19:54:07.226 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check olh does not find new leftover OLH entries after fix 2024-04-22T19:54:07.226 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:07.228 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-04-22T19:54:07.257 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:07.257 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:07.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.291+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.293+0000 7ffa2b83f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.293+0000 7ffa2b83f800 20 realm 2024-04-22T19:54:07.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.293+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.293+0000 7ffa2b83f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.293+0000 7ffa2b83f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:07.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.293+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.293+0000 7ffa2b83f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.293+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.294+0000 7ffa2b83f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:07.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.294+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.294+0000 7ffa2b83f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:07.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.294+0000 7ffa2b83f800 20 searching for the correct realm 2024-04-22T19:54:07.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.298+0000 7ffa2933e640 0 -- 172.21.15.136:0/627480410 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x55e6b313b0b0 msgr2=0x55e6b313d570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:07.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.301+0000 7ffa28b3d640 0 -- 172.21.15.136:0/627480410 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7ffa2406f2a0 msgr2=0x7ffa24089050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.305+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:07.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 20 zone default found 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 4 Realm: () 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 20 zonegroup default 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.306+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.307+0000 7ffa2b83f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.307+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.307+0000 7ffa2b83f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:07.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.307+0000 7ffa2b83f800 20 started sync module instance, tier type = 2024-04-22T19:54:07.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.307+0000 7ffa2b83f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:07.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.323+0000 7ffa2b83f800 20 add_watcher() i=0 2024-04-22T19:54:07.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.323+0000 7ffa2b83f800 20 add_watcher() i=1 2024-04-22T19:54:07.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.323+0000 7ffa2b83f800 20 add_watcher() i=2 2024-04-22T19:54:07.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.323+0000 7ffa2b83f800 20 add_watcher() i=3 2024-04-22T19:54:07.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.323+0000 7ffa2b83f800 20 add_watcher() i=4 2024-04-22T19:54:07.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.323+0000 7ffa2b83f800 20 add_watcher() i=5 2024-04-22T19:54:07.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.323+0000 7ffa2b83f800 20 add_watcher() i=6 2024-04-22T19:54:07.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.325+0000 7ffa2b83f800 20 add_watcher() i=7 2024-04-22T19:54:07.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.325+0000 7ffa2b83f800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:07.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.327+0000 7ff8df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.327+0000 7ffa2b83f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.327+0000 7ffa2b83f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.327+0000 7ffa2b83f800 5 note: GC not initialized 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ff8de7e4640 20 reqs_thread_entry: start 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 20 Filter name: none 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ff84ffff640 20 reqs_thread_entry: start 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:54:07.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.328+0000 7ffa2b83f800 20 rados->read ofs=0 len=0 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 1 NOTICE: finished shard 0 (0 entries found) 2024-04-22T19:54:07.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.329+0000 7ffa2b83f800 1 NOTICE: finished all shards (0 entries found) 2024-04-22T19:54:07.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.330+0000 7ff8df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:07.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.333+0000 7ffa2b83f800 20 remove_watcher() i=0 2024-04-22T19:54:07.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.333+0000 7ffa2b83f800 2 removed watcher, disabling cache 2024-04-22T19:54:07.334 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.334+0000 7ffa2b83f800 20 remove_watcher() i=1 2024-04-22T19:54:07.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.335+0000 7ffa2b83f800 20 remove_watcher() i=2 2024-04-22T19:54:07.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.336+0000 7ffa2b83f800 20 remove_watcher() i=3 2024-04-22T19:54:07.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.337+0000 7ffa2b83f800 20 remove_watcher() i=4 2024-04-22T19:54:07.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.338+0000 7ffa2b83f800 20 remove_watcher() i=5 2024-04-22T19:54:07.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.339+0000 7ffa2b83f800 20 remove_watcher() i=6 2024-04-22T19:54:07.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:07.340+0000 7ffa2b83f800 20 remove_watcher() i=7 2024-04-22T19:54:07.349 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: verify that bucket check fixes do not affect null version objects 2024-04-22T19:54:07.349 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:07.351 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.351 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.378 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.378 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.401 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.401 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.465 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: bucket check stats are correct in the presence of unlinked entries 2024-04-22T19:54:07.466 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:54:07.466 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.466 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.531 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.531 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.732 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.732 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.792 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.792 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.816 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.816 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.873 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.874 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.898 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.898 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.950 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.950 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:07.974 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:07.974 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:08.026 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:08.026 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:08.049 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:08.049 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:08.101 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:08.101 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:08.125 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:08.177 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:08.177 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:08.203 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-04-22T19:54:08.606 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-04-22T19:54:10.001 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:10.001 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:10.088 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T19:54:10.090 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-04-22T19:54:10.120 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:10.120 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:10.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.355+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.357+0000 7f78aa439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.357+0000 7f78aa439800 20 realm 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.357+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.357+0000 7f78aa439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.357+0000 7f78aa439800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.357+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.357+0000 7f78aa439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.357+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.358+0000 7f78aa439800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:10.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.358+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.359+0000 7f78aa439800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:10.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.359+0000 7f78aa439800 20 searching for the correct realm 2024-04-22T19:54:10.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.361+0000 7f78a59a8640 0 -- 172.21.15.136:0/3368950079 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f78a0036070 msgr2=0x7f78a0017930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:10.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.566+0000 7f78a49a6640 0 -- 172.21.15.136:0/3368950079 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f78a0017040 msgr2=0x7f78a005ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:10.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:10.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:10.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:10.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:10.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:10.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:10.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:10.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.773+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:10.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 20 zone default found 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 4 Realm: () 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 20 zonegroup default 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.774+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.775+0000 7f78aa439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.775+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.775+0000 7f78aa439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.775+0000 7f78aa439800 20 started sync module instance, tier type = 2024-04-22T19:54:10.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.775+0000 7f78aa439800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:10.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 20 add_watcher() i=0 2024-04-22T19:54:10.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 20 add_watcher() i=1 2024-04-22T19:54:10.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 20 add_watcher() i=2 2024-04-22T19:54:10.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 20 add_watcher() i=3 2024-04-22T19:54:10.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 20 add_watcher() i=4 2024-04-22T19:54:10.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 20 add_watcher() i=5 2024-04-22T19:54:10.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 20 add_watcher() i=6 2024-04-22T19:54:10.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 20 add_watcher() i=7 2024-04-22T19:54:10.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.790+0000 7f78aa439800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:10.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.793+0000 7f775b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:10.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.793+0000 7f78aa439800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:10.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.793+0000 7f78aa439800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:10.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.793+0000 7f78aa439800 5 note: GC not initialized 2024-04-22T19:54:10.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.793+0000 7f775a7e4640 20 reqs_thread_entry: start 2024-04-22T19:54:10.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.793+0000 7f78aa439800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:10.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.793+0000 7f78aa439800 20 Filter name: none 2024-04-22T19:54:10.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.793+0000 7f76cbfff640 20 reqs_thread_entry: start 2024-04-22T19:54:10.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.794+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.794+0000 7f78aa439800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:10.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.794+0000 7f78aa439800 20 rados->read ofs=0 len=0 2024-04-22T19:54:10.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.794+0000 7f78aa439800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:10.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.794+0000 7f78aa439800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:10.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.794+0000 7f78aa439800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:10.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.794+0000 7f78aa439800 20 INFO: main: starting shard=0 2024-04-22T19:54:10.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.794+0000 7f78aa439800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:10.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.795+0000 7f78aa439800 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-04-22T19:54:10.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.795+0000 7f78aa439800 20 INFO: main: done 2024-04-22T19:54:10.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.796+0000 7f775b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:10.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.799+0000 7f78aa439800 20 remove_watcher() i=0 2024-04-22T19:54:10.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.799+0000 7f78aa439800 2 removed watcher, disabling cache 2024-04-22T19:54:10.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.800+0000 7f78aa439800 20 remove_watcher() i=1 2024-04-22T19:54:10.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.801+0000 7f78aa439800 20 remove_watcher() i=2 2024-04-22T19:54:10.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.803+0000 7f78aa439800 20 remove_watcher() i=3 2024-04-22T19:54:10.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.804+0000 7f78aa439800 20 remove_watcher() i=4 2024-04-22T19:54:10.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.805+0000 7f78aa439800 20 remove_watcher() i=5 2024-04-22T19:54:10.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.806+0000 7f78aa439800 20 remove_watcher() i=6 2024-04-22T19:54:10.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:10.807+0000 7f78aa439800 20 remove_watcher() i=7 2024-04-22T19:54:10.820 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:10.820 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:10.843 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:10.844 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:10.867 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:10.867 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:10.953 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T19:54:10.955 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-04-22T19:54:10.984 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:10.984 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:11.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.017+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.019+0000 7f4601e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.019+0000 7f4601e62800 20 realm 2024-04-22T19:54:11.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.019+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.019+0000 7f4601e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.019+0000 7f4601e62800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:11.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.019+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.019+0000 7f4601e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.019+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.020+0000 7f4601e62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:11.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.020+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.020+0000 7f4601e62800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:11.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.020+0000 7f4601e62800 20 searching for the correct realm 2024-04-22T19:54:11.023 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.023+0000 7f4600162640 0 -- 172.21.15.136:0/3348323677 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f45f8096040 msgr2=0x7f45f8098950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:11.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.229+0000 7f45f7fff640 0 -- 172.21.15.136:0/3348323677 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f45f805c040 msgr2=0x7f45f8096960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:11.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:11.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:11.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:11.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:11.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:11.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:11.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.435+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:11.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:11.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 20 zone default found 2024-04-22T19:54:11.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 4 Realm: () 2024-04-22T19:54:11.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 20 zonegroup default 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.436+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.437+0000 7f4601e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.437+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.437+0000 7f4601e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.437+0000 7f4601e62800 20 started sync module instance, tier type = 2024-04-22T19:54:11.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.437+0000 7f4601e62800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:11.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.448+0000 7f4601e62800 20 add_watcher() i=0 2024-04-22T19:54:11.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.448+0000 7f4601e62800 20 add_watcher() i=1 2024-04-22T19:54:11.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.448+0000 7f4601e62800 20 add_watcher() i=2 2024-04-22T19:54:11.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.448+0000 7f4601e62800 20 add_watcher() i=3 2024-04-22T19:54:11.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.448+0000 7f4601e62800 20 add_watcher() i=4 2024-04-22T19:54:11.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.448+0000 7f4601e62800 20 add_watcher() i=5 2024-04-22T19:54:11.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.448+0000 7f4601e62800 20 add_watcher() i=6 2024-04-22T19:54:11.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.449+0000 7f4601e62800 20 add_watcher() i=7 2024-04-22T19:54:11.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.449+0000 7f4601e62800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:11.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.450+0000 7f44b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:11.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.451+0000 7f4601e62800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:11.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.451+0000 7f4601e62800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:11.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.451+0000 7f4601e62800 5 note: GC not initialized 2024-04-22T19:54:11.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.451+0000 7f44b4fe1640 20 reqs_thread_entry: start 2024-04-22T19:54:11.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.451+0000 7f4601e62800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:11.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.451+0000 7f4601e62800 20 Filter name: none 2024-04-22T19:54:11.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.451+0000 7f442e7fc640 20 reqs_thread_entry: start 2024-04-22T19:54:11.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.451+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.452+0000 7f4601e62800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.452+0000 7f4601e62800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.452+0000 7f4601e62800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.452+0000 7f4601e62800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.452+0000 7f4601e62800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.452+0000 7f4601e62800 20 INFO: main: starting shard=0 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.452+0000 7f4601e62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.453+0000 7f4601e62800 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-04-22T19:54:11.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.453+0000 7f4601e62800 20 INFO: main: done 2024-04-22T19:54:11.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.454+0000 7f44b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:11.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.456+0000 7f4601e62800 20 remove_watcher() i=0 2024-04-22T19:54:11.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.456+0000 7f4601e62800 2 removed watcher, disabling cache 2024-04-22T19:54:11.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.457+0000 7f4601e62800 20 remove_watcher() i=1 2024-04-22T19:54:11.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.458+0000 7f4601e62800 20 remove_watcher() i=2 2024-04-22T19:54:11.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.459+0000 7f4601e62800 20 remove_watcher() i=3 2024-04-22T19:54:11.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.460+0000 7f4601e62800 20 remove_watcher() i=4 2024-04-22T19:54:11.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.461+0000 7f4601e62800 20 remove_watcher() i=5 2024-04-22T19:54:11.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.462+0000 7f4601e62800 20 remove_watcher() i=6 2024-04-22T19:54:11.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.464+0000 7f4601e62800 20 remove_watcher() i=7 2024-04-22T19:54:11.475 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:11.475 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:11.498 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:11.499 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:11.582 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T19:54:11.585 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-04-22T19:54:11.613 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:11.613 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:11.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.646+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.648+0000 7f2fc3236800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.648+0000 7f2fc3236800 20 realm 2024-04-22T19:54:11.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.648+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.648+0000 7f2fc3236800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.648+0000 7f2fc3236800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:11.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.648+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.648+0000 7f2fc3236800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:11.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.648+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.649+0000 7f2fc3236800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:11.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.649+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:11.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.650+0000 7f2fc3236800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:11.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.650+0000 7f2fc3236800 20 searching for the correct realm 2024-04-22T19:54:11.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.652+0000 7f2fc1536640 0 -- 172.21.15.136:0/1596805402 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f2fbc033410 msgr2=0x7f2fbc033870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:11.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:11.858+0000 7f2fbbfff640 0 -- 172.21.15.136:0/1596805402 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f2fbc057060 msgr2=0x7f2fbc059440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.064+0000 7f2fc3236800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.064+0000 7f2fc3236800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.064+0000 7f2fc3236800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.064+0000 7f2fc3236800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.064+0000 7f2fc3236800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.064+0000 7f2fc3236800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.064+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 20 zone default found 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 4 Realm: () 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 20 zonegroup default 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.065+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.066+0000 7f2fc3236800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.066+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.066+0000 7f2fc3236800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.066+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.066+0000 7f2fc3236800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.066+0000 7f2fc3236800 20 started sync module instance, tier type = 2024-04-22T19:54:12.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.066+0000 7f2fc3236800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:12.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.083+0000 7f2fc3236800 20 add_watcher() i=0 2024-04-22T19:54:12.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.083+0000 7f2fc3236800 20 add_watcher() i=1 2024-04-22T19:54:12.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.083+0000 7f2fc3236800 20 add_watcher() i=2 2024-04-22T19:54:12.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.083+0000 7f2fc3236800 20 add_watcher() i=3 2024-04-22T19:54:12.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.083+0000 7f2fc3236800 20 add_watcher() i=4 2024-04-22T19:54:12.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.083+0000 7f2fc3236800 20 add_watcher() i=5 2024-04-22T19:54:12.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.083+0000 7f2fc3236800 20 add_watcher() i=6 2024-04-22T19:54:12.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.086+0000 7f2fc3236800 20 add_watcher() i=7 2024-04-22T19:54:12.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.086+0000 7f2fc3236800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:12.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.088+0000 7f2e76fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:12.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.088+0000 7f2fc3236800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:12.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.088+0000 7f2fc3236800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:12.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.088+0000 7f2fc3236800 5 note: GC not initialized 2024-04-22T19:54:12.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.088+0000 7f2e75fe3640 20 reqs_thread_entry: start 2024-04-22T19:54:12.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.088+0000 7f2fc3236800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:12.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.088+0000 7f2fc3236800 20 Filter name: none 2024-04-22T19:54:12.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2de77fe640 20 reqs_thread_entry: start 2024-04-22T19:54:12.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2fc3236800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:12.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2fc3236800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2fc3236800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:12.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2fc3236800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:12.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2fc3236800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:12.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2fc3236800 20 INFO: main: starting shard=0 2024-04-22T19:54:12.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.089+0000 7f2fc3236800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:12.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.090+0000 7f2fc3236800 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-22T19:54:12.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.090+0000 7f2fc3236800 20 INFO: main: done 2024-04-22T19:54:12.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.091+0000 7f2e76fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:12.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.093+0000 7f2fc3236800 20 remove_watcher() i=0 2024-04-22T19:54:12.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.093+0000 7f2fc3236800 2 removed watcher, disabling cache 2024-04-22T19:54:12.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.094+0000 7f2fc3236800 20 remove_watcher() i=1 2024-04-22T19:54:12.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.096+0000 7f2fc3236800 20 remove_watcher() i=2 2024-04-22T19:54:12.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.097+0000 7f2fc3236800 20 remove_watcher() i=3 2024-04-22T19:54:12.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.098+0000 7f2fc3236800 20 remove_watcher() i=4 2024-04-22T19:54:12.099 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.099+0000 7f2fc3236800 20 remove_watcher() i=5 2024-04-22T19:54:12.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.100+0000 7f2fc3236800 20 remove_watcher() i=6 2024-04-22T19:54:12.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.101+0000 7f2fc3236800 20 remove_watcher() i=7 2024-04-22T19:54:12.113 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:12.113 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:12.137 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:12.137 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:12.220 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T19:54:12.222 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-04-22T19:54:12.251 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:12.251 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:12.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.285+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.287+0000 7f51be134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.287+0000 7f51be134800 20 realm 2024-04-22T19:54:12.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.287+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.287+0000 7f51be134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.287+0000 7f51be134800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:12.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.287+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.287+0000 7f51be134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.287+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.288+0000 7f51be134800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:12.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.288+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.289+0000 7f51be134800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:12.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.289+0000 7f51be134800 20 searching for the correct realm 2024-04-22T19:54:12.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.291+0000 7f51b3fff640 0 -- 172.21.15.136:0/1927448980 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f51ac05b040 msgr2=0x7f51ac070480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:12.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.295+0000 7f51b3fff640 0 -- 172.21.15.136:0/1927448980 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f51b4055970 msgr2=0x7f51b40523e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:12.501 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:12.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:12.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:12.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:12.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:12.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:12.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.501+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 20 zone default found 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 4 Realm: () 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 20 zonegroup default 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.502+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.503+0000 7f51be134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.503+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.503+0000 7f51be134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:12.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.503+0000 7f51be134800 20 started sync module instance, tier type = 2024-04-22T19:54:12.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.503+0000 7f51be134800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:12.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.518+0000 7f51be134800 20 add_watcher() i=0 2024-04-22T19:54:12.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.518+0000 7f51be134800 20 add_watcher() i=1 2024-04-22T19:54:12.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.518+0000 7f51be134800 20 add_watcher() i=2 2024-04-22T19:54:12.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.518+0000 7f51be134800 20 add_watcher() i=3 2024-04-22T19:54:12.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.518+0000 7f51be134800 20 add_watcher() i=4 2024-04-22T19:54:12.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.518+0000 7f51be134800 20 add_watcher() i=5 2024-04-22T19:54:12.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.518+0000 7f51be134800 20 add_watcher() i=6 2024-04-22T19:54:12.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.520+0000 7f51be134800 20 add_watcher() i=7 2024-04-22T19:54:12.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.520+0000 7f51be134800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:12.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.522+0000 7f506b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.522+0000 7f51be134800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.522+0000 7f51be134800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.522+0000 7f51be134800 5 note: GC not initialized 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.522+0000 7f506a7fc640 20 reqs_thread_entry: start 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.522+0000 7f51be134800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.522+0000 7f51be134800 20 Filter name: none 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f4fdbfff640 20 reqs_thread_entry: start 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f51be134800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f51be134800 20 rados->read ofs=0 len=0 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f51be134800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:12.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f51be134800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:12.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f51be134800 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-04-22T19:54:12.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f51be134800 20 INFO: main: starting shard=0 2024-04-22T19:54:12.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.523+0000 7f51be134800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:12.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.524+0000 7f51be134800 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-22T19:54:12.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.525+0000 7f51be134800 20 INFO: main: done 2024-04-22T19:54:12.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.525+0000 7f506b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:12.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.528+0000 7f51be134800 20 remove_watcher() i=0 2024-04-22T19:54:12.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.528+0000 7f51be134800 2 removed watcher, disabling cache 2024-04-22T19:54:12.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.529+0000 7f51be134800 20 remove_watcher() i=1 2024-04-22T19:54:12.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.531+0000 7f51be134800 20 remove_watcher() i=2 2024-04-22T19:54:12.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.532+0000 7f51be134800 20 remove_watcher() i=3 2024-04-22T19:54:12.533 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.533+0000 7f51be134800 20 remove_watcher() i=4 2024-04-22T19:54:12.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.534+0000 7f51be134800 20 remove_watcher() i=5 2024-04-22T19:54:12.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.535+0000 7f51be134800 20 remove_watcher() i=6 2024-04-22T19:54:12.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:12.537+0000 7f51be134800 20 remove_watcher() i=7 2024-04-22T19:54:12.550 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:12.550 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:12.576 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-04-22T19:54:12.971 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-04-22T19:54:13.367 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-04-22T19:54:13.395 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:13.395 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:13.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.429+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.430+0000 7f7bf6013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.430+0000 7f7bf6013800 20 realm 2024-04-22T19:54:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.430+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.430+0000 7f7bf6013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.430+0000 7f7bf6013800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.430+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.430+0000 7f7bf6013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.430+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.431+0000 7f7bf6013800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:13.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.431+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.432+0000 7f7bf6013800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:13.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.432+0000 7f7bf6013800 20 searching for the correct realm 2024-04-22T19:54:13.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.434+0000 7f7bf1f48640 0 -- 172.21.15.136:0/682088414 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f7bec0ada90 msgr2=0x7f7bec009250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:13.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.640+0000 7f7bf0f46640 0 -- 172.21.15.136:0/682088414 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f7bec0bd6a0 msgr2=0x7f7bec0c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:13.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:13.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:13.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:13.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:13.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:13.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:13.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:13.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.846+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:13.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 20 zone default found 2024-04-22T19:54:13.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 4 Realm: () 2024-04-22T19:54:13.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:13.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:13.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:13.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 20 zonegroup default 2024-04-22T19:54:13.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.847+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.848+0000 7f7bf6013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.848+0000 7f7bf6013800 20 started sync module instance, tier type = 2024-04-22T19:54:13.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.848+0000 7f7bf6013800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:13.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.859+0000 7f7bf6013800 20 add_watcher() i=0 2024-04-22T19:54:13.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.859+0000 7f7bf6013800 20 add_watcher() i=1 2024-04-22T19:54:13.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.859+0000 7f7bf6013800 20 add_watcher() i=2 2024-04-22T19:54:13.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.859+0000 7f7bf6013800 20 add_watcher() i=3 2024-04-22T19:54:13.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.859+0000 7f7bf6013800 20 add_watcher() i=4 2024-04-22T19:54:13.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.859+0000 7f7bf6013800 20 add_watcher() i=5 2024-04-22T19:54:13.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.859+0000 7f7bf6013800 20 add_watcher() i=6 2024-04-22T19:54:13.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.862+0000 7f7bf6013800 20 add_watcher() i=7 2024-04-22T19:54:13.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.862+0000 7f7bf6013800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:13.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.864+0000 7f7aa7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:13.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.864+0000 7f7bf6013800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:13.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.864+0000 7f7bf6013800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:13.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.864+0000 7f7bf6013800 5 note: GC not initialized 2024-04-22T19:54:13.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.864+0000 7f7aa6fe5640 20 reqs_thread_entry: start 2024-04-22T19:54:13.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:13.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 20 Filter name: none 2024-04-22T19:54:13.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7aa4fe1640 20 reqs_thread_entry: start 2024-04-22T19:54:13.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:54:13.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:13.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:54:13.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:54:13.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:54:13.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:13.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:54:13.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:54:13.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:54:13.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:54:13.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:54:13.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.865+0000 7f7bf6013800 20 list_objects_ordered: starting attempt 1 2024-04-22T19:54:13.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.866+0000 7f7bf6013800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T19:54:13.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.866+0000 7f7bf6013800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T19:54:13.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.866+0000 7f7bf6013800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T19:54:13.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.866+0000 7f7bf6013800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T19:54:13.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.866+0000 7f7bf6013800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T19:54:13.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.869+0000 7f7aa7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:13.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.872+0000 7f7bf6013800 20 remove_watcher() i=0 2024-04-22T19:54:13.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.872+0000 7f7bf6013800 2 removed watcher, disabling cache 2024-04-22T19:54:13.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.873+0000 7f7bf6013800 20 remove_watcher() i=1 2024-04-22T19:54:13.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.874+0000 7f7bf6013800 20 remove_watcher() i=2 2024-04-22T19:54:13.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.875+0000 7f7bf6013800 20 remove_watcher() i=3 2024-04-22T19:54:13.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.877+0000 7f7bf6013800 20 remove_watcher() i=4 2024-04-22T19:54:13.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.877+0000 7f7bf6013800 20 remove_watcher() i=5 2024-04-22T19:54:13.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.879+0000 7f7bf6013800 20 remove_watcher() i=6 2024-04-22T19:54:13.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.879+0000 7f7bf6013800 20 remove_watcher() i=7 2024-04-22T19:54:13.890 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-04-22T19:54:13.925 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:13.925 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:13.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.958+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.959+0000 7fc17a340800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.959+0000 7fc17a340800 20 realm 2024-04-22T19:54:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.959+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.960+0000 7fc17a340800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.960+0000 7fc17a340800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.960+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.960+0000 7fc17a340800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.960+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.960+0000 7fc17a340800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:13.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.960+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:13.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.961+0000 7fc17a340800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:13.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.961+0000 7fc17a340800 20 searching for the correct realm 2024-04-22T19:54:13.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:13.963+0000 7fc178640640 0 -- 172.21.15.136:0/1282343750 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fc170079c20 msgr2=0x7fc170059270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:14.169 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.169+0000 7fc17763e640 0 -- 172.21.15.136:0/1282343750 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fc170017020 msgr2=0x7fc17008a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.375+0000 7fc17a340800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.375+0000 7fc17a340800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.375+0000 7fc17a340800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.375+0000 7fc17a340800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.375+0000 7fc17a340800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.375+0000 7fc17a340800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.375+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.376+0000 7fc17a340800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.376+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.376+0000 7fc17a340800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:14.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.376+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.376+0000 7fc17a340800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 zone default found 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 4 Realm: () 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 zonegroup default 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 started sync module instance, tier type = 2024-04-22T19:54:14.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.377+0000 7fc17a340800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:14.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.391+0000 7fc17a340800 20 add_watcher() i=0 2024-04-22T19:54:14.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.391+0000 7fc17a340800 20 add_watcher() i=1 2024-04-22T19:54:14.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.391+0000 7fc17a340800 20 add_watcher() i=2 2024-04-22T19:54:14.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.391+0000 7fc17a340800 20 add_watcher() i=3 2024-04-22T19:54:14.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.391+0000 7fc17a340800 20 add_watcher() i=4 2024-04-22T19:54:14.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.391+0000 7fc17a340800 20 add_watcher() i=5 2024-04-22T19:54:14.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.391+0000 7fc17a340800 20 add_watcher() i=6 2024-04-22T19:54:14.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.394+0000 7fc17a340800 20 add_watcher() i=7 2024-04-22T19:54:14.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.394+0000 7fc17a340800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:14.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.396+0000 7fc02dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:14.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.396+0000 7fc17a340800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:14.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.396+0000 7fc17a340800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:14.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.396+0000 7fc17a340800 5 note: GC not initialized 2024-04-22T19:54:14.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc02cfe1640 20 reqs_thread_entry: start 2024-04-22T19:54:14.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:14.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 20 Filter name: none 2024-04-22T19:54:14.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fbfa67fc640 20 reqs_thread_entry: start 2024-04-22T19:54:14.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-04-22T19:54:14.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 : miss 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.397+0000 7fc17a340800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.398+0000 7fc17a340800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.398+0000 7fc17a340800 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 info.flags=0x17 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.398+0000 7fc17a340800 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 to cache LRU end 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.398+0000 7fc17a340800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.398+0000 7fc17a340800 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.398+0000 7fc17a340800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1 2024-04-22T19:54:14.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.398+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.399+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.399+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:c state=0x5590bba881a8 s->prefetch_data=0 2024-04-22T19:54:14.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.400+0000 7fc17a340800 20 get_obj_state: setting s->obj_tag to 0jxf9u2fnld8kev4carrplwkuanzsj8n 2024-04-22T19:54:14.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.400+0000 7fc17a340800 20 get_obj_state_impl: setting s->olh_tag to sgzt2see532087h9t4xwazj40uwiuuat 2024-04-22T19:54:14.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.400+0000 7fc17a340800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=sgzt2see532087h9t4xwazj40uwiuuat 2024-04-22T19:54:14.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.402+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.405+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.406+0000 7fc17a340800 20 olh_log_entry: epoch=3 op=3 key=c[pZwvkH4HQ73jlO3dCYhY.lTzRS8XCn9] 2024-04-22T19:54:14.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.406+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:_:pZwvkH4HQ73jlO3dCYhY.lTzRS8XCn9_c state=0x5590bbacb368 s->prefetch_data=0 2024-04-22T19:54:14.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.406+0000 7fc17a340800 10 manifest: total_size = 11 2024-04-22T19:54:14.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.406+0000 7fc17a340800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13930749923848752128 2024-04-22T19:54:14.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.406+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:_:pZwvkH4HQ73jlO3dCYhY.lTzRS8XCn9_c state=0x5590bbacb368 s->prefetch_data=0 2024-04-22T19:54:14.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.406+0000 7fc17a340800 20 prepare_atomic_modification: state is not atomic. state=0x5590bbacb368 2024-04-22T19:54:14.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.406+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.409+0000 7fc17a340800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T19:54:14.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.409+0000 7fc17a340800 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2024-04-22T19:54:14.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.409+0000 7fc17a340800 0 deleting objects inline since gc isn't initialized 2024-04-22T19:54:14.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.409+0000 7fc17a340800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1__shadow:pZwvkH4HQ73jlO3dCYhY.lTzRS8XCn9_.iiA8k3CVmQx9oav84XSrV0rfwo442Ho_0 2024-04-22T19:54:14.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.411+0000 7fc056ffd640 20 handle_completion(): completion ok for obj=c[pZwvkH4HQ73jlO3dCYhY.lTzRS8XCn9] 2024-04-22T19:54:14.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.412+0000 7fc17a340800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T19:54:14.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.416+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.416+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.420+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.420 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.420+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.420 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.420+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:d state=0x5590bbacdff8 s->prefetch_data=0 2024-04-22T19:54:14.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.421+0000 7fc17a340800 20 get_obj_state: setting s->obj_tag to byhwc6ym23eyhzc85vlqe4hodnfhdv71 2024-04-22T19:54:14.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.421+0000 7fc17a340800 20 get_obj_state_impl: setting s->olh_tag to mv570g4gwjbpb7pwt6a09lwy64pd5lnt 2024-04-22T19:54:14.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.421+0000 7fc17a340800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=mv570g4gwjbpb7pwt6a09lwy64pd5lnt 2024-04-22T19:54:14.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.422+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.426+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.426+0000 7fc17a340800 20 olh_log_entry: epoch=3 op=3 key=d[9S3dbqrDVjLMILuGB9PyBI88OBKpEVx] 2024-04-22T19:54:14.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.426+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:_:9S3dbqrDVjLMILuGB9PyBI88OBKpEVx_d state=0x5590bbaca358 s->prefetch_data=0 2024-04-22T19:54:14.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.427+0000 7fc17a340800 10 manifest: total_size = 9 2024-04-22T19:54:14.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.427+0000 7fc17a340800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9302430133569012361 2024-04-22T19:54:14.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.427+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:_:9S3dbqrDVjLMILuGB9PyBI88OBKpEVx_d state=0x5590bbaca358 s->prefetch_data=0 2024-04-22T19:54:14.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.427+0000 7fc17a340800 20 prepare_atomic_modification: state is not atomic. state=0x5590bbaca358 2024-04-22T19:54:14.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.427+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.430+0000 7fc17a340800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T19:54:14.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.430+0000 7fc17a340800 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-04-22T19:54:14.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.430+0000 7fc17a340800 0 deleting objects inline since gc isn't initialized 2024-04-22T19:54:14.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.430+0000 7fc17a340800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1__shadow:9S3dbqrDVjLMILuGB9PyBI88OBKpEVx_.KoTs8diYBm_GK5xbukoV5wYgn6TpI60_0 2024-04-22T19:54:14.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.433+0000 7fc17a340800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T19:54:14.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.433+0000 7fc0567fc640 20 handle_completion(): completion ok for obj=d[9S3dbqrDVjLMILuGB9PyBI88OBKpEVx] 2024-04-22T19:54:14.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.434+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.434+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.436+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.436+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.436+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:e state=0x5590bbacdff8 s->prefetch_data=0 2024-04-22T19:54:14.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.437+0000 7fc17a340800 20 get_obj_state: setting s->obj_tag to 7zv8ki0p3d9aihrrcc2u8os7r0mm20gk 2024-04-22T19:54:14.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.437+0000 7fc17a340800 20 get_obj_state_impl: setting s->olh_tag to 9y27pckton4zchd304hhdsy2rn3b7pnd 2024-04-22T19:54:14.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.437+0000 7fc17a340800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=9y27pckton4zchd304hhdsy2rn3b7pnd 2024-04-22T19:54:14.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.438+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-04-22T19:54:14.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 20 olh_log_entry: epoch=2 op=3 key=e[veedsWXqX1SmYw6rK150WOXQIHI0-Zm] 2024-04-22T19:54:14.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:_:veedsWXqX1SmYw6rK150WOXQIHI0-Zm_e state=0x5590bba8b908 s->prefetch_data=0 2024-04-22T19:54:14.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 10 manifest: total_size = 9 2024-04-22T19:54:14.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1895177074165250612 2024-04-22T19:54:14.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:_:veedsWXqX1SmYw6rK150WOXQIHI0-Zm_e state=0x5590bba8b908 s->prefetch_data=0 2024-04-22T19:54:14.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 20 prepare_atomic_modification: state is not atomic. state=0x5590bba8b908 2024-04-22T19:54:14.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.440+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.443+0000 7fc17a340800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T19:54:14.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.443+0000 7fc17a340800 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-04-22T19:54:14.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.443+0000 7fc17a340800 0 deleting objects inline since gc isn't initialized 2024-04-22T19:54:14.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.443+0000 7fc17a340800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1__shadow:veedsWXqX1SmYw6rK150WOXQIHI0-Zm_.9wJycppiWJspj7pEDz7XRQxe4TBbwgS_0 2024-04-22T19:54:14.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.444+0000 7fc056ffd640 20 handle_completion(): completion ok for obj=e[veedsWXqX1SmYw6rK150WOXQIHI0-Zm] 2024-04-22T19:54:14.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.446+0000 7fc17a340800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T19:54:14.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.447+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:e state=0x5590bbacdff8 s->prefetch_data=0 2024-04-22T19:54:14.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.447+0000 7fc17a340800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=9y27pckton4zchd304hhdsy2rn3b7pnd 2024-04-22T19:54:14.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.454+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.455+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.456+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.456+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:f state=0x5590bbacdff8 s->prefetch_data=0 2024-04-22T19:54:14.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.456+0000 7fc17a340800 20 get_obj_state: setting s->obj_tag to vkvdhixfp823cx38si50nk78epxja0oe 2024-04-22T19:54:14.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.456+0000 7fc17a340800 20 get_obj_state_impl: setting s->olh_tag to y7g2q8ueq6tqvu6rt6k0ipktf7x932ai 2024-04-22T19:54:14.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.456+0000 7fc17a340800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=y7g2q8ueq6tqvu6rt6k0ipktf7x932ai 2024-04-22T19:54:14.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.458+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.461+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.461+0000 7fc17a340800 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-04-22T19:54:14.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.461+0000 7fc17a340800 20 olh_log_entry: epoch=2 op=3 key=f[DE4GR0aD-91Ea.fHG127nlLnWuaGI-R] 2024-04-22T19:54:14.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.461+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:_:DE4GR0aD-91Ea.fHG127nlLnWuaGI-R_f state=0x5590bbacc378 s->prefetch_data=0 2024-04-22T19:54:14.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.461+0000 7fc17a340800 10 manifest: total_size = 9 2024-04-22T19:54:14.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.462+0000 7fc17a340800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3154896755426345352 2024-04-22T19:54:14.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.462+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:_:DE4GR0aD-91Ea.fHG127nlLnWuaGI-R_f state=0x5590bbacc378 s->prefetch_data=0 2024-04-22T19:54:14.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.462+0000 7fc17a340800 20 prepare_atomic_modification: state is not atomic. state=0x5590bbacc378 2024-04-22T19:54:14.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.462+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.465+0000 7fc17a340800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T19:54:14.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.465+0000 7fc17a340800 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-04-22T19:54:14.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.465+0000 7fc17a340800 0 deleting objects inline since gc isn't initialized 2024-04-22T19:54:14.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.465+0000 7fc17a340800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1__shadow:DE4GR0aD-91Ea.fHG127nlLnWuaGI-R_.EN1akL088LSEstQU4RFUir1IrtMZvpX_0 2024-04-22T19:54:14.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.467+0000 7fc0567fc640 20 handle_completion(): completion ok for obj=f[DE4GR0aD-91Ea.fHG127nlLnWuaGI-R] 2024-04-22T19:54:14.468 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.468+0000 7fc17a340800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T19:54:14.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.470+0000 7fc17a340800 20 get_obj_state: octx=0x5590bb7bb570 obj=check-bucket:f state=0x5590bbacdff8 s->prefetch_data=0 2024-04-22T19:54:14.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.470+0000 7fc17a340800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=y7g2q8ueq6tqvu6rt6k0ipktf7x932ai 2024-04-22T19:54:14.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.472+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.474+0000 7fc17a340800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1.0 2024-04-22T19:54:14.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.474+0000 7fc17a340800 1 NOTICE: finished shard 0 (4 entries removed) 2024-04-22T19:54:14.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.474+0000 7fc17a340800 1 NOTICE: finished all shards (4 entries removed) 2024-04-22T19:54:14.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.475+0000 7fc02dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:14.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.477+0000 7fc17a340800 20 remove_watcher() i=0 2024-04-22T19:54:14.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.477+0000 7fc17a340800 2 removed watcher, disabling cache 2024-04-22T19:54:14.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.478+0000 7fc17a340800 20 remove_watcher() i=1 2024-04-22T19:54:14.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.479+0000 7fc17a340800 20 remove_watcher() i=2 2024-04-22T19:54:14.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.480+0000 7fc17a340800 20 remove_watcher() i=3 2024-04-22T19:54:14.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.481+0000 7fc17a340800 20 remove_watcher() i=4 2024-04-22T19:54:14.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.482+0000 7fc17a340800 20 remove_watcher() i=5 2024-04-22T19:54:14.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.483+0000 7fc17a340800 20 remove_watcher() i=6 2024-04-22T19:54:14.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.484+0000 7fc17a340800 20 remove_watcher() i=7 2024-04-22T19:54:14.496 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:14.496 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:14.525 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:14.525 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:14.704 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-04-22T19:54:14.732 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:54:14.732 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:54:14.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.765+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.766+0000 7f13b4d25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.766+0000 7f13b4d25800 20 realm 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.766+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.766+0000 7f13b4d25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.766+0000 7f13b4d25800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.766+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.766+0000 7f13b4d25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.766+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.767+0000 7f13b4d25800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:14.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.767+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.768+0000 7f13b4d25800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T19:54:14.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.768+0000 7f13b4d25800 20 searching for the correct realm 2024-04-22T19:54:14.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.769+0000 7f13b12b8640 0 -- 172.21.15.136:0/2669210889 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x556fdc572810 msgr2=0x556fdc574c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:14.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.773+0000 7f13b12b8640 0 -- 172.21.15.136:0/2669210889 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f127c078f40 msgr2=0x7f127c07b370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T19:54:14.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T19:54:14.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T19:54:14.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.778+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 20 zone default found 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 4 Realm: () 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T19:54:14.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T19:54:14.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 20 zonegroup default 2024-04-22T19:54:14.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.779+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.780+0000 7f13b4d25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.780+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.780+0000 7f13b4d25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T19:54:14.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.780+0000 7f13b4d25800 20 started sync module instance, tier type = 2024-04-22T19:54:14.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.780+0000 7f13b4d25800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T19:54:14.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.789+0000 7f13b4d25800 20 add_watcher() i=0 2024-04-22T19:54:14.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.789+0000 7f13b4d25800 20 add_watcher() i=1 2024-04-22T19:54:14.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.789+0000 7f13b4d25800 20 add_watcher() i=2 2024-04-22T19:54:14.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.789+0000 7f13b4d25800 20 add_watcher() i=3 2024-04-22T19:54:14.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.789+0000 7f13b4d25800 20 add_watcher() i=4 2024-04-22T19:54:14.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.789+0000 7f13b4d25800 20 add_watcher() i=5 2024-04-22T19:54:14.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.789+0000 7f13b4d25800 20 add_watcher() i=6 2024-04-22T19:54:14.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.790+0000 7f13b4d25800 20 add_watcher() i=7 2024-04-22T19:54:14.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.790+0000 7f13b4d25800 2 all 8 watchers are set, enabling cache 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.792+0000 7f1266fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.792+0000 7f13b4d25800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.792+0000 7f13b4d25800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.792+0000 7f13b4d25800 5 note: GC not initialized 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.792+0000 7f1265fe3640 20 reqs_thread_entry: start 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f13b4d25800 20 init_complete bucket index max shards: 1 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f13b4d25800 20 Filter name: none 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f11d77fe640 20 reqs_thread_entry: start 2024-04-22T19:54:14.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f13b4d25800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:14.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f13b4d25800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:14.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f13b4d25800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:14.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.793+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.794+0000 7f13b4d25800 20 rados_obj.operate() r=0 bl.length=190 2024-04-22T19:54:14.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.794+0000 7f13b4d25800 20 rados->read ofs=0 len=0 2024-04-22T19:54:14.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.794+0000 7f13b4d25800 20 rados_obj.operate() r=0 bl.length=404 2024-04-22T19:54:14.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.794+0000 7f13b4d25800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T19:54:14.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.795+0000 7f1266fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T19:54:14.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.797+0000 7f13b4d25800 20 remove_watcher() i=0 2024-04-22T19:54:14.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.797+0000 7f13b4d25800 2 removed watcher, disabling cache 2024-04-22T19:54:14.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.798+0000 7f13b4d25800 20 remove_watcher() i=1 2024-04-22T19:54:14.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.799+0000 7f13b4d25800 20 remove_watcher() i=2 2024-04-22T19:54:14.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.800+0000 7f13b4d25800 20 remove_watcher() i=3 2024-04-22T19:54:14.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.801+0000 7f13b4d25800 20 remove_watcher() i=4 2024-04-22T19:54:14.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.802+0000 7f13b4d25800 20 remove_watcher() i=5 2024-04-22T19:54:14.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.803+0000 7f13b4d25800 20 remove_watcher() i=6 2024-04-22T19:54:14.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:54:14.804+0000 7f13b4d25800 20 remove_watcher() i=7 2024-04-22T19:54:14.814 INFO:tasks.workunit.client.0.smithi136.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-04-22T19:54:14.814 INFO:tasks.workunit.client.0.smithi136.stderr:Deleting bucket check-bucket 2024-04-22T19:54:14.815 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:14.815 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:14.840 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T19:54:14.840 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T19:54:14.973 INFO:tasks.workunit.client.0.smithi136.stderr:Completed bucket check tests 2024-04-22T19:54:14.973 INFO:tasks.workunit.client.0.smithi136.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2024-04-22T19:54:14.973 INFO:tasks.workunit.client.0.smithi136.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2024-04-22T19:54:14.973 INFO:tasks.workunit.client.0.smithi136.stdout:connected to https://localhost:443 2024-04-22T19:54:15.038 INFO:tasks.workunit.client.0.smithi136.stderr:+ deactivate 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -n /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin ']' 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ PATH=/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ export PATH 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ unset _OLD_VIRTUAL_PATH 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -n '' ']' 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ hash -r 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -n '' ']' 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ unset VIRTUAL_ENV 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' '!' '' = nondestructive ']' 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ unset -f deactivate 2024-04-22T19:54:15.039 INFO:tasks.workunit.client.0.smithi136.stderr:+ echo OK. 2024-04-22T19:54:15.040 INFO:tasks.workunit.client.0.smithi136.stdout:OK. 2024-04-22T19:54:15.040 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T19:54:15.041 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T19:54:15.117 INFO:tasks.workunit:Stopping ['rgw/run-bucket-check.sh'] on client.0... 2024-04-22T19:54:15.117 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T19:54:15.710 DEBUG:teuthology.parallel:result is None 2024-04-22T19:54:15.710 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T19:54:15.746 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T19:54:15.746 DEBUG:teuthology.orchestra.run.smithi136:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T19:54:15.845 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T19:54:15.845 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T19:54:15.855 INFO:tasks.workunit:Pulling workunits from ref 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:54:15.855 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T19:54:15.855 DEBUG:teuthology.orchestra.run.smithi136:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T19:54:15.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T19:54:15.882 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-22T19:54:15.883 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T19:54:15.950 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-22T19:54:15.950 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-22T19:54:16.017 INFO:tasks.workunit:timeout=3h 2024-04-22T19:54:16.017 INFO:tasks.workunit:cleanup=True 2024-04-22T19:54:16.017 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T19:54:16.086 INFO:tasks.workunit.client.0.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T19:55:43.463 INFO:tasks.workunit.client.0.smithi136.stderr:Note: switching to '9ed8263edff7e87907bf24b054e5c7144c194dbe'. 2024-04-22T19:55:43.463 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:55:43.463 INFO:tasks.workunit.client.0.smithi136.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T19:55:43.463 INFO:tasks.workunit.client.0.smithi136.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T19:55:43.463 INFO:tasks.workunit.client.0.smithi136.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T19:55:43.463 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr: git switch -c 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr:Or undo this operation with: 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr: git switch - 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T19:55:43.464 INFO:tasks.workunit.client.0.smithi136.stderr:HEAD is now at 9ed8263edff rgw: apply default quota config on account creation 2024-04-22T19:55:43.469 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T19:55:43.535 INFO:tasks.workunit.client.0.smithi136.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T19:55:43.537 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T19:55:43.537 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T19:55:43.585 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T19:55:43.634 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T19:55:43.673 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T19:55:43.675 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T19:55:43.675 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T19:55:43.714 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T19:55:43.717 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T19:55:43.717 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T19:55:43.784 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2024-04-22T19:55:43.785 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2024-04-22T19:55:43.785 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2024-04-22T19:55:43.864 INFO:tasks.workunit.client.0.smithi136.stdout:Running main() from gmock_main.cc 2024-04-22T19:55:43.865 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 11 tests from 1 test suite. 2024-04-22T19:55:43.865 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T19:55:43.865 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 11 tests from ClsLock 2024-04-22T19:55:43.865 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestMultiLocking 2024-04-22T19:55:45.808 INFO:tasks.workunit.client.0.smithi136.stdout:lock: mylock 2024-04-22T19:55:45.808 INFO:tasks.workunit.client.0.smithi136.stdout: lock_type: exclusive 2024-04-22T19:55:45.808 INFO:tasks.workunit.client.0.smithi136.stdout: tag: 2024-04-22T19:55:45.808 INFO:tasks.workunit.client.0.smithi136.stdout: lockers: 2024-04-22T19:55:45.808 INFO:tasks.workunit.client.0.smithi136.stdout: client.5015 expiration=2024-04-22T19:57:45.800999+0000 addr=v1:172.21.15.136:0/2897768190 cookie= 2024-04-22T19:55:45.814 INFO:tasks.workunit.client.0.smithi136.stdout:lock: mylock 2024-04-22T19:55:45.814 INFO:tasks.workunit.client.0.smithi136.stdout: lock_type: shared 2024-04-22T19:55:45.814 INFO:tasks.workunit.client.0.smithi136.stdout: tag: 2024-04-22T19:55:45.814 INFO:tasks.workunit.client.0.smithi136.stdout: lockers: 2024-04-22T19:55:45.814 INFO:tasks.workunit.client.0.smithi136.stdout: client.5015 expiration=2024-04-22T19:57:45.814114+0000 addr=v1:172.21.15.136:0/2897768190 cookie= 2024-04-22T19:55:45.815 INFO:tasks.workunit.client.0.smithi136.stdout: client.5021 expiration=0.000000 addr=v1:172.21.15.136:0/3825060892 cookie= 2024-04-22T19:55:45.816 INFO:tasks.workunit.client.0.smithi136.stdout:lock: mylock 2024-04-22T19:55:45.816 INFO:tasks.workunit.client.0.smithi136.stdout: lock_type: shared 2024-04-22T19:55:45.817 INFO:tasks.workunit.client.0.smithi136.stdout: tag: 2024-04-22T19:55:45.817 INFO:tasks.workunit.client.0.smithi136.stdout: lockers: 2024-04-22T19:55:45.817 INFO:tasks.workunit.client.0.smithi136.stdout: client.5021 expiration=0.000000 addr=v1:172.21.15.136:0/3825060892 cookie= 2024-04-22T19:55:46.215 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestMultiLocking (2350 ms) 2024-04-22T19:55:46.215 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestMeta 2024-04-22T19:55:48.474 INFO:tasks.workunit.client.0.smithi136.stdout:lock: mylock 2024-04-22T19:55:48.474 INFO:tasks.workunit.client.0.smithi136.stdout: lock_type: shared 2024-04-22T19:55:48.474 INFO:tasks.workunit.client.0.smithi136.stdout: tag: 2024-04-22T19:55:48.474 INFO:tasks.workunit.client.0.smithi136.stdout: lockers: 2024-04-22T19:55:48.474 INFO:tasks.workunit.client.0.smithi136.stdout: client.5033 expiration=0.000000 addr=v1:172.21.15.136:0/2638522334 cookie= 2024-04-22T19:55:48.477 INFO:tasks.workunit.client.0.smithi136.stdout:lock: mylock 2024-04-22T19:55:48.477 INFO:tasks.workunit.client.0.smithi136.stdout: lock_type: exclusive 2024-04-22T19:55:48.477 INFO:tasks.workunit.client.0.smithi136.stdout: tag: new_tag 2024-04-22T19:55:48.477 INFO:tasks.workunit.client.0.smithi136.stdout: lockers: 2024-04-22T19:55:48.477 INFO:tasks.workunit.client.0.smithi136.stdout: client.5181 expiration=0.000000 addr=v1:172.21.15.136:0/4166567616 cookie= 2024-04-22T19:55:49.243 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestMeta (3028 ms) 2024-04-22T19:55:49.243 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestCookie 2024-04-22T19:55:51.260 INFO:tasks.workunit.client.0.smithi136.stdout:lock: mylock 2024-04-22T19:55:51.261 INFO:tasks.workunit.client.0.smithi136.stdout: lock_type: exclusive 2024-04-22T19:55:51.261 INFO:tasks.workunit.client.0.smithi136.stdout: tag: 2024-04-22T19:55:51.261 INFO:tasks.workunit.client.0.smithi136.stdout: lockers: 2024-04-22T19:55:51.263 INFO:tasks.workunit.client.0.smithi136.stdout:lock: mylock 2024-04-22T19:55:51.263 INFO:tasks.workunit.client.0.smithi136.stdout: lock_type: shared 2024-04-22T19:55:51.263 INFO:tasks.workunit.client.0.smithi136.stdout: tag: 2024-04-22T19:55:51.263 INFO:tasks.workunit.client.0.smithi136.stdout: lockers: 2024-04-22T19:55:51.263 INFO:tasks.workunit.client.0.smithi136.stdout: client.5196 expiration=0.000000 addr=v1:172.21.15.136:0/1982590116 cookie= 2024-04-22T19:55:51.263 INFO:tasks.workunit.client.0.smithi136.stdout: client.5196 expiration=0.000000 addr=v1:172.21.15.136:0/1982590116 cookie=new cookie 2024-04-22T19:55:52.252 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestCookie (3009 ms) 2024-04-22T19:55:52.252 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestMultipleLocks 2024-04-22T19:55:55.278 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestMultipleLocks (3027 ms) 2024-04-22T19:55:55.279 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestLockDuration 2024-04-22T19:56:02.372 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestLockDuration (7093 ms) 2024-04-22T19:56:02.372 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestAssertLocked 2024-04-22T19:56:05.396 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestAssertLocked (3023 ms) 2024-04-22T19:56:05.396 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestSetCookie 2024-04-22T19:56:08.401 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestSetCookie (3007 ms) 2024-04-22T19:56:08.402 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestRenew 2024-04-22T19:56:29.457 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestRenew (21055 ms) 2024-04-22T19:56:29.457 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2024-04-22T19:56:35.539 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (6082 ms) 2024-04-22T19:56:35.539 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2024-04-22T19:56:43.576 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8037 ms) 2024-04-22T19:56:43.577 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2024-04-22T19:56:50.650 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (7074 ms) 2024-04-22T19:56:50.651 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 11 tests from ClsLock (66786 ms total) 2024-04-22T19:56:50.651 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T19:56:50.651 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T19:56:50.651 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 11 tests from 1 test suite ran. (66786 ms total) 2024-04-22T19:56:50.651 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 11 tests. 2024-04-22T19:56:50.654 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T19:56:50.654 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T19:56:50.735 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2024-04-22T19:56:50.736 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2024-04-22T19:56:50.736 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2024-04-22T19:56:50.818 INFO:tasks.workunit.client.0.smithi136.stdout:Running main() from gmock_main.cc 2024-04-22T19:56:50.818 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 4 tests from 1 test suite. 2024-04-22T19:56:50.818 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T19:56:50.818 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 4 tests from cls_log 2024-04-22T19:56:50.818 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_log.test_log_add_same_time 2024-04-22T19:56:53.689 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_log.test_log_add_same_time (2871 ms) 2024-04-22T19:56:53.689 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_log.test_log_add_different_time 2024-04-22T19:56:56.697 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_log.test_log_add_different_time (3007 ms) 2024-04-22T19:56:56.697 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_log.trim_by_time 2024-04-22T19:56:59.713 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_log.trim_by_time (3017 ms) 2024-04-22T19:56:59.713 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_log.trim_by_marker 2024-04-22T19:57:02.719 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_log.trim_by_marker (3006 ms) 2024-04-22T19:57:02.719 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 4 tests from cls_log (11901 ms total) 2024-04-22T19:57:02.719 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T19:57:02.720 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T19:57:02.720 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 4 tests from 1 test suite ran. (11901 ms total) 2024-04-22T19:57:02.720 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 4 tests. 2024-04-22T19:57:02.723 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T19:57:02.723 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T19:57:02.804 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2024-04-22T19:57:02.806 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2024-04-22T19:57:02.806 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2024-04-22T19:57:02.888 INFO:tasks.workunit.client.0.smithi136.stdout:Running main() from gmock_main.cc 2024-04-22T19:57:02.889 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 10 tests from 1 test suite. 2024-04-22T19:57:02.889 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T19:57:02.889 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 10 tests from cls_refcount 2024-04-22T19:57:02.889 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.test_implicit 2024-04-22T19:57:05.750 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.test_implicit (2861 ms) 2024-04-22T19:57:05.750 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.test_implicit_idempotent 2024-04-22T19:57:08.777 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.test_implicit_idempotent (3026 ms) 2024-04-22T19:57:08.777 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.test_put_snap 2024-04-22T19:57:17.839 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.test_put_snap (9061 ms) 2024-04-22T19:57:17.839 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.test_explicit 2024-04-22T19:57:20.849 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.test_explicit (3011 ms) 2024-04-22T19:57:20.850 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.set 2024-04-22T19:57:23.889 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.set (3040 ms) 2024-04-22T19:57:23.889 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.test_implicit_ec 2024-04-22T19:57:30.917 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.test_implicit_ec (7029 ms) 2024-04-22T19:57:30.917 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.test_implicit_idempotent_ec 2024-04-22T19:57:37.969 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.test_implicit_idempotent_ec (7052 ms) 2024-04-22T19:57:37.969 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.test_put_snap_ec 2024-04-22T19:57:50.049 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.test_put_snap_ec (12081 ms) 2024-04-22T19:57:50.050 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.test_explicit_ec 2024-04-22T19:57:57.099 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.test_explicit_ec (7049 ms) 2024-04-22T19:57:57.099 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_refcount.set_ec 2024-04-22T19:58:04.127 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_refcount.set_ec (7028 ms) 2024-04-22T19:58:04.127 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 10 tests from cls_refcount (61238 ms total) 2024-04-22T19:58:04.127 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T19:58:04.127 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T19:58:04.127 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 10 tests from 1 test suite ran. (61238 ms total) 2024-04-22T19:58:04.127 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 10 tests. 2024-04-22T19:58:04.130 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T19:58:04.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T19:58:04.212 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2024-04-22T19:58:04.213 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2024-04-22T19:58:04.213 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2024-04-22T19:58:04.297 INFO:tasks.workunit.client.0.smithi136.stdout:Running main() from gmock_main.cc 2024-04-22T19:58:04.298 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 16 tests from 1 test suite. 2024-04-22T19:58:04.298 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T19:58:04.298 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 16 tests from cls_rgw 2024-04-22T19:58:06.135 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.index_basic 2024-04-22T19:58:06.172 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.index_basic (38 ms) 2024-04-22T19:58:06.172 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2024-04-22T19:58:06.200 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (27 ms) 2024-04-22T19:58:06.200 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.index_remove_object 2024-04-22T19:58:06.245 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.index_remove_object (46 ms) 2024-04-22T19:58:06.246 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.index_suggest 2024-04-22T19:58:07.736 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.index_suggest (1490 ms) 2024-04-22T19:58:07.736 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.index_suggest_complete 2024-04-22T19:58:07.743 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.index_suggest_complete (8 ms) 2024-04-22T19:58:07.743 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.index_list 2024-04-22T19:58:07.765 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.index_list (22 ms) 2024-04-22T19:58:07.765 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.index_list_delimited 2024-04-22T19:58:27.083 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.index_list_delimited (19317 ms) 2024-04-22T19:58:27.083 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.bi_list 2024-04-22T19:58:27.153 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.bi_list (70 ms) 2024-04-22T19:58:27.153 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.gc_set 2024-04-22T19:58:27.163 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.gc_set (11 ms) 2024-04-22T19:58:27.163 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.gc_list 2024-04-22T19:58:27.174 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.gc_list (10 ms) 2024-04-22T19:58:27.174 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.gc_defer 2024-04-22T19:58:35.202 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.gc_defer (8028 ms) 2024-04-22T19:58:35.203 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.usage_basic 2024-04-22T19:58:35.304 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.usage_basic (103 ms) 2024-04-22T19:58:35.304 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2024-04-22T19:58:35.305 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.usage_clear_no_obj (1 ms) 2024-04-22T19:58:35.306 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.usage_clear 2024-04-22T19:58:35.399 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.usage_clear (94 ms) 2024-04-22T19:58:35.399 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.bi_log_trim 2024-04-22T19:58:35.440 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.bi_log_trim (40 ms) 2024-04-22T19:58:35.440 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw.index_racing_removes 2024-04-22T19:58:35.454 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw.index_racing_removes (15 ms) 2024-04-22T19:58:36.207 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 16 tests from cls_rgw (29320 ms total) 2024-04-22T19:58:36.207 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T19:58:36.207 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T19:58:36.207 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 16 tests from 1 test suite ran. (31908 ms total) 2024-04-22T19:58:36.207 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 16 tests. 2024-04-22T19:58:36.209 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T19:58:36.210 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T19:58:36.252 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_gc.sh on client.0... 2024-04-22T19:58:36.253 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2024-04-22T19:58:36.253 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2024-04-22T19:58:36.335 INFO:tasks.workunit.client.0.smithi136.stdout:Running main() from gmock_main.cc 2024-04-22T19:58:36.335 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 9 tests from 1 test suite. 2024-04-22T19:58:36.335 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T19:58:36.335 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 9 tests from cls_rgw_gc 2024-04-22T19:58:36.335 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.init 2024-04-22T19:58:38.225 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.init (1890 ms) 2024-04-22T19:58:38.225 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2024-04-22T19:58:38.239 INFO:tasks.workunit.client.0.smithi136.stderr:[ ] list info tag = chain-0 2024-04-22T19:58:38.240 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (14 ms) 2024-04-22T19:58:38.240 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2024-04-22T19:58:38.246 INFO:tasks.workunit.client.0.smithi136.stderr:[ ] next_marker is: = 0/4263 2024-04-22T19:58:38.246 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (6 ms) 2024-04-22T19:58:38.246 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2024-04-22T19:58:38.254 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (8 ms) 2024-04-22T19:58:38.254 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2024-04-22T19:58:38.383 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (129 ms) 2024-04-22T19:58:38.383 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2024-04-22T19:58:38.391 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (9 ms) 2024-04-22T19:58:38.391 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2024-04-22T19:58:38.399 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (8 ms) 2024-04-22T19:58:38.399 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2024-04-22T19:58:38.406 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (6 ms) 2024-04-22T19:58:38.406 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_gc.finalize 2024-04-22T19:58:39.230 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_gc.finalize (824 ms) 2024-04-22T19:58:39.230 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 9 tests from cls_rgw_gc (2895 ms total) 2024-04-22T19:58:39.230 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T19:58:39.230 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T19:58:39.230 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 9 tests from 1 test suite ran. (2895 ms total) 2024-04-22T19:58:39.230 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 9 tests. 2024-04-22T19:58:39.233 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T19:58:39.233 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T19:58:39.315 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_stats.sh on client.0... 2024-04-22T19:58:39.317 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2024-04-22T19:58:39.317 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2024-04-22T19:58:39.409 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T19:58:39.409 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T19:58:39.410 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 1 test from 1 test suite. 2024-04-22T19:58:39.410 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T19:58:41.239 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 1 test from cls_rgw_stats 2024-04-22T19:58:41.239 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] cls_rgw_stats.simulate 2024-04-22T19:58:41.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.252+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=WLKEW1UN3PG0 type=del size=1509720 2024-04-22T19:58:41.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.254+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=1OWT14D0EQ5E type=write size=298107 2024-04-22T19:58:41.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.255+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=5XF8SL40HVWI type=write size=70642 2024-04-22T19:58:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.257+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=O1PSWVMOB59P type=del size=1965480 2024-04-22T19:58:41.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.259+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~SVYANHZ25D7M.0 size=989684 2024-04-22T19:58:41.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.259+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=SVYANHZ25D7M tag=PW9FY1928Z07 type=write size=2038260 2024-04-22T19:58:41.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.260+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=DNLDF4ZQ0867 type=write size=745951 2024-04-22T19:58:41.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.262+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=RPVXK21EXOTC type=del size=3774385 2024-04-22T19:58:41.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.263+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=LU3KYISJXV4F type=write size=56143 2024-04-22T19:58:41.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.265+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~4W3JNY5WI89U.0 size=1048576 2024-04-22T19:58:41.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.266+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~4W3JNY5WI89U.1 size=1048576 2024-04-22T19:58:41.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.267+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~4W3JNY5WI89U.2 size=288390 2024-04-22T19:58:41.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.267+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=4W3JNY5WI89U tag=BAIARVS5UUZV type=write size=3434118 2024-04-22T19:58:41.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.269+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=R7506KBC0IRU type=del size=3081964 2024-04-22T19:58:41.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.270+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=2BONZNP4JXEN type=del size=1733456 2024-04-22T19:58:41.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.272+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~XTQ74YS12OBY.0 size=1048576 2024-04-22T19:58:41.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.273+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~XTQ74YS12OBY.1 size=1048576 2024-04-22T19:58:41.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.274+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~XTQ74YS12OBY.2 size=1025413 2024-04-22T19:58:41.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.274+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=XTQ74YS12OBY tag=K8SNQVI0IVZ2 type=write size=4171141 2024-04-22T19:58:41.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.277+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~Z4L0IUKM8JGD.0 size=1048576 2024-04-22T19:58:41.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.278+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~Z4L0IUKM8JGD.1 size=1048576 2024-04-22T19:58:41.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.279+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~Z4L0IUKM8JGD.2 size=535086 2024-04-22T19:58:41.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.279+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=Z4L0IUKM8JGD tag=KDXQ49HR6U1S type=write size=3680814 2024-04-22T19:58:41.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.281+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~XQZBX8NX8HJK.0 size=1048576 2024-04-22T19:58:41.282 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.282+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~XQZBX8NX8HJK.1 size=207003 2024-04-22T19:58:41.282 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.282+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=XQZBX8NX8HJK tag=F3KZC1O8N9A1 type=write size=2304155 2024-04-22T19:58:41.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.284+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=1I2138MJFIKP type=del size=1239844 2024-04-22T19:58:41.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.285+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=L44DHE9C4R8Y type=write size=280108 2024-04-22T19:58:41.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.287+0000 7f5269a8e780 1 < suggested remove operation key=B0Y7ZTPC0975 tag=WLKEW1UN3PG0 type=del 2024-04-22T19:58:41.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.290+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~21W8H0IWQJKV.0 size=190864 2024-04-22T19:58:41.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.290+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=21W8H0IWQJKV tag=ZL4YURX4DZQO type=write size=1239440 2024-04-22T19:58:41.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.291+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=1OWT14D0EQ5E type=write size=298107 2024-04-22T19:58:41.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.292+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=7UGX5UZ3PZ46 type=del size=2841397 2024-04-22T19:58:41.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.293+0000 7f5269a8e780 1 < completed write operation key=GX6NILQG878S tag=5XF8SL40HVWI type=write size=70642 2024-04-22T19:58:41.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.295+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=BPZ44ECZSHEJ type=del size=788653 2024-04-22T19:58:41.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.296+0000 7f5269a8e780 1 < canceled operation key=637M7AP8X9E9 tag=O1PSWVMOB59P type=del size=1965480 2024-04-22T19:58:41.297 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.297+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=Z39U3TBARIZ2 type=write size=621767 2024-04-22T19:58:41.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.298+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~SVYANHZ25D7M.0 size=989684 2024-04-22T19:58:41.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.300+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=SVYANHZ25D7M tag=PW9FY1928Z07 type=write size=2038260 2024-04-22T19:58:41.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.302+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~9E6609FDOHRI.0 size=1001246 2024-04-22T19:58:41.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.302+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=9E6609FDOHRI tag=S1FTTONA8HMQ type=write size=2049822 2024-04-22T19:58:41.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.304+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=DNLDF4ZQ0867 type=write size=745951 2024-04-22T19:58:41.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.305+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=X3BXTQL401XI type=del size=2243876 2024-04-22T19:58:41.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.306+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=RPVXK21EXOTC type=del 2024-04-22T19:58:41.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.307+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=2KEFDJPFWD10 type=write size=744539 2024-04-22T19:58:41.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.308+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=LU3KYISJXV4F type=write size=56143 2024-04-22T19:58:41.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.310+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=CL7FTBK7XWUH type=del size=2469187 2024-04-22T19:58:41.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.311+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~4W3JNY5WI89U.0 size=1048576 2024-04-22T19:58:41.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.312+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~4W3JNY5WI89U.1 size=1048576 2024-04-22T19:58:41.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.313+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~4W3JNY5WI89U.2 size=288390 2024-04-22T19:58:41.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.315+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=4W3JNY5WI89U tag=BAIARVS5UUZV type=write size=3434118 2024-04-22T19:58:41.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.317+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~4U4AN2P2LDCE.0 size=660517 2024-04-22T19:58:41.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.317+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=4U4AN2P2LDCE tag=0GNXSDKRNUKW type=write size=1709093 2024-04-22T19:58:41.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.319+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=R7506KBC0IRU type=del size=3081964 2024-04-22T19:58:41.321 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.321+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=M775NJS8ND6C type=del size=738138 2024-04-22T19:58:41.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.322+0000 7f5269a8e780 1 < canceled operation key=70MLQGJPQYPY tag=2BONZNP4JXEN type=del size=1733456 2024-04-22T19:58:41.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.324+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~QAZMW1JE5P6R.0 size=464768 2024-04-22T19:58:41.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.324+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=QAZMW1JE5P6R tag=44UFYB900U11 type=write size=1513344 2024-04-22T19:58:41.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.326+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~XTQ74YS12OBY.0 size=1048576 2024-04-22T19:58:41.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.328+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~XTQ74YS12OBY.1 size=1048576 2024-04-22T19:58:41.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.329+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~XTQ74YS12OBY.2 size=1025413 2024-04-22T19:58:41.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.332+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=XTQ74YS12OBY tag=K8SNQVI0IVZ2 type=write size=4171141 2024-04-22T19:58:41.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.333+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=BO2OOXA5TARQ type=del size=330156 2024-04-22T19:58:41.334 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.334+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~Z4L0IUKM8JGD.0 size=1048576 2024-04-22T19:58:41.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.335+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~Z4L0IUKM8JGD.1 size=1048576 2024-04-22T19:58:41.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.336+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~Z4L0IUKM8JGD.2 size=535086 2024-04-22T19:58:41.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.338+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=Z4L0IUKM8JGD tag=KDXQ49HR6U1S type=write size=3680814 2024-04-22T19:58:41.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.339+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=L8NCKKH7HWRY type=del size=2528210 2024-04-22T19:58:41.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.339+0000 7f5269a8e780 1 < canceled part key=_multipart_A853MOZPT9BG.2~XQZBX8NX8HJK.0 size=1048576 2024-04-22T19:58:41.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.340+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~XQZBX8NX8HJK.1 size=207003 2024-04-22T19:58:41.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.343+0000 7f5269a8e780 1 < canceled multipart upload key=A853MOZPT9BG upload=XQZBX8NX8HJK tag=F3KZC1O8N9A1 type=write size=2304155 2024-04-22T19:58:41.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.344+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=ZEFY0FTBGDCP type=del size=2614565 2024-04-22T19:58:41.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.345+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=1I2138MJFIKP type=del 2024-04-22T19:58:41.346 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.346+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=3D4C84G1V1V1 type=del size=757519 2024-04-22T19:58:41.348 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.348+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=L44DHE9C4R8Y type=write size=280108 2024-04-22T19:58:41.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.349+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=X1DYX0QBLRD7 type=del size=2503030 2024-04-22T19:58:41.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.350+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~21W8H0IWQJKV.0 size=190864 2024-04-22T19:58:41.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.352+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=21W8H0IWQJKV tag=ZL4YURX4DZQO type=write size=1239440 2024-04-22T19:58:41.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.353+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=HJW58UPQI7LG type=write size=743509 2024-04-22T19:58:41.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.354+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=7UGX5UZ3PZ46 type=del 2024-04-22T19:58:41.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.355+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=EPV9OM0NFP1H type=del size=63240 2024-04-22T19:58:41.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.356+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=BPZ44ECZSHEJ type=del 2024-04-22T19:58:41.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.357+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=CMPXK3E3W1QU type=del size=1169484 2024-04-22T19:58:41.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.359+0000 7f5269a8e780 1 < canceled operation key=GP1C5ULUU143 tag=Z39U3TBARIZ2 type=write size=621767 2024-04-22T19:58:41.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.360+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=1FBLEUN6TPUN type=del size=4177076 2024-04-22T19:58:41.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.361+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~9E6609FDOHRI.0 size=1001246 2024-04-22T19:58:41.363 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.363+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=9E6609FDOHRI tag=S1FTTONA8HMQ type=write size=2049822 2024-04-22T19:58:41.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.364+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=CIEIAL9KEG1W type=del size=875693 2024-04-22T19:58:41.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.366+0000 7f5269a8e780 1 < suggested update operation key=KV5DYQ49ZKCX tag=X3BXTQL401XI type=del size=2243876 2024-04-22T19:58:41.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.369+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~16L0UH0M4SM2.0 size=1048576 2024-04-22T19:58:41.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.370+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~16L0UH0M4SM2.1 size=1048576 2024-04-22T19:58:41.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.371+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~16L0UH0M4SM2.2 size=197525 2024-04-22T19:58:41.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.371+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=16L0UH0M4SM2 tag=078HCN89PPWJ type=write size=3343253 2024-04-22T19:58:41.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.372+0000 7f5269a8e780 1 < completed write operation key=B0Y7ZTPC0975 tag=2KEFDJPFWD10 type=write size=744539 2024-04-22T19:58:41.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.373+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=DCGS49P19VCF type=del size=1146684 2024-04-22T19:58:41.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.374+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=CL7FTBK7XWUH type=del 2024-04-22T19:58:41.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.376+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~Q1M8TMZCL0L3.0 size=622669 2024-04-22T19:58:41.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.376+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=Q1M8TMZCL0L3 tag=I6RMW1TQCSFF type=write size=1671245 2024-04-22T19:58:41.377 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.377+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~4U4AN2P2LDCE.0 size=660517 2024-04-22T19:58:41.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.380+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=4U4AN2P2LDCE tag=0GNXSDKRNUKW type=write size=1709093 2024-04-22T19:58:41.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.383+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~PIR1D8LT2TXX.0 size=1048576 2024-04-22T19:58:41.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.385+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~PIR1D8LT2TXX.1 size=492318 2024-04-22T19:58:41.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.385+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=PIR1D8LT2TXX tag=9XUDU99V8BL5 type=write size=2589470 2024-04-22T19:58:41.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.386+0000 7f5269a8e780 1 < canceled operation key=GRSL48RUEBX8 tag=M775NJS8ND6C type=del size=738138 2024-04-22T19:58:41.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.387+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=A3BLVEWCX9TV type=write size=647653 2024-04-22T19:58:41.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.388+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~QAZMW1JE5P6R.0 size=464768 2024-04-22T19:58:41.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.390+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=QAZMW1JE5P6R tag=44UFYB900U11 type=write size=1513344 2024-04-22T19:58:41.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.391+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=MLABE26DQUG4 type=del size=1781187 2024-04-22T19:58:41.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.393+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=BO2OOXA5TARQ type=del 2024-04-22T19:58:41.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.394+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=154NRANHRU1Q type=del size=3567495 2024-04-22T19:58:41.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.395+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=L8NCKKH7HWRY type=del 2024-04-22T19:58:41.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.396+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=LEBZ3D8YHC8K type=del size=982357 2024-04-22T19:58:41.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.397+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=ZEFY0FTBGDCP type=del 2024-04-22T19:58:41.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.399+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~R0FN7Z63BA8S.0 size=1048576 2024-04-22T19:58:41.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.400+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~R0FN7Z63BA8S.1 size=768146 2024-04-22T19:58:41.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.400+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=R0FN7Z63BA8S tag=VB7TYY5CL7VZ type=write size=2865298 2024-04-22T19:58:41.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.402+0000 7f5269a8e780 1 < suggested update operation key=RY1O7X1CWZMK tag=3D4C84G1V1V1 type=del size=757519 2024-04-22T19:58:41.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.403+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=7VL3G0R4P09S type=del size=857983 2024-04-22T19:58:41.404 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.404+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=X1DYX0QBLRD7 type=del 2024-04-22T19:58:41.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.406+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~7K83H5W7TFGP.0 size=1048576 2024-04-22T19:58:41.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.407+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~7K83H5W7TFGP.1 size=1048576 2024-04-22T19:58:41.408 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.408+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~7K83H5W7TFGP.2 size=33722 2024-04-22T19:58:41.408 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.408+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=7K83H5W7TFGP tag=3FARVB1Y72YI type=write size=3179450 2024-04-22T19:58:41.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.409+0000 7f5269a8e780 1 < completed write operation key=IBH6W1NINUHC tag=HJW58UPQI7LG type=write size=743509 2024-04-22T19:58:41.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.410+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=TZUIAGBH7U6O type=del size=36245 2024-04-22T19:58:41.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.412+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=EPV9OM0NFP1H type=del 2024-04-22T19:58:41.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.414+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~QZVJPJ9A75AG.0 size=1048576 2024-04-22T19:58:41.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.415+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~QZVJPJ9A75AG.1 size=502365 2024-04-22T19:58:41.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.415+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=QZVJPJ9A75AG tag=STE22B7G2P5P type=write size=2599517 2024-04-22T19:58:41.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.417+0000 7f5269a8e780 1 < suggested update operation key=RY1O7X1CWZMK tag=CMPXK3E3W1QU type=del size=1169484 2024-04-22T19:58:41.418 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.418+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=DVDJGVIPH2NC type=del size=2432130 2024-04-22T19:58:41.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.419+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=1FBLEUN6TPUN type=del 2024-04-22T19:58:41.420 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.420+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=SYAWA8HCUQPB type=write size=45864 2024-04-22T19:58:41.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.421+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=CIEIAL9KEG1W type=del 2024-04-22T19:58:41.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.423+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~A4WO5OYFG1Q8.0 size=738331 2024-04-22T19:58:41.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.423+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=A4WO5OYFG1Q8 tag=WTWRXTU6LESO type=write size=1786907 2024-04-22T19:58:41.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.424+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~16L0UH0M4SM2.0 size=1048576 2024-04-22T19:58:41.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.425+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~16L0UH0M4SM2.1 size=1048576 2024-04-22T19:58:41.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.426+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~16L0UH0M4SM2.2 size=197525 2024-04-22T19:58:41.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.428+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=16L0UH0M4SM2 tag=078HCN89PPWJ type=write size=3343253 2024-04-22T19:58:41.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.429+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=7JHL7GDJ55CO type=write size=303524 2024-04-22T19:58:41.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.430+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=DCGS49P19VCF type=del 2024-04-22T19:58:41.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.433+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~05Q191M7HU6G.0 size=1048576 2024-04-22T19:58:41.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.433+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~05Q191M7HU6G.1 size=1048576 2024-04-22T19:58:41.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.434+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~05Q191M7HU6G.2 size=666407 2024-04-22T19:58:41.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.434+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=05Q191M7HU6G tag=UPMEOCZMYEYL type=write size=3812135 2024-04-22T19:58:41.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.436+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~Q1M8TMZCL0L3.0 size=622669 2024-04-22T19:58:41.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.437+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=Q1M8TMZCL0L3 tag=I6RMW1TQCSFF type=write size=1671245 2024-04-22T19:58:41.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.439+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=AHUHMUR37EZN type=write size=523933 2024-04-22T19:58:41.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.440+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~PIR1D8LT2TXX.0 size=1048576 2024-04-22T19:58:41.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.441+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~PIR1D8LT2TXX.1 size=492318 2024-04-22T19:58:41.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.443+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=PIR1D8LT2TXX tag=9XUDU99V8BL5 type=write size=2589470 2024-04-22T19:58:41.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.445+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~ODQECR1PY0Q2.0 size=1048576 2024-04-22T19:58:41.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.446+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~ODQECR1PY0Q2.1 size=922823 2024-04-22T19:58:41.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.446+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=ODQECR1PY0Q2 tag=8XGG9BCMGAB5 type=write size=3019975 2024-04-22T19:58:41.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.447+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=A3BLVEWCX9TV type=write size=647653 2024-04-22T19:58:41.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.448+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=L07I6XKRGN8G type=write size=1011544 2024-04-22T19:58:41.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.450+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=MLABE26DQUG4 type=del 2024-04-22T19:58:41.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.452+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~IDELTKRN0B5J.0 size=1048576 2024-04-22T19:58:41.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.453+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~IDELTKRN0B5J.1 size=612368 2024-04-22T19:58:41.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.453+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=IDELTKRN0B5J tag=U7AB7G2QXT4W type=write size=2709520 2024-04-22T19:58:41.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.455+0000 7f5269a8e780 1 < suggested update operation key=Y5HR8XW4L0SP tag=154NRANHRU1Q type=del size=3567495 2024-04-22T19:58:41.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.457+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~URKFFFWQEVAK.0 size=1048576 2024-04-22T19:58:41.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.458+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~URKFFFWQEVAK.1 size=630354 2024-04-22T19:58:41.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.458+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=URKFFFWQEVAK tag=1486MJC3T9T1 type=write size=2727506 2024-04-22T19:58:41.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.459+0000 7f5269a8e780 1 < canceled operation key=I91AD8FJ3P4T tag=LEBZ3D8YHC8K type=del size=982357 2024-04-22T19:58:41.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.461+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~0TY4NAVOYZCE.0 size=360063 2024-04-22T19:58:41.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.461+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=0TY4NAVOYZCE tag=Y6YDKZKADZK9 type=write size=1408639 2024-04-22T19:58:41.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.462+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~R0FN7Z63BA8S.0 size=1048576 2024-04-22T19:58:41.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.463+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~R0FN7Z63BA8S.1 size=768146 2024-04-22T19:58:41.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.465+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=R0FN7Z63BA8S tag=VB7TYY5CL7VZ type=write size=2865298 2024-04-22T19:58:41.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.467+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=P3ISLB4YXI04 type=del size=1719362 2024-04-22T19:58:41.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.469+0000 7f5269a8e780 1 < suggested update operation key=GP1C5ULUU143 tag=7VL3G0R4P09S type=del size=857983 2024-04-22T19:58:41.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.470+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=7AHKHAJT02XR type=del size=3499624 2024-04-22T19:58:41.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.472+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~7K83H5W7TFGP.0 size=1048576 2024-04-22T19:58:41.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.473+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~7K83H5W7TFGP.1 size=1048576 2024-04-22T19:58:41.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.474+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~7K83H5W7TFGP.2 size=33722 2024-04-22T19:58:41.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.476+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=7K83H5W7TFGP tag=3FARVB1Y72YI type=write size=3179450 2024-04-22T19:58:41.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.477+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=C7GWX0YHFZB8 type=del size=2437088 2024-04-22T19:58:41.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.478+0000 7f5269a8e780 -1 < failed to complete operation key=RY1O7X1CWZMK tag=TZUIAGBH7U6O type=del size=36245: (22) Invalid argument 2024-04-22T19:58:41.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.480+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~ITT40LD3I2OE.0 size=594817 2024-04-22T19:58:41.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.480+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=ITT40LD3I2OE tag=7MII0FBPRHVY type=write size=1643393 2024-04-22T19:58:41.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.482+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~QZVJPJ9A75AG.0 size=1048576 2024-04-22T19:58:41.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.482+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~QZVJPJ9A75AG.1 size=502365 2024-04-22T19:58:41.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.484+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=QZVJPJ9A75AG tag=STE22B7G2P5P type=write size=2599517 2024-04-22T19:58:41.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.485+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=R9491MDOZHPK type=del size=2555920 2024-04-22T19:58:41.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.486+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=DVDJGVIPH2NC type=del 2024-04-22T19:58:41.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.488+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=HTPZ13CX1X5A type=del size=3248289 2024-04-22T19:58:41.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.489+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=SYAWA8HCUQPB type=write size=45864 2024-04-22T19:58:41.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.490+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=BHIIDGBPUJU8 type=del size=2928974 2024-04-22T19:58:41.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.492+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~A4WO5OYFG1Q8.0 size=738331 2024-04-22T19:58:41.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.494+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=A4WO5OYFG1Q8 tag=WTWRXTU6LESO type=write size=1786907 2024-04-22T19:58:41.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.496+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~3RWXLFCACSYA.0 size=291197 2024-04-22T19:58:41.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.496+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=3RWXLFCACSYA tag=7LZXDTXUNVX3 type=write size=1339773 2024-04-22T19:58:41.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.498+0000 7f5269a8e780 1 < suggested update operation key=M9M4SEEEZSFV tag=7JHL7GDJ55CO type=write size=303524 2024-04-22T19:58:41.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.501+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~WD9PJ4MHJYZ1.0 size=1048576 2024-04-22T19:58:41.501 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.501+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~WD9PJ4MHJYZ1.1 size=1048576 2024-04-22T19:58:41.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.502+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~WD9PJ4MHJYZ1.2 size=582234 2024-04-22T19:58:41.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.502+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=WD9PJ4MHJYZ1 tag=H08LBF9RA4FH type=write size=3727962 2024-04-22T19:58:41.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.503+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~05Q191M7HU6G.0 size=1048576 2024-04-22T19:58:41.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.504+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~05Q191M7HU6G.1 size=1048576 2024-04-22T19:58:41.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.505+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~05Q191M7HU6G.2 size=666407 2024-04-22T19:58:41.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.507+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=05Q191M7HU6G tag=UPMEOCZMYEYL type=write size=3812135 2024-04-22T19:58:41.508 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.508+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=DXW8BEHZK259 type=del size=613231 2024-04-22T19:58:41.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.510+0000 7f5269a8e780 1 < completed write operation key=SN2BZATJGUWA tag=AHUHMUR37EZN type=write size=523933 2024-04-22T19:58:41.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.511+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=WRLPWP8SF67Y type=del size=2860816 2024-04-22T19:58:41.512 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.512+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~ODQECR1PY0Q2.0 size=1048576 2024-04-22T19:58:41.513 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.513+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~ODQECR1PY0Q2.1 size=922823 2024-04-22T19:58:41.515 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.515+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=ODQECR1PY0Q2 tag=8XGG9BCMGAB5 type=write size=3019975 2024-04-22T19:58:41.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.517+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~2KNFWYW1YJ7X.0 size=1048576 2024-04-22T19:58:41.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.518+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~2KNFWYW1YJ7X.1 size=761545 2024-04-22T19:58:41.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.518+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=2KNFWYW1YJ7X tag=KNMD3TCLVRGS type=write size=2858697 2024-04-22T19:58:41.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.519+0000 7f5269a8e780 1 < completed write operation key=O9NFAVE09ZN7 tag=L07I6XKRGN8G type=write size=1011544 2024-04-22T19:58:41.521 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.521+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~G03ICZAXVG83.0 size=59049 2024-04-22T19:58:41.521 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.521+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=G03ICZAXVG83 tag=SH4JAVQ83HUU type=write size=1107625 2024-04-22T19:58:41.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.522+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~IDELTKRN0B5J.0 size=1048576 2024-04-22T19:58:41.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.522+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~IDELTKRN0B5J.1 size=612368 2024-04-22T19:58:41.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.524+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=IDELTKRN0B5J tag=U7AB7G2QXT4W type=write size=2709520 2024-04-22T19:58:41.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.526+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~91LG9IUWC32X.0 size=1048576 2024-04-22T19:58:41.527 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.527+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~91LG9IUWC32X.1 size=1048576 2024-04-22T19:58:41.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.528+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~91LG9IUWC32X.2 size=625812 2024-04-22T19:58:41.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.528+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=91LG9IUWC32X tag=M201191FKLLH type=write size=3771540 2024-04-22T19:58:41.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.529+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~URKFFFWQEVAK.0 size=1048576 2024-04-22T19:58:41.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.530+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~URKFFFWQEVAK.1 size=630354 2024-04-22T19:58:41.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.532+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=URKFFFWQEVAK tag=1486MJC3T9T1 type=write size=2727506 2024-04-22T19:58:41.533 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.533+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=FLGY4NSDGF8J type=write size=977519 2024-04-22T19:58:41.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.535+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~0TY4NAVOYZCE.0 size=360063 2024-04-22T19:58:41.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.537+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=0TY4NAVOYZCE tag=Y6YDKZKADZK9 type=write size=1408639 2024-04-22T19:58:41.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.538+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=7QE1W7U1UJXU type=del size=114350 2024-04-22T19:58:41.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.539+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=P3ISLB4YXI04 type=del 2024-04-22T19:58:41.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.541+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~UPP224810O5C.0 size=814799 2024-04-22T19:58:41.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.541+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=UPP224810O5C tag=1Y8NWXGTVOQS type=write size=1863375 2024-04-22T19:58:41.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.543+0000 7f5269a8e780 1 < suggested update operation key=86GW6HV9G1IZ tag=7AHKHAJT02XR type=del size=3499624 2024-04-22T19:58:41.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.544+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=70PT0AV2O8C6 type=del size=2203471 2024-04-22T19:58:41.546 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.547+0000 7f5269a8e780 1 < suggested remove operation key=70MLQGJPQYPY tag=C7GWX0YHFZB8 type=del 2024-04-22T19:58:41.548 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.548+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=SJBDBDL5AL6G type=del size=3510698 2024-04-22T19:58:41.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.549+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~ITT40LD3I2OE.0 size=594817 2024-04-22T19:58:41.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.551+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=ITT40LD3I2OE tag=7MII0FBPRHVY type=write size=1643393 2024-04-22T19:58:41.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.553+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~AS6L278CZS0U.0 size=769425 2024-04-22T19:58:41.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.553+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=AS6L278CZS0U tag=XNKF4OU6WZLT type=write size=1818001 2024-04-22T19:58:41.554 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.554+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=R9491MDOZHPK type=del 2024-04-22T19:58:41.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.556+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~EJR8NDTT4ICB.0 size=694981 2024-04-22T19:58:41.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.556+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=EJR8NDTT4ICB tag=I3J1GH0WEJ83 type=write size=1743557 2024-04-22T19:58:41.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.557+0000 7f5269a8e780 -1 < failed to complete operation key=86GW6HV9G1IZ tag=HTPZ13CX1X5A type=del size=3248289: (22) Invalid argument 2024-04-22T19:58:41.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.559+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~OU7T6CG7VAAI.0 size=1048576 2024-04-22T19:58:41.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.561+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~OU7T6CG7VAAI.1 size=96685 2024-04-22T19:58:41.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.561+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=OU7T6CG7VAAI tag=MQBSOEOVH2PW type=write size=2193837 2024-04-22T19:58:41.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.562+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=BHIIDGBPUJU8 type=del 2024-04-22T19:58:41.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.564+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~0VF2VAWL1O7B.0 size=452161 2024-04-22T19:58:41.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.564+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=0VF2VAWL1O7B tag=YBJDSSGL740S type=write size=1500737 2024-04-22T19:58:41.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.565+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~3RWXLFCACSYA.0 size=291197 2024-04-22T19:58:41.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.567+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=3RWXLFCACSYA tag=7LZXDTXUNVX3 type=write size=1339773 2024-04-22T19:58:41.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.568+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=S33G0DQRC1HY type=del size=3473375 2024-04-22T19:58:41.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.570+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~WD9PJ4MHJYZ1.0 size=1048576 2024-04-22T19:58:41.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.571+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~WD9PJ4MHJYZ1.1 size=1048576 2024-04-22T19:58:41.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.572+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~WD9PJ4MHJYZ1.2 size=582234 2024-04-22T19:58:41.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.574+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=WD9PJ4MHJYZ1 tag=H08LBF9RA4FH type=write size=3727962 2024-04-22T19:58:41.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.576+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~0YUCGY8TX3IH.0 size=416466 2024-04-22T19:58:41.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.576+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=0YUCGY8TX3IH tag=PAW01P79VBLQ type=write size=1465042 2024-04-22T19:58:41.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.578+0000 7f5269a8e780 1 < suggested update operation key=4GX83ZJWZCD1 tag=DXW8BEHZK259 type=del size=613231 2024-04-22T19:58:41.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.579+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=TV0UHFZOHKEK type=del size=3561706 2024-04-22T19:58:41.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.581+0000 7f5269a8e780 1 < suggested remove operation key=T2BV2LUCY4E3 tag=WRLPWP8SF67Y type=del 2024-04-22T19:58:41.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.583+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~YSPCBULIAJ4A.0 size=980659 2024-04-22T19:58:41.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.583+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=YSPCBULIAJ4A tag=2F6M00VP6KFE type=write size=2029235 2024-04-22T19:58:41.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.585+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~2KNFWYW1YJ7X.0 size=1048576 2024-04-22T19:58:41.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.585+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~2KNFWYW1YJ7X.1 size=761545 2024-04-22T19:58:41.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.587+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=2KNFWYW1YJ7X tag=KNMD3TCLVRGS type=write size=2858697 2024-04-22T19:58:41.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.590+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~O2XRHO2542LH.0 size=1048576 2024-04-22T19:58:41.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.591+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~O2XRHO2542LH.1 size=1048576 2024-04-22T19:58:41.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.591+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~O2XRHO2542LH.2 size=655980 2024-04-22T19:58:41.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.591+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=O2XRHO2542LH tag=EKJQ351D2QX2 type=write size=3801708 2024-04-22T19:58:41.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.592+0000 7f5269a8e780 1 < canceled part key=_multipart_HIAXPVKZ1IIO.2~G03ICZAXVG83.0 size=59049 2024-04-22T19:58:41.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.594+0000 7f5269a8e780 1 < canceled multipart upload key=HIAXPVKZ1IIO upload=G03ICZAXVG83 tag=SH4JAVQ83HUU type=write size=1107625 2024-04-22T19:58:41.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.596+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=IMYV2V6FL5PM type=write size=610558 2024-04-22T19:58:41.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.597+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~91LG9IUWC32X.0 size=1048576 2024-04-22T19:58:41.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.598+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~91LG9IUWC32X.1 size=1048576 2024-04-22T19:58:41.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.599+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~91LG9IUWC32X.2 size=625812 2024-04-22T19:58:41.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.601+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=91LG9IUWC32X tag=M201191FKLLH type=write size=3771540 2024-04-22T19:58:41.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.603+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~H2TVJUPQEI4S.0 size=289297 2024-04-22T19:58:41.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.603+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=H2TVJUPQEI4S tag=6QO367NRFO7Z type=write size=1337873 2024-04-22T19:58:41.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.604+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=FLGY4NSDGF8J type=write size=977519 2024-04-22T19:58:41.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.606+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=MTMTKXRLOBSV type=del size=1422721 2024-04-22T19:58:41.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.607+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=7QE1W7U1UJXU type=del 2024-04-22T19:58:41.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.608+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~6TOKXYEENQKJ.0 size=233580 2024-04-22T19:58:41.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.609+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=6TOKXYEENQKJ tag=PV2FWXSKF9Q3 type=write size=1282156 2024-04-22T19:58:41.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.610+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~UPP224810O5C.0 size=814799 2024-04-22T19:58:41.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.611+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=UPP224810O5C tag=1Y8NWXGTVOQS type=write size=1863375 2024-04-22T19:58:41.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.613+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=IHJ2Q5224UIQ type=del size=3647449 2024-04-22T19:58:41.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.615+0000 7f5269a8e780 1 < suggested update operation key=2YFD2U6HVNE7 tag=70PT0AV2O8C6 type=del size=2203471 2024-04-22T19:58:41.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.616+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=9OSL7GTFL5MI type=del size=1944640 2024-04-22T19:58:41.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.618+0000 7f5269a8e780 1 < suggested update operation key=70MLQGJPQYPY tag=SJBDBDL5AL6G type=del size=3510698 2024-04-22T19:58:41.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.619+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=4W755Q1BE4HN type=del size=1587854 2024-04-22T19:58:41.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.620+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~AS6L278CZS0U.0 size=769425 2024-04-22T19:58:41.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.622+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=AS6L278CZS0U tag=XNKF4OU6WZLT type=write size=1818001 2024-04-22T19:58:41.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.623+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=FUQMIMT1BUU8 type=write size=838182 2024-04-22T19:58:41.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.624+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~EJR8NDTT4ICB.0 size=694981 2024-04-22T19:58:41.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.626+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=EJR8NDTT4ICB tag=I3J1GH0WEJ83 type=write size=1743557 2024-04-22T19:58:41.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.627+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=4Q9T8W1B645Y type=del size=1671033 2024-04-22T19:58:41.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.628+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~OU7T6CG7VAAI.0 size=1048576 2024-04-22T19:58:41.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.629+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~OU7T6CG7VAAI.1 size=96685 2024-04-22T19:58:41.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.631+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=OU7T6CG7VAAI tag=MQBSOEOVH2PW type=write size=2193837 2024-04-22T19:58:41.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.632+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=YB3Q6UCZZNMY type=del size=4034857 2024-04-22T19:58:41.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.633+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~0VF2VAWL1O7B.0 size=452161 2024-04-22T19:58:41.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.635+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=0VF2VAWL1O7B tag=YBJDSSGL740S type=write size=1500737 2024-04-22T19:58:41.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.637+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~R8AGJ9U0D3C5.0 size=483556 2024-04-22T19:58:41.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.637+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=R8AGJ9U0D3C5 tag=RRW0ZY2671FM type=write size=1532132 2024-04-22T19:58:41.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.639+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=S33G0DQRC1HY type=del 2024-04-22T19:58:41.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.640+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=9BC5E55298F4 type=write size=996343 2024-04-22T19:58:41.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.641+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~0YUCGY8TX3IH.0 size=416466 2024-04-22T19:58:41.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.643+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=0YUCGY8TX3IH tag=PAW01P79VBLQ type=write size=1465042 2024-04-22T19:58:41.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.644+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=JQ8QGUG7D83Q type=del size=2689528 2024-04-22T19:58:41.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.646+0000 7f5269a8e780 1 < suggested update operation key=HV8J6YM0YNC1 tag=TV0UHFZOHKEK type=del size=3561706 2024-04-22T19:58:41.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.648+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~OC1K2YO03YT3.0 size=1048576 2024-04-22T19:58:41.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.649+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~OC1K2YO03YT3.1 size=967406 2024-04-22T19:58:41.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.649+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=OC1K2YO03YT3 tag=OMJQLNGONBOO type=write size=3064558 2024-04-22T19:58:41.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.649+0000 7f5269a8e780 1 < canceled part key=_multipart_5UBYMQT7ZWKX.2~YSPCBULIAJ4A.0 size=980659 2024-04-22T19:58:41.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.651+0000 7f5269a8e780 1 < canceled multipart upload key=5UBYMQT7ZWKX upload=YSPCBULIAJ4A tag=2F6M00VP6KFE type=write size=2029235 2024-04-22T19:58:41.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.652+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=BKXWSAO935Y8 type=del size=1632186 2024-04-22T19:58:41.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.654+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~O2XRHO2542LH.0 size=1048576 2024-04-22T19:58:41.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.655+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~O2XRHO2542LH.1 size=1048576 2024-04-22T19:58:41.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.655+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~O2XRHO2542LH.2 size=655980 2024-04-22T19:58:41.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.657+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=O2XRHO2542LH tag=EKJQ351D2QX2 type=write size=3801708 2024-04-22T19:58:41.659 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.659+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=597U5NUXBR7F type=del size=89869 2024-04-22T19:58:41.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.660+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=IMYV2V6FL5PM type=write size=610558 2024-04-22T19:58:41.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.661+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=UTXCYGN66904 type=del size=760049 2024-04-22T19:58:41.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.662+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~H2TVJUPQEI4S.0 size=289297 2024-04-22T19:58:41.664 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.664+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=H2TVJUPQEI4S tag=6QO367NRFO7Z type=write size=1337873 2024-04-22T19:58:41.665 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.665+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=PNPGRMFT20KQ type=del size=2862041 2024-04-22T19:58:41.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.666+0000 7f5269a8e780 1 < canceled operation key=HIAXPVKZ1IIO tag=MTMTKXRLOBSV type=del size=1422721 2024-04-22T19:58:41.668 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.668+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=06DI75WNKMON type=del size=1514698 2024-04-22T19:58:41.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.669+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~6TOKXYEENQKJ.0 size=233580 2024-04-22T19:58:41.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.671+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=6TOKXYEENQKJ tag=PV2FWXSKF9Q3 type=write size=1282156 2024-04-22T19:58:41.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.672+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=OW83VD67NL07 type=del size=514425 2024-04-22T19:58:41.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.673+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=IHJ2Q5224UIQ type=del 2024-04-22T19:58:41.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.674+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=EVJSITMT6R8A type=del size=2186954 2024-04-22T19:58:41.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.676+0000 7f5269a8e780 1 < suggested remove operation key=I91AD8FJ3P4T tag=9OSL7GTFL5MI type=del 2024-04-22T19:58:41.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.677+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=SAVKAE156ROX type=del size=33881 2024-04-22T19:58:41.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.678+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=4W755Q1BE4HN type=del 2024-04-22T19:58:41.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.680+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~PZ613B11Z14J.0 size=1048576 2024-04-22T19:58:41.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.681+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~PZ613B11Z14J.1 size=478180 2024-04-22T19:58:41.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.681+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=PZ613B11Z14J tag=YC7OH85RWFCF type=write size=2575332 2024-04-22T19:58:41.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.682+0000 7f5269a8e780 1 < completed write operation key=KE7YGPTF1MOR tag=FUQMIMT1BUU8 type=write size=838182 2024-04-22T19:58:41.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.683+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=19A54ZM515W4 type=del size=1883135 2024-04-22T19:58:41.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.686+0000 7f5269a8e780 1 < suggested update operation key=B0Y7ZTPC0975 tag=4Q9T8W1B645Y type=del size=1671033 2024-04-22T19:58:41.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.687+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~XMFZCC1CI4WN.0 size=111751 2024-04-22T19:58:41.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.687+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=XMFZCC1CI4WN tag=84LND5RYP6KP type=write size=1160327 2024-04-22T19:58:41.689 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.689+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=YB3Q6UCZZNMY type=del 2024-04-22T19:58:41.690 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.690+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=3F5YSG3OZCN0 type=del size=2069583 2024-04-22T19:58:41.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.691+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~R8AGJ9U0D3C5.0 size=483556 2024-04-22T19:58:41.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.693+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=R8AGJ9U0D3C5 tag=RRW0ZY2671FM type=write size=1532132 2024-04-22T19:58:41.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.694+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=0680GGLFVXE0 type=del size=3701393 2024-04-22T19:58:41.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.695+0000 7f5269a8e780 1 < completed write operation key=HIAXPVKZ1IIO tag=9BC5E55298F4 type=write size=996343 2024-04-22T19:58:41.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.696+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=0N984O3J18OY type=del size=2074624 2024-04-22T19:58:41.697 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.697+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=JQ8QGUG7D83Q type=del 2024-04-22T19:58:41.699 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.699+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~TO0DZ8O20NLY.0 size=1048576 2024-04-22T19:58:41.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.700+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~TO0DZ8O20NLY.1 size=1048576 2024-04-22T19:58:41.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.702+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~TO0DZ8O20NLY.2 size=471041 2024-04-22T19:58:41.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.702+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=TO0DZ8O20NLY tag=JP4H94T4YDHE type=write size=3616769 2024-04-22T19:58:41.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.703+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~OC1K2YO03YT3.0 size=1048576 2024-04-22T19:58:41.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.704+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~OC1K2YO03YT3.1 size=967406 2024-04-22T19:58:41.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.706+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=OC1K2YO03YT3 tag=OMJQLNGONBOO type=write size=3064558 2024-04-22T19:58:41.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.707+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~C0HQQ1DX37IW.0 size=1048576 2024-04-22T19:58:41.708 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.708+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~C0HQQ1DX37IW.1 size=1048576 2024-04-22T19:58:41.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.709+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~C0HQQ1DX37IW.2 size=231438 2024-04-22T19:58:41.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.709+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=C0HQQ1DX37IW tag=M69W7F8KJURD type=write size=3377166 2024-04-22T19:58:41.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.711+0000 7f5269a8e780 1 < suggested remove operation key=GX6NILQG878S tag=BKXWSAO935Y8 type=del 2024-04-22T19:58:41.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.712+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=XQWCGDYSZ1MG type=del size=3082781 2024-04-22T19:58:41.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.713+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=597U5NUXBR7F type=del 2024-04-22T19:58:41.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.714+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=982MR4SUT5YY type=del size=923039 2024-04-22T19:58:41.715 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.715+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=UTXCYGN66904 type=del 2024-04-22T19:58:41.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.717+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~95LX603TL938.0 size=1048576 2024-04-22T19:58:41.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.718+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~95LX603TL938.1 size=229132 2024-04-22T19:58:41.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.718+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=95LX603TL938 tag=VJQM9QX3DPXN type=write size=2326284 2024-04-22T19:58:41.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.719+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=PNPGRMFT20KQ type=del 2024-04-22T19:58:41.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.720+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=BSO59WUSRGQ1 type=write size=858938 2024-04-22T19:58:41.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.722+0000 7f5269a8e780 1 < suggested update operation key=B0Y7ZTPC0975 tag=06DI75WNKMON type=del size=1514698 2024-04-22T19:58:41.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.724+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=31TUH2FK54KU type=write size=373599 2024-04-22T19:58:41.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.726+0000 7f5269a8e780 1 < suggested update operation key=RY1O7X1CWZMK tag=OW83VD67NL07 type=del size=514425 2024-04-22T19:58:41.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.728+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~C8415OIQJPY0.0 size=364425 2024-04-22T19:58:41.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.728+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=C8415OIQJPY0 tag=TSRYWYFO5Y9F type=write size=1413001 2024-04-22T19:58:41.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.729+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=EVJSITMT6R8A type=del 2024-04-22T19:58:41.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.731+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~V1MCRHIRQDJ2.0 size=1048576 2024-04-22T19:58:41.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.732+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~V1MCRHIRQDJ2.1 size=1048576 2024-04-22T19:58:41.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.732+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~V1MCRHIRQDJ2.2 size=719284 2024-04-22T19:58:41.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.732+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=V1MCRHIRQDJ2 tag=WV5152BFZ2UI type=write size=3865012 2024-04-22T19:58:41.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.735+0000 7f5269a8e780 1 < suggested remove operation key=VPAVX5FXYWZO tag=SAVKAE156ROX type=del 2024-04-22T19:58:41.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.736+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=K98Y3CV1KR2W type=write size=1001022 2024-04-22T19:58:41.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.737+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~PZ613B11Z14J.0 size=1048576 2024-04-22T19:58:41.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.737+0000 7f5269a8e780 1 < canceled part key=_multipart_8V6ESIGMRO5G.2~PZ613B11Z14J.1 size=478180 2024-04-22T19:58:41.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.739+0000 7f5269a8e780 1 < canceled multipart upload key=8V6ESIGMRO5G upload=PZ613B11Z14J tag=YC7OH85RWFCF type=write size=2575332 2024-04-22T19:58:41.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.740+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=OY0MWY7J6ST9 type=del size=958738 2024-04-22T19:58:41.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.742+0000 7f5269a8e780 1 < suggested remove operation key=HV8J6YM0YNC1 tag=19A54ZM515W4 type=del 2024-04-22T19:58:41.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.743+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=KXE7VO4P83K6 type=del size=20419 2024-04-22T19:58:41.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.744+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~XMFZCC1CI4WN.0 size=111751 2024-04-22T19:58:41.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.746+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=XMFZCC1CI4WN tag=84LND5RYP6KP type=write size=1160327 2024-04-22T19:58:41.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.747+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=9B255DQRSIM2 type=del size=1844655 2024-04-22T19:58:41.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.749+0000 7f5269a8e780 1 < suggested remove operation key=DX5YUYQRZKWA tag=3F5YSG3OZCN0 type=del 2024-04-22T19:58:41.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.751+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~JQ9E69UCMVZU.0 size=132521 2024-04-22T19:58:41.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.751+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=JQ9E69UCMVZU tag=YIMOQOSKMQCB type=write size=1181097 2024-04-22T19:58:41.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.753+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=0680GGLFVXE0 type=del 2024-04-22T19:58:41.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.754+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=WEY1IKYRC4Q2 type=del size=750443 2024-04-22T19:58:41.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.756+0000 7f5269a8e780 1 < suggested remove operation key=A853MOZPT9BG tag=0N984O3J18OY type=del 2024-04-22T19:58:41.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.758+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~W2ZWWHTMV79N.0 size=1048576 2024-04-22T19:58:41.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.759+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~W2ZWWHTMV79N.1 size=858734 2024-04-22T19:58:41.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.759+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=W2ZWWHTMV79N tag=9QDRPSCMC1WV type=write size=2955886 2024-04-22T19:58:41.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.760+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~TO0DZ8O20NLY.0 size=1048576 2024-04-22T19:58:41.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.761+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~TO0DZ8O20NLY.1 size=1048576 2024-04-22T19:58:41.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.762+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~TO0DZ8O20NLY.2 size=471041 2024-04-22T19:58:41.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.764+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=TO0DZ8O20NLY tag=JP4H94T4YDHE type=write size=3616769 2024-04-22T19:58:41.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.766+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~WHE4P11HSPZM.0 size=669473 2024-04-22T19:58:41.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.766+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=WHE4P11HSPZM tag=2R3ZBXZP8Q9K type=write size=1718049 2024-04-22T19:58:41.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.767+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~C0HQQ1DX37IW.0 size=1048576 2024-04-22T19:58:41.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.768+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~C0HQQ1DX37IW.1 size=1048576 2024-04-22T19:58:41.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.769+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~C0HQQ1DX37IW.2 size=231438 2024-04-22T19:58:41.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.771+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=C0HQQ1DX37IW tag=M69W7F8KJURD type=write size=3377166 2024-04-22T19:58:41.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.772+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=QXZ5ZI7MXHOR type=del size=4007401 2024-04-22T19:58:41.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.773+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=XQWCGDYSZ1MG type=del 2024-04-22T19:58:41.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.775+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~QCY26FM3XAV1.0 size=21605 2024-04-22T19:58:41.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.775+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=QCY26FM3XAV1 tag=Y5W38FZOHZS2 type=write size=1070181 2024-04-22T19:58:41.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.777+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=982MR4SUT5YY type=del 2024-04-22T19:58:41.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.778+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=VQX1GK25IUWK type=del size=766900 2024-04-22T19:58:41.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.778+0000 7f5269a8e780 1 < canceled part key=_multipart_RY1O7X1CWZMK.2~95LX603TL938.0 size=1048576 2024-04-22T19:58:41.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.779+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~95LX603TL938.1 size=229132 2024-04-22T19:58:41.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.781+0000 7f5269a8e780 1 < canceled multipart upload key=RY1O7X1CWZMK upload=95LX603TL938 tag=VJQM9QX3DPXN type=write size=2326284 2024-04-22T19:58:41.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.783+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~7LZ7KEZV83OJ.0 size=1048576 2024-04-22T19:58:41.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.783+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~7LZ7KEZV83OJ.1 size=1048576 2024-04-22T19:58:41.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.784+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~7LZ7KEZV83OJ.2 size=820857 2024-04-22T19:58:41.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.784+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=7LZ7KEZV83OJ tag=P2MAN6N67NC0 type=write size=3966585 2024-04-22T19:58:41.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.785+0000 7f5269a8e780 1 < completed write operation key=86GW6HV9G1IZ tag=BSO59WUSRGQ1 type=write size=858938 2024-04-22T19:58:41.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.787+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=L4AQ3BTTHUKQ type=del size=1171325 2024-04-22T19:58:41.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.788+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=31TUH2FK54KU type=write size=373599 2024-04-22T19:58:41.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.789+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=ORQC1P2JNF2L type=del size=1422328 2024-04-22T19:58:41.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.790+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~C8415OIQJPY0.0 size=364425 2024-04-22T19:58:41.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.792+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=C8415OIQJPY0 tag=TSRYWYFO5Y9F type=write size=1413001 2024-04-22T19:58:41.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.794+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~CGGNGS98FF5V.0 size=1048576 2024-04-22T19:58:41.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.795+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~CGGNGS98FF5V.1 size=138890 2024-04-22T19:58:41.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.795+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=CGGNGS98FF5V tag=022C3NUJGXQT type=write size=2236042 2024-04-22T19:58:41.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.796+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~V1MCRHIRQDJ2.0 size=1048576 2024-04-22T19:58:41.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.797+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~V1MCRHIRQDJ2.1 size=1048576 2024-04-22T19:58:41.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.798+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~V1MCRHIRQDJ2.2 size=719284 2024-04-22T19:58:41.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.800+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=V1MCRHIRQDJ2 tag=WV5152BFZ2UI type=write size=3865012 2024-04-22T19:58:41.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.801+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=U0BJ8NTLVUPU type=write size=639862 2024-04-22T19:58:41.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.802+0000 7f5269a8e780 1 < completed write operation key=HIAXPVKZ1IIO tag=K98Y3CV1KR2W type=write size=1001022 2024-04-22T19:58:41.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.803+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=WIHYWV45KE16 type=write size=254010 2024-04-22T19:58:41.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.805+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=OY0MWY7J6ST9 type=del 2024-04-22T19:58:41.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.806+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=6GL44VDTPFAP type=write size=1043241 2024-04-22T19:58:41.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.807+0000 7f5269a8e780 -1 < no bi entry to suggest for operation key=A853MOZPT9BG tag=KXE7VO4P83K6 type=del size=20419: (2) No such file or directory 2024-04-22T19:58:41.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.808+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=COPKIBO6A9D3 type=del size=1413048 2024-04-22T19:58:41.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.810+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=9B255DQRSIM2 type=del size=1844655 2024-04-22T19:58:41.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.811+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=3DYZF9QOX00U type=write size=809091 2024-04-22T19:58:41.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.812+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~JQ9E69UCMVZU.0 size=132521 2024-04-22T19:58:41.814 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.814+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=JQ9E69UCMVZU tag=YIMOQOSKMQCB type=write size=1181097 2024-04-22T19:58:41.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.815+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=PNNS4VR3G6XH type=del size=2989356 2024-04-22T19:58:41.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.818+0000 7f5269a8e780 1 < suggested remove operation key=Q1UKF4UMJ2HO tag=WEY1IKYRC4Q2 type=del 2024-04-22T19:58:41.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.819+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=NGBGOQY0Y2AN type=del size=3367752 2024-04-22T19:58:41.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.820+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~W2ZWWHTMV79N.0 size=1048576 2024-04-22T19:58:41.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.821+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~W2ZWWHTMV79N.1 size=858734 2024-04-22T19:58:41.823 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.823+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=W2ZWWHTMV79N tag=9QDRPSCMC1WV type=write size=2955886 2024-04-22T19:58:41.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.824+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=7MA62YV4E8SY type=write size=933545 2024-04-22T19:58:41.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.825+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~WHE4P11HSPZM.0 size=669473 2024-04-22T19:58:41.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.827+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=WHE4P11HSPZM tag=2R3ZBXZP8Q9K type=write size=1718049 2024-04-22T19:58:41.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.829+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~T4EI4H1LZYBY.0 size=1048576 2024-04-22T19:58:41.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.830+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~T4EI4H1LZYBY.1 size=1048576 2024-04-22T19:58:41.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.831+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~T4EI4H1LZYBY.2 size=921675 2024-04-22T19:58:41.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.831+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=T4EI4H1LZYBY tag=EF9GLJ56N8CB type=write size=4067403 2024-04-22T19:58:41.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.832+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=QXZ5ZI7MXHOR type=del 2024-04-22T19:58:41.833 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.833+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=U6H5VVYP3Z5D type=del size=2504484 2024-04-22T19:58:41.835 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.835+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~QCY26FM3XAV1.0 size=21605 2024-04-22T19:58:41.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.837+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=QCY26FM3XAV1 tag=Y5W38FZOHZS2 type=write size=1070181 2024-04-22T19:58:41.838 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.838+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=JFZSZYPSCVNN type=del size=3442041 2024-04-22T19:58:41.839 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.839+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=VQX1GK25IUWK type=del 2024-04-22T19:58:41.840 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.840+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=L8LJMB93VTL1 type=del size=2442003 2024-04-22T19:58:41.841 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.841+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~7LZ7KEZV83OJ.0 size=1048576 2024-04-22T19:58:41.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.842+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~7LZ7KEZV83OJ.1 size=1048576 2024-04-22T19:58:41.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.843+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~7LZ7KEZV83OJ.2 size=820857 2024-04-22T19:58:41.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.845+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=7LZ7KEZV83OJ tag=P2MAN6N67NC0 type=write size=3966585 2024-04-22T19:58:41.846 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.847+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=B5EGR5V86SOV type=del size=177930 2024-04-22T19:58:41.848 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.848+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=L4AQ3BTTHUKQ type=del 2024-04-22T19:58:41.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.850+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~X9PSFMIWS4KJ.0 size=885182 2024-04-22T19:58:41.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.850+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=X9PSFMIWS4KJ tag=EN5PUOKJD9YV type=write size=1933758 2024-04-22T19:58:41.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.851+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=ORQC1P2JNF2L type=del 2024-04-22T19:58:41.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.853+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=AYDNBAVWCJOB type=write size=301079 2024-04-22T19:58:41.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.853+0000 7f5269a8e780 1 < canceled part key=_multipart_I91AD8FJ3P4T.2~CGGNGS98FF5V.0 size=1048576 2024-04-22T19:58:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.854+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~CGGNGS98FF5V.1 size=138890 2024-04-22T19:58:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.856+0000 7f5269a8e780 1 < canceled multipart upload key=I91AD8FJ3P4T upload=CGGNGS98FF5V tag=022C3NUJGXQT type=write size=2236042 2024-04-22T19:58:41.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.858+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~TSTWANAAFTX8.0 size=1048576 2024-04-22T19:58:41.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.859+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~TSTWANAAFTX8.1 size=1048576 2024-04-22T19:58:41.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.860+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~TSTWANAAFTX8.2 size=533013 2024-04-22T19:58:41.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.860+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=TSTWANAAFTX8 tag=R7X9ZXSTBRU8 type=write size=3678741 2024-04-22T19:58:41.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.862+0000 7f5269a8e780 1 < suggested remove operation key=8V6ESIGMRO5G tag=U0BJ8NTLVUPU type=write 2024-04-22T19:58:41.863 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.863+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=E5RBDP3J8GP2 type=del size=2582333 2024-04-22T19:58:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.865+0000 7f5269a8e780 1 < completed write operation key=RY1O7X1CWZMK tag=WIHYWV45KE16 type=write size=254010 2024-04-22T19:58:41.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.866+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=YCDPO2H4C6OH type=del size=419028 2024-04-22T19:58:41.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.867+0000 7f5269a8e780 1 < completed write operation key=70MLQGJPQYPY tag=6GL44VDTPFAP type=write size=1043241 2024-04-22T19:58:41.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.869+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~K5BDVTNZORK3.0 size=1048576 2024-04-22T19:58:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.871+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~K5BDVTNZORK3.1 size=567884 2024-04-22T19:58:41.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.871+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=K5BDVTNZORK3 tag=OCW2JZ2W8OPW type=write size=2665036 2024-04-22T19:58:41.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.873+0000 7f5269a8e780 1 < suggested remove operation key=637M7AP8X9E9 tag=COPKIBO6A9D3 type=del 2024-04-22T19:58:41.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.876+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~TF7AIWOLZUTD.0 size=1048576 2024-04-22T19:58:41.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.877+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~TF7AIWOLZUTD.1 size=1048576 2024-04-22T19:58:41.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.877+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~TF7AIWOLZUTD.2 size=466783 2024-04-22T19:58:41.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.877+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=TF7AIWOLZUTD tag=9JV1EJT1KLFC type=write size=3612511 2024-04-22T19:58:41.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.879+0000 7f5269a8e780 1 < completed write operation key=SN2BZATJGUWA tag=3DYZF9QOX00U type=write size=809091 2024-04-22T19:58:41.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.880+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=1IY06POK7B5E type=write size=168612 2024-04-22T19:58:41.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.881+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=PNNS4VR3G6XH type=del 2024-04-22T19:58:41.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.882+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=H9JGGG285LV1 type=del size=1424303 2024-04-22T19:58:41.884 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.884+0000 7f5269a8e780 1 < suggested update operation key=I91AD8FJ3P4T tag=NGBGOQY0Y2AN type=del size=3367752 2024-04-22T19:58:41.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.887+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~RF7OEOCYYIDJ.0 size=1048576 2024-04-22T19:58:41.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.887+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~RF7OEOCYYIDJ.1 size=291580 2024-04-22T19:58:41.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.887+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=RF7OEOCYYIDJ tag=FVJAXEHNF6KO type=write size=2388732 2024-04-22T19:58:41.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.890+0000 7f5269a8e780 1 < suggested update operation key=GX6NILQG878S tag=7MA62YV4E8SY type=write size=933545 2024-04-22T19:58:41.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.891+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=206AJ5L83XZF type=write size=1040829 2024-04-22T19:58:41.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.892+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~T4EI4H1LZYBY.0 size=1048576 2024-04-22T19:58:41.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.893+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~T4EI4H1LZYBY.1 size=1048576 2024-04-22T19:58:41.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.893+0000 7f5269a8e780 1 < canceled part key=_multipart_M9M4SEEEZSFV.2~T4EI4H1LZYBY.2 size=921675 2024-04-22T19:58:41.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.895+0000 7f5269a8e780 1 < canceled multipart upload key=M9M4SEEEZSFV upload=T4EI4H1LZYBY tag=EF9GLJ56N8CB type=write size=4067403 2024-04-22T19:58:41.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.896+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=TQSMT85Y2SI8 type=write size=538255 2024-04-22T19:58:41.897 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.897+0000 7f5269a8e780 -1 < failed to complete operation key=I91AD8FJ3P4T tag=U6H5VVYP3Z5D type=del size=2504484: (22) Invalid argument 2024-04-22T19:58:41.898 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.898+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=EJJLEALIHEIJ type=del size=48012 2024-04-22T19:58:41.899 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.899+0000 7f5269a8e780 1 < canceled operation key=8HFSDSNZONIN tag=JFZSZYPSCVNN type=del size=3442041 2024-04-22T19:58:41.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.901+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~JZXRZDL4JQBI.0 size=551851 2024-04-22T19:58:41.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.901+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=JZXRZDL4JQBI tag=41BBW0MKG5IR type=write size=1600427 2024-04-22T19:58:41.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.902+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=L8LJMB93VTL1 type=del 2024-04-22T19:58:41.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.903+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=D1H9I1MOWU0J type=del size=3585351 2024-04-22T19:58:41.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.904+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=B5EGR5V86SOV type=del 2024-04-22T19:58:41.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.906+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~LA1LPZSSYBYL.0 size=1048576 2024-04-22T19:58:41.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.907+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~LA1LPZSSYBYL.1 size=1048576 2024-04-22T19:58:41.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.908+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~LA1LPZSSYBYL.2 size=305125 2024-04-22T19:58:41.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.908+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=LA1LPZSSYBYL tag=X5Y00CS1R9UX type=write size=3450853 2024-04-22T19:58:41.909 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.910+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~X9PSFMIWS4KJ.0 size=885182 2024-04-22T19:58:41.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.912+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=X9PSFMIWS4KJ tag=EN5PUOKJD9YV type=write size=1933758 2024-04-22T19:58:41.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.914+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~NWE65VR2350Z.0 size=925494 2024-04-22T19:58:41.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.914+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=NWE65VR2350Z tag=AOFWD5C3DPPW type=write size=1974070 2024-04-22T19:58:41.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.916+0000 7f5269a8e780 1 < completed write operation key=DX5YUYQRZKWA tag=AYDNBAVWCJOB type=write size=301079 2024-04-22T19:58:41.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.917+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=NYY5V1JMGQ6N type=del size=1725173 2024-04-22T19:58:41.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.919+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~TSTWANAAFTX8.0 size=1048576 2024-04-22T19:58:41.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.919+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~TSTWANAAFTX8.1 size=1048576 2024-04-22T19:58:41.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.920+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~TSTWANAAFTX8.2 size=533013 2024-04-22T19:58:41.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.922+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=TSTWANAAFTX8 tag=R7X9ZXSTBRU8 type=write size=3678741 2024-04-22T19:58:41.923 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.923+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=QZKF28Z502L8 type=del size=1102995 2024-04-22T19:58:41.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.925+0000 7f5269a8e780 1 < suggested update operation key=A853MOZPT9BG tag=E5RBDP3J8GP2 type=del size=2582333 2024-04-22T19:58:41.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.926+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=F729JGBZSMKH type=write size=1010715 2024-04-22T19:58:41.929 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.929+0000 7f5269a8e780 1 < suggested update operation key=DX5YUYQRZKWA tag=YCDPO2H4C6OH type=del size=419028 2024-04-22T19:58:41.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.930+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=1R2Y2WESZXA3 type=del size=1159275 2024-04-22T19:58:41.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.931+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~K5BDVTNZORK3.0 size=1048576 2024-04-22T19:58:41.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.932+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~K5BDVTNZORK3.1 size=567884 2024-04-22T19:58:41.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.934+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=K5BDVTNZORK3 tag=OCW2JZ2W8OPW type=write size=2665036 2024-04-22T19:58:41.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.935+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=140PHEFUW0ZI type=del size=2110330 2024-04-22T19:58:41.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.936+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~TF7AIWOLZUTD.0 size=1048576 2024-04-22T19:58:41.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.936+0000 7f5269a8e780 1 < canceled part key=_multipart_HV8J6YM0YNC1.2~TF7AIWOLZUTD.1 size=1048576 2024-04-22T19:58:41.937 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.937+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~TF7AIWOLZUTD.2 size=466783 2024-04-22T19:58:41.939 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.939+0000 7f5269a8e780 1 < canceled multipart upload key=HV8J6YM0YNC1 upload=TF7AIWOLZUTD tag=9JV1EJT1KLFC type=write size=3612511 2024-04-22T19:58:41.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.940+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=SCAA8Q8733N3 type=del size=2419669 2024-04-22T19:58:41.941 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.941+0000 7f5269a8e780 1 < completed write operation key=ZC4ZOHUJ3HQD tag=1IY06POK7B5E type=write size=168612 2024-04-22T19:58:41.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.944+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~OY9M579ZW2S1.0 size=1048576 2024-04-22T19:58:41.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.945+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~OY9M579ZW2S1.1 size=1048576 2024-04-22T19:58:41.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.945+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~OY9M579ZW2S1.2 size=147155 2024-04-22T19:58:41.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.945+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=OY9M579ZW2S1 tag=RNA9YLKTV5VT type=write size=3292883 2024-04-22T19:58:41.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.948+0000 7f5269a8e780 1 < suggested update operation key=HIAXPVKZ1IIO tag=H9JGGG285LV1 type=del size=1424303 2024-04-22T19:58:41.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.950+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~Q6A0S1W7ZTYO.0 size=310501 2024-04-22T19:58:41.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.950+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=Q6A0S1W7ZTYO tag=G2Z5AVGAV4IV type=write size=1359077 2024-04-22T19:58:41.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.951+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~RF7OEOCYYIDJ.0 size=1048576 2024-04-22T19:58:41.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.952+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~RF7OEOCYYIDJ.1 size=291580 2024-04-22T19:58:41.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.954+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=RF7OEOCYYIDJ tag=FVJAXEHNF6KO type=write size=2388732 2024-04-22T19:58:41.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.956+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~UCC7OL02QDLV.0 size=766683 2024-04-22T19:58:41.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.956+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=UCC7OL02QDLV tag=S3SQXJHXJ15J type=write size=1815259 2024-04-22T19:58:41.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.957+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=206AJ5L83XZF type=write size=1040829 2024-04-22T19:58:41.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.958+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=M8WUGZKZCL9M type=write size=194631 2024-04-22T19:58:41.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.959+0000 7f5269a8e780 1 < completed write operation key=637M7AP8X9E9 tag=TQSMT85Y2SI8 type=write size=538255 2024-04-22T19:58:41.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.961+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=4UXYYD0MXXDP type=del size=106388 2024-04-22T19:58:41.962 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.962+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=EJJLEALIHEIJ type=del 2024-04-22T19:58:41.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.963+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=UGHTMOF3J2U5 type=write size=22050 2024-04-22T19:58:41.964 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.964+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~JZXRZDL4JQBI.0 size=551851 2024-04-22T19:58:41.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.966+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=JZXRZDL4JQBI tag=41BBW0MKG5IR type=write size=1600427 2024-04-22T19:58:41.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.967+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=T0EEBWXUKB03 type=write size=412443 2024-04-22T19:58:41.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.968+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=D1H9I1MOWU0J type=del 2024-04-22T19:58:41.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.970+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~7PC60CJA06K1.0 size=1048576 2024-04-22T19:58:41.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.971+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~7PC60CJA06K1.1 size=781551 2024-04-22T19:58:41.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.971+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=7PC60CJA06K1 tag=18W50HJRY722 type=write size=2878703 2024-04-22T19:58:41.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.972+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~LA1LPZSSYBYL.0 size=1048576 2024-04-22T19:58:41.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.973+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~LA1LPZSSYBYL.1 size=1048576 2024-04-22T19:58:41.974 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.974+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~LA1LPZSSYBYL.2 size=305125 2024-04-22T19:58:41.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.976+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=LA1LPZSSYBYL tag=X5Y00CS1R9UX type=write size=3450853 2024-04-22T19:58:41.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.978+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~NY3JFA7ZMWUH.0 size=1048576 2024-04-22T19:58:41.979 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.979+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~NY3JFA7ZMWUH.1 size=1048576 2024-04-22T19:58:41.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.980+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~NY3JFA7ZMWUH.2 size=230317 2024-04-22T19:58:41.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.980+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=NY3JFA7ZMWUH tag=2Y7N8TBPLJ6H type=write size=3376045 2024-04-22T19:58:41.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.981+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~NWE65VR2350Z.0 size=925494 2024-04-22T19:58:41.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.984+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=NWE65VR2350Z tag=AOFWD5C3DPPW type=write size=1974070 2024-04-22T19:58:41.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.986+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~RMVJSX6N984Y.0 size=1048576 2024-04-22T19:58:41.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.987+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~RMVJSX6N984Y.1 size=1048576 2024-04-22T19:58:41.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.988+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~RMVJSX6N984Y.2 size=1013962 2024-04-22T19:58:41.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.988+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=RMVJSX6N984Y tag=KRTVBPOR2000 type=write size=4159690 2024-04-22T19:58:41.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.989+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=NYY5V1JMGQ6N type=del 2024-04-22T19:58:41.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.990+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=52DNUGFXKTT7 type=del size=2908424 2024-04-22T19:58:41.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.992+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=QZKF28Z502L8 type=del 2024-04-22T19:58:41.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.993+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=TRZQM123UE8T type=del size=3658050 2024-04-22T19:58:41.994 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.994+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=F729JGBZSMKH type=write size=1010715 2024-04-22T19:58:41.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.996+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~LZ9PL6DV3E3S.0 size=1048576 2024-04-22T19:58:41.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.997+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~LZ9PL6DV3E3S.1 size=310466 2024-04-22T19:58:41.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.997+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=LZ9PL6DV3E3S tag=JQ7L0BPPN3EG type=write size=2407618 2024-04-22T19:58:41.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:41.998+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=1R2Y2WESZXA3 type=del 2024-04-22T19:58:42.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.000+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~40E0OZHV265Y.0 size=694170 2024-04-22T19:58:42.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.000+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=40E0OZHV265Y tag=EU3CQ421PNKS type=write size=1742746 2024-04-22T19:58:42.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.002+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=140PHEFUW0ZI type=del 2024-04-22T19:58:42.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.004+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~A7ACK2RDZ3ZV.0 size=553494 2024-04-22T19:58:42.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.004+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=A7ACK2RDZ3ZV tag=NZ6L7QM80A8U type=write size=1602070 2024-04-22T19:58:42.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.006+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=SCAA8Q8733N3 type=del size=2419669 2024-04-22T19:58:42.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.007+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=K6HAJL51ZAWA type=write size=347992 2024-04-22T19:58:42.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.008+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~OY9M579ZW2S1.0 size=1048576 2024-04-22T19:58:42.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.009+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~OY9M579ZW2S1.1 size=1048576 2024-04-22T19:58:42.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.010+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~OY9M579ZW2S1.2 size=147155 2024-04-22T19:58:42.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.012+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=OY9M579ZW2S1 tag=RNA9YLKTV5VT type=write size=3292883 2024-04-22T19:58:42.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.013+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=GPR52S8T63Z6 type=del size=1737840 2024-04-22T19:58:42.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.013+0000 7f5269a8e780 1 < canceled part key=_multipart_HV8J6YM0YNC1.2~Q6A0S1W7ZTYO.0 size=310501 2024-04-22T19:58:42.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.015+0000 7f5269a8e780 1 < canceled multipart upload key=HV8J6YM0YNC1 upload=Q6A0S1W7ZTYO tag=G2Z5AVGAV4IV type=write size=1359077 2024-04-22T19:58:42.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.016+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=IPUN16SYM1X0 type=del size=2510817 2024-04-22T19:58:42.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.017+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~UCC7OL02QDLV.0 size=766683 2024-04-22T19:58:42.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.019+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=UCC7OL02QDLV tag=S3SQXJHXJ15J type=write size=1815259 2024-04-22T19:58:42.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.021+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~0BWOIQPHJQQS.0 size=1048576 2024-04-22T19:58:42.023 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.023+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~0BWOIQPHJQQS.1 size=1048576 2024-04-22T19:58:42.023 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.024+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~0BWOIQPHJQQS.2 size=583494 2024-04-22T19:58:42.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.024+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=0BWOIQPHJQQS tag=M3SEIAYFJ7AD type=write size=3729222 2024-04-22T19:58:42.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.027+0000 7f5269a8e780 1 < suggested update operation key=4EA3LUQRL8II tag=M8WUGZKZCL9M type=write size=194631 2024-04-22T19:58:42.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.028+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=0ZKE78HMMSWL type=write size=71194 2024-04-22T19:58:42.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.030+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=4UXYYD0MXXDP type=del 2024-04-22T19:58:42.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.032+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~AL08NS46NI25.0 size=65766 2024-04-22T19:58:42.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.032+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=AL08NS46NI25 tag=N79VR46HET13 type=write size=1114342 2024-04-22T19:58:42.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.033+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=UGHTMOF3J2U5 type=write size=22050 2024-04-22T19:58:42.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.034+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=IKKH0JKOZ0N9 type=write size=345783 2024-04-22T19:58:42.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.035+0000 7f5269a8e780 1 < completed write operation key=637M7AP8X9E9 tag=T0EEBWXUKB03 type=write size=412443 2024-04-22T19:58:42.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.037+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~DYXPCQ2YNREL.0 size=1048576 2024-04-22T19:58:42.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.038+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~DYXPCQ2YNREL.1 size=83381 2024-04-22T19:58:42.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.038+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=DYXPCQ2YNREL tag=8Y4M2HU112MG type=write size=2180533 2024-04-22T19:58:42.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.039+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~7PC60CJA06K1.0 size=1048576 2024-04-22T19:58:42.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.040+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~7PC60CJA06K1.1 size=781551 2024-04-22T19:58:42.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.041+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=7PC60CJA06K1 tag=18W50HJRY722 type=write size=2878703 2024-04-22T19:58:42.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.042+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=HQ3TSWOC6LFV type=del size=456379 2024-04-22T19:58:42.043 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.044+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~NY3JFA7ZMWUH.0 size=1048576 2024-04-22T19:58:42.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.044+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~NY3JFA7ZMWUH.1 size=1048576 2024-04-22T19:58:42.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.045+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~NY3JFA7ZMWUH.2 size=230317 2024-04-22T19:58:42.047 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.047+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=NY3JFA7ZMWUH tag=2Y7N8TBPLJ6H type=write size=3376045 2024-04-22T19:58:42.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.049+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~CTR0NO5PXYVU.0 size=1048576 2024-04-22T19:58:42.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.050+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~CTR0NO5PXYVU.1 size=666647 2024-04-22T19:58:42.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.050+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=CTR0NO5PXYVU tag=9LYKXNJIEL1B type=write size=2763799 2024-04-22T19:58:42.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.051+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~RMVJSX6N984Y.0 size=1048576 2024-04-22T19:58:42.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.051+0000 7f5269a8e780 1 < canceled part key=_multipart_VPAVX5FXYWZO.2~RMVJSX6N984Y.1 size=1048576 2024-04-22T19:58:42.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.051+0000 7f5269a8e780 1 < canceled part key=_multipart_VPAVX5FXYWZO.2~RMVJSX6N984Y.2 size=1013962 2024-04-22T19:58:42.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.054+0000 7f5269a8e780 1 < canceled multipart upload key=VPAVX5FXYWZO upload=RMVJSX6N984Y tag=KRTVBPOR2000 type=write size=4159690 2024-04-22T19:58:42.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.056+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~GA57QNYA6R13.0 size=1048576 2024-04-22T19:58:42.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.057+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~GA57QNYA6R13.1 size=736637 2024-04-22T19:58:42.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.057+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=GA57QNYA6R13 tag=CZUYA3AB7ELP type=write size=2833789 2024-04-22T19:58:42.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.058+0000 7f5269a8e780 1 < canceled operation key=SN2BZATJGUWA tag=52DNUGFXKTT7 type=del size=2908424 2024-04-22T19:58:42.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.060+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~67ZD7ATYXCBH.0 size=1048576 2024-04-22T19:58:42.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.061+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~67ZD7ATYXCBH.1 size=275499 2024-04-22T19:58:42.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.061+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=67ZD7ATYXCBH tag=XEC9N6ZWEKNZ type=write size=2372651 2024-04-22T19:58:42.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.062+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=TRZQM123UE8T type=del 2024-04-22T19:58:42.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.063+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=3RDLBJZI67VN type=del size=3256038 2024-04-22T19:58:42.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.064+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~LZ9PL6DV3E3S.0 size=1048576 2024-04-22T19:58:42.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.064+0000 7f5269a8e780 1 < canceled part key=_multipart_4GX83ZJWZCD1.2~LZ9PL6DV3E3S.1 size=310466 2024-04-22T19:58:42.066 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.066+0000 7f5269a8e780 1 < canceled multipart upload key=4GX83ZJWZCD1 upload=LZ9PL6DV3E3S tag=JQ7L0BPPN3EG type=write size=2407618 2024-04-22T19:58:42.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.067+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=3ZQAFL08QIQB type=del size=1248310 2024-04-22T19:58:42.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.068+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~40E0OZHV265Y.0 size=694170 2024-04-22T19:58:42.070 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.070+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=40E0OZHV265Y tag=EU3CQ421PNKS type=write size=1742746 2024-04-22T19:58:42.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.072+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~3N8R7DAPJ8UH.0 size=1048576 2024-04-22T19:58:42.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.073+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~3N8R7DAPJ8UH.1 size=304991 2024-04-22T19:58:42.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.073+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=3N8R7DAPJ8UH tag=JQCQNQ60OAQN type=write size=2402143 2024-04-22T19:58:42.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.074+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~A7ACK2RDZ3ZV.0 size=553494 2024-04-22T19:58:42.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.076+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=A7ACK2RDZ3ZV tag=NZ6L7QM80A8U type=write size=1602070 2024-04-22T19:58:42.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.078+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~IZ4BVWLBM49G.0 size=1048576 2024-04-22T19:58:42.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.079+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~IZ4BVWLBM49G.1 size=1048576 2024-04-22T19:58:42.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.080+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~IZ4BVWLBM49G.2 size=237945 2024-04-22T19:58:42.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.080+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=IZ4BVWLBM49G tag=ANUM8AXZ4S2Z type=write size=3383673 2024-04-22T19:58:42.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.082+0000 7f5269a8e780 1 < suggested remove operation key=RY1O7X1CWZMK tag=K6HAJL51ZAWA type=write 2024-04-22T19:58:42.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.083+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=W7YFF2CSGZO2 type=write size=673659 2024-04-22T19:58:42.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.083+0000 7f5269a8e780 -1 < no bi entry to suggest for operation key=RY1O7X1CWZMK tag=GPR52S8T63Z6 type=del size=1737840: (2) No such file or directory 2024-04-22T19:58:42.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.085+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=EULS7BJ7HHJB type=del size=3185214 2024-04-22T19:58:42.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.086+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=IPUN16SYM1X0 type=del 2024-04-22T19:58:42.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.087+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=V74P8UQGKKWJ type=del size=1774792 2024-04-22T19:58:42.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.088+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~0BWOIQPHJQQS.0 size=1048576 2024-04-22T19:58:42.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.089+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~0BWOIQPHJQQS.1 size=1048576 2024-04-22T19:58:42.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.090+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~0BWOIQPHJQQS.2 size=583494 2024-04-22T19:58:42.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.092+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=0BWOIQPHJQQS tag=M3SEIAYFJ7AD type=write size=3729222 2024-04-22T19:58:42.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.094+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~VO065SGOJY8B.0 size=1048576 2024-04-22T19:58:42.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.095+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~VO065SGOJY8B.1 size=1048576 2024-04-22T19:58:42.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.096+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~VO065SGOJY8B.2 size=706235 2024-04-22T19:58:42.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.096+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=VO065SGOJY8B tag=FICAA8FDDPFV type=write size=3851963 2024-04-22T19:58:42.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.097+0000 7f5269a8e780 1 < canceled operation key=637M7AP8X9E9 tag=0ZKE78HMMSWL type=write size=71194 2024-04-22T19:58:42.099 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.099+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~O9UZG1IPQTKU.0 size=1048576 2024-04-22T19:58:42.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.100+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~O9UZG1IPQTKU.1 size=1048576 2024-04-22T19:58:42.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.101+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~O9UZG1IPQTKU.2 size=458668 2024-04-22T19:58:42.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.101+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=O9UZG1IPQTKU tag=XJ0KK66LLYBL type=write size=3604396 2024-04-22T19:58:42.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.102+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~AL08NS46NI25.0 size=65766 2024-04-22T19:58:42.104 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.104+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=AL08NS46NI25 tag=N79VR46HET13 type=write size=1114342 2024-04-22T19:58:42.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.105+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=R5YT25HBCWX5 type=del size=3998999 2024-04-22T19:58:42.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.106+0000 7f5269a8e780 1 < canceled operation key=KV5DYQ49ZKCX tag=IKKH0JKOZ0N9 type=write size=345783 2024-04-22T19:58:42.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.107+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=PZWOF1NRUWMA type=del size=961670 2024-04-22T19:58:42.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.108+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~DYXPCQ2YNREL.0 size=1048576 2024-04-22T19:58:42.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.109+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~DYXPCQ2YNREL.1 size=83381 2024-04-22T19:58:42.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.111+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=DYXPCQ2YNREL tag=8Y4M2HU112MG type=write size=2180533 2024-04-22T19:58:42.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.112+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=FI6XH7BEN5HP type=del size=3709061 2024-04-22T19:58:42.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.113+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=HQ3TSWOC6LFV type=del 2024-04-22T19:58:42.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.115+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~80J25WPN06UP.0 size=1048576 2024-04-22T19:58:42.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.116+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~80J25WPN06UP.1 size=828921 2024-04-22T19:58:42.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.116+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=80J25WPN06UP tag=JM79HW25Q4Q7 type=write size=2926073 2024-04-22T19:58:42.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.117+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~CTR0NO5PXYVU.0 size=1048576 2024-04-22T19:58:42.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.118+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~CTR0NO5PXYVU.1 size=666647 2024-04-22T19:58:42.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.120+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=CTR0NO5PXYVU tag=9LYKXNJIEL1B type=write size=2763799 2024-04-22T19:58:42.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.121+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=4LUIY68C5R0V type=del size=3588974 2024-04-22T19:58:42.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.122+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~GA57QNYA6R13.0 size=1048576 2024-04-22T19:58:42.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.123+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~GA57QNYA6R13.1 size=736637 2024-04-22T19:58:42.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.125+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=GA57QNYA6R13 tag=CZUYA3AB7ELP type=write size=2833789 2024-04-22T19:58:42.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.128+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~KOYK1DE8T8A7.0 size=119699 2024-04-22T19:58:42.128 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.128+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=KOYK1DE8T8A7 tag=0XY2DG0UGZTW type=write size=1168275 2024-04-22T19:58:42.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.130+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~67ZD7ATYXCBH.0 size=1048576 2024-04-22T19:58:42.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.131+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~67ZD7ATYXCBH.1 size=275499 2024-04-22T19:58:42.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.133+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=67ZD7ATYXCBH tag=XEC9N6ZWEKNZ type=write size=2372651 2024-04-22T19:58:42.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.135+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~IBW7X653WZUM.0 size=1048576 2024-04-22T19:58:42.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.136+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~IBW7X653WZUM.1 size=464854 2024-04-22T19:58:42.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.136+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=IBW7X653WZUM tag=SS844NV3XI17 type=write size=2562006 2024-04-22T19:58:42.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.138+0000 7f5269a8e780 1 < suggested update operation key=4GX83ZJWZCD1 tag=3RDLBJZI67VN type=del size=3256038 2024-04-22T19:58:42.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.139+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=GJFL1R59EUVA type=del size=60887 2024-04-22T19:58:42.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.140+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=3ZQAFL08QIQB type=del 2024-04-22T19:58:42.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.142+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~E91WN88E1DBM.0 size=1048576 2024-04-22T19:58:42.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.143+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~E91WN88E1DBM.1 size=1048576 2024-04-22T19:58:42.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.144+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~E91WN88E1DBM.2 size=319277 2024-04-22T19:58:42.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.144+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=E91WN88E1DBM tag=9T2C5TD3143Y type=write size=3465005 2024-04-22T19:58:42.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.145+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~3N8R7DAPJ8UH.0 size=1048576 2024-04-22T19:58:42.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.146+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~3N8R7DAPJ8UH.1 size=304991 2024-04-22T19:58:42.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.147+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=3N8R7DAPJ8UH tag=JQCQNQ60OAQN type=write size=2402143 2024-04-22T19:58:42.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.148+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=YA738LX3ANSE type=write size=905698 2024-04-22T19:58:42.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.150+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~IZ4BVWLBM49G.0 size=1048576 2024-04-22T19:58:42.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.150+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~IZ4BVWLBM49G.1 size=1048576 2024-04-22T19:58:42.151 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.151+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~IZ4BVWLBM49G.2 size=237945 2024-04-22T19:58:42.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.153+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=IZ4BVWLBM49G tag=ANUM8AXZ4S2Z type=write size=3383673 2024-04-22T19:58:42.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.154+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=W8IWEOL01BF5 type=write size=667415 2024-04-22T19:58:42.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.156+0000 7f5269a8e780 1 < suggested update operation key=M9M4SEEEZSFV tag=W7YFF2CSGZO2 type=write size=673659 2024-04-22T19:58:42.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.159+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~CX4AHG3HP1I0.0 size=879566 2024-04-22T19:58:42.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.159+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=CX4AHG3HP1I0 tag=NLTMHABN2I8T type=write size=1928142 2024-04-22T19:58:42.161 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.161+0000 7f5269a8e780 1 < suggested update operation key=8HFSDSNZONIN tag=EULS7BJ7HHJB type=del size=3185214 2024-04-22T19:58:42.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.163+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~JMGF5VQUFFZW.0 size=1026944 2024-04-22T19:58:42.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.163+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=JMGF5VQUFFZW tag=CERV09QR14VC type=write size=2075520 2024-04-22T19:58:42.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.164+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=V74P8UQGKKWJ type=del 2024-04-22T19:58:42.165 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.165+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=YCLO03LA6JKZ type=del size=1063916 2024-04-22T19:58:42.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.166+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~VO065SGOJY8B.0 size=1048576 2024-04-22T19:58:42.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.167+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~VO065SGOJY8B.1 size=1048576 2024-04-22T19:58:42.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.168+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~VO065SGOJY8B.2 size=706235 2024-04-22T19:58:42.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.170+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=VO065SGOJY8B tag=FICAA8FDDPFV type=write size=3851963 2024-04-22T19:58:42.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.172+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~PZO8HKQO8MBV.0 size=1048576 2024-04-22T19:58:42.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.173+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~PZO8HKQO8MBV.1 size=9461 2024-04-22T19:58:42.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.173+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=PZO8HKQO8MBV tag=DL7FONSI24AY type=write size=2106613 2024-04-22T19:58:42.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.175+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~O9UZG1IPQTKU.0 size=1048576 2024-04-22T19:58:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.176+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~O9UZG1IPQTKU.1 size=1048576 2024-04-22T19:58:42.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.177+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~O9UZG1IPQTKU.2 size=458668 2024-04-22T19:58:42.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.179+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=O9UZG1IPQTKU tag=XJ0KK66LLYBL type=write size=3604396 2024-04-22T19:58:42.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.181+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~TDFZE0YG2MYX.0 size=1048576 2024-04-22T19:58:42.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.182+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~TDFZE0YG2MYX.1 size=1048576 2024-04-22T19:58:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.183+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~TDFZE0YG2MYX.2 size=925009 2024-04-22T19:58:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.183+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=TDFZE0YG2MYX tag=WQEB7YFX29EJ type=write size=4070737 2024-04-22T19:58:42.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.184+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=R5YT25HBCWX5 type=del 2024-04-22T19:58:42.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.186+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=21T2P8QNZS5O type=del size=2173750 2024-04-22T19:58:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.187+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=PZWOF1NRUWMA type=del 2024-04-22T19:58:42.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.189+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~1S7XN3TAFP31.0 size=1048576 2024-04-22T19:58:42.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.190+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~1S7XN3TAFP31.1 size=665337 2024-04-22T19:58:42.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.190+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=1S7XN3TAFP31 tag=I20O15DU1JKQ type=write size=2762489 2024-04-22T19:58:42.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.191+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=FI6XH7BEN5HP type=del 2024-04-22T19:58:42.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.192+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=JQX9RE2XS7E9 type=del size=3047268 2024-04-22T19:58:42.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.195+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~80J25WPN06UP.0 size=1048576 2024-04-22T19:58:42.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.196+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~80J25WPN06UP.1 size=828921 2024-04-22T19:58:42.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.198+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=80J25WPN06UP tag=JM79HW25Q4Q7 type=write size=2926073 2024-04-22T19:58:42.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.199+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=T73XJ5TO0PAS type=write size=945931 2024-04-22T19:58:42.200 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.200+0000 7f5269a8e780 1 < canceled operation key=RY1O7X1CWZMK tag=4LUIY68C5R0V type=del size=3588974 2024-04-22T19:58:42.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.201+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=7TAB9H0ATSN1 type=del size=3404551 2024-04-22T19:58:42.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.202+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~KOYK1DE8T8A7.0 size=119699 2024-04-22T19:58:42.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.204+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=KOYK1DE8T8A7 tag=0XY2DG0UGZTW type=write size=1168275 2024-04-22T19:58:42.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.205+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=GR8O2TZELS98 type=del size=385746 2024-04-22T19:58:42.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.206+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~IBW7X653WZUM.0 size=1048576 2024-04-22T19:58:42.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.207+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~IBW7X653WZUM.1 size=464854 2024-04-22T19:58:42.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.209+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=IBW7X653WZUM tag=SS844NV3XI17 type=write size=2562006 2024-04-22T19:58:42.211 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.211+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~TUXXWCAQ2ZBZ.0 size=1048576 2024-04-22T19:58:42.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.212+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~TUXXWCAQ2ZBZ.1 size=240079 2024-04-22T19:58:42.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.212+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=TUXXWCAQ2ZBZ tag=TXZQIPWAZ9UJ type=write size=2337231 2024-04-22T19:58:42.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.214+0000 7f5269a8e780 1 < suggested remove operation key=DX5YUYQRZKWA tag=GJFL1R59EUVA type=del 2024-04-22T19:58:42.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.215+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=9OB0RSOTU1DL type=del size=2672812 2024-04-22T19:58:42.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.216+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~E91WN88E1DBM.0 size=1048576 2024-04-22T19:58:42.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.217+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~E91WN88E1DBM.1 size=1048576 2024-04-22T19:58:42.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.217+0000 7f5269a8e780 1 < canceled part key=_multipart_IBH6W1NINUHC.2~E91WN88E1DBM.2 size=319277 2024-04-22T19:58:42.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.219+0000 7f5269a8e780 1 < canceled multipart upload key=IBH6W1NINUHC upload=E91WN88E1DBM tag=9T2C5TD3143Y type=write size=3465005 2024-04-22T19:58:42.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.220+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=0OVG6GE6KPNH type=del size=3935720 2024-04-22T19:58:42.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.221+0000 7f5269a8e780 1 < canceled operation key=2YFD2U6HVNE7 tag=YA738LX3ANSE type=write size=905698 2024-04-22T19:58:42.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.223+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=S5QEVN6YASKR type=del size=2589494 2024-04-22T19:58:42.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.224+0000 7f5269a8e780 1 < canceled operation key=70MLQGJPQYPY tag=W8IWEOL01BF5 type=write size=667415 2024-04-22T19:58:42.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.225+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=ATZCJWJLV7Y0 type=write size=289159 2024-04-22T19:58:42.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.226+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~CX4AHG3HP1I0.0 size=879566 2024-04-22T19:58:42.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.228+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=CX4AHG3HP1I0 tag=NLTMHABN2I8T type=write size=1928142 2024-04-22T19:58:42.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.230+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=V9B5XDX4KIEN type=del size=2579185 2024-04-22T19:58:42.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.236+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~JMGF5VQUFFZW.0 size=1026944 2024-04-22T19:58:42.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.238+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=JMGF5VQUFFZW tag=CERV09QR14VC type=write size=2075520 2024-04-22T19:58:42.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.243+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~HKUQPFKT0G8L.0 size=1048576 2024-04-22T19:58:42.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.244+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~HKUQPFKT0G8L.1 size=720630 2024-04-22T19:58:42.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.244+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=HKUQPFKT0G8L tag=C05LQJHR1C8F type=write size=2817782 2024-04-22T19:58:42.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.245+0000 7f5269a8e780 1 < canceled operation key=B0Y7ZTPC0975 tag=YCLO03LA6JKZ type=del size=1063916 2024-04-22T19:58:42.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.247+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~2Q4XPCEQYCAZ.0 size=749649 2024-04-22T19:58:42.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.247+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=2Q4XPCEQYCAZ tag=5ZSADVGV7PQ5 type=write size=1798225 2024-04-22T19:58:42.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.249+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~PZO8HKQO8MBV.0 size=1048576 2024-04-22T19:58:42.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.250+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~PZO8HKQO8MBV.1 size=9461 2024-04-22T19:58:42.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.252+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=PZO8HKQO8MBV tag=DL7FONSI24AY type=write size=2106613 2024-04-22T19:58:42.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.253+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=S6VS5ZZX3X7W type=write size=785369 2024-04-22T19:58:42.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.254+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~TDFZE0YG2MYX.0 size=1048576 2024-04-22T19:58:42.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.255+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~TDFZE0YG2MYX.1 size=1048576 2024-04-22T19:58:42.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.256+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~TDFZE0YG2MYX.2 size=925009 2024-04-22T19:58:42.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.258+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=TDFZE0YG2MYX tag=WQEB7YFX29EJ type=write size=4070737 2024-04-22T19:58:42.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.259+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=2VCN18L8ICCF type=write size=1041252 2024-04-22T19:58:42.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.261+0000 7f5269a8e780 1 < suggested remove operation key=4GX83ZJWZCD1 tag=21T2P8QNZS5O type=del 2024-04-22T19:58:42.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.262+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=8IIOYN1LE9I6 type=write size=463092 2024-04-22T19:58:42.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.263+0000 7f5269a8e780 1 < canceled part key=_multipart_SN2BZATJGUWA.2~1S7XN3TAFP31.0 size=1048576 2024-04-22T19:58:42.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.263+0000 7f5269a8e780 1 < canceled part key=_multipart_SN2BZATJGUWA.2~1S7XN3TAFP31.1 size=665337 2024-04-22T19:58:42.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.265+0000 7f5269a8e780 1 < canceled multipart upload key=SN2BZATJGUWA upload=1S7XN3TAFP31 tag=I20O15DU1JKQ type=write size=2762489 2024-04-22T19:58:42.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.267+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~LI5MAVLBZHSE.0 size=141969 2024-04-22T19:58:42.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.267+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=LI5MAVLBZHSE tag=DFQ3NP24TYG6 type=write size=1190545 2024-04-22T19:58:42.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.268+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=JQX9RE2XS7E9 type=del 2024-04-22T19:58:42.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.270+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~WGB1WZY0BKIT.0 size=1048576 2024-04-22T19:58:42.271 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.271+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~WGB1WZY0BKIT.1 size=1048576 2024-04-22T19:58:42.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.272+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~WGB1WZY0BKIT.2 size=992133 2024-04-22T19:58:42.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.272+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=WGB1WZY0BKIT tag=1TI688IDQ56F type=write size=4137861 2024-04-22T19:58:42.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.273+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=T73XJ5TO0PAS type=write size=945931 2024-04-22T19:58:42.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.274+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=SAEZVJY3FYC6 type=write size=405046 2024-04-22T19:58:42.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.275+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=7TAB9H0ATSN1 type=del 2024-04-22T19:58:42.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.276+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=VINYPATVIX7Y type=write size=307677 2024-04-22T19:58:42.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.278+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=GR8O2TZELS98 type=del 2024-04-22T19:58:42.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.280+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~O78A3CZ4B31F.0 size=1048576 2024-04-22T19:58:42.280 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.280+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~O78A3CZ4B31F.1 size=902360 2024-04-22T19:58:42.280 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.280+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=O78A3CZ4B31F tag=S7BM480HKRGP type=write size=2999512 2024-04-22T19:58:42.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.282+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~TUXXWCAQ2ZBZ.0 size=1048576 2024-04-22T19:58:42.282 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.283+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~TUXXWCAQ2ZBZ.1 size=240079 2024-04-22T19:58:42.284 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.284+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=TUXXWCAQ2ZBZ tag=TXZQIPWAZ9UJ type=write size=2337231 2024-04-22T19:58:42.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.286+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=EK7020TH018R type=del size=1587579 2024-04-22T19:58:42.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.288+0000 7f5269a8e780 1 < suggested update operation key=8V6ESIGMRO5G tag=9OB0RSOTU1DL type=del size=2672812 2024-04-22T19:58:42.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.289+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=EY5648RGGRBN type=del size=3878660 2024-04-22T19:58:42.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.290+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=0OVG6GE6KPNH type=del 2024-04-22T19:58:42.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.292+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~EUV01MRKN1EC.0 size=1048576 2024-04-22T19:58:42.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.293+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~EUV01MRKN1EC.1 size=729726 2024-04-22T19:58:42.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.293+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=EUV01MRKN1EC tag=9KP7NT32IOIN type=write size=2826878 2024-04-22T19:58:42.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.294+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=S5QEVN6YASKR type=del 2024-04-22T19:58:42.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.296+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~L99NRGU12AHI.0 size=1048576 2024-04-22T19:58:42.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.298+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~L99NRGU12AHI.1 size=1048576 2024-04-22T19:58:42.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.299+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~L99NRGU12AHI.2 size=617771 2024-04-22T19:58:42.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.299+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=L99NRGU12AHI tag=CYKRRPRK8BYA type=write size=3763499 2024-04-22T19:58:42.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.301+0000 7f5269a8e780 1 < suggested remove operation key=T2BV2LUCY4E3 tag=ATZCJWJLV7Y0 type=write 2024-04-22T19:58:42.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.303+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~0A2MB9ESTCN2.0 size=675501 2024-04-22T19:58:42.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.303+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=0A2MB9ESTCN2 tag=CYPXL0HDB2EW type=write size=1724077 2024-04-22T19:58:42.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.304+0000 7f5269a8e780 1 < canceled operation key=SN2BZATJGUWA tag=V9B5XDX4KIEN type=del size=2579185 2024-04-22T19:58:42.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.306+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~86H60GJESCQN.0 size=1048576 2024-04-22T19:58:42.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.307+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~86H60GJESCQN.1 size=30249 2024-04-22T19:58:42.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.307+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=86H60GJESCQN tag=WULPDJQ8CS6M type=write size=2127401 2024-04-22T19:58:42.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.307+0000 7f5269a8e780 1 < canceled part key=_multipart_GX6NILQG878S.2~HKUQPFKT0G8L.0 size=1048576 2024-04-22T19:58:42.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.308+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~HKUQPFKT0G8L.1 size=720630 2024-04-22T19:58:42.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.310+0000 7f5269a8e780 1 < canceled multipart upload key=GX6NILQG878S upload=HKUQPFKT0G8L tag=C05LQJHR1C8F type=write size=2817782 2024-04-22T19:58:42.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.312+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~S7U4LFCD8Z8F.0 size=394246 2024-04-22T19:58:42.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.312+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=S7U4LFCD8Z8F tag=MELKOHWRQ4I6 type=write size=1442822 2024-04-22T19:58:42.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.313+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~2Q4XPCEQYCAZ.0 size=749649 2024-04-22T19:58:42.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.315+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=2Q4XPCEQYCAZ tag=5ZSADVGV7PQ5 type=write size=1798225 2024-04-22T19:58:42.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.316+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=E11SDF1ACQ5G type=del size=2906881 2024-04-22T19:58:42.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.317+0000 7f5269a8e780 1 < completed write operation key=637M7AP8X9E9 tag=S6VS5ZZX3X7W type=write size=785369 2024-04-22T19:58:42.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.318+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=BOT5E6KEY7ZT type=del size=3000072 2024-04-22T19:58:42.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.320+0000 7f5269a8e780 1 < suggested update operation key=KE7YGPTF1MOR tag=2VCN18L8ICCF type=write size=1041252 2024-04-22T19:58:42.321 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.321+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=74LA4W5F9SBC type=write size=592547 2024-04-22T19:58:42.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.322+0000 7f5269a8e780 1 < completed write operation key=A853MOZPT9BG tag=8IIOYN1LE9I6 type=write size=463092 2024-04-22T19:58:42.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.324+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~E0Q1P56DOOCI.0 size=1048576 2024-04-22T19:58:42.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.325+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~E0Q1P56DOOCI.1 size=1048576 2024-04-22T19:58:42.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.326+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~E0Q1P56DOOCI.2 size=256566 2024-04-22T19:58:42.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.326+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=E0Q1P56DOOCI tag=JO5805GSO3ID type=write size=3402294 2024-04-22T19:58:42.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.327+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~LI5MAVLBZHSE.0 size=141969 2024-04-22T19:58:42.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.329+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=LI5MAVLBZHSE tag=DFQ3NP24TYG6 type=write size=1190545 2024-04-22T19:58:42.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.331+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~7UH4P01FEMBP.0 size=563948 2024-04-22T19:58:42.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.331+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=7UH4P01FEMBP tag=UWWZ354Z7E50 type=write size=1612524 2024-04-22T19:58:42.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.333+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~WGB1WZY0BKIT.0 size=1048576 2024-04-22T19:58:42.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.334+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~WGB1WZY0BKIT.1 size=1048576 2024-04-22T19:58:42.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.335+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~WGB1WZY0BKIT.2 size=992133 2024-04-22T19:58:42.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.337+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=WGB1WZY0BKIT tag=1TI688IDQ56F type=write size=4137861 2024-04-22T19:58:42.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.338+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=73G0EKY7NRGC type=del size=2418927 2024-04-22T19:58:42.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.339+0000 7f5269a8e780 1 < completed write operation key=VPAVX5FXYWZO tag=SAEZVJY3FYC6 type=write size=405046 2024-04-22T19:58:42.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.340+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=VEG8F2OIRFPZ type=del size=2720392 2024-04-22T19:58:42.341 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.341+0000 7f5269a8e780 1 < completed write operation key=B0Y7ZTPC0975 tag=VINYPATVIX7Y type=write size=307677 2024-04-22T19:58:42.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.343+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=MPFN8M0CZPVK type=del size=1668518 2024-04-22T19:58:42.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.344+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~O78A3CZ4B31F.0 size=1048576 2024-04-22T19:58:42.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.345+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~O78A3CZ4B31F.1 size=902360 2024-04-22T19:58:42.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.347+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=O78A3CZ4B31F tag=S7BM480HKRGP type=write size=2999512 2024-04-22T19:58:42.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.349+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=0R8AXVSWR0L2 type=del size=4138235 2024-04-22T19:58:42.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.351+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=EK7020TH018R type=del 2024-04-22T19:58:42.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.353+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=PL81AESGJ82Y type=del size=3701951 2024-04-22T19:58:42.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.355+0000 7f5269a8e780 1 < suggested update operation key=2YFD2U6HVNE7 tag=EY5648RGGRBN type=del size=3878660 2024-04-22T19:58:42.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.357+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~50NBJCB79UJT.0 size=1048576 2024-04-22T19:58:42.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.358+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~50NBJCB79UJT.1 size=1048576 2024-04-22T19:58:42.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.359+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~50NBJCB79UJT.2 size=640909 2024-04-22T19:58:42.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.359+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=50NBJCB79UJT tag=OI97EXJD0GSR type=write size=3786637 2024-04-22T19:58:42.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.360+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~EUV01MRKN1EC.0 size=1048576 2024-04-22T19:58:42.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.361+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~EUV01MRKN1EC.1 size=729726 2024-04-22T19:58:42.363 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.363+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=EUV01MRKN1EC tag=9KP7NT32IOIN type=write size=2826878 2024-04-22T19:58:42.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.364+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=QORDUDKOXYZ3 type=del size=1635273 2024-04-22T19:58:42.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.365+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~L99NRGU12AHI.0 size=1048576 2024-04-22T19:58:42.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.366+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~L99NRGU12AHI.1 size=1048576 2024-04-22T19:58:42.367 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.367+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~L99NRGU12AHI.2 size=617771 2024-04-22T19:58:42.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.369+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=L99NRGU12AHI tag=CYKRRPRK8BYA type=write size=3763499 2024-04-22T19:58:42.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.371+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~G6CGU18VV0BA.0 size=1048576 2024-04-22T19:58:42.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.372+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~G6CGU18VV0BA.1 size=277011 2024-04-22T19:58:42.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.372+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=G6CGU18VV0BA tag=TC1Y5N9FI69I type=write size=2374163 2024-04-22T19:58:42.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.374+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~0A2MB9ESTCN2.0 size=675501 2024-04-22T19:58:42.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.376+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=0A2MB9ESTCN2 tag=CYPXL0HDB2EW type=write size=1724077 2024-04-22T19:58:42.377 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.377+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=IT92EOS80U40 type=del size=925881 2024-04-22T19:58:42.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.378+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~86H60GJESCQN.0 size=1048576 2024-04-22T19:58:42.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.379+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~86H60GJESCQN.1 size=30249 2024-04-22T19:58:42.381 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.381+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=86H60GJESCQN tag=WULPDJQ8CS6M type=write size=2127401 2024-04-22T19:58:42.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.382+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=J95KH5GYW0MO type=write size=736105 2024-04-22T19:58:42.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.383+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~S7U4LFCD8Z8F.0 size=394246 2024-04-22T19:58:42.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.385+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=S7U4LFCD8Z8F tag=MELKOHWRQ4I6 type=write size=1442822 2024-04-22T19:58:42.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.386+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=L8A9G1H6R7NL type=del size=2436556 2024-04-22T19:58:42.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.387+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=E11SDF1ACQ5G type=del 2024-04-22T19:58:42.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.390+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~78XDUP6RFD2K.0 size=1048576 2024-04-22T19:58:42.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.390+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~78XDUP6RFD2K.1 size=1048576 2024-04-22T19:58:42.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.391+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~78XDUP6RFD2K.2 size=102199 2024-04-22T19:58:42.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.391+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=78XDUP6RFD2K tag=IVF77TVDRJL5 type=write size=3247927 2024-04-22T19:58:42.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.392+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=BOT5E6KEY7ZT type=del 2024-04-22T19:58:42.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.394+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=O53HAWU1ETM6 type=del size=2283089 2024-04-22T19:58:42.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.395+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=74LA4W5F9SBC type=write size=592547 2024-04-22T19:58:42.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.396+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=12SKR06VHCDH type=del size=1900455 2024-04-22T19:58:42.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.397+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~E0Q1P56DOOCI.0 size=1048576 2024-04-22T19:58:42.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.398+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~E0Q1P56DOOCI.1 size=1048576 2024-04-22T19:58:42.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.399+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~E0Q1P56DOOCI.2 size=256566 2024-04-22T19:58:42.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.401+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=E0Q1P56DOOCI tag=JO5805GSO3ID type=write size=3402294 2024-04-22T19:58:42.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.402+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=6XTV64CHSIB3 type=del size=328176 2024-04-22T19:58:42.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.403+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~7UH4P01FEMBP.0 size=563948 2024-04-22T19:58:42.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.405+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=7UH4P01FEMBP tag=UWWZ354Z7E50 type=write size=1612524 2024-04-22T19:58:42.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.406+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=95Y5CE681R9O type=del size=3739635 2024-04-22T19:58:42.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.408+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=73G0EKY7NRGC type=del 2024-04-22T19:58:42.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.409+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~4RKGLU4N530Z.0 size=1048576 2024-04-22T19:58:42.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.410+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~4RKGLU4N530Z.1 size=1048576 2024-04-22T19:58:42.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.411+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~4RKGLU4N530Z.2 size=82360 2024-04-22T19:58:42.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.411+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=4RKGLU4N530Z tag=LA65SQEP3YEI type=write size=3228088 2024-04-22T19:58:42.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.412+0000 7f5269a8e780 1 < canceled operation key=EHRS6DQDRXVS tag=VEG8F2OIRFPZ type=del size=2720392 2024-04-22T19:58:42.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.415+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~4SLM5YDWHJEB.0 size=1048576 2024-04-22T19:58:42.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.415+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~4SLM5YDWHJEB.1 size=539833 2024-04-22T19:58:42.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.415+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=4SLM5YDWHJEB tag=5LU2EY8ZZBV9 type=write size=2636985 2024-04-22T19:58:42.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.417+0000 7f5269a8e780 1 < canceled operation key=GX6NILQG878S tag=MPFN8M0CZPVK type=del size=1668518 2024-04-22T19:58:42.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.419+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~7IIC2101WMAC.0 size=1048576 2024-04-22T19:58:42.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.422+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~7IIC2101WMAC.1 size=578416 2024-04-22T19:58:42.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.422+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=7IIC2101WMAC tag=AXCNMTTJOC1G type=write size=2675568 2024-04-22T19:58:42.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.423+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=0R8AXVSWR0L2 type=del 2024-04-22T19:58:42.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.424+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=V9HEQZRDP8A9 type=del size=1787079 2024-04-22T19:58:42.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.426+0000 7f5269a8e780 1 < canceled operation key=HIAXPVKZ1IIO tag=PL81AESGJ82Y type=del size=3701951 2024-04-22T19:58:42.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.427+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=BE9L3NL640GT type=del size=1155414 2024-04-22T19:58:42.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.428+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~50NBJCB79UJT.0 size=1048576 2024-04-22T19:58:42.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.429+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~50NBJCB79UJT.1 size=1048576 2024-04-22T19:58:42.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.430+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~50NBJCB79UJT.2 size=640909 2024-04-22T19:58:42.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.432+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=50NBJCB79UJT tag=OI97EXJD0GSR type=write size=3786637 2024-04-22T19:58:42.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.433+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=59ZSCNOJUQ0H type=del size=3071800 2024-04-22T19:58:42.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.434+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=QORDUDKOXYZ3 type=del 2024-04-22T19:58:42.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.435+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=698N13MWDYHF type=write size=88086 2024-04-22T19:58:42.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.436+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~G6CGU18VV0BA.0 size=1048576 2024-04-22T19:58:42.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.437+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~G6CGU18VV0BA.1 size=277011 2024-04-22T19:58:42.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.439+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=G6CGU18VV0BA tag=TC1Y5N9FI69I type=write size=2374163 2024-04-22T19:58:42.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.440+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=FSY1RDGKUJ4J type=del size=1308977 2024-04-22T19:58:42.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.441+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=IT92EOS80U40 type=del 2024-04-22T19:58:42.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.443+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~TC5YISPS08NS.0 size=1048576 2024-04-22T19:58:42.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.444+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~TC5YISPS08NS.1 size=124190 2024-04-22T19:58:42.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.444+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=TC5YISPS08NS tag=22RRN04RP1GI type=write size=2221342 2024-04-22T19:58:42.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.445+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=J95KH5GYW0MO type=write size=736105 2024-04-22T19:58:42.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.448+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~NYX7VQDV5KLQ.0 size=462569 2024-04-22T19:58:42.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.448+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=NYX7VQDV5KLQ tag=DG5CUXONZIF6 type=write size=1511145 2024-04-22T19:58:42.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.449+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=L8A9G1H6R7NL type=del 2024-04-22T19:58:42.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.450+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=9B9AOTSERJGT type=del size=201428 2024-04-22T19:58:42.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.451+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~78XDUP6RFD2K.0 size=1048576 2024-04-22T19:58:42.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.452+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~78XDUP6RFD2K.1 size=1048576 2024-04-22T19:58:42.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.453+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~78XDUP6RFD2K.2 size=102199 2024-04-22T19:58:42.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.455+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=78XDUP6RFD2K tag=IVF77TVDRJL5 type=write size=3247927 2024-04-22T19:58:42.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.457+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=YZCAKWV3XGO4 type=del size=1499843 2024-04-22T19:58:42.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.460+0000 7f5269a8e780 1 < suggested remove operation key=I91AD8FJ3P4T tag=O53HAWU1ETM6 type=del 2024-04-22T19:58:42.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.462+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~X1YZNQNOR65M.0 size=666979 2024-04-22T19:58:42.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.462+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=X1YZNQNOR65M tag=18G97FZL65PM type=write size=1715555 2024-04-22T19:58:42.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.463+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=12SKR06VHCDH type=del 2024-04-22T19:58:42.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.464+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=VH9IVZF13LFE type=write size=513849 2024-04-22T19:58:42.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.466+0000 7f5269a8e780 1 < canceled operation key=RY1O7X1CWZMK tag=6XTV64CHSIB3 type=del size=328176 2024-04-22T19:58:42.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.468+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~KXC065YCDR5W.0 size=1048576 2024-04-22T19:58:42.468 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.468+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~KXC065YCDR5W.1 size=828907 2024-04-22T19:58:42.468 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.468+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=KXC065YCDR5W tag=04BY6PH21Q79 type=write size=2926059 2024-04-22T19:58:42.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.471+0000 7f5269a8e780 1 < suggested update operation key=HV8J6YM0YNC1 tag=95Y5CE681R9O type=del size=3739635 2024-04-22T19:58:42.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.472+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=QUTR2GHERW2B type=del size=785298 2024-04-22T19:58:42.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.473+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~4RKGLU4N530Z.0 size=1048576 2024-04-22T19:58:42.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.473+0000 7f5269a8e780 1 < canceled part key=_multipart_4GX83ZJWZCD1.2~4RKGLU4N530Z.1 size=1048576 2024-04-22T19:58:42.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.474+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~4RKGLU4N530Z.2 size=82360 2024-04-22T19:58:42.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.476+0000 7f5269a8e780 1 < canceled multipart upload key=4GX83ZJWZCD1 upload=4RKGLU4N530Z tag=LA65SQEP3YEI type=write size=3228088 2024-04-22T19:58:42.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.478+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=Y1FA4Z30AKIE type=write size=426252 2024-04-22T19:58:42.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.479+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~4SLM5YDWHJEB.0 size=1048576 2024-04-22T19:58:42.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.479+0000 7f5269a8e780 1 < canceled part key=_multipart_IBH6W1NINUHC.2~4SLM5YDWHJEB.1 size=539833 2024-04-22T19:58:42.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.481+0000 7f5269a8e780 1 < canceled multipart upload key=IBH6W1NINUHC upload=4SLM5YDWHJEB tag=5LU2EY8ZZBV9 type=write size=2636985 2024-04-22T19:58:42.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.482+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=EXVV7Q685WK4 type=del size=2300451 2024-04-22T19:58:42.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.483+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~7IIC2101WMAC.0 size=1048576 2024-04-22T19:58:42.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.484+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~7IIC2101WMAC.1 size=578416 2024-04-22T19:58:42.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.486+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=7IIC2101WMAC tag=AXCNMTTJOC1G type=write size=2675568 2024-04-22T19:58:42.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.489+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~Y080LWUXNWXG.0 size=1048576 2024-04-22T19:58:42.489 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.489+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~Y080LWUXNWXG.1 size=266601 2024-04-22T19:58:42.489 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.489+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=Y080LWUXNWXG tag=OJZ5MGYIPW71 type=write size=2363753 2024-04-22T19:58:42.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.492+0000 7f5269a8e780 1 < suggested remove operation key=AMBJHMJGLUUS tag=V9HEQZRDP8A9 type=del 2024-04-22T19:58:42.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.493+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=R5BT21FDOXU9 type=del size=604891 2024-04-22T19:58:42.495 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.495+0000 7f5269a8e780 1 < suggested update operation key=EHRS6DQDRXVS tag=BE9L3NL640GT type=del size=1155414 2024-04-22T19:58:42.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.498+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~5WSJTWJIUED9.0 size=1048576 2024-04-22T19:58:42.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.499+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~5WSJTWJIUED9.1 size=1048576 2024-04-22T19:58:42.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.500+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~5WSJTWJIUED9.2 size=393739 2024-04-22T19:58:42.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.500+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=5WSJTWJIUED9 tag=WCP7BP1W9IRJ type=write size=3539467 2024-04-22T19:58:42.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.502+0000 7f5269a8e780 1 < suggested remove operation key=637M7AP8X9E9 tag=59ZSCNOJUQ0H type=del 2024-04-22T19:58:42.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.503+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=NTN3W7FH5S6J type=del size=1288984 2024-04-22T19:58:42.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.504+0000 7f5269a8e780 1 < canceled operation key=M9M4SEEEZSFV tag=698N13MWDYHF type=write size=88086 2024-04-22T19:58:42.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.505+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=CI2OPYCO38UA type=del size=593064 2024-04-22T19:58:42.506 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.506+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=FSY1RDGKUJ4J type=del 2024-04-22T19:58:42.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.509+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~EHFDM6R719HT.0 size=1048576 2024-04-22T19:58:42.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.509+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~EHFDM6R719HT.1 size=315657 2024-04-22T19:58:42.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.509+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=EHFDM6R719HT tag=215TP4CSH54T type=write size=2412809 2024-04-22T19:58:42.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.511+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~TC5YISPS08NS.0 size=1048576 2024-04-22T19:58:42.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.512+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~TC5YISPS08NS.1 size=124190 2024-04-22T19:58:42.513 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.513+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=TC5YISPS08NS tag=22RRN04RP1GI type=write size=2221342 2024-04-22T19:58:42.514 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.515+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=7SUOA6OD2DWK type=del size=2844268 2024-04-22T19:58:42.515 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.515+0000 7f5269a8e780 1 < canceled part key=_multipart_GP1C5ULUU143.2~NYX7VQDV5KLQ.0 size=462569 2024-04-22T19:58:42.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.517+0000 7f5269a8e780 1 < canceled multipart upload key=GP1C5ULUU143 upload=NYX7VQDV5KLQ tag=DG5CUXONZIF6 type=write size=1511145 2024-04-22T19:58:42.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.518+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=YOW3YW8VUSHR type=del size=1288813 2024-04-22T19:58:42.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.519+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=9B9AOTSERJGT type=del 2024-04-22T19:58:42.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.520+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=AXSGWL9Z4TZ7 type=del size=1828865 2024-04-22T19:58:42.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.522+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=YZCAKWV3XGO4 type=del size=1499843 2024-04-22T19:58:42.523 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.524+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=PQ6BWN6PDXE8 type=del size=3284080 2024-04-22T19:58:42.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.525+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~X1YZNQNOR65M.0 size=666979 2024-04-22T19:58:42.527 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.527+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=X1YZNQNOR65M tag=18G97FZL65PM type=write size=1715555 2024-04-22T19:58:42.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.528+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=OZESO159SQ0I type=del size=3599596 2024-04-22T19:58:42.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.529+0000 7f5269a8e780 1 < completed write operation key=A853MOZPT9BG tag=VH9IVZF13LFE type=write size=513849 2024-04-22T19:58:42.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.531+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=FV25869X58Y9 type=write size=210605 2024-04-22T19:58:42.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.532+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~KXC065YCDR5W.0 size=1048576 2024-04-22T19:58:42.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.532+0000 7f5269a8e780 1 < canceled part key=_multipart_I91AD8FJ3P4T.2~KXC065YCDR5W.1 size=828907 2024-04-22T19:58:42.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.534+0000 7f5269a8e780 1 < canceled multipart upload key=I91AD8FJ3P4T upload=KXC065YCDR5W tag=04BY6PH21Q79 type=write size=2926059 2024-04-22T19:58:42.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.535+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=RCG0ILEPWH51 type=del size=3285531 2024-04-22T19:58:42.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.536+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=QUTR2GHERW2B type=del 2024-04-22T19:58:42.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.538+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~RT54WD6OOCYR.0 size=673950 2024-04-22T19:58:42.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.538+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=RT54WD6OOCYR tag=PDI5UDU5CSE3 type=write size=1722526 2024-04-22T19:58:42.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.539+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=Y1FA4Z30AKIE type=write size=426252 2024-04-22T19:58:42.540 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.540+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=241EF1CW7MWF type=del size=911027 2024-04-22T19:58:42.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.542+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=EXVV7Q685WK4 type=del 2024-04-22T19:58:42.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.543+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=EE5KJ6Q2ZNYW type=del size=4039035 2024-04-22T19:58:42.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.544+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~Y080LWUXNWXG.0 size=1048576 2024-04-22T19:58:42.545 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.545+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~Y080LWUXNWXG.1 size=266601 2024-04-22T19:58:42.547 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.547+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=Y080LWUXNWXG tag=OJZ5MGYIPW71 type=write size=2363753 2024-04-22T19:58:42.548 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.548+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=7OQ6UVNOU8JS type=del size=615005 2024-04-22T19:58:42.550 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.550+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=R5BT21FDOXU9 type=del 2024-04-22T19:58:42.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.552+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~046UF5RPGQLN.0 size=1019266 2024-04-22T19:58:42.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.552+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=046UF5RPGQLN tag=QSV4VVJZLSU7 type=write size=2067842 2024-04-22T19:58:42.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.553+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~5WSJTWJIUED9.0 size=1048576 2024-04-22T19:58:42.554 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.554+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~5WSJTWJIUED9.1 size=1048576 2024-04-22T19:58:42.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.555+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~5WSJTWJIUED9.2 size=393739 2024-04-22T19:58:42.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.557+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=5WSJTWJIUED9 tag=WCP7BP1W9IRJ type=write size=3539467 2024-04-22T19:58:42.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.559+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~TF7GP0A9X6PB.0 size=1048576 2024-04-22T19:58:42.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.560+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~TF7GP0A9X6PB.1 size=19582 2024-04-22T19:58:42.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.560+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=TF7GP0A9X6PB tag=VBOE2ZACU1XF type=write size=2116734 2024-04-22T19:58:42.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.561+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=NTN3W7FH5S6J type=del 2024-04-22T19:58:42.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.562+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=IDO34YTMF7DR type=del size=3266066 2024-04-22T19:58:42.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.563+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=CI2OPYCO38UA type=del 2024-04-22T19:58:42.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.565+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~PGKDFXG68YVK.0 size=1048576 2024-04-22T19:58:42.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.566+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~PGKDFXG68YVK.1 size=86316 2024-04-22T19:58:42.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.566+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=PGKDFXG68YVK tag=JCOEH0MV0729 type=write size=2183468 2024-04-22T19:58:42.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.567+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~EHFDM6R719HT.0 size=1048576 2024-04-22T19:58:42.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.567+0000 7f5269a8e780 1 < canceled part key=_multipart_Y5HR8XW4L0SP.2~EHFDM6R719HT.1 size=315657 2024-04-22T19:58:42.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.570+0000 7f5269a8e780 1 < canceled multipart upload key=Y5HR8XW4L0SP upload=EHFDM6R719HT tag=215TP4CSH54T type=write size=2412809 2024-04-22T19:58:42.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.572+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~PL2NLYT7OIE0.0 size=1048576 2024-04-22T19:58:42.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.573+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~PL2NLYT7OIE0.1 size=498583 2024-04-22T19:58:42.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.573+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=PL2NLYT7OIE0 tag=LLYQI66K2OT7 type=write size=2595735 2024-04-22T19:58:42.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.574+0000 7f5269a8e780 1 < canceled operation key=637M7AP8X9E9 tag=7SUOA6OD2DWK type=del size=2844268 2024-04-22T19:58:42.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.575+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=F6RK74EDKBXA type=del size=3072601 2024-04-22T19:58:42.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.577+0000 7f5269a8e780 1 < suggested update operation key=KE7YGPTF1MOR tag=YOW3YW8VUSHR type=del size=1288813 2024-04-22T19:58:42.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.578+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=120HMJTON9CI type=del size=449356 2024-04-22T19:58:42.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.579+0000 7f5269a8e780 1 < canceled operation key=2YFD2U6HVNE7 tag=AXSGWL9Z4TZ7 type=del size=1828865 2024-04-22T19:58:42.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.580+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=6A7VQMFFT4BV type=del size=225409 2024-04-22T19:58:42.582 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.582+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=PQ6BWN6PDXE8 type=del 2024-04-22T19:58:42.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.584+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~LL2W0IWS6Q77.0 size=1048576 2024-04-22T19:58:42.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.585+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~LL2W0IWS6Q77.1 size=1048576 2024-04-22T19:58:42.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.586+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~LL2W0IWS6Q77.2 size=255257 2024-04-22T19:58:42.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.586+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=LL2W0IWS6Q77 tag=P19L0ZZIFZ4G type=write size=3400985 2024-04-22T19:58:42.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.587+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=OZESO159SQ0I type=del 2024-04-22T19:58:42.588 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.589+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=CZNRKKCJZHVF type=del size=1118898 2024-04-22T19:58:42.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.590+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=FV25869X58Y9 type=write size=210605 2024-04-22T19:58:42.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.591+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=IG572QN0L9CY type=del size=357755 2024-04-22T19:58:42.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.592+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=RCG0ILEPWH51 type=del 2024-04-22T19:58:42.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.594+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=ICLEQCYBK5X0 type=write size=606631 2024-04-22T19:58:42.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.594+0000 7f5269a8e780 1 < canceled part key=_multipart_GX6NILQG878S.2~RT54WD6OOCYR.0 size=673950 2024-04-22T19:58:42.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.596+0000 7f5269a8e780 1 < canceled multipart upload key=GX6NILQG878S upload=RT54WD6OOCYR tag=PDI5UDU5CSE3 type=write size=1722526 2024-04-22T19:58:42.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.597+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=XKKPXK3UW0XZ type=del size=3465865 2024-04-22T19:58:42.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.599+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=241EF1CW7MWF type=del 2024-04-22T19:58:42.600 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.600+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=FAZFN0N00PNI type=del size=2628755 2024-04-22T19:58:42.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.602+0000 7f5269a8e780 1 < suggested update operation key=4GX83ZJWZCD1 tag=EE5KJ6Q2ZNYW type=del size=4039035 2024-04-22T19:58:42.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.603+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=DPJFGD7N2ZNF type=del size=957650 2024-04-22T19:58:42.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.605+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=7OQ6UVNOU8JS type=del 2024-04-22T19:58:42.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.607+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~7DXFMNW7EXTD.0 size=933409 2024-04-22T19:58:42.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.607+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=7DXFMNW7EXTD tag=947EHB3AYMEG type=write size=1981985 2024-04-22T19:58:42.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.608+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~046UF5RPGQLN.0 size=1019266 2024-04-22T19:58:42.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.610+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=046UF5RPGQLN tag=QSV4VVJZLSU7 type=write size=2067842 2024-04-22T19:58:42.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.612+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~L4WUBEZ5D04J.0 size=1048576 2024-04-22T19:58:42.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.613+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~L4WUBEZ5D04J.1 size=1048576 2024-04-22T19:58:42.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.614+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~L4WUBEZ5D04J.2 size=604389 2024-04-22T19:58:42.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.614+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=L4WUBEZ5D04J tag=S50O4K5E03Z3 type=write size=3750117 2024-04-22T19:58:42.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.615+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~TF7GP0A9X6PB.0 size=1048576 2024-04-22T19:58:42.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.616+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~TF7GP0A9X6PB.1 size=19582 2024-04-22T19:58:42.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.618+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=TF7GP0A9X6PB tag=VBOE2ZACU1XF type=write size=2116734 2024-04-22T19:58:42.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.620+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~6FYJ1PEP933Y.0 size=163217 2024-04-22T19:58:42.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.620+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=6FYJ1PEP933Y tag=TE4WLKNGWZX5 type=write size=1211793 2024-04-22T19:58:42.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.622+0000 7f5269a8e780 1 < suggested remove operation key=HIAXPVKZ1IIO tag=IDO34YTMF7DR type=del 2024-04-22T19:58:42.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.624+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~RNTNIKWQZFO4.0 size=1048576 2024-04-22T19:58:42.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.625+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~RNTNIKWQZFO4.1 size=1048576 2024-04-22T19:58:42.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.625+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~RNTNIKWQZFO4.2 size=348213 2024-04-22T19:58:42.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.625+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=RNTNIKWQZFO4 tag=DWYE4CWIJPXC type=write size=3493941 2024-04-22T19:58:42.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.627+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~PGKDFXG68YVK.0 size=1048576 2024-04-22T19:58:42.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.627+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~PGKDFXG68YVK.1 size=86316 2024-04-22T19:58:42.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.629+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=PGKDFXG68YVK tag=JCOEH0MV0729 type=write size=2183468 2024-04-22T19:58:42.630 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.630+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=2MO8Q15UZN5P type=write size=706492 2024-04-22T19:58:42.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.632+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~PL2NLYT7OIE0.0 size=1048576 2024-04-22T19:58:42.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.632+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~PL2NLYT7OIE0.1 size=498583 2024-04-22T19:58:42.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.634+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=PL2NLYT7OIE0 tag=LLYQI66K2OT7 type=write size=2595735 2024-04-22T19:58:42.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.636+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~YHSECSUMZIQZ.0 size=104616 2024-04-22T19:58:42.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.636+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=YHSECSUMZIQZ tag=0A1R7WC2TMDK type=write size=1153192 2024-04-22T19:58:42.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.637+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=F6RK74EDKBXA type=del 2024-04-22T19:58:42.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.639+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=35VMCTJAJPFG type=del size=3562458 2024-04-22T19:58:42.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.641+0000 7f5269a8e780 1 < suggested remove operation key=B0Y7ZTPC0975 tag=120HMJTON9CI type=del 2024-04-22T19:58:42.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.643+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~X8DG522N0M07.0 size=303498 2024-04-22T19:58:42.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.643+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=X8DG522N0M07 tag=I59W3LHPP5VF type=write size=1352074 2024-04-22T19:58:42.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.644+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=6A7VQMFFT4BV type=del 2024-04-22T19:58:42.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.646+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~PTRZ0XSMVBSH.0 size=1048576 2024-04-22T19:58:42.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.647+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~PTRZ0XSMVBSH.1 size=17711 2024-04-22T19:58:42.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.647+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=PTRZ0XSMVBSH tag=XR0SHX6A76D6 type=write size=2114863 2024-04-22T19:58:42.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.648+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~LL2W0IWS6Q77.0 size=1048576 2024-04-22T19:58:42.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.649+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~LL2W0IWS6Q77.1 size=1048576 2024-04-22T19:58:42.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.650+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~LL2W0IWS6Q77.2 size=255257 2024-04-22T19:58:42.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.652+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=LL2W0IWS6Q77 tag=P19L0ZZIFZ4G type=write size=3400985 2024-04-22T19:58:42.653 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.654+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~7NVOTR5ZVYRD.0 size=1048576 2024-04-22T19:58:42.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.654+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~7NVOTR5ZVYRD.1 size=1048576 2024-04-22T19:58:42.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.655+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~7NVOTR5ZVYRD.2 size=336776 2024-04-22T19:58:42.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.655+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=7NVOTR5ZVYRD tag=2I7BYY0LLLKI type=write size=3482504 2024-04-22T19:58:42.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.656+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=CZNRKKCJZHVF type=del 2024-04-22T19:58:42.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.657+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=JY4COA5TGO21 type=del size=3016298 2024-04-22T19:58:42.658 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.659+0000 7f5269a8e780 -1 < failed to complete operation key=4GX83ZJWZCD1 tag=IG572QN0L9CY type=del size=357755: (22) Invalid argument 2024-04-22T19:58:42.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.661+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~IXXXV4L35QG6.0 size=416545 2024-04-22T19:58:42.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.661+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=IXXXV4L35QG6 tag=KP7XYQ7WRIL0 type=write size=1465121 2024-04-22T19:58:42.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.662+0000 7f5269a8e780 1 < completed write operation key=KE7YGPTF1MOR tag=ICLEQCYBK5X0 type=write size=606631 2024-04-22T19:58:42.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.663+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=DG3D4ZH2KZ71 type=del size=3653682 2024-04-22T19:58:42.664 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.664+0000 7f5269a8e780 -1 < failed to complete operation key=HIAXPVKZ1IIO tag=XKKPXK3UW0XZ type=del size=3465865: (22) Invalid argument 2024-04-22T19:58:42.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.666+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~BLQZKRXJOP1C.0 size=1048576 2024-04-22T19:58:42.667 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.667+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~BLQZKRXJOP1C.1 size=105240 2024-04-22T19:58:42.667 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.667+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=BLQZKRXJOP1C tag=Q9WELK6CWB6Z type=write size=2202392 2024-04-22T19:58:42.668 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.668+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=FAZFN0N00PNI type=del 2024-04-22T19:58:42.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.669+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=4B1FZHZR2DFE type=write size=517584 2024-04-22T19:58:42.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.670+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=DPJFGD7N2ZNF type=del 2024-04-22T19:58:42.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.672+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~DTMFFZH2MGCF.0 size=1048576 2024-04-22T19:58:42.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.673+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~DTMFFZH2MGCF.1 size=1048576 2024-04-22T19:58:42.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.674+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~DTMFFZH2MGCF.2 size=362190 2024-04-22T19:58:42.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.674+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=DTMFFZH2MGCF tag=ARE983VVBBZE type=write size=3507918 2024-04-22T19:58:42.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.676+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~7DXFMNW7EXTD.0 size=933409 2024-04-22T19:58:42.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.678+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=7DXFMNW7EXTD tag=947EHB3AYMEG type=write size=1981985 2024-04-22T19:58:42.679 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.679+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=F1EX6HDFIF30 type=del size=2219386 2024-04-22T19:58:42.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.680+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~L4WUBEZ5D04J.0 size=1048576 2024-04-22T19:58:42.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.681+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~L4WUBEZ5D04J.1 size=1048576 2024-04-22T19:58:42.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.682+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~L4WUBEZ5D04J.2 size=604389 2024-04-22T19:58:42.684 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.684+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=L4WUBEZ5D04J tag=S50O4K5E03Z3 type=write size=3750117 2024-04-22T19:58:42.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.686+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~28Y2OFDDJXUX.0 size=1048576 2024-04-22T19:58:42.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.686+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~28Y2OFDDJXUX.1 size=486578 2024-04-22T19:58:42.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.686+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=28Y2OFDDJXUX tag=JHR1JGGS1G2O type=write size=2583730 2024-04-22T19:58:42.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.687+0000 7f5269a8e780 1 < canceled part key=_multipart_T2BV2LUCY4E3.2~6FYJ1PEP933Y.0 size=163217 2024-04-22T19:58:42.689 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.689+0000 7f5269a8e780 1 < canceled multipart upload key=T2BV2LUCY4E3 upload=6FYJ1PEP933Y tag=TE4WLKNGWZX5 type=write size=1211793 2024-04-22T19:58:42.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.691+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~3QNW28H7J6NO.0 size=815775 2024-04-22T19:58:42.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.691+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=3QNW28H7J6NO tag=K4EUKOPJ74HV type=write size=1864351 2024-04-22T19:58:42.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.691+0000 7f5269a8e780 1 < canceled part key=_multipart_8HFSDSNZONIN.2~RNTNIKWQZFO4.0 size=1048576 2024-04-22T19:58:42.692 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.692+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~RNTNIKWQZFO4.1 size=1048576 2024-04-22T19:58:42.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.693+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~RNTNIKWQZFO4.2 size=348213 2024-04-22T19:58:42.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.695+0000 7f5269a8e780 1 < canceled multipart upload key=8HFSDSNZONIN upload=RNTNIKWQZFO4 tag=DWYE4CWIJPXC type=write size=3493941 2024-04-22T19:58:42.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.696+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=G97IR8KT4K9F type=write size=1004885 2024-04-22T19:58:42.698 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.698+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=2MO8Q15UZN5P type=write size=706492 2024-04-22T19:58:42.699 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.699+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=XG4N1211P87F type=del size=3498631 2024-04-22T19:58:42.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.700+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~YHSECSUMZIQZ.0 size=104616 2024-04-22T19:58:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.702+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=YHSECSUMZIQZ tag=0A1R7WC2TMDK type=write size=1153192 2024-04-22T19:58:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.704+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~L3CJ6WP6M42X.0 size=1048576 2024-04-22T19:58:42.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.705+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~L3CJ6WP6M42X.1 size=1048576 2024-04-22T19:58:42.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.706+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~L3CJ6WP6M42X.2 size=375994 2024-04-22T19:58:42.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.706+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=L3CJ6WP6M42X tag=L2N0Q5RXKCCK type=write size=3521722 2024-04-22T19:58:42.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.708+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=35VMCTJAJPFG type=del 2024-04-22T19:58:42.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.709+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=ITDLX53QKNE4 type=del size=2188171 2024-04-22T19:58:42.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.709+0000 7f5269a8e780 1 < canceled part key=_multipart_DX5YUYQRZKWA.2~X8DG522N0M07.0 size=303498 2024-04-22T19:58:42.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.711+0000 7f5269a8e780 1 < canceled multipart upload key=DX5YUYQRZKWA upload=X8DG522N0M07 tag=I59W3LHPP5VF type=write size=1352074 2024-04-22T19:58:42.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.712+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=4CVPTOAET47W type=del size=560955 2024-04-22T19:58:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.713+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~PTRZ0XSMVBSH.0 size=1048576 2024-04-22T19:58:42.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.714+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~PTRZ0XSMVBSH.1 size=17711 2024-04-22T19:58:42.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.716+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=PTRZ0XSMVBSH tag=XR0SHX6A76D6 type=write size=2114863 2024-04-22T19:58:42.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.717+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=MLE1P1BM57Y8 type=del size=3786428 2024-04-22T19:58:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.719+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~7NVOTR5ZVYRD.0 size=1048576 2024-04-22T19:58:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.719+0000 7f5269a8e780 1 < canceled part key=_multipart_DX5YUYQRZKWA.2~7NVOTR5ZVYRD.1 size=1048576 2024-04-22T19:58:42.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.719+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~7NVOTR5ZVYRD.2 size=336776 2024-04-22T19:58:42.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.721+0000 7f5269a8e780 1 < canceled multipart upload key=DX5YUYQRZKWA upload=7NVOTR5ZVYRD tag=2I7BYY0LLLKI type=write size=3482504 2024-04-22T19:58:42.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.723+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=CYJNSSYJURUI type=del size=682715 2024-04-22T19:58:42.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.724+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=JY4COA5TGO21 type=del 2024-04-22T19:58:42.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.726+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~VWOH9YOVJ0GZ.0 size=1048576 2024-04-22T19:58:42.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.727+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~VWOH9YOVJ0GZ.1 size=60335 2024-04-22T19:58:42.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.727+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=VWOH9YOVJ0GZ tag=9HTBAOLPUJA1 type=write size=2157487 2024-04-22T19:58:42.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.728+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~IXXXV4L35QG6.0 size=416545 2024-04-22T19:58:42.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.730+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=IXXXV4L35QG6 tag=KP7XYQ7WRIL0 type=write size=1465121 2024-04-22T19:58:42.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.731+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=K2YLD60AGBGT type=write size=532381 2024-04-22T19:58:42.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.733+0000 7f5269a8e780 1 < suggested update operation key=2YFD2U6HVNE7 tag=DG3D4ZH2KZ71 type=del size=3653682 2024-04-22T19:58:42.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.734+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=0WWEBQ9VH26J type=del size=3482476 2024-04-22T19:58:42.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.735+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~BLQZKRXJOP1C.0 size=1048576 2024-04-22T19:58:42.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.736+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~BLQZKRXJOP1C.1 size=105240 2024-04-22T19:58:42.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.738+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=BLQZKRXJOP1C tag=Q9WELK6CWB6Z type=write size=2202392 2024-04-22T19:58:42.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.740+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~1U7WODR9CHNH.0 size=1048576 2024-04-22T19:58:42.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.741+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~1U7WODR9CHNH.1 size=327957 2024-04-22T19:58:42.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.741+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=1U7WODR9CHNH tag=W2KP1XLQGXW7 type=write size=2425109 2024-04-22T19:58:42.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.742+0000 7f5269a8e780 1 < completed write operation key=M9M4SEEEZSFV tag=4B1FZHZR2DFE type=write size=517584 2024-04-22T19:58:42.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.743+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=DLZ6H81OBCV1 type=del size=3443956 2024-04-22T19:58:42.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.744+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~DTMFFZH2MGCF.0 size=1048576 2024-04-22T19:58:42.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.745+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~DTMFFZH2MGCF.1 size=1048576 2024-04-22T19:58:42.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.747+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~DTMFFZH2MGCF.2 size=362190 2024-04-22T19:58:42.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.749+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=DTMFFZH2MGCF tag=ARE983VVBBZE type=write size=3507918 2024-04-22T19:58:42.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.750+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=AW1YAFTS9VES type=del size=495850 2024-04-22T19:58:42.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.751+0000 7f5269a8e780 1 < canceled operation key=RY1O7X1CWZMK tag=F1EX6HDFIF30 type=del size=2219386 2024-04-22T19:58:42.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.753+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=CC6RG28FMZ71 type=del size=1939487 2024-04-22T19:58:42.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.754+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~28Y2OFDDJXUX.0 size=1048576 2024-04-22T19:58:42.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.754+0000 7f5269a8e780 1 < canceled part key=_multipart_IBH6W1NINUHC.2~28Y2OFDDJXUX.1 size=486578 2024-04-22T19:58:42.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.758+0000 7f5269a8e780 1 < canceled multipart upload key=IBH6W1NINUHC upload=28Y2OFDDJXUX tag=JHR1JGGS1G2O type=write size=2583730 2024-04-22T19:58:42.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.760+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~JSTLQSM0VJTU.0 size=638715 2024-04-22T19:58:42.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.760+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=JSTLQSM0VJTU tag=VO1XIGIWK248 type=write size=1687291 2024-04-22T19:58:42.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.761+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~3QNW28H7J6NO.0 size=815775 2024-04-22T19:58:42.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.763+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=3QNW28H7J6NO tag=K4EUKOPJ74HV type=write size=1864351 2024-04-22T19:58:42.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.765+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~6OKQRJLH2D2Z.0 size=989876 2024-04-22T19:58:42.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.765+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=6OKQRJLH2D2Z tag=3HWGYYFJ2OC0 type=write size=2038452 2024-04-22T19:58:42.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.766+0000 7f5269a8e780 1 < completed write operation key=4GX83ZJWZCD1 tag=G97IR8KT4K9F type=write size=1004885 2024-04-22T19:58:42.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.768+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~Y29RX0AOWDO2.0 size=1048576 2024-04-22T19:58:42.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.769+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~Y29RX0AOWDO2.1 size=572877 2024-04-22T19:58:42.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.769+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=Y29RX0AOWDO2 tag=EGLC9JAVKRAN type=write size=2670029 2024-04-22T19:58:42.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.771+0000 7f5269a8e780 1 < suggested update operation key=Y5HR8XW4L0SP tag=XG4N1211P87F type=del size=3498631 2024-04-22T19:58:42.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.772+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=T2ATWACYS7O9 type=del size=2634950 2024-04-22T19:58:42.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.774+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~L3CJ6WP6M42X.0 size=1048576 2024-04-22T19:58:42.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.775+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~L3CJ6WP6M42X.1 size=1048576 2024-04-22T19:58:42.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.776+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~L3CJ6WP6M42X.2 size=375994 2024-04-22T19:58:42.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.777+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=L3CJ6WP6M42X tag=L2N0Q5RXKCCK type=write size=3521722 2024-04-22T19:58:42.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.779+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~17HWG52F91U9.0 size=786342 2024-04-22T19:58:42.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.779+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=17HWG52F91U9 tag=2EMIY5M6HOBD type=write size=1834918 2024-04-22T19:58:42.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.781+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=ITDLX53QKNE4 type=del 2024-04-22T19:58:42.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.783+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~9I5UDCP1KDW1.0 size=1048576 2024-04-22T19:58:42.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.785+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~9I5UDCP1KDW1.1 size=1048576 2024-04-22T19:58:42.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.785+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~9I5UDCP1KDW1.2 size=45139 2024-04-22T19:58:42.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.786+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=9I5UDCP1KDW1 tag=KLW7E4PCYC7P type=write size=3190867 2024-04-22T19:58:42.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.789+0000 7f5269a8e780 1 < suggested update operation key=5UBYMQT7ZWKX tag=4CVPTOAET47W type=del size=560955 2024-04-22T19:58:42.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.790+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=GH7P4I3FQ0CH type=del size=4008482 2024-04-22T19:58:42.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.791+0000 7f5269a8e780 1 < canceled operation key=VPAVX5FXYWZO tag=MLE1P1BM57Y8 type=del size=3786428 2024-04-22T19:58:42.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.793+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=SKS9JZB2IP8Y type=write size=64287 2024-04-22T19:58:42.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.794+0000 7f5269a8e780 1 < canceled operation key=70MLQGJPQYPY tag=CYJNSSYJURUI type=del size=682715 2024-04-22T19:58:42.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.795+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=AACXT14O20XJ type=del size=1138581 2024-04-22T19:58:42.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.796+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~VWOH9YOVJ0GZ.0 size=1048576 2024-04-22T19:58:42.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.796+0000 7f5269a8e780 1 < canceled part key=_multipart_4EA3LUQRL8II.2~VWOH9YOVJ0GZ.1 size=60335 2024-04-22T19:58:42.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.799+0000 7f5269a8e780 1 < canceled multipart upload key=4EA3LUQRL8II upload=VWOH9YOVJ0GZ tag=9HTBAOLPUJA1 type=write size=2157487 2024-04-22T19:58:42.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.800+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=Q34AQAFIWHS4 type=del size=3113868 2024-04-22T19:58:42.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.802+0000 7f5269a8e780 1 < suggested remove operation key=KV5DYQ49ZKCX tag=K2YLD60AGBGT type=write 2024-04-22T19:58:42.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.803+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=7XVGLU0FAUMI type=write size=741428 2024-04-22T19:58:42.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.806+0000 7f5269a8e780 1 < suggested update operation key=GRSL48RUEBX8 tag=0WWEBQ9VH26J type=del size=3482476 2024-04-22T19:58:42.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.808+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~Z0WJ30KGQ6X2.0 size=648968 2024-04-22T19:58:42.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.808+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=Z0WJ30KGQ6X2 tag=OMZS309E2YPP type=write size=1697544 2024-04-22T19:58:42.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.809+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~1U7WODR9CHNH.0 size=1048576 2024-04-22T19:58:42.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.810+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~1U7WODR9CHNH.1 size=327957 2024-04-22T19:58:42.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.813+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=1U7WODR9CHNH tag=W2KP1XLQGXW7 type=write size=2425109 2024-04-22T19:58:42.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.815+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~1EKPPOBSREWA.0 size=1048576 2024-04-22T19:58:42.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.816+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~1EKPPOBSREWA.1 size=1048576 2024-04-22T19:58:42.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.816+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~1EKPPOBSREWA.2 size=478905 2024-04-22T19:58:42.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.816+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=1EKPPOBSREWA tag=SU3QOD7OI96O type=write size=3624633 2024-04-22T19:58:42.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.818+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=DLZ6H81OBCV1 type=del 2024-04-22T19:58:42.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.819+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=LF0MB5T59S01 type=del size=741091 2024-04-22T19:58:42.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.820+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=AW1YAFTS9VES type=del 2024-04-22T19:58:42.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.821+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=2GEVQJTEM79O type=write size=75731 2024-04-22T19:58:42.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.823+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=CC6RG28FMZ71 type=del 2024-04-22T19:58:42.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.824+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=UHTF9HMKVS5O type=del size=3545479 2024-04-22T19:58:42.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.825+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~JSTLQSM0VJTU.0 size=638715 2024-04-22T19:58:42.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.827+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=JSTLQSM0VJTU tag=VO1XIGIWK248 type=write size=1687291 2024-04-22T19:58:42.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.829+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~78IJ73C5P5H7.0 size=1048576 2024-04-22T19:58:42.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.830+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~78IJ73C5P5H7.1 size=282074 2024-04-22T19:58:42.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.830+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=78IJ73C5P5H7 tag=Y1NKJ0WIZWJT type=write size=2379226 2024-04-22T19:58:42.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.831+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~6OKQRJLH2D2Z.0 size=989876 2024-04-22T19:58:42.833 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.834+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=6OKQRJLH2D2Z tag=3HWGYYFJ2OC0 type=write size=2038452 2024-04-22T19:58:42.835 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.835+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=MEM0QXO78DZS type=write size=750686 2024-04-22T19:58:42.836 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.836+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~Y29RX0AOWDO2.0 size=1048576 2024-04-22T19:58:42.836 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.837+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~Y29RX0AOWDO2.1 size=572877 2024-04-22T19:58:42.838 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.839+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=Y29RX0AOWDO2 tag=EGLC9JAVKRAN type=write size=2670029 2024-04-22T19:58:42.840 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.841+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~PZAKHXG56E22.0 size=1048576 2024-04-22T19:58:42.841 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.841+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~PZAKHXG56E22.1 size=700945 2024-04-22T19:58:42.841 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.841+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=PZAKHXG56E22 tag=TDZGH10JWPQV type=write size=2798097 2024-04-22T19:58:42.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.843+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=T2ATWACYS7O9 type=del 2024-04-22T19:58:42.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.844+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=50R2KGM77WCM type=del size=3628953 2024-04-22T19:58:42.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.845+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~17HWG52F91U9.0 size=786342 2024-04-22T19:58:42.847 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.847+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=17HWG52F91U9 tag=2EMIY5M6HOBD type=write size=1834918 2024-04-22T19:58:42.848 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.848+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=MM1P64MYY6UI type=del size=1719349 2024-04-22T19:58:42.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.849+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~9I5UDCP1KDW1.0 size=1048576 2024-04-22T19:58:42.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.850+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~9I5UDCP1KDW1.1 size=1048576 2024-04-22T19:58:42.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.851+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~9I5UDCP1KDW1.2 size=45139 2024-04-22T19:58:42.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.853+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=9I5UDCP1KDW1 tag=KLW7E4PCYC7P type=write size=3190867 2024-04-22T19:58:42.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.855+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~8F0D5BMJGETJ.0 size=1048576 2024-04-22T19:58:42.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.856+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~8F0D5BMJGETJ.1 size=1048576 2024-04-22T19:58:42.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.857+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~8F0D5BMJGETJ.2 size=256033 2024-04-22T19:58:42.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.857+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=8F0D5BMJGETJ tag=YCXJSV4JOSJ2 type=write size=3401761 2024-04-22T19:58:42.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.858+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=GH7P4I3FQ0CH type=del 2024-04-22T19:58:42.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.859+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=B6I6MK4VK91E type=del size=529686 2024-04-22T19:58:42.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.860+0000 7f5269a8e780 1 < completed write operation key=DX5YUYQRZKWA tag=SKS9JZB2IP8Y type=write size=64287 2024-04-22T19:58:42.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.862+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~QMBRWQJD4NJ5.0 size=255985 2024-04-22T19:58:42.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.862+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=QMBRWQJD4NJ5 tag=OVEM533IH268 type=write size=1304561 2024-04-22T19:58:42.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.864+0000 7f5269a8e780 1 < suggested remove operation key=I91AD8FJ3P4T tag=AACXT14O20XJ type=del 2024-04-22T19:58:42.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.866+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~GG87MQ7RO9IJ.0 size=1048576 2024-04-22T19:58:42.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.867+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~GG87MQ7RO9IJ.1 size=849793 2024-04-22T19:58:42.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.867+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=GG87MQ7RO9IJ tag=ZXE6NEDK3WJ0 type=write size=2946945 2024-04-22T19:58:42.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.868+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=Q34AQAFIWHS4 type=del 2024-04-22T19:58:42.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.870+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~O43JZF3XMDXS.0 size=180720 2024-04-22T19:58:42.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.870+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=O43JZF3XMDXS tag=CCJT1GCUUS4V type=write size=1229296 2024-04-22T19:58:42.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.872+0000 7f5269a8e780 1 < suggested remove operation key=Y5HR8XW4L0SP tag=7XVGLU0FAUMI type=write 2024-04-22T19:58:42.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.873+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=KN0SYH3DBJ41 type=del size=2976588 2024-04-22T19:58:42.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.874+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~Z0WJ30KGQ6X2.0 size=648968 2024-04-22T19:58:42.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.876+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=Z0WJ30KGQ6X2 tag=OMZS309E2YPP type=write size=1697544 2024-04-22T19:58:42.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.878+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~CXL1NVUUAKOF.0 size=1048576 2024-04-22T19:58:42.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.879+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~CXL1NVUUAKOF.1 size=1048576 2024-04-22T19:58:42.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.879+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~CXL1NVUUAKOF.2 size=81802 2024-04-22T19:58:42.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.879+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=CXL1NVUUAKOF tag=4WWYB26626WP type=write size=3227530 2024-04-22T19:58:42.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.881+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~1EKPPOBSREWA.0 size=1048576 2024-04-22T19:58:42.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.881+0000 7f5269a8e780 1 < canceled part key=_multipart_HV8J6YM0YNC1.2~1EKPPOBSREWA.1 size=1048576 2024-04-22T19:58:42.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.881+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~1EKPPOBSREWA.2 size=478905 2024-04-22T19:58:42.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.883+0000 7f5269a8e780 1 < canceled multipart upload key=HV8J6YM0YNC1 upload=1EKPPOBSREWA tag=SU3QOD7OI96O type=write size=3624633 2024-04-22T19:58:42.885 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.885+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~L3MRB4LFS3DV.0 size=1048576 2024-04-22T19:58:42.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.886+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~L3MRB4LFS3DV.1 size=529833 2024-04-22T19:58:42.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.886+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=L3MRB4LFS3DV tag=CB559PB02DNZ type=write size=2626985 2024-04-22T19:58:42.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.887+0000 7f5269a8e780 -1 < failed to complete operation key=Y5HR8XW4L0SP tag=LF0MB5T59S01 type=del size=741091: (22) Invalid argument 2024-04-22T19:58:42.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.889+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~IWFRSWWT7P62.0 size=1048576 2024-04-22T19:58:42.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.890+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~IWFRSWWT7P62.1 size=977275 2024-04-22T19:58:42.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.890+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=IWFRSWWT7P62 tag=O6KPQTNFGJGB type=write size=3074427 2024-04-22T19:58:42.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.892+0000 7f5269a8e780 1 < completed write operation key=GRSL48RUEBX8 tag=2GEVQJTEM79O type=write size=75731 2024-04-22T19:58:42.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.893+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=ZVD3JC496C1Q type=del size=1559492 2024-04-22T19:58:42.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.894+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=UHTF9HMKVS5O type=del 2024-04-22T19:58:42.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.895+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=W6HCQQAILG6B type=del size=2377098 2024-04-22T19:58:42.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.897+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~78IJ73C5P5H7.0 size=1048576 2024-04-22T19:58:42.897 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.897+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~78IJ73C5P5H7.1 size=282074 2024-04-22T19:58:42.899 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.899+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=78IJ73C5P5H7 tag=Y1NKJ0WIZWJT type=write size=2379226 2024-04-22T19:58:42.900 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.900+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=H7123GTN43VF type=del size=2442296 2024-04-22T19:58:42.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.901+0000 7f5269a8e780 1 < completed write operation key=HIAXPVKZ1IIO tag=MEM0QXO78DZS type=write size=750686 2024-04-22T19:58:42.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.903+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~DYA8M4TASP8K.0 size=289523 2024-04-22T19:58:42.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.903+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=DYA8M4TASP8K tag=WIFULVGTHDZ8 type=write size=1338099 2024-04-22T19:58:42.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.905+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~PZAKHXG56E22.0 size=1048576 2024-04-22T19:58:42.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.905+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~PZAKHXG56E22.1 size=700945 2024-04-22T19:58:42.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.907+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=PZAKHXG56E22 tag=TDZGH10JWPQV type=write size=2798097 2024-04-22T19:58:42.909 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.909+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=TDE2WRXK697G type=del size=189501 2024-04-22T19:58:42.910 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.910+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=50R2KGM77WCM type=del 2024-04-22T19:58:42.912 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.913+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~OC4VBOBQ9K08.0 size=1048576 2024-04-22T19:58:42.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.913+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~OC4VBOBQ9K08.1 size=925365 2024-04-22T19:58:42.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.913+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=OC4VBOBQ9K08 tag=H1ON2JM22M7S type=write size=3022517 2024-04-22T19:58:42.915 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.915+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=MM1P64MYY6UI type=del 2024-04-22T19:58:42.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.916+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=5C20S3R24QF9 type=del size=1876320 2024-04-22T19:58:42.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.917+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~8F0D5BMJGETJ.0 size=1048576 2024-04-22T19:58:42.918 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.918+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~8F0D5BMJGETJ.1 size=1048576 2024-04-22T19:58:42.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.919+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~8F0D5BMJGETJ.2 size=256033 2024-04-22T19:58:42.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.921+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=8F0D5BMJGETJ tag=YCXJSV4JOSJ2 type=write size=3401761 2024-04-22T19:58:42.923 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.923+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~GOWDJ1IKZS5F.0 size=1048576 2024-04-22T19:58:42.924 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.924+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~GOWDJ1IKZS5F.1 size=883643 2024-04-22T19:58:42.924 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.924+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=GOWDJ1IKZS5F tag=72WAVNM5DTBH type=write size=2980795 2024-04-22T19:58:42.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.926+0000 7f5269a8e780 1 < canceled operation key=T2BV2LUCY4E3 tag=B6I6MK4VK91E type=del size=529686 2024-04-22T19:58:42.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.927+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=04JY8O2K0LIE type=del size=2094588 2024-04-22T19:58:42.928 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.928+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~QMBRWQJD4NJ5.0 size=255985 2024-04-22T19:58:42.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.930+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=QMBRWQJD4NJ5 tag=OVEM533IH268 type=write size=1304561 2024-04-22T19:58:42.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.932+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~9MUEC8DT138H.0 size=1048576 2024-04-22T19:58:42.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.933+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~9MUEC8DT138H.1 size=1048576 2024-04-22T19:58:42.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.934+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~9MUEC8DT138H.2 size=57452 2024-04-22T19:58:42.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.934+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=9MUEC8DT138H tag=GHNV8II9XZYL type=write size=3203180 2024-04-22T19:58:42.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.935+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~GG87MQ7RO9IJ.0 size=1048576 2024-04-22T19:58:42.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.936+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~GG87MQ7RO9IJ.1 size=849793 2024-04-22T19:58:42.937 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.938+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=GG87MQ7RO9IJ tag=ZXE6NEDK3WJ0 type=write size=2946945 2024-04-22T19:58:42.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.940+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~0ERL3MUTT2W3.0 size=1048576 2024-04-22T19:58:42.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.940+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~0ERL3MUTT2W3.1 size=1048576 2024-04-22T19:58:42.941 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.941+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~0ERL3MUTT2W3.2 size=902497 2024-04-22T19:58:42.941 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.941+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=0ERL3MUTT2W3 tag=0624I7Y1I9SQ type=write size=4048225 2024-04-22T19:58:42.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.942+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~O43JZF3XMDXS.0 size=180720 2024-04-22T19:58:42.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.944+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=O43JZF3XMDXS tag=CCJT1GCUUS4V type=write size=1229296 2024-04-22T19:58:42.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.945+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=1YW11OG2E5MN type=del size=638661 2024-04-22T19:58:42.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.948+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=KN0SYH3DBJ41 type=del size=2976588 2024-04-22T19:58:42.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.949+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=M5B0AGSECUZ8 type=del size=1330872 2024-04-22T19:58:42.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.950+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~CXL1NVUUAKOF.0 size=1048576 2024-04-22T19:58:42.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.951+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~CXL1NVUUAKOF.1 size=1048576 2024-04-22T19:58:42.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.952+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~CXL1NVUUAKOF.2 size=81802 2024-04-22T19:58:42.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.953+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=CXL1NVUUAKOF tag=4WWYB26626WP type=write size=3227530 2024-04-22T19:58:42.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.955+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~2L5K916VUCLI.0 size=1048576 2024-04-22T19:58:42.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.956+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~2L5K916VUCLI.1 size=618422 2024-04-22T19:58:42.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.956+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=2L5K916VUCLI tag=DIP5LUFKA9LE type=write size=2715574 2024-04-22T19:58:42.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.957+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~L3MRB4LFS3DV.0 size=1048576 2024-04-22T19:58:42.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.957+0000 7f5269a8e780 1 < canceled part key=_multipart_VPAVX5FXYWZO.2~L3MRB4LFS3DV.1 size=529833 2024-04-22T19:58:42.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.959+0000 7f5269a8e780 1 < canceled multipart upload key=VPAVX5FXYWZO upload=L3MRB4LFS3DV tag=CB559PB02DNZ type=write size=2626985 2024-04-22T19:58:42.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.961+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=GJAZKJU2X3RG type=write size=675627 2024-04-22T19:58:42.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.961+0000 7f5269a8e780 1 < canceled part key=_multipart_I91AD8FJ3P4T.2~IWFRSWWT7P62.0 size=1048576 2024-04-22T19:58:42.962 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.962+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~IWFRSWWT7P62.1 size=977275 2024-04-22T19:58:42.964 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.964+0000 7f5269a8e780 1 < canceled multipart upload key=I91AD8FJ3P4T upload=IWFRSWWT7P62 tag=O6KPQTNFGJGB type=write size=3074427 2024-04-22T19:58:42.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.966+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~F4W0H5QDMQ86.0 size=1048576 2024-04-22T19:58:42.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.967+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~F4W0H5QDMQ86.1 size=11431 2024-04-22T19:58:42.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.967+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=F4W0H5QDMQ86 tag=WNWM94XTN5LE type=write size=2108583 2024-04-22T19:58:42.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.968+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=ZVD3JC496C1Q type=del 2024-04-22T19:58:42.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.970+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~FFDXXF8WUNZO.0 size=1048576 2024-04-22T19:58:42.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.971+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~FFDXXF8WUNZO.1 size=1048576 2024-04-22T19:58:42.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.972+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~FFDXXF8WUNZO.2 size=959175 2024-04-22T19:58:42.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.972+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=FFDXXF8WUNZO tag=WCIIDLNDLJBY type=write size=4104903 2024-04-22T19:58:42.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.973+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=W6HCQQAILG6B type=del 2024-04-22T19:58:42.974 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.974+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=LQZ79WLU5QOH type=del size=1663318 2024-04-22T19:58:42.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.975+0000 7f5269a8e780 1 < canceled operation key=86GW6HV9G1IZ tag=H7123GTN43VF type=del size=2442296 2024-04-22T19:58:42.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.977+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=O5VKVPPNUQJL type=del size=3352337 2024-04-22T19:58:42.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.978+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~DYA8M4TASP8K.0 size=289523 2024-04-22T19:58:42.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.980+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=DYA8M4TASP8K tag=WIFULVGTHDZ8 type=write size=1338099 2024-04-22T19:58:42.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.981+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=0VT9S51Z9P7W type=del size=2734001 2024-04-22T19:58:42.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.982+0000 7f5269a8e780 1 < canceled operation key=4GX83ZJWZCD1 tag=TDE2WRXK697G type=del size=189501 2024-04-22T19:58:42.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.984+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~4RK6F1U2KPX9.0 size=1048576 2024-04-22T19:58:42.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.985+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~4RK6F1U2KPX9.1 size=1048576 2024-04-22T19:58:42.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.986+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~4RK6F1U2KPX9.2 size=245630 2024-04-22T19:58:42.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.986+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=4RK6F1U2KPX9 tag=QMCQY6Z0O83N type=write size=3391358 2024-04-22T19:58:42.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.987+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~OC4VBOBQ9K08.0 size=1048576 2024-04-22T19:58:42.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.987+0000 7f5269a8e780 1 < canceled part key=_multipart_86GW6HV9G1IZ.2~OC4VBOBQ9K08.1 size=925365 2024-04-22T19:58:42.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.989+0000 7f5269a8e780 1 < canceled multipart upload key=86GW6HV9G1IZ upload=OC4VBOBQ9K08 tag=H1ON2JM22M7S type=write size=3022517 2024-04-22T19:58:42.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.991+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~KCAOMX345NZ0.0 size=1048576 2024-04-22T19:58:42.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.992+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~KCAOMX345NZ0.1 size=1048576 2024-04-22T19:58:42.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.993+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~KCAOMX345NZ0.2 size=528426 2024-04-22T19:58:42.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.993+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=KCAOMX345NZ0 tag=KKLM3KOMZJBK type=write size=3674154 2024-04-22T19:58:42.994 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.994+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=5C20S3R24QF9 type=del 2024-04-22T19:58:42.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.995+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=LFOPZD9P4ZE1 type=del size=3497167 2024-04-22T19:58:42.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.996+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~GOWDJ1IKZS5F.0 size=1048576 2024-04-22T19:58:42.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.998+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~GOWDJ1IKZS5F.1 size=883643 2024-04-22T19:58:42.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:42.999+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=GOWDJ1IKZS5F tag=72WAVNM5DTBH type=write size=2980795 2024-04-22T19:58:43.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.001+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=MWX1090YL0T5 type=del size=1898245 2024-04-22T19:58:43.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.002+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=04JY8O2K0LIE type=del 2024-04-22T19:58:43.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.004+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~6WHO507ADKW1.0 size=1048576 2024-04-22T19:58:43.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.005+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~6WHO507ADKW1.1 size=1048576 2024-04-22T19:58:43.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.006+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~6WHO507ADKW1.2 size=546571 2024-04-22T19:58:43.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.006+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=6WHO507ADKW1 tag=ID3PZKI2S910 type=write size=3692299 2024-04-22T19:58:43.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.007+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~9MUEC8DT138H.0 size=1048576 2024-04-22T19:58:43.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.008+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~9MUEC8DT138H.1 size=1048576 2024-04-22T19:58:43.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.009+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~9MUEC8DT138H.2 size=57452 2024-04-22T19:58:43.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.011+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=9MUEC8DT138H tag=GHNV8II9XZYL type=write size=3203180 2024-04-22T19:58:43.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.013+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~TXWZ64P4PTIR.0 size=179811 2024-04-22T19:58:43.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.013+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=TXWZ64P4PTIR tag=TB3WPWJZP2YZ type=write size=1228387 2024-04-22T19:58:43.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.014+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~0ERL3MUTT2W3.0 size=1048576 2024-04-22T19:58:43.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.015+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~0ERL3MUTT2W3.1 size=1048576 2024-04-22T19:58:43.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.016+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~0ERL3MUTT2W3.2 size=902497 2024-04-22T19:58:43.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.018+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=0ERL3MUTT2W3 tag=0624I7Y1I9SQ type=write size=4048225 2024-04-22T19:58:43.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.020+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~10ZW7VHKUVUS.0 size=1048576 2024-04-22T19:58:43.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.021+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~10ZW7VHKUVUS.1 size=95678 2024-04-22T19:58:43.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.021+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=10ZW7VHKUVUS tag=K5HQ0B36NKFK type=write size=2192830 2024-04-22T19:58:43.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.022+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=1YW11OG2E5MN type=del 2024-04-22T19:58:43.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.025+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~DAE0QF4YBCUW.0 size=1048576 2024-04-22T19:58:43.025 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.025+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~DAE0QF4YBCUW.1 size=1048576 2024-04-22T19:58:43.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.026+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~DAE0QF4YBCUW.2 size=168651 2024-04-22T19:58:43.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.026+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=DAE0QF4YBCUW tag=QWN2W311Z2O5 type=write size=3314379 2024-04-22T19:58:43.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.029+0000 7f5269a8e780 1 < suggested update operation key=GP1C5ULUU143 tag=M5B0AGSECUZ8 type=del size=1330872 2024-04-22T19:58:43.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.031+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~DQEELLZYRT9U.0 size=174970 2024-04-22T19:58:43.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.031+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=DQEELLZYRT9U tag=ADLVFDBO318V type=write size=1223546 2024-04-22T19:58:43.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.032+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~2L5K916VUCLI.0 size=1048576 2024-04-22T19:58:43.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.033+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~2L5K916VUCLI.1 size=618422 2024-04-22T19:58:43.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.035+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=2L5K916VUCLI tag=DIP5LUFKA9LE type=write size=2715574 2024-04-22T19:58:43.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.036+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=R5HJ2C3CUE9R type=del size=2317745 2024-04-22T19:58:43.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.038+0000 7f5269a8e780 1 < suggested remove operation key=M9M4SEEEZSFV tag=GJAZKJU2X3RG type=write 2024-04-22T19:58:43.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.039+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=5NSWK8RC86OQ type=del size=2627054 2024-04-22T19:58:43.040 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.040+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~F4W0H5QDMQ86.0 size=1048576 2024-04-22T19:58:43.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.041+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~F4W0H5QDMQ86.1 size=11431 2024-04-22T19:58:43.043 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.043+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=F4W0H5QDMQ86 tag=WNWM94XTN5LE type=write size=2108583 2024-04-22T19:58:43.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.044+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=SWO8TBCFZTCD type=del size=1839325 2024-04-22T19:58:43.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.045+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~FFDXXF8WUNZO.0 size=1048576 2024-04-22T19:58:43.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.046+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~FFDXXF8WUNZO.1 size=1048576 2024-04-22T19:58:43.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.047+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~FFDXXF8WUNZO.2 size=959175 2024-04-22T19:58:43.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.048+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=FFDXXF8WUNZO tag=WCIIDLNDLJBY type=write size=4104903 2024-04-22T19:58:43.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.050+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~91FFTUE6523N.0 size=1048576 2024-04-22T19:58:43.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.051+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~91FFTUE6523N.1 size=1048576 2024-04-22T19:58:43.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.052+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~91FFTUE6523N.2 size=197513 2024-04-22T19:58:43.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.052+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=91FFTUE6523N tag=SM25EEJ4MCYI type=write size=3343241 2024-04-22T19:58:43.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.053+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=LQZ79WLU5QOH type=del 2024-04-22T19:58:43.054 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.054+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=Q6M6O49US7CN type=del size=1431869 2024-04-22T19:58:43.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.056+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=O5VKVPPNUQJL type=del 2024-04-22T19:58:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.057+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=WYDKCVYOTX8D type=del size=1934622 2024-04-22T19:58:43.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.058+0000 7f5269a8e780 -1 < failed to complete operation key=M9M4SEEEZSFV tag=0VT9S51Z9P7W type=del size=2734001: (22) Invalid argument 2024-04-22T19:58:43.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.059+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=ATC9ZC9W2QG2 type=write size=374918 2024-04-22T19:58:43.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.060+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~4RK6F1U2KPX9.0 size=1048576 2024-04-22T19:58:43.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.061+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~4RK6F1U2KPX9.1 size=1048576 2024-04-22T19:58:43.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.061+0000 7f5269a8e780 1 < canceled part key=_multipart_HIAXPVKZ1IIO.2~4RK6F1U2KPX9.2 size=245630 2024-04-22T19:58:43.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.063+0000 7f5269a8e780 1 < canceled multipart upload key=HIAXPVKZ1IIO upload=4RK6F1U2KPX9 tag=QMCQY6Z0O83N type=write size=3391358 2024-04-22T19:58:43.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.064+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=68F15YQF2Y2A type=write size=122853 2024-04-22T19:58:43.066 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.066+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~KCAOMX345NZ0.0 size=1048576 2024-04-22T19:58:43.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.067+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~KCAOMX345NZ0.1 size=1048576 2024-04-22T19:58:43.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.068+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~KCAOMX345NZ0.2 size=528426 2024-04-22T19:58:43.070 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.070+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=KCAOMX345NZ0 tag=KKLM3KOMZJBK type=write size=3674154 2024-04-22T19:58:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.071+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=AXCQ59ESFXMV type=del size=1731083 2024-04-22T19:58:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.073+0000 7f5269a8e780 1 < canceled operation key=HIAXPVKZ1IIO tag=LFOPZD9P4ZE1 type=del size=3497167 2024-04-22T19:58:43.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.074+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=N34NX1B45BP5 type=del size=5533 2024-04-22T19:58:43.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.076+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=MWX1090YL0T5 type=del size=1898245 2024-04-22T19:58:43.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.078+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MPK0SN92KEBG.0 size=1048576 2024-04-22T19:58:43.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.079+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MPK0SN92KEBG.1 size=331385 2024-04-22T19:58:43.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.079+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=MPK0SN92KEBG tag=7A0Q3IMB13QA type=write size=2428537 2024-04-22T19:58:43.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.080+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~6WHO507ADKW1.0 size=1048576 2024-04-22T19:58:43.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.081+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~6WHO507ADKW1.1 size=1048576 2024-04-22T19:58:43.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.082+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~6WHO507ADKW1.2 size=546571 2024-04-22T19:58:43.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.084+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=6WHO507ADKW1 tag=ID3PZKI2S910 type=write size=3692299 2024-04-22T19:58:43.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.086+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=9V155WEOB3CZ type=write size=241305 2024-04-22T19:58:43.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.086+0000 7f5269a8e780 1 < canceled part key=_multipart_5UBYMQT7ZWKX.2~TXWZ64P4PTIR.0 size=179811 2024-04-22T19:58:43.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.088+0000 7f5269a8e780 1 < canceled multipart upload key=5UBYMQT7ZWKX upload=TXWZ64P4PTIR tag=TB3WPWJZP2YZ type=write size=1228387 2024-04-22T19:58:43.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.089+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=65Y3W5IFOCI5 type=del size=713411 2024-04-22T19:58:43.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.090+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~10ZW7VHKUVUS.0 size=1048576 2024-04-22T19:58:43.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.091+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~10ZW7VHKUVUS.1 size=95678 2024-04-22T19:58:43.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.093+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=10ZW7VHKUVUS tag=K5HQ0B36NKFK type=write size=2192830 2024-04-22T19:58:43.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.094+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=MPJZPATUCSCY type=del size=2927559 2024-04-22T19:58:43.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.095+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~DAE0QF4YBCUW.0 size=1048576 2024-04-22T19:58:43.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.096+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~DAE0QF4YBCUW.1 size=1048576 2024-04-22T19:58:43.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.097+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~DAE0QF4YBCUW.2 size=168651 2024-04-22T19:58:43.099 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.099+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=DAE0QF4YBCUW tag=QWN2W311Z2O5 type=write size=3314379 2024-04-22T19:58:43.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.100+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=11DMAA9SFON3 type=write size=139461 2024-04-22T19:58:43.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.101+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~DQEELLZYRT9U.0 size=174970 2024-04-22T19:58:43.104 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.104+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=DQEELLZYRT9U tag=ADLVFDBO318V type=write size=1223546 2024-04-22T19:58:43.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.105+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=X90N41H8YYNN type=del size=830721 2024-04-22T19:58:43.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.107+0000 7f5269a8e780 1 < suggested remove operation key=RY1O7X1CWZMK tag=R5HJ2C3CUE9R type=del 2024-04-22T19:58:43.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.109+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~YZOAHPN8BW0Y.0 size=1048576 2024-04-22T19:58:43.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.110+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~YZOAHPN8BW0Y.1 size=1048576 2024-04-22T19:58:43.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.111+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~YZOAHPN8BW0Y.2 size=751956 2024-04-22T19:58:43.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.111+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=YZOAHPN8BW0Y tag=MO08I944V5FJ type=write size=3897684 2024-04-22T19:58:43.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.112+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=5NSWK8RC86OQ type=del 2024-04-22T19:58:43.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.113+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=ZQTR4XXQSF3Q type=write size=873007 2024-04-22T19:58:43.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.114+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=SWO8TBCFZTCD type=del 2024-04-22T19:58:43.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.115+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=PC7WIT4MLSU9 type=del size=3195509 2024-04-22T19:58:43.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.116+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~91FFTUE6523N.0 size=1048576 2024-04-22T19:58:43.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.117+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~91FFTUE6523N.1 size=1048576 2024-04-22T19:58:43.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.118+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~91FFTUE6523N.2 size=197513 2024-04-22T19:58:43.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.120+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=91FFTUE6523N tag=SM25EEJ4MCYI type=write size=3343241 2024-04-22T19:58:43.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.121+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=RXX8B8OQCGDY type=write size=409824 2024-04-22T19:58:43.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.122+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=Q6M6O49US7CN type=del 2024-04-22T19:58:43.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.123+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=OQ12F6IBXGK3 type=del size=4054903 2024-04-22T19:58:43.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.125+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=WYDKCVYOTX8D type=del 2024-04-22T19:58:43.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.126+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=O723HHOLOA8U type=del size=1220343 2024-04-22T19:58:43.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.127+0000 7f5269a8e780 1 < completed write operation key=GRSL48RUEBX8 tag=ATC9ZC9W2QG2 type=write size=374918 2024-04-22T19:58:43.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.129+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~DYTFGROSVQ0Y.0 size=1048576 2024-04-22T19:58:43.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.130+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~DYTFGROSVQ0Y.1 size=1048576 2024-04-22T19:58:43.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.131+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~DYTFGROSVQ0Y.2 size=150631 2024-04-22T19:58:43.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.131+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=DYTFGROSVQ0Y tag=BSV3U4D3Z0J9 type=write size=3296359 2024-04-22T19:58:43.132 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.132+0000 7f5269a8e780 1 < canceled operation key=GX6NILQG878S tag=68F15YQF2Y2A type=write size=122853 2024-04-22T19:58:43.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.133+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=GNTK3SG482U7 type=write size=265605 2024-04-22T19:58:43.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.135+0000 7f5269a8e780 1 < suggested remove operation key=5UBYMQT7ZWKX tag=AXCQ59ESFXMV type=del 2024-04-22T19:58:43.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.136+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=UDMRTMEW4WHF type=del size=2818111 2024-04-22T19:58:43.137 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.137+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=N34NX1B45BP5 type=del 2024-04-22T19:58:43.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.138+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=U1YSQ51FFS9T type=write size=490757 2024-04-22T19:58:43.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.140+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~MPK0SN92KEBG.0 size=1048576 2024-04-22T19:58:43.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.140+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~MPK0SN92KEBG.1 size=331385 2024-04-22T19:58:43.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.142+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=MPK0SN92KEBG tag=7A0Q3IMB13QA type=write size=2428537 2024-04-22T19:58:43.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.143+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=SUX6CAOIJ9K8 type=del size=1310736 2024-04-22T19:58:43.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.145+0000 7f5269a8e780 1 < completed write operation key=AMBJHMJGLUUS tag=9V155WEOB3CZ type=write size=241305 2024-04-22T19:58:43.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.146+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=5O4BPELZI9SL type=del size=1344393 2024-04-22T19:58:43.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.147+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=65Y3W5IFOCI5 type=del 2024-04-22T19:58:43.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.148+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=65MP094SUCDW type=del size=3056771 2024-04-22T19:58:43.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.149+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=MPJZPATUCSCY type=del 2024-04-22T19:58:43.151 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.151+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~E58EQZGKDCP2.0 size=1048576 2024-04-22T19:58:43.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.152+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~E58EQZGKDCP2.1 size=687929 2024-04-22T19:58:43.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.152+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=E58EQZGKDCP2 tag=XMO3BLNJ3G0Z type=write size=2785081 2024-04-22T19:58:43.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.153+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=11DMAA9SFON3 type=write size=139461 2024-04-22T19:58:43.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.154+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=WEP04FEMNOK7 type=del size=190144 2024-04-22T19:58:43.155 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.155+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=X90N41H8YYNN type=del 2024-04-22T19:58:43.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.157+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=OSR7AUDBVJSQ type=del size=261665 2024-04-22T19:58:43.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.158+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~YZOAHPN8BW0Y.0 size=1048576 2024-04-22T19:58:43.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.159+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~YZOAHPN8BW0Y.1 size=1048576 2024-04-22T19:58:43.160 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.160+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~YZOAHPN8BW0Y.2 size=751956 2024-04-22T19:58:43.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.162+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=YZOAHPN8BW0Y tag=MO08I944V5FJ type=write size=3897684 2024-04-22T19:58:43.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.163+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=O893PO1SCHJ2 type=del size=630686 2024-04-22T19:58:43.165 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.165+0000 7f5269a8e780 1 < suggested remove operation key=Q1UKF4UMJ2HO tag=ZQTR4XXQSF3Q type=write 2024-04-22T19:58:43.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.167+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~4QM0VDJ4ER5A.0 size=398020 2024-04-22T19:58:43.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.167+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=4QM0VDJ4ER5A tag=D1Z7EWJFWVF3 type=write size=1446596 2024-04-22T19:58:43.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.168+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=PC7WIT4MLSU9 type=del 2024-04-22T19:58:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.170+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~FRFFJCP22Z9D.0 size=1048576 2024-04-22T19:58:43.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.171+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~FRFFJCP22Z9D.1 size=1048576 2024-04-22T19:58:43.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.172+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~FRFFJCP22Z9D.2 size=859444 2024-04-22T19:58:43.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.172+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=FRFFJCP22Z9D tag=VUSVL7DZ69KH type=write size=4005172 2024-04-22T19:58:43.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.173+0000 7f5269a8e780 1 < completed write operation key=DX5YUYQRZKWA tag=RXX8B8OQCGDY type=write size=409824 2024-04-22T19:58:43.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.174+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=4J0BGKUEF4BI type=del size=699668 2024-04-22T19:58:43.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.176+0000 7f5269a8e780 1 < suggested update operation key=8V6ESIGMRO5G tag=OQ12F6IBXGK3 type=del size=4054903 2024-04-22T19:58:43.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.177+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=E3O2UWG1YHZF type=del size=2581619 2024-04-22T19:58:43.178 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.178+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=O723HHOLOA8U type=del 2024-04-22T19:58:43.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.180+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~F659637KTREB.0 size=1048576 2024-04-22T19:58:43.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.181+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~F659637KTREB.1 size=1048576 2024-04-22T19:58:43.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.182+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~F659637KTREB.2 size=8982 2024-04-22T19:58:43.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.182+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=F659637KTREB tag=3XBKBJGRC2LE type=write size=3154710 2024-04-22T19:58:43.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.183+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~DYTFGROSVQ0Y.0 size=1048576 2024-04-22T19:58:43.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.184+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~DYTFGROSVQ0Y.1 size=1048576 2024-04-22T19:58:43.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.185+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~DYTFGROSVQ0Y.2 size=150631 2024-04-22T19:58:43.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.187+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=DYTFGROSVQ0Y tag=BSV3U4D3Z0J9 type=write size=3296359 2024-04-22T19:58:43.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.189+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=WN8LX53XIQY0 type=del size=272954 2024-04-22T19:58:43.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.190+0000 7f5269a8e780 1 < completed write operation key=2YFD2U6HVNE7 tag=GNTK3SG482U7 type=write size=265605 2024-04-22T19:58:43.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.192+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MW3CFBXL92G3.0 size=88178 2024-04-22T19:58:43.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.192+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=MW3CFBXL92G3 tag=C7VQH4ZHSZFX type=write size=1136754 2024-04-22T19:58:43.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.194+0000 7f5269a8e780 1 < suggested update operation key=B0Y7ZTPC0975 tag=UDMRTMEW4WHF type=del size=2818111 2024-04-22T19:58:43.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.195+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=NC603TJ35KHZ type=write size=647078 2024-04-22T19:58:43.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.196+0000 7f5269a8e780 1 < completed write operation key=2YFD2U6HVNE7 tag=U1YSQ51FFS9T type=write size=490757 2024-04-22T19:58:43.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.198+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~HJ9SVERXWZS4.0 size=1008149 2024-04-22T19:58:43.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.198+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=HJ9SVERXWZS4 tag=5P4H0TRFWH4M type=write size=2056725 2024-04-22T19:58:43.200 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.200+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=SUX6CAOIJ9K8 type=del 2024-04-22T19:58:43.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.201+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=10GV34HRSGBP type=del size=3985907 2024-04-22T19:58:43.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.202+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=5O4BPELZI9SL type=del 2024-04-22T19:58:43.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.203+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=XSX0G30CH1UW type=del size=997764 2024-04-22T19:58:43.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.204+0000 7f5269a8e780 -1 < failed to complete operation key=B0Y7ZTPC0975 tag=65MP094SUCDW type=del size=3056771: (22) Invalid argument 2024-04-22T19:58:43.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.205+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=JS4M9XMU0BFC type=write size=716273 2024-04-22T19:58:43.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.206+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~E58EQZGKDCP2.0 size=1048576 2024-04-22T19:58:43.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.207+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~E58EQZGKDCP2.1 size=687929 2024-04-22T19:58:43.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.210+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=E58EQZGKDCP2 tag=XMO3BLNJ3G0Z type=write size=2785081 2024-04-22T19:58:43.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.212+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~7T6WIXP1AJOB.0 size=1048576 2024-04-22T19:58:43.213 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.213+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~7T6WIXP1AJOB.1 size=1048576 2024-04-22T19:58:43.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.214+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~7T6WIXP1AJOB.2 size=678594 2024-04-22T19:58:43.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.214+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=7T6WIXP1AJOB tag=DRY0VQM3QND8 type=write size=3824322 2024-04-22T19:58:43.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.215+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=WEP04FEMNOK7 type=del 2024-04-22T19:58:43.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.216+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=N89JPZPX4MTL type=del size=2786101 2024-04-22T19:58:43.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.217+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=OSR7AUDBVJSQ type=del 2024-04-22T19:58:43.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.218+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=4GA8ZJQGGCVQ type=del size=212479 2024-04-22T19:58:43.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.219+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=O893PO1SCHJ2 type=del 2024-04-22T19:58:43.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.221+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=A5ZJQBJIW863 type=del size=1089869 2024-04-22T19:58:43.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.222+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~4QM0VDJ4ER5A.0 size=398020 2024-04-22T19:58:43.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.224+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=4QM0VDJ4ER5A tag=D1Z7EWJFWVF3 type=write size=1446596 2024-04-22T19:58:43.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.226+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~BC8LTOSVZQPL.0 size=1048576 2024-04-22T19:58:43.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.227+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~BC8LTOSVZQPL.1 size=773610 2024-04-22T19:58:43.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.227+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=BC8LTOSVZQPL tag=NI1JSKWCUVBQ type=write size=2870762 2024-04-22T19:58:43.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.228+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~FRFFJCP22Z9D.0 size=1048576 2024-04-22T19:58:43.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.229+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~FRFFJCP22Z9D.1 size=1048576 2024-04-22T19:58:43.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.229+0000 7f5269a8e780 1 < canceled part key=_multipart_HV8J6YM0YNC1.2~FRFFJCP22Z9D.2 size=859444 2024-04-22T19:58:43.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.231+0000 7f5269a8e780 1 < canceled multipart upload key=HV8J6YM0YNC1 upload=FRFFJCP22Z9D tag=VUSVL7DZ69KH type=write size=4005172 2024-04-22T19:58:43.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.232+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=09ZL4CL1EIYI type=del size=2899 2024-04-22T19:58:43.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.233+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=4J0BGKUEF4BI type=del 2024-04-22T19:58:43.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.235+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=5DC7G1X3HJPA type=del size=710401 2024-04-22T19:58:43.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.236+0000 7f5269a8e780 1 < canceled operation key=EHRS6DQDRXVS tag=E3O2UWG1YHZF type=del size=2581619 2024-04-22T19:58:43.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.237+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=8DOPTAPYC4JC type=del size=886808 2024-04-22T19:58:43.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.239+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~F659637KTREB.0 size=1048576 2024-04-22T19:58:43.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.240+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~F659637KTREB.1 size=1048576 2024-04-22T19:58:43.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.241+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~F659637KTREB.2 size=8982 2024-04-22T19:58:43.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.242+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=F659637KTREB tag=3XBKBJGRC2LE type=write size=3154710 2024-04-22T19:58:43.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.244+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=01RW7HH6LFUW type=del size=3309371 2024-04-22T19:58:43.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.245+0000 7f5269a8e780 1 < canceled operation key=HV8J6YM0YNC1 tag=WN8LX53XIQY0 type=del size=272954 2024-04-22T19:58:43.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.247+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~QPT4PS4HMX0H.0 size=1048576 2024-04-22T19:58:43.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.248+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~QPT4PS4HMX0H.1 size=1048576 2024-04-22T19:58:43.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.248+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~QPT4PS4HMX0H.2 size=179701 2024-04-22T19:58:43.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.248+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=QPT4PS4HMX0H tag=XLGDL8BJR2UA type=write size=3325429 2024-04-22T19:58:43.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.250+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~MW3CFBXL92G3.0 size=88178 2024-04-22T19:58:43.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.252+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=MW3CFBXL92G3 tag=C7VQH4ZHSZFX type=write size=1136754 2024-04-22T19:58:43.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.253+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=BZ3AZCGBCZ4P type=del size=3156296 2024-04-22T19:58:43.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.254+0000 7f5269a8e780 1 < completed write operation key=2YFD2U6HVNE7 tag=NC603TJ35KHZ type=write size=647078 2024-04-22T19:58:43.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.255+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=KOSPZE2DUIBS type=del size=3107387 2024-04-22T19:58:43.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.257+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~HJ9SVERXWZS4.0 size=1008149 2024-04-22T19:58:43.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.259+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=HJ9SVERXWZS4 tag=5P4H0TRFWH4M type=write size=2056725 2024-04-22T19:58:43.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.260+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=ZGI8AUR3GD6M type=write size=380843 2024-04-22T19:58:43.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.261+0000 7f5269a8e780 1 < canceled operation key=VPAVX5FXYWZO tag=10GV34HRSGBP type=del size=3985907 2024-04-22T19:58:43.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.263+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~7T3ET4B2A7GD.0 size=1048576 2024-04-22T19:58:43.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.264+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~7T3ET4B2A7GD.1 size=1007315 2024-04-22T19:58:43.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.264+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=7T3ET4B2A7GD tag=VF5RK4ZG42DF type=write size=3104467 2024-04-22T19:58:43.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.265+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=XSX0G30CH1UW type=del 2024-04-22T19:58:43.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.266+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=DGGFA3MNUJKJ type=del size=1374865 2024-04-22T19:58:43.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.267+0000 7f5269a8e780 1 < completed write operation key=Y5HR8XW4L0SP tag=JS4M9XMU0BFC type=write size=716273 2024-04-22T19:58:43.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.269+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=Y1XA0ZN0LYNU type=del size=888672 2024-04-22T19:58:43.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.269+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~7T6WIXP1AJOB.0 size=1048576 2024-04-22T19:58:43.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.269+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~7T6WIXP1AJOB.1 size=1048576 2024-04-22T19:58:43.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.270+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~7T6WIXP1AJOB.2 size=678594 2024-04-22T19:58:43.271 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.272+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=7T6WIXP1AJOB tag=DRY0VQM3QND8 type=write size=3824322 2024-04-22T19:58:43.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.273+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=S8L3VOLROMAK type=del size=2575469 2024-04-22T19:58:43.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.274+0000 7f5269a8e780 1 < canceled operation key=IBH6W1NINUHC tag=N89JPZPX4MTL type=del size=2786101 2024-04-22T19:58:43.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.276+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~LR8X91BM3SIY.0 size=1048576 2024-04-22T19:58:43.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.277+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~LR8X91BM3SIY.1 size=1048576 2024-04-22T19:58:43.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.278+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~LR8X91BM3SIY.2 size=1035269 2024-04-22T19:58:43.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.278+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=LR8X91BM3SIY tag=O34X9B9PFDGE type=write size=4180997 2024-04-22T19:58:43.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.279+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=4GA8ZJQGGCVQ type=del 2024-04-22T19:58:43.280 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.280+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=OD9NJH78S508 type=del size=4122844 2024-04-22T19:58:43.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.281+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=A5ZJQBJIW863 type=del 2024-04-22T19:58:43.283 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.283+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~IHB3G9CMUGGR.0 size=1048576 2024-04-22T19:58:43.284 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.284+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~IHB3G9CMUGGR.1 size=1048576 2024-04-22T19:58:43.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.285+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~IHB3G9CMUGGR.2 size=578098 2024-04-22T19:58:43.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.285+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=IHB3G9CMUGGR tag=P0NMCHXV1SBX type=write size=3723826 2024-04-22T19:58:43.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.286+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~BC8LTOSVZQPL.0 size=1048576 2024-04-22T19:58:43.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.287+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~BC8LTOSVZQPL.1 size=773610 2024-04-22T19:58:43.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.289+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=BC8LTOSVZQPL tag=NI1JSKWCUVBQ type=write size=2870762 2024-04-22T19:58:43.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.291+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~5YSQFFLF22NT.0 size=1048576 2024-04-22T19:58:43.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.292+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~5YSQFFLF22NT.1 size=503161 2024-04-22T19:58:43.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.292+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=5YSQFFLF22NT tag=S6A0GPQXHA53 type=write size=2600313 2024-04-22T19:58:43.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.293+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=09ZL4CL1EIYI type=del 2024-04-22T19:58:43.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.294+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=RCOTBTQFVBH6 type=del size=2430649 2024-04-22T19:58:43.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.295+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=5DC7G1X3HJPA type=del 2024-04-22T19:58:43.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.296+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=MV7J6K103PLD type=write size=552582 2024-04-22T19:58:43.297 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.297+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=8DOPTAPYC4JC type=del 2024-04-22T19:58:43.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.299+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~GRABB8SL0CMB.0 size=424801 2024-04-22T19:58:43.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.299+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=GRABB8SL0CMB tag=H8P0VLPNF94M type=write size=1473377 2024-04-22T19:58:43.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.301+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=01RW7HH6LFUW type=del 2024-04-22T19:58:43.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.302+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=LEV2236J7JPF type=del size=131220 2024-04-22T19:58:43.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.303+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~QPT4PS4HMX0H.0 size=1048576 2024-04-22T19:58:43.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.304+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~QPT4PS4HMX0H.1 size=1048576 2024-04-22T19:58:43.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.305+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~QPT4PS4HMX0H.2 size=179701 2024-04-22T19:58:43.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.307+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=QPT4PS4HMX0H tag=XLGDL8BJR2UA type=write size=3325429 2024-04-22T19:58:43.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.309+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~YYQI8BQAL9EJ.0 size=1048576 2024-04-22T19:58:43.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.310+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~YYQI8BQAL9EJ.1 size=1048576 2024-04-22T19:58:43.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.311+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~YYQI8BQAL9EJ.2 size=486790 2024-04-22T19:58:43.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.311+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=YYQI8BQAL9EJ tag=HTNXQU70HYMX type=write size=3632518 2024-04-22T19:58:43.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.312+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=BZ3AZCGBCZ4P type=del 2024-04-22T19:58:43.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.313+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=K3ECLPOYNYAC type=write size=704521 2024-04-22T19:58:43.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.314+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=KOSPZE2DUIBS type=del 2024-04-22T19:58:43.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.317+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~PASEBTAIKY95.0 size=1048576 2024-04-22T19:58:43.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.318+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~PASEBTAIKY95.1 size=1048576 2024-04-22T19:58:43.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.319+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~PASEBTAIKY95.2 size=713006 2024-04-22T19:58:43.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.319+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=PASEBTAIKY95 tag=T23JWFQ6B9YV type=write size=3858734 2024-04-22T19:58:43.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.322+0000 7f5269a8e780 1 < suggested update operation key=M9M4SEEEZSFV tag=ZGI8AUR3GD6M type=write size=380843 2024-04-22T19:58:43.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.323+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=RT0SX0BEX252 type=write size=341408 2024-04-22T19:58:43.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.324+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~7T3ET4B2A7GD.0 size=1048576 2024-04-22T19:58:43.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.325+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~7T3ET4B2A7GD.1 size=1007315 2024-04-22T19:58:43.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.328+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=7T3ET4B2A7GD tag=VF5RK4ZG42DF type=write size=3104467 2024-04-22T19:58:43.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.329+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=A29Z8DYYYIEZ type=del size=1431463 2024-04-22T19:58:43.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.330+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=DGGFA3MNUJKJ type=del 2024-04-22T19:58:43.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.331+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=43ZGR87PJ2TT type=del size=3986772 2024-04-22T19:58:43.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.333+0000 7f5269a8e780 1 < suggested update operation key=I91AD8FJ3P4T tag=Y1XA0ZN0LYNU type=del size=888672 2024-04-22T19:58:43.334 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.335+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=C023ULYT1JO8 type=del size=4175431 2024-04-22T19:58:43.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.336+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=S8L3VOLROMAK type=del 2024-04-22T19:58:43.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.337+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=Q1O9OPG7B5FF type=del size=2992753 2024-04-22T19:58:43.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.338+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~LR8X91BM3SIY.0 size=1048576 2024-04-22T19:58:43.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.339+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~LR8X91BM3SIY.1 size=1048576 2024-04-22T19:58:43.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.340+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~LR8X91BM3SIY.2 size=1035269 2024-04-22T19:58:43.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.342+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=LR8X91BM3SIY tag=O34X9B9PFDGE type=write size=4180997 2024-04-22T19:58:43.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.343+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=8W6H5TWJKCEH type=del size=3279023 2024-04-22T19:58:43.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.344+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=OD9NJH78S508 type=del 2024-04-22T19:58:43.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.345+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=C0N9LJF1OZX0 type=del size=2138295 2024-04-22T19:58:43.346 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.346+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~IHB3G9CMUGGR.0 size=1048576 2024-04-22T19:58:43.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.347+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~IHB3G9CMUGGR.1 size=1048576 2024-04-22T19:58:43.348 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.348+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~IHB3G9CMUGGR.2 size=578098 2024-04-22T19:58:43.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.350+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=IHB3G9CMUGGR tag=P0NMCHXV1SBX type=write size=3723826 2024-04-22T19:58:43.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.352+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=T2BECY4WYLTQ type=del size=3243690 2024-04-22T19:58:43.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.353+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~5YSQFFLF22NT.0 size=1048576 2024-04-22T19:58:43.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.354+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~5YSQFFLF22NT.1 size=503161 2024-04-22T19:58:43.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.356+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=5YSQFFLF22NT tag=S6A0GPQXHA53 type=write size=2600313 2024-04-22T19:58:43.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.358+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~JA2XFQ8RN1VS.0 size=1048576 2024-04-22T19:58:43.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.359+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~JA2XFQ8RN1VS.1 size=1048576 2024-04-22T19:58:43.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.360+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~JA2XFQ8RN1VS.2 size=817061 2024-04-22T19:58:43.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.360+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=JA2XFQ8RN1VS tag=7JE84UWUDV5W type=write size=3962789 2024-04-22T19:58:43.362 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.362+0000 7f5269a8e780 1 < suggested remove operation key=IBH6W1NINUHC tag=RCOTBTQFVBH6 type=del 2024-04-22T19:58:43.363 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.363+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=ABT8DX6WEUJV type=del size=1901337 2024-04-22T19:58:43.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.364+0000 7f5269a8e780 1 < completed write operation key=4EA3LUQRL8II tag=MV7J6K103PLD type=write size=552582 2024-04-22T19:58:43.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.365+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=TW0MLV2GVVAA type=write size=683314 2024-04-22T19:58:43.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.366+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~GRABB8SL0CMB.0 size=424801 2024-04-22T19:58:43.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.368+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=GRABB8SL0CMB tag=H8P0VLPNF94M type=write size=1473377 2024-04-22T19:58:43.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.370+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~N1UR5IVPDRIS.0 size=1048576 2024-04-22T19:58:43.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.371+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~N1UR5IVPDRIS.1 size=730458 2024-04-22T19:58:43.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.371+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=N1UR5IVPDRIS tag=L0U4BWPTH0ZE type=write size=2827610 2024-04-22T19:58:43.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.372+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=LEV2236J7JPF type=del 2024-04-22T19:58:43.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.373+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=HOQD760X2EHF type=del size=1482916 2024-04-22T19:58:43.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.374+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~YYQI8BQAL9EJ.0 size=1048576 2024-04-22T19:58:43.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.374+0000 7f5269a8e780 1 < canceled part key=_multipart_8V6ESIGMRO5G.2~YYQI8BQAL9EJ.1 size=1048576 2024-04-22T19:58:43.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.375+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~YYQI8BQAL9EJ.2 size=486790 2024-04-22T19:58:43.377 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.377+0000 7f5269a8e780 1 < canceled multipart upload key=8V6ESIGMRO5G upload=YYQI8BQAL9EJ tag=HTNXQU70HYMX type=write size=3632518 2024-04-22T19:58:43.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.378+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=YVSCUSMV908C type=del size=3266897 2024-04-22T19:58:43.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.380+0000 7f5269a8e780 1 < suggested remove operation key=GP1C5ULUU143 tag=K3ECLPOYNYAC type=write 2024-04-22T19:58:43.381 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.381+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=QHCWICIJOW44 type=del size=2547031 2024-04-22T19:58:43.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.382+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~PASEBTAIKY95.0 size=1048576 2024-04-22T19:58:43.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.383+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~PASEBTAIKY95.1 size=1048576 2024-04-22T19:58:43.384 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.384+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~PASEBTAIKY95.2 size=713006 2024-04-22T19:58:43.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.386+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=PASEBTAIKY95 tag=T23JWFQ6B9YV type=write size=3858734 2024-04-22T19:58:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.388+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~1A04S2R0NHTA.0 size=1048576 2024-04-22T19:58:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.389+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~1A04S2R0NHTA.1 size=497831 2024-04-22T19:58:43.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.389+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=1A04S2R0NHTA tag=D1IFFLOJ554I type=write size=2594983 2024-04-22T19:58:43.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.390+0000 7f5269a8e780 1 < canceled operation key=B0Y7ZTPC0975 tag=RT0SX0BEX252 type=write size=341408 2024-04-22T19:58:43.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.391+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=1CEWT6UHIOS6 type=del size=1914207 2024-04-22T19:58:43.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.392+0000 7f5269a8e780 -1 < failed to complete operation key=IBH6W1NINUHC tag=A29Z8DYYYIEZ type=del size=1431463: (22) Invalid argument 2024-04-22T19:58:43.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.394+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~QXHPHDTHINF5.0 size=1048576 2024-04-22T19:58:43.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.395+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~QXHPHDTHINF5.1 size=63895 2024-04-22T19:58:43.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.395+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=QXHPHDTHINF5 tag=BZG0S3P3GZZD type=write size=2161047 2024-04-22T19:58:43.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.396+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=43ZGR87PJ2TT type=del 2024-04-22T19:58:43.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.398+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~GQ3LOLGK4GLF.0 size=1048576 2024-04-22T19:58:43.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.398+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~GQ3LOLGK4GLF.1 size=257334 2024-04-22T19:58:43.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.398+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=GQ3LOLGK4GLF tag=EYBCMS4VQ85V type=write size=2354486 2024-04-22T19:58:43.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.401+0000 7f5269a8e780 1 < suggested remove operation key=GRSL48RUEBX8 tag=C023ULYT1JO8 type=del 2024-04-22T19:58:43.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.402+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=4EIBLM0BRYC8 type=del size=2621875 2024-04-22T19:58:43.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.403+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=Q1O9OPG7B5FF type=del 2024-04-22T19:58:43.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.405+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~EAO3VJZZM1ON.0 size=75719 2024-04-22T19:58:43.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.405+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=EAO3VJZZM1ON tag=PB31BIFXAB7A type=write size=1124295 2024-04-22T19:58:43.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.406+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=8W6H5TWJKCEH type=del 2024-04-22T19:58:43.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.407+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=GR9BNBQ45MTK type=del size=3220617 2024-04-22T19:58:43.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.409+0000 7f5269a8e780 1 < suggested update operation key=HIAXPVKZ1IIO tag=C0N9LJF1OZX0 type=del size=2138295 2024-04-22T19:58:43.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.410+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=U3RO5WJICH4X type=del size=1185696 2024-04-22T19:58:43.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.411+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=T2BECY4WYLTQ type=del 2024-04-22T19:58:43.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.413+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~8H793VESD0GL.0 size=293660 2024-04-22T19:58:43.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.413+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=8H793VESD0GL tag=6OC41VM4821K type=write size=1342236 2024-04-22T19:58:43.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.414+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~JA2XFQ8RN1VS.0 size=1048576 2024-04-22T19:58:43.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.415+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~JA2XFQ8RN1VS.1 size=1048576 2024-04-22T19:58:43.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.416+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~JA2XFQ8RN1VS.2 size=817061 2024-04-22T19:58:43.418 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.418+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=JA2XFQ8RN1VS tag=7JE84UWUDV5W type=write size=3962789 2024-04-22T19:58:43.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.419+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=92F343MT8LVK type=del size=3374287 2024-04-22T19:58:43.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.421+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=ABT8DX6WEUJV type=del 2024-04-22T19:58:43.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.424+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~LA9AQ4X75EQG.0 size=1048576 2024-04-22T19:58:43.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.426+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~LA9AQ4X75EQG.1 size=390640 2024-04-22T19:58:43.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.426+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=LA9AQ4X75EQG tag=ZX7GHDQCHZF1 type=write size=2487792 2024-04-22T19:58:43.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.427+0000 7f5269a8e780 1 < completed write operation key=70MLQGJPQYPY tag=TW0MLV2GVVAA type=write size=683314 2024-04-22T19:58:43.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.429+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~2XZYPHX3GPZ9.0 size=596367 2024-04-22T19:58:43.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.429+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=2XZYPHX3GPZ9 tag=ZBKMLP5LF7UY type=write size=1644943 2024-04-22T19:58:43.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.430+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~N1UR5IVPDRIS.0 size=1048576 2024-04-22T19:58:43.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.431+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~N1UR5IVPDRIS.1 size=730458 2024-04-22T19:58:43.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.433+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=N1UR5IVPDRIS tag=L0U4BWPTH0ZE type=write size=2827610 2024-04-22T19:58:43.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.435+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~CAVR903VXC32.0 size=1048576 2024-04-22T19:58:43.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.436+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~CAVR903VXC32.1 size=1048576 2024-04-22T19:58:43.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.437+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~CAVR903VXC32.2 size=125509 2024-04-22T19:58:43.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.437+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=CAVR903VXC32 tag=RTF6FTXXTXD8 type=write size=3271237 2024-04-22T19:58:43.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.438+0000 7f5269a8e780 -1 < failed to complete operation key=GP1C5ULUU143 tag=HOQD760X2EHF type=del size=1482916: (22) Invalid argument 2024-04-22T19:58:43.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.439+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=ZST45T23L0Y7 type=del size=2985420 2024-04-22T19:58:43.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.440+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=YVSCUSMV908C type=del 2024-04-22T19:58:43.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.443+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~W5PYMC1N9ZPU.0 size=1048576 2024-04-22T19:58:43.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.444+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~W5PYMC1N9ZPU.1 size=564015 2024-04-22T19:58:43.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.444+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=W5PYMC1N9ZPU tag=9NJS9LYCQY3V type=write size=2661167 2024-04-22T19:58:43.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.445+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=QHCWICIJOW44 type=del 2024-04-22T19:58:43.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.446+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=GKP9DQDXRNRG type=del size=2338270 2024-04-22T19:58:43.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.447+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~1A04S2R0NHTA.0 size=1048576 2024-04-22T19:58:43.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.448+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~1A04S2R0NHTA.1 size=497831 2024-04-22T19:58:43.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.449+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=1A04S2R0NHTA tag=D1IFFLOJ554I type=write size=2594983 2024-04-22T19:58:43.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.451+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=6BLEKY3OZKMY type=del size=2665420 2024-04-22T19:58:43.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.452+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=1CEWT6UHIOS6 type=del 2024-04-22T19:58:43.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.453+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=ZP4KKP5YMTZT type=del size=2655725 2024-04-22T19:58:43.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.454+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~QXHPHDTHINF5.0 size=1048576 2024-04-22T19:58:43.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.455+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~QXHPHDTHINF5.1 size=63895 2024-04-22T19:58:43.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.456+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=QXHPHDTHINF5 tag=BZG0S3P3GZZD type=write size=2161047 2024-04-22T19:58:43.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.458+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=XQ1RJ5STI567 type=del size=3947385 2024-04-22T19:58:43.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.459+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~GQ3LOLGK4GLF.0 size=1048576 2024-04-22T19:58:43.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.459+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~GQ3LOLGK4GLF.1 size=257334 2024-04-22T19:58:43.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.461+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=GQ3LOLGK4GLF tag=EYBCMS4VQ85V type=write size=2354486 2024-04-22T19:58:43.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.463+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~M16NEN76KDLA.0 size=287889 2024-04-22T19:58:43.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.463+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=M16NEN76KDLA tag=C7SLW152A1MD type=write size=1336465 2024-04-22T19:58:43.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.464+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=4EIBLM0BRYC8 type=del 2024-04-22T19:58:43.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.465+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=5FDK8NVW9RYS type=del size=1029441 2024-04-22T19:58:43.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.466+0000 7f5269a8e780 1 < canceled part key=_multipart_ZC4ZOHUJ3HQD.2~EAO3VJZZM1ON.0 size=75719 2024-04-22T19:58:43.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.467+0000 7f5269a8e780 1 < canceled multipart upload key=ZC4ZOHUJ3HQD upload=EAO3VJZZM1ON tag=PB31BIFXAB7A type=write size=1124295 2024-04-22T19:58:43.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.470+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~6JMFCZIM8ZNG.0 size=1048576 2024-04-22T19:58:43.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.470+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~6JMFCZIM8ZNG.1 size=323368 2024-04-22T19:58:43.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.470+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=6JMFCZIM8ZNG tag=KC6YD5YMFJYK type=write size=2420520 2024-04-22T19:58:43.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.472+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=GR9BNBQ45MTK type=del 2024-04-22T19:58:43.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.474+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~PPLX8G9H3UY9.0 size=326338 2024-04-22T19:58:43.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.474+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=PPLX8G9H3UY9 tag=VOBYY5BNAPJN type=write size=1374914 2024-04-22T19:58:43.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.475+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=U3RO5WJICH4X type=del 2024-04-22T19:58:43.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.477+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~ARPN5GFF5RKR.0 size=1048576 2024-04-22T19:58:43.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.478+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~ARPN5GFF5RKR.1 size=734957 2024-04-22T19:58:43.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.478+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=ARPN5GFF5RKR tag=2SYWMFN51KFN type=write size=2832109 2024-04-22T19:58:43.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.479+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~8H793VESD0GL.0 size=293660 2024-04-22T19:58:43.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.481+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=8H793VESD0GL tag=6OC41VM4821K type=write size=1342236 2024-04-22T19:58:43.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.483+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~21C6SD44RK56.0 size=1048576 2024-04-22T19:58:43.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.483+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~21C6SD44RK56.1 size=579318 2024-04-22T19:58:43.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.483+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=21C6SD44RK56 tag=OLZ2HR765QFQ type=write size=2676470 2024-04-22T19:58:43.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.485+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=92F343MT8LVK type=del 2024-04-22T19:58:43.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.486+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=MRGSWQ629Z7E type=write size=703522 2024-04-22T19:58:43.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.487+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~LA9AQ4X75EQG.0 size=1048576 2024-04-22T19:58:43.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.488+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~LA9AQ4X75EQG.1 size=390640 2024-04-22T19:58:43.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.490+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=LA9AQ4X75EQG tag=ZX7GHDQCHZF1 type=write size=2487792 2024-04-22T19:58:43.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.491+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=5AU79J03UUDL type=del size=1530383 2024-04-22T19:58:43.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.492+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~2XZYPHX3GPZ9.0 size=596367 2024-04-22T19:58:43.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.494+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=2XZYPHX3GPZ9 tag=ZBKMLP5LF7UY type=write size=1644943 2024-04-22T19:58:43.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.496+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~9HPB4QIB30OM.0 size=1048576 2024-04-22T19:58:43.497 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.497+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~9HPB4QIB30OM.1 size=102495 2024-04-22T19:58:43.497 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.497+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=9HPB4QIB30OM tag=5BXKN05NH5E2 type=write size=2199647 2024-04-22T19:58:43.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.498+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~CAVR903VXC32.0 size=1048576 2024-04-22T19:58:43.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.499+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~CAVR903VXC32.1 size=1048576 2024-04-22T19:58:43.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.500+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~CAVR903VXC32.2 size=125509 2024-04-22T19:58:43.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.502+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=CAVR903VXC32 tag=RTF6FTXXTXD8 type=write size=3271237 2024-04-22T19:58:43.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.503+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=ISK22RW7W6JT type=del size=2903027 2024-04-22T19:58:43.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.504+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=ZST45T23L0Y7 type=del 2024-04-22T19:58:43.506 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.506+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~M78B97MB1DX5.0 size=1048576 2024-04-22T19:58:43.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.507+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~M78B97MB1DX5.1 size=1048576 2024-04-22T19:58:43.508 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.508+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~M78B97MB1DX5.2 size=20346 2024-04-22T19:58:43.508 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.508+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=M78B97MB1DX5 tag=BHLIGBK97U2B type=write size=3166074 2024-04-22T19:58:43.510 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.510+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~W5PYMC1N9ZPU.0 size=1048576 2024-04-22T19:58:43.510 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.511+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~W5PYMC1N9ZPU.1 size=564015 2024-04-22T19:58:43.512 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.512+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=W5PYMC1N9ZPU tag=9NJS9LYCQY3V type=write size=2661167 2024-04-22T19:58:43.513 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.514+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=1N5NBXR8UOJZ type=del size=483179 2024-04-22T19:58:43.515 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.515+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=GKP9DQDXRNRG type=del 2024-04-22T19:58:43.516 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.516+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=MFV1PVACX7Y5 type=del size=2909695 2024-04-22T19:58:43.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.517+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=6BLEKY3OZKMY type=del 2024-04-22T19:58:43.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.520+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~VLXT1NOTE756.0 size=479261 2024-04-22T19:58:43.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.520+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=VLXT1NOTE756 tag=MQ8LBTS9MQ9N type=write size=1527837 2024-04-22T19:58:43.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.522+0000 7f5269a8e780 1 < suggested remove operation key=A853MOZPT9BG tag=ZP4KKP5YMTZT type=del 2024-04-22T19:58:43.523 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.523+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=F5LISBWPYTG8 type=del size=1087626 2024-04-22T19:58:43.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.525+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=XQ1RJ5STI567 type=del 2024-04-22T19:58:43.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.526+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=GV3RQIUSHM2U type=del size=913996 2024-04-22T19:58:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.528+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~M16NEN76KDLA.0 size=287889 2024-04-22T19:58:43.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.530+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=M16NEN76KDLA tag=C7SLW152A1MD type=write size=1336465 2024-04-22T19:58:43.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.531+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=0HK558310PNP type=del size=2533494 2024-04-22T19:58:43.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.534+0000 7f5269a8e780 1 < suggested update operation key=4GX83ZJWZCD1 tag=5FDK8NVW9RYS type=del size=1029441 2024-04-22T19:58:43.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.536+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~JUEJ8XW1ZMN5.0 size=1048576 2024-04-22T19:58:43.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.537+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~JUEJ8XW1ZMN5.1 size=494014 2024-04-22T19:58:43.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.537+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=JUEJ8XW1ZMN5 tag=7RORFLJBENIM type=write size=2591166 2024-04-22T19:58:43.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.538+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~6JMFCZIM8ZNG.0 size=1048576 2024-04-22T19:58:43.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.539+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~6JMFCZIM8ZNG.1 size=323368 2024-04-22T19:58:43.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.541+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=6JMFCZIM8ZNG tag=KC6YD5YMFJYK type=write size=2420520 2024-04-22T19:58:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.543+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=9VTYX28ZX342 type=del size=3444905 2024-04-22T19:58:43.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.544+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~PPLX8G9H3UY9.0 size=326338 2024-04-22T19:58:43.546 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.546+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=PPLX8G9H3UY9 tag=VOBYY5BNAPJN type=write size=1374914 2024-04-22T19:58:43.548 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.548+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~KCJK7X2IY60I.0 size=1048576 2024-04-22T19:58:43.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.549+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~KCJK7X2IY60I.1 size=34927 2024-04-22T19:58:43.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.549+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=KCJK7X2IY60I tag=0B067QTHAPXI type=write size=2132079 2024-04-22T19:58:43.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.551+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~ARPN5GFF5RKR.0 size=1048576 2024-04-22T19:58:43.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.552+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~ARPN5GFF5RKR.1 size=734957 2024-04-22T19:58:43.554 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.554+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=ARPN5GFF5RKR tag=2SYWMFN51KFN type=write size=2832109 2024-04-22T19:58:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.555+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=YSI68PSESXVA type=write size=895842 2024-04-22T19:58:43.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.556+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~21C6SD44RK56.0 size=1048576 2024-04-22T19:58:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.557+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~21C6SD44RK56.1 size=579318 2024-04-22T19:58:43.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.559+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=21C6SD44RK56 tag=OLZ2HR765QFQ type=write size=2676470 2024-04-22T19:58:43.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.560+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=5XI1ODOI4RDH type=write size=758057 2024-04-22T19:58:43.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.561+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=MRGSWQ629Z7E type=write size=703522 2024-04-22T19:58:43.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.563+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=B7P9H4R4V2K9 type=del size=1475090 2024-04-22T19:58:43.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.565+0000 7f5269a8e780 1 < suggested remove operation key=KV5DYQ49ZKCX tag=5AU79J03UUDL type=del 2024-04-22T19:58:43.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.566+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=95S8O58MHO9H type=del size=2160254 2024-04-22T19:58:43.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.567+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~9HPB4QIB30OM.0 size=1048576 2024-04-22T19:58:43.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.568+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~9HPB4QIB30OM.1 size=102495 2024-04-22T19:58:43.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.570+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=9HPB4QIB30OM tag=5BXKN05NH5E2 type=write size=2199647 2024-04-22T19:58:43.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.572+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=106WRM588L45 type=del size=389706 2024-04-22T19:58:43.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.573+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=ISK22RW7W6JT type=del 2024-04-22T19:58:43.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.574+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=15W6BX9WO3Y3 type=del size=1986868 2024-04-22T19:58:43.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.574+0000 7f5269a8e780 1 < canceled part key=_multipart_5UBYMQT7ZWKX.2~M78B97MB1DX5.0 size=1048576 2024-04-22T19:58:43.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.575+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~M78B97MB1DX5.1 size=1048576 2024-04-22T19:58:43.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.576+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~M78B97MB1DX5.2 size=20346 2024-04-22T19:58:43.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.578+0000 7f5269a8e780 1 < canceled multipart upload key=5UBYMQT7ZWKX upload=M78B97MB1DX5 tag=BHLIGBK97U2B type=write size=3166074 2024-04-22T19:58:43.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.579+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=UE7TE2M40Z3H type=del size=1955899 2024-04-22T19:58:43.582 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.582+0000 7f5269a8e780 1 < suggested update operation key=86GW6HV9G1IZ tag=1N5NBXR8UOJZ type=del size=483179 2024-04-22T19:58:43.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.584+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~NZYJAT5SBZGT.0 size=1048576 2024-04-22T19:58:43.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.585+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~NZYJAT5SBZGT.1 size=880079 2024-04-22T19:58:43.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.585+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=NZYJAT5SBZGT tag=9VPW57K63FKJ type=write size=2977231 2024-04-22T19:58:43.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.586+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=MFV1PVACX7Y5 type=del 2024-04-22T19:58:43.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.587+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~MJMOM09RE87Z.0 size=1048576 2024-04-22T19:58:43.588 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.588+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~MJMOM09RE87Z.1 size=422853 2024-04-22T19:58:43.588 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.588+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=MJMOM09RE87Z tag=UC0Y99YDGRU5 type=write size=2520005 2024-04-22T19:58:43.589 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.590+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~VLXT1NOTE756.0 size=479261 2024-04-22T19:58:43.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.591+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=VLXT1NOTE756 tag=MQ8LBTS9MQ9N type=write size=1527837 2024-04-22T19:58:43.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.593+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=PIZKBSBFNKMH type=write size=1017096 2024-04-22T19:58:43.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.594+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=F5LISBWPYTG8 type=del 2024-04-22T19:58:43.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.595+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=TH1Z814YB3LE type=del size=2379204 2024-04-22T19:58:43.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.596+0000 7f5269a8e780 1 < canceled operation key=O9NFAVE09ZN7 tag=GV3RQIUSHM2U type=del size=913996 2024-04-22T19:58:43.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.598+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=MA611V2JM9T4 type=del size=1186838 2024-04-22T19:58:43.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.599+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=0HK558310PNP type=del 2024-04-22T19:58:43.600 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.600+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=SGL2A2LYWMC7 type=del size=2162214 2024-04-22T19:58:43.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.601+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~JUEJ8XW1ZMN5.0 size=1048576 2024-04-22T19:58:43.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.602+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~JUEJ8XW1ZMN5.1 size=494014 2024-04-22T19:58:43.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.604+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=JUEJ8XW1ZMN5 tag=7RORFLJBENIM type=write size=2591166 2024-04-22T19:58:43.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.605+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=853EJ9FO1XWM type=del size=2873248 2024-04-22T19:58:43.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.606+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=9VTYX28ZX342 type=del 2024-04-22T19:58:43.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.608+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~HC2AXN4PBY3G.0 size=1048576 2024-04-22T19:58:43.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.609+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~HC2AXN4PBY3G.1 size=355939 2024-04-22T19:58:43.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.609+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=HC2AXN4PBY3G tag=Z15AHRA4IASM type=write size=2453091 2024-04-22T19:58:43.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.610+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~KCJK7X2IY60I.0 size=1048576 2024-04-22T19:58:43.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.611+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~KCJK7X2IY60I.1 size=34927 2024-04-22T19:58:43.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.613+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=KCJK7X2IY60I tag=0B067QTHAPXI type=write size=2132079 2024-04-22T19:58:43.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.615+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~ABCBA2GY3QE0.0 size=1048576 2024-04-22T19:58:43.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.616+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~ABCBA2GY3QE0.1 size=198848 2024-04-22T19:58:43.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.616+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=ABCBA2GY3QE0 tag=P2N5OYNHJRP7 type=write size=2296000 2024-04-22T19:58:43.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.617+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=YSI68PSESXVA type=write size=895842 2024-04-22T19:58:43.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.619+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~PR73HRFR9LM0.0 size=1048576 2024-04-22T19:58:43.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.620+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~PR73HRFR9LM0.1 size=1048576 2024-04-22T19:58:43.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.621+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~PR73HRFR9LM0.2 size=832446 2024-04-22T19:58:43.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.621+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=PR73HRFR9LM0 tag=6N1Z3LJKY484 type=write size=3978174 2024-04-22T19:58:43.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.622+0000 7f5269a8e780 -1 < failed to complete operation key=KV5DYQ49ZKCX tag=5XI1ODOI4RDH type=write size=758057: (22) Invalid argument 2024-04-22T19:58:43.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.623+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=JU14UBSZWM55 type=del size=2621291 2024-04-22T19:58:43.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.625+0000 7f5269a8e780 1 < suggested remove operation key=A853MOZPT9BG tag=B7P9H4R4V2K9 type=del 2024-04-22T19:58:43.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.627+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~VR3LTFVRO52D.0 size=1048576 2024-04-22T19:58:43.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.628+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~VR3LTFVRO52D.1 size=1048576 2024-04-22T19:58:43.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.629+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~VR3LTFVRO52D.2 size=1027647 2024-04-22T19:58:43.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.629+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=VR3LTFVRO52D tag=IP4YB876FP12 type=write size=4173375 2024-04-22T19:58:43.630 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.630+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=95S8O58MHO9H type=del 2024-04-22T19:58:43.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.631+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=MRE89V9NQ2LZ type=del size=778828 2024-04-22T19:58:43.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.632+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=106WRM588L45 type=del 2024-04-22T19:58:43.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.634+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~67TM7RIC0NPO.0 size=1048576 2024-04-22T19:58:43.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.636+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~67TM7RIC0NPO.1 size=1048576 2024-04-22T19:58:43.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.637+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~67TM7RIC0NPO.2 size=811107 2024-04-22T19:58:43.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.637+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=67TM7RIC0NPO tag=H7C7NBMD53CT type=write size=3956835 2024-04-22T19:58:43.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.639+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=15W6BX9WO3Y3 type=del 2024-04-22T19:58:43.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.641+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=BGRT2JET0EDM type=del size=2327783 2024-04-22T19:58:43.642 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.642+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=UE7TE2M40Z3H type=del 2024-04-22T19:58:43.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.643+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=FRKOHRGE5VDD type=del size=1560423 2024-04-22T19:58:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.644+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~NZYJAT5SBZGT.0 size=1048576 2024-04-22T19:58:43.645 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.645+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~NZYJAT5SBZGT.1 size=880079 2024-04-22T19:58:43.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.647+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=NZYJAT5SBZGT tag=9VPW57K63FKJ type=write size=2977231 2024-04-22T19:58:43.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.649+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~0WCYTHTACZY3.0 size=1048576 2024-04-22T19:58:43.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.650+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~0WCYTHTACZY3.1 size=1048576 2024-04-22T19:58:43.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.651+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~0WCYTHTACZY3.2 size=219069 2024-04-22T19:58:43.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.651+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=0WCYTHTACZY3 tag=9SM11XPNWEI2 type=write size=3364797 2024-04-22T19:58:43.653 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.653+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~MJMOM09RE87Z.0 size=1048576 2024-04-22T19:58:43.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.654+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~MJMOM09RE87Z.1 size=422853 2024-04-22T19:58:43.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.656+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=MJMOM09RE87Z tag=UC0Y99YDGRU5 type=write size=2520005 2024-04-22T19:58:43.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.657+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=B2217F9CQTAQ type=del size=1466644 2024-04-22T19:58:43.658 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.658+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=PIZKBSBFNKMH type=write size=1017096 2024-04-22T19:58:43.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.661+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~DJ1DTTZWNCVL.0 size=1048576 2024-04-22T19:58:43.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.661+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~DJ1DTTZWNCVL.1 size=1048576 2024-04-22T19:58:43.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.662+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~DJ1DTTZWNCVL.2 size=976791 2024-04-22T19:58:43.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.662+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=DJ1DTTZWNCVL tag=8C1RJTSC5404 type=write size=4122519 2024-04-22T19:58:43.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.663+0000 7f5269a8e780 1 < canceled operation key=I91AD8FJ3P4T tag=TH1Z814YB3LE type=del size=2379204 2024-04-22T19:58:43.665 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.665+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~SZ4SAQ3F50R0.0 size=1048576 2024-04-22T19:58:43.667 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.667+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~SZ4SAQ3F50R0.1 size=183242 2024-04-22T19:58:43.667 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.667+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=SZ4SAQ3F50R0 tag=TYSDHJQ8NPP7 type=write size=2280394 2024-04-22T19:58:43.668 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.668+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=MA611V2JM9T4 type=del 2024-04-22T19:58:43.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.670+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~8RZE90KZ71L4.0 size=1048576 2024-04-22T19:58:43.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.671+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~8RZE90KZ71L4.1 size=727250 2024-04-22T19:58:43.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.671+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=8RZE90KZ71L4 tag=P2P0BMLXIIIY type=write size=2824402 2024-04-22T19:58:43.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.672+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=SGL2A2LYWMC7 type=del 2024-04-22T19:58:43.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.673+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=26U2EQAM4QFC type=write size=459629 2024-04-22T19:58:43.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.675+0000 7f5269a8e780 1 < suggested update operation key=B0Y7ZTPC0975 tag=853EJ9FO1XWM type=del size=2873248 2024-04-22T19:58:43.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.676+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=460ODVLYETJB type=write size=931088 2024-04-22T19:58:43.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.677+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~HC2AXN4PBY3G.0 size=1048576 2024-04-22T19:58:43.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.678+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~HC2AXN4PBY3G.1 size=355939 2024-04-22T19:58:43.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.680+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=HC2AXN4PBY3G tag=Z15AHRA4IASM type=write size=2453091 2024-04-22T19:58:43.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.681+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=Y2ICR0USRG2E type=del size=1296039 2024-04-22T19:58:43.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.682+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~ABCBA2GY3QE0.0 size=1048576 2024-04-22T19:58:43.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.683+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~ABCBA2GY3QE0.1 size=198848 2024-04-22T19:58:43.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.685+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=ABCBA2GY3QE0 tag=P2N5OYNHJRP7 type=write size=2296000 2024-04-22T19:58:43.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.686+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=O83SKSONKVJ4 type=del size=620107 2024-04-22T19:58:43.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.687+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~PR73HRFR9LM0.0 size=1048576 2024-04-22T19:58:43.688 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.688+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~PR73HRFR9LM0.1 size=1048576 2024-04-22T19:58:43.689 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.689+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~PR73HRFR9LM0.2 size=832446 2024-04-22T19:58:43.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.691+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=PR73HRFR9LM0 tag=6N1Z3LJKY484 type=write size=3978174 2024-04-22T19:58:43.692 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.692+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=OGXDDU7GEJZN type=del size=2179467 2024-04-22T19:58:43.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.694+0000 7f5269a8e780 1 < suggested remove operation key=Q1UKF4UMJ2HO tag=JU14UBSZWM55 type=del 2024-04-22T19:58:43.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.696+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~D857M2POKXFR.0 size=793767 2024-04-22T19:58:43.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.696+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=D857M2POKXFR tag=F1UPG8H9ZYC4 type=write size=1842343 2024-04-22T19:58:43.697 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.697+0000 7f5269a8e780 1 < canceled part key=_multipart_I91AD8FJ3P4T.2~VR3LTFVRO52D.0 size=1048576 2024-04-22T19:58:43.697 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.697+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~VR3LTFVRO52D.1 size=1048576 2024-04-22T19:58:43.697 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.697+0000 7f5269a8e780 1 < canceled part key=_multipart_I91AD8FJ3P4T.2~VR3LTFVRO52D.2 size=1027647 2024-04-22T19:58:43.699 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.699+0000 7f5269a8e780 1 < canceled multipart upload key=I91AD8FJ3P4T upload=VR3LTFVRO52D tag=IP4YB876FP12 type=write size=4173375 2024-04-22T19:58:43.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.701+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~HUHNJC5LJW6B.0 size=914836 2024-04-22T19:58:43.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.701+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=HUHNJC5LJW6B tag=D9XYI360DWSP type=write size=1963412 2024-04-22T19:58:43.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.704+0000 7f5269a8e780 1 < suggested remove operation key=GRSL48RUEBX8 tag=MRE89V9NQ2LZ type=del 2024-04-22T19:58:43.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.705+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=K7PEYL4XU9TB type=del size=1433090 2024-04-22T19:58:43.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.706+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~67TM7RIC0NPO.0 size=1048576 2024-04-22T19:58:43.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.707+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~67TM7RIC0NPO.1 size=1048576 2024-04-22T19:58:43.708 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.708+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~67TM7RIC0NPO.2 size=811107 2024-04-22T19:58:43.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.710+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=67TM7RIC0NPO tag=H7C7NBMD53CT type=write size=3956835 2024-04-22T19:58:43.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.711+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=1PD02PWMSZ65 type=write size=1046862 2024-04-22T19:58:43.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.712+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=BGRT2JET0EDM type=del 2024-04-22T19:58:43.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.713+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=YMBHVBHM9MVC type=del size=1352068 2024-04-22T19:58:43.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.715+0000 7f5269a8e780 1 < suggested remove operation key=B0Y7ZTPC0975 tag=FRKOHRGE5VDD type=del 2024-04-22T19:58:43.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.717+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=VP8N2N9QIKPW type=del size=549710 2024-04-22T19:58:43.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.718+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~0WCYTHTACZY3.0 size=1048576 2024-04-22T19:58:43.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.719+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~0WCYTHTACZY3.1 size=1048576 2024-04-22T19:58:43.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.720+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~0WCYTHTACZY3.2 size=219069 2024-04-22T19:58:43.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.722+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=0WCYTHTACZY3 tag=9SM11XPNWEI2 type=write size=3364797 2024-04-22T19:58:43.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.723+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=1B25Y1G3K7GB type=write size=151178 2024-04-22T19:58:43.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.724+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=B2217F9CQTAQ type=del 2024-04-22T19:58:43.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.726+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~5P43XD3T05V3.0 size=301331 2024-04-22T19:58:43.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.726+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=5P43XD3T05V3 tag=QGDKB98WW7S8 type=write size=1349907 2024-04-22T19:58:43.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.727+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~DJ1DTTZWNCVL.0 size=1048576 2024-04-22T19:58:43.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.727+0000 7f5269a8e780 1 < canceled part key=_multipart_70MLQGJPQYPY.2~DJ1DTTZWNCVL.1 size=1048576 2024-04-22T19:58:43.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.728+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~DJ1DTTZWNCVL.2 size=976791 2024-04-22T19:58:43.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.729+0000 7f5269a8e780 1 < canceled multipart upload key=70MLQGJPQYPY upload=DJ1DTTZWNCVL tag=8C1RJTSC5404 type=write size=4122519 2024-04-22T19:58:43.730 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.730+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=UGR2TBQ27F05 type=del size=2064978 2024-04-22T19:58:43.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.732+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~SZ4SAQ3F50R0.0 size=1048576 2024-04-22T19:58:43.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.732+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~SZ4SAQ3F50R0.1 size=183242 2024-04-22T19:58:43.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.734+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=SZ4SAQ3F50R0 tag=TYSDHJQ8NPP7 type=write size=2280394 2024-04-22T19:58:43.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.736+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~1RRTN6RTAGTM.0 size=1048576 2024-04-22T19:58:43.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.737+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~1RRTN6RTAGTM.1 size=348591 2024-04-22T19:58:43.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.737+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=1RRTN6RTAGTM tag=MUSZ6MDQ6IYW type=write size=2445743 2024-04-22T19:58:43.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.738+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~8RZE90KZ71L4.0 size=1048576 2024-04-22T19:58:43.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.739+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~8RZE90KZ71L4.1 size=727250 2024-04-22T19:58:43.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.742+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=8RZE90KZ71L4 tag=P2P0BMLXIIIY type=write size=2824402 2024-04-22T19:58:43.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.743+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=33K3NYU597L9 type=del size=916194 2024-04-22T19:58:43.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.746+0000 7f5269a8e780 1 < suggested update operation key=8HFSDSNZONIN tag=26U2EQAM4QFC type=write size=459629 2024-04-22T19:58:43.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.747+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=2OIRTJ99ENTG type=write size=635060 2024-04-22T19:58:43.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.749+0000 7f5269a8e780 1 < completed write operation key=4EA3LUQRL8II tag=460ODVLYETJB type=write size=931088 2024-04-22T19:58:43.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.750+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=9RWR28GRWFFN type=del size=2108466 2024-04-22T19:58:43.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.751+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=Y2ICR0USRG2E type=del 2024-04-22T19:58:43.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.752+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=U06UBGXDH971 type=write size=502280 2024-04-22T19:58:43.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.754+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=O83SKSONKVJ4 type=del 2024-04-22T19:58:43.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.756+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~HTJR0DHK9JE2.0 size=1048576 2024-04-22T19:58:43.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.757+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~HTJR0DHK9JE2.1 size=7556 2024-04-22T19:58:43.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.757+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=HTJR0DHK9JE2 tag=CEZZGUDC845C type=write size=2104708 2024-04-22T19:58:43.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.759+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=OGXDDU7GEJZN type=del 2024-04-22T19:58:43.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.760+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=ZT1RVTW48SYT type=del size=3732253 2024-04-22T19:58:43.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.762+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~D857M2POKXFR.0 size=793767 2024-04-22T19:58:43.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.764+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=D857M2POKXFR tag=F1UPG8H9ZYC4 type=write size=1842343 2024-04-22T19:58:43.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.766+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~JK86Y5CUO4AN.0 size=1048576 2024-04-22T19:58:43.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.767+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~JK86Y5CUO4AN.1 size=191869 2024-04-22T19:58:43.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.767+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=JK86Y5CUO4AN tag=NT11LD0DAAST type=write size=2289021 2024-04-22T19:58:43.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.768+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~HUHNJC5LJW6B.0 size=914836 2024-04-22T19:58:43.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.770+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=HUHNJC5LJW6B tag=D9XYI360DWSP type=write size=1963412 2024-04-22T19:58:43.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.772+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~QE2KR2DZIKBF.0 size=1048576 2024-04-22T19:58:43.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.772+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~QE2KR2DZIKBF.1 size=789779 2024-04-22T19:58:43.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.772+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=QE2KR2DZIKBF tag=AZTVIZ0JT7X3 type=write size=2886931 2024-04-22T19:58:43.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.774+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=K7PEYL4XU9TB type=del 2024-04-22T19:58:43.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.775+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=MBG99JPG8WZ0 type=del size=1729914 2024-04-22T19:58:43.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.776+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=1PD02PWMSZ65 type=write size=1046862 2024-04-22T19:58:43.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.777+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=P44K3VVDDG13 type=del size=2620367 2024-04-22T19:58:43.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.778+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=YMBHVBHM9MVC type=del 2024-04-22T19:58:43.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.780+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~TQU7NGVHW5MT.0 size=1048576 2024-04-22T19:58:43.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.781+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~TQU7NGVHW5MT.1 size=1048576 2024-04-22T19:58:43.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.782+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~TQU7NGVHW5MT.2 size=372114 2024-04-22T19:58:43.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.782+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=TQU7NGVHW5MT tag=8S49XPCB5147 type=write size=3517842 2024-04-22T19:58:43.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.783+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=VP8N2N9QIKPW type=del 2024-04-22T19:58:43.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.785+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~ZCDK73162ZY9.0 size=445970 2024-04-22T19:58:43.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.785+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=ZCDK73162ZY9 tag=GKY44VTHT2B4 type=write size=1494546 2024-04-22T19:58:43.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.786+0000 7f5269a8e780 1 < canceled operation key=A853MOZPT9BG tag=1B25Y1G3K7GB type=write size=151178 2024-04-22T19:58:43.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.788+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=U1E5JC0ZCJ9J type=del size=3086598 2024-04-22T19:58:43.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.789+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~5P43XD3T05V3.0 size=301331 2024-04-22T19:58:43.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.791+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=5P43XD3T05V3 tag=QGDKB98WW7S8 type=write size=1349907 2024-04-22T19:58:43.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.793+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~P1CDNAUNHHS0.0 size=1048576 2024-04-22T19:58:43.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.794+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~P1CDNAUNHHS0.1 size=1048576 2024-04-22T19:58:43.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.795+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~P1CDNAUNHHS0.2 size=905586 2024-04-22T19:58:43.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.795+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=P1CDNAUNHHS0 tag=7OYVKLQWVMS3 type=write size=4051314 2024-04-22T19:58:43.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.796+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=UGR2TBQ27F05 type=del 2024-04-22T19:58:43.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.797+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=ESBOO66MP5KQ type=del size=2567607 2024-04-22T19:58:43.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.799+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~1RRTN6RTAGTM.0 size=1048576 2024-04-22T19:58:43.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.800+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~1RRTN6RTAGTM.1 size=348591 2024-04-22T19:58:43.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.802+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=1RRTN6RTAGTM tag=MUSZ6MDQ6IYW type=write size=2445743 2024-04-22T19:58:43.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.803+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=TVOVYCXV63TY type=del size=1081358 2024-04-22T19:58:43.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.804+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=33K3NYU597L9 type=del 2024-04-22T19:58:43.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.806+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~G4BLNJGF2N12.0 size=1048576 2024-04-22T19:58:43.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.807+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~G4BLNJGF2N12.1 size=99403 2024-04-22T19:58:43.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.807+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=G4BLNJGF2N12 tag=KLQ7Z4YKXQDS type=write size=2196555 2024-04-22T19:58:43.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.808+0000 7f5269a8e780 1 < completed write operation key=GRSL48RUEBX8 tag=2OIRTJ99ENTG type=write size=635060 2024-04-22T19:58:43.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.809+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=4BDBHNZO52BN type=del size=909461 2024-04-22T19:58:43.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.812+0000 7f5269a8e780 1 < suggested update operation key=KV5DYQ49ZKCX tag=9RWR28GRWFFN type=del size=2108466 2024-04-22T19:58:43.814 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.814+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~2DKPZN19QHTF.0 size=1048576 2024-04-22T19:58:43.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.815+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~2DKPZN19QHTF.1 size=1048576 2024-04-22T19:58:43.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.816+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~2DKPZN19QHTF.2 size=60456 2024-04-22T19:58:43.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.816+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=2DKPZN19QHTF tag=9ZNDXQW0ITNV type=write size=3206184 2024-04-22T19:58:43.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.817+0000 7f5269a8e780 1 < completed write operation key=VPAVX5FXYWZO tag=U06UBGXDH971 type=write size=502280 2024-04-22T19:58:43.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.818+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=5GY4BLMU4IMG type=del size=3262928 2024-04-22T19:58:43.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.819+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~HTJR0DHK9JE2.0 size=1048576 2024-04-22T19:58:43.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.820+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~HTJR0DHK9JE2.1 size=7556 2024-04-22T19:58:43.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.822+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=HTJR0DHK9JE2 tag=CEZZGUDC845C type=write size=2104708 2024-04-22T19:58:43.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.824+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~O7G6WBDNSIVQ.0 size=388542 2024-04-22T19:58:43.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.824+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=O7G6WBDNSIVQ tag=6XE6WJFSB7SO type=write size=1437118 2024-04-22T19:58:43.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.825+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=ZT1RVTW48SYT type=del 2024-04-22T19:58:43.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.826+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=2JR3O20ZQ3X6 type=del size=3417961 2024-04-22T19:58:43.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.827+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~JK86Y5CUO4AN.0 size=1048576 2024-04-22T19:58:43.828 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.828+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~JK86Y5CUO4AN.1 size=191869 2024-04-22T19:58:43.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.830+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=JK86Y5CUO4AN tag=NT11LD0DAAST type=write size=2289021 2024-04-22T19:58:43.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.831+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=UR36A50FXM2K type=del size=2771962 2024-04-22T19:58:43.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.832+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~QE2KR2DZIKBF.0 size=1048576 2024-04-22T19:58:43.833 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.833+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~QE2KR2DZIKBF.1 size=789779 2024-04-22T19:58:43.835 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.835+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=QE2KR2DZIKBF tag=AZTVIZ0JT7X3 type=write size=2886931 2024-04-22T19:58:43.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.837+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~IE3CY6YCT3O7.0 size=746219 2024-04-22T19:58:43.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.837+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=IE3CY6YCT3O7 tag=FA54VR4337BV type=write size=1794795 2024-04-22T19:58:43.838 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.838+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=MBG99JPG8WZ0 type=del 2024-04-22T19:58:43.839 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.839+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=5S845J028GLC type=del size=408964 2024-04-22T19:58:43.840 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.840+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=P44K3VVDDG13 type=del 2024-04-22T19:58:43.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.842+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~C9OKUDFAKKKE.0 size=1048576 2024-04-22T19:58:43.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.843+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~C9OKUDFAKKKE.1 size=677065 2024-04-22T19:58:43.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.843+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=C9OKUDFAKKKE tag=YBUNXA3AAAO2 type=write size=2774217 2024-04-22T19:58:43.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.844+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~TQU7NGVHW5MT.0 size=1048576 2024-04-22T19:58:43.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.845+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~TQU7NGVHW5MT.1 size=1048576 2024-04-22T19:58:43.846 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.846+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~TQU7NGVHW5MT.2 size=372114 2024-04-22T19:58:43.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.849+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=TQU7NGVHW5MT tag=8S49XPCB5147 type=write size=3517842 2024-04-22T19:58:43.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.852+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~HDQ3XNSPDE1B.0 size=90780 2024-04-22T19:58:43.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.852+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=HDQ3XNSPDE1B tag=WBBPAVNVXWSN type=write size=1139356 2024-04-22T19:58:43.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.853+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~ZCDK73162ZY9.0 size=445970 2024-04-22T19:58:43.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.856+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=ZCDK73162ZY9 tag=GKY44VTHT2B4 type=write size=1494546 2024-04-22T19:58:43.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.857+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=CCQC763WP18D type=del size=70208 2024-04-22T19:58:43.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.858+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=U1E5JC0ZCJ9J type=del 2024-04-22T19:58:43.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.859+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=DW9RLXMYC667 type=write size=620394 2024-04-22T19:58:43.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.860+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~P1CDNAUNHHS0.0 size=1048576 2024-04-22T19:58:43.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.861+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~P1CDNAUNHHS0.1 size=1048576 2024-04-22T19:58:43.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.862+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~P1CDNAUNHHS0.2 size=905586 2024-04-22T19:58:43.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.864+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=P1CDNAUNHHS0 tag=7OYVKLQWVMS3 type=write size=4051314 2024-04-22T19:58:43.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.866+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~GOLEX6FCKJ8E.0 size=1048576 2024-04-22T19:58:43.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.867+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~GOLEX6FCKJ8E.1 size=1048576 2024-04-22T19:58:43.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.869+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~GOLEX6FCKJ8E.2 size=577181 2024-04-22T19:58:43.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.869+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=GOLEX6FCKJ8E tag=NJRUIV3VNV8U type=write size=3722909 2024-04-22T19:58:43.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.870+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=ESBOO66MP5KQ type=del 2024-04-22T19:58:43.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.871+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=MSEO661WP43I type=write size=414594 2024-04-22T19:58:43.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.873+0000 7f5269a8e780 1 < canceled operation key=RY1O7X1CWZMK tag=TVOVYCXV63TY type=del size=1081358 2024-04-22T19:58:43.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.874+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~OPROKYJZK9LK.0 size=944110 2024-04-22T19:58:43.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.874+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=OPROKYJZK9LK tag=SK0U53ATT6CH type=write size=1992686 2024-04-22T19:58:43.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.875+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~G4BLNJGF2N12.0 size=1048576 2024-04-22T19:58:43.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.877+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~G4BLNJGF2N12.1 size=99403 2024-04-22T19:58:43.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.878+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=G4BLNJGF2N12 tag=KLQ7Z4YKXQDS type=write size=2196555 2024-04-22T19:58:43.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.880+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~T29DM26R7J4K.0 size=1048576 2024-04-22T19:58:43.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.881+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~T29DM26R7J4K.1 size=1048576 2024-04-22T19:58:43.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.882+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~T29DM26R7J4K.2 size=956416 2024-04-22T19:58:43.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.882+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=T29DM26R7J4K tag=0E463PG6JYPR type=write size=4102144 2024-04-22T19:58:43.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.883+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=4BDBHNZO52BN type=del 2024-04-22T19:58:43.885 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.885+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~O70ZGBF2AK3U.0 size=1048576 2024-04-22T19:58:43.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.886+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~O70ZGBF2AK3U.1 size=440336 2024-04-22T19:58:43.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.886+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=O70ZGBF2AK3U tag=ARQWRYW5Z98F type=write size=2537488 2024-04-22T19:58:43.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.887+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~2DKPZN19QHTF.0 size=1048576 2024-04-22T19:58:43.888 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.888+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~2DKPZN19QHTF.1 size=1048576 2024-04-22T19:58:43.888 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.889+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~2DKPZN19QHTF.2 size=60456 2024-04-22T19:58:43.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.891+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=2DKPZN19QHTF tag=9ZNDXQW0ITNV type=write size=3206184 2024-04-22T19:58:43.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.893+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~U6W70CTG0H0B.0 size=179362 2024-04-22T19:58:43.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.893+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=U6W70CTG0H0B tag=OXKRQ854LSEI type=write size=1227938 2024-04-22T19:58:43.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.894+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=5GY4BLMU4IMG type=del 2024-04-22T19:58:43.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.896+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~G13RW0UOFETK.0 size=999121 2024-04-22T19:58:43.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.896+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=G13RW0UOFETK tag=JK3DUIGU46UV type=write size=2047697 2024-04-22T19:58:43.897 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.897+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~O7G6WBDNSIVQ.0 size=388542 2024-04-22T19:58:43.899 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.899+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=O7G6WBDNSIVQ tag=6XE6WJFSB7SO type=write size=1437118 2024-04-22T19:58:43.900 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.901+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=EPIN0YWNXROB type=write size=258435 2024-04-22T19:58:43.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.902+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=2JR3O20ZQ3X6 type=del 2024-04-22T19:58:43.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.903+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=L1WXG1D7F3IU type=del size=36897 2024-04-22T19:58:43.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.904+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=UR36A50FXM2K type=del 2024-04-22T19:58:43.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.905+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=JVPQO7PW3P6S type=del size=2468072 2024-04-22T19:58:43.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.906+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~IE3CY6YCT3O7.0 size=746219 2024-04-22T19:58:43.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.908+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=IE3CY6YCT3O7 tag=FA54VR4337BV type=write size=1794795 2024-04-22T19:58:43.910 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.910+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~OIRGJ810BNVJ.0 size=514067 2024-04-22T19:58:43.910 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.910+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=OIRGJ810BNVJ tag=QT03POKRIS9G type=write size=1562643 2024-04-22T19:58:43.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.912+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=5S845J028GLC type=del 2024-04-22T19:58:43.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.913+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~M1UK78F8DTTI.0 size=123796 2024-04-22T19:58:43.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.913+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=M1UK78F8DTTI tag=MAHQZKEDB8T3 type=write size=1172372 2024-04-22T19:58:43.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.915+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~C9OKUDFAKKKE.0 size=1048576 2024-04-22T19:58:43.915 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.915+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~C9OKUDFAKKKE.1 size=677065 2024-04-22T19:58:43.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.917+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=C9OKUDFAKKKE tag=YBUNXA3AAAO2 type=write size=2774217 2024-04-22T19:58:43.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.919+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~3RUB1M4QV39K.0 size=999697 2024-04-22T19:58:43.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.919+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=3RUB1M4QV39K tag=72A3Y43DSFHY type=write size=2048273 2024-04-22T19:58:43.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.920+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~HDQ3XNSPDE1B.0 size=90780 2024-04-22T19:58:43.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.922+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=HDQ3XNSPDE1B tag=WBBPAVNVXWSN type=write size=1139356 2024-04-22T19:58:43.923 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.923+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=3IZTDA1WB1YV type=del size=2363609 2024-04-22T19:58:43.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.925+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=CCQC763WP18D type=del 2024-04-22T19:58:43.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.926+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=SNA565LJORE1 type=del size=2326716 2024-04-22T19:58:43.928 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.928+0000 7f5269a8e780 1 < suggested update operation key=637M7AP8X9E9 tag=DW9RLXMYC667 type=write size=620394 2024-04-22T19:58:43.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.930+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~Y66BGTH7YFY4.0 size=1048576 2024-04-22T19:58:43.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.931+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~Y66BGTH7YFY4.1 size=1048576 2024-04-22T19:58:43.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.932+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~Y66BGTH7YFY4.2 size=527555 2024-04-22T19:58:43.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.932+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=Y66BGTH7YFY4 tag=BRTB2XZFLALJ type=write size=3673283 2024-04-22T19:58:43.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.933+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~GOLEX6FCKJ8E.0 size=1048576 2024-04-22T19:58:43.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.933+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~GOLEX6FCKJ8E.1 size=1048576 2024-04-22T19:58:43.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.934+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~GOLEX6FCKJ8E.2 size=577181 2024-04-22T19:58:43.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.936+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=GOLEX6FCKJ8E tag=NJRUIV3VNV8U type=write size=3722909 2024-04-22T19:58:43.937 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.937+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=JQTTID2WQFTY type=del size=3580425 2024-04-22T19:58:43.938 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.938+0000 7f5269a8e780 1 < canceled operation key=70MLQGJPQYPY tag=MSEO661WP43I type=write size=414594 2024-04-22T19:58:43.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.940+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~S7ERD4ER9GOI.0 size=1048576 2024-04-22T19:58:43.941 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.941+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~S7ERD4ER9GOI.1 size=1048576 2024-04-22T19:58:43.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.942+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~S7ERD4ER9GOI.2 size=497258 2024-04-22T19:58:43.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.942+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=S7ERD4ER9GOI tag=Z7U2IXOQIMQ9 type=write size=3642986 2024-04-22T19:58:43.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.943+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~OPROKYJZK9LK.0 size=944110 2024-04-22T19:58:43.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.945+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=OPROKYJZK9LK tag=SK0U53ATT6CH type=write size=1992686 2024-04-22T19:58:43.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.946+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=8BVHJRV5RFXN type=write size=824628 2024-04-22T19:58:43.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.948+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~T29DM26R7J4K.0 size=1048576 2024-04-22T19:58:43.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.949+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~T29DM26R7J4K.1 size=1048576 2024-04-22T19:58:43.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.950+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~T29DM26R7J4K.2 size=956416 2024-04-22T19:58:43.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.952+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=T29DM26R7J4K tag=0E463PG6JYPR type=write size=4102144 2024-04-22T19:58:43.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.954+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~6LVPI4EZI6PZ.0 size=1048576 2024-04-22T19:58:43.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.955+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~6LVPI4EZI6PZ.1 size=900803 2024-04-22T19:58:43.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.955+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=6LVPI4EZI6PZ tag=3CNSJFCCU43T type=write size=2997955 2024-04-22T19:58:43.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.957+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~O70ZGBF2AK3U.0 size=1048576 2024-04-22T19:58:43.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.958+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~O70ZGBF2AK3U.1 size=440336 2024-04-22T19:58:43.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.960+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=O70ZGBF2AK3U tag=ARQWRYW5Z98F type=write size=2537488 2024-04-22T19:58:43.962 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.962+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~KXBFHERZG0BJ.0 size=1048576 2024-04-22T19:58:43.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.963+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~KXBFHERZG0BJ.1 size=286011 2024-04-22T19:58:43.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.963+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=KXBFHERZG0BJ tag=VWDZUYSJ7MRV type=write size=2383163 2024-04-22T19:58:43.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.965+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~U6W70CTG0H0B.0 size=179362 2024-04-22T19:58:43.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.966+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=U6W70CTG0H0B tag=OXKRQ854LSEI type=write size=1227938 2024-04-22T19:58:43.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.968+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=G4HPW1HB0218 type=del size=2633615 2024-04-22T19:58:43.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.969+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~G13RW0UOFETK.0 size=999121 2024-04-22T19:58:43.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.971+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=G13RW0UOFETK tag=JK3DUIGU46UV type=write size=2047697 2024-04-22T19:58:43.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.973+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~L7S2FKU8HMJ6.0 size=1048576 2024-04-22T19:58:43.974 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.974+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~L7S2FKU8HMJ6.1 size=666020 2024-04-22T19:58:43.974 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.974+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=L7S2FKU8HMJ6 tag=3Y29JWZCC0A5 type=write size=2763172 2024-04-22T19:58:43.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.975+0000 7f5269a8e780 1 < completed write operation key=M9M4SEEEZSFV tag=EPIN0YWNXROB type=write size=258435 2024-04-22T19:58:43.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.977+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~3YLBA7B9M21I.0 size=710878 2024-04-22T19:58:43.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.977+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=3YLBA7B9M21I tag=X7RK15ZEDJ3G type=write size=1759454 2024-04-22T19:58:43.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.979+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=L1WXG1D7F3IU type=del 2024-04-22T19:58:43.979 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.980+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=2LRRAXZ68WJX type=write size=269191 2024-04-22T19:58:43.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.981+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=JVPQO7PW3P6S type=del 2024-04-22T19:58:43.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.982+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=9NP6S0NX42QW type=write size=214880 2024-04-22T19:58:43.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.983+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~OIRGJ810BNVJ.0 size=514067 2024-04-22T19:58:43.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.985+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=OIRGJ810BNVJ tag=QT03POKRIS9G type=write size=1562643 2024-04-22T19:58:43.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.989+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~TVAW6Z29VCEO.0 size=753976 2024-04-22T19:58:43.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.989+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=TVAW6Z29VCEO tag=IE6XZTOSWX3N type=write size=1802552 2024-04-22T19:58:43.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.990+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~M1UK78F8DTTI.0 size=123796 2024-04-22T19:58:43.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.992+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=M1UK78F8DTTI tag=MAHQZKEDB8T3 type=write size=1172372 2024-04-22T19:58:43.994 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.994+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~PYDA2S777YM7.0 size=1048576 2024-04-22T19:58:43.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.995+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~PYDA2S777YM7.1 size=1048576 2024-04-22T19:58:43.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.996+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~PYDA2S777YM7.2 size=180786 2024-04-22T19:58:43.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.996+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=PYDA2S777YM7 tag=F3FQU9W0GCA5 type=write size=3326514 2024-04-22T19:58:43.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.997+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~3RUB1M4QV39K.0 size=999697 2024-04-22T19:58:43.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:43.999+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=3RUB1M4QV39K tag=72A3Y43DSFHY type=write size=2048273 2024-04-22T19:58:44.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.001+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~ULQWF0AJMTSZ.0 size=1048576 2024-04-22T19:58:44.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.002+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~ULQWF0AJMTSZ.1 size=428305 2024-04-22T19:58:44.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.002+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=ULQWF0AJMTSZ tag=J7T9Q2ZD5NDF type=write size=2525457 2024-04-22T19:58:44.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.003+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=3IZTDA1WB1YV type=del 2024-04-22T19:58:44.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.004+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=H5ZVECFNF0Q8 type=del size=2169453 2024-04-22T19:58:44.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.006+0000 7f5269a8e780 1 < suggested remove operation key=70MLQGJPQYPY tag=SNA565LJORE1 type=del 2024-04-22T19:58:44.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.008+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~SFD5EEI6Z3DU.0 size=557087 2024-04-22T19:58:44.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.008+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=SFD5EEI6Z3DU tag=3EFBMOPEKV2T type=write size=1605663 2024-04-22T19:58:44.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.009+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~Y66BGTH7YFY4.0 size=1048576 2024-04-22T19:58:44.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.010+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~Y66BGTH7YFY4.1 size=1048576 2024-04-22T19:58:44.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.011+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~Y66BGTH7YFY4.2 size=527555 2024-04-22T19:58:44.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.013+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=Y66BGTH7YFY4 tag=BRTB2XZFLALJ type=write size=3673283 2024-04-22T19:58:44.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.014+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=4KREK4UJ9JA6 type=del size=344852 2024-04-22T19:58:44.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.016+0000 7f5269a8e780 1 < suggested update operation key=2YFD2U6HVNE7 tag=JQTTID2WQFTY type=del size=3580425 2024-04-22T19:58:44.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.018+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~BNUS2SGUSB7U.0 size=1048576 2024-04-22T19:58:44.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.019+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~BNUS2SGUSB7U.1 size=129864 2024-04-22T19:58:44.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.019+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=BNUS2SGUSB7U tag=TZTZ2H20GSLD type=write size=2227016 2024-04-22T19:58:44.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.020+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~S7ERD4ER9GOI.0 size=1048576 2024-04-22T19:58:44.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.021+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~S7ERD4ER9GOI.1 size=1048576 2024-04-22T19:58:44.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.022+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~S7ERD4ER9GOI.2 size=497258 2024-04-22T19:58:44.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.024+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=S7ERD4ER9GOI tag=Z7U2IXOQIMQ9 type=write size=3642986 2024-04-22T19:58:44.025 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.025+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=QCA3GI27JVXY type=write size=691068 2024-04-22T19:58:44.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.026+0000 7f5269a8e780 1 < completed write operation key=RY1O7X1CWZMK tag=8BVHJRV5RFXN type=write size=824628 2024-04-22T19:58:44.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.027+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=83NE493E6A6J type=del size=1990496 2024-04-22T19:58:44.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.028+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~6LVPI4EZI6PZ.0 size=1048576 2024-04-22T19:58:44.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.029+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~6LVPI4EZI6PZ.1 size=900803 2024-04-22T19:58:44.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.031+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=6LVPI4EZI6PZ tag=3CNSJFCCU43T type=write size=2997955 2024-04-22T19:58:44.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.033+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~AK9GXM3C2U61.0 size=1048576 2024-04-22T19:58:44.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.034+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~AK9GXM3C2U61.1 size=975555 2024-04-22T19:58:44.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.034+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=AK9GXM3C2U61 tag=B0YJ8WFJVENE type=write size=3072707 2024-04-22T19:58:44.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.035+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~KXBFHERZG0BJ.0 size=1048576 2024-04-22T19:58:44.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.036+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~KXBFHERZG0BJ.1 size=286011 2024-04-22T19:58:44.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.038+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=KXBFHERZG0BJ tag=VWDZUYSJ7MRV type=write size=2383163 2024-04-22T19:58:44.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.039+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=ON3M6JJ02QFL type=write size=782789 2024-04-22T19:58:44.040 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.040+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=G4HPW1HB0218 type=del 2024-04-22T19:58:44.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.041+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=Q04XAPX64BO1 type=del size=1316605 2024-04-22T19:58:44.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.043+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~L7S2FKU8HMJ6.0 size=1048576 2024-04-22T19:58:44.043 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.044+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~L7S2FKU8HMJ6.1 size=666020 2024-04-22T19:58:44.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.045+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=L7S2FKU8HMJ6 tag=3Y29JWZCC0A5 type=write size=2763172 2024-04-22T19:58:44.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.047+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=AIO7IDUJNIHO type=write size=895791 2024-04-22T19:58:44.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.048+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~3YLBA7B9M21I.0 size=710878 2024-04-22T19:58:44.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.050+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=3YLBA7B9M21I tag=X7RK15ZEDJ3G type=write size=1759454 2024-04-22T19:58:44.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.051+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=UD540ZTML7E9 type=del size=1792387 2024-04-22T19:58:44.054 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.054+0000 7f5269a8e780 1 < suggested remove operation key=M9M4SEEEZSFV tag=2LRRAXZ68WJX type=write 2024-04-22T19:58:44.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.056+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~WDCR4JGLZZ5S.0 size=1048576 2024-04-22T19:58:44.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.057+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~WDCR4JGLZZ5S.1 size=985661 2024-04-22T19:58:44.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.057+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=WDCR4JGLZZ5S tag=SUQ2VDKF44VW type=write size=3082813 2024-04-22T19:58:44.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.058+0000 7f5269a8e780 1 < canceled operation key=Y5HR8XW4L0SP tag=9NP6S0NX42QW type=write size=214880 2024-04-22T19:58:44.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.059+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=D2GBT62H0ON4 type=del size=1262540 2024-04-22T19:58:44.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.060+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~TVAW6Z29VCEO.0 size=753976 2024-04-22T19:58:44.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.062+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=TVAW6Z29VCEO tag=IE6XZTOSWX3N type=write size=1802552 2024-04-22T19:58:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.064+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=XG782YTPDK1V type=del size=1129284 2024-04-22T19:58:44.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.066+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~PYDA2S777YM7.0 size=1048576 2024-04-22T19:58:44.066 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.067+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~PYDA2S777YM7.1 size=1048576 2024-04-22T19:58:44.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.068+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~PYDA2S777YM7.2 size=180786 2024-04-22T19:58:44.069 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.070+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=PYDA2S777YM7 tag=F3FQU9W0GCA5 type=write size=3326514 2024-04-22T19:58:44.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.072+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~2NR4YJ1XX2P9.0 size=429111 2024-04-22T19:58:44.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.072+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=2NR4YJ1XX2P9 tag=7XLDB8W6ER79 type=write size=1477687 2024-04-22T19:58:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.073+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~ULQWF0AJMTSZ.0 size=1048576 2024-04-22T19:58:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.073+0000 7f5269a8e780 1 < canceled part key=_multipart_B0Y7ZTPC0975.2~ULQWF0AJMTSZ.1 size=428305 2024-04-22T19:58:44.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.075+0000 7f5269a8e780 1 < canceled multipart upload key=B0Y7ZTPC0975 upload=ULQWF0AJMTSZ tag=J7T9Q2ZD5NDF type=write size=2525457 2024-04-22T19:58:44.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.076+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=IER4CAD9TORU type=del size=1110830 2024-04-22T19:58:44.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.077+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=H5ZVECFNF0Q8 type=del 2024-04-22T19:58:44.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.079+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~M8VJ30LSABM5.0 size=1048576 2024-04-22T19:58:44.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.080+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~M8VJ30LSABM5.1 size=1048576 2024-04-22T19:58:44.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.081+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~M8VJ30LSABM5.2 size=259557 2024-04-22T19:58:44.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.081+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=M8VJ30LSABM5 tag=36Z0Z0Y5X4YR type=write size=3405285 2024-04-22T19:58:44.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.082+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~SFD5EEI6Z3DU.0 size=557087 2024-04-22T19:58:44.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.084+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=SFD5EEI6Z3DU tag=3EFBMOPEKV2T type=write size=1605663 2024-04-22T19:58:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.086+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~LTIB9DQ6VXHN.0 size=1048576 2024-04-22T19:58:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.086+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~LTIB9DQ6VXHN.1 size=170198 2024-04-22T19:58:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.086+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=LTIB9DQ6VXHN tag=7L6IHJ9G3C0X type=write size=2267350 2024-04-22T19:58:44.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.088+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=4KREK4UJ9JA6 type=del 2024-04-22T19:58:44.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.089+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=7DX4MUOIU6NN type=del size=1636202 2024-04-22T19:58:44.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.090+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~BNUS2SGUSB7U.0 size=1048576 2024-04-22T19:58:44.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.091+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~BNUS2SGUSB7U.1 size=129864 2024-04-22T19:58:44.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.092+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=BNUS2SGUSB7U tag=TZTZ2H20GSLD type=write size=2227016 2024-04-22T19:58:44.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.094+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~CTNLY8ZWWB7Y.0 size=1048576 2024-04-22T19:58:44.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.095+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~CTNLY8ZWWB7Y.1 size=141012 2024-04-22T19:58:44.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.095+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=CTNLY8ZWWB7Y tag=XEP5EVUV18BR type=write size=2238164 2024-04-22T19:58:44.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.096+0000 7f5269a8e780 1 < canceled operation key=I91AD8FJ3P4T tag=QCA3GI27JVXY type=write size=691068 2024-04-22T19:58:44.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.097+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=2GNUCA7S5MXX type=del size=943019 2024-04-22T19:58:44.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.099+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=83NE493E6A6J type=del 2024-04-22T19:58:44.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.101+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~CYVTTWISEYEU.0 size=1048576 2024-04-22T19:58:44.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.102+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~CYVTTWISEYEU.1 size=942158 2024-04-22T19:58:44.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.102+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=CYVTTWISEYEU tag=CAEEDTVE93YZ type=write size=3039310 2024-04-22T19:58:44.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.102+0000 7f5269a8e780 1 < canceled part key=_multipart_KE7YGPTF1MOR.2~AK9GXM3C2U61.0 size=1048576 2024-04-22T19:58:44.103 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.103+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~AK9GXM3C2U61.1 size=975555 2024-04-22T19:58:44.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.105+0000 7f5269a8e780 1 < canceled multipart upload key=KE7YGPTF1MOR upload=AK9GXM3C2U61 tag=B0YJ8WFJVENE type=write size=3072707 2024-04-22T19:58:44.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.106+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=2BKEBSYX8IWS type=del size=1456000 2024-04-22T19:58:44.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.107+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=ON3M6JJ02QFL type=write size=782789 2024-04-22T19:58:44.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.108+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=KIO456Q2I6AP type=del size=836641 2024-04-22T19:58:44.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.110+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=Q04XAPX64BO1 type=del 2024-04-22T19:58:44.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.111+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=WSDY3U6G8S3E type=del size=1887559 2024-04-22T19:58:44.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.112+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=AIO7IDUJNIHO type=write size=895791 2024-04-22T19:58:44.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.113+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=JSSQIY8I74J5 type=write size=653432 2024-04-22T19:58:44.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.115+0000 7f5269a8e780 1 < suggested remove operation key=HV8J6YM0YNC1 tag=UD540ZTML7E9 type=del 2024-04-22T19:58:44.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.117+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~WEAY3PAX716B.0 size=1048576 2024-04-22T19:58:44.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.118+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~WEAY3PAX716B.1 size=817359 2024-04-22T19:58:44.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.118+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=WEAY3PAX716B tag=JU1JLUU8UP03 type=write size=2914511 2024-04-22T19:58:44.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.119+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~WDCR4JGLZZ5S.0 size=1048576 2024-04-22T19:58:44.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.120+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~WDCR4JGLZZ5S.1 size=985661 2024-04-22T19:58:44.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.122+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=WDCR4JGLZZ5S tag=SUQ2VDKF44VW type=write size=3082813 2024-04-22T19:58:44.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.124+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~OLULUQEYERER.0 size=1048576 2024-04-22T19:58:44.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.124+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~OLULUQEYERER.1 size=119517 2024-04-22T19:58:44.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.124+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=OLULUQEYERER tag=C6NAG1QR0O9M type=write size=2216669 2024-04-22T19:58:44.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.125+0000 7f5269a8e780 1 < canceled operation key=EHRS6DQDRXVS tag=D2GBT62H0ON4 type=del size=1262540 2024-04-22T19:58:44.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.127+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=QB57L3UAOKYC type=del size=2159037 2024-04-22T19:58:44.128 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.128+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=XG782YTPDK1V type=del 2024-04-22T19:58:44.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.129+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=0JLNQGLKLFO6 type=del size=2248204 2024-04-22T19:58:44.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.130+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~2NR4YJ1XX2P9.0 size=429111 2024-04-22T19:58:44.132 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.132+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=2NR4YJ1XX2P9 tag=7XLDB8W6ER79 type=write size=1477687 2024-04-22T19:58:44.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.133+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=OH2S9BB4SQRU type=del size=3491454 2024-04-22T19:58:44.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.135+0000 7f5269a8e780 1 < suggested remove operation key=Y5HR8XW4L0SP tag=IER4CAD9TORU type=del 2024-04-22T19:58:44.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.136+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=U8E3U51XFXWD type=write size=401849 2024-04-22T19:58:44.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.138+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~M8VJ30LSABM5.0 size=1048576 2024-04-22T19:58:44.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.139+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~M8VJ30LSABM5.1 size=1048576 2024-04-22T19:58:44.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.139+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~M8VJ30LSABM5.2 size=259557 2024-04-22T19:58:44.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.141+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=M8VJ30LSABM5 tag=36Z0Z0Y5X4YR type=write size=3405285 2024-04-22T19:58:44.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.144+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~UWFFTILBNP1S.0 size=545591 2024-04-22T19:58:44.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.144+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=UWFFTILBNP1S tag=XJ8EXN12SSMW type=write size=1594167 2024-04-22T19:58:44.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.145+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~LTIB9DQ6VXHN.0 size=1048576 2024-04-22T19:58:44.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.146+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~LTIB9DQ6VXHN.1 size=170198 2024-04-22T19:58:44.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.147+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=LTIB9DQ6VXHN tag=7L6IHJ9G3C0X type=write size=2267350 2024-04-22T19:58:44.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.149+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=SMXWUEHE2III type=del size=2886501 2024-04-22T19:58:44.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.150+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=7DX4MUOIU6NN type=del 2024-04-22T19:58:44.151 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.151+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=I79FP63Z55Y1 type=del size=3187880 2024-04-22T19:58:44.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.152+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~CTNLY8ZWWB7Y.0 size=1048576 2024-04-22T19:58:44.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.153+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~CTNLY8ZWWB7Y.1 size=141012 2024-04-22T19:58:44.155 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.155+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=CTNLY8ZWWB7Y tag=XEP5EVUV18BR type=write size=2238164 2024-04-22T19:58:44.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.157+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~8Z8YF94HH206.0 size=612690 2024-04-22T19:58:44.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.157+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=8Z8YF94HH206 tag=XO8JARGYI6ZN type=write size=1661266 2024-04-22T19:58:44.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.158+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=2GNUCA7S5MXX type=del 2024-04-22T19:58:44.160 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.160+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=2EMZR8530WXO type=del size=3912376 2024-04-22T19:58:44.161 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.161+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~CYVTTWISEYEU.0 size=1048576 2024-04-22T19:58:44.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.162+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~CYVTTWISEYEU.1 size=942158 2024-04-22T19:58:44.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.164+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=CYVTTWISEYEU tag=CAEEDTVE93YZ type=write size=3039310 2024-04-22T19:58:44.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.166+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~U4DU2O7K1IOX.0 size=1048576 2024-04-22T19:58:44.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.167+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~U4DU2O7K1IOX.1 size=1048576 2024-04-22T19:58:44.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.168+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~U4DU2O7K1IOX.2 size=808053 2024-04-22T19:58:44.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.168+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=U4DU2O7K1IOX tag=25LQTVSNUV9Q type=write size=3953781 2024-04-22T19:58:44.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.170+0000 7f5269a8e780 1 < suggested remove operation key=B0Y7ZTPC0975 tag=2BKEBSYX8IWS type=del 2024-04-22T19:58:44.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.173+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~LNV9RJJ2JX5P.0 size=694040 2024-04-22T19:58:44.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.173+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=LNV9RJJ2JX5P tag=0ZYTZ60RJP3V type=write size=1742616 2024-04-22T19:58:44.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.175+0000 7f5269a8e780 1 < suggested update operation key=GP1C5ULUU143 tag=KIO456Q2I6AP type=del size=836641 2024-04-22T19:58:44.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.177+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~USALX40H9Q4L.0 size=1048576 2024-04-22T19:58:44.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.178+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~USALX40H9Q4L.1 size=1048576 2024-04-22T19:58:44.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.179+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~USALX40H9Q4L.2 size=601829 2024-04-22T19:58:44.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.179+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=USALX40H9Q4L tag=ZO3XA9YKWQGO type=write size=3747557 2024-04-22T19:58:44.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.180+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=WSDY3U6G8S3E type=del 2024-04-22T19:58:44.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.181+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=80WWEX9MYQQE type=del size=1511093 2024-04-22T19:58:44.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.183+0000 7f5269a8e780 1 < suggested remove operation key=KV5DYQ49ZKCX tag=JSSQIY8I74J5 type=write 2024-04-22T19:58:44.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.185+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~BGU25INFZAFA.0 size=1048576 2024-04-22T19:58:44.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.186+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~BGU25INFZAFA.1 size=403678 2024-04-22T19:58:44.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.186+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=BGU25INFZAFA tag=B9N94CYG1OY4 type=write size=2500830 2024-04-22T19:58:44.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.187+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~WEAY3PAX716B.0 size=1048576 2024-04-22T19:58:44.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.188+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~WEAY3PAX716B.1 size=817359 2024-04-22T19:58:44.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.190+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=WEAY3PAX716B tag=JU1JLUU8UP03 type=write size=2914511 2024-04-22T19:58:44.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.191+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=IW0PNDEMHDY2 type=del size=849787 2024-04-22T19:58:44.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.192+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~OLULUQEYERER.0 size=1048576 2024-04-22T19:58:44.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.193+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~OLULUQEYERER.1 size=119517 2024-04-22T19:58:44.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.195+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=OLULUQEYERER tag=C6NAG1QR0O9M type=write size=2216669 2024-04-22T19:58:44.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.196+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=93LRTA4PTYF2 type=del size=588765 2024-04-22T19:58:44.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.197+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=QB57L3UAOKYC type=del 2024-04-22T19:58:44.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.198+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=I157DGYD93HF type=del size=3229687 2024-04-22T19:58:44.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.199+0000 7f5269a8e780 -1 < failed to complete operation key=Y5HR8XW4L0SP tag=0JLNQGLKLFO6 type=del size=2248204: (22) Invalid argument 2024-04-22T19:58:44.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.201+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=4T9RXTZKR5VR type=del size=1971817 2024-04-22T19:58:44.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.202+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=OH2S9BB4SQRU type=del 2024-04-22T19:58:44.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.204+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~X0K00OZE1I0K.0 size=1048576 2024-04-22T19:58:44.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.205+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~X0K00OZE1I0K.1 size=1048576 2024-04-22T19:58:44.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.206+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~X0K00OZE1I0K.2 size=374957 2024-04-22T19:58:44.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.206+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=X0K00OZE1I0K tag=OOX6XSPUVMN5 type=write size=3520685 2024-04-22T19:58:44.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.208+0000 7f5269a8e780 1 < suggested update operation key=HV8J6YM0YNC1 tag=U8E3U51XFXWD type=write size=401849 2024-04-22T19:58:44.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.210+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~IDXAI6JEQKLX.0 size=1048576 2024-04-22T19:58:44.211 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.211+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~IDXAI6JEQKLX.1 size=450241 2024-04-22T19:58:44.211 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.211+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=IDXAI6JEQKLX tag=X20RZCX6M4J3 type=write size=2547393 2024-04-22T19:58:44.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.212+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~UWFFTILBNP1S.0 size=545591 2024-04-22T19:58:44.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.215+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=UWFFTILBNP1S tag=XJ8EXN12SSMW type=write size=1594167 2024-04-22T19:58:44.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.217+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=MVN2HKDBFRCJ type=del size=2294463 2024-04-22T19:58:44.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.219+0000 7f5269a8e780 1 < suggested update operation key=A853MOZPT9BG tag=SMXWUEHE2III type=del size=2886501 2024-04-22T19:58:44.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.221+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~UQ4N5K1QLZUU.0 size=894673 2024-04-22T19:58:44.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.221+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=UQ4N5K1QLZUU tag=IZ820CSHSHTA type=write size=1943249 2024-04-22T19:58:44.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.222+0000 7f5269a8e780 -1 < failed to complete operation key=HV8J6YM0YNC1 tag=I79FP63Z55Y1 type=del size=3187880: (22) Invalid argument 2024-04-22T19:58:44.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.223+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=TTUIEB9SWGM3 type=del size=4106593 2024-04-22T19:58:44.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.224+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~8Z8YF94HH206.0 size=612690 2024-04-22T19:58:44.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.226+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=8Z8YF94HH206 tag=XO8JARGYI6ZN type=write size=1661266 2024-04-22T19:58:44.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.227+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=CDA3QBK25CBO type=del size=4084763 2024-04-22T19:58:44.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.228+0000 7f5269a8e780 1 < canceled operation key=Q1UKF4UMJ2HO tag=2EMZR8530WXO type=del size=3912376 2024-04-22T19:58:44.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.229+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=GOA006KTQAFB type=write size=275265 2024-04-22T19:58:44.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.230+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~U4DU2O7K1IOX.0 size=1048576 2024-04-22T19:58:44.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.231+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~U4DU2O7K1IOX.1 size=1048576 2024-04-22T19:58:44.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.232+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~U4DU2O7K1IOX.2 size=808053 2024-04-22T19:58:44.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.234+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=U4DU2O7K1IOX tag=25LQTVSNUV9Q type=write size=3953781 2024-04-22T19:58:44.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.236+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~KXL3R3HD538K.0 size=436592 2024-04-22T19:58:44.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.236+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=KXL3R3HD538K tag=IWWK9K00EJTX type=write size=1485168 2024-04-22T19:58:44.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.237+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~LNV9RJJ2JX5P.0 size=694040 2024-04-22T19:58:44.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.239+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=LNV9RJJ2JX5P tag=0ZYTZ60RJP3V type=write size=1742616 2024-04-22T19:58:44.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.240+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=WMUSY9J66CUM type=del size=3892944 2024-04-22T19:58:44.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.241+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~USALX40H9Q4L.0 size=1048576 2024-04-22T19:58:44.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.242+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~USALX40H9Q4L.1 size=1048576 2024-04-22T19:58:44.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.243+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~USALX40H9Q4L.2 size=601829 2024-04-22T19:58:44.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.245+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=USALX40H9Q4L tag=ZO3XA9YKWQGO type=write size=3747557 2024-04-22T19:58:44.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.246+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=LN19TYVRGQPY type=del size=2019717 2024-04-22T19:58:44.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.247+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=80WWEX9MYQQE type=del 2024-04-22T19:58:44.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.249+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~E5Q7QUMFNEBX.0 size=1048576 2024-04-22T19:58:44.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.250+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~E5Q7QUMFNEBX.1 size=127705 2024-04-22T19:58:44.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.250+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=E5Q7QUMFNEBX tag=RZOF80K8HRU0 type=write size=2224857 2024-04-22T19:58:44.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.251+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~BGU25INFZAFA.0 size=1048576 2024-04-22T19:58:44.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.252+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~BGU25INFZAFA.1 size=403678 2024-04-22T19:58:44.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.254+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=BGU25INFZAFA tag=B9N94CYG1OY4 type=write size=2500830 2024-04-22T19:58:44.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.255+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=HXHZLLD3Y1CB type=del size=1522799 2024-04-22T19:58:44.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.257+0000 7f5269a8e780 1 < canceled operation key=ZC4ZOHUJ3HQD tag=IW0PNDEMHDY2 type=del size=849787 2024-04-22T19:58:44.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.258+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=3D6Y8NKMQQJL type=del size=2582085 2024-04-22T19:58:44.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.259+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=93LRTA4PTYF2 type=del 2024-04-22T19:58:44.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.260+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=VTGG7P8G8I0V type=del size=3450010 2024-04-22T19:58:44.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.261+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=I157DGYD93HF type=del 2024-04-22T19:58:44.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.262+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=2SD0QZRBDUPF type=del size=1520124 2024-04-22T19:58:44.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.264+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=4T9RXTZKR5VR type=del 2024-04-22T19:58:44.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.266+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~EVESKRRK634I.0 size=1048576 2024-04-22T19:58:44.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.267+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~EVESKRRK634I.1 size=102154 2024-04-22T19:58:44.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.267+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=EVESKRRK634I tag=P7H3GX4KBAWF type=write size=2199306 2024-04-22T19:58:44.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.268+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~X0K00OZE1I0K.0 size=1048576 2024-04-22T19:58:44.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.269+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~X0K00OZE1I0K.1 size=1048576 2024-04-22T19:58:44.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.269+0000 7f5269a8e780 1 < canceled part key=_multipart_VPAVX5FXYWZO.2~X0K00OZE1I0K.2 size=374957 2024-04-22T19:58:44.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.272+0000 7f5269a8e780 1 < canceled multipart upload key=VPAVX5FXYWZO upload=X0K00OZE1I0K tag=OOX6XSPUVMN5 type=write size=3520685 2024-04-22T19:58:44.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.273+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=XA84O1KBDEEL type=del size=1972332 2024-04-22T19:58:44.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.274+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~IDXAI6JEQKLX.0 size=1048576 2024-04-22T19:58:44.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.276+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~IDXAI6JEQKLX.1 size=450241 2024-04-22T19:58:44.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.278+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=IDXAI6JEQKLX tag=X20RZCX6M4J3 type=write size=2547393 2024-04-22T19:58:44.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.279+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=YA9AC324NXEP type=del size=2536759 2024-04-22T19:58:44.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.281+0000 7f5269a8e780 1 < suggested update operation key=Y5HR8XW4L0SP tag=MVN2HKDBFRCJ type=del size=2294463 2024-04-22T19:58:44.282 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.282+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=O9T9IQ6JL6LM type=del size=349557 2024-04-22T19:58:44.283 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.283+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~UQ4N5K1QLZUU.0 size=894673 2024-04-22T19:58:44.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.285+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=UQ4N5K1QLZUU tag=IZ820CSHSHTA type=write size=1943249 2024-04-22T19:58:44.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.287+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~IRP5JZUQN8U0.0 size=326858 2024-04-22T19:58:44.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.287+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=IRP5JZUQN8U0 tag=U4I2CHZEWXPC type=write size=1375434 2024-04-22T19:58:44.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.288+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=TTUIEB9SWGM3 type=del 2024-04-22T19:58:44.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.289+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=JO90XIUNWJ3H type=del size=3094662 2024-04-22T19:58:44.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.290+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=CDA3QBK25CBO type=del 2024-04-22T19:58:44.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.291+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=IUPTVAFFQD7P type=del size=1984998 2024-04-22T19:58:44.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.293+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=GOA006KTQAFB type=write size=275265 2024-04-22T19:58:44.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.295+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~EFVJEA2QVEKO.0 size=1048576 2024-04-22T19:58:44.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.295+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~EFVJEA2QVEKO.1 size=66071 2024-04-22T19:58:44.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.295+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=EFVJEA2QVEKO tag=8XW32TGVBBYI type=write size=2163223 2024-04-22T19:58:44.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.297+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~KXL3R3HD538K.0 size=436592 2024-04-22T19:58:44.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.299+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=KXL3R3HD538K tag=IWWK9K00EJTX type=write size=1485168 2024-04-22T19:58:44.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.300+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=HPLX3X4GAW68 type=del size=954143 2024-04-22T19:58:44.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.301+0000 7f5269a8e780 -1 < failed to complete operation key=Y5HR8XW4L0SP tag=WMUSY9J66CUM type=del size=3892944: (22) Invalid argument 2024-04-22T19:58:44.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.302+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=9DJKRPK6CFFM type=write size=942177 2024-04-22T19:58:44.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.304+0000 7f5269a8e780 1 < canceled operation key=GX6NILQG878S tag=LN19TYVRGQPY type=del size=2019717 2024-04-22T19:58:44.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.305+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=15V5O84WLOJQ type=del size=1171085 2024-04-22T19:58:44.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.306+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~E5Q7QUMFNEBX.0 size=1048576 2024-04-22T19:58:44.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.307+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~E5Q7QUMFNEBX.1 size=127705 2024-04-22T19:58:44.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.308+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=E5Q7QUMFNEBX tag=RZOF80K8HRU0 type=write size=2224857 2024-04-22T19:58:44.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.310+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=OJ2VC3XX5WBI type=write size=222598 2024-04-22T19:58:44.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.312+0000 7f5269a8e780 1 < suggested remove operation key=GRSL48RUEBX8 tag=HXHZLLD3Y1CB type=del 2024-04-22T19:58:44.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.313+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=T19ROHS4ZNLT type=write size=732134 2024-04-22T19:58:44.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.314+0000 7f5269a8e780 1 < canceled operation key=I91AD8FJ3P4T tag=3D6Y8NKMQQJL type=del size=2582085 2024-04-22T19:58:44.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.315+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=40VRZSGOKTH2 type=del size=3262422 2024-04-22T19:58:44.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.316+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=VTGG7P8G8I0V type=del 2024-04-22T19:58:44.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.318+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~MEDHZBJF8B6B.0 size=543679 2024-04-22T19:58:44.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.318+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=MEDHZBJF8B6B tag=QFPZIA3GPPQ0 type=write size=1592255 2024-04-22T19:58:44.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.320+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=2SD0QZRBDUPF type=del 2024-04-22T19:58:44.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.321+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=7GD0K7DXYKBH type=del size=1638528 2024-04-22T19:58:44.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.322+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~EVESKRRK634I.0 size=1048576 2024-04-22T19:58:44.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.323+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~EVESKRRK634I.1 size=102154 2024-04-22T19:58:44.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.324+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=EVESKRRK634I tag=P7H3GX4KBAWF type=write size=2199306 2024-04-22T19:58:44.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.326+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=W8PF0L4OE832 type=del size=4064121 2024-04-22T19:58:44.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.327+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=XA84O1KBDEEL type=del 2024-04-22T19:58:44.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.328+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=N9Q7FIXAER5K type=del size=2559155 2024-04-22T19:58:44.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.329+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=YA9AC324NXEP type=del 2024-04-22T19:58:44.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.331+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~D1W22J3Z5DEQ.0 size=1048576 2024-04-22T19:58:44.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.332+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~D1W22J3Z5DEQ.1 size=1048576 2024-04-22T19:58:44.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.333+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~D1W22J3Z5DEQ.2 size=734888 2024-04-22T19:58:44.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.333+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=D1W22J3Z5DEQ tag=9QTPDX9F3M22 type=write size=3880616 2024-04-22T19:58:44.334 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.334+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=O9T9IQ6JL6LM type=del 2024-04-22T19:58:44.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.335+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=MWVZ2WQRHPVW type=write size=878662 2024-04-22T19:58:44.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.335+0000 7f5269a8e780 1 < canceled part key=_multipart_GRSL48RUEBX8.2~IRP5JZUQN8U0.0 size=326858 2024-04-22T19:58:44.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.337+0000 7f5269a8e780 1 < canceled multipart upload key=GRSL48RUEBX8 upload=IRP5JZUQN8U0 tag=U4I2CHZEWXPC type=write size=1375434 2024-04-22T19:58:44.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.339+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~O5VKIX2OE9IN.0 size=1048576 2024-04-22T19:58:44.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.340+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~O5VKIX2OE9IN.1 size=693238 2024-04-22T19:58:44.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.340+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=O5VKIX2OE9IN tag=NNQF8G82H0QW type=write size=2790390 2024-04-22T19:58:44.341 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.341+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=JO90XIUNWJ3H type=del 2024-04-22T19:58:44.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.343+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~0CIDBRF9KBE3.0 size=1048576 2024-04-22T19:58:44.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.344+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~0CIDBRF9KBE3.1 size=1048576 2024-04-22T19:58:44.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.345+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~0CIDBRF9KBE3.2 size=531588 2024-04-22T19:58:44.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.345+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=0CIDBRF9KBE3 tag=4FXLIO53MG1Z type=write size=3677316 2024-04-22T19:58:44.346 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.346+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=IUPTVAFFQD7P type=del 2024-04-22T19:58:44.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.347+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=JF1MELHUC821 type=del size=1426218 2024-04-22T19:58:44.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.348+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~EFVJEA2QVEKO.0 size=1048576 2024-04-22T19:58:44.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.349+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~EFVJEA2QVEKO.1 size=66071 2024-04-22T19:58:44.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.351+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=EFVJEA2QVEKO tag=8XW32TGVBBYI type=write size=2163223 2024-04-22T19:58:44.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.353+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~HW9IC429651C.0 size=1048576 2024-04-22T19:58:44.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.354+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~HW9IC429651C.1 size=198910 2024-04-22T19:58:44.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.354+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=HW9IC429651C tag=KUO7EK2JQ946 type=write size=2296062 2024-04-22T19:58:44.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.355+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=HPLX3X4GAW68 type=del 2024-04-22T19:58:44.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.356+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=JSP8RSCRXHXM type=del size=806654 2024-04-22T19:58:44.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.357+0000 7f5269a8e780 1 < canceled operation key=HIAXPVKZ1IIO tag=9DJKRPK6CFFM type=write size=942177 2024-04-22T19:58:44.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.359+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~56XH5WM8IVDX.0 size=1048576 2024-04-22T19:58:44.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.360+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~56XH5WM8IVDX.1 size=1048576 2024-04-22T19:58:44.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.361+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~56XH5WM8IVDX.2 size=817263 2024-04-22T19:58:44.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.361+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=56XH5WM8IVDX tag=13HW2JLO3TFE type=write size=3962991 2024-04-22T19:58:44.362 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.362+0000 7f5269a8e780 1 < canceled operation key=T2BV2LUCY4E3 tag=15V5O84WLOJQ type=del size=1171085 2024-04-22T19:58:44.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.364+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~ZEN6MXN4HDGD.0 size=1021229 2024-04-22T19:58:44.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.364+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=ZEN6MXN4HDGD tag=SXJLBL0GV8QR type=write size=2069805 2024-04-22T19:58:44.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.365+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=OJ2VC3XX5WBI type=write size=222598 2024-04-22T19:58:44.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.366+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=ZROIOTHK8WBF type=del size=1870006 2024-04-22T19:58:44.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.368+0000 7f5269a8e780 1 < completed write operation key=GX6NILQG878S tag=T19ROHS4ZNLT type=write size=732134 2024-04-22T19:58:44.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.369+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=PEU75VPQTD9Y type=del size=1266922 2024-04-22T19:58:44.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.370+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=40VRZSGOKTH2 type=del 2024-04-22T19:58:44.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.371+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=0AGWBXZNUJI1 type=write size=214118 2024-04-22T19:58:44.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.373+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~MEDHZBJF8B6B.0 size=543679 2024-04-22T19:58:44.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.375+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=MEDHZBJF8B6B tag=QFPZIA3GPPQ0 type=write size=1592255 2024-04-22T19:58:44.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.376+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=CEOWVCAUYOH0 type=write size=789230 2024-04-22T19:58:44.377 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.377+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=7GD0K7DXYKBH type=del 2024-04-22T19:58:44.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.379+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~NBAE0XMFRX7P.0 size=1048576 2024-04-22T19:58:44.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.380+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~NBAE0XMFRX7P.1 size=793480 2024-04-22T19:58:44.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.380+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=NBAE0XMFRX7P tag=K8N59LGQVXCF type=write size=2890632 2024-04-22T19:58:44.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.382+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=W8PF0L4OE832 type=del 2024-04-22T19:58:44.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.383+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=4B232BRC9PGW type=del size=312713 2024-04-22T19:58:44.384 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.385+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=N9Q7FIXAER5K type=del 2024-04-22T19:58:44.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.387+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~T71EMLKMORHU.0 size=277065 2024-04-22T19:58:44.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.387+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=T71EMLKMORHU tag=AZ8HGY12H3ZE type=write size=1325641 2024-04-22T19:58:44.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.388+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~D1W22J3Z5DEQ.0 size=1048576 2024-04-22T19:58:44.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.388+0000 7f5269a8e780 1 < canceled part key=_multipart_KE7YGPTF1MOR.2~D1W22J3Z5DEQ.1 size=1048576 2024-04-22T19:58:44.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.389+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~D1W22J3Z5DEQ.2 size=734888 2024-04-22T19:58:44.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.391+0000 7f5269a8e780 1 < canceled multipart upload key=KE7YGPTF1MOR upload=D1W22J3Z5DEQ tag=9QTPDX9F3M22 type=write size=3880616 2024-04-22T19:58:44.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.393+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~Y8LNTHNITFZU.0 size=1048576 2024-04-22T19:58:44.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.393+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~Y8LNTHNITFZU.1 size=1006538 2024-04-22T19:58:44.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.393+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=Y8LNTHNITFZU tag=7OZSU14012X0 type=write size=3103690 2024-04-22T19:58:44.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.395+0000 7f5269a8e780 1 < completed write operation key=GX6NILQG878S tag=MWVZ2WQRHPVW type=write size=878662 2024-04-22T19:58:44.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.396+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=LZ426AS9XUTO type=del size=471248 2024-04-22T19:58:44.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.397+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~O5VKIX2OE9IN.0 size=1048576 2024-04-22T19:58:44.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.398+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~O5VKIX2OE9IN.1 size=693238 2024-04-22T19:58:44.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.400+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=O5VKIX2OE9IN tag=NNQF8G82H0QW type=write size=2790390 2024-04-22T19:58:44.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.402+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~LGVTLKJCZKAD.0 size=1048576 2024-04-22T19:58:44.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.403+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~LGVTLKJCZKAD.1 size=1048576 2024-04-22T19:58:44.404 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.404+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~LGVTLKJCZKAD.2 size=475543 2024-04-22T19:58:44.404 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.404+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=LGVTLKJCZKAD tag=GTUXUSGZMKJU type=write size=3621271 2024-04-22T19:58:44.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.405+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~0CIDBRF9KBE3.0 size=1048576 2024-04-22T19:58:44.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.406+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~0CIDBRF9KBE3.1 size=1048576 2024-04-22T19:58:44.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.406+0000 7f5269a8e780 1 < canceled part key=_multipart_8HFSDSNZONIN.2~0CIDBRF9KBE3.2 size=531588 2024-04-22T19:58:44.408 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.408+0000 7f5269a8e780 1 < canceled multipart upload key=8HFSDSNZONIN upload=0CIDBRF9KBE3 tag=4FXLIO53MG1Z type=write size=3677316 2024-04-22T19:58:44.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.409+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=BYBARCPGTO94 type=del size=3447638 2024-04-22T19:58:44.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.410+0000 7f5269a8e780 1 < canceled operation key=KV5DYQ49ZKCX tag=JF1MELHUC821 type=del size=1426218 2024-04-22T19:58:44.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.412+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~ZP0NE4FFQQEB.0 size=1048576 2024-04-22T19:58:44.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.413+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~ZP0NE4FFQQEB.1 size=1048576 2024-04-22T19:58:44.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.414+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~ZP0NE4FFQQEB.2 size=600210 2024-04-22T19:58:44.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.414+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=ZP0NE4FFQQEB tag=V5FS5SENFU0L type=write size=3745938 2024-04-22T19:58:44.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.414+0000 7f5269a8e780 1 < canceled part key=_multipart_637M7AP8X9E9.2~HW9IC429651C.0 size=1048576 2024-04-22T19:58:44.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.414+0000 7f5269a8e780 1 < canceled part key=_multipart_637M7AP8X9E9.2~HW9IC429651C.1 size=198910 2024-04-22T19:58:44.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.416+0000 7f5269a8e780 1 < canceled multipart upload key=637M7AP8X9E9 upload=HW9IC429651C tag=KUO7EK2JQ946 type=write size=2296062 2024-04-22T19:58:44.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.417+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=6IJKZQ6TJANE type=del size=2615168 2024-04-22T19:58:44.418 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.418+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=JSP8RSCRXHXM type=del 2024-04-22T19:58:44.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.420+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=86N96BGFCM1F type=write size=162786 2024-04-22T19:58:44.420 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.420+0000 7f5269a8e780 1 < canceled part key=_multipart_70MLQGJPQYPY.2~56XH5WM8IVDX.0 size=1048576 2024-04-22T19:58:44.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.421+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~56XH5WM8IVDX.1 size=1048576 2024-04-22T19:58:44.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.422+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~56XH5WM8IVDX.2 size=817263 2024-04-22T19:58:44.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.423+0000 7f5269a8e780 1 < canceled multipart upload key=70MLQGJPQYPY upload=56XH5WM8IVDX tag=13HW2JLO3TFE type=write size=3962991 2024-04-22T19:58:44.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.424+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=P1J63R21151K type=write size=86210 2024-04-22T19:58:44.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.426+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~ZEN6MXN4HDGD.0 size=1021229 2024-04-22T19:58:44.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.427+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=ZEN6MXN4HDGD tag=SXJLBL0GV8QR type=write size=2069805 2024-04-22T19:58:44.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.429+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=JVFHVJUEYB4D type=del size=3600138 2024-04-22T19:58:44.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.430+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=ZROIOTHK8WBF type=del 2024-04-22T19:58:44.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.431+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=ZR9UT3BHBV93 type=del size=132863 2024-04-22T19:58:44.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.432+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=PEU75VPQTD9Y type=del 2024-04-22T19:58:44.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.434+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~IVAMQOWLMDBN.0 size=59296 2024-04-22T19:58:44.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.434+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=IVAMQOWLMDBN tag=HXXHMEBG15HM type=write size=1107872 2024-04-22T19:58:44.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.435+0000 7f5269a8e780 1 < completed write operation key=IBH6W1NINUHC tag=0AGWBXZNUJI1 type=write size=214118 2024-04-22T19:58:44.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.437+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~SJ49ALO0597O.0 size=1048576 2024-04-22T19:58:44.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.438+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~SJ49ALO0597O.1 size=117791 2024-04-22T19:58:44.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.438+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=SJ49ALO0597O tag=74V3P4I3QXTF type=write size=2214943 2024-04-22T19:58:44.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.440+0000 7f5269a8e780 1 < suggested update operation key=5UBYMQT7ZWKX tag=CEOWVCAUYOH0 type=write size=789230 2024-04-22T19:58:44.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.441+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=SA6DUT2DT25P type=del size=597078 2024-04-22T19:58:44.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.442+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~NBAE0XMFRX7P.0 size=1048576 2024-04-22T19:58:44.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.443+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~NBAE0XMFRX7P.1 size=793480 2024-04-22T19:58:44.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.445+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=NBAE0XMFRX7P tag=K8N59LGQVXCF type=write size=2890632 2024-04-22T19:58:44.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.447+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~4QA6GQABSD26.0 size=1048576 2024-04-22T19:58:44.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.448+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~4QA6GQABSD26.1 size=1048576 2024-04-22T19:58:44.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.449+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~4QA6GQABSD26.2 size=542571 2024-04-22T19:58:44.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.449+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=4QA6GQABSD26 tag=13RDCG72PQH8 type=write size=3688299 2024-04-22T19:58:44.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.450+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=4B232BRC9PGW type=del 2024-04-22T19:58:44.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.452+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~QOCFGQUFWPIL.0 size=914684 2024-04-22T19:58:44.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.452+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=QOCFGQUFWPIL tag=IQU38WQUJORN type=write size=1963260 2024-04-22T19:58:44.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.454+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~T71EMLKMORHU.0 size=277065 2024-04-22T19:58:44.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.456+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=T71EMLKMORHU tag=AZ8HGY12H3ZE type=write size=1325641 2024-04-22T19:58:44.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.457+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=A54R4O1BCAA1 type=del size=1957471 2024-04-22T19:58:44.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.458+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~Y8LNTHNITFZU.0 size=1048576 2024-04-22T19:58:44.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.458+0000 7f5269a8e780 1 < canceled part key=_multipart_M9M4SEEEZSFV.2~Y8LNTHNITFZU.1 size=1006538 2024-04-22T19:58:44.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.460+0000 7f5269a8e780 1 < canceled multipart upload key=M9M4SEEEZSFV upload=Y8LNTHNITFZU tag=7OZSU14012X0 type=write size=3103690 2024-04-22T19:58:44.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.461+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=8K15S1CSFZE0 type=del size=255361 2024-04-22T19:58:44.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.463+0000 7f5269a8e780 1 < suggested remove operation key=VPAVX5FXYWZO tag=LZ426AS9XUTO type=del 2024-04-22T19:58:44.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.466+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~7NUFRMPCPKMM.0 size=1048576 2024-04-22T19:58:44.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.467+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~7NUFRMPCPKMM.1 size=103606 2024-04-22T19:58:44.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.467+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=7NUFRMPCPKMM tag=40612N0S29SQ type=write size=2200758 2024-04-22T19:58:44.468 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.468+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~LGVTLKJCZKAD.0 size=1048576 2024-04-22T19:58:44.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.469+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~LGVTLKJCZKAD.1 size=1048576 2024-04-22T19:58:44.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.470+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~LGVTLKJCZKAD.2 size=475543 2024-04-22T19:58:44.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.472+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=LGVTLKJCZKAD tag=GTUXUSGZMKJU type=write size=3621271 2024-04-22T19:58:44.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.473+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=SR1NXEZP85BG type=del size=2299912 2024-04-22T19:58:44.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.474+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=BYBARCPGTO94 type=del 2024-04-22T19:58:44.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.476+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~AH5MD5X69KHD.0 size=1048576 2024-04-22T19:58:44.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.477+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~AH5MD5X69KHD.1 size=1048576 2024-04-22T19:58:44.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.478+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~AH5MD5X69KHD.2 size=61969 2024-04-22T19:58:44.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.478+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=AH5MD5X69KHD tag=4T24A0PHX9VJ type=write size=3207697 2024-04-22T19:58:44.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.479+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~ZP0NE4FFQQEB.0 size=1048576 2024-04-22T19:58:44.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.480+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~ZP0NE4FFQQEB.1 size=1048576 2024-04-22T19:58:44.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.481+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~ZP0NE4FFQQEB.2 size=600210 2024-04-22T19:58:44.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.483+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=ZP0NE4FFQQEB tag=V5FS5SENFU0L type=write size=3745938 2024-04-22T19:58:44.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.485+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~CJLWI692XMRU.0 size=1048576 2024-04-22T19:58:44.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.487+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~CJLWI692XMRU.1 size=795939 2024-04-22T19:58:44.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.487+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=CJLWI692XMRU tag=883IQFO3SXZA type=write size=2893091 2024-04-22T19:58:44.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.490+0000 7f5269a8e780 1 < suggested remove operation key=AMBJHMJGLUUS tag=6IJKZQ6TJANE type=del 2024-04-22T19:58:44.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.492+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~R1T1GAN7P54B.0 size=796450 2024-04-22T19:58:44.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.492+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=R1T1GAN7P54B tag=9V5YXHCISG7E type=write size=1845026 2024-04-22T19:58:44.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.493+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=86N96BGFCM1F type=write size=162786 2024-04-22T19:58:44.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.494+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=P06EXSV0LRLP type=del size=405779 2024-04-22T19:58:44.495 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.496+0000 7f5269a8e780 1 < completed write operation key=4GX83ZJWZCD1 tag=P1J63R21151K type=write size=86210 2024-04-22T19:58:44.497 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.497+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=KV0Y6DZPQOPO type=del size=567568 2024-04-22T19:58:44.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.498+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=JVFHVJUEYB4D type=del 2024-04-22T19:58:44.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.499+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=7WJXN7QXRY4V type=del size=3853164 2024-04-22T19:58:44.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.500+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=ZR9UT3BHBV93 type=del 2024-04-22T19:58:44.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.502+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~BXUWBXENZE25.0 size=1048576 2024-04-22T19:58:44.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.503+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~BXUWBXENZE25.1 size=1048576 2024-04-22T19:58:44.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.503+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~BXUWBXENZE25.2 size=46913 2024-04-22T19:58:44.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.504+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=BXUWBXENZE25 tag=GCU7YZCERMN4 type=write size=3192641 2024-04-22T19:58:44.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.505+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~IVAMQOWLMDBN.0 size=59296 2024-04-22T19:58:44.506 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.506+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=IVAMQOWLMDBN tag=HXXHMEBG15HM type=write size=1107872 2024-04-22T19:58:44.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.507+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=4SBP2KS2WX9O type=del size=2105033 2024-04-22T19:58:44.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.509+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~SJ49ALO0597O.0 size=1048576 2024-04-22T19:58:44.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.510+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~SJ49ALO0597O.1 size=117791 2024-04-22T19:58:44.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.511+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=SJ49ALO0597O tag=74V3P4I3QXTF type=write size=2214943 2024-04-22T19:58:44.513 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.513+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~J7S8U6KPSA1B.0 size=616574 2024-04-22T19:58:44.513 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.513+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=J7S8U6KPSA1B tag=2ZT6KDULASL3 type=write size=1665150 2024-04-22T19:58:44.514 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.515+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=SA6DUT2DT25P type=del 2024-04-22T19:58:44.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.517+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~YSCOD3565UZP.0 size=1048576 2024-04-22T19:58:44.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.518+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~YSCOD3565UZP.1 size=1048576 2024-04-22T19:58:44.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.519+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~YSCOD3565UZP.2 size=796618 2024-04-22T19:58:44.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.519+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=YSCOD3565UZP tag=711NXKRM1FOM type=write size=3942346 2024-04-22T19:58:44.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.520+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~4QA6GQABSD26.0 size=1048576 2024-04-22T19:58:44.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.520+0000 7f5269a8e780 1 < canceled part key=_multipart_2YFD2U6HVNE7.2~4QA6GQABSD26.1 size=1048576 2024-04-22T19:58:44.521 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.521+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~4QA6GQABSD26.2 size=542571 2024-04-22T19:58:44.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.524+0000 7f5269a8e780 1 < canceled multipart upload key=2YFD2U6HVNE7 upload=4QA6GQABSD26 tag=13RDCG72PQH8 type=write size=3688299 2024-04-22T19:58:44.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.526+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~OFRZFXU6CVPC.0 size=1048576 2024-04-22T19:58:44.527 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.527+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~OFRZFXU6CVPC.1 size=1048576 2024-04-22T19:58:44.527 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.528+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~OFRZFXU6CVPC.2 size=255302 2024-04-22T19:58:44.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.528+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=OFRZFXU6CVPC tag=GHDHMYS1RTP6 type=write size=3401030 2024-04-22T19:58:44.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.529+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~QOCFGQUFWPIL.0 size=914684 2024-04-22T19:58:44.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.531+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=QOCFGQUFWPIL tag=IQU38WQUJORN type=write size=1963260 2024-04-22T19:58:44.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.532+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=RS0IQ12MWIZX type=del size=953476 2024-04-22T19:58:44.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.534+0000 7f5269a8e780 1 < suggested remove operation key=O9NFAVE09ZN7 tag=A54R4O1BCAA1 type=del 2024-04-22T19:58:44.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.535+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=EU6SPPN2EPTU type=del size=1119423 2024-04-22T19:58:44.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.537+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=8K15S1CSFZE0 type=del 2024-04-22T19:58:44.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.538+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=TXAFAG3DKII3 type=del size=785450 2024-04-22T19:58:44.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.539+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~7NUFRMPCPKMM.0 size=1048576 2024-04-22T19:58:44.540 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.540+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~7NUFRMPCPKMM.1 size=103606 2024-04-22T19:58:44.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.542+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=7NUFRMPCPKMM tag=40612N0S29SQ type=write size=2200758 2024-04-22T19:58:44.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.543+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=R5X753IC0KXS type=del size=2049944 2024-04-22T19:58:44.545 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.545+0000 7f5269a8e780 1 < suggested remove operation key=M9M4SEEEZSFV tag=SR1NXEZP85BG type=del 2024-04-22T19:58:44.547 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.547+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=3TVVZZQWD95M type=del size=1297891 2024-04-22T19:58:44.548 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.548+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~AH5MD5X69KHD.0 size=1048576 2024-04-22T19:58:44.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.549+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~AH5MD5X69KHD.1 size=1048576 2024-04-22T19:58:44.550 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.550+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~AH5MD5X69KHD.2 size=61969 2024-04-22T19:58:44.554 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.552+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=AH5MD5X69KHD tag=4T24A0PHX9VJ type=write size=3207697 2024-04-22T19:58:44.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.554+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~JYRMOMN93BKX.0 size=700317 2024-04-22T19:58:44.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.554+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=JYRMOMN93BKX tag=8TXJGWM68BMO type=write size=1748893 2024-04-22T19:58:44.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.555+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~CJLWI692XMRU.0 size=1048576 2024-04-22T19:58:44.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.556+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~CJLWI692XMRU.1 size=795939 2024-04-22T19:58:44.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.558+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=CJLWI692XMRU tag=883IQFO3SXZA type=write size=2893091 2024-04-22T19:58:44.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.559+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=KOEDEUSM2YJE type=del size=3109931 2024-04-22T19:58:44.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.560+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~R1T1GAN7P54B.0 size=796450 2024-04-22T19:58:44.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.562+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=R1T1GAN7P54B tag=9V5YXHCISG7E type=write size=1845026 2024-04-22T19:58:44.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.563+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=3CXZDPC3N0QV type=del size=2016406 2024-04-22T19:58:44.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.565+0000 7f5269a8e780 1 < suggested update operation key=KE7YGPTF1MOR tag=P06EXSV0LRLP type=del size=405779 2024-04-22T19:58:44.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.566+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=9JC1MZITQGIG type=del size=2632642 2024-04-22T19:58:44.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.567+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=KV0Y6DZPQOPO type=del 2024-04-22T19:58:44.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.569+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~4ULWV0U2A1HT.0 size=1048576 2024-04-22T19:58:44.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.570+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~4ULWV0U2A1HT.1 size=551992 2024-04-22T19:58:44.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.570+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=4ULWV0U2A1HT tag=3YWF6AWNFMTA type=write size=2649144 2024-04-22T19:58:44.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.571+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=7WJXN7QXRY4V type=del 2024-04-22T19:58:44.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.573+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~L9WJQIAUWJUH.0 size=772701 2024-04-22T19:58:44.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.573+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=L9WJQIAUWJUH tag=7IRP0PTDU45G type=write size=1821277 2024-04-22T19:58:44.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.574+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~BXUWBXENZE25.0 size=1048576 2024-04-22T19:58:44.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.575+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~BXUWBXENZE25.1 size=1048576 2024-04-22T19:58:44.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.575+0000 7f5269a8e780 1 < canceled part key=_multipart_DX5YUYQRZKWA.2~BXUWBXENZE25.2 size=46913 2024-04-22T19:58:44.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.577+0000 7f5269a8e780 1 < canceled multipart upload key=DX5YUYQRZKWA upload=BXUWBXENZE25 tag=GCU7YZCERMN4 type=write size=3192641 2024-04-22T19:58:44.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.578+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=ZH7YMWUIL2EK type=del size=370209 2024-04-22T19:58:44.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.579+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=4SBP2KS2WX9O type=del 2024-04-22T19:58:44.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.581+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=0EE4NJLBYDFR type=del size=2769485 2024-04-22T19:58:44.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.582+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~J7S8U6KPSA1B.0 size=616574 2024-04-22T19:58:44.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.584+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=J7S8U6KPSA1B tag=2ZT6KDULASL3 type=write size=1665150 2024-04-22T19:58:44.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.585+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=NS7DWF74A6RP type=write size=976580 2024-04-22T19:58:44.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.586+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~YSCOD3565UZP.0 size=1048576 2024-04-22T19:58:44.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.587+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~YSCOD3565UZP.1 size=1048576 2024-04-22T19:58:44.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.588+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~YSCOD3565UZP.2 size=796618 2024-04-22T19:58:44.589 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.590+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=YSCOD3565UZP tag=711NXKRM1FOM type=write size=3942346 2024-04-22T19:58:44.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.591+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=Z4JEGF1N2Q1J type=del size=1588577 2024-04-22T19:58:44.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.592+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~OFRZFXU6CVPC.0 size=1048576 2024-04-22T19:58:44.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.594+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~OFRZFXU6CVPC.1 size=1048576 2024-04-22T19:58:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.595+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~OFRZFXU6CVPC.2 size=255302 2024-04-22T19:58:44.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.597+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=OFRZFXU6CVPC tag=GHDHMYS1RTP6 type=write size=3401030 2024-04-22T19:58:44.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.599+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~QJX0CNG6Z3RE.0 size=1048576 2024-04-22T19:58:44.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.600+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~QJX0CNG6Z3RE.1 size=984504 2024-04-22T19:58:44.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.600+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=QJX0CNG6Z3RE tag=F66QA9H2MBHY type=write size=3081656 2024-04-22T19:58:44.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.601+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=RS0IQ12MWIZX type=del 2024-04-22T19:58:44.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.603+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~G8UILXVNW2RN.0 size=1048576 2024-04-22T19:58:44.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.604+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~G8UILXVNW2RN.1 size=228730 2024-04-22T19:58:44.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.604+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=G8UILXVNW2RN tag=CCNL1P49EIDK type=write size=2325882 2024-04-22T19:58:44.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.605+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=EU6SPPN2EPTU type=del 2024-04-22T19:58:44.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.606+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=PIDKCFHQKQK3 type=del size=2055718 2024-04-22T19:58:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.607+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=TXAFAG3DKII3 type=del 2024-04-22T19:58:44.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.609+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~VZORJP4OQN5E.0 size=1048576 2024-04-22T19:58:44.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.610+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~VZORJP4OQN5E.1 size=324752 2024-04-22T19:58:44.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.610+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=VZORJP4OQN5E tag=8WGJ5JZ4HQRF type=write size=2421904 2024-04-22T19:58:44.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.611+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=R5X753IC0KXS type=del 2024-04-22T19:58:44.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.612+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=KZOZVFP68Y61 type=del size=1876963 2024-04-22T19:58:44.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.613+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=3TVVZZQWD95M type=del 2024-04-22T19:58:44.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.614+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=8SIKP0VNEGWM type=del size=3102052 2024-04-22T19:58:44.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.616+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~JYRMOMN93BKX.0 size=700317 2024-04-22T19:58:44.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.617+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=JYRMOMN93BKX tag=8TXJGWM68BMO type=write size=1748893 2024-04-22T19:58:44.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.619+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~J1JZSV79G6OO.0 size=1048576 2024-04-22T19:58:44.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.620+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~J1JZSV79G6OO.1 size=1048576 2024-04-22T19:58:44.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.621+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~J1JZSV79G6OO.2 size=532376 2024-04-22T19:58:44.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.621+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=J1JZSV79G6OO tag=JVXA1TGML1X8 type=write size=3678104 2024-04-22T19:58:44.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.623+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=KOEDEUSM2YJE type=del 2024-04-22T19:58:44.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.624+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=WZQ26X2OCMEY type=del size=1723161 2024-04-22T19:58:44.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.625+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=3CXZDPC3N0QV type=del 2024-04-22T19:58:44.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.627+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~0PHL6H4I6UXN.0 size=1048576 2024-04-22T19:58:44.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.628+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~0PHL6H4I6UXN.1 size=1048576 2024-04-22T19:58:44.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.629+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~0PHL6H4I6UXN.2 size=408425 2024-04-22T19:58:44.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.629+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=0PHL6H4I6UXN tag=2AB29IRDT05Z type=write size=3554153 2024-04-22T19:58:44.630 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.630+0000 7f5269a8e780 1 < canceled operation key=IBH6W1NINUHC tag=9JC1MZITQGIG type=del size=2632642 2024-04-22T19:58:44.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.633+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~6D15A0FTUXW5.0 size=678157 2024-04-22T19:58:44.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.633+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=6D15A0FTUXW5 tag=MZXCX6O0MTBW type=write size=1726733 2024-04-22T19:58:44.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.634+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~4ULWV0U2A1HT.0 size=1048576 2024-04-22T19:58:44.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.635+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~4ULWV0U2A1HT.1 size=551992 2024-04-22T19:58:44.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.637+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=4ULWV0U2A1HT tag=3YWF6AWNFMTA type=write size=2649144 2024-04-22T19:58:44.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.639+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~HT90CIRS4L90.0 size=1048576 2024-04-22T19:58:44.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.640+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~HT90CIRS4L90.1 size=371064 2024-04-22T19:58:44.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.640+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=HT90CIRS4L90 tag=MMKRRMLUSQKO type=write size=2468216 2024-04-22T19:58:44.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.640+0000 7f5269a8e780 1 < canceled part key=_multipart_8V6ESIGMRO5G.2~L9WJQIAUWJUH.0 size=772701 2024-04-22T19:58:44.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.642+0000 7f5269a8e780 1 < canceled multipart upload key=8V6ESIGMRO5G upload=L9WJQIAUWJUH tag=7IRP0PTDU45G type=write size=1821277 2024-04-22T19:58:44.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.644+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~LZKIJCE7AMOO.0 size=909475 2024-04-22T19:58:44.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.644+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=LZKIJCE7AMOO tag=1NA5LT5GS1VD type=write size=1958051 2024-04-22T19:58:44.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.645+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=ZH7YMWUIL2EK type=del 2024-04-22T19:58:44.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.646+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=8RG6HVM6FOJ3 type=del size=1653705 2024-04-22T19:58:44.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.647+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=0EE4NJLBYDFR type=del 2024-04-22T19:58:44.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.648+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=0XBYFNMZJBHU type=write size=1018315 2024-04-22T19:58:44.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.649+0000 7f5269a8e780 1 < completed write operation key=2YFD2U6HVNE7 tag=NS7DWF74A6RP type=write size=976580 2024-04-22T19:58:44.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.651+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~S27RV9H18U37.0 size=988579 2024-04-22T19:58:44.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.651+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=S27RV9H18U37 tag=L6PLG60XTEZW type=write size=2037155 2024-04-22T19:58:44.653 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.653+0000 7f5269a8e780 1 < suggested update operation key=ZC4ZOHUJ3HQD tag=Z4JEGF1N2Q1J type=del size=1588577 2024-04-22T19:58:44.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.654+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=N3AFL34M9OBY type=del size=3776923 2024-04-22T19:58:44.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.655+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~QJX0CNG6Z3RE.0 size=1048576 2024-04-22T19:58:44.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.655+0000 7f5269a8e780 1 < canceled part key=_multipart_T2BV2LUCY4E3.2~QJX0CNG6Z3RE.1 size=984504 2024-04-22T19:58:44.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.657+0000 7f5269a8e780 1 < canceled multipart upload key=T2BV2LUCY4E3 upload=QJX0CNG6Z3RE tag=F66QA9H2MBHY type=write size=3081656 2024-04-22T19:58:44.658 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.658+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=7YB1M6BXMMXN type=write size=284758 2024-04-22T19:58:44.659 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.659+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~G8UILXVNW2RN.0 size=1048576 2024-04-22T19:58:44.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.660+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~G8UILXVNW2RN.1 size=228730 2024-04-22T19:58:44.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.662+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=G8UILXVNW2RN tag=CCNL1P49EIDK type=write size=2325882 2024-04-22T19:58:44.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.663+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=Q6UP4LGNLWYO type=write size=546056 2024-04-22T19:58:44.664 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.664+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=PIDKCFHQKQK3 type=del 2024-04-22T19:58:44.665 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.665+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=EA8EM6G11XBC type=del size=357489 2024-04-22T19:58:44.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.666+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~VZORJP4OQN5E.0 size=1048576 2024-04-22T19:58:44.667 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.667+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~VZORJP4OQN5E.1 size=324752 2024-04-22T19:58:44.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.669+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=VZORJP4OQN5E tag=8WGJ5JZ4HQRF type=write size=2421904 2024-04-22T19:58:44.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.670+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=KHP29DO44513 type=del size=2458408 2024-04-22T19:58:44.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.671+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=KZOZVFP68Y61 type=del 2024-04-22T19:58:44.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.672+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=SKA7XU2UK3A3 type=del size=181578 2024-04-22T19:58:44.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.674+0000 7f5269a8e780 1 < suggested update operation key=Q1UKF4UMJ2HO tag=8SIKP0VNEGWM type=del size=3102052 2024-04-22T19:58:44.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.675+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=S9EF0Y9UE9LT type=write size=778865 2024-04-22T19:58:44.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.676+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~J1JZSV79G6OO.0 size=1048576 2024-04-22T19:58:44.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.677+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~J1JZSV79G6OO.1 size=1048576 2024-04-22T19:58:44.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.677+0000 7f5269a8e780 1 < canceled part key=_multipart_I91AD8FJ3P4T.2~J1JZSV79G6OO.2 size=532376 2024-04-22T19:58:44.679 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.679+0000 7f5269a8e780 1 < canceled multipart upload key=I91AD8FJ3P4T upload=J1JZSV79G6OO tag=JVXA1TGML1X8 type=write size=3678104 2024-04-22T19:58:44.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.680+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=SJJPAPW8NBOY type=write size=43002 2024-04-22T19:58:44.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.681+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=WZQ26X2OCMEY type=del 2024-04-22T19:58:44.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.683+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~GEFOVEFKSU6T.0 size=1048576 2024-04-22T19:58:44.684 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.684+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~GEFOVEFKSU6T.1 size=548765 2024-04-22T19:58:44.684 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.684+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=GEFOVEFKSU6T tag=VA9L6OFAN8JC type=write size=2645917 2024-04-22T19:58:44.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.685+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~0PHL6H4I6UXN.0 size=1048576 2024-04-22T19:58:44.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.686+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~0PHL6H4I6UXN.1 size=1048576 2024-04-22T19:58:44.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.687+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~0PHL6H4I6UXN.2 size=408425 2024-04-22T19:58:44.689 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.689+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=0PHL6H4I6UXN tag=2AB29IRDT05Z type=write size=3554153 2024-04-22T19:58:44.690 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.690+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=H2S2R3EBN1CE type=del size=2923454 2024-04-22T19:58:44.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.691+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~6D15A0FTUXW5.0 size=678157 2024-04-22T19:58:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.693+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=6D15A0FTUXW5 tag=MZXCX6O0MTBW type=write size=1726733 2024-04-22T19:58:44.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.694+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=QU3RN3LPTTE1 type=del size=363784 2024-04-22T19:58:44.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.695+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~HT90CIRS4L90.0 size=1048576 2024-04-22T19:58:44.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.697+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~HT90CIRS4L90.1 size=371064 2024-04-22T19:58:44.698 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.698+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=HT90CIRS4L90 tag=MMKRRMLUSQKO type=write size=2468216 2024-04-22T19:58:44.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.700+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~6PZJLTIK3TQX.0 size=1048576 2024-04-22T19:58:44.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.701+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~6PZJLTIK3TQX.1 size=310273 2024-04-22T19:58:44.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.701+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=6PZJLTIK3TQX tag=GN49P71L87T5 type=write size=2407425 2024-04-22T19:58:44.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.702+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~LZKIJCE7AMOO.0 size=909475 2024-04-22T19:58:44.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.704+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=LZKIJCE7AMOO tag=1NA5LT5GS1VD type=write size=1958051 2024-04-22T19:58:44.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.706+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~QL3Z24RUQRKD.0 size=846163 2024-04-22T19:58:44.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.706+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=QL3Z24RUQRKD tag=H3EHXW2AHMR4 type=write size=1894739 2024-04-22T19:58:44.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.707+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=8RG6HVM6FOJ3 type=del 2024-04-22T19:58:44.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.708+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=JAW6T2TO2WGN type=del size=4110513 2024-04-22T19:58:44.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.709+0000 7f5269a8e780 1 < canceled operation key=VPAVX5FXYWZO tag=0XBYFNMZJBHU type=write size=1018315 2024-04-22T19:58:44.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.711+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~UXA28FNNFUG9.0 size=1048576 2024-04-22T19:58:44.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.712+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~UXA28FNNFUG9.1 size=1048576 2024-04-22T19:58:44.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.713+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~UXA28FNNFUG9.2 size=638955 2024-04-22T19:58:44.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.713+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=UXA28FNNFUG9 tag=WY1D6H03AJAS type=write size=3784683 2024-04-22T19:58:44.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.714+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~S27RV9H18U37.0 size=988579 2024-04-22T19:58:44.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.716+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=S27RV9H18U37 tag=L6PLG60XTEZW type=write size=2037155 2024-04-22T19:58:44.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.717+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=DNLC575OSKEN type=del size=4168123 2024-04-22T19:58:44.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.718+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=N3AFL34M9OBY type=del 2024-04-22T19:58:44.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.719+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=7K9BY5AZQ3IM type=del size=2965149 2024-04-22T19:58:44.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.721+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=7YB1M6BXMMXN type=write size=284758 2024-04-22T19:58:44.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.722+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~GC5JI575ENDS.0 size=1048576 2024-04-22T19:58:44.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.723+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~GC5JI575ENDS.1 size=402688 2024-04-22T19:58:44.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.723+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=GC5JI575ENDS tag=ZL8YWCZSGXGW type=write size=2499840 2024-04-22T19:58:44.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.725+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=Q6UP4LGNLWYO type=write size=546056 2024-04-22T19:58:44.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.726+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=YYC3SNX2OH5A type=del size=2290588 2024-04-22T19:58:44.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.727+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=EA8EM6G11XBC type=del 2024-04-22T19:58:44.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.729+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~NVA9VVCS2AGY.0 size=875463 2024-04-22T19:58:44.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.729+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=NVA9VVCS2AGY tag=TZ20E0ELIHXL type=write size=1924039 2024-04-22T19:58:44.730 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.730+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=KHP29DO44513 type=del 2024-04-22T19:58:44.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.731+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=9UCCWIQB7A81 type=del size=2670092 2024-04-22T19:58:44.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.732+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=SKA7XU2UK3A3 type=del 2024-04-22T19:58:44.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.733+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=Q2TCNED0UQUF type=write size=292227 2024-04-22T19:58:44.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.734+0000 7f5269a8e780 1 < completed write operation key=HIAXPVKZ1IIO tag=S9EF0Y9UE9LT type=write size=778865 2024-04-22T19:58:44.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.735+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=UE89BGH35CBQ type=write size=494012 2024-04-22T19:58:44.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.736+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=SJJPAPW8NBOY type=write size=43002 2024-04-22T19:58:44.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.737+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=H6N2Y8MGTX5U type=del size=825001 2024-04-22T19:58:44.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.738+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~GEFOVEFKSU6T.0 size=1048576 2024-04-22T19:58:44.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.739+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~GEFOVEFKSU6T.1 size=548765 2024-04-22T19:58:44.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.741+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=GEFOVEFKSU6T tag=VA9L6OFAN8JC type=write size=2645917 2024-04-22T19:58:44.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.743+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~M9HF0V6H6VLI.0 size=824913 2024-04-22T19:58:44.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.743+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=M9HF0V6H6VLI tag=MO8756JKY2QU type=write size=1873489 2024-04-22T19:58:44.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.744+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=H2S2R3EBN1CE type=del 2024-04-22T19:58:44.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.746+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~A0I2387Q76ZB.0 size=1048576 2024-04-22T19:58:44.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.747+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~A0I2387Q76ZB.1 size=1048576 2024-04-22T19:58:44.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.748+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~A0I2387Q76ZB.2 size=82849 2024-04-22T19:58:44.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.748+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=A0I2387Q76ZB tag=VVM3YMQOXKN0 type=write size=3228577 2024-04-22T19:58:44.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.749+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=QU3RN3LPTTE1 type=del 2024-04-22T19:58:44.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.750+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=OHHYLOQWN5UD type=del size=43143 2024-04-22T19:58:44.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.751+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~6PZJLTIK3TQX.0 size=1048576 2024-04-22T19:58:44.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.751+0000 7f5269a8e780 1 < canceled part key=_multipart_RY1O7X1CWZMK.2~6PZJLTIK3TQX.1 size=310273 2024-04-22T19:58:44.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.753+0000 7f5269a8e780 1 < canceled multipart upload key=RY1O7X1CWZMK upload=6PZJLTIK3TQX tag=GN49P71L87T5 type=write size=2407425 2024-04-22T19:58:44.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.755+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~ULDRO34KKA3L.0 size=1048576 2024-04-22T19:58:44.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.756+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~ULDRO34KKA3L.1 size=1048576 2024-04-22T19:58:44.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.757+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~ULDRO34KKA3L.2 size=17303 2024-04-22T19:58:44.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.757+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=ULDRO34KKA3L tag=GK1SNA8EZE0I type=write size=3163031 2024-04-22T19:58:44.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.758+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~QL3Z24RUQRKD.0 size=846163 2024-04-22T19:58:44.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.761+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=QL3Z24RUQRKD tag=H3EHXW2AHMR4 type=write size=1894739 2024-04-22T19:58:44.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.762+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=7HQBK5P85B6J type=del size=2377712 2024-04-22T19:58:44.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.764+0000 7f5269a8e780 1 < suggested update operation key=8HFSDSNZONIN tag=JAW6T2TO2WGN type=del size=4110513 2024-04-22T19:58:44.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.765+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=MZAJ5WMA5UKW type=del size=258693 2024-04-22T19:58:44.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.766+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~UXA28FNNFUG9.0 size=1048576 2024-04-22T19:58:44.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.766+0000 7f5269a8e780 1 < canceled part key=_multipart_8HFSDSNZONIN.2~UXA28FNNFUG9.1 size=1048576 2024-04-22T19:58:44.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.767+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~UXA28FNNFUG9.2 size=638955 2024-04-22T19:58:44.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.769+0000 7f5269a8e780 1 < canceled multipart upload key=8HFSDSNZONIN upload=UXA28FNNFUG9 tag=WY1D6H03AJAS type=write size=3784683 2024-04-22T19:58:44.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.771+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=Y503YSVDM2JR type=del size=3183706 2024-04-22T19:58:44.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.772+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=DNLC575OSKEN type=del 2024-04-22T19:58:44.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.773+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=BT1J0QD0ESHQ type=del size=1194286 2024-04-22T19:58:44.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.775+0000 7f5269a8e780 1 < suggested remove operation key=IBH6W1NINUHC tag=7K9BY5AZQ3IM type=del 2024-04-22T19:58:44.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.776+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=AI2381H8KBEB type=del size=3596428 2024-04-22T19:58:44.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.777+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~GC5JI575ENDS.0 size=1048576 2024-04-22T19:58:44.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.779+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~GC5JI575ENDS.1 size=402688 2024-04-22T19:58:44.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.780+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=GC5JI575ENDS tag=ZL8YWCZSGXGW type=write size=2499840 2024-04-22T19:58:44.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.783+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~OVTY2Z3ZKIPS.0 size=545866 2024-04-22T19:58:44.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.783+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=OVTY2Z3ZKIPS tag=NWKV023MA2ZJ type=write size=1594442 2024-04-22T19:58:44.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.784+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=YYC3SNX2OH5A type=del 2024-04-22T19:58:44.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.785+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=0DJ340I24NQ2 type=write size=612211 2024-04-22T19:58:44.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.786+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~NVA9VVCS2AGY.0 size=875463 2024-04-22T19:58:44.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.788+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=NVA9VVCS2AGY tag=TZ20E0ELIHXL type=write size=1924039 2024-04-22T19:58:44.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.789+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=7GEE4H8M7TUI type=del size=2932373 2024-04-22T19:58:44.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.790+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=9UCCWIQB7A81 type=del 2024-04-22T19:58:44.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.792+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~NUBB9WQIAW01.0 size=1048576 2024-04-22T19:58:44.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.793+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~NUBB9WQIAW01.1 size=701363 2024-04-22T19:58:44.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.793+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=NUBB9WQIAW01 tag=D9PGFDPVIRUG type=write size=2798515 2024-04-22T19:58:44.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.794+0000 7f5269a8e780 1 < canceled operation key=4EA3LUQRL8II tag=Q2TCNED0UQUF type=write size=292227 2024-04-22T19:58:44.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.796+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=QC383PG1IUWT type=write size=1034033 2024-04-22T19:58:44.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.797+0000 7f5269a8e780 1 < completed write operation key=HV8J6YM0YNC1 tag=UE89BGH35CBQ type=write size=494012 2024-04-22T19:58:44.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.799+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~CHECWVUJDICR.0 size=1048576 2024-04-22T19:58:44.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.800+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~CHECWVUJDICR.1 size=1048576 2024-04-22T19:58:44.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.802+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~CHECWVUJDICR.2 size=262832 2024-04-22T19:58:44.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.802+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=CHECWVUJDICR tag=HLY0EVTBMBYX type=write size=3408560 2024-04-22T19:58:44.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.803+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=H6N2Y8MGTX5U type=del 2024-04-22T19:58:44.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.804+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=YUW3R8Z312K1 type=del size=2375434 2024-04-22T19:58:44.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.804+0000 7f5269a8e780 1 < canceled part key=_multipart_SN2BZATJGUWA.2~M9HF0V6H6VLI.0 size=824913 2024-04-22T19:58:44.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.806+0000 7f5269a8e780 1 < canceled multipart upload key=SN2BZATJGUWA upload=M9HF0V6H6VLI tag=MO8756JKY2QU type=write size=1873489 2024-04-22T19:58:44.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.807+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=4ZAVYPSC6UIJ type=del size=1576041 2024-04-22T19:58:44.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.807+0000 7f5269a8e780 1 < canceled part key=_multipart_A853MOZPT9BG.2~A0I2387Q76ZB.0 size=1048576 2024-04-22T19:58:44.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.808+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~A0I2387Q76ZB.1 size=1048576 2024-04-22T19:58:44.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.809+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~A0I2387Q76ZB.2 size=82849 2024-04-22T19:58:44.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.811+0000 7f5269a8e780 1 < canceled multipart upload key=A853MOZPT9BG upload=A0I2387Q76ZB tag=VVM3YMQOXKN0 type=write size=3228577 2024-04-22T19:58:44.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.812+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=W24HDBL9RI20 type=write size=747614 2024-04-22T19:58:44.813 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.813+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=OHHYLOQWN5UD type=del 2024-04-22T19:58:44.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.815+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=IG589VC947LT type=del size=87554 2024-04-22T19:58:44.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.816+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~ULDRO34KKA3L.0 size=1048576 2024-04-22T19:58:44.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.817+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~ULDRO34KKA3L.1 size=1048576 2024-04-22T19:58:44.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.817+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~ULDRO34KKA3L.2 size=17303 2024-04-22T19:58:44.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.819+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=ULDRO34KKA3L tag=GK1SNA8EZE0I type=write size=3163031 2024-04-22T19:58:44.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.821+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~H0EZZNSOPWZQ.0 size=96616 2024-04-22T19:58:44.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.821+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=H0EZZNSOPWZQ tag=JQG7JQLLNBIB type=write size=1145192 2024-04-22T19:58:44.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.822+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=7HQBK5P85B6J type=del 2024-04-22T19:58:44.823 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.823+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=4EO8ORZX1H1D type=del size=1086716 2024-04-22T19:58:44.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.824+0000 7f5269a8e780 1 < canceled operation key=RY1O7X1CWZMK tag=MZAJ5WMA5UKW type=del size=258693 2024-04-22T19:58:44.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.825+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=IMCUXZFB31IB type=write size=244616 2024-04-22T19:58:44.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.826+0000 7f5269a8e780 -1 < no bi entry to suggest for operation key=IBH6W1NINUHC tag=Y503YSVDM2JR type=del size=3183706: (2) No such file or directory 2024-04-22T19:58:44.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.827+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=BWXHPJ3UXE45 type=del size=1532121 2024-04-22T19:58:44.828 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.828+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=BT1J0QD0ESHQ type=del 2024-04-22T19:58:44.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.829+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=95KP12WXVY32 type=del size=1359187 2024-04-22T19:58:44.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.830+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=AI2381H8KBEB type=del 2024-04-22T19:58:44.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.832+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~E2FUTQGY0EYQ.0 size=259289 2024-04-22T19:58:44.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.832+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=E2FUTQGY0EYQ tag=W1UZONIP4829 type=write size=1307865 2024-04-22T19:58:44.834 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.834+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~OVTY2Z3ZKIPS.0 size=545866 2024-04-22T19:58:44.836 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.836+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=OVTY2Z3ZKIPS tag=NWKV023MA2ZJ type=write size=1594442 2024-04-22T19:58:44.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.837+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=QXQ69A24QW7S type=del size=2987523 2024-04-22T19:58:44.838 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.838+0000 7f5269a8e780 1 < completed write operation key=HIAXPVKZ1IIO tag=0DJ340I24NQ2 type=write size=612211 2024-04-22T19:58:44.839 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.839+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=39UTTM64E271 type=write size=51450 2024-04-22T19:58:44.840 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.841+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=7GEE4H8M7TUI type=del 2024-04-22T19:58:44.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.843+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~RS192OIO31E1.0 size=1048576 2024-04-22T19:58:44.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.843+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~RS192OIO31E1.1 size=1048576 2024-04-22T19:58:44.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.844+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~RS192OIO31E1.2 size=304029 2024-04-22T19:58:44.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.844+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=RS192OIO31E1 tag=PYD16S7CE7WN type=write size=3449757 2024-04-22T19:58:44.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.845+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~NUBB9WQIAW01.0 size=1048576 2024-04-22T19:58:44.846 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.846+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~NUBB9WQIAW01.1 size=701363 2024-04-22T19:58:44.848 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.848+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=NUBB9WQIAW01 tag=D9PGFDPVIRUG type=write size=2798515 2024-04-22T19:58:44.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.850+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~MJPZP89XQK06.0 size=1048576 2024-04-22T19:58:44.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.851+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~MJPZP89XQK06.1 size=1048576 2024-04-22T19:58:44.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.852+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~MJPZP89XQK06.2 size=684398 2024-04-22T19:58:44.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.852+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=MJPZP89XQK06 tag=CJ8PLTH8FH9X type=write size=3830126 2024-04-22T19:58:44.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.853+0000 7f5269a8e780 1 < completed write operation key=Q1UKF4UMJ2HO tag=QC383PG1IUWT type=write size=1034033 2024-04-22T19:58:44.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.854+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=EW0G5LU724H6 type=del size=1133999 2024-04-22T19:58:44.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.855+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~CHECWVUJDICR.0 size=1048576 2024-04-22T19:58:44.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.856+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~CHECWVUJDICR.1 size=1048576 2024-04-22T19:58:44.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.857+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~CHECWVUJDICR.2 size=262832 2024-04-22T19:58:44.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.859+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=CHECWVUJDICR tag=HLY0EVTBMBYX type=write size=3408560 2024-04-22T19:58:44.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.860+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=05XCV9K0T9YQ type=del size=1519836 2024-04-22T19:58:44.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.862+0000 7f5269a8e780 1 < suggested remove operation key=4EA3LUQRL8II tag=YUW3R8Z312K1 type=del 2024-04-22T19:58:44.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.864+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~6CU6YD38H1LU.0 size=1048576 2024-04-22T19:58:44.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.865+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~6CU6YD38H1LU.1 size=301863 2024-04-22T19:58:44.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.865+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=6CU6YD38H1LU tag=O96SPKXC3RL5 type=write size=2399015 2024-04-22T19:58:44.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.866+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=4ZAVYPSC6UIJ type=del 2024-04-22T19:58:44.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.868+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=24WA12V6R3LU type=write size=622701 2024-04-22T19:58:44.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.869+0000 7f5269a8e780 1 < completed write operation key=RY1O7X1CWZMK tag=W24HDBL9RI20 type=write size=747614 2024-04-22T19:58:44.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.870+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=I0OEFEOFJRS2 type=del size=1707393 2024-04-22T19:58:44.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.871+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=IG589VC947LT type=del 2024-04-22T19:58:44.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.873+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~2ZWC8LXAYZ51.0 size=696942 2024-04-22T19:58:44.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.873+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=2ZWC8LXAYZ51 tag=LBOJ8JLPFY46 type=write size=1745518 2024-04-22T19:58:44.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.874+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~H0EZZNSOPWZQ.0 size=96616 2024-04-22T19:58:44.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.876+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=H0EZZNSOPWZQ tag=JQG7JQLLNBIB type=write size=1145192 2024-04-22T19:58:44.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.877+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=AGRSC3D5KAMZ type=del size=1039987 2024-04-22T19:58:44.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.879+0000 7f5269a8e780 1 < suggested update operation key=HIAXPVKZ1IIO tag=4EO8ORZX1H1D type=del size=1086716 2024-04-22T19:58:44.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.881+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=6PAPTF3EQLD4 type=write size=830965 2024-04-22T19:58:44.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.883+0000 7f5269a8e780 1 < suggested remove operation key=8V6ESIGMRO5G tag=IMCUXZFB31IB type=write 2024-04-22T19:58:44.884 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.884+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=405HX0MO7PEL type=del size=2640832 2024-04-22T19:58:44.885 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.885+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=BWXHPJ3UXE45 type=del 2024-04-22T19:58:44.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.886+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=4E5Y9K0P8QNJ type=del size=970470 2024-04-22T19:58:44.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.889+0000 7f5269a8e780 1 < suggested update operation key=SN2BZATJGUWA tag=95KP12WXVY32 type=del size=1359187 2024-04-22T19:58:44.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.890+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=GWPS90WBN04G type=del size=2862727 2024-04-22T19:58:44.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.891+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~E2FUTQGY0EYQ.0 size=259289 2024-04-22T19:58:44.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.893+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=E2FUTQGY0EYQ tag=W1UZONIP4829 type=write size=1307865 2024-04-22T19:58:44.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.894+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=VL8KRESFXH7J type=del size=3442287 2024-04-22T19:58:44.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.895+0000 7f5269a8e780 -1 < failed to complete operation key=HIAXPVKZ1IIO tag=QXQ69A24QW7S type=del size=2987523: (22) Invalid argument 2024-04-22T19:58:44.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.896+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=3MFT0CHC4UW5 type=del size=2405237 2024-04-22T19:58:44.898 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.898+0000 7f5269a8e780 1 < suggested remove operation key=SN2BZATJGUWA tag=39UTTM64E271 type=write 2024-04-22T19:58:44.899 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.899+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=JENMD7UF7AO8 type=del size=1335225 2024-04-22T19:58:44.900 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.901+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~RS192OIO31E1.0 size=1048576 2024-04-22T19:58:44.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.902+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~RS192OIO31E1.1 size=1048576 2024-04-22T19:58:44.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.902+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~RS192OIO31E1.2 size=304029 2024-04-22T19:58:44.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.904+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=RS192OIO31E1 tag=PYD16S7CE7WN type=write size=3449757 2024-04-22T19:58:44.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.905+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=YZAIJHZGLB0O type=del size=206362 2024-04-22T19:58:44.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.907+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~MJPZP89XQK06.0 size=1048576 2024-04-22T19:58:44.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.907+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~MJPZP89XQK06.1 size=1048576 2024-04-22T19:58:44.909 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.909+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~MJPZP89XQK06.2 size=684398 2024-04-22T19:58:44.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.911+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=MJPZP89XQK06 tag=CJ8PLTH8FH9X type=write size=3830126 2024-04-22T19:58:44.912 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.912+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=A51FSJRNOEHU type=del size=1008485 2024-04-22T19:58:44.915 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.915+0000 7f5269a8e780 1 < suggested remove operation key=70MLQGJPQYPY tag=EW0G5LU724H6 type=del 2024-04-22T19:58:44.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.917+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~MUXA1GTIRB10.0 size=439208 2024-04-22T19:58:44.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.917+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=MUXA1GTIRB10 tag=WMM6WINAPMSU type=write size=1487784 2024-04-22T19:58:44.918 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.918+0000 7f5269a8e780 1 < canceled operation key=5UBYMQT7ZWKX tag=05XCV9K0T9YQ type=del size=1519836 2024-04-22T19:58:44.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.920+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=YHUQ8R1SUWV5 type=del size=1143218 2024-04-22T19:58:44.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.920+0000 7f5269a8e780 1 < canceled part key=_multipart_RY1O7X1CWZMK.2~6CU6YD38H1LU.0 size=1048576 2024-04-22T19:58:44.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.921+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~6CU6YD38H1LU.1 size=301863 2024-04-22T19:58:44.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.922+0000 7f5269a8e780 1 < canceled multipart upload key=RY1O7X1CWZMK upload=6CU6YD38H1LU tag=O96SPKXC3RL5 type=write size=2399015 2024-04-22T19:58:44.923 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.924+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=0LWHCY84JJAO type=del size=1315798 2024-04-22T19:58:44.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.926+0000 7f5269a8e780 1 < suggested remove operation key=5UBYMQT7ZWKX tag=24WA12V6R3LU type=write 2024-04-22T19:58:44.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.927+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=Y8P8YPZZEIJU type=del size=1259160 2024-04-22T19:58:44.928 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.928+0000 7f5269a8e780 -1 < failed to complete operation key=8V6ESIGMRO5G tag=I0OEFEOFJRS2 type=del size=1707393: (22) Invalid argument 2024-04-22T19:58:44.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.930+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~XX03F46YE1E4.0 size=1048576 2024-04-22T19:58:44.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.931+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~XX03F46YE1E4.1 size=125558 2024-04-22T19:58:44.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.931+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=XX03F46YE1E4 tag=F32WQZ1RPOIZ type=write size=2222710 2024-04-22T19:58:44.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.932+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~2ZWC8LXAYZ51.0 size=696942 2024-04-22T19:58:44.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.934+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=2ZWC8LXAYZ51 tag=LBOJ8JLPFY46 type=write size=1745518 2024-04-22T19:58:44.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.936+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~X8DYO4V9MGFI.0 size=1048576 2024-04-22T19:58:44.937 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.937+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~X8DYO4V9MGFI.1 size=1048576 2024-04-22T19:58:44.938 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.939+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~X8DYO4V9MGFI.2 size=699185 2024-04-22T19:58:44.938 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.939+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=X8DYO4V9MGFI tag=87WWRQZUKF4J type=write size=3844913 2024-04-22T19:58:44.939 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.940+0000 7f5269a8e780 -1 < failed to complete operation key=8V6ESIGMRO5G tag=AGRSC3D5KAMZ type=del size=1039987: (22) Invalid argument 2024-04-22T19:58:44.941 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.941+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=U0IS5MLWOPZA type=del size=1253629 2024-04-22T19:58:44.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.942+0000 7f5269a8e780 1 < completed write operation key=EHRS6DQDRXVS tag=6PAPTF3EQLD4 type=write size=830965 2024-04-22T19:58:44.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.944+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~B1U1JL7QWTGN.0 size=1048576 2024-04-22T19:58:44.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.945+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~B1U1JL7QWTGN.1 size=1048576 2024-04-22T19:58:44.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.946+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~B1U1JL7QWTGN.2 size=389355 2024-04-22T19:58:44.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.946+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=B1U1JL7QWTGN tag=F9LWIK5H9Z65 type=write size=3535083 2024-04-22T19:58:44.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.947+0000 7f5269a8e780 -1 < failed to complete operation key=SN2BZATJGUWA tag=405HX0MO7PEL type=del size=2640832: (22) Invalid argument 2024-04-22T19:58:44.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.948+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=NHSSU2PER354 type=del size=616355 2024-04-22T19:58:44.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.949+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=4E5Y9K0P8QNJ type=del 2024-04-22T19:58:44.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.950+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=T0CGB6NOKJS1 type=del size=2428514 2024-04-22T19:58:44.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.952+0000 7f5269a8e780 1 < suggested remove operation key=AMBJHMJGLUUS tag=GWPS90WBN04G type=del 2024-04-22T19:58:44.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.953+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=3IJN3SQMO2FU type=write size=174165 2024-04-22T19:58:44.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.955+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=VL8KRESFXH7J type=del 2024-04-22T19:58:44.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.957+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~NOO7921AS24F.0 size=1048576 2024-04-22T19:58:44.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.958+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~NOO7921AS24F.1 size=1048576 2024-04-22T19:58:44.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.958+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~NOO7921AS24F.2 size=980322 2024-04-22T19:58:44.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.958+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=NOO7921AS24F tag=W3G6P4AZDK64 type=write size=4126050 2024-04-22T19:58:44.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.961+0000 7f5269a8e780 1 < suggested remove operation key=4EA3LUQRL8II tag=3MFT0CHC4UW5 type=del 2024-04-22T19:58:44.962 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.962+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=YKJUV49RV13K type=write size=436955 2024-04-22T19:58:44.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.963+0000 7f5269a8e780 -1 < failed to complete operation key=4EA3LUQRL8II tag=JENMD7UF7AO8 type=del size=1335225: (22) Invalid argument 2024-04-22T19:58:44.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.964+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=F6SIHGM9YWV8 type=del size=2855649 2024-04-22T19:58:44.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.965+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=YZAIJHZGLB0O type=del 2024-04-22T19:58:44.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.967+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~2EST76XDDDNQ.0 size=1048576 2024-04-22T19:58:44.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.968+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~2EST76XDDDNQ.1 size=1048576 2024-04-22T19:58:44.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.969+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~2EST76XDDDNQ.2 size=328140 2024-04-22T19:58:44.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.969+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=2EST76XDDDNQ tag=TYN05LMLNI7P type=write size=3473868 2024-04-22T19:58:44.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.970+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=A51FSJRNOEHU type=del 2024-04-22T19:58:44.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.971+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=Y6PPOZ3EYZL8 type=del size=2092083 2024-04-22T19:58:44.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.973+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~MUXA1GTIRB10.0 size=439208 2024-04-22T19:58:44.974 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.974+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=MUXA1GTIRB10 tag=WMM6WINAPMSU type=write size=1487784 2024-04-22T19:58:44.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.977+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~DEXPPFS5DE2D.0 size=965674 2024-04-22T19:58:44.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.977+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=DEXPPFS5DE2D tag=LPKFSD4MUK4S type=write size=2014250 2024-04-22T19:58:44.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.978+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=YHUQ8R1SUWV5 type=del 2024-04-22T19:58:44.979 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.979+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=MWWER5V2P1I7 type=write size=762893 2024-04-22T19:58:44.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.980+0000 7f5269a8e780 -1 < failed to complete operation key=5UBYMQT7ZWKX tag=0LWHCY84JJAO type=del size=1315798: (22) Invalid argument 2024-04-22T19:58:44.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.981+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=PFG8I0KFZSQB type=write size=5842 2024-04-22T19:58:44.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.982+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=Y8P8YPZZEIJU type=del 2024-04-22T19:58:44.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.984+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=CQO8NZ8MBJMA type=del size=2831387 2024-04-22T19:58:44.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.985+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~XX03F46YE1E4.0 size=1048576 2024-04-22T19:58:44.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.985+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~XX03F46YE1E4.1 size=125558 2024-04-22T19:58:44.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.987+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=XX03F46YE1E4 tag=F32WQZ1RPOIZ type=write size=2222710 2024-04-22T19:58:44.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.989+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~4C9EK64OB6NG.0 size=1048576 2024-04-22T19:58:44.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.990+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~4C9EK64OB6NG.1 size=581762 2024-04-22T19:58:44.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.990+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=4C9EK64OB6NG tag=IQOPNHQBKSZ9 type=write size=2678914 2024-04-22T19:58:44.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.991+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~X8DYO4V9MGFI.0 size=1048576 2024-04-22T19:58:44.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.992+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~X8DYO4V9MGFI.1 size=1048576 2024-04-22T19:58:44.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.993+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~X8DYO4V9MGFI.2 size=699185 2024-04-22T19:58:44.994 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.995+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=X8DYO4V9MGFI tag=87WWRQZUKF4J type=write size=3844913 2024-04-22T19:58:44.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.996+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=SVF82WB3R6PZ type=write size=10622 2024-04-22T19:58:44.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.997+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=U0IS5MLWOPZA type=del 2024-04-22T19:58:44.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.998+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=13PQ0C15DE1H type=del size=2232057 2024-04-22T19:58:44.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:44.999+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~B1U1JL7QWTGN.0 size=1048576 2024-04-22T19:58:45.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.000+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~B1U1JL7QWTGN.1 size=1048576 2024-04-22T19:58:45.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.001+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~B1U1JL7QWTGN.2 size=389355 2024-04-22T19:58:45.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.003+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=B1U1JL7QWTGN tag=F9LWIK5H9Z65 type=write size=3535083 2024-04-22T19:58:45.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.004+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=XXHJUK3UPT89 type=del size=3614132 2024-04-22T19:58:45.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.005+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=NHSSU2PER354 type=del 2024-04-22T19:58:45.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.006+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=L7RK3QIS7XCS type=del size=2892270 2024-04-22T19:58:45.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.007+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=T0CGB6NOKJS1 type=del 2024-04-22T19:58:45.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.008+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=JEMX4SEN7VMB type=del size=1997567 2024-04-22T19:58:45.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.009+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=3IJN3SQMO2FU type=write size=174165 2024-04-22T19:58:45.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.012+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~ILFGFZ1T2FL9.0 size=1048576 2024-04-22T19:58:45.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.012+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~ILFGFZ1T2FL9.1 size=918224 2024-04-22T19:58:45.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.012+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=ILFGFZ1T2FL9 tag=OAQEVAK2QOLZ type=write size=3015376 2024-04-22T19:58:45.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.013+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~NOO7921AS24F.0 size=1048576 2024-04-22T19:58:45.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.015+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~NOO7921AS24F.1 size=1048576 2024-04-22T19:58:45.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.017+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~NOO7921AS24F.2 size=980322 2024-04-22T19:58:45.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.019+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=NOO7921AS24F tag=W3G6P4AZDK64 type=write size=4126050 2024-04-22T19:58:45.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.021+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~RAXT7LX827SO.0 size=1048576 2024-04-22T19:58:45.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.022+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~RAXT7LX827SO.1 size=247347 2024-04-22T19:58:45.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.022+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=RAXT7LX827SO tag=MYTBUVTD2OTT type=write size=2344499 2024-04-22T19:58:45.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.024+0000 7f5269a8e780 1 < suggested remove operation key=86GW6HV9G1IZ tag=YKJUV49RV13K type=write 2024-04-22T19:58:45.025 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.025+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=N9PAA4HJ7JFK type=del size=1105462 2024-04-22T19:58:45.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.026+0000 7f5269a8e780 1 < canceled operation key=SN2BZATJGUWA tag=F6SIHGM9YWV8 type=del size=2855649 2024-04-22T19:58:45.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.027+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=OLD93SU76LK0 type=del size=2318331 2024-04-22T19:58:45.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.028+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~2EST76XDDDNQ.0 size=1048576 2024-04-22T19:58:45.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.029+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~2EST76XDDDNQ.1 size=1048576 2024-04-22T19:58:45.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.030+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~2EST76XDDDNQ.2 size=328140 2024-04-22T19:58:45.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.031+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=2EST76XDDDNQ tag=TYN05LMLNI7P type=write size=3473868 2024-04-22T19:58:45.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.032+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=8Y1OTNIK9FLN type=del size=1626996 2024-04-22T19:58:45.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.034+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=Y6PPOZ3EYZL8 type=del 2024-04-22T19:58:45.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.035+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=GRHFI1Y1O8IO type=write size=35550 2024-04-22T19:58:45.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.036+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~DEXPPFS5DE2D.0 size=965674 2024-04-22T19:58:45.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.038+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=DEXPPFS5DE2D tag=LPKFSD4MUK4S type=write size=2014250 2024-04-22T19:58:45.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.039+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=5NI9IAI7ZJPJ type=del size=834684 2024-04-22T19:58:45.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.041+0000 7f5269a8e780 1 < suggested update operation key=2YFD2U6HVNE7 tag=MWWER5V2P1I7 type=write size=762893 2024-04-22T19:58:45.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.042+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=UAB7SEKL2GQB type=write size=919732 2024-04-22T19:58:45.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.045+0000 7f5269a8e780 1 < suggested remove operation key=A853MOZPT9BG tag=PFG8I0KFZSQB type=write 2024-04-22T19:58:45.047 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.047+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~IW5K72IH1WFM.0 size=1048576 2024-04-22T19:58:45.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.048+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~IW5K72IH1WFM.1 size=1048576 2024-04-22T19:58:45.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.049+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~IW5K72IH1WFM.2 size=728288 2024-04-22T19:58:45.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.049+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=IW5K72IH1WFM tag=48ULI0UGTNN8 type=write size=3874016 2024-04-22T19:58:45.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.051+0000 7f5269a8e780 1 < suggested remove operation key=HV8J6YM0YNC1 tag=CQO8NZ8MBJMA type=del 2024-04-22T19:58:45.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.052+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=3RKBSW73ZQ8G type=del size=1453724 2024-04-22T19:58:45.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.054+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~4C9EK64OB6NG.0 size=1048576 2024-04-22T19:58:45.054 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.055+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~4C9EK64OB6NG.1 size=581762 2024-04-22T19:58:45.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.057+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=4C9EK64OB6NG tag=IQOPNHQBKSZ9 type=write size=2678914 2024-04-22T19:58:45.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.058+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=CGPUBA98DTN1 type=write size=32132 2024-04-22T19:58:45.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.059+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=SVF82WB3R6PZ type=write size=10622 2024-04-22T19:58:45.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.060+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=HEQEARJRF8MV type=del size=3401101 2024-04-22T19:58:45.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.061+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=13PQ0C15DE1H type=del 2024-04-22T19:58:45.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.062+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=X4BBXE2JZRXE type=del size=758909 2024-04-22T19:58:45.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.063+0000 7f5269a8e780 1 < canceled operation key=I91AD8FJ3P4T tag=XXHJUK3UPT89 type=del size=3614132 2024-04-22T19:58:45.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.064+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=DG2FU4OQQHYZ type=del size=42301 2024-04-22T19:58:45.066 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.067+0000 7f5269a8e780 1 < suggested update operation key=5UBYMQT7ZWKX tag=L7RK3QIS7XCS type=del size=2892270 2024-04-22T19:58:45.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.068+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=A8Y8Y8EKHIIU type=del size=516367 2024-04-22T19:58:45.069 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.069+0000 7f5269a8e780 1 < canceled operation key=AMBJHMJGLUUS tag=JEMX4SEN7VMB type=del size=1997567 2024-04-22T19:58:45.070 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.070+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=ITRZEPWZFHXL type=write size=134600 2024-04-22T19:58:45.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.071+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~ILFGFZ1T2FL9.0 size=1048576 2024-04-22T19:58:45.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.072+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~ILFGFZ1T2FL9.1 size=918224 2024-04-22T19:58:45.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.074+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=ILFGFZ1T2FL9 tag=OAQEVAK2QOLZ type=write size=3015376 2024-04-22T19:58:45.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.076+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~3KP6BJ1HZ69G.0 size=658042 2024-04-22T19:58:45.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.076+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=3KP6BJ1HZ69G tag=RE77I2R3XQT3 type=write size=1706618 2024-04-22T19:58:45.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.078+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~RAXT7LX827SO.0 size=1048576 2024-04-22T19:58:45.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.078+0000 7f5269a8e780 1 < canceled part key=_multipart_ZC4ZOHUJ3HQD.2~RAXT7LX827SO.1 size=247347 2024-04-22T19:58:45.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.080+0000 7f5269a8e780 1 < canceled multipart upload key=ZC4ZOHUJ3HQD upload=RAXT7LX827SO tag=MYTBUVTD2OTT type=write size=2344499 2024-04-22T19:58:45.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.081+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=QL6HSNICUTTD type=del size=3681985 2024-04-22T19:58:45.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.083+0000 7f5269a8e780 1 < suggested remove operation key=Q1UKF4UMJ2HO tag=N9PAA4HJ7JFK type=del 2024-04-22T19:58:45.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.085+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=DSB0YKEMWKHB type=write size=369353 2024-04-22T19:58:45.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.087+0000 7f5269a8e780 1 < suggested update operation key=HIAXPVKZ1IIO tag=OLD93SU76LK0 type=del size=2318331 2024-04-22T19:58:45.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.088+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=42FQ2NANNZXT type=del size=2315117 2024-04-22T19:58:45.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.090+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=8Y1OTNIK9FLN type=del 2024-04-22T19:58:45.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.091+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=H79QEJLVJ8UA type=del size=260981 2024-04-22T19:58:45.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.092+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=GRHFI1Y1O8IO type=write size=35550 2024-04-22T19:58:45.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.093+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=H9L9ECHPVBVP type=del size=3679566 2024-04-22T19:58:45.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.094+0000 7f5269a8e780 -1 < failed to complete operation key=HV8J6YM0YNC1 tag=5NI9IAI7ZJPJ type=del size=834684: (22) Invalid argument 2024-04-22T19:58:45.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.095+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=2ZGU037OGIH1 type=write size=216187 2024-04-22T19:58:45.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.096+0000 7f5269a8e780 1 < completed write operation key=637M7AP8X9E9 tag=UAB7SEKL2GQB type=write size=919732 2024-04-22T19:58:45.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.098+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=3P927BT9BOCK type=del size=3300972 2024-04-22T19:58:45.099 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.099+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~IW5K72IH1WFM.0 size=1048576 2024-04-22T19:58:45.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.100+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~IW5K72IH1WFM.1 size=1048576 2024-04-22T19:58:45.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.101+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~IW5K72IH1WFM.2 size=728288 2024-04-22T19:58:45.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.103+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=IW5K72IH1WFM tag=48ULI0UGTNN8 type=write size=3874016 2024-04-22T19:58:45.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.105+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~5811MGCK0RH4.0 size=1048576 2024-04-22T19:58:45.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.106+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~5811MGCK0RH4.1 size=1048576 2024-04-22T19:58:45.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.107+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~5811MGCK0RH4.2 size=1043444 2024-04-22T19:58:45.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.107+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=5811MGCK0RH4 tag=UDVOYGWFZ57S type=write size=4189172 2024-04-22T19:58:45.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.108+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=3RKBSW73ZQ8G type=del 2024-04-22T19:58:45.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.110+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~OLP08SSZPYIC.0 size=868215 2024-04-22T19:58:45.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.110+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=OLP08SSZPYIC tag=IETQPYO9LMZV type=write size=1916791 2024-04-22T19:58:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.111+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=CGPUBA98DTN1 type=write size=32132 2024-04-22T19:58:45.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.112+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=AXKS9FE68BNS type=del size=3591892 2024-04-22T19:58:45.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.113+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=HEQEARJRF8MV type=del 2024-04-22T19:58:45.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.115+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~0TKCQA1ABIID.0 size=1048576 2024-04-22T19:58:45.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.116+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~0TKCQA1ABIID.1 size=1048576 2024-04-22T19:58:45.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.117+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~0TKCQA1ABIID.2 size=422536 2024-04-22T19:58:45.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.117+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=0TKCQA1ABIID tag=I2AOG844YJVW type=write size=3568264 2024-04-22T19:58:45.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.118+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=X4BBXE2JZRXE type=del 2024-04-22T19:58:45.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.120+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~C0PBZBW8DKNX.0 size=305062 2024-04-22T19:58:45.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.120+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=C0PBZBW8DKNX tag=0NMJD42TL5GG type=write size=1353638 2024-04-22T19:58:45.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.122+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=DG2FU4OQQHYZ type=del 2024-04-22T19:58:45.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.124+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~3EFW02Z6YFFC.0 size=229754 2024-04-22T19:58:45.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.124+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=3EFW02Z6YFFC tag=GBW5TUFS0COD type=write size=1278330 2024-04-22T19:58:45.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.126+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=A8Y8Y8EKHIIU type=del 2024-04-22T19:58:45.128 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.128+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=WXYQTUAHY18Q type=del size=136775 2024-04-22T19:58:45.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.129+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=ITRZEPWZFHXL type=write size=134600 2024-04-22T19:58:45.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.130+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=VA36B39WECQZ type=del size=964654 2024-04-22T19:58:45.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.131+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~3KP6BJ1HZ69G.0 size=658042 2024-04-22T19:58:45.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.133+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=3KP6BJ1HZ69G tag=RE77I2R3XQT3 type=write size=1706618 2024-04-22T19:58:45.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.135+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=WVHL8LCJJE3H type=del size=2842544 2024-04-22T19:58:45.137 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.137+0000 7f5269a8e780 1 < suggested update operation key=8V6ESIGMRO5G tag=QL6HSNICUTTD type=del size=3681985 2024-04-22T19:58:45.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.138+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=LSEUWDWAY7HN type=del size=1955663 2024-04-22T19:58:45.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.141+0000 7f5269a8e780 1 < suggested remove operation key=HV8J6YM0YNC1 tag=DSB0YKEMWKHB type=write 2024-04-22T19:58:45.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.142+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=AX83064BAOOT type=del size=1135122 2024-04-22T19:58:45.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.143+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=42FQ2NANNZXT type=del 2024-04-22T19:58:45.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.145+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~T9GY5EQKJOQN.0 size=895045 2024-04-22T19:58:45.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.145+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=T9GY5EQKJOQN tag=6VXQTAB3RN72 type=write size=1943621 2024-04-22T19:58:45.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.146+0000 7f5269a8e780 1 < canceled operation key=4GX83ZJWZCD1 tag=H79QEJLVJ8UA type=del size=260981 2024-04-22T19:58:45.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.147+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=YTURX2XE6DI9 type=del size=3187232 2024-04-22T19:58:45.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.148+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=H9L9ECHPVBVP type=del 2024-04-22T19:58:45.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.152+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~O2JGNSLFH8BR.0 size=1048576 2024-04-22T19:58:45.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.153+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~O2JGNSLFH8BR.1 size=1048576 2024-04-22T19:58:45.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.154+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~O2JGNSLFH8BR.2 size=118609 2024-04-22T19:58:45.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.154+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=O2JGNSLFH8BR tag=D66TFAX6OI1V type=write size=3264337 2024-04-22T19:58:45.155 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.155+0000 7f5269a8e780 1 < completed write operation key=4EA3LUQRL8II tag=2ZGU037OGIH1 type=write size=216187 2024-04-22T19:58:45.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.156+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=DJARQPK2WUTO type=del size=2555402 2024-04-22T19:58:45.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.157+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=3P927BT9BOCK type=del 2024-04-22T19:58:45.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.159+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=4PESXHKQTJNW type=del size=3634608 2024-04-22T19:58:45.160 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.160+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~5811MGCK0RH4.0 size=1048576 2024-04-22T19:58:45.161 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.161+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~5811MGCK0RH4.1 size=1048576 2024-04-22T19:58:45.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.162+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~5811MGCK0RH4.2 size=1043444 2024-04-22T19:58:45.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.164+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=5811MGCK0RH4 tag=UDVOYGWFZ57S type=write size=4189172 2024-04-22T19:58:45.165 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.165+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=DNQ7FU0ZHC4H type=del size=1867451 2024-04-22T19:58:45.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.166+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~OLP08SSZPYIC.0 size=868215 2024-04-22T19:58:45.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.168+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=OLP08SSZPYIC tag=IETQPYO9LMZV type=write size=1916791 2024-04-22T19:58:45.169 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.169+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=E26XVWIOC9UI type=del size=1000961 2024-04-22T19:58:45.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.172+0000 7f5269a8e780 1 < suggested update operation key=VPAVX5FXYWZO tag=AXKS9FE68BNS type=del size=3591892 2024-04-22T19:58:45.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.174+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~T4BXE0HNU3ZH.0 size=1048576 2024-04-22T19:58:45.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.175+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~T4BXE0HNU3ZH.1 size=1048576 2024-04-22T19:58:45.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.175+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~T4BXE0HNU3ZH.2 size=596588 2024-04-22T19:58:45.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.175+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=T4BXE0HNU3ZH tag=X7AWQLXBN83N type=write size=3742316 2024-04-22T19:58:45.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.177+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~0TKCQA1ABIID.0 size=1048576 2024-04-22T19:58:45.178 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.178+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~0TKCQA1ABIID.1 size=1048576 2024-04-22T19:58:45.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.179+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~0TKCQA1ABIID.2 size=422536 2024-04-22T19:58:45.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.181+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=0TKCQA1ABIID tag=I2AOG844YJVW type=write size=3568264 2024-04-22T19:58:45.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.183+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~23W2VG1ZEZGB.0 size=1048576 2024-04-22T19:58:45.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.183+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~23W2VG1ZEZGB.1 size=1048576 2024-04-22T19:58:45.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.184+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~23W2VG1ZEZGB.2 size=975324 2024-04-22T19:58:45.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.184+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=23W2VG1ZEZGB tag=HS7O49VEDWPO type=write size=4121052 2024-04-22T19:58:45.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.186+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~C0PBZBW8DKNX.0 size=305062 2024-04-22T19:58:45.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.188+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=C0PBZBW8DKNX tag=0NMJD42TL5GG type=write size=1353638 2024-04-22T19:58:45.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.189+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=K2TXIY0ZEVY3 type=write size=1007510 2024-04-22T19:58:45.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.190+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~3EFW02Z6YFFC.0 size=229754 2024-04-22T19:58:45.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.192+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=3EFW02Z6YFFC tag=GBW5TUFS0COD type=write size=1278330 2024-04-22T19:58:45.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.193+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=0LTL3RBAUUY2 type=del size=1104383 2024-04-22T19:58:45.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.194+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=WXYQTUAHY18Q type=del 2024-04-22T19:58:45.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.196+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=2U3YKZQY9QRD type=del size=3873311 2024-04-22T19:58:45.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.197+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=VA36B39WECQZ type=del 2024-04-22T19:58:45.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.198+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=6V1T5MJQRB52 type=del size=1789467 2024-04-22T19:58:45.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.199+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=WVHL8LCJJE3H type=del 2024-04-22T19:58:45.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.202+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~9GQV0V6IX65Q.0 size=1048576 2024-04-22T19:58:45.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.203+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~9GQV0V6IX65Q.1 size=150023 2024-04-22T19:58:45.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.203+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=9GQV0V6IX65Q tag=AOJPSJA8B56S type=write size=2247175 2024-04-22T19:58:45.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.204+0000 7f5269a8e780 1 < canceled operation key=I91AD8FJ3P4T tag=LSEUWDWAY7HN type=del size=1955663 2024-04-22T19:58:45.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.206+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~THIE8THT79QD.0 size=50685 2024-04-22T19:58:45.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.206+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=THIE8THT79QD tag=JC6XHA4V26F8 type=write size=1099261 2024-04-22T19:58:45.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.208+0000 7f5269a8e780 1 < suggested remove operation key=GRSL48RUEBX8 tag=AX83064BAOOT type=del 2024-04-22T19:58:45.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.209+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=0XLTT7LTIV8W type=write size=153073 2024-04-22T19:58:45.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.210+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~T9GY5EQKJOQN.0 size=895045 2024-04-22T19:58:45.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.212+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=T9GY5EQKJOQN tag=6VXQTAB3RN72 type=write size=1943621 2024-04-22T19:58:45.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.214+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=5AQL4GCSBW5A type=write size=142349 2024-04-22T19:58:45.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.216+0000 7f5269a8e780 1 < suggested remove operation key=GX6NILQG878S tag=YTURX2XE6DI9 type=del 2024-04-22T19:58:45.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.218+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~HZUC6KZB73K0.0 size=383964 2024-04-22T19:58:45.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.218+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=HZUC6KZB73K0 tag=LFG7ZWANLED9 type=write size=1432540 2024-04-22T19:58:45.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.219+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~O2JGNSLFH8BR.0 size=1048576 2024-04-22T19:58:45.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.220+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~O2JGNSLFH8BR.1 size=1048576 2024-04-22T19:58:45.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.221+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~O2JGNSLFH8BR.2 size=118609 2024-04-22T19:58:45.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.223+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=O2JGNSLFH8BR tag=D66TFAX6OI1V type=write size=3264337 2024-04-22T19:58:45.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.224+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=EQSVEOD2QROB type=del size=3111620 2024-04-22T19:58:45.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.226+0000 7f5269a8e780 1 < suggested remove operation key=HIAXPVKZ1IIO tag=DJARQPK2WUTO type=del 2024-04-22T19:58:45.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.228+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=WFMHTND00HR6 type=del size=1242096 2024-04-22T19:58:45.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.229+0000 7f5269a8e780 -1 < failed to complete operation key=HIAXPVKZ1IIO tag=4PESXHKQTJNW type=del size=3634608: (22) Invalid argument 2024-04-22T19:58:45.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.231+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~697C4O3RUZT5.0 size=1048576 2024-04-22T19:58:45.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.232+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~697C4O3RUZT5.1 size=1048576 2024-04-22T19:58:45.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.233+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~697C4O3RUZT5.2 size=847879 2024-04-22T19:58:45.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.233+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=697C4O3RUZT5 tag=WGYIG5GY67E3 type=write size=3993607 2024-04-22T19:58:45.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.234+0000 7f5269a8e780 -1 < failed to complete operation key=HIAXPVKZ1IIO tag=DNQ7FU0ZHC4H type=del size=1867451: (22) Invalid argument 2024-04-22T19:58:45.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.235+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=DFR7556RD1S8 type=del size=319242 2024-04-22T19:58:45.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.236+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=E26XVWIOC9UI type=del 2024-04-22T19:58:45.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.238+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=LVKF2Z3IY43V type=del size=3691602 2024-04-22T19:58:45.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.240+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~T4BXE0HNU3ZH.0 size=1048576 2024-04-22T19:58:45.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.241+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~T4BXE0HNU3ZH.1 size=1048576 2024-04-22T19:58:45.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.241+0000 7f5269a8e780 1 < canceled part key=_multipart_Q1UKF4UMJ2HO.2~T4BXE0HNU3ZH.2 size=596588 2024-04-22T19:58:45.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.243+0000 7f5269a8e780 1 < canceled multipart upload key=Q1UKF4UMJ2HO upload=T4BXE0HNU3ZH tag=X7AWQLXBN83N type=write size=3742316 2024-04-22T19:58:45.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.244+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=2ZKQY57O8J1U type=del size=250031 2024-04-22T19:58:45.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.245+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~23W2VG1ZEZGB.0 size=1048576 2024-04-22T19:58:45.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.246+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~23W2VG1ZEZGB.1 size=1048576 2024-04-22T19:58:45.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.247+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~23W2VG1ZEZGB.2 size=975324 2024-04-22T19:58:45.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.249+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=23W2VG1ZEZGB tag=HS7O49VEDWPO type=write size=4121052 2024-04-22T19:58:45.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.251+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~TVSE18BRZ0XI.0 size=1048576 2024-04-22T19:58:45.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.252+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~TVSE18BRZ0XI.1 size=993007 2024-04-22T19:58:45.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.252+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=TVSE18BRZ0XI tag=JAZY0L4B2FMU type=write size=3090159 2024-04-22T19:58:45.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.253+0000 7f5269a8e780 1 < completed write operation key=EHRS6DQDRXVS tag=K2TXIY0ZEVY3 type=write size=1007510 2024-04-22T19:58:45.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.254+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=9TSMV6YA9VTZ type=del size=3414534 2024-04-22T19:58:45.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.255+0000 7f5269a8e780 -1 < failed to complete operation key=GRSL48RUEBX8 tag=0LTL3RBAUUY2 type=del size=1104383: (22) Invalid argument 2024-04-22T19:58:45.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.257+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~51IB2ZJ3II95.0 size=987802 2024-04-22T19:58:45.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.257+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=51IB2ZJ3II95 tag=WK8ULNHOKXCA type=write size=2036378 2024-04-22T19:58:45.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.258+0000 7f5269a8e780 1 < canceled operation key=Q1UKF4UMJ2HO tag=2U3YKZQY9QRD type=del size=3873311 2024-04-22T19:58:45.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.260+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~UOVV47T7UP64.0 size=1048576 2024-04-22T19:58:45.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.261+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~UOVV47T7UP64.1 size=1048576 2024-04-22T19:58:45.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.262+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~UOVV47T7UP64.2 size=360731 2024-04-22T19:58:45.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.262+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=UOVV47T7UP64 tag=3RHO00YCH03A type=write size=3506459 2024-04-22T19:58:45.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.265+0000 7f5269a8e780 1 < suggested remove operation key=4GX83ZJWZCD1 tag=6V1T5MJQRB52 type=del 2024-04-22T19:58:45.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.266+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=7YLL54ZN59OQ type=del size=747864 2024-04-22T19:58:45.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.267+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~9GQV0V6IX65Q.0 size=1048576 2024-04-22T19:58:45.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.268+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~9GQV0V6IX65Q.1 size=150023 2024-04-22T19:58:45.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.270+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=9GQV0V6IX65Q tag=AOJPSJA8B56S type=write size=2247175 2024-04-22T19:58:45.271 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.271+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=AQUX2D15T126 type=del size=3666703 2024-04-22T19:58:45.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.272+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~THIE8THT79QD.0 size=50685 2024-04-22T19:58:45.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.274+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=THIE8THT79QD tag=JC6XHA4V26F8 type=write size=1099261 2024-04-22T19:58:45.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.276+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~A1VML5TQGZD1.0 size=1048576 2024-04-22T19:58:45.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.277+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~A1VML5TQGZD1.1 size=1048576 2024-04-22T19:58:45.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.278+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~A1VML5TQGZD1.2 size=253846 2024-04-22T19:58:45.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.278+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=A1VML5TQGZD1 tag=I6P0FOOC08J1 type=write size=3399574 2024-04-22T19:58:45.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.279+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=0XLTT7LTIV8W type=write size=153073 2024-04-22T19:58:45.280 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.280+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=6DORYZ8VFI6L type=del size=260600 2024-04-22T19:58:45.282 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.282+0000 7f5269a8e780 1 < suggested remove operation key=5UBYMQT7ZWKX tag=5AQL4GCSBW5A type=write 2024-04-22T19:58:45.284 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.284+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~F917K2NPFOHK.0 size=1048576 2024-04-22T19:58:45.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.285+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~F917K2NPFOHK.1 size=661355 2024-04-22T19:58:45.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.285+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=F917K2NPFOHK tag=LJENUNZU4GN4 type=write size=2758507 2024-04-22T19:58:45.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.285+0000 7f5269a8e780 1 < canceled part key=_multipart_Q1UKF4UMJ2HO.2~HZUC6KZB73K0.0 size=383964 2024-04-22T19:58:45.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.287+0000 7f5269a8e780 1 < canceled multipart upload key=Q1UKF4UMJ2HO upload=HZUC6KZB73K0 tag=LFG7ZWANLED9 type=write size=1432540 2024-04-22T19:58:45.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.289+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=90AS7A4RDUKA type=del size=1421688 2024-04-22T19:58:45.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.290+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=EQSVEOD2QROB type=del 2024-04-22T19:58:45.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.291+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=QL448KFVH7Q4 type=del size=23081 2024-04-22T19:58:45.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.292+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=WFMHTND00HR6 type=del 2024-04-22T19:58:45.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.294+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=C03KD8BTV40M type=del size=1745762 2024-04-22T19:58:45.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.295+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~697C4O3RUZT5.0 size=1048576 2024-04-22T19:58:45.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.296+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~697C4O3RUZT5.1 size=1048576 2024-04-22T19:58:45.297 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.297+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~697C4O3RUZT5.2 size=847879 2024-04-22T19:58:45.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.299+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=697C4O3RUZT5 tag=WGYIG5GY67E3 type=write size=3993607 2024-04-22T19:58:45.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.300+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=YVSMYKLJ0LAS type=write size=233295 2024-04-22T19:58:45.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.301+0000 7f5269a8e780 1 < canceled operation key=Y5HR8XW4L0SP tag=DFR7556RD1S8 type=del size=319242 2024-04-22T19:58:45.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.303+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~O7FZVDVJNXD9.0 size=1048576 2024-04-22T19:58:45.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.304+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~O7FZVDVJNXD9.1 size=40768 2024-04-22T19:58:45.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.304+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=O7FZVDVJNXD9 tag=8RJIAC77LKSM type=write size=2137920 2024-04-22T19:58:45.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.307+0000 7f5269a8e780 1 < suggested update operation key=O9NFAVE09ZN7 tag=LVKF2Z3IY43V type=del size=3691602 2024-04-22T19:58:45.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.308+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=QK5TE133O1PP type=del size=2685909 2024-04-22T19:58:45.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.309+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=2ZKQY57O8J1U type=del 2024-04-22T19:58:45.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.311+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~K78C19G1MLTM.0 size=1048576 2024-04-22T19:58:45.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.312+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~K78C19G1MLTM.1 size=1048576 2024-04-22T19:58:45.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.313+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~K78C19G1MLTM.2 size=942313 2024-04-22T19:58:45.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.313+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=K78C19G1MLTM tag=1K53BKS6FD8A type=write size=4088041 2024-04-22T19:58:45.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.314+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~TVSE18BRZ0XI.0 size=1048576 2024-04-22T19:58:45.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.315+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~TVSE18BRZ0XI.1 size=993007 2024-04-22T19:58:45.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.317+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=TVSE18BRZ0XI tag=JAZY0L4B2FMU type=write size=3090159 2024-04-22T19:58:45.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.319+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=IZKQ3A1LD3IE type=del size=2859984 2024-04-22T19:58:45.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.320+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=9TSMV6YA9VTZ type=del 2024-04-22T19:58:45.321 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.321+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=0M76R18B0UKM type=del size=775028 2024-04-22T19:58:45.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.322+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~51IB2ZJ3II95.0 size=987802 2024-04-22T19:58:45.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.324+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=51IB2ZJ3II95 tag=WK8ULNHOKXCA type=write size=2036378 2024-04-22T19:58:45.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.326+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=1LK5CLR8KOVY type=del size=3484960 2024-04-22T19:58:45.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.327+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~UOVV47T7UP64.0 size=1048576 2024-04-22T19:58:45.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.328+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~UOVV47T7UP64.1 size=1048576 2024-04-22T19:58:45.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.329+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~UOVV47T7UP64.2 size=360731 2024-04-22T19:58:45.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.331+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=UOVV47T7UP64 tag=3RHO00YCH03A type=write size=3506459 2024-04-22T19:58:45.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.332+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=OMDQFE5X09Y9 type=write size=117410 2024-04-22T19:58:45.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.333+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=7YLL54ZN59OQ type=del 2024-04-22T19:58:45.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.335+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~2TAZ9WFYJMJM.0 size=1048576 2024-04-22T19:58:45.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.336+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~2TAZ9WFYJMJM.1 size=492439 2024-04-22T19:58:45.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.336+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=2TAZ9WFYJMJM tag=KJMXGJZGD63P type=write size=2589591 2024-04-22T19:58:45.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.337+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=AQUX2D15T126 type=del 2024-04-22T19:58:45.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.339+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~AG99RIL17KOO.0 size=1048576 2024-04-22T19:58:45.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.340+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~AG99RIL17KOO.1 size=127305 2024-04-22T19:58:45.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.340+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=AG99RIL17KOO tag=G20CA5URWC1H type=write size=2224457 2024-04-22T19:58:45.341 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.341+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~A1VML5TQGZD1.0 size=1048576 2024-04-22T19:58:45.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.342+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~A1VML5TQGZD1.1 size=1048576 2024-04-22T19:58:45.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.343+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~A1VML5TQGZD1.2 size=253846 2024-04-22T19:58:45.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.346+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=A1VML5TQGZD1 tag=I6P0FOOC08J1 type=write size=3399574 2024-04-22T19:58:45.348 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.348+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=4UOS17QHP5S3 type=del size=3966699 2024-04-22T19:58:45.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.349+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=6DORYZ8VFI6L type=del 2024-04-22T19:58:45.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.351+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~JLBW2ILS2PH3.0 size=1048576 2024-04-22T19:58:45.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.352+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~JLBW2ILS2PH3.1 size=537950 2024-04-22T19:58:45.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.352+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=JLBW2ILS2PH3 tag=WVEU81XB3YOY type=write size=2635102 2024-04-22T19:58:45.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.353+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~F917K2NPFOHK.0 size=1048576 2024-04-22T19:58:45.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.354+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~F917K2NPFOHK.1 size=661355 2024-04-22T19:58:45.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.356+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=F917K2NPFOHK tag=LJENUNZU4GN4 type=write size=2758507 2024-04-22T19:58:45.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.357+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=305JWP3EVW4I type=del size=332228 2024-04-22T19:58:45.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.358+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=90AS7A4RDUKA type=del 2024-04-22T19:58:45.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.360+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~WWWB2Z1D9A6X.0 size=986293 2024-04-22T19:58:45.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.360+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=WWWB2Z1D9A6X tag=JOKNNJC4G8X3 type=write size=2034869 2024-04-22T19:58:45.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.361+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=QL448KFVH7Q4 type=del 2024-04-22T19:58:45.362 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.363+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=4ZU0XT8N8MV9 type=write size=75781 2024-04-22T19:58:45.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.364+0000 7f5269a8e780 1 < canceled operation key=70MLQGJPQYPY tag=C03KD8BTV40M type=del size=1745762 2024-04-22T19:58:45.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.365+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=VFHMT4P5GR58 type=del size=4172458 2024-04-22T19:58:45.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.366+0000 7f5269a8e780 1 < completed write operation key=Q1UKF4UMJ2HO tag=YVSMYKLJ0LAS type=write size=233295 2024-04-22T19:58:45.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.368+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~7FGAWJLAE80N.0 size=1048576 2024-04-22T19:58:45.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.369+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~7FGAWJLAE80N.1 size=132521 2024-04-22T19:58:45.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.369+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=7FGAWJLAE80N tag=8S7Z0WT6S3AM type=write size=2229673 2024-04-22T19:58:45.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.369+0000 7f5269a8e780 1 < canceled part key=_multipart_DX5YUYQRZKWA.2~O7FZVDVJNXD9.0 size=1048576 2024-04-22T19:58:45.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.369+0000 7f5269a8e780 1 < canceled part key=_multipart_DX5YUYQRZKWA.2~O7FZVDVJNXD9.1 size=40768 2024-04-22T19:58:45.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.371+0000 7f5269a8e780 1 < canceled multipart upload key=DX5YUYQRZKWA upload=O7FZVDVJNXD9 tag=8RJIAC77LKSM type=write size=2137920 2024-04-22T19:58:45.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.374+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~AO1UV04J2TLH.0 size=1048576 2024-04-22T19:58:45.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.375+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~AO1UV04J2TLH.1 size=925886 2024-04-22T19:58:45.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.375+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=AO1UV04J2TLH tag=Z38AGIPUS93Z type=write size=3023038 2024-04-22T19:58:45.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.377+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=QK5TE133O1PP type=del 2024-04-22T19:58:45.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.378+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=11JM3IIU9I6P type=del size=876496 2024-04-22T19:58:45.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.379+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~K78C19G1MLTM.0 size=1048576 2024-04-22T19:58:45.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.380+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~K78C19G1MLTM.1 size=1048576 2024-04-22T19:58:45.381 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.381+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~K78C19G1MLTM.2 size=942313 2024-04-22T19:58:45.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.383+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=K78C19G1MLTM tag=1K53BKS6FD8A type=write size=4088041 2024-04-22T19:58:45.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.385+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~QZ2IC0GD5UC3.0 size=320398 2024-04-22T19:58:45.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.385+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=QZ2IC0GD5UC3 tag=L0AXFJH1WZMS type=write size=1368974 2024-04-22T19:58:45.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.386+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=IZKQ3A1LD3IE type=del 2024-04-22T19:58:45.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.388+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=MSYJ3M37KGIN type=del size=4153643 2024-04-22T19:58:45.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.389+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=0M76R18B0UKM type=del 2024-04-22T19:58:45.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.391+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~A3R7I975ITRK.0 size=320495 2024-04-22T19:58:45.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.391+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=A3R7I975ITRK tag=QK24B76GDBGV type=write size=1369071 2024-04-22T19:58:45.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.393+0000 7f5269a8e780 1 < suggested remove operation key=I91AD8FJ3P4T tag=1LK5CLR8KOVY type=del 2024-04-22T19:58:45.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.394+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=DUKULFFXC7F4 type=del size=909658 2024-04-22T19:58:45.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.395+0000 7f5269a8e780 1 < completed write operation key=O9NFAVE09ZN7 tag=OMDQFE5X09Y9 type=write size=117410 2024-04-22T19:58:45.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.397+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~LKX83LT91WCA.0 size=900521 2024-04-22T19:58:45.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.397+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=LKX83LT91WCA tag=O5GS8RCNKXPD type=write size=1949097 2024-04-22T19:58:45.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.398+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~2TAZ9WFYJMJM.0 size=1048576 2024-04-22T19:58:45.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.399+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~2TAZ9WFYJMJM.1 size=492439 2024-04-22T19:58:45.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.401+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=2TAZ9WFYJMJM tag=KJMXGJZGD63P type=write size=2589591 2024-04-22T19:58:45.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.403+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~DAM06X9UQGY0.0 size=847507 2024-04-22T19:58:45.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.403+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=DAM06X9UQGY0 tag=G1FS0TTN5P1K type=write size=1896083 2024-04-22T19:58:45.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.404+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~AG99RIL17KOO.0 size=1048576 2024-04-22T19:58:45.404 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.405+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~AG99RIL17KOO.1 size=127305 2024-04-22T19:58:45.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.406+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=AG99RIL17KOO tag=G20CA5URWC1H type=write size=2224457 2024-04-22T19:58:45.408 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.408+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~TTZQK79NTPZT.0 size=1048576 2024-04-22T19:58:45.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.409+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~TTZQK79NTPZT.1 size=1048576 2024-04-22T19:58:45.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.410+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~TTZQK79NTPZT.2 size=49065 2024-04-22T19:58:45.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.410+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=TTZQK79NTPZT tag=B0DX5EKPWXDU type=write size=3194793 2024-04-22T19:58:45.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.411+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=4UOS17QHP5S3 type=del 2024-04-22T19:58:45.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.413+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~4BMG7SXFCXNK.0 size=1048576 2024-04-22T19:58:45.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.414+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~4BMG7SXFCXNK.1 size=1048576 2024-04-22T19:58:45.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.415+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~4BMG7SXFCXNK.2 size=132510 2024-04-22T19:58:45.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.415+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=4BMG7SXFCXNK tag=SKNRE72IDD0G type=write size=3278238 2024-04-22T19:58:45.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.416+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~JLBW2ILS2PH3.0 size=1048576 2024-04-22T19:58:45.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.417+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~JLBW2ILS2PH3.1 size=537950 2024-04-22T19:58:45.418 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.419+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=JLBW2ILS2PH3 tag=WVEU81XB3YOY type=write size=2635102 2024-04-22T19:58:45.420 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.420+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=8T8JN52CY9YD type=del size=3446745 2024-04-22T19:58:45.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.421+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=305JWP3EVW4I type=del 2024-04-22T19:58:45.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.423+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~LVH6ZGORU121.0 size=1048576 2024-04-22T19:58:45.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.424+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~LVH6ZGORU121.1 size=783668 2024-04-22T19:58:45.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.424+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=LVH6ZGORU121 tag=DIEC00CL8KAI type=write size=2880820 2024-04-22T19:58:45.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.425+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~WWWB2Z1D9A6X.0 size=986293 2024-04-22T19:58:45.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.427+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=WWWB2Z1D9A6X tag=JOKNNJC4G8X3 type=write size=2034869 2024-04-22T19:58:45.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.428+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=89FX2H3UJXG3 type=del size=2803171 2024-04-22T19:58:45.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.430+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=4ZU0XT8N8MV9 type=write size=75781 2024-04-22T19:58:45.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.431+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=A1EGTDJJWJAD type=del size=1518837 2024-04-22T19:58:45.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.433+0000 7f5269a8e780 1 < suggested remove operation key=86GW6HV9G1IZ tag=VFHMT4P5GR58 type=del 2024-04-22T19:58:45.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.434+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=Z6A533FMDN9U type=del size=2721654 2024-04-22T19:58:45.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.435+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~7FGAWJLAE80N.0 size=1048576 2024-04-22T19:58:45.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.437+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~7FGAWJLAE80N.1 size=132521 2024-04-22T19:58:45.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.438+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=7FGAWJLAE80N tag=8S7Z0WT6S3AM type=write size=2229673 2024-04-22T19:58:45.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.440+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~S1Q9PJ2H38IZ.0 size=1048576 2024-04-22T19:58:45.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.441+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~S1Q9PJ2H38IZ.1 size=1048576 2024-04-22T19:58:45.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.442+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~S1Q9PJ2H38IZ.2 size=955497 2024-04-22T19:58:45.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.442+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=S1Q9PJ2H38IZ tag=GEW2GWFEE1CF type=write size=4101225 2024-04-22T19:58:45.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.443+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~AO1UV04J2TLH.0 size=1048576 2024-04-22T19:58:45.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.444+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~AO1UV04J2TLH.1 size=925886 2024-04-22T19:58:45.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.446+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=AO1UV04J2TLH tag=Z38AGIPUS93Z type=write size=3023038 2024-04-22T19:58:45.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.448+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~LHJMCZ0L0BXE.0 size=250161 2024-04-22T19:58:45.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.448+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=LHJMCZ0L0BXE tag=J20OS9KZ2X2H type=write size=1298737 2024-04-22T19:58:45.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.449+0000 7f5269a8e780 1 < canceled operation key=M9M4SEEEZSFV tag=11JM3IIU9I6P type=del size=876496 2024-04-22T19:58:45.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.450+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=F8VBBY4VOD03 type=write size=23538 2024-04-22T19:58:45.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.453+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~QZ2IC0GD5UC3.0 size=320398 2024-04-22T19:58:45.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.455+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=QZ2IC0GD5UC3 tag=L0AXFJH1WZMS type=write size=1368974 2024-04-22T19:58:45.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.456+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=C7QVULANH8Q1 type=del size=2289851 2024-04-22T19:58:45.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.458+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=MSYJ3M37KGIN type=del 2024-04-22T19:58:45.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.460+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~W8XIX0KK0TYB.0 size=268541 2024-04-22T19:58:45.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.460+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=W8XIX0KK0TYB tag=5061GYG4K6UN type=write size=1317117 2024-04-22T19:58:45.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.461+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~A3R7I975ITRK.0 size=320495 2024-04-22T19:58:45.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.463+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=A3R7I975ITRK tag=QK24B76GDBGV type=write size=1369071 2024-04-22T19:58:45.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.464+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~KSP8W5WJNV78.0 size=878663 2024-04-22T19:58:45.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.464+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=KSP8W5WJNV78 tag=WCC3MB8ZTFJI type=write size=1927239 2024-04-22T19:58:45.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.466+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=DUKULFFXC7F4 type=del 2024-04-22T19:58:45.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.467+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=W5NX10OT644M type=del size=1958640 2024-04-22T19:58:45.468 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.468+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~LKX83LT91WCA.0 size=900521 2024-04-22T19:58:45.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.470+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=LKX83LT91WCA tag=O5GS8RCNKXPD type=write size=1949097 2024-04-22T19:58:45.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.471+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=40YBGD0ZTYGG type=del size=3227898 2024-04-22T19:58:45.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.472+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~DAM06X9UQGY0.0 size=847507 2024-04-22T19:58:45.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.474+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=DAM06X9UQGY0 tag=G1FS0TTN5P1K type=write size=1896083 2024-04-22T19:58:45.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.475+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=BJ4ZM41M025I type=del size=3477662 2024-04-22T19:58:45.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.476+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~TTZQK79NTPZT.0 size=1048576 2024-04-22T19:58:45.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.477+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~TTZQK79NTPZT.1 size=1048576 2024-04-22T19:58:45.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.478+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~TTZQK79NTPZT.2 size=49065 2024-04-22T19:58:45.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.480+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=TTZQK79NTPZT tag=B0DX5EKPWXDU type=write size=3194793 2024-04-22T19:58:45.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.481+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=Q939I9O8M6P8 type=del size=3070567 2024-04-22T19:58:45.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.481+0000 7f5269a8e780 1 < canceled part key=_multipart_GRSL48RUEBX8.2~4BMG7SXFCXNK.0 size=1048576 2024-04-22T19:58:45.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.482+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~4BMG7SXFCXNK.1 size=1048576 2024-04-22T19:58:45.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.483+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~4BMG7SXFCXNK.2 size=132510 2024-04-22T19:58:45.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.485+0000 7f5269a8e780 1 < canceled multipart upload key=GRSL48RUEBX8 upload=4BMG7SXFCXNK tag=SKNRE72IDD0G type=write size=3278238 2024-04-22T19:58:45.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.487+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=49B0R5KD0F02 type=write size=644930 2024-04-22T19:58:45.489 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.489+0000 7f5269a8e780 1 < suggested remove operation key=B0Y7ZTPC0975 tag=8T8JN52CY9YD type=del 2024-04-22T19:58:45.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.491+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=P7XLX14IJTI6 type=del size=3032190 2024-04-22T19:58:45.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.492+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~LVH6ZGORU121.0 size=1048576 2024-04-22T19:58:45.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.493+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~LVH6ZGORU121.1 size=783668 2024-04-22T19:58:45.495 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.495+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=LVH6ZGORU121 tag=DIEC00CL8KAI type=write size=2880820 2024-04-22T19:58:45.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.496+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=B5F0GY6ZRTYA type=del size=28500 2024-04-22T19:58:45.497 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.497+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=89FX2H3UJXG3 type=del 2024-04-22T19:58:45.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.498+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=S8AUWO6BGDV6 type=del size=3573236 2024-04-22T19:58:45.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.499+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=A1EGTDJJWJAD type=del 2024-04-22T19:58:45.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.501+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=5K6O0FWN2KNX type=write size=628373 2024-04-22T19:58:45.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.502+0000 7f5269a8e780 1 < canceled operation key=8HFSDSNZONIN tag=Z6A533FMDN9U type=del size=2721654 2024-04-22T19:58:45.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.504+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~67QB0R819QJ9.0 size=1048576 2024-04-22T19:58:45.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.505+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~67QB0R819QJ9.1 size=247669 2024-04-22T19:58:45.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.505+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=67QB0R819QJ9 tag=47HUJ2K4RXBD type=write size=2344821 2024-04-22T19:58:45.506 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.506+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~S1Q9PJ2H38IZ.0 size=1048576 2024-04-22T19:58:45.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.507+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~S1Q9PJ2H38IZ.1 size=1048576 2024-04-22T19:58:45.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.508+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~S1Q9PJ2H38IZ.2 size=955497 2024-04-22T19:58:45.510 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.510+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=S1Q9PJ2H38IZ tag=GEW2GWFEE1CF type=write size=4101225 2024-04-22T19:58:45.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.511+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=2C8SR5WSJ6KJ type=write size=953797 2024-04-22T19:58:45.512 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.512+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~LHJMCZ0L0BXE.0 size=250161 2024-04-22T19:58:45.513 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.514+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=LHJMCZ0L0BXE tag=J20OS9KZ2X2H type=write size=1298737 2024-04-22T19:58:45.516 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.516+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~RNHXVMRPNUYT.0 size=1048576 2024-04-22T19:58:45.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.517+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~RNHXVMRPNUYT.1 size=1048576 2024-04-22T19:58:45.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.517+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~RNHXVMRPNUYT.2 size=743923 2024-04-22T19:58:45.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.517+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=RNHXVMRPNUYT tag=RHTGOW7CZMB0 type=write size=3889651 2024-04-22T19:58:45.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.519+0000 7f5269a8e780 1 < completed write operation key=HIAXPVKZ1IIO tag=F8VBBY4VOD03 type=write size=23538 2024-04-22T19:58:45.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.520+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=0PST6BFBZHQY type=del size=736386 2024-04-22T19:58:45.521 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.521+0000 7f5269a8e780 1 < canceled operation key=AMBJHMJGLUUS tag=C7QVULANH8Q1 type=del size=2289851 2024-04-22T19:58:45.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.522+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=ZUEY4NQO60E8 type=del size=733634 2024-04-22T19:58:45.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.522+0000 7f5269a8e780 1 < canceled part key=_multipart_EHRS6DQDRXVS.2~W8XIX0KK0TYB.0 size=268541 2024-04-22T19:58:45.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.524+0000 7f5269a8e780 1 < canceled multipart upload key=EHRS6DQDRXVS upload=W8XIX0KK0TYB tag=5061GYG4K6UN type=write size=1317117 2024-04-22T19:58:45.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.525+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=MPFY2KS5P6CT type=del size=2297938 2024-04-22T19:58:45.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.526+0000 7f5269a8e780 1 < canceled part key=_multipart_DX5YUYQRZKWA.2~KSP8W5WJNV78.0 size=878663 2024-04-22T19:58:45.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.527+0000 7f5269a8e780 1 < canceled multipart upload key=DX5YUYQRZKWA upload=KSP8W5WJNV78 tag=WCC3MB8ZTFJI type=write size=1927239 2024-04-22T19:58:45.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.529+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~ZR5ETCRZ44NS.0 size=1048576 2024-04-22T19:58:45.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.530+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~ZR5ETCRZ44NS.1 size=1048576 2024-04-22T19:58:45.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.531+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~ZR5ETCRZ44NS.2 size=450505 2024-04-22T19:58:45.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.531+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=ZR5ETCRZ44NS tag=KHZ6NWHMTY0D type=write size=3596233 2024-04-22T19:58:45.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.532+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=W5NX10OT644M type=del 2024-04-22T19:58:45.533 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.533+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=S6LIPDKAMAC2 type=del size=3325314 2024-04-22T19:58:45.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.534+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=40YBGD0ZTYGG type=del 2024-04-22T19:58:45.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.536+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=A1GMM5TXR8CU type=del size=3987711 2024-04-22T19:58:45.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.537+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=BJ4ZM41M025I type=del 2024-04-22T19:58:45.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.538+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=O2QYUGWG2CZL type=del size=209048 2024-04-22T19:58:45.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.539+0000 7f5269a8e780 1 < canceled operation key=AMBJHMJGLUUS tag=Q939I9O8M6P8 type=del size=3070567 2024-04-22T19:58:45.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.541+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~LC68ZWK91UB4.0 size=1048576 2024-04-22T19:58:45.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.542+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~LC68ZWK91UB4.1 size=1048576 2024-04-22T19:58:45.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.543+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~LC68ZWK91UB4.2 size=100650 2024-04-22T19:58:45.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.543+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=LC68ZWK91UB4 tag=FOILC2O7459X type=write size=3246378 2024-04-22T19:58:45.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.545+0000 7f5269a8e780 1 < suggested update operation key=VPAVX5FXYWZO tag=49B0R5KD0F02 type=write size=644930 2024-04-22T19:58:45.547 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.547+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~LF2UKIUET65U.0 size=728952 2024-04-22T19:58:45.547 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.547+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=LF2UKIUET65U tag=OZ0ZZ9CCKSLL type=write size=1777528 2024-04-22T19:58:45.547 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.548+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=P7XLX14IJTI6 type=del 2024-04-22T19:58:45.550 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.550+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~4U0CYYZ23P9N.0 size=1048576 2024-04-22T19:58:45.550 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.551+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~4U0CYYZ23P9N.1 size=1048576 2024-04-22T19:58:45.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.551+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~4U0CYYZ23P9N.2 size=1042872 2024-04-22T19:58:45.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.551+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=4U0CYYZ23P9N tag=9NNYXY72S2AH type=write size=4188600 2024-04-22T19:58:45.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.552+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=B5F0GY6ZRTYA type=del 2024-04-22T19:58:45.554 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.554+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=4L819KWRGHEX type=del size=2911000 2024-04-22T19:58:45.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.555+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=S8AUWO6BGDV6 type=del 2024-04-22T19:58:45.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.556+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=LEINGUBKNUB1 type=del size=2970726 2024-04-22T19:58:45.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.557+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=5K6O0FWN2KNX type=write size=628373 2024-04-22T19:58:45.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.559+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=1U45GEX21T1H type=del size=3209077 2024-04-22T19:58:45.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.560+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~67QB0R819QJ9.0 size=1048576 2024-04-22T19:58:45.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.561+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~67QB0R819QJ9.1 size=247669 2024-04-22T19:58:45.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.563+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=67QB0R819QJ9 tag=47HUJ2K4RXBD type=write size=2344821 2024-04-22T19:58:45.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.565+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~B0ZV5320FEGD.0 size=1039080 2024-04-22T19:58:45.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.565+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=B0ZV5320FEGD tag=MV3Y9UCSONDV type=write size=2087656 2024-04-22T19:58:45.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.567+0000 7f5269a8e780 1 < suggested update operation key=A853MOZPT9BG tag=2C8SR5WSJ6KJ type=write size=953797 2024-04-22T19:58:45.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.568+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=IF5XBNHNCDHB type=del size=2659293 2024-04-22T19:58:45.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.569+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~RNHXVMRPNUYT.0 size=1048576 2024-04-22T19:58:45.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.570+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~RNHXVMRPNUYT.1 size=1048576 2024-04-22T19:58:45.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.571+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~RNHXVMRPNUYT.2 size=743923 2024-04-22T19:58:45.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.573+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=RNHXVMRPNUYT tag=RHTGOW7CZMB0 type=write size=3889651 2024-04-22T19:58:45.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.574+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=KR5OXRZPDAZT type=del size=3540115 2024-04-22T19:58:45.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.576+0000 7f5269a8e780 -1 < failed to complete operation key=VPAVX5FXYWZO tag=0PST6BFBZHQY type=del size=736386: (22) Invalid argument 2024-04-22T19:58:45.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.577+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=PZNU4D5D5X12 type=del size=2202658 2024-04-22T19:58:45.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.578+0000 7f5269a8e780 1 < canceled operation key=O9NFAVE09ZN7 tag=ZUEY4NQO60E8 type=del size=733634 2024-04-22T19:58:45.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.579+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=4WZXJ8TN7DPG type=del size=3440509 2024-04-22T19:58:45.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.580+0000 7f5269a8e780 -1 < failed to complete operation key=VPAVX5FXYWZO tag=MPFY2KS5P6CT type=del size=2297938: (22) Invalid argument 2024-04-22T19:58:45.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.581+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=1WY126HXWC2D type=del size=4017699 2024-04-22T19:58:45.582 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.582+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~ZR5ETCRZ44NS.0 size=1048576 2024-04-22T19:58:45.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.583+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~ZR5ETCRZ44NS.1 size=1048576 2024-04-22T19:58:45.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.584+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~ZR5ETCRZ44NS.2 size=450505 2024-04-22T19:58:45.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.586+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=ZR5ETCRZ44NS tag=KHZ6NWHMTY0D type=write size=3596233 2024-04-22T19:58:45.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.587+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=OJX6XQYCTAD0 type=del size=2979548 2024-04-22T19:58:45.589 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.589+0000 7f5269a8e780 1 < canceled operation key=SN2BZATJGUWA tag=S6LIPDKAMAC2 type=del size=3325314 2024-04-22T19:58:45.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.590+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=U20G3ISC3LMW type=write size=896185 2024-04-22T19:58:45.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.591+0000 7f5269a8e780 -1 < failed to complete operation key=A853MOZPT9BG tag=A1GMM5TXR8CU type=del size=3987711: (22) Invalid argument 2024-04-22T19:58:45.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.592+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=6O63EBJZL1WD type=del size=2775808 2024-04-22T19:58:45.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.594+0000 7f5269a8e780 1 < suggested remove operation key=IBH6W1NINUHC tag=O2QYUGWG2CZL type=del 2024-04-22T19:58:45.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.595+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=Q6ALSBY558FS type=write size=110573 2024-04-22T19:58:45.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.597+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~LC68ZWK91UB4.0 size=1048576 2024-04-22T19:58:45.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.598+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~LC68ZWK91UB4.1 size=1048576 2024-04-22T19:58:45.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.599+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~LC68ZWK91UB4.2 size=100650 2024-04-22T19:58:45.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.601+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=LC68ZWK91UB4 tag=FOILC2O7459X type=write size=3246378 2024-04-22T19:58:45.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.603+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~VM0DKAYF6UU3.0 size=1048576 2024-04-22T19:58:45.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.604+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~VM0DKAYF6UU3.1 size=1048576 2024-04-22T19:58:45.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.605+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~VM0DKAYF6UU3.2 size=157934 2024-04-22T19:58:45.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.605+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=VM0DKAYF6UU3 tag=DMSZ33B2SH7J type=write size=3303662 2024-04-22T19:58:45.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.606+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~LF2UKIUET65U.0 size=728952 2024-04-22T19:58:45.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.608+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=LF2UKIUET65U tag=OZ0ZZ9CCKSLL type=write size=1777528 2024-04-22T19:58:45.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.610+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~853IH0O8DTI9.0 size=347762 2024-04-22T19:58:45.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.610+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=853IH0O8DTI9 tag=0QQSS7BKI18H type=write size=1396338 2024-04-22T19:58:45.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.611+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~4U0CYYZ23P9N.0 size=1048576 2024-04-22T19:58:45.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.611+0000 7f5269a8e780 1 < canceled part key=_multipart_HIAXPVKZ1IIO.2~4U0CYYZ23P9N.1 size=1048576 2024-04-22T19:58:45.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.612+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~4U0CYYZ23P9N.2 size=1042872 2024-04-22T19:58:45.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.614+0000 7f5269a8e780 1 < canceled multipart upload key=HIAXPVKZ1IIO upload=4U0CYYZ23P9N tag=9NNYXY72S2AH type=write size=4188600 2024-04-22T19:58:45.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.615+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=8DBB4T58VTWT type=write size=20864 2024-04-22T19:58:45.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.616+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=4L819KWRGHEX type=del 2024-04-22T19:58:45.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.617+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=8WB7QYO1PUE5 type=del size=3778509 2024-04-22T19:58:45.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.617+0000 7f5269a8e780 -1 < no bi entry to suggest for operation key=IBH6W1NINUHC tag=LEINGUBKNUB1 type=del size=2970726: (2) No such file or directory 2024-04-22T19:58:45.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.618+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=D20S0JHBJY9X type=write size=183871 2024-04-22T19:58:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.619+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=1U45GEX21T1H type=del 2024-04-22T19:58:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.621+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~HDQ8OYL7DG3V.0 size=1048576 2024-04-22T19:58:45.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.622+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~HDQ8OYL7DG3V.1 size=292617 2024-04-22T19:58:45.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.622+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=HDQ8OYL7DG3V tag=8F2F7SHAS46Y type=write size=2389769 2024-04-22T19:58:45.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.623+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~B0ZV5320FEGD.0 size=1039080 2024-04-22T19:58:45.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.625+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=B0ZV5320FEGD tag=MV3Y9UCSONDV type=write size=2087656 2024-04-22T19:58:45.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.626+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=B0F4FCV2XDBN type=write size=180193 2024-04-22T19:58:45.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.627+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=IF5XBNHNCDHB type=del 2024-04-22T19:58:45.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.628+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=NRLI12DAWTKR type=del size=3596450 2024-04-22T19:58:45.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.631+0000 7f5269a8e780 1 < suggested update operation key=DX5YUYQRZKWA tag=KR5OXRZPDAZT type=del size=3540115 2024-04-22T19:58:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.632+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=BQ6TGJ0BBGEA type=del size=1137070 2024-04-22T19:58:45.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.633+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=PZNU4D5D5X12 type=del 2024-04-22T19:58:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.635+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=Z34GREMZ1RL2 type=del size=1420538 2024-04-22T19:58:45.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.637+0000 7f5269a8e780 1 < suggested update operation key=M9M4SEEEZSFV tag=4WZXJ8TN7DPG type=del size=3440509 2024-04-22T19:58:45.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.638+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=UMXFSY5LSL6C type=del size=62917 2024-04-22T19:58:45.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.639+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=1WY126HXWC2D type=del 2024-04-22T19:58:45.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.642+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~RO4OW0HGOJWR.0 size=1048576 2024-04-22T19:58:45.642 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.642+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~RO4OW0HGOJWR.1 size=1048576 2024-04-22T19:58:45.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.643+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~RO4OW0HGOJWR.2 size=764638 2024-04-22T19:58:45.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.643+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=RO4OW0HGOJWR tag=7BX9V7KUDDIL type=write size=3910366 2024-04-22T19:58:45.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.645+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=OJX6XQYCTAD0 type=del 2024-04-22T19:58:45.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.647+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~2J22ZY3BEGBG.0 size=582553 2024-04-22T19:58:45.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.647+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=2J22ZY3BEGBG tag=DU8G0004080W type=write size=1631129 2024-04-22T19:58:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.648+0000 7f5269a8e780 -1 < failed to complete operation key=IBH6W1NINUHC tag=U20G3ISC3LMW type=write size=896185: (22) Invalid argument 2024-04-22T19:58:45.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.649+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=Z7VU2NUCT0PU type=del size=3559751 2024-04-22T19:58:45.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.650+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=6O63EBJZL1WD type=del 2024-04-22T19:58:45.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.652+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=U800ZESFE5A2 type=write size=569819 2024-04-22T19:58:45.653 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.653+0000 7f5269a8e780 1 < completed write operation key=ZC4ZOHUJ3HQD tag=Q6ALSBY558FS type=write size=110573 2024-04-22T19:58:45.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.655+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~23CS2CKGYJGK.0 size=50504 2024-04-22T19:58:45.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.655+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=23CS2CKGYJGK tag=5N8K9PCLRZHB type=write size=1099080 2024-04-22T19:58:45.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.657+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~VM0DKAYF6UU3.0 size=1048576 2024-04-22T19:58:45.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.657+0000 7f5269a8e780 1 < canceled part key=_multipart_A853MOZPT9BG.2~VM0DKAYF6UU3.1 size=1048576 2024-04-22T19:58:45.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.657+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~VM0DKAYF6UU3.2 size=157934 2024-04-22T19:58:45.659 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.659+0000 7f5269a8e780 1 < canceled multipart upload key=A853MOZPT9BG upload=VM0DKAYF6UU3 tag=DMSZ33B2SH7J type=write size=3303662 2024-04-22T19:58:45.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.660+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=UP4RVP3R4UB1 type=write size=813637 2024-04-22T19:58:45.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.662+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~853IH0O8DTI9.0 size=347762 2024-04-22T19:58:45.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.663+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=853IH0O8DTI9 tag=0QQSS7BKI18H type=write size=1396338 2024-04-22T19:58:45.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.666+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MI3WX0507HWR.0 size=1048576 2024-04-22T19:58:45.668 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.668+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MI3WX0507HWR.1 size=1048576 2024-04-22T19:58:45.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.669+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MI3WX0507HWR.2 size=254326 2024-04-22T19:58:45.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.669+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=MI3WX0507HWR tag=Z5KW4M073VWT type=write size=3400054 2024-04-22T19:58:45.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.670+0000 7f5269a8e780 1 < completed write operation key=O9NFAVE09ZN7 tag=8DBB4T58VTWT type=write size=20864 2024-04-22T19:58:45.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.672+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~MU45Y7GVBMOZ.0 size=1048576 2024-04-22T19:58:45.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.673+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~MU45Y7GVBMOZ.1 size=1048576 2024-04-22T19:58:45.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.674+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~MU45Y7GVBMOZ.2 size=925871 2024-04-22T19:58:45.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.674+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=MU45Y7GVBMOZ tag=0XRRTF00R3QB type=write size=4071599 2024-04-22T19:58:45.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.675+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=8WB7QYO1PUE5 type=del 2024-04-22T19:58:45.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.677+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~UYESSBXAZUPR.0 size=1048576 2024-04-22T19:58:45.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.678+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~UYESSBXAZUPR.1 size=1048576 2024-04-22T19:58:45.679 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.679+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~UYESSBXAZUPR.2 size=66843 2024-04-22T19:58:45.679 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.679+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=UYESSBXAZUPR tag=0ZZ4ZZF3G92P type=write size=3212571 2024-04-22T19:58:45.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.680+0000 7f5269a8e780 1 < completed write operation key=RY1O7X1CWZMK tag=D20S0JHBJY9X type=write size=183871 2024-04-22T19:58:45.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.681+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=3KRXFRKF580Y type=del size=3671230 2024-04-22T19:58:45.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.682+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~HDQ8OYL7DG3V.0 size=1048576 2024-04-22T19:58:45.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.683+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~HDQ8OYL7DG3V.1 size=292617 2024-04-22T19:58:45.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.686+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=HDQ8OYL7DG3V tag=8F2F7SHAS46Y type=write size=2389769 2024-04-22T19:58:45.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.687+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=42SF4Q5ZH1V7 type=write size=642786 2024-04-22T19:58:45.689 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.689+0000 7f5269a8e780 1 < completed write operation key=Q1UKF4UMJ2HO tag=B0F4FCV2XDBN type=write size=180193 2024-04-22T19:58:45.690 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.690+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=YS8NB3K62LOZ type=write size=495792 2024-04-22T19:58:45.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.691+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=NRLI12DAWTKR type=del 2024-04-22T19:58:45.692 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.692+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=7ZFQNYPXG55E type=del size=332838 2024-04-22T19:58:45.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.694+0000 7f5269a8e780 1 < suggested update operation key=4GX83ZJWZCD1 tag=BQ6TGJ0BBGEA type=del size=1137070 2024-04-22T19:58:45.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.695+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=E84H7GA3IK02 type=write size=602450 2024-04-22T19:58:45.697 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.697+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=Z34GREMZ1RL2 type=del 2024-04-22T19:58:45.698 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.698+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=O2BJETO9D6IU type=del size=305210 2024-04-22T19:58:45.699 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.699+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=UMXFSY5LSL6C type=del 2024-04-22T19:58:45.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.701+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~C18XFERN3QNB.0 size=1048576 2024-04-22T19:58:45.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.702+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~C18XFERN3QNB.1 size=1048576 2024-04-22T19:58:45.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.703+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~C18XFERN3QNB.2 size=214101 2024-04-22T19:58:45.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.703+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=C18XFERN3QNB tag=0S3RADM00PHE type=write size=3359829 2024-04-22T19:58:45.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.704+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~RO4OW0HGOJWR.0 size=1048576 2024-04-22T19:58:45.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.705+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~RO4OW0HGOJWR.1 size=1048576 2024-04-22T19:58:45.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.706+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~RO4OW0HGOJWR.2 size=764638 2024-04-22T19:58:45.708 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.708+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=RO4OW0HGOJWR tag=7BX9V7KUDDIL type=write size=3910366 2024-04-22T19:58:45.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.710+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~RNJMS82NGWV0.0 size=735742 2024-04-22T19:58:45.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.710+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=RNJMS82NGWV0 tag=KOGSJO7NIGLW type=write size=1784318 2024-04-22T19:58:45.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.711+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~2J22ZY3BEGBG.0 size=582553 2024-04-22T19:58:45.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.713+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=2J22ZY3BEGBG tag=DU8G0004080W type=write size=1631129 2024-04-22T19:58:45.715 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.715+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~3J4PCQXM8KTN.0 size=1048576 2024-04-22T19:58:45.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.716+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~3J4PCQXM8KTN.1 size=1048576 2024-04-22T19:58:45.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.717+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~3J4PCQXM8KTN.2 size=228335 2024-04-22T19:58:45.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.717+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=3J4PCQXM8KTN tag=9Z87YSBYV39B type=write size=3374063 2024-04-22T19:58:45.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.718+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=Z7VU2NUCT0PU type=del 2024-04-22T19:58:45.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.720+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~R5VB5FZW08OV.0 size=1048576 2024-04-22T19:58:45.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.721+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~R5VB5FZW08OV.1 size=820052 2024-04-22T19:58:45.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.721+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=R5VB5FZW08OV tag=9FKC7SA67ULA type=write size=2917204 2024-04-22T19:58:45.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.722+0000 7f5269a8e780 1 < completed write operation key=IBH6W1NINUHC tag=U800ZESFE5A2 type=write size=569819 2024-04-22T19:58:45.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.723+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=DZ3B8XPJQG7D type=del size=1178752 2024-04-22T19:58:45.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.724+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~23CS2CKGYJGK.0 size=50504 2024-04-22T19:58:45.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.726+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=23CS2CKGYJGK tag=5N8K9PCLRZHB type=write size=1099080 2024-04-22T19:58:45.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.728+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~FJBGO2QV4F00.0 size=1048576 2024-04-22T19:58:45.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.729+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~FJBGO2QV4F00.1 size=544190 2024-04-22T19:58:45.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.729+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=FJBGO2QV4F00 tag=1ERTOJ8I5OJD type=write size=2641342 2024-04-22T19:58:45.730 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.730+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=UP4RVP3R4UB1 type=write size=813637 2024-04-22T19:58:45.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.732+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~DDI0PFKP35XN.0 size=898768 2024-04-22T19:58:45.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.732+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=DDI0PFKP35XN tag=1AL0OQ2AMRHF type=write size=1947344 2024-04-22T19:58:45.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.733+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~MI3WX0507HWR.0 size=1048576 2024-04-22T19:58:45.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.734+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~MI3WX0507HWR.1 size=1048576 2024-04-22T19:58:45.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.735+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~MI3WX0507HWR.2 size=254326 2024-04-22T19:58:45.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.737+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=MI3WX0507HWR tag=Z5KW4M073VWT type=write size=3400054 2024-04-22T19:58:45.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.739+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~Q0CWSS5D0JON.0 size=21527 2024-04-22T19:58:45.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.739+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=Q0CWSS5D0JON tag=MHZ08VSW2KU2 type=write size=1070103 2024-04-22T19:58:45.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.740+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~MU45Y7GVBMOZ.0 size=1048576 2024-04-22T19:58:45.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.741+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~MU45Y7GVBMOZ.1 size=1048576 2024-04-22T19:58:45.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.742+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~MU45Y7GVBMOZ.2 size=925871 2024-04-22T19:58:45.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.744+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=MU45Y7GVBMOZ tag=0XRRTF00R3QB type=write size=4071599 2024-04-22T19:58:45.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.746+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~EAS9UF4O5W10.0 size=1048576 2024-04-22T19:58:45.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.747+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~EAS9UF4O5W10.1 size=448085 2024-04-22T19:58:45.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.747+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=EAS9UF4O5W10 tag=OIF94E0CFCYR type=write size=2545237 2024-04-22T19:58:45.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.748+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~UYESSBXAZUPR.0 size=1048576 2024-04-22T19:58:45.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.749+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~UYESSBXAZUPR.1 size=1048576 2024-04-22T19:58:45.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.750+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~UYESSBXAZUPR.2 size=66843 2024-04-22T19:58:45.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.752+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=UYESSBXAZUPR tag=0ZZ4ZZF3G92P type=write size=3212571 2024-04-22T19:58:45.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.754+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~Q3I1Y3JVLX9B.0 size=1048576 2024-04-22T19:58:45.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.755+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~Q3I1Y3JVLX9B.1 size=101661 2024-04-22T19:58:45.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.755+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=Q3I1Y3JVLX9B tag=WIBEV3SKDS42 type=write size=2198813 2024-04-22T19:58:45.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.756+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=3KRXFRKF580Y type=del 2024-04-22T19:58:45.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.757+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=4S0CHX0LBWRH type=del size=1749764 2024-04-22T19:58:45.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.758+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=42SF4Q5ZH1V7 type=write size=642786 2024-04-22T19:58:45.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.760+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=SLDBUDMCYPBJ type=write size=735142 2024-04-22T19:58:45.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.761+0000 7f5269a8e780 1 < canceled operation key=637M7AP8X9E9 tag=YS8NB3K62LOZ type=write size=495792 2024-04-22T19:58:45.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.762+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=XRZN7LSQ2CRP type=del size=725329 2024-04-22T19:58:45.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.763+0000 7f5269a8e780 -1 < failed to complete operation key=4GX83ZJWZCD1 tag=7ZFQNYPXG55E type=del size=332838: (22) Invalid argument 2024-04-22T19:58:45.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.764+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=G1Z7S3OV4MUY type=del size=1819706 2024-04-22T19:58:45.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.765+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=E84H7GA3IK02 type=write size=602450 2024-04-22T19:58:45.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.766+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=CFG4L352YOYH type=write size=180169 2024-04-22T19:58:45.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.767+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=O2BJETO9D6IU type=del 2024-04-22T19:58:45.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.769+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~QB2QU2K3HCEK.0 size=1048576 2024-04-22T19:58:45.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.770+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~QB2QU2K3HCEK.1 size=367818 2024-04-22T19:58:45.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.770+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=QB2QU2K3HCEK tag=XYO65KZ8SDZ0 type=write size=2464970 2024-04-22T19:58:45.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.771+0000 7f5269a8e780 1 < canceled part key=_multipart_4GX83ZJWZCD1.2~C18XFERN3QNB.0 size=1048576 2024-04-22T19:58:45.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.771+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~C18XFERN3QNB.1 size=1048576 2024-04-22T19:58:45.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.773+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~C18XFERN3QNB.2 size=214101 2024-04-22T19:58:45.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.777+0000 7f5269a8e780 1 < canceled multipart upload key=4GX83ZJWZCD1 upload=C18XFERN3QNB tag=0S3RADM00PHE type=write size=3359829 2024-04-22T19:58:45.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.778+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=GQQAS8AQ462U type=del size=3819412 2024-04-22T19:58:45.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.779+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~RNJMS82NGWV0.0 size=735742 2024-04-22T19:58:45.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.781+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=RNJMS82NGWV0 tag=KOGSJO7NIGLW type=write size=1784318 2024-04-22T19:58:45.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.783+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~1S5ETD6RUKJ9.0 size=191831 2024-04-22T19:58:45.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.783+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=1S5ETD6RUKJ9 tag=X446TWVQ1IJ0 type=write size=1240407 2024-04-22T19:58:45.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.784+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~3J4PCQXM8KTN.0 size=1048576 2024-04-22T19:58:45.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.784+0000 7f5269a8e780 1 < canceled part key=_multipart_O9NFAVE09ZN7.2~3J4PCQXM8KTN.1 size=1048576 2024-04-22T19:58:45.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.785+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~3J4PCQXM8KTN.2 size=228335 2024-04-22T19:58:45.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.787+0000 7f5269a8e780 1 < canceled multipart upload key=O9NFAVE09ZN7 upload=3J4PCQXM8KTN tag=9Z87YSBYV39B type=write size=3374063 2024-04-22T19:58:45.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.788+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=Z2DQ3A8J2KP1 type=del size=2027814 2024-04-22T19:58:45.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.789+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~R5VB5FZW08OV.0 size=1048576 2024-04-22T19:58:45.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.792+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~R5VB5FZW08OV.1 size=820052 2024-04-22T19:58:45.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.793+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=R5VB5FZW08OV tag=9FKC7SA67ULA type=write size=2917204 2024-04-22T19:58:45.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.795+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=Y8HXFEUSMU7T type=del size=2333231 2024-04-22T19:58:45.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.797+0000 7f5269a8e780 1 < suggested remove operation key=IBH6W1NINUHC tag=DZ3B8XPJQG7D type=del 2024-04-22T19:58:45.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.798+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=5SGCQFKNLIXA type=del size=184611 2024-04-22T19:58:45.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.799+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~FJBGO2QV4F00.0 size=1048576 2024-04-22T19:58:45.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.800+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~FJBGO2QV4F00.1 size=544190 2024-04-22T19:58:45.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.802+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=FJBGO2QV4F00 tag=1ERTOJ8I5OJD type=write size=2641342 2024-04-22T19:58:45.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.804+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~YO2HH34ADJCQ.0 size=1048576 2024-04-22T19:58:45.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.805+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~YO2HH34ADJCQ.1 size=251062 2024-04-22T19:58:45.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.805+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=YO2HH34ADJCQ tag=YN8L61BTP9ZV type=write size=2348214 2024-04-22T19:58:45.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.806+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~DDI0PFKP35XN.0 size=898768 2024-04-22T19:58:45.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.808+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=DDI0PFKP35XN tag=1AL0OQ2AMRHF type=write size=1947344 2024-04-22T19:58:45.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.810+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=SMXUZ2JISCC8 type=del size=1851621 2024-04-22T19:58:45.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.811+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~Q0CWSS5D0JON.0 size=21527 2024-04-22T19:58:45.813 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.813+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=Q0CWSS5D0JON tag=MHZ08VSW2KU2 type=write size=1070103 2024-04-22T19:58:45.814 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.814+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=6NIIDLUAGY11 type=del size=4155659 2024-04-22T19:58:45.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.815+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~EAS9UF4O5W10.0 size=1048576 2024-04-22T19:58:45.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.816+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~EAS9UF4O5W10.1 size=448085 2024-04-22T19:58:45.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.818+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=EAS9UF4O5W10 tag=OIF94E0CFCYR type=write size=2545237 2024-04-22T19:58:45.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.819+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=24OVYRGSXN56 type=del size=1334980 2024-04-22T19:58:45.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.820+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~Q3I1Y3JVLX9B.0 size=1048576 2024-04-22T19:58:45.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.821+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~Q3I1Y3JVLX9B.1 size=101661 2024-04-22T19:58:45.823 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.823+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=Q3I1Y3JVLX9B tag=WIBEV3SKDS42 type=write size=2198813 2024-04-22T19:58:45.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.824+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=WQ1RB5R0TJAX type=del size=2777291 2024-04-22T19:58:45.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.827+0000 7f5269a8e780 1 < suggested update operation key=8HFSDSNZONIN tag=4S0CHX0LBWRH type=del size=1749764 2024-04-22T19:58:45.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.829+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~8E5MTTJI20HM.0 size=1048576 2024-04-22T19:58:45.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.830+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~8E5MTTJI20HM.1 size=430605 2024-04-22T19:58:45.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.830+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=8E5MTTJI20HM tag=VP6WDTB73DRJ type=write size=2527757 2024-04-22T19:58:45.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.832+0000 7f5269a8e780 1 < suggested update operation key=GX6NILQG878S tag=SLDBUDMCYPBJ type=write size=735142 2024-04-22T19:58:45.834 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.834+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~0OPI9EU2TO6G.0 size=62470 2024-04-22T19:58:45.834 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.834+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=0OPI9EU2TO6G tag=V923SDMS8P6L type=write size=1111046 2024-04-22T19:58:45.835 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.835+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=XRZN7LSQ2CRP type=del 2024-04-22T19:58:45.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.836+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=DCO6S3A4PWMG type=del size=44627 2024-04-22T19:58:45.838 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.838+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=G1Z7S3OV4MUY type=del 2024-04-22T19:58:45.839 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.839+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=SSVTVXHX3U68 type=del size=2277022 2024-04-22T19:58:45.840 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.840+0000 7f5269a8e780 -1 < failed to complete operation key=IBH6W1NINUHC tag=CFG4L352YOYH type=write size=180169: (22) Invalid argument 2024-04-22T19:58:45.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.842+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~KCI9K9G19TVI.0 size=1048576 2024-04-22T19:58:45.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.843+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~KCI9K9G19TVI.1 size=107972 2024-04-22T19:58:45.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.843+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=KCI9K9G19TVI tag=FCQ9929QC17M type=write size=2205124 2024-04-22T19:58:45.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.844+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~QB2QU2K3HCEK.0 size=1048576 2024-04-22T19:58:45.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.845+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~QB2QU2K3HCEK.1 size=367818 2024-04-22T19:58:45.847 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.847+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=QB2QU2K3HCEK tag=XYO65KZ8SDZ0 type=write size=2464970 2024-04-22T19:58:45.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.849+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~SE8J19X3U7VD.0 size=371015 2024-04-22T19:58:45.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.849+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=SE8J19X3U7VD tag=W9HHQM9SNRLG type=write size=1419591 2024-04-22T19:58:45.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.851+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=GQQAS8AQ462U type=del 2024-04-22T19:58:45.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.852+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=SZF0ZRE113OH type=del size=2679954 2024-04-22T19:58:45.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.853+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~1S5ETD6RUKJ9.0 size=191831 2024-04-22T19:58:45.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.855+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=1S5ETD6RUKJ9 tag=X446TWVQ1IJ0 type=write size=1240407 2024-04-22T19:58:45.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.856+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=DZUMNHSMW3L0 type=del size=366055 2024-04-22T19:58:45.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.858+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=Z2DQ3A8J2KP1 type=del 2024-04-22T19:58:45.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.859+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=77V5UF24FVTN type=del size=671961 2024-04-22T19:58:45.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.860+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=Y8HXFEUSMU7T type=del 2024-04-22T19:58:45.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.861+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=N7P8H0RGUVKL type=del size=3317575 2024-04-22T19:58:45.863 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.863+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=5SGCQFKNLIXA type=del 2024-04-22T19:58:45.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.864+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=BN18X5B1CUAR type=del size=11195 2024-04-22T19:58:45.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.865+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~YO2HH34ADJCQ.0 size=1048576 2024-04-22T19:58:45.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.866+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~YO2HH34ADJCQ.1 size=251062 2024-04-22T19:58:45.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.868+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=YO2HH34ADJCQ tag=YN8L61BTP9ZV type=write size=2348214 2024-04-22T19:58:45.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.869+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=R3RW5TVVP082 type=del size=3140747 2024-04-22T19:58:45.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.870+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=SMXUZ2JISCC8 type=del 2024-04-22T19:58:45.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.871+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=HQETNKE46EHZ type=del size=893948 2024-04-22T19:58:45.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.873+0000 7f5269a8e780 1 < suggested update operation key=SN2BZATJGUWA tag=6NIIDLUAGY11 type=del size=4155659 2024-04-22T19:58:45.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.876+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~T6XGT20BJDHK.0 size=1048576 2024-04-22T19:58:45.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.876+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~T6XGT20BJDHK.1 size=1048576 2024-04-22T19:58:45.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.877+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~T6XGT20BJDHK.2 size=399092 2024-04-22T19:58:45.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.877+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=T6XGT20BJDHK tag=L13QAMQCKOZO type=write size=3544820 2024-04-22T19:58:45.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.878+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=24OVYRGSXN56 type=del 2024-04-22T19:58:45.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.880+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~A1AFTP1NSBXF.0 size=1048576 2024-04-22T19:58:45.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.881+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~A1AFTP1NSBXF.1 size=426338 2024-04-22T19:58:45.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.881+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=A1AFTP1NSBXF tag=01GNHYWM5KL6 type=write size=2523490 2024-04-22T19:58:45.885 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.885+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=WQ1RB5R0TJAX type=del size=2777291 2024-04-22T19:58:45.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.889+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~N89Y8YWZZQJ1.0 size=871302 2024-04-22T19:58:45.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.889+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=N89Y8YWZZQJ1 tag=8MQ6JF18FBGU type=write size=1919878 2024-04-22T19:58:45.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.890+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~8E5MTTJI20HM.0 size=1048576 2024-04-22T19:58:45.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.891+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~8E5MTTJI20HM.1 size=430605 2024-04-22T19:58:45.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.893+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=8E5MTTJI20HM tag=VP6WDTB73DRJ type=write size=2527757 2024-04-22T19:58:45.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.894+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=RGLYM3EQQKB4 type=del size=3612672 2024-04-22T19:58:45.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.894+0000 7f5269a8e780 1 < canceled part key=_multipart_GRSL48RUEBX8.2~0OPI9EU2TO6G.0 size=62470 2024-04-22T19:58:45.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.896+0000 7f5269a8e780 1 < canceled multipart upload key=GRSL48RUEBX8 upload=0OPI9EU2TO6G tag=V923SDMS8P6L type=write size=1111046 2024-04-22T19:58:45.897 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.897+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=CBFPNU21O8NS type=del size=366318 2024-04-22T19:58:45.898 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.898+0000 7f5269a8e780 1 < canceled operation key=B0Y7ZTPC0975 tag=DCO6S3A4PWMG type=del size=44627 2024-04-22T19:58:45.900 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.900+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~UR2WB0L5VRL4.0 size=818915 2024-04-22T19:58:45.900 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.900+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=UR2WB0L5VRL4 tag=D4XFTXSESPAD type=write size=1867491 2024-04-22T19:58:45.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.901+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=SSVTVXHX3U68 type=del 2024-04-22T19:58:45.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.904+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~89J27LDA9933.0 size=1048576 2024-04-22T19:58:45.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.904+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~89J27LDA9933.1 size=1048576 2024-04-22T19:58:45.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.905+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~89J27LDA9933.2 size=271125 2024-04-22T19:58:45.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.905+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=89J27LDA9933 tag=INSBJVYJBW1O type=write size=3416853 2024-04-22T19:58:45.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.907+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~KCI9K9G19TVI.0 size=1048576 2024-04-22T19:58:45.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.908+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~KCI9K9G19TVI.1 size=107972 2024-04-22T19:58:45.909 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.910+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=KCI9K9G19TVI tag=FCQ9929QC17M type=write size=2205124 2024-04-22T19:58:45.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.911+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=G6TSM88HHDKT type=del size=1500577 2024-04-22T19:58:45.912 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.912+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~SE8J19X3U7VD.0 size=371015 2024-04-22T19:58:45.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.914+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=SE8J19X3U7VD tag=W9HHQM9SNRLG type=write size=1419591 2024-04-22T19:58:45.915 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.915+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=LFMJP5W9JC8B type=del size=3651239 2024-04-22T19:58:45.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.916+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=SZF0ZRE113OH type=del 2024-04-22T19:58:45.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.917+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=3PEVZ9KKSYLA type=write size=1009673 2024-04-22T19:58:45.918 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.918+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=DZUMNHSMW3L0 type=del 2024-04-22T19:58:45.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.921+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~O4RUCQ0I2HNY.0 size=812087 2024-04-22T19:58:45.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.921+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=O4RUCQ0I2HNY tag=HI5VJJSZRRAQ type=write size=1860663 2024-04-22T19:58:45.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.922+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=77V5UF24FVTN type=del 2024-04-22T19:58:45.923 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.923+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=OTMEO9T2SFSZ type=del size=3505045 2024-04-22T19:58:45.924 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.924+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=N7P8H0RGUVKL type=del 2024-04-22T19:58:45.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.925+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=6D7GYY61Z8IB type=del size=3475158 2024-04-22T19:58:45.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.927+0000 7f5269a8e780 1 < suggested update operation key=IBH6W1NINUHC tag=BN18X5B1CUAR type=del size=11195 2024-04-22T19:58:45.928 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.929+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=6EGDUOP38RSJ type=write size=328972 2024-04-22T19:58:45.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.930+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=R3RW5TVVP082 type=del 2024-04-22T19:58:45.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.931+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=3G0IFFMNK3US type=del size=1311221 2024-04-22T19:58:45.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.932+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=HQETNKE46EHZ type=del 2024-04-22T19:58:45.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.933+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=VYGHB5G4AZ7Q type=del size=1091964 2024-04-22T19:58:45.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.934+0000 7f5269a8e780 1 < canceled part key=_multipart_637M7AP8X9E9.2~T6XGT20BJDHK.0 size=1048576 2024-04-22T19:58:45.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.934+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~T6XGT20BJDHK.1 size=1048576 2024-04-22T19:58:45.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.935+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~T6XGT20BJDHK.2 size=399092 2024-04-22T19:58:45.937 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.937+0000 7f5269a8e780 1 < canceled multipart upload key=637M7AP8X9E9 upload=T6XGT20BJDHK tag=L13QAMQCKOZO type=write size=3544820 2024-04-22T19:58:45.939 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.939+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~W5Q3ISR3MSTP.0 size=1048576 2024-04-22T19:58:45.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.940+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~W5Q3ISR3MSTP.1 size=811728 2024-04-22T19:58:45.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.940+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=W5Q3ISR3MSTP tag=FPF5APZ3T0BX type=write size=2908880 2024-04-22T19:58:45.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.941+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~A1AFTP1NSBXF.0 size=1048576 2024-04-22T19:58:45.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.941+0000 7f5269a8e780 1 < canceled part key=_multipart_ZC4ZOHUJ3HQD.2~A1AFTP1NSBXF.1 size=426338 2024-04-22T19:58:45.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.943+0000 7f5269a8e780 1 < canceled multipart upload key=ZC4ZOHUJ3HQD upload=A1AFTP1NSBXF tag=01GNHYWM5KL6 type=write size=2523490 2024-04-22T19:58:45.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.944+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=ZDYOMACGYDBQ type=del size=2079475 2024-04-22T19:58:45.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.945+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~N89Y8YWZZQJ1.0 size=871302 2024-04-22T19:58:45.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.947+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=N89Y8YWZZQJ1 tag=8MQ6JF18FBGU type=write size=1919878 2024-04-22T19:58:45.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.949+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=GS5W40CQSOQY type=del size=2071733 2024-04-22T19:58:45.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.950+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=RGLYM3EQQKB4 type=del 2024-04-22T19:58:45.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.952+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~7OJF6DYS6K2O.0 size=1048576 2024-04-22T19:58:45.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.953+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~7OJF6DYS6K2O.1 size=1048576 2024-04-22T19:58:45.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.953+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~7OJF6DYS6K2O.2 size=233002 2024-04-22T19:58:45.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.953+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=7OJF6DYS6K2O tag=90VON73N0K46 type=write size=3378730 2024-04-22T19:58:45.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.955+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=CBFPNU21O8NS type=del 2024-04-22T19:58:45.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.957+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~0RYKLZCSC02U.0 size=318724 2024-04-22T19:58:45.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.957+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=0RYKLZCSC02U tag=Z93QDZG88BTE type=write size=1367300 2024-04-22T19:58:45.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.958+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~UR2WB0L5VRL4.0 size=818915 2024-04-22T19:58:45.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.960+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=UR2WB0L5VRL4 tag=D4XFTXSESPAD type=write size=1867491 2024-04-22T19:58:45.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.961+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=D4NMUYL4L3A6 type=write size=457939 2024-04-22T19:58:45.962 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.962+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~89J27LDA9933.0 size=1048576 2024-04-22T19:58:45.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.963+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~89J27LDA9933.1 size=1048576 2024-04-22T19:58:45.964 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.964+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~89J27LDA9933.2 size=271125 2024-04-22T19:58:45.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.966+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=89J27LDA9933 tag=INSBJVYJBW1O type=write size=3416853 2024-04-22T19:58:45.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.968+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~Q2Q5ZZY2RB65.0 size=1048576 2024-04-22T19:58:45.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.969+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~Q2Q5ZZY2RB65.1 size=1048576 2024-04-22T19:58:45.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.970+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~Q2Q5ZZY2RB65.2 size=163944 2024-04-22T19:58:45.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.970+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=Q2Q5ZZY2RB65 tag=VOGDLFIFJ8BX type=write size=3309672 2024-04-22T19:58:45.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.972+0000 7f5269a8e780 1 < suggested remove operation key=8V6ESIGMRO5G tag=G6TSM88HHDKT type=del 2024-04-22T19:58:45.974 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.974+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~FJHJSJBMVL4T.0 size=1048576 2024-04-22T19:58:45.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.975+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~FJHJSJBMVL4T.1 size=1048576 2024-04-22T19:58:45.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.976+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~FJHJSJBMVL4T.2 size=433642 2024-04-22T19:58:45.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.976+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=FJHJSJBMVL4T tag=D7N4GN34Q4O1 type=write size=3579370 2024-04-22T19:58:45.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.977+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=LFMJP5W9JC8B type=del 2024-04-22T19:58:45.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.978+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=HEOIH1U30Y2Z type=write size=644820 2024-04-22T19:58:45.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.979+0000 7f5269a8e780 1 < completed write operation key=O9NFAVE09ZN7 tag=3PEVZ9KKSYLA type=write size=1009673 2024-04-22T19:58:45.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.981+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~E3255HZ1ILBZ.0 size=201763 2024-04-22T19:58:45.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.981+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=E3255HZ1ILBZ tag=GD09HHE00582 type=write size=1250339 2024-04-22T19:58:45.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.982+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~O4RUCQ0I2HNY.0 size=812087 2024-04-22T19:58:45.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.984+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=O4RUCQ0I2HNY tag=HI5VJJSZRRAQ type=write size=1860663 2024-04-22T19:58:45.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.985+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=VBXTWL1HLO8Q type=write size=496781 2024-04-22T19:58:45.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.986+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=OTMEO9T2SFSZ type=del 2024-04-22T19:58:45.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.987+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=SCMN958O3WYC type=del size=1637385 2024-04-22T19:58:45.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.988+0000 7f5269a8e780 -1 < failed to complete operation key=8V6ESIGMRO5G tag=6D7GYY61Z8IB type=del size=3475158: (22) Invalid argument 2024-04-22T19:58:45.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.989+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=HOSWYE3N07AD type=del size=1679500 2024-04-22T19:58:45.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.991+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=6EGDUOP38RSJ type=write size=328972 2024-04-22T19:58:45.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.993+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=6VM27X1KIL22 type=del size=511275 2024-04-22T19:58:45.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.995+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=3G0IFFMNK3US type=del 2024-04-22T19:58:45.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.997+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=0IST3RP1A8FI type=del size=145776 2024-04-22T19:58:45.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:45.998+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=VYGHB5G4AZ7Q type=del 2024-04-22T19:58:46.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.000+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~KI7YFLDBB2OU.0 size=1048576 2024-04-22T19:58:46.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.001+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~KI7YFLDBB2OU.1 size=968489 2024-04-22T19:58:46.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.001+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=KI7YFLDBB2OU tag=OE279DNEWTNF type=write size=3065641 2024-04-22T19:58:46.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.002+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~W5Q3ISR3MSTP.0 size=1048576 2024-04-22T19:58:46.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.003+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~W5Q3ISR3MSTP.1 size=811728 2024-04-22T19:58:46.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.005+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=W5Q3ISR3MSTP tag=FPF5APZ3T0BX type=write size=2908880 2024-04-22T19:58:46.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.006+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=RA369EDHAI1P type=del size=4157863 2024-04-22T19:58:46.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.007+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=ZDYOMACGYDBQ type=del 2024-04-22T19:58:46.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.009+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=IBCMJ27X9W7L type=write size=426301 2024-04-22T19:58:46.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.010+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=GS5W40CQSOQY type=del 2024-04-22T19:58:46.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.011+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=CDKBXZQLZW9I type=del size=248210 2024-04-22T19:58:46.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.013+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~7OJF6DYS6K2O.0 size=1048576 2024-04-22T19:58:46.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.013+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~7OJF6DYS6K2O.1 size=1048576 2024-04-22T19:58:46.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.014+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~7OJF6DYS6K2O.2 size=233002 2024-04-22T19:58:46.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.016+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=7OJF6DYS6K2O tag=90VON73N0K46 type=write size=3378730 2024-04-22T19:58:46.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.018+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=Z4SC3EQK7RG0 type=del size=4079805 2024-04-22T19:58:46.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.019+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~0RYKLZCSC02U.0 size=318724 2024-04-22T19:58:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.021+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=0RYKLZCSC02U tag=Z93QDZG88BTE type=write size=1367300 2024-04-22T19:58:46.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.022+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=X6YDDVAU3EM7 type=del size=1935430 2024-04-22T19:58:46.023 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.023+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=D4NMUYL4L3A6 type=write size=457939 2024-04-22T19:58:46.025 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.025+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~KR8IHQLLKUGY.0 size=1048576 2024-04-22T19:58:46.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.026+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~KR8IHQLLKUGY.1 size=82274 2024-04-22T19:58:46.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.026+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=KR8IHQLLKUGY tag=KHWSC0IEGNI6 type=write size=2179426 2024-04-22T19:58:46.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.027+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~Q2Q5ZZY2RB65.0 size=1048576 2024-04-22T19:58:46.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.028+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~Q2Q5ZZY2RB65.1 size=1048576 2024-04-22T19:58:46.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.029+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~Q2Q5ZZY2RB65.2 size=163944 2024-04-22T19:58:46.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.031+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=Q2Q5ZZY2RB65 tag=VOGDLFIFJ8BX type=write size=3309672 2024-04-22T19:58:46.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.033+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~RWDHU4YCUM55.0 size=1041216 2024-04-22T19:58:46.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.033+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=RWDHU4YCUM55 tag=T7CJEQ27PXGO type=write size=2089792 2024-04-22T19:58:46.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.034+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~FJHJSJBMVL4T.0 size=1048576 2024-04-22T19:58:46.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.035+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~FJHJSJBMVL4T.1 size=1048576 2024-04-22T19:58:46.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.036+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~FJHJSJBMVL4T.2 size=433642 2024-04-22T19:58:46.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.038+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=FJHJSJBMVL4T tag=D7N4GN34Q4O1 type=write size=3579370 2024-04-22T19:58:46.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.039+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=O68FV01V84JA type=del size=1529583 2024-04-22T19:58:46.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.041+0000 7f5269a8e780 1 < suggested remove operation key=M9M4SEEEZSFV tag=HEOIH1U30Y2Z type=write 2024-04-22T19:58:46.043 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.043+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~VR3BLPRYED1M.0 size=1048576 2024-04-22T19:58:46.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.044+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~VR3BLPRYED1M.1 size=354215 2024-04-22T19:58:46.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.044+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=VR3BLPRYED1M tag=L4VFFK553B3O type=write size=2451367 2024-04-22T19:58:46.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.045+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~E3255HZ1ILBZ.0 size=201763 2024-04-22T19:58:46.047 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.047+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=E3255HZ1ILBZ tag=GD09HHE00582 type=write size=1250339 2024-04-22T19:58:46.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.048+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=2X5UBTBX4V3E type=del size=174610 2024-04-22T19:58:46.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.049+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=VBXTWL1HLO8Q type=write size=496781 2024-04-22T19:58:46.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.050+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=EYJK72DTSFVE type=del size=2148893 2024-04-22T19:58:46.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.051+0000 7f5269a8e780 1 < canceled operation key=T2BV2LUCY4E3 tag=SCMN958O3WYC type=del size=1637385 2024-04-22T19:58:46.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.052+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=6FSBKD17IQW1 type=write size=333997 2024-04-22T19:58:46.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.053+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=HOSWYE3N07AD type=del 2024-04-22T19:58:46.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.055+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~OVDAKI20RIPW.0 size=751292 2024-04-22T19:58:46.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.055+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=OVDAKI20RIPW tag=09BMSHLHW53T type=write size=1799868 2024-04-22T19:58:46.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.057+0000 7f5269a8e780 1 < suggested update operation key=I91AD8FJ3P4T tag=6VM27X1KIL22 type=del size=511275 2024-04-22T19:58:46.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.059+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=7ZI2NO38FLIP type=del size=2024039 2024-04-22T19:58:46.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.060+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=0IST3RP1A8FI type=del 2024-04-22T19:58:46.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.061+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=1650OTO91OQI type=del size=3346643 2024-04-22T19:58:46.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.062+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~KI7YFLDBB2OU.0 size=1048576 2024-04-22T19:58:46.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.063+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~KI7YFLDBB2OU.1 size=968489 2024-04-22T19:58:46.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.065+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=KI7YFLDBB2OU tag=OE279DNEWTNF type=write size=3065641 2024-04-22T19:58:46.066 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.066+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=Y1WUMYBGFVT9 type=del size=2119769 2024-04-22T19:58:46.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.067+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=RA369EDHAI1P type=del 2024-04-22T19:58:46.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.068+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=GTWPMKDGZP5Y type=del size=3995370 2024-04-22T19:58:46.070 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.069+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=IBCMJ27X9W7L type=write size=426301 2024-04-22T19:58:46.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.071+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~0AR5MA7UJ1EB.0 size=1048576 2024-04-22T19:58:46.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.072+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~0AR5MA7UJ1EB.1 size=156952 2024-04-22T19:58:46.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.072+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=0AR5MA7UJ1EB tag=N887V1FH6K9Y type=write size=2254104 2024-04-22T19:58:46.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.074+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=CDKBXZQLZW9I type=del 2024-04-22T19:58:46.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.075+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=AW0F04RK41SU type=del size=35704 2024-04-22T19:58:46.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.076+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=Z4SC3EQK7RG0 type=del 2024-04-22T19:58:46.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.077+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=K87QBSREZ4WM type=write size=415984 2024-04-22T19:58:46.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.078+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=X6YDDVAU3EM7 type=del 2024-04-22T19:58:46.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.080+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~JFQQCB1DDZ7Z.0 size=1048576 2024-04-22T19:58:46.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.081+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~JFQQCB1DDZ7Z.1 size=1048576 2024-04-22T19:58:46.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.082+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~JFQQCB1DDZ7Z.2 size=996263 2024-04-22T19:58:46.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.082+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=JFQQCB1DDZ7Z tag=CXB0LMGTHRP9 type=write size=4141991 2024-04-22T19:58:46.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.083+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~KR8IHQLLKUGY.0 size=1048576 2024-04-22T19:58:46.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.083+0000 7f5269a8e780 1 < canceled part key=_multipart_ZC4ZOHUJ3HQD.2~KR8IHQLLKUGY.1 size=82274 2024-04-22T19:58:46.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.085+0000 7f5269a8e780 1 < canceled multipart upload key=ZC4ZOHUJ3HQD upload=KR8IHQLLKUGY tag=KHWSC0IEGNI6 type=write size=2179426 2024-04-22T19:58:46.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.086+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=0URKGJQFYPCH type=write size=543283 2024-04-22T19:58:46.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.087+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~RWDHU4YCUM55.0 size=1041216 2024-04-22T19:58:46.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.089+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=RWDHU4YCUM55 tag=T7CJEQ27PXGO type=write size=2089792 2024-04-22T19:58:46.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.090+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=I9WQ39ZQS3FZ type=del size=755881 2024-04-22T19:58:46.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.091+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=O68FV01V84JA type=del 2024-04-22T19:58:46.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.092+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=KNM96J1QWI2Y type=del size=1703018 2024-04-22T19:58:46.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.093+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~VR3BLPRYED1M.0 size=1048576 2024-04-22T19:58:46.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.094+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~VR3BLPRYED1M.1 size=354215 2024-04-22T19:58:46.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.096+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=VR3BLPRYED1M tag=L4VFFK553B3O type=write size=2451367 2024-04-22T19:58:46.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.097+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=LJEJO4SY6MQI type=del size=2256013 2024-04-22T19:58:46.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.098+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=2X5UBTBX4V3E type=del 2024-04-22T19:58:46.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.101+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=OU7ZN55MBPIR type=del size=2793144 2024-04-22T19:58:46.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.102+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=EYJK72DTSFVE type=del 2024-04-22T19:58:46.103 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.103+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=PS227IYQNK3I type=del size=3712597 2024-04-22T19:58:46.104 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.105+0000 7f5269a8e780 1 < completed write operation key=M9M4SEEEZSFV tag=6FSBKD17IQW1 type=write size=333997 2024-04-22T19:58:46.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.106+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=Q0FKO7F8DHVF type=del size=4022254 2024-04-22T19:58:46.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.106+0000 7f5269a8e780 1 < canceled part key=_multipart_B0Y7ZTPC0975.2~OVDAKI20RIPW.0 size=751292 2024-04-22T19:58:46.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.108+0000 7f5269a8e780 1 < canceled multipart upload key=B0Y7ZTPC0975 upload=OVDAKI20RIPW tag=09BMSHLHW53T type=write size=1799868 2024-04-22T19:58:46.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.110+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~MMTDMY4ZXT0N.0 size=1048576 2024-04-22T19:58:46.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.111+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~MMTDMY4ZXT0N.1 size=1048576 2024-04-22T19:58:46.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.112+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~MMTDMY4ZXT0N.2 size=25274 2024-04-22T19:58:46.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.112+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=MMTDMY4ZXT0N tag=E5UEY7L3TMY0 type=write size=3171002 2024-04-22T19:58:46.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.113+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=7ZI2NO38FLIP type=del 2024-04-22T19:58:46.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.114+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=8FSPQ8KSSKUE type=del size=2170575 2024-04-22T19:58:46.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.116+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=1650OTO91OQI type=del 2024-04-22T19:58:46.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.117+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=3ZL76EI8C273 type=del size=1918940 2024-04-22T19:58:46.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.118+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=Y1WUMYBGFVT9 type=del 2024-04-22T19:58:46.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.120+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~WB417M1BIC38.0 size=1048576 2024-04-22T19:58:46.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.121+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~WB417M1BIC38.1 size=814705 2024-04-22T19:58:46.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.121+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=WB417M1BIC38 tag=3A0ITLPK2LIP type=write size=2911857 2024-04-22T19:58:46.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.122+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=GTWPMKDGZP5Y type=del 2024-04-22T19:58:46.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.123+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=WDFITOHYLCMT type=del size=2494030 2024-04-22T19:58:46.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.124+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~0AR5MA7UJ1EB.0 size=1048576 2024-04-22T19:58:46.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.125+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~0AR5MA7UJ1EB.1 size=156952 2024-04-22T19:58:46.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.127+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=0AR5MA7UJ1EB tag=N887V1FH6K9Y type=write size=2254104 2024-04-22T19:58:46.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.129+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~3R2KK1AX1WJH.0 size=1048576 2024-04-22T19:58:46.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.130+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~3R2KK1AX1WJH.1 size=468244 2024-04-22T19:58:46.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.130+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=3R2KK1AX1WJH tag=80F9SXP1Y0H7 type=write size=2565396 2024-04-22T19:58:46.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.131+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=AW0F04RK41SU type=del 2024-04-22T19:58:46.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.133+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~0VQ0ZKAWYP72.0 size=1048576 2024-04-22T19:58:46.134 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.134+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~0VQ0ZKAWYP72.1 size=1048576 2024-04-22T19:58:46.134 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.135+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~0VQ0ZKAWYP72.2 size=358458 2024-04-22T19:58:46.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.135+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=0VQ0ZKAWYP72 tag=DWNEQ4K6R4I1 type=write size=3504186 2024-04-22T19:58:46.137 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.137+0000 7f5269a8e780 1 < suggested remove operation key=DX5YUYQRZKWA tag=K87QBSREZ4WM type=write 2024-04-22T19:58:46.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.138+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=KTPHAIFL8KM4 type=del size=2643771 2024-04-22T19:58:46.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.139+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~JFQQCB1DDZ7Z.0 size=1048576 2024-04-22T19:58:46.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.140+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~JFQQCB1DDZ7Z.1 size=1048576 2024-04-22T19:58:46.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.141+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~JFQQCB1DDZ7Z.2 size=996263 2024-04-22T19:58:46.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.143+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=JFQQCB1DDZ7Z tag=CXB0LMGTHRP9 type=write size=4141991 2024-04-22T19:58:46.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.144+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=C8EJP75W8CCL type=del size=2439679 2024-04-22T19:58:46.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.145+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=0URKGJQFYPCH type=write size=543283 2024-04-22T19:58:46.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.147+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~1MBGHBM3LIR2.0 size=786348 2024-04-22T19:58:46.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.147+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=1MBGHBM3LIR2 tag=AIJJ5QTYZR0C type=write size=1834924 2024-04-22T19:58:46.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.149+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=I9WQ39ZQS3FZ type=del 2024-04-22T19:58:46.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.150+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=RG77WTMQ24BL type=del size=2860490 2024-04-22T19:58:46.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.153+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=KNM96J1QWI2Y type=del 2024-04-22T19:58:46.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.154+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=JFGKURVACGZK type=write size=1025370 2024-04-22T19:58:46.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.156+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=LJEJO4SY6MQI type=del 2024-04-22T19:58:46.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.158+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~XDCR84RL3G8L.0 size=1048576 2024-04-22T19:58:46.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.159+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~XDCR84RL3G8L.1 size=624286 2024-04-22T19:58:46.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.159+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=XDCR84RL3G8L tag=ZLTMC1S4S5T0 type=write size=2721438 2024-04-22T19:58:46.160 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.160+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=OU7ZN55MBPIR type=del 2024-04-22T19:58:46.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.162+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~3OD9R82X3VE1.0 size=1048576 2024-04-22T19:58:46.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.163+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~3OD9R82X3VE1.1 size=1048576 2024-04-22T19:58:46.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.164+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~3OD9R82X3VE1.2 size=513262 2024-04-22T19:58:46.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.164+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=3OD9R82X3VE1 tag=IYDYHHB8VAL8 type=write size=3658990 2024-04-22T19:58:46.165 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.165+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=PS227IYQNK3I type=del 2024-04-22T19:58:46.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.167+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~PY89T2TY7MEH.0 size=618401 2024-04-22T19:58:46.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.167+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=PY89T2TY7MEH tag=J2TNVNLIH5WX type=write size=1666977 2024-04-22T19:58:46.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.168+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=Q0FKO7F8DHVF type=del 2024-04-22T19:58:46.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.170+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~ECKF8BIQYE6X.0 size=728013 2024-04-22T19:58:46.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.170+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=ECKF8BIQYE6X tag=1IDH940U8147 type=write size=1776589 2024-04-22T19:58:46.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.170+0000 7f5269a8e780 1 < canceled part key=_multipart_SN2BZATJGUWA.2~MMTDMY4ZXT0N.0 size=1048576 2024-04-22T19:58:46.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.171+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~MMTDMY4ZXT0N.1 size=1048576 2024-04-22T19:58:46.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.171+0000 7f5269a8e780 1 < canceled part key=_multipart_SN2BZATJGUWA.2~MMTDMY4ZXT0N.2 size=25274 2024-04-22T19:58:46.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.173+0000 7f5269a8e780 1 < canceled multipart upload key=SN2BZATJGUWA upload=MMTDMY4ZXT0N tag=E5UEY7L3TMY0 type=write size=3171002 2024-04-22T19:58:46.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.175+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~DQB9ZGKQDEV5.0 size=1048576 2024-04-22T19:58:46.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.175+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~DQB9ZGKQDEV5.1 size=1048576 2024-04-22T19:58:46.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.176+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~DQB9ZGKQDEV5.2 size=949934 2024-04-22T19:58:46.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.176+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=DQB9ZGKQDEV5 tag=U801RM5MQ6MK type=write size=4095662 2024-04-22T19:58:46.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.177+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=8FSPQ8KSSKUE type=del 2024-04-22T19:58:46.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.179+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~IMQGT28U5X9O.0 size=1048576 2024-04-22T19:58:46.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.180+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~IMQGT28U5X9O.1 size=583130 2024-04-22T19:58:46.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.180+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=IMQGT28U5X9O tag=VBGO7OGOC1YQ type=write size=2680282 2024-04-22T19:58:46.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.182+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=3ZL76EI8C273 type=del size=1918940 2024-04-22T19:58:46.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.183+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=GEZI5D2522KU type=del size=2522263 2024-04-22T19:58:46.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.184+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~WB417M1BIC38.0 size=1048576 2024-04-22T19:58:46.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.185+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~WB417M1BIC38.1 size=814705 2024-04-22T19:58:46.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.187+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=WB417M1BIC38 tag=3A0ITLPK2LIP type=write size=2911857 2024-04-22T19:58:46.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.189+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~RO2ETZEHY575.0 size=1048576 2024-04-22T19:58:46.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.190+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~RO2ETZEHY575.1 size=1048576 2024-04-22T19:58:46.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.191+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~RO2ETZEHY575.2 size=275628 2024-04-22T19:58:46.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.191+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=RO2ETZEHY575 tag=X9FJG1CZ1S4V type=write size=3421356 2024-04-22T19:58:46.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.192+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=WDFITOHYLCMT type=del 2024-04-22T19:58:46.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.193+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=AN4E10U2V7OF type=del size=2660027 2024-04-22T19:58:46.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.194+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~3R2KK1AX1WJH.0 size=1048576 2024-04-22T19:58:46.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.195+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~3R2KK1AX1WJH.1 size=468244 2024-04-22T19:58:46.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.197+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=3R2KK1AX1WJH tag=80F9SXP1Y0H7 type=write size=2565396 2024-04-22T19:58:46.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.198+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=2XGQK326EQPK type=del size=472534 2024-04-22T19:58:46.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.200+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~0VQ0ZKAWYP72.0 size=1048576 2024-04-22T19:58:46.200 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.201+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~0VQ0ZKAWYP72.1 size=1048576 2024-04-22T19:58:46.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.202+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~0VQ0ZKAWYP72.2 size=358458 2024-04-22T19:58:46.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.204+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=0VQ0ZKAWYP72 tag=DWNEQ4K6R4I1 type=write size=3504186 2024-04-22T19:58:46.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.205+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=HTBWGXE1DPN9 type=del size=229237 2024-04-22T19:58:46.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.207+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=KTPHAIFL8KM4 type=del 2024-04-22T19:58:46.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.209+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=W5KP1K5IWLL2 type=del size=1082991 2024-04-22T19:58:46.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.212+0000 7f5269a8e780 1 < suggested update operation key=ZC4ZOHUJ3HQD tag=C8EJP75W8CCL type=del size=2439679 2024-04-22T19:58:46.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.214+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~VHAE4P30QZT7.0 size=961910 2024-04-22T19:58:46.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.214+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=VHAE4P30QZT7 tag=TJAZV1DF3EYL type=write size=2010486 2024-04-22T19:58:46.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.215+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~1MBGHBM3LIR2.0 size=786348 2024-04-22T19:58:46.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.218+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=1MBGHBM3LIR2 tag=AIJJ5QTYZR0C type=write size=1834924 2024-04-22T19:58:46.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.219+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=NN78FS3GC5WI type=write size=158578 2024-04-22T19:58:46.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.221+0000 7f5269a8e780 1 < suggested update operation key=RY1O7X1CWZMK tag=RG77WTMQ24BL type=del size=2860490 2024-04-22T19:58:46.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.224+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~G2SPTU0WGL83.0 size=1048576 2024-04-22T19:58:46.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.225+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~G2SPTU0WGL83.1 size=1048576 2024-04-22T19:58:46.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.226+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~G2SPTU0WGL83.2 size=698996 2024-04-22T19:58:46.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.226+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=G2SPTU0WGL83 tag=T8KIZ7F9LEQ1 type=write size=3844724 2024-04-22T19:58:46.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.228+0000 7f5269a8e780 1 < suggested update operation key=ZC4ZOHUJ3HQD tag=JFGKURVACGZK type=write size=1025370 2024-04-22T19:58:46.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.230+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~IZ6C12MG3XFJ.0 size=1048576 2024-04-22T19:58:46.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.231+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~IZ6C12MG3XFJ.1 size=228680 2024-04-22T19:58:46.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.231+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=IZ6C12MG3XFJ tag=QFOGBCRJO1ZL type=write size=2325832 2024-04-22T19:58:46.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.232+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~XDCR84RL3G8L.0 size=1048576 2024-04-22T19:58:46.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.233+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~XDCR84RL3G8L.1 size=624286 2024-04-22T19:58:46.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.235+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=XDCR84RL3G8L tag=ZLTMC1S4S5T0 type=write size=2721438 2024-04-22T19:58:46.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.237+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~LOJOI3QQXGI0.0 size=1048576 2024-04-22T19:58:46.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.238+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~LOJOI3QQXGI0.1 size=873269 2024-04-22T19:58:46.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.238+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=LOJOI3QQXGI0 tag=4XGUDHWECZ76 type=write size=2970421 2024-04-22T19:58:46.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.239+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~3OD9R82X3VE1.0 size=1048576 2024-04-22T19:58:46.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.240+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~3OD9R82X3VE1.1 size=1048576 2024-04-22T19:58:46.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.240+0000 7f5269a8e780 1 < canceled part key=_multipart_VPAVX5FXYWZO.2~3OD9R82X3VE1.2 size=513262 2024-04-22T19:58:46.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.242+0000 7f5269a8e780 1 < canceled multipart upload key=VPAVX5FXYWZO upload=3OD9R82X3VE1 tag=IYDYHHB8VAL8 type=write size=3658990 2024-04-22T19:58:46.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.243+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=0I3TRDJXUS37 type=del size=1403817 2024-04-22T19:58:46.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.244+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~PY89T2TY7MEH.0 size=618401 2024-04-22T19:58:46.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.246+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=PY89T2TY7MEH tag=J2TNVNLIH5WX type=write size=1666977 2024-04-22T19:58:46.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.247+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=R38XQJXRAO4N type=del size=1011947 2024-04-22T19:58:46.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.247+0000 7f5269a8e780 1 < canceled part key=_multipart_70MLQGJPQYPY.2~ECKF8BIQYE6X.0 size=728013 2024-04-22T19:58:46.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.249+0000 7f5269a8e780 1 < canceled multipart upload key=70MLQGJPQYPY upload=ECKF8BIQYE6X tag=1IDH940U8147 type=write size=1776589 2024-04-22T19:58:46.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.250+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=N2L33WW050KH type=del size=697696 2024-04-22T19:58:46.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.251+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~DQB9ZGKQDEV5.0 size=1048576 2024-04-22T19:58:46.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.252+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~DQB9ZGKQDEV5.1 size=1048576 2024-04-22T19:58:46.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.253+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~DQB9ZGKQDEV5.2 size=949934 2024-04-22T19:58:46.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.255+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=DQB9ZGKQDEV5 tag=U801RM5MQ6MK type=write size=4095662 2024-04-22T19:58:46.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.257+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~GRT1ATSWQFV1.0 size=1048576 2024-04-22T19:58:46.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.258+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~GRT1ATSWQFV1.1 size=1048576 2024-04-22T19:58:46.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.259+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~GRT1ATSWQFV1.2 size=665624 2024-04-22T19:58:46.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.259+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=GRT1ATSWQFV1 tag=BCHWNI0LQIMX type=write size=3811352 2024-04-22T19:58:46.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.260+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~IMQGT28U5X9O.0 size=1048576 2024-04-22T19:58:46.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.261+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~IMQGT28U5X9O.1 size=583130 2024-04-22T19:58:46.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.263+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=IMQGT28U5X9O tag=VBGO7OGOC1YQ type=write size=2680282 2024-04-22T19:58:46.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.265+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~9A3C1AZ62XXN.0 size=1048576 2024-04-22T19:58:46.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.265+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~9A3C1AZ62XXN.1 size=1048576 2024-04-22T19:58:46.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.266+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~9A3C1AZ62XXN.2 size=670655 2024-04-22T19:58:46.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.266+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=9A3C1AZ62XXN tag=2KVHNERN4TC4 type=write size=3816383 2024-04-22T19:58:46.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.267+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=GEZI5D2522KU type=del 2024-04-22T19:58:46.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.270+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~DP1NGS4YZURM.0 size=1048576 2024-04-22T19:58:46.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.270+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~DP1NGS4YZURM.1 size=1048576 2024-04-22T19:58:46.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.271+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~DP1NGS4YZURM.2 size=979926 2024-04-22T19:58:46.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.271+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=DP1NGS4YZURM tag=I63LFBY915TN type=write size=4125654 2024-04-22T19:58:46.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.273+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~RO2ETZEHY575.0 size=1048576 2024-04-22T19:58:46.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.274+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~RO2ETZEHY575.1 size=1048576 2024-04-22T19:58:46.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.275+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~RO2ETZEHY575.2 size=275628 2024-04-22T19:58:46.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.277+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=RO2ETZEHY575 tag=X9FJG1CZ1S4V type=write size=3421356 2024-04-22T19:58:46.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.279+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~Z5SVR46O4314.0 size=1048576 2024-04-22T19:58:46.280 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.280+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~Z5SVR46O4314.1 size=1048576 2024-04-22T19:58:46.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.281+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~Z5SVR46O4314.2 size=35601 2024-04-22T19:58:46.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.281+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=Z5SVR46O4314 tag=IBXU1ECRX9SE type=write size=3181329 2024-04-22T19:58:46.282 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.282+0000 7f5269a8e780 1 < canceled operation key=IBH6W1NINUHC tag=AN4E10U2V7OF type=del size=2660027 2024-04-22T19:58:46.284 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.284+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~NRV3VX829SV7.0 size=534519 2024-04-22T19:58:46.284 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.284+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=NRV3VX829SV7 tag=Z53B4PNLYC2B type=write size=1583095 2024-04-22T19:58:46.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.287+0000 7f5269a8e780 1 < suggested remove operation key=Y5HR8XW4L0SP tag=2XGQK326EQPK type=del 2024-04-22T19:58:46.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.289+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~5ARG5DTL24WS.0 size=1048576 2024-04-22T19:58:46.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.290+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~5ARG5DTL24WS.1 size=1048576 2024-04-22T19:58:46.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.291+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~5ARG5DTL24WS.2 size=883765 2024-04-22T19:58:46.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.291+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=5ARG5DTL24WS tag=RBDCOYDLSMSL type=write size=4029493 2024-04-22T19:58:46.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.292+0000 7f5269a8e780 1 < canceled operation key=DX5YUYQRZKWA tag=HTBWGXE1DPN9 type=del size=229237 2024-04-22T19:58:46.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.294+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~GYRYOPCBSEB4.0 size=1048576 2024-04-22T19:58:46.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.295+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~GYRYOPCBSEB4.1 size=1048576 2024-04-22T19:58:46.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.296+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~GYRYOPCBSEB4.2 size=544840 2024-04-22T19:58:46.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.296+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=GYRYOPCBSEB4 tag=RQYKOAIQTR6I type=write size=3690568 2024-04-22T19:58:46.297 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.297+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=W5KP1K5IWLL2 type=del 2024-04-22T19:58:46.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.298+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=LB74GSQ5TFI5 type=del size=864713 2024-04-22T19:58:46.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.299+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~VHAE4P30QZT7.0 size=961910 2024-04-22T19:58:46.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.301+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=VHAE4P30QZT7 tag=TJAZV1DF3EYL type=write size=2010486 2024-04-22T19:58:46.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.302+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=6HSJUPYUGL8G type=del size=3143419 2024-04-22T19:58:46.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.304+0000 7f5269a8e780 1 < completed write operation key=Q1UKF4UMJ2HO tag=NN78FS3GC5WI type=write size=158578 2024-04-22T19:58:46.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.306+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~U1BRTUSAJNM3.0 size=1048576 2024-04-22T19:58:46.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.307+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~U1BRTUSAJNM3.1 size=608123 2024-04-22T19:58:46.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.307+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=U1BRTUSAJNM3 tag=031F2JIX7EXG type=write size=2705275 2024-04-22T19:58:46.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.308+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~G2SPTU0WGL83.0 size=1048576 2024-04-22T19:58:46.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.308+0000 7f5269a8e780 1 < canceled part key=_multipart_KE7YGPTF1MOR.2~G2SPTU0WGL83.1 size=1048576 2024-04-22T19:58:46.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.309+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~G2SPTU0WGL83.2 size=698996 2024-04-22T19:58:46.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.311+0000 7f5269a8e780 1 < canceled multipart upload key=KE7YGPTF1MOR upload=G2SPTU0WGL83 tag=T8KIZ7F9LEQ1 type=write size=3844724 2024-04-22T19:58:46.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.312+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=KXBFJ5QJXVNH type=del size=1507675 2024-04-22T19:58:46.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.313+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~IZ6C12MG3XFJ.0 size=1048576 2024-04-22T19:58:46.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.315+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~IZ6C12MG3XFJ.1 size=228680 2024-04-22T19:58:46.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.317+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=IZ6C12MG3XFJ tag=QFOGBCRJO1ZL type=write size=2325832 2024-04-22T19:58:46.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.320+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~ZJEMD0QQW38D.0 size=1048576 2024-04-22T19:58:46.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.321+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~ZJEMD0QQW38D.1 size=739393 2024-04-22T19:58:46.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.321+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=ZJEMD0QQW38D tag=RL8LIPOTMG3S type=write size=2836545 2024-04-22T19:58:46.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.322+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~LOJOI3QQXGI0.0 size=1048576 2024-04-22T19:58:46.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.323+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~LOJOI3QQXGI0.1 size=873269 2024-04-22T19:58:46.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.325+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=LOJOI3QQXGI0 tag=4XGUDHWECZ76 type=write size=2970421 2024-04-22T19:58:46.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.326+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=8Z0Q3EFSWJZU type=write size=219357 2024-04-22T19:58:46.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.328+0000 7f5269a8e780 1 < suggested remove operation key=GRSL48RUEBX8 tag=0I3TRDJXUS37 type=del 2024-04-22T19:58:46.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.329+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=IKA0HALDDQWQ type=del size=3498128 2024-04-22T19:58:46.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.330+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=R38XQJXRAO4N type=del 2024-04-22T19:58:46.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.332+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~FNBGBFE199XR.0 size=1048576 2024-04-22T19:58:46.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.333+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~FNBGBFE199XR.1 size=1041973 2024-04-22T19:58:46.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.333+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=FNBGBFE199XR tag=6WKE8Z0L3KFC type=write size=3139125 2024-04-22T19:58:46.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.335+0000 7f5269a8e780 1 < suggested remove operation key=O9NFAVE09ZN7 tag=N2L33WW050KH type=del 2024-04-22T19:58:46.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.336+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=QK83YUZL7SL4 type=del size=1196458 2024-04-22T19:58:46.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.339+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~GRT1ATSWQFV1.0 size=1048576 2024-04-22T19:58:46.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.340+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~GRT1ATSWQFV1.1 size=1048576 2024-04-22T19:58:46.341 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.341+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~GRT1ATSWQFV1.2 size=665624 2024-04-22T19:58:46.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.343+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=GRT1ATSWQFV1 tag=BCHWNI0LQIMX type=write size=3811352 2024-04-22T19:58:46.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.344+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=2LJNZQL2H1GK type=del size=3679634 2024-04-22T19:58:46.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.345+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~9A3C1AZ62XXN.0 size=1048576 2024-04-22T19:58:46.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.346+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~9A3C1AZ62XXN.1 size=1048576 2024-04-22T19:58:46.346 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.346+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~9A3C1AZ62XXN.2 size=670655 2024-04-22T19:58:46.348 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.348+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=9A3C1AZ62XXN tag=2KVHNERN4TC4 type=write size=3816383 2024-04-22T19:58:46.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.350+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=6UWNX0LVMLPA type=del size=3143629 2024-04-22T19:58:46.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.351+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~DP1NGS4YZURM.0 size=1048576 2024-04-22T19:58:46.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.352+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~DP1NGS4YZURM.1 size=1048576 2024-04-22T19:58:46.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.353+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~DP1NGS4YZURM.2 size=979926 2024-04-22T19:58:46.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.355+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=DP1NGS4YZURM tag=I63LFBY915TN type=write size=4125654 2024-04-22T19:58:46.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.356+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=JV40EIL1UFKH type=write size=407214 2024-04-22T19:58:46.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.357+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~Z5SVR46O4314.0 size=1048576 2024-04-22T19:58:46.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.358+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~Z5SVR46O4314.1 size=1048576 2024-04-22T19:58:46.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.358+0000 7f5269a8e780 1 < canceled part key=_multipart_Q1UKF4UMJ2HO.2~Z5SVR46O4314.2 size=35601 2024-04-22T19:58:46.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.360+0000 7f5269a8e780 1 < canceled multipart upload key=Q1UKF4UMJ2HO upload=Z5SVR46O4314 tag=IBXU1ECRX9SE type=write size=3181329 2024-04-22T19:58:46.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.361+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=OC7IN7YT5U9Q type=del size=1344387 2024-04-22T19:58:46.362 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.362+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~NRV3VX829SV7.0 size=534519 2024-04-22T19:58:46.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.364+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=NRV3VX829SV7 tag=Z53B4PNLYC2B type=write size=1583095 2024-04-22T19:58:46.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.366+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~O9OS200NJCDK.0 size=1048576 2024-04-22T19:58:46.367 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.367+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~O9OS200NJCDK.1 size=1048576 2024-04-22T19:58:46.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.368+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~O9OS200NJCDK.2 size=874574 2024-04-22T19:58:46.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.368+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=O9OS200NJCDK tag=TAZQ52B5MRCG type=write size=4020302 2024-04-22T19:58:46.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.369+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~5ARG5DTL24WS.0 size=1048576 2024-04-22T19:58:46.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.369+0000 7f5269a8e780 1 < canceled part key=_multipart_HV8J6YM0YNC1.2~5ARG5DTL24WS.1 size=1048576 2024-04-22T19:58:46.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.370+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~5ARG5DTL24WS.2 size=883765 2024-04-22T19:58:46.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.372+0000 7f5269a8e780 1 < canceled multipart upload key=HV8J6YM0YNC1 upload=5ARG5DTL24WS tag=RBDCOYDLSMSL type=write size=4029493 2024-04-22T19:58:46.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.373+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=TRRABNTCEZ7J type=del size=3718420 2024-04-22T19:58:46.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.374+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~GYRYOPCBSEB4.0 size=1048576 2024-04-22T19:58:46.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.375+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~GYRYOPCBSEB4.1 size=1048576 2024-04-22T19:58:46.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.375+0000 7f5269a8e780 1 < canceled part key=_multipart_IBH6W1NINUHC.2~GYRYOPCBSEB4.2 size=544840 2024-04-22T19:58:46.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.377+0000 7f5269a8e780 1 < canceled multipart upload key=IBH6W1NINUHC upload=GYRYOPCBSEB4 tag=RQYKOAIQTR6I type=write size=3690568 2024-04-22T19:58:46.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.378+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~B2D9SPJ3DW6D.0 size=498961 2024-04-22T19:58:46.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.378+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=B2D9SPJ3DW6D tag=RZE1JEQ2VZ4P type=write size=1547537 2024-04-22T19:58:46.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.380+0000 7f5269a8e780 1 < canceled operation key=GP1C5ULUU143 tag=LB74GSQ5TFI5 type=del size=864713 2024-04-22T19:58:46.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.381+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=4921B4194VGH type=write size=385478 2024-04-22T19:58:46.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.382+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=6HSJUPYUGL8G type=del 2024-04-22T19:58:46.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.383+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=8YSNY0NFEHM2 type=del size=3591843 2024-04-22T19:58:46.384 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.384+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~U1BRTUSAJNM3.0 size=1048576 2024-04-22T19:58:46.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.385+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~U1BRTUSAJNM3.1 size=608123 2024-04-22T19:58:46.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.387+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=U1BRTUSAJNM3 tag=031F2JIX7EXG type=write size=2705275 2024-04-22T19:58:46.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.388+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=KFAXW1NX1TU0 type=del size=574284 2024-04-22T19:58:46.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.389+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=KXBFJ5QJXVNH type=del 2024-04-22T19:58:46.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.390+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=CFGN5G7N7ZSZ type=del size=1056501 2024-04-22T19:58:46.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.391+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~ZJEMD0QQW38D.0 size=1048576 2024-04-22T19:58:46.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.392+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~ZJEMD0QQW38D.1 size=739393 2024-04-22T19:58:46.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.394+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=ZJEMD0QQW38D tag=RL8LIPOTMG3S type=write size=2836545 2024-04-22T19:58:46.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.395+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=BTGOPRRY20SP type=del size=1931971 2024-04-22T19:58:46.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.396+0000 7f5269a8e780 1 < completed write operation key=IBH6W1NINUHC tag=8Z0Q3EFSWJZU type=write size=219357 2024-04-22T19:58:46.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.398+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~E5N7JAG2CTOT.0 size=1048576 2024-04-22T19:58:46.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.399+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~E5N7JAG2CTOT.1 size=1048576 2024-04-22T19:58:46.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.400+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~E5N7JAG2CTOT.2 size=946049 2024-04-22T19:58:46.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.400+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=E5N7JAG2CTOT tag=Z5YTBTO4UI85 type=write size=4091777 2024-04-22T19:58:46.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.402+0000 7f5269a8e780 1 < suggested update operation key=I91AD8FJ3P4T tag=IKA0HALDDQWQ type=del size=3498128 2024-04-22T19:58:46.404 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.404+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=715GD1B6Z7AF type=write size=900790 2024-04-22T19:58:46.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.405+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~FNBGBFE199XR.0 size=1048576 2024-04-22T19:58:46.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.406+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~FNBGBFE199XR.1 size=1041973 2024-04-22T19:58:46.408 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.408+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=FNBGBFE199XR tag=6WKE8Z0L3KFC type=write size=3139125 2024-04-22T19:58:46.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.409+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=4NE2P9YWJV19 type=del size=1746572 2024-04-22T19:58:46.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.410+0000 7f5269a8e780 1 < canceled operation key=EHRS6DQDRXVS tag=QK83YUZL7SL4 type=del size=1196458 2024-04-22T19:58:46.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.411+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=66BBJI7A65G4 type=del size=3403035 2024-04-22T19:58:46.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.412+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=2LJNZQL2H1GK type=del 2024-04-22T19:58:46.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.414+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=P3TJB0QMGH5P type=del size=3443463 2024-04-22T19:58:46.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.415+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=6UWNX0LVMLPA type=del 2024-04-22T19:58:46.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.417+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~HMGJH2DD16SO.0 size=338932 2024-04-22T19:58:46.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.417+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=HMGJH2DD16SO tag=274UQRT1JXS9 type=write size=1387508 2024-04-22T19:58:46.418 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.418+0000 7f5269a8e780 1 < canceled operation key=70MLQGJPQYPY tag=JV40EIL1UFKH type=write size=407214 2024-04-22T19:58:46.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.419+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=2B1STING84QZ type=write size=115745 2024-04-22T19:58:46.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.421+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=OC7IN7YT5U9Q type=del 2024-04-22T19:58:46.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.426+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~98LOK8LHVPOZ.0 size=1048576 2024-04-22T19:58:46.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.427+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~98LOK8LHVPOZ.1 size=1048576 2024-04-22T19:58:46.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.428+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~98LOK8LHVPOZ.2 size=624572 2024-04-22T19:58:46.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.428+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=98LOK8LHVPOZ tag=PJ08FTXB25I7 type=write size=3770300 2024-04-22T19:58:46.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.429+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~O9OS200NJCDK.0 size=1048576 2024-04-22T19:58:46.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.430+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~O9OS200NJCDK.1 size=1048576 2024-04-22T19:58:46.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.431+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~O9OS200NJCDK.2 size=874574 2024-04-22T19:58:46.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.433+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=O9OS200NJCDK tag=TAZQ52B5MRCG type=write size=4020302 2024-04-22T19:58:46.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.434+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=BWHA5NUR1UUC type=del size=305091 2024-04-22T19:58:46.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.436+0000 7f5269a8e780 1 < suggested update operation key=T2BV2LUCY4E3 tag=TRRABNTCEZ7J type=del size=3718420 2024-04-22T19:58:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.438+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~2Y3VW5PRNY6G.0 size=782120 2024-04-22T19:58:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.438+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=2Y3VW5PRNY6G tag=IZD3WLR56BPR type=write size=1830696 2024-04-22T19:58:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.440+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~B2D9SPJ3DW6D.0 size=498961 2024-04-22T19:58:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.441+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=B2D9SPJ3DW6D tag=RZE1JEQ2VZ4P type=write size=1547537 2024-04-22T19:58:46.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.443+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~ND8U24XNWSFE.0 size=1048576 2024-04-22T19:58:46.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.444+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~ND8U24XNWSFE.1 size=249117 2024-04-22T19:58:46.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.444+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=ND8U24XNWSFE tag=3SS4P268Y8ER type=write size=2346269 2024-04-22T19:58:46.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.445+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=4921B4194VGH type=write size=385478 2024-04-22T19:58:46.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.447+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=A1M3UBWWM17Z type=write size=25431 2024-04-22T19:58:46.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.448+0000 7f5269a8e780 -1 < failed to complete operation key=T2BV2LUCY4E3 tag=8YSNY0NFEHM2 type=del size=3591843: (22) Invalid argument 2024-04-22T19:58:46.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.450+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~0YTXF0TRKE5S.0 size=1048576 2024-04-22T19:58:46.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.450+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~0YTXF0TRKE5S.1 size=894940 2024-04-22T19:58:46.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.450+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=0YTXF0TRKE5S tag=HFASSFFA6PDX type=write size=2992092 2024-04-22T19:58:46.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.453+0000 7f5269a8e780 1 < suggested update operation key=M9M4SEEEZSFV tag=KFAXW1NX1TU0 type=del size=574284 2024-04-22T19:58:46.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.455+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~S6758NBE2M0I.0 size=1048576 2024-04-22T19:58:46.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.456+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~S6758NBE2M0I.1 size=565174 2024-04-22T19:58:46.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.456+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=S6758NBE2M0I tag=T1QNUXHE486L type=write size=2662326 2024-04-22T19:58:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.457+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=CFGN5G7N7ZSZ type=del 2024-04-22T19:58:46.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.459+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~ZD58LXS75CD3.0 size=1048576 2024-04-22T19:58:46.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.460+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~ZD58LXS75CD3.1 size=240064 2024-04-22T19:58:46.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.460+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=ZD58LXS75CD3 tag=C4OM42A6SJH3 type=write size=2337216 2024-04-22T19:58:46.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.462+0000 7f5269a8e780 1 < suggested update operation key=8V6ESIGMRO5G tag=BTGOPRRY20SP type=del size=1931971 2024-04-22T19:58:46.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.463+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=YCG7BZH3UD8Z type=del size=1614032 2024-04-22T19:58:46.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.465+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~E5N7JAG2CTOT.0 size=1048576 2024-04-22T19:58:46.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.466+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~E5N7JAG2CTOT.1 size=1048576 2024-04-22T19:58:46.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.466+0000 7f5269a8e780 1 < canceled part key=_multipart_B0Y7ZTPC0975.2~E5N7JAG2CTOT.2 size=946049 2024-04-22T19:58:46.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.468+0000 7f5269a8e780 1 < canceled multipart upload key=B0Y7ZTPC0975 upload=E5N7JAG2CTOT tag=Z5YTBTO4UI85 type=write size=4091777 2024-04-22T19:58:46.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.470+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~RGGOFT03EZMQ.0 size=1048576 2024-04-22T19:58:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.471+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~RGGOFT03EZMQ.1 size=1048576 2024-04-22T19:58:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.472+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~RGGOFT03EZMQ.2 size=786637 2024-04-22T19:58:46.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.472+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=RGGOFT03EZMQ tag=11YDZ2NQNDTE type=write size=3932365 2024-04-22T19:58:46.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.473+0000 7f5269a8e780 -1 < failed to complete operation key=T2BV2LUCY4E3 tag=715GD1B6Z7AF type=write size=900790: (22) Invalid argument 2024-04-22T19:58:46.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.474+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=1ZP85QLORYCL type=write size=931412 2024-04-22T19:58:46.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.475+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=4NE2P9YWJV19 type=del 2024-04-22T19:58:46.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.476+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=FEGBMW4HPOMX type=del size=1299901 2024-04-22T19:58:46.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.477+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=66BBJI7A65G4 type=del 2024-04-22T19:58:46.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.479+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~JI0DX16SOGRC.0 size=1048576 2024-04-22T19:58:46.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.480+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~JI0DX16SOGRC.1 size=1048576 2024-04-22T19:58:46.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.481+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~JI0DX16SOGRC.2 size=40044 2024-04-22T19:58:46.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.481+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=JI0DX16SOGRC tag=QEBV33BMNSRV type=write size=3185772 2024-04-22T19:58:46.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.482+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=P3TJB0QMGH5P type=del 2024-04-22T19:58:46.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.483+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=ERNR4BTRAOV9 type=write size=520613 2024-04-22T19:58:46.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.484+0000 7f5269a8e780 1 < canceled part key=_multipart_70MLQGJPQYPY.2~HMGJH2DD16SO.0 size=338932 2024-04-22T19:58:46.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.486+0000 7f5269a8e780 1 < canceled multipart upload key=70MLQGJPQYPY upload=HMGJH2DD16SO tag=274UQRT1JXS9 type=write size=1387508 2024-04-22T19:58:46.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.487+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=TYB8YG48EX5S type=del size=1659416 2024-04-22T19:58:46.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.488+0000 7f5269a8e780 1 < completed write operation key=4EA3LUQRL8II tag=2B1STING84QZ type=write size=115745 2024-04-22T19:58:46.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.490+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~1ZJIZRDU4JLA.0 size=1048576 2024-04-22T19:58:46.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.491+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~1ZJIZRDU4JLA.1 size=1048576 2024-04-22T19:58:46.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.492+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~1ZJIZRDU4JLA.2 size=828084 2024-04-22T19:58:46.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.492+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=1ZJIZRDU4JLA tag=KJJHRBQZ9HBC type=write size=3973812 2024-04-22T19:58:46.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.493+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~98LOK8LHVPOZ.0 size=1048576 2024-04-22T19:58:46.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.494+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~98LOK8LHVPOZ.1 size=1048576 2024-04-22T19:58:46.495 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.495+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~98LOK8LHVPOZ.2 size=624572 2024-04-22T19:58:46.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.497+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=98LOK8LHVPOZ tag=PJ08FTXB25I7 type=write size=3770300 2024-04-22T19:58:46.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.499+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~4TXXHK4NWZWE.0 size=1048576 2024-04-22T19:58:46.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.500+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~4TXXHK4NWZWE.1 size=1048576 2024-04-22T19:58:46.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.501+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~4TXXHK4NWZWE.2 size=558873 2024-04-22T19:58:46.501 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.501+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=4TXXHK4NWZWE tag=3694G0C0V7LO type=write size=3704601 2024-04-22T19:58:46.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.502+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=BWHA5NUR1UUC type=del 2024-04-22T19:58:46.504 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.504+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~K32LXPNPV2U9.0 size=1048576 2024-04-22T19:58:46.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.505+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~K32LXPNPV2U9.1 size=1048576 2024-04-22T19:58:46.506 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.506+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~K32LXPNPV2U9.2 size=741169 2024-04-22T19:58:46.506 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.506+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=K32LXPNPV2U9 tag=4XE35FU5AU9U type=write size=3886897 2024-04-22T19:58:46.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.507+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~2Y3VW5PRNY6G.0 size=782120 2024-04-22T19:58:46.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.509+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=2Y3VW5PRNY6G tag=IZD3WLR56BPR type=write size=1830696 2024-04-22T19:58:46.510 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.510+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=78OHAFE0ADCJ type=del size=75811 2024-04-22T19:58:46.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.511+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~ND8U24XNWSFE.0 size=1048576 2024-04-22T19:58:46.512 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.512+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~ND8U24XNWSFE.1 size=249117 2024-04-22T19:58:46.514 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.514+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=ND8U24XNWSFE tag=3SS4P268Y8ER type=write size=2346269 2024-04-22T19:58:46.515 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.515+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=BD4OT0SMCAPV type=del size=430418 2024-04-22T19:58:46.516 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.516+0000 7f5269a8e780 1 < completed write operation key=4GX83ZJWZCD1 tag=A1M3UBWWM17Z type=write size=25431 2024-04-22T19:58:46.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.518+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~CQJXIU6E51VP.0 size=1048576 2024-04-22T19:58:46.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.519+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~CQJXIU6E51VP.1 size=798369 2024-04-22T19:58:46.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.519+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=CQJXIU6E51VP tag=EEJLOT1RDDZZ type=write size=2895521 2024-04-22T19:58:46.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.520+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~0YTXF0TRKE5S.0 size=1048576 2024-04-22T19:58:46.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.520+0000 7f5269a8e780 1 < canceled part key=_multipart_DX5YUYQRZKWA.2~0YTXF0TRKE5S.1 size=894940 2024-04-22T19:58:46.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.522+0000 7f5269a8e780 1 < canceled multipart upload key=DX5YUYQRZKWA upload=0YTXF0TRKE5S tag=HFASSFFA6PDX type=write size=2992092 2024-04-22T19:58:46.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.524+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~1Q8WO20VWKCR.0 size=1048576 2024-04-22T19:58:46.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.525+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~1Q8WO20VWKCR.1 size=583771 2024-04-22T19:58:46.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.525+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=1Q8WO20VWKCR tag=ZNL2ZURGL19H type=write size=2680923 2024-04-22T19:58:46.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.527+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~S6758NBE2M0I.0 size=1048576 2024-04-22T19:58:46.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.529+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~S6758NBE2M0I.1 size=565174 2024-04-22T19:58:46.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.531+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=S6758NBE2M0I tag=T1QNUXHE486L type=write size=2662326 2024-04-22T19:58:46.533 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.533+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~2GK0AFRRSQGT.0 size=1048576 2024-04-22T19:58:46.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.534+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~2GK0AFRRSQGT.1 size=1048576 2024-04-22T19:58:46.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.535+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~2GK0AFRRSQGT.2 size=1000531 2024-04-22T19:58:46.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.535+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=2GK0AFRRSQGT tag=4CT3JWQ1MXFM type=write size=4146259 2024-04-22T19:58:46.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.535+0000 7f5269a8e780 1 < canceled part key=_multipart_DX5YUYQRZKWA.2~ZD58LXS75CD3.0 size=1048576 2024-04-22T19:58:46.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.536+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~ZD58LXS75CD3.1 size=240064 2024-04-22T19:58:46.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.538+0000 7f5269a8e780 1 < canceled multipart upload key=DX5YUYQRZKWA upload=ZD58LXS75CD3 tag=C4OM42A6SJH3 type=write size=2337216 2024-04-22T19:58:46.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.539+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=BT3YQPJZBHQS type=del size=3965917 2024-04-22T19:58:46.540 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.540+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=YCG7BZH3UD8Z type=del 2024-04-22T19:58:46.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.541+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=3CXF1ACYZH8C type=write size=367870 2024-04-22T19:58:46.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.543+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~RGGOFT03EZMQ.0 size=1048576 2024-04-22T19:58:46.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.544+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~RGGOFT03EZMQ.1 size=1048576 2024-04-22T19:58:46.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.544+0000 7f5269a8e780 1 < canceled part key=_multipart_A853MOZPT9BG.2~RGGOFT03EZMQ.2 size=786637 2024-04-22T19:58:46.546 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.546+0000 7f5269a8e780 1 < canceled multipart upload key=A853MOZPT9BG upload=RGGOFT03EZMQ tag=11YDZ2NQNDTE type=write size=3932365 2024-04-22T19:58:46.548 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.548+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~WIPUDA9HM0BB.0 size=697951 2024-04-22T19:58:46.548 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.548+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=WIPUDA9HM0BB tag=IZYKFVGY94JY type=write size=1746527 2024-04-22T19:58:46.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.549+0000 7f5269a8e780 1 < completed write operation key=ZC4ZOHUJ3HQD tag=1ZP85QLORYCL type=write size=931412 2024-04-22T19:58:46.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.551+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~YGP8XZ2QKGH2.0 size=1048576 2024-04-22T19:58:46.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.552+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~YGP8XZ2QKGH2.1 size=1048576 2024-04-22T19:58:46.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.553+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~YGP8XZ2QKGH2.2 size=173004 2024-04-22T19:58:46.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.553+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=YGP8XZ2QKGH2 tag=1QBEOEKQJYZ3 type=write size=3318732 2024-04-22T19:58:46.554 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.554+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=FEGBMW4HPOMX type=del 2024-04-22T19:58:46.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.555+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=TVN4B9JOR59F type=write size=685586 2024-04-22T19:58:46.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.556+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~JI0DX16SOGRC.0 size=1048576 2024-04-22T19:58:46.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.557+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~JI0DX16SOGRC.1 size=1048576 2024-04-22T19:58:46.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.558+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~JI0DX16SOGRC.2 size=40044 2024-04-22T19:58:46.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.560+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=JI0DX16SOGRC tag=QEBV33BMNSRV type=write size=3185772 2024-04-22T19:58:46.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.562+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=VURW1MNZZ6R9 type=del size=3965819 2024-04-22T19:58:46.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.563+0000 7f5269a8e780 1 < completed write operation key=RY1O7X1CWZMK tag=ERNR4BTRAOV9 type=write size=520613 2024-04-22T19:58:46.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.565+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~0OH1ILON2220.0 size=1048576 2024-04-22T19:58:46.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.566+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~0OH1ILON2220.1 size=1048576 2024-04-22T19:58:46.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.567+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~0OH1ILON2220.2 size=997529 2024-04-22T19:58:46.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.567+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=0OH1ILON2220 tag=PQNJR4A3AAIA type=write size=4143257 2024-04-22T19:58:46.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.568+0000 7f5269a8e780 1 < canceled operation key=B0Y7ZTPC0975 tag=TYB8YG48EX5S type=del size=1659416 2024-04-22T19:58:46.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.569+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=OZBE140RL3ER type=del size=3030487 2024-04-22T19:58:46.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.570+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~1ZJIZRDU4JLA.0 size=1048576 2024-04-22T19:58:46.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.571+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~1ZJIZRDU4JLA.1 size=1048576 2024-04-22T19:58:46.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.572+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~1ZJIZRDU4JLA.2 size=828084 2024-04-22T19:58:46.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.575+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=1ZJIZRDU4JLA tag=KJJHRBQZ9HBC type=write size=3973812 2024-04-22T19:58:46.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.576+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=8N4FTSW2XVP8 type=del size=3414719 2024-04-22T19:58:46.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.577+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~4TXXHK4NWZWE.0 size=1048576 2024-04-22T19:58:46.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.578+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~4TXXHK4NWZWE.1 size=1048576 2024-04-22T19:58:46.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.579+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~4TXXHK4NWZWE.2 size=558873 2024-04-22T19:58:46.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.581+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=4TXXHK4NWZWE tag=3694G0C0V7LO type=write size=3704601 2024-04-22T19:58:46.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.583+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~ZA2COAGCW2OI.0 size=1048576 2024-04-22T19:58:46.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.584+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~ZA2COAGCW2OI.1 size=258323 2024-04-22T19:58:46.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.584+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=ZA2COAGCW2OI tag=EA5SCBFVZNGM type=write size=2355475 2024-04-22T19:58:46.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.585+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~K32LXPNPV2U9.0 size=1048576 2024-04-22T19:58:46.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.586+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~K32LXPNPV2U9.1 size=1048576 2024-04-22T19:58:46.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.586+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~K32LXPNPV2U9.2 size=741169 2024-04-22T19:58:46.588 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.588+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=K32LXPNPV2U9 tag=4XE35FU5AU9U type=write size=3886897 2024-04-22T19:58:46.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.591+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~EXP4GOZ1VVX0.0 size=383296 2024-04-22T19:58:46.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.591+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=EXP4GOZ1VVX0 tag=FVJGAV2NYWKX type=write size=1431872 2024-04-22T19:58:46.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.592+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=78OHAFE0ADCJ type=del 2024-04-22T19:58:46.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.594+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~QS9FJJQ99AEC.0 size=1048576 2024-04-22T19:58:46.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.595+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~QS9FJJQ99AEC.1 size=1048576 2024-04-22T19:58:46.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.596+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~QS9FJJQ99AEC.2 size=514323 2024-04-22T19:58:46.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.596+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=QS9FJJQ99AEC tag=FBDRZGKSF39W type=write size=3660051 2024-04-22T19:58:46.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.597+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=BD4OT0SMCAPV type=del 2024-04-22T19:58:46.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.599+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~686Y8KOY2U86.0 size=1048576 2024-04-22T19:58:46.600 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.600+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~686Y8KOY2U86.1 size=1048576 2024-04-22T19:58:46.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.601+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~686Y8KOY2U86.2 size=627648 2024-04-22T19:58:46.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.601+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=686Y8KOY2U86 tag=RAV8O7Q20WUW type=write size=3773376 2024-04-22T19:58:46.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.602+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~CQJXIU6E51VP.0 size=1048576 2024-04-22T19:58:46.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.603+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~CQJXIU6E51VP.1 size=798369 2024-04-22T19:58:46.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.605+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=CQJXIU6E51VP tag=EEJLOT1RDDZZ type=write size=2895521 2024-04-22T19:58:46.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.606+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=WBDU86R8ACXI type=write size=911875 2024-04-22T19:58:46.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.607+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~1Q8WO20VWKCR.0 size=1048576 2024-04-22T19:58:46.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.608+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~1Q8WO20VWKCR.1 size=583771 2024-04-22T19:58:46.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.610+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=1Q8WO20VWKCR tag=ZNL2ZURGL19H type=write size=2680923 2024-04-22T19:58:46.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.612+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~13T90UWB2OYT.0 size=1048576 2024-04-22T19:58:46.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.613+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~13T90UWB2OYT.1 size=1048576 2024-04-22T19:58:46.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.614+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~13T90UWB2OYT.2 size=56555 2024-04-22T19:58:46.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.614+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=13T90UWB2OYT tag=N2CJJ0X1AE2K type=write size=3202283 2024-04-22T19:58:46.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.615+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~2GK0AFRRSQGT.0 size=1048576 2024-04-22T19:58:46.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.615+0000 7f5269a8e780 1 < canceled part key=_multipart_8V6ESIGMRO5G.2~2GK0AFRRSQGT.1 size=1048576 2024-04-22T19:58:46.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.616+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~2GK0AFRRSQGT.2 size=1000531 2024-04-22T19:58:46.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.618+0000 7f5269a8e780 1 < canceled multipart upload key=8V6ESIGMRO5G upload=2GK0AFRRSQGT tag=4CT3JWQ1MXFM type=write size=4146259 2024-04-22T19:58:46.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.619+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=F3CZVZG12CCY type=del size=3559586 2024-04-22T19:58:46.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.621+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=BT3YQPJZBHQS type=del 2024-04-22T19:58:46.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.622+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=MNVO1K4AP8CF type=write size=238924 2024-04-22T19:58:46.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.623+0000 7f5269a8e780 1 < completed write operation key=GX6NILQG878S tag=3CXF1ACYZH8C type=write size=367870 2024-04-22T19:58:46.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.624+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=7XJMI96OSF33 type=del size=3306000 2024-04-22T19:58:46.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.625+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~WIPUDA9HM0BB.0 size=697951 2024-04-22T19:58:46.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.627+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=WIPUDA9HM0BB tag=IZYKFVGY94JY type=write size=1746527 2024-04-22T19:58:46.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.629+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~W3C0QWM5T8U7.0 size=1048576 2024-04-22T19:58:46.630 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.630+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~W3C0QWM5T8U7.1 size=1048576 2024-04-22T19:58:46.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.631+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~W3C0QWM5T8U7.2 size=483113 2024-04-22T19:58:46.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.631+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=W3C0QWM5T8U7 tag=KF0AF7JRB5NO type=write size=3628841 2024-04-22T19:58:46.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.632+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~YGP8XZ2QKGH2.0 size=1048576 2024-04-22T19:58:46.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.634+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~YGP8XZ2QKGH2.1 size=1048576 2024-04-22T19:58:46.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.635+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~YGP8XZ2QKGH2.2 size=173004 2024-04-22T19:58:46.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.637+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=YGP8XZ2QKGH2 tag=1QBEOEKQJYZ3 type=write size=3318732 2024-04-22T19:58:46.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.639+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~SJC6H3Z6N49F.0 size=1048576 2024-04-22T19:58:46.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.640+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~SJC6H3Z6N49F.1 size=1048576 2024-04-22T19:58:46.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.641+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~SJC6H3Z6N49F.2 size=625522 2024-04-22T19:58:46.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.641+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=SJC6H3Z6N49F tag=VWE0BT0B0WDI type=write size=3771250 2024-04-22T19:58:46.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.643+0000 7f5269a8e780 1 < suggested remove operation key=EHRS6DQDRXVS tag=TVN4B9JOR59F type=write 2024-04-22T19:58:46.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.644+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=7H09NN07TORK type=del size=3653582 2024-04-22T19:58:46.645 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.645+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=VURW1MNZZ6R9 type=del 2024-04-22T19:58:46.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.647+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~HJUCKZB8DP5N.0 size=335402 2024-04-22T19:58:46.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.647+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=HJUCKZB8DP5N tag=GYTL7PEJ5ZV6 type=write size=1383978 2024-04-22T19:58:46.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.648+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~0OH1ILON2220.0 size=1048576 2024-04-22T19:58:46.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.649+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~0OH1ILON2220.1 size=1048576 2024-04-22T19:58:46.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.650+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~0OH1ILON2220.2 size=997529 2024-04-22T19:58:46.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.652+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=0OH1ILON2220 tag=PQNJR4A3AAIA type=write size=4143257 2024-04-22T19:58:46.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.654+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~9X4ADDTLUVXR.0 size=1048576 2024-04-22T19:58:46.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.655+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~9X4ADDTLUVXR.1 size=1048576 2024-04-22T19:58:46.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.655+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~9X4ADDTLUVXR.2 size=671328 2024-04-22T19:58:46.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.655+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=9X4ADDTLUVXR tag=PXMO83Q7XFE2 type=write size=3817056 2024-04-22T19:58:46.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.657+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=OZBE140RL3ER type=del 2024-04-22T19:58:46.658 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.659+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~6I1L0ZJF1C95.0 size=1048576 2024-04-22T19:58:46.659 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.659+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~6I1L0ZJF1C95.1 size=1048576 2024-04-22T19:58:46.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.660+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~6I1L0ZJF1C95.2 size=340564 2024-04-22T19:58:46.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.660+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=6I1L0ZJF1C95 tag=5ZXHUKAT9D5C type=write size=3486292 2024-04-22T19:58:46.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.662+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=8N4FTSW2XVP8 type=del 2024-04-22T19:58:46.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.663+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=06IIJF3QH74T type=del size=1913710 2024-04-22T19:58:46.664 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.664+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~ZA2COAGCW2OI.0 size=1048576 2024-04-22T19:58:46.665 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.665+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~ZA2COAGCW2OI.1 size=258323 2024-04-22T19:58:46.667 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.667+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=ZA2COAGCW2OI tag=EA5SCBFVZNGM type=write size=2355475 2024-04-22T19:58:46.668 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.668+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=208BFALWHD49 type=del size=2338674 2024-04-22T19:58:46.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.669+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~EXP4GOZ1VVX0.0 size=383296 2024-04-22T19:58:46.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.671+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=EXP4GOZ1VVX0 tag=FVJGAV2NYWKX type=write size=1431872 2024-04-22T19:58:46.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.672+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=TB9J6LDB19DU type=del size=3389360 2024-04-22T19:58:46.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.673+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~QS9FJJQ99AEC.0 size=1048576 2024-04-22T19:58:46.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.674+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~QS9FJJQ99AEC.1 size=1048576 2024-04-22T19:58:46.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.675+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~QS9FJJQ99AEC.2 size=514323 2024-04-22T19:58:46.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.677+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=QS9FJJQ99AEC tag=FBDRZGKSF39W type=write size=3660051 2024-04-22T19:58:46.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.678+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=M5YLB0GABF52 type=del size=894581 2024-04-22T19:58:46.679 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.679+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~686Y8KOY2U86.0 size=1048576 2024-04-22T19:58:46.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.680+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~686Y8KOY2U86.1 size=1048576 2024-04-22T19:58:46.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.681+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~686Y8KOY2U86.2 size=627648 2024-04-22T19:58:46.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.683+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=686Y8KOY2U86 tag=RAV8O7Q20WUW type=write size=3773376 2024-04-22T19:58:46.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.685+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~AG50UBPXEGQU.0 size=1048576 2024-04-22T19:58:46.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.686+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~AG50UBPXEGQU.1 size=1048576 2024-04-22T19:58:46.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.687+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~AG50UBPXEGQU.2 size=1022812 2024-04-22T19:58:46.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.687+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=AG50UBPXEGQU tag=ITUBEZRAWPER type=write size=4168540 2024-04-22T19:58:46.689 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.689+0000 7f5269a8e780 1 < suggested update operation key=Y5HR8XW4L0SP tag=WBDU86R8ACXI type=write size=911875 2024-04-22T19:58:46.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.691+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~XQMHG4S32G16.0 size=1048576 2024-04-22T19:58:46.692 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.692+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~XQMHG4S32G16.1 size=126831 2024-04-22T19:58:46.692 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.692+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=XQMHG4S32G16 tag=V3Q1HERNTF5I type=write size=2223983 2024-04-22T19:58:46.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.693+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~13T90UWB2OYT.0 size=1048576 2024-04-22T19:58:46.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.694+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~13T90UWB2OYT.1 size=1048576 2024-04-22T19:58:46.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.695+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~13T90UWB2OYT.2 size=56555 2024-04-22T19:58:46.697 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.698+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=13T90UWB2OYT tag=N2CJJ0X1AE2K type=write size=3202283 2024-04-22T19:58:46.699 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.699+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=1KERH422LSKO type=del size=840418 2024-04-22T19:58:46.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.700+0000 7f5269a8e780 1 < canceled operation key=B0Y7ZTPC0975 tag=F3CZVZG12CCY type=del size=3559586 2024-04-22T19:58:46.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.702+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~J5V2PUX5KBT6.0 size=1048576 2024-04-22T19:58:46.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.703+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~J5V2PUX5KBT6.1 size=592752 2024-04-22T19:58:46.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.703+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=J5V2PUX5KBT6 tag=HI3USZCGQMDH type=write size=2689904 2024-04-22T19:58:46.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.704+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=MNVO1K4AP8CF type=write size=238924 2024-04-22T19:58:46.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.705+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=61Q4DU8JDHMK type=write size=624722 2024-04-22T19:58:46.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.706+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=7XJMI96OSF33 type=del 2024-04-22T19:58:46.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.709+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~3ZGOYOWSWBZC.0 size=1033443 2024-04-22T19:58:46.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.709+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=3ZGOYOWSWBZC tag=IDNK4GV1MCRY type=write size=2082019 2024-04-22T19:58:46.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.710+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~W3C0QWM5T8U7.0 size=1048576 2024-04-22T19:58:46.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.711+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~W3C0QWM5T8U7.1 size=1048576 2024-04-22T19:58:46.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.712+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~W3C0QWM5T8U7.2 size=483113 2024-04-22T19:58:46.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.714+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=W3C0QWM5T8U7 tag=KF0AF7JRB5NO type=write size=3628841 2024-04-22T19:58:46.715 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.715+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=KIXTZIK8IQNF type=write size=176748 2024-04-22T19:58:46.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.716+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~SJC6H3Z6N49F.0 size=1048576 2024-04-22T19:58:46.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.717+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~SJC6H3Z6N49F.1 size=1048576 2024-04-22T19:58:46.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.718+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~SJC6H3Z6N49F.2 size=625522 2024-04-22T19:58:46.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.720+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=SJC6H3Z6N49F tag=VWE0BT0B0WDI type=write size=3771250 2024-04-22T19:58:46.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.722+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~FB7Y4HMX539D.0 size=1048576 2024-04-22T19:58:46.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.723+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~FB7Y4HMX539D.1 size=1048576 2024-04-22T19:58:46.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.724+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~FB7Y4HMX539D.2 size=861773 2024-04-22T19:58:46.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.724+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=FB7Y4HMX539D tag=M3WEONUT30XK type=write size=4007501 2024-04-22T19:58:46.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.725+0000 7f5269a8e780 1 < canceled operation key=4GX83ZJWZCD1 tag=7H09NN07TORK type=del size=3653582 2024-04-22T19:58:46.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.726+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=9B10PX43KJCE type=del size=2865365 2024-04-22T19:58:46.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.727+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~HJUCKZB8DP5N.0 size=335402 2024-04-22T19:58:46.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.729+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=HJUCKZB8DP5N tag=GYTL7PEJ5ZV6 type=write size=1383978 2024-04-22T19:58:46.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.731+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~IZ3U74U0CG45.0 size=1048576 2024-04-22T19:58:46.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.732+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~IZ3U74U0CG45.1 size=1048576 2024-04-22T19:58:46.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.733+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~IZ3U74U0CG45.2 size=212543 2024-04-22T19:58:46.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.733+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=IZ3U74U0CG45 tag=B8YI377UDYGS type=write size=3358271 2024-04-22T19:58:46.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.734+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~9X4ADDTLUVXR.0 size=1048576 2024-04-22T19:58:46.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.735+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~9X4ADDTLUVXR.1 size=1048576 2024-04-22T19:58:46.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.737+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~9X4ADDTLUVXR.2 size=671328 2024-04-22T19:58:46.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.739+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=9X4ADDTLUVXR tag=PXMO83Q7XFE2 type=write size=3817056 2024-04-22T19:58:46.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.740+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=36RC8A8LDDQY type=del size=1348239 2024-04-22T19:58:46.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.741+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~6I1L0ZJF1C95.0 size=1048576 2024-04-22T19:58:46.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.742+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~6I1L0ZJF1C95.1 size=1048576 2024-04-22T19:58:46.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.743+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~6I1L0ZJF1C95.2 size=340564 2024-04-22T19:58:46.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.745+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=6I1L0ZJF1C95 tag=5ZXHUKAT9D5C type=write size=3486292 2024-04-22T19:58:46.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.746+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=3PLDOHOG7EIR type=write size=657408 2024-04-22T19:58:46.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.748+0000 7f5269a8e780 1 < suggested update operation key=DX5YUYQRZKWA tag=06IIJF3QH74T type=del size=1913710 2024-04-22T19:58:46.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.750+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=Q93X8UAGHNO3 type=del size=1548869 2024-04-22T19:58:46.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.751+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=208BFALWHD49 type=del 2024-04-22T19:58:46.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.752+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=HB33FB1KB9FP type=del size=1463464 2024-04-22T19:58:46.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.753+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=TB9J6LDB19DU type=del 2024-04-22T19:58:46.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.754+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=HM1IYYOLFH20 type=del size=57672 2024-04-22T19:58:46.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.755+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=M5YLB0GABF52 type=del 2024-04-22T19:58:46.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.757+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=T4APFRRX9YE8 type=del size=3401318 2024-04-22T19:58:46.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.758+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~AG50UBPXEGQU.0 size=1048576 2024-04-22T19:58:46.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.759+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~AG50UBPXEGQU.1 size=1048576 2024-04-22T19:58:46.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.760+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~AG50UBPXEGQU.2 size=1022812 2024-04-22T19:58:46.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.762+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=AG50UBPXEGQU tag=ITUBEZRAWPER type=write size=4168540 2024-04-22T19:58:46.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.763+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=AQV7VVIVZHQO type=del size=181683 2024-04-22T19:58:46.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.764+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~XQMHG4S32G16.0 size=1048576 2024-04-22T19:58:46.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.765+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~XQMHG4S32G16.1 size=126831 2024-04-22T19:58:46.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.767+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=XQMHG4S32G16 tag=V3Q1HERNTF5I type=write size=2223983 2024-04-22T19:58:46.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.768+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=56HMMI71WBTR type=write size=1023827 2024-04-22T19:58:46.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.769+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=1KERH422LSKO type=del 2024-04-22T19:58:46.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.770+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=ZRGKXD6FVQ45 type=del size=2478841 2024-04-22T19:58:46.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.772+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~J5V2PUX5KBT6.0 size=1048576 2024-04-22T19:58:46.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.772+0000 7f5269a8e780 1 < canceled part key=_multipart_ZC4ZOHUJ3HQD.2~J5V2PUX5KBT6.1 size=592752 2024-04-22T19:58:46.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.774+0000 7f5269a8e780 1 < canceled multipart upload key=ZC4ZOHUJ3HQD upload=J5V2PUX5KBT6 tag=HI3USZCGQMDH type=write size=2689904 2024-04-22T19:58:46.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.776+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~FOU6ZHDF2C2I.0 size=102046 2024-04-22T19:58:46.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.776+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=FOU6ZHDF2C2I tag=GQWE2LZARQ02 type=write size=1150622 2024-04-22T19:58:46.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.777+0000 7f5269a8e780 1 < completed write operation key=86GW6HV9G1IZ tag=61Q4DU8JDHMK type=write size=624722 2024-04-22T19:58:46.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.778+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=5ZOEGDPDZ0AZ type=del size=3971520 2024-04-22T19:58:46.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.779+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~3ZGOYOWSWBZC.0 size=1033443 2024-04-22T19:58:46.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.781+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=3ZGOYOWSWBZC tag=IDNK4GV1MCRY type=write size=2082019 2024-04-22T19:58:46.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.782+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=KFZZRIOG4QW6 type=del size=1145784 2024-04-22T19:58:46.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.783+0000 7f5269a8e780 1 < completed write operation key=B0Y7ZTPC0975 tag=KIXTZIK8IQNF type=write size=176748 2024-04-22T19:58:46.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.785+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~RANBEIB1TA7G.0 size=1048576 2024-04-22T19:58:46.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.787+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~RANBEIB1TA7G.1 size=1048576 2024-04-22T19:58:46.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.787+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~RANBEIB1TA7G.2 size=613353 2024-04-22T19:58:46.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.787+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=RANBEIB1TA7G tag=5VVQ8526V5HC type=write size=3759081 2024-04-22T19:58:46.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.789+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~FB7Y4HMX539D.0 size=1048576 2024-04-22T19:58:46.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.790+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~FB7Y4HMX539D.1 size=1048576 2024-04-22T19:58:46.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.791+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~FB7Y4HMX539D.2 size=861773 2024-04-22T19:58:46.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.793+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=FB7Y4HMX539D tag=M3WEONUT30XK type=write size=4007501 2024-04-22T19:58:46.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.795+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~YT8M67E6UMQO.0 size=504767 2024-04-22T19:58:46.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.795+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=YT8M67E6UMQO tag=IVR80QZHAS1L type=write size=1553343 2024-04-22T19:58:46.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.797+0000 7f5269a8e780 1 < suggested remove operation key=GRSL48RUEBX8 tag=9B10PX43KJCE type=del 2024-04-22T19:58:46.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.799+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~6RZLAB0SJBR7.0 size=1048576 2024-04-22T19:58:46.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.800+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~6RZLAB0SJBR7.1 size=656846 2024-04-22T19:58:46.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.800+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=6RZLAB0SJBR7 tag=S11OT512KYU5 type=write size=2753998 2024-04-22T19:58:46.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.801+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~IZ3U74U0CG45.0 size=1048576 2024-04-22T19:58:46.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.802+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~IZ3U74U0CG45.1 size=1048576 2024-04-22T19:58:46.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.803+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~IZ3U74U0CG45.2 size=212543 2024-04-22T19:58:46.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.805+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=IZ3U74U0CG45 tag=B8YI377UDYGS type=write size=3358271 2024-04-22T19:58:46.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.806+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=RC5310V0NDV0 type=del size=1434139 2024-04-22T19:58:46.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.807+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=36RC8A8LDDQY type=del 2024-04-22T19:58:46.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.808+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=Q5ENH497M47Q type=del size=2827462 2024-04-22T19:58:46.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.809+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=3PLDOHOG7EIR type=write size=657408 2024-04-22T19:58:46.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.811+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~PING7WCX31QD.0 size=1048576 2024-04-22T19:58:46.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.812+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~PING7WCX31QD.1 size=574737 2024-04-22T19:58:46.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.812+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=PING7WCX31QD tag=M76ON2SRW73A type=write size=2671889 2024-04-22T19:58:46.813 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.814+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=Q93X8UAGHNO3 type=del 2024-04-22T19:58:46.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.816+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~8NJITWO4SAKK.0 size=1048576 2024-04-22T19:58:46.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.817+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~8NJITWO4SAKK.1 size=734881 2024-04-22T19:58:46.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.817+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=8NJITWO4SAKK tag=UMF1MJH2U69V type=write size=2832033 2024-04-22T19:58:46.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.818+0000 7f5269a8e780 1 < canceled operation key=70MLQGJPQYPY tag=HB33FB1KB9FP type=del size=1463464 2024-04-22T19:58:46.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.819+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=3UFWKHL71KQ6 type=del size=3151264 2024-04-22T19:58:46.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.821+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=HM1IYYOLFH20 type=del 2024-04-22T19:58:46.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.822+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=3KYZMMYL7YS8 type=del size=2400032 2024-04-22T19:58:46.823 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.823+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=T4APFRRX9YE8 type=del 2024-04-22T19:58:46.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.824+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=XLO5DLTMFN7N type=del size=3088201 2024-04-22T19:58:46.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.825+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=AQV7VVIVZHQO type=del 2024-04-22T19:58:46.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.827+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=QNE2097AE70R type=write size=894795 2024-04-22T19:58:46.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.828+0000 7f5269a8e780 1 < completed write operation key=IBH6W1NINUHC tag=56HMMI71WBTR type=write size=1023827 2024-04-22T19:58:46.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.830+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~7X6PVI2Q431A.0 size=1048576 2024-04-22T19:58:46.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.831+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~7X6PVI2Q431A.1 size=1048576 2024-04-22T19:58:46.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.832+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~7X6PVI2Q431A.2 size=758632 2024-04-22T19:58:46.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.832+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=7X6PVI2Q431A tag=SYQQ9G3VLLOX type=write size=3904360 2024-04-22T19:58:46.833 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.833+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=ZRGKXD6FVQ45 type=del 2024-04-22T19:58:46.834 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.834+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=J7QW87053K5R type=del size=965579 2024-04-22T19:58:46.835 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.835+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~FOU6ZHDF2C2I.0 size=102046 2024-04-22T19:58:46.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.837+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=FOU6ZHDF2C2I tag=GQWE2LZARQ02 type=write size=1150622 2024-04-22T19:58:46.839 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.839+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~I7GTVEH40CHH.0 size=597713 2024-04-22T19:58:46.839 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.839+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=I7GTVEH40CHH tag=1463RPE3LFFS type=write size=1646289 2024-04-22T19:58:46.840 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.840+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=5ZOEGDPDZ0AZ type=del 2024-04-22T19:58:46.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.842+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~H9D0KN5QMBRF.0 size=1048576 2024-04-22T19:58:46.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.843+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~H9D0KN5QMBRF.1 size=1048576 2024-04-22T19:58:46.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.845+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~H9D0KN5QMBRF.2 size=786707 2024-04-22T19:58:46.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.845+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=H9D0KN5QMBRF tag=AND8LNX3T7WJ type=write size=3932435 2024-04-22T19:58:46.846 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.846+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=KFZZRIOG4QW6 type=del 2024-04-22T19:58:46.847 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.848+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=17606JODZQVX type=del size=1613829 2024-04-22T19:58:46.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.849+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~RANBEIB1TA7G.0 size=1048576 2024-04-22T19:58:46.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.850+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~RANBEIB1TA7G.1 size=1048576 2024-04-22T19:58:46.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.850+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~RANBEIB1TA7G.2 size=613353 2024-04-22T19:58:46.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.852+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=RANBEIB1TA7G tag=5VVQ8526V5HC type=write size=3759081 2024-04-22T19:58:46.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.854+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~7OJH3J2YPEHJ.0 size=248342 2024-04-22T19:58:46.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.854+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=7OJH3J2YPEHJ tag=ABQLQBQRNU72 type=write size=1296918 2024-04-22T19:58:46.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.856+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~YT8M67E6UMQO.0 size=504767 2024-04-22T19:58:46.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.857+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=YT8M67E6UMQO tag=IVR80QZHAS1L type=write size=1553343 2024-04-22T19:58:46.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.859+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~XKVGG63JXKM7.0 size=568476 2024-04-22T19:58:46.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.859+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=XKVGG63JXKM7 tag=R5QTMY1334DW type=write size=1617052 2024-04-22T19:58:46.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.860+0000 7f5269a8e780 1 < canceled part key=_multipart_GX6NILQG878S.2~6RZLAB0SJBR7.0 size=1048576 2024-04-22T19:58:46.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.860+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~6RZLAB0SJBR7.1 size=656846 2024-04-22T19:58:46.863 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.862+0000 7f5269a8e780 1 < canceled multipart upload key=GX6NILQG878S upload=6RZLAB0SJBR7 tag=S11OT512KYU5 type=write size=2753998 2024-04-22T19:58:46.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.864+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=CMIFIBZKL8C8 type=del size=1969898 2024-04-22T19:58:46.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.866+0000 7f5269a8e780 1 < suggested update operation key=DX5YUYQRZKWA tag=RC5310V0NDV0 type=del size=1434139 2024-04-22T19:58:46.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.868+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~AEIP4OCY872R.0 size=1048576 2024-04-22T19:58:46.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.869+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~AEIP4OCY872R.1 size=1048576 2024-04-22T19:58:46.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.870+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~AEIP4OCY872R.2 size=261285 2024-04-22T19:58:46.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.870+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=AEIP4OCY872R tag=BSGR530QGVBI type=write size=3407013 2024-04-22T19:58:46.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.871+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=Q5ENH497M47Q type=del 2024-04-22T19:58:46.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.873+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~NGIV06UFGJUA.0 size=1048576 2024-04-22T19:58:46.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.874+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~NGIV06UFGJUA.1 size=765210 2024-04-22T19:58:46.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.874+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=NGIV06UFGJUA tag=KXHFK5HMJO6X type=write size=2862362 2024-04-22T19:58:46.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.876+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~PING7WCX31QD.0 size=1048576 2024-04-22T19:58:46.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.876+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~PING7WCX31QD.1 size=574737 2024-04-22T19:58:46.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.878+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=PING7WCX31QD tag=M76ON2SRW73A type=write size=2671889 2024-04-22T19:58:46.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.881+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~N4V0L4MZHFTC.0 size=1048576 2024-04-22T19:58:46.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.882+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~N4V0L4MZHFTC.1 size=776126 2024-04-22T19:58:46.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.882+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=N4V0L4MZHFTC tag=1UUT7YV5QD99 type=write size=2873278 2024-04-22T19:58:46.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.882+0000 7f5269a8e780 1 < canceled part key=_multipart_KE7YGPTF1MOR.2~8NJITWO4SAKK.0 size=1048576 2024-04-22T19:58:46.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.883+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~8NJITWO4SAKK.1 size=734881 2024-04-22T19:58:46.885 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.885+0000 7f5269a8e780 1 < canceled multipart upload key=KE7YGPTF1MOR upload=8NJITWO4SAKK tag=UMF1MJH2U69V type=write size=2832033 2024-04-22T19:58:46.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.887+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~99HF23POE8AN.0 size=232426 2024-04-22T19:58:46.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.887+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=99HF23POE8AN tag=J6M0KP2EIRA0 type=write size=1281002 2024-04-22T19:58:46.888 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.888+0000 7f5269a8e780 1 < canceled operation key=637M7AP8X9E9 tag=3UFWKHL71KQ6 type=del size=3151264 2024-04-22T19:58:46.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.889+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=VXJ2AB0RQB4T type=del size=2896849 2024-04-22T19:58:46.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.892+0000 7f5269a8e780 1 < suggested remove operation key=GP1C5ULUU143 tag=3KYZMMYL7YS8 type=del 2024-04-22T19:58:46.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.893+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=GA8C5GOAB59Y type=del size=2032509 2024-04-22T19:58:46.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.894+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=XLO5DLTMFN7N type=del 2024-04-22T19:58:46.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.896+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~H2F91LAF8V1K.0 size=1048576 2024-04-22T19:58:46.897 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.898+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~H2F91LAF8V1K.1 size=1048576 2024-04-22T19:58:46.899 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.899+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~H2F91LAF8V1K.2 size=35237 2024-04-22T19:58:46.899 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.899+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=H2F91LAF8V1K tag=4IESIRKWBORE type=write size=3180965 2024-04-22T19:58:46.900 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.900+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=QNE2097AE70R type=write size=894795 2024-04-22T19:58:46.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.902+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~BJ7SRMAFHNJD.0 size=56770 2024-04-22T19:58:46.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.902+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=BJ7SRMAFHNJD tag=VLLCMS0ZF3NL type=write size=1105346 2024-04-22T19:58:46.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.903+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~7X6PVI2Q431A.0 size=1048576 2024-04-22T19:58:46.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.904+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~7X6PVI2Q431A.1 size=1048576 2024-04-22T19:58:46.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.904+0000 7f5269a8e780 1 < canceled part key=_multipart_Y5HR8XW4L0SP.2~7X6PVI2Q431A.2 size=758632 2024-04-22T19:58:46.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.906+0000 7f5269a8e780 1 < canceled multipart upload key=Y5HR8XW4L0SP upload=7X6PVI2Q431A tag=SYQQ9G3VLLOX type=write size=3904360 2024-04-22T19:58:46.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.908+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~3KPQ01QH5V2O.0 size=1048576 2024-04-22T19:58:46.909 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.909+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~3KPQ01QH5V2O.1 size=1048576 2024-04-22T19:58:46.910 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.910+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~3KPQ01QH5V2O.2 size=297783 2024-04-22T19:58:46.910 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.910+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=3KPQ01QH5V2O tag=GCB9VTBWQAQM type=write size=3443511 2024-04-22T19:58:46.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.911+0000 7f5269a8e780 -1 < failed to complete operation key=GP1C5ULUU143 tag=J7QW87053K5R type=del size=965579: (22) Invalid argument 2024-04-22T19:58:46.912 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.912+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=YXBKC4FTIFUT type=write size=516654 2024-04-22T19:58:46.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.913+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~I7GTVEH40CHH.0 size=597713 2024-04-22T19:58:46.915 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.915+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=I7GTVEH40CHH tag=1463RPE3LFFS type=write size=1646289 2024-04-22T19:58:46.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.917+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MA1TFUYLGL3C.0 size=1048576 2024-04-22T19:58:46.918 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.918+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MA1TFUYLGL3C.1 size=1048576 2024-04-22T19:58:46.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.919+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~MA1TFUYLGL3C.2 size=260700 2024-04-22T19:58:46.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.919+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=MA1TFUYLGL3C tag=GPCW0V22KFJL type=write size=3406428 2024-04-22T19:58:46.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.920+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~H9D0KN5QMBRF.0 size=1048576 2024-04-22T19:58:46.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.921+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~H9D0KN5QMBRF.1 size=1048576 2024-04-22T19:58:46.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.922+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~H9D0KN5QMBRF.2 size=786707 2024-04-22T19:58:46.924 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.924+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=H9D0KN5QMBRF tag=AND8LNX3T7WJ type=write size=3932435 2024-04-22T19:58:46.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.925+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=F0JNAJGVTZK1 type=del size=2899395 2024-04-22T19:58:46.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.926+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=17606JODZQVX type=del 2024-04-22T19:58:46.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.927+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=RZHR1OMKVM44 type=del size=1782366 2024-04-22T19:58:46.928 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.928+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~7OJH3J2YPEHJ.0 size=248342 2024-04-22T19:58:46.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.930+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=7OJH3J2YPEHJ tag=ABQLQBQRNU72 type=write size=1296918 2024-04-22T19:58:46.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.931+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=SUFTJBAZUR9T type=del size=2956773 2024-04-22T19:58:46.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.932+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~XKVGG63JXKM7.0 size=568476 2024-04-22T19:58:46.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.934+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=XKVGG63JXKM7 tag=R5QTMY1334DW type=write size=1617052 2024-04-22T19:58:46.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.935+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=YMDUYLGWP5JS type=del size=3886074 2024-04-22T19:58:46.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.936+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=CMIFIBZKL8C8 type=del 2024-04-22T19:58:46.937 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.937+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=SQQT3H0YN9HX type=del size=152659 2024-04-22T19:58:46.939 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.939+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~AEIP4OCY872R.0 size=1048576 2024-04-22T19:58:46.939 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.940+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~AEIP4OCY872R.1 size=1048576 2024-04-22T19:58:46.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.940+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~AEIP4OCY872R.2 size=261285 2024-04-22T19:58:46.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.942+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=AEIP4OCY872R tag=BSGR530QGVBI type=write size=3407013 2024-04-22T19:58:46.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.943+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=3P20WRQKY8YO type=del size=411079 2024-04-22T19:58:46.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.944+0000 7f5269a8e780 1 < canceled part key=_multipart_70MLQGJPQYPY.2~NGIV06UFGJUA.0 size=1048576 2024-04-22T19:58:46.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.944+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~NGIV06UFGJUA.1 size=765210 2024-04-22T19:58:46.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.946+0000 7f5269a8e780 1 < canceled multipart upload key=70MLQGJPQYPY upload=NGIV06UFGJUA tag=KXHFK5HMJO6X type=write size=2862362 2024-04-22T19:58:46.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.948+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~L3ZY8T1WJ5QG.0 size=1013202 2024-04-22T19:58:46.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.948+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=L3ZY8T1WJ5QG tag=3BEDN28340Y1 type=write size=2061778 2024-04-22T19:58:46.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.950+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~N4V0L4MZHFTC.0 size=1048576 2024-04-22T19:58:46.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.951+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~N4V0L4MZHFTC.1 size=776126 2024-04-22T19:58:46.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.954+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=N4V0L4MZHFTC tag=1UUT7YV5QD99 type=write size=2873278 2024-04-22T19:58:46.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.956+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~09AEQCD8R9C4.0 size=1048576 2024-04-22T19:58:46.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.957+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~09AEQCD8R9C4.1 size=1048576 2024-04-22T19:58:46.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.958+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~09AEQCD8R9C4.2 size=172415 2024-04-22T19:58:46.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.958+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=09AEQCD8R9C4 tag=L2HMBD82TLYP type=write size=3318143 2024-04-22T19:58:46.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.959+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~99HF23POE8AN.0 size=232426 2024-04-22T19:58:46.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.961+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=99HF23POE8AN tag=J6M0KP2EIRA0 type=write size=1281002 2024-04-22T19:58:46.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.963+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~H59YYZIDKKB2.0 size=58134 2024-04-22T19:58:46.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.963+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=H59YYZIDKKB2 tag=X9LIXDK6MVI3 type=write size=1106710 2024-04-22T19:58:46.964 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.964+0000 7f5269a8e780 1 < canceled operation key=Y5HR8XW4L0SP tag=VXJ2AB0RQB4T type=del size=2896849 2024-04-22T19:58:46.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.965+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=28DA67CY02O5 type=write size=268865 2024-04-22T19:58:46.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.967+0000 7f5269a8e780 1 < suggested remove operation key=SN2BZATJGUWA tag=GA8C5GOAB59Y type=del 2024-04-22T19:58:46.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.968+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=3ZECS4NJCAMY type=del size=4104598 2024-04-22T19:58:46.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.970+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~H2F91LAF8V1K.0 size=1048576 2024-04-22T19:58:46.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.970+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~H2F91LAF8V1K.1 size=1048576 2024-04-22T19:58:46.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.971+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~H2F91LAF8V1K.2 size=35237 2024-04-22T19:58:46.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.973+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=H2F91LAF8V1K tag=4IESIRKWBORE type=write size=3180965 2024-04-22T19:58:46.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.975+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~7RZPDEF15KE7.0 size=1048576 2024-04-22T19:58:46.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.976+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~7RZPDEF15KE7.1 size=159745 2024-04-22T19:58:46.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.976+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=7RZPDEF15KE7 tag=Z1YBUYL5IESR type=write size=2256897 2024-04-22T19:58:46.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.977+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~BJ7SRMAFHNJD.0 size=56770 2024-04-22T19:58:46.979 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.980+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=BJ7SRMAFHNJD tag=VLLCMS0ZF3NL type=write size=1105346 2024-04-22T19:58:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.981+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=73B928PP4H6Y type=write size=359820 2024-04-22T19:58:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.981+0000 7f5269a8e780 1 < canceled part key=_multipart_GRSL48RUEBX8.2~3KPQ01QH5V2O.0 size=1048576 2024-04-22T19:58:46.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.982+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~3KPQ01QH5V2O.1 size=1048576 2024-04-22T19:58:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.983+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~3KPQ01QH5V2O.2 size=297783 2024-04-22T19:58:46.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.985+0000 7f5269a8e780 1 < canceled multipart upload key=GRSL48RUEBX8 upload=3KPQ01QH5V2O tag=GCB9VTBWQAQM type=write size=3443511 2024-04-22T19:58:46.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.986+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=P7POVWODYPC3 type=del size=3069969 2024-04-22T19:58:46.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.988+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=YXBKC4FTIFUT type=write size=516654 2024-04-22T19:58:46.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.990+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~01MU0RX66OCP.0 size=1048576 2024-04-22T19:58:46.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.991+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~01MU0RX66OCP.1 size=283050 2024-04-22T19:58:46.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.991+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=01MU0RX66OCP tag=AJL1WN94OFRT type=write size=2380202 2024-04-22T19:58:46.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.991+0000 7f5269a8e780 1 < canceled part key=_multipart_Y5HR8XW4L0SP.2~MA1TFUYLGL3C.0 size=1048576 2024-04-22T19:58:46.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.992+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~MA1TFUYLGL3C.1 size=1048576 2024-04-22T19:58:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.993+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~MA1TFUYLGL3C.2 size=260700 2024-04-22T19:58:46.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.995+0000 7f5269a8e780 1 < canceled multipart upload key=Y5HR8XW4L0SP upload=MA1TFUYLGL3C tag=GPCW0V22KFJL type=write size=3406428 2024-04-22T19:58:46.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.996+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=9HJ5JA4Y4UXC type=del size=2578436 2024-04-22T19:58:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.998+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=F0JNAJGVTZK1 type=del 2024-04-22T19:58:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:46.999+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=J7C4ESV0Y7GV type=del size=1365407 2024-04-22T19:58:47.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.000+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=RZHR1OMKVM44 type=del 2024-04-22T19:58:47.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.002+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=0GOHGFAP3U6N type=del size=2793655 2024-04-22T19:58:47.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.003+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=SUFTJBAZUR9T type=del 2024-04-22T19:58:47.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.004+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=P01OVLVFBUKP type=del size=4158069 2024-04-22T19:58:47.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.006+0000 7f5269a8e780 1 < suggested remove operation key=M9M4SEEEZSFV tag=YMDUYLGWP5JS type=del 2024-04-22T19:58:47.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.008+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~W342YOM9Q9HQ.0 size=806090 2024-04-22T19:58:47.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.008+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=W342YOM9Q9HQ tag=E18JAU6FUWVO type=write size=1854666 2024-04-22T19:58:47.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.010+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=SQQT3H0YN9HX type=del 2024-04-22T19:58:47.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.011+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=ULHP0HV3CUB0 type=del size=3285800 2024-04-22T19:58:47.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.012+0000 7f5269a8e780 -1 < failed to complete operation key=M9M4SEEEZSFV tag=3P20WRQKY8YO type=del size=411079: (22) Invalid argument 2024-04-22T19:58:47.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.013+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=RSVMFPGIJS1X type=write size=558444 2024-04-22T19:58:47.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.015+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~L3ZY8T1WJ5QG.0 size=1013202 2024-04-22T19:58:47.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.017+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=L3ZY8T1WJ5QG tag=3BEDN28340Y1 type=write size=2061778 2024-04-22T19:58:47.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.018+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=1DIAEPP3TLSK type=del size=447225 2024-04-22T19:58:47.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.019+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~09AEQCD8R9C4.0 size=1048576 2024-04-22T19:58:47.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.020+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~09AEQCD8R9C4.1 size=1048576 2024-04-22T19:58:47.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.021+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~09AEQCD8R9C4.2 size=172415 2024-04-22T19:58:47.023 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.023+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=09AEQCD8R9C4 tag=L2HMBD82TLYP type=write size=3318143 2024-04-22T19:58:47.025 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.025+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~57GO6U1AY8Y8.0 size=963795 2024-04-22T19:58:47.025 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.025+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=57GO6U1AY8Y8 tag=94M99VXU0AYU type=write size=2012371 2024-04-22T19:58:47.025 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.025+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~H59YYZIDKKB2.0 size=58134 2024-04-22T19:58:47.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.027+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=H59YYZIDKKB2 tag=X9LIXDK6MVI3 type=write size=1106710 2024-04-22T19:58:47.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.029+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~TFSMOUT4N87V.0 size=1048576 2024-04-22T19:58:47.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.030+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~TFSMOUT4N87V.1 size=1048576 2024-04-22T19:58:47.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.031+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~TFSMOUT4N87V.2 size=907945 2024-04-22T19:58:47.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.031+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=TFSMOUT4N87V tag=B4JG90KI63KZ type=write size=4053673 2024-04-22T19:58:47.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.032+0000 7f5269a8e780 1 < completed write operation key=70MLQGJPQYPY tag=28DA67CY02O5 type=write size=268865 2024-04-22T19:58:47.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.034+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~0ZPSTOIT40XF.0 size=1048576 2024-04-22T19:58:47.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.034+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~0ZPSTOIT40XF.1 size=1048576 2024-04-22T19:58:47.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.035+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~0ZPSTOIT40XF.2 size=796954 2024-04-22T19:58:47.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.035+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=0ZPSTOIT40XF tag=GKTHTUSJTXG7 type=write size=3942682 2024-04-22T19:58:47.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.037+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=3ZECS4NJCAMY type=del 2024-04-22T19:58:47.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.039+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~XO14429CI9DP.0 size=1048576 2024-04-22T19:58:47.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.039+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~XO14429CI9DP.1 size=127621 2024-04-22T19:58:47.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.039+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=XO14429CI9DP tag=ODGVJ09998E0 type=write size=2224773 2024-04-22T19:58:47.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.041+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~7RZPDEF15KE7.0 size=1048576 2024-04-22T19:58:47.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.041+0000 7f5269a8e780 1 < canceled part key=_multipart_2YFD2U6HVNE7.2~7RZPDEF15KE7.1 size=159745 2024-04-22T19:58:47.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.042+0000 7f5269a8e780 1 < canceled multipart upload key=2YFD2U6HVNE7 upload=7RZPDEF15KE7 tag=Z1YBUYL5IESR type=write size=2256897 2024-04-22T19:58:47.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.044+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~4Y7EWJPT9A2U.0 size=899098 2024-04-22T19:58:47.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.044+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=4Y7EWJPT9A2U tag=QGRIUZC3FJNA type=write size=1947674 2024-04-22T19:58:47.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.047+0000 7f5269a8e780 1 < suggested remove operation key=DX5YUYQRZKWA tag=73B928PP4H6Y type=write 2024-04-22T19:58:47.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.048+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=QL35L96OGOQ3 type=del size=387710 2024-04-22T19:58:47.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.049+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=P7POVWODYPC3 type=del 2024-04-22T19:58:47.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.050+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=LM0Z5E535330 type=del size=1043835 2024-04-22T19:58:47.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.051+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~01MU0RX66OCP.0 size=1048576 2024-04-22T19:58:47.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.052+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~01MU0RX66OCP.1 size=283050 2024-04-22T19:58:47.054 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.054+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=01MU0RX66OCP tag=AJL1WN94OFRT type=write size=2380202 2024-04-22T19:58:47.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.055+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=A0S5UR911TGN type=del size=3105429 2024-04-22T19:58:47.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.056+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=9HJ5JA4Y4UXC type=del 2024-04-22T19:58:47.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.060+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~0EUZLG62KYGK.0 size=1048576 2024-04-22T19:58:47.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.060+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~0EUZLG62KYGK.1 size=1048576 2024-04-22T19:58:47.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.061+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~0EUZLG62KYGK.2 size=751477 2024-04-22T19:58:47.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.061+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=0EUZLG62KYGK tag=BPQHLZKG52JB type=write size=3897205 2024-04-22T19:58:47.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.063+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=J7C4ESV0Y7GV type=del 2024-04-22T19:58:47.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.064+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=DHIGZPUZTPZD type=del size=1146534 2024-04-22T19:58:47.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.065+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=0GOHGFAP3U6N type=del 2024-04-22T19:58:47.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.067+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=987Y3YFO5C8P type=del size=3893033 2024-04-22T19:58:47.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.068+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=P01OVLVFBUKP type=del 2024-04-22T19:58:47.069 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.069+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=ZKLXUEKVQGYP type=del size=1120629 2024-04-22T19:58:47.070 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.070+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~W342YOM9Q9HQ.0 size=806090 2024-04-22T19:58:47.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.072+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=W342YOM9Q9HQ tag=E18JAU6FUWVO type=write size=1854666 2024-04-22T19:58:47.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.074+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~VAUA3YJQ4YGU.0 size=1048576 2024-04-22T19:58:47.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.075+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~VAUA3YJQ4YGU.1 size=842559 2024-04-22T19:58:47.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.075+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=VAUA3YJQ4YGU tag=00V6YQ3KP4IT type=write size=2939711 2024-04-22T19:58:47.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.076+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=ULHP0HV3CUB0 type=del 2024-04-22T19:58:47.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.077+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=WAG62M2PT1M3 type=write size=237788 2024-04-22T19:58:47.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.079+0000 7f5269a8e780 1 < completed write operation key=70MLQGJPQYPY tag=RSVMFPGIJS1X type=write size=558444 2024-04-22T19:58:47.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.080+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=AN0CNDF3ER8R type=del size=483781 2024-04-22T19:58:47.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.081+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=1DIAEPP3TLSK type=del 2024-04-22T19:58:47.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.083+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~XD0P0ARFJXNR.0 size=1048576 2024-04-22T19:58:47.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.084+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~XD0P0ARFJXNR.1 size=711364 2024-04-22T19:58:47.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.084+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=XD0P0ARFJXNR tag=8SP98E07928K type=write size=2808516 2024-04-22T19:58:47.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.084+0000 7f5269a8e780 1 < canceled part key=_multipart_GRSL48RUEBX8.2~57GO6U1AY8Y8.0 size=963795 2024-04-22T19:58:47.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.087+0000 7f5269a8e780 1 < canceled multipart upload key=GRSL48RUEBX8 upload=57GO6U1AY8Y8 tag=94M99VXU0AYU type=write size=2012371 2024-04-22T19:58:47.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.089+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~XGNSDSR1ZBD3.0 size=283656 2024-04-22T19:58:47.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.089+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=XGNSDSR1ZBD3 tag=58BM6GX332FL type=write size=1332232 2024-04-22T19:58:47.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.090+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~TFSMOUT4N87V.0 size=1048576 2024-04-22T19:58:47.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.091+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~TFSMOUT4N87V.1 size=1048576 2024-04-22T19:58:47.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.092+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~TFSMOUT4N87V.2 size=907945 2024-04-22T19:58:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.094+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=TFSMOUT4N87V tag=B4JG90KI63KZ type=write size=4053673 2024-04-22T19:58:47.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.095+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=PGFW8E9ZIVSF type=del size=2571610 2024-04-22T19:58:47.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.097+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~0ZPSTOIT40XF.0 size=1048576 2024-04-22T19:58:47.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.098+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~0ZPSTOIT40XF.1 size=1048576 2024-04-22T19:58:47.099 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.099+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~0ZPSTOIT40XF.2 size=796954 2024-04-22T19:58:47.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.101+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=0ZPSTOIT40XF tag=GKTHTUSJTXG7 type=write size=3942682 2024-04-22T19:58:47.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.102+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=RGTIXBDT2F3V type=del size=2246585 2024-04-22T19:58:47.103 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.103+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~XO14429CI9DP.0 size=1048576 2024-04-22T19:58:47.104 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.104+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~XO14429CI9DP.1 size=127621 2024-04-22T19:58:47.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.106+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=XO14429CI9DP tag=ODGVJ09998E0 type=write size=2224773 2024-04-22T19:58:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.107+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=SI3JR4NSSPFS type=del size=114363 2024-04-22T19:58:47.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.108+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~4Y7EWJPT9A2U.0 size=899098 2024-04-22T19:58:47.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.110+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=4Y7EWJPT9A2U tag=QGRIUZC3FJNA type=write size=1947674 2024-04-22T19:58:47.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.111+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=0E2I8FP7SNWW type=write size=697276 2024-04-22T19:58:47.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.112+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=QL35L96OGOQ3 type=del 2024-04-22T19:58:47.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.115+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~021GO5R8AKCB.0 size=160120 2024-04-22T19:58:47.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.115+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=021GO5R8AKCB tag=OC4Y3K81NWIY type=write size=1208696 2024-04-22T19:58:47.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.116+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=LM0Z5E535330 type=del 2024-04-22T19:58:47.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.117+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=J4QT3I3K7ZR3 type=del size=3745196 2024-04-22T19:58:47.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.118+0000 7f5269a8e780 1 < canceled operation key=M9M4SEEEZSFV tag=A0S5UR911TGN type=del size=3105429 2024-04-22T19:58:47.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.120+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~X0WIKHH7AI8T.0 size=1048576 2024-04-22T19:58:47.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.121+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~X0WIKHH7AI8T.1 size=330129 2024-04-22T19:58:47.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.121+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=X0WIKHH7AI8T tag=M8L04HXHPENL type=write size=2427281 2024-04-22T19:58:47.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.122+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~0EUZLG62KYGK.0 size=1048576 2024-04-22T19:58:47.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.123+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~0EUZLG62KYGK.1 size=1048576 2024-04-22T19:58:47.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.123+0000 7f5269a8e780 1 < canceled part key=_multipart_70MLQGJPQYPY.2~0EUZLG62KYGK.2 size=751477 2024-04-22T19:58:47.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.125+0000 7f5269a8e780 1 < canceled multipart upload key=70MLQGJPQYPY upload=0EUZLG62KYGK tag=BPQHLZKG52JB type=write size=3897205 2024-04-22T19:58:47.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.127+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~L59R9X76CJ8E.0 size=183880 2024-04-22T19:58:47.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.127+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=L59R9X76CJ8E tag=WLK32NRX8LGC type=write size=1232456 2024-04-22T19:58:47.128 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.128+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=DHIGZPUZTPZD type=del 2024-04-22T19:58:47.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.130+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~L40TAZRPWBD4.0 size=1048576 2024-04-22T19:58:47.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.131+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~L40TAZRPWBD4.1 size=1048576 2024-04-22T19:58:47.132 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.132+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~L40TAZRPWBD4.2 size=558482 2024-04-22T19:58:47.132 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.132+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=L40TAZRPWBD4 tag=YUUIV7WUOX1T type=write size=3704210 2024-04-22T19:58:47.134 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.134+0000 7f5269a8e780 1 < suggested update operation key=5UBYMQT7ZWKX tag=987Y3YFO5C8P type=del size=3893033 2024-04-22T19:58:47.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.136+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~DHK2TROERFPT.0 size=1048576 2024-04-22T19:58:47.137 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.137+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~DHK2TROERFPT.1 size=1048576 2024-04-22T19:58:47.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.138+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~DHK2TROERFPT.2 size=1063 2024-04-22T19:58:47.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.138+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=DHK2TROERFPT tag=Y4R0TRCH0KYQ type=write size=3146791 2024-04-22T19:58:47.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.139+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=ZKLXUEKVQGYP type=del 2024-04-22T19:58:47.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.140+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=KC3MXL8XIO7K type=del size=2793151 2024-04-22T19:58:47.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.141+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~VAUA3YJQ4YGU.0 size=1048576 2024-04-22T19:58:47.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.142+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~VAUA3YJQ4YGU.1 size=842559 2024-04-22T19:58:47.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.144+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=VAUA3YJQ4YGU tag=00V6YQ3KP4IT type=write size=2939711 2024-04-22T19:58:47.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.145+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=GL84ZUWQUFZU type=del size=3201414 2024-04-22T19:58:47.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.146+0000 7f5269a8e780 1 < completed write operation key=EHRS6DQDRXVS tag=WAG62M2PT1M3 type=write size=237788 2024-04-22T19:58:47.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.148+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~IWW6MPDLHHRT.0 size=1048576 2024-04-22T19:58:47.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.149+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~IWW6MPDLHHRT.1 size=1048576 2024-04-22T19:58:47.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.150+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~IWW6MPDLHHRT.2 size=950957 2024-04-22T19:58:47.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.150+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=IWW6MPDLHHRT tag=MNOC4RXUN3JK type=write size=4096685 2024-04-22T19:58:47.151 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.151+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=AN0CNDF3ER8R type=del 2024-04-22T19:58:47.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.152+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=PD0DOS37DMWN type=del size=2186934 2024-04-22T19:58:47.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.153+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~XD0P0ARFJXNR.0 size=1048576 2024-04-22T19:58:47.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.155+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~XD0P0ARFJXNR.1 size=711364 2024-04-22T19:58:47.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.157+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=XD0P0ARFJXNR tag=8SP98E07928K type=write size=2808516 2024-04-22T19:58:47.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.158+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=OCNG4AMJTRJ4 type=del size=558857 2024-04-22T19:58:47.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.159+0000 7f5269a8e780 1 < canceled part key=_multipart_GX6NILQG878S.2~XGNSDSR1ZBD3.0 size=283656 2024-04-22T19:58:47.160 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.160+0000 7f5269a8e780 1 < canceled multipart upload key=GX6NILQG878S upload=XGNSDSR1ZBD3 tag=58BM6GX332FL type=write size=1332232 2024-04-22T19:58:47.161 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.161+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=750OEAIHZPNV type=del size=972881 2024-04-22T19:58:47.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.163+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=PGFW8E9ZIVSF type=del 2024-04-22T19:58:47.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.164+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=IIC3G1VBDYG9 type=del size=366324 2024-04-22T19:58:47.165 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.165+0000 7f5269a8e780 1 < canceled operation key=SN2BZATJGUWA tag=RGTIXBDT2F3V type=del size=2246585 2024-04-22T19:58:47.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.167+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=WA8WSYGH22WU type=del size=1023910 2024-04-22T19:58:47.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.168+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=SI3JR4NSSPFS type=del 2024-04-22T19:58:47.169 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.170+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=QXVR0P0GXB6Y type=write size=546963 2024-04-22T19:58:47.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.171+0000 7f5269a8e780 1 < completed write operation key=AMBJHMJGLUUS tag=0E2I8FP7SNWW type=write size=697276 2024-04-22T19:58:47.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.172+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=O1JPMKW8D34T type=write size=252153 2024-04-22T19:58:47.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.173+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~021GO5R8AKCB.0 size=160120 2024-04-22T19:58:47.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.175+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=021GO5R8AKCB tag=OC4Y3K81NWIY type=write size=1208696 2024-04-22T19:58:47.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.176+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=D0G0TD4P3B9F type=del size=2597644 2024-04-22T19:58:47.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.177+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=J4QT3I3K7ZR3 type=del 2024-04-22T19:58:47.178 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.178+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=C9U9YAHR2ZCC type=del size=1246065 2024-04-22T19:58:47.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.179+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~X0WIKHH7AI8T.0 size=1048576 2024-04-22T19:58:47.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.180+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~X0WIKHH7AI8T.1 size=330129 2024-04-22T19:58:47.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.182+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=X0WIKHH7AI8T tag=M8L04HXHPENL type=write size=2427281 2024-04-22T19:58:47.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.183+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=TMQ8BMH81OL5 type=del size=3629740 2024-04-22T19:58:47.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.184+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~L59R9X76CJ8E.0 size=183880 2024-04-22T19:58:47.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.186+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=L59R9X76CJ8E tag=WLK32NRX8LGC type=write size=1232456 2024-04-22T19:58:47.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.187+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=XOOUBM6S9UHJ type=del size=3396655 2024-04-22T19:58:47.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.189+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~L40TAZRPWBD4.0 size=1048576 2024-04-22T19:58:47.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.189+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~L40TAZRPWBD4.1 size=1048576 2024-04-22T19:58:47.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.190+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~L40TAZRPWBD4.2 size=558482 2024-04-22T19:58:47.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.192+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=L40TAZRPWBD4 tag=YUUIV7WUOX1T type=write size=3704210 2024-04-22T19:58:47.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.194+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~357FSBEJMW5S.0 size=59036 2024-04-22T19:58:47.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.194+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=357FSBEJMW5S tag=H31PCP9P2DGU type=write size=1107612 2024-04-22T19:58:47.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.196+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~DHK2TROERFPT.0 size=1048576 2024-04-22T19:58:47.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.197+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~DHK2TROERFPT.1 size=1048576 2024-04-22T19:58:47.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.197+0000 7f5269a8e780 1 < canceled part key=_multipart_8HFSDSNZONIN.2~DHK2TROERFPT.2 size=1063 2024-04-22T19:58:47.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.199+0000 7f5269a8e780 1 < canceled multipart upload key=8HFSDSNZONIN upload=DHK2TROERFPT tag=Y4R0TRCH0KYQ type=write size=3146791 2024-04-22T19:58:47.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.200+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=354OVPDW3YYF type=del size=2363259 2024-04-22T19:58:47.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.201+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=KC3MXL8XIO7K type=del 2024-04-22T19:58:47.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.202+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=FPJ9F4FIIQ2N type=del size=1641440 2024-04-22T19:58:47.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.203+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=GL84ZUWQUFZU type=del 2024-04-22T19:58:47.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.205+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~E7RQIK89XG3B.0 size=1048576 2024-04-22T19:58:47.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.206+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~E7RQIK89XG3B.1 size=1048576 2024-04-22T19:58:47.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.207+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~E7RQIK89XG3B.2 size=391955 2024-04-22T19:58:47.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.207+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=E7RQIK89XG3B tag=U2L286VNGJH3 type=write size=3537683 2024-04-22T19:58:47.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.208+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~IWW6MPDLHHRT.0 size=1048576 2024-04-22T19:58:47.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.209+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~IWW6MPDLHHRT.1 size=1048576 2024-04-22T19:58:47.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.210+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~IWW6MPDLHHRT.2 size=950957 2024-04-22T19:58:47.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.212+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=IWW6MPDLHHRT tag=MNOC4RXUN3JK type=write size=4096685 2024-04-22T19:58:47.213 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.214+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=VZCKR1NUMTET type=del size=2883734 2024-04-22T19:58:47.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.215+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=PD0DOS37DMWN type=del 2024-04-22T19:58:47.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.217+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~KS2S00J5NX3A.0 size=1048576 2024-04-22T19:58:47.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.218+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~KS2S00J5NX3A.1 size=1048576 2024-04-22T19:58:47.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.219+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~KS2S00J5NX3A.2 size=827388 2024-04-22T19:58:47.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.219+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=KS2S00J5NX3A tag=SSXCELONPOKR type=write size=3973116 2024-04-22T19:58:47.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.220+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=OCNG4AMJTRJ4 type=del 2024-04-22T19:58:47.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.222+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~ZG7Q84UFA20R.0 size=1048576 2024-04-22T19:58:47.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.223+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~ZG7Q84UFA20R.1 size=794677 2024-04-22T19:58:47.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.223+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=ZG7Q84UFA20R tag=QEKXO4F90D3W type=write size=2891829 2024-04-22T19:58:47.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.226+0000 7f5269a8e780 1 < suggested remove operation key=GX6NILQG878S tag=750OEAIHZPNV type=del 2024-04-22T19:58:47.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.227+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=TIQKIDKOJWX1 type=del size=1733097 2024-04-22T19:58:47.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.229+0000 7f5269a8e780 1 < suggested update operation key=86GW6HV9G1IZ tag=IIC3G1VBDYG9 type=del size=366324 2024-04-22T19:58:47.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.230+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=7VUIK65SK3E6 type=del size=1403665 2024-04-22T19:58:47.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.232+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=WA8WSYGH22WU type=del 2024-04-22T19:58:47.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.233+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=QK0RKIDCZEGU type=write size=437643 2024-04-22T19:58:47.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.235+0000 7f5269a8e780 1 < suggested remove operation key=I91AD8FJ3P4T tag=QXVR0P0GXB6Y type=write 2024-04-22T19:58:47.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.236+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=DUFI2LH5Y2J7 type=del size=1539299 2024-04-22T19:58:47.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.237+0000 7f5269a8e780 1 < canceled operation key=AMBJHMJGLUUS tag=O1JPMKW8D34T type=write size=252153 2024-04-22T19:58:47.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.239+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=PK8JVDPUXH6K type=write size=248525 2024-04-22T19:58:47.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.240+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=D0G0TD4P3B9F type=del 2024-04-22T19:58:47.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.242+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~B2HKVH753BVW.0 size=106836 2024-04-22T19:58:47.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.242+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=B2HKVH753BVW tag=QM1SO4NNAVJV type=write size=1155412 2024-04-22T19:58:47.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.243+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=C9U9YAHR2ZCC type=del 2024-04-22T19:58:47.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.244+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=I9WHWQINRMV6 type=del size=4111961 2024-04-22T19:58:47.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.245+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=TMQ8BMH81OL5 type=del 2024-04-22T19:58:47.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.247+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~N98XVQ9GOUKK.0 size=1048576 2024-04-22T19:58:47.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.248+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~N98XVQ9GOUKK.1 size=1048576 2024-04-22T19:58:47.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.249+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~N98XVQ9GOUKK.2 size=891421 2024-04-22T19:58:47.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.249+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=N98XVQ9GOUKK tag=RW96CJ3M9I8C type=write size=4037149 2024-04-22T19:58:47.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.250+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=XOOUBM6S9UHJ type=del 2024-04-22T19:58:47.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.252+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~POV3C97AI0SJ.0 size=1048576 2024-04-22T19:58:47.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.253+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~POV3C97AI0SJ.1 size=876297 2024-04-22T19:58:47.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.253+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=POV3C97AI0SJ tag=C24TR8XQJ3CH type=write size=2973449 2024-04-22T19:58:47.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.254+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~357FSBEJMW5S.0 size=59036 2024-04-22T19:58:47.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.256+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=357FSBEJMW5S tag=H31PCP9P2DGU type=write size=1107612 2024-04-22T19:58:47.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.258+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~XRWYWHOBCRWZ.0 size=1048576 2024-04-22T19:58:47.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.259+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~XRWYWHOBCRWZ.1 size=312186 2024-04-22T19:58:47.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.259+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=XRWYWHOBCRWZ tag=70EEEMUS9QE8 type=write size=2409338 2024-04-22T19:58:47.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.260+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=354OVPDW3YYF type=del 2024-04-22T19:58:47.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.261+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=A57EO2VF2AK9 type=del size=2622430 2024-04-22T19:58:47.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.262+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=FPJ9F4FIIQ2N type=del 2024-04-22T19:58:47.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.264+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~TEO2DMOATRL7.0 size=1048576 2024-04-22T19:58:47.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.265+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~TEO2DMOATRL7.1 size=717544 2024-04-22T19:58:47.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.265+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=TEO2DMOATRL7 tag=9XH1YQUHFOZB type=write size=2814696 2024-04-22T19:58:47.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.267+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~E7RQIK89XG3B.0 size=1048576 2024-04-22T19:58:47.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.268+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~E7RQIK89XG3B.1 size=1048576 2024-04-22T19:58:47.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.269+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~E7RQIK89XG3B.2 size=391955 2024-04-22T19:58:47.271 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.271+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=E7RQIK89XG3B tag=U2L286VNGJH3 type=write size=3537683 2024-04-22T19:58:47.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.273+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~H00PCEGQEGJ4.0 size=197093 2024-04-22T19:58:47.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.273+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=H00PCEGQEGJ4 tag=ZY3FAEW0FMX0 type=write size=1245669 2024-04-22T19:58:47.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.275+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=VZCKR1NUMTET type=del 2024-04-22T19:58:47.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.278+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~6UE6T5A82JII.0 size=1048576 2024-04-22T19:58:47.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.279+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~6UE6T5A82JII.1 size=1048576 2024-04-22T19:58:47.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.280+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~6UE6T5A82JII.2 size=774874 2024-04-22T19:58:47.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.280+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=6UE6T5A82JII tag=AN3ALMLZ7Z20 type=write size=3920602 2024-04-22T19:58:47.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.281+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~KS2S00J5NX3A.0 size=1048576 2024-04-22T19:58:47.282 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.282+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~KS2S00J5NX3A.1 size=1048576 2024-04-22T19:58:47.283 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.283+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~KS2S00J5NX3A.2 size=827388 2024-04-22T19:58:47.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.285+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=KS2S00J5NX3A tag=SSXCELONPOKR type=write size=3973116 2024-04-22T19:58:47.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.286+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=CHJ3OA25VL65 type=del size=2991906 2024-04-22T19:58:47.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.287+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~ZG7Q84UFA20R.0 size=1048576 2024-04-22T19:58:47.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.288+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~ZG7Q84UFA20R.1 size=794677 2024-04-22T19:58:47.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.290+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=ZG7Q84UFA20R tag=QEKXO4F90D3W type=write size=2891829 2024-04-22T19:58:47.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.291+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=8AXX87EAOWWL type=del size=1636400 2024-04-22T19:58:47.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.292+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=TIQKIDKOJWX1 type=del 2024-04-22T19:58:47.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.294+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~SJBPFGSA6ESS.0 size=1048576 2024-04-22T19:58:47.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.295+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~SJBPFGSA6ESS.1 size=680125 2024-04-22T19:58:47.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.295+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=SJBPFGSA6ESS tag=K35D15AHNJUO type=write size=2777277 2024-04-22T19:58:47.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.297+0000 7f5269a8e780 1 < canceled operation key=5UBYMQT7ZWKX tag=7VUIK65SK3E6 type=del size=1403665 2024-04-22T19:58:47.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.299+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~NZN79E50UR88.0 size=1048576 2024-04-22T19:58:47.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.299+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~NZN79E50UR88.1 size=797893 2024-04-22T19:58:47.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.299+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=NZN79E50UR88 tag=JREQSZF14GQ6 type=write size=2895045 2024-04-22T19:58:47.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.301+0000 7f5269a8e780 1 < canceled operation key=Y5HR8XW4L0SP tag=QK0RKIDCZEGU type=write size=437643 2024-04-22T19:58:47.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.302+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=G6F8GA9DPIV3 type=del size=765933 2024-04-22T19:58:47.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.304+0000 7f5269a8e780 1 < suggested update operation key=SN2BZATJGUWA tag=DUFI2LH5Y2J7 type=del size=1539299 2024-04-22T19:58:47.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.306+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~2TNCVWI1YQZB.0 size=233175 2024-04-22T19:58:47.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.306+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=2TNCVWI1YQZB tag=PLWK8YKCM4B8 type=write size=1281751 2024-04-22T19:58:47.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.307+0000 7f5269a8e780 1 < canceled operation key=86GW6HV9G1IZ tag=PK8JVDPUXH6K type=write size=248525 2024-04-22T19:58:47.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.309+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~W34O7WH9N9IB.0 size=1048576 2024-04-22T19:58:47.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.310+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~W34O7WH9N9IB.1 size=326555 2024-04-22T19:58:47.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.310+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=W34O7WH9N9IB tag=WZE3SM0AAEY9 type=write size=2423707 2024-04-22T19:58:47.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.311+0000 7f5269a8e780 1 < canceled part key=_multipart_8HFSDSNZONIN.2~B2HKVH753BVW.0 size=106836 2024-04-22T19:58:47.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.313+0000 7f5269a8e780 1 < canceled multipart upload key=8HFSDSNZONIN upload=B2HKVH753BVW tag=QM1SO4NNAVJV type=write size=1155412 2024-04-22T19:58:47.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.314+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=8RCLTW59RFZ4 type=del size=3601211 2024-04-22T19:58:47.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.316+0000 7f5269a8e780 1 < suggested remove operation key=5UBYMQT7ZWKX tag=I9WHWQINRMV6 type=del 2024-04-22T19:58:47.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.317+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=1E697EZ02PJI type=del size=1690599 2024-04-22T19:58:47.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.319+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~N98XVQ9GOUKK.0 size=1048576 2024-04-22T19:58:47.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.319+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~N98XVQ9GOUKK.1 size=1048576 2024-04-22T19:58:47.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.320+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~N98XVQ9GOUKK.2 size=891421 2024-04-22T19:58:47.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.322+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=N98XVQ9GOUKK tag=RW96CJ3M9I8C type=write size=4037149 2024-04-22T19:58:47.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.324+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=NNHSQI5SKFTO type=del size=2410996 2024-04-22T19:58:47.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.325+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~POV3C97AI0SJ.0 size=1048576 2024-04-22T19:58:47.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.326+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~POV3C97AI0SJ.1 size=876297 2024-04-22T19:58:47.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.328+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=POV3C97AI0SJ tag=C24TR8XQJ3CH type=write size=2973449 2024-04-22T19:58:47.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.329+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=4OPKPMQLQ5WV type=write size=979344 2024-04-22T19:58:47.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.330+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~XRWYWHOBCRWZ.0 size=1048576 2024-04-22T19:58:47.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.331+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~XRWYWHOBCRWZ.1 size=312186 2024-04-22T19:58:47.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.333+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=XRWYWHOBCRWZ tag=70EEEMUS9QE8 type=write size=2409338 2024-04-22T19:58:47.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.335+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~DGYPFMENBRD9.0 size=1048576 2024-04-22T19:58:47.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.336+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~DGYPFMENBRD9.1 size=189494 2024-04-22T19:58:47.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.336+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=DGYPFMENBRD9 tag=ALGHA8SAWQ77 type=write size=2286646 2024-04-22T19:58:47.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.337+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=A57EO2VF2AK9 type=del 2024-04-22T19:58:47.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.338+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=30PMVZ15VYEZ type=del size=4072586 2024-04-22T19:58:47.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.340+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~TEO2DMOATRL7.0 size=1048576 2024-04-22T19:58:47.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.340+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~TEO2DMOATRL7.1 size=717544 2024-04-22T19:58:47.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.342+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=TEO2DMOATRL7 tag=9XH1YQUHFOZB type=write size=2814696 2024-04-22T19:58:47.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.344+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=Y1JJP1JJLEIS type=del size=2425796 2024-04-22T19:58:47.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.345+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~H00PCEGQEGJ4.0 size=197093 2024-04-22T19:58:47.346 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.346+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=H00PCEGQEGJ4 tag=ZY3FAEW0FMX0 type=write size=1245669 2024-04-22T19:58:47.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.348+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=7RJEMYVB5KL2 type=del size=3539920 2024-04-22T19:58:47.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.349+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~6UE6T5A82JII.0 size=1048576 2024-04-22T19:58:47.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.350+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~6UE6T5A82JII.1 size=1048576 2024-04-22T19:58:47.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.350+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~6UE6T5A82JII.2 size=774874 2024-04-22T19:58:47.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.353+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=6UE6T5A82JII tag=AN3ALMLZ7Z20 type=write size=3920602 2024-04-22T19:58:47.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.354+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=UATA98MXDY4Y type=del size=1329890 2024-04-22T19:58:47.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.355+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=CHJ3OA25VL65 type=del 2024-04-22T19:58:47.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.357+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~TFZJMEV9MTGJ.0 size=1048576 2024-04-22T19:58:47.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.358+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~TFZJMEV9MTGJ.1 size=1048576 2024-04-22T19:58:47.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.359+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~TFZJMEV9MTGJ.2 size=984370 2024-04-22T19:58:47.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.359+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=TFZJMEV9MTGJ tag=H9O04BL0PFKX type=write size=4130098 2024-04-22T19:58:47.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.361+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=8AXX87EAOWWL type=del 2024-04-22T19:58:47.362 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.363+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~3Y8ZVSYLO4MO.0 size=154801 2024-04-22T19:58:47.363 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.363+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=3Y8ZVSYLO4MO tag=IJB26D5W1F1S type=write size=1203377 2024-04-22T19:58:47.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.364+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~SJBPFGSA6ESS.0 size=1048576 2024-04-22T19:58:47.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.365+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~SJBPFGSA6ESS.1 size=680125 2024-04-22T19:58:47.367 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.367+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=SJBPFGSA6ESS tag=K35D15AHNJUO type=write size=2777277 2024-04-22T19:58:47.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.368+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=0YF0BY7U9RPS type=write size=728178 2024-04-22T19:58:47.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.369+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~NZN79E50UR88.0 size=1048576 2024-04-22T19:58:47.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.370+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~NZN79E50UR88.1 size=797893 2024-04-22T19:58:47.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.372+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=NZN79E50UR88 tag=JREQSZF14GQ6 type=write size=2895045 2024-04-22T19:58:47.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.374+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~1RAB8W6BYOZF.0 size=1048576 2024-04-22T19:58:47.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.375+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~1RAB8W6BYOZF.1 size=782062 2024-04-22T19:58:47.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.375+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=1RAB8W6BYOZF tag=0X5DSBQCFHOH type=write size=2879214 2024-04-22T19:58:47.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.376+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=G6F8GA9DPIV3 type=del 2024-04-22T19:58:47.377 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.377+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=05ZRJE6W9PQ5 type=del size=3281584 2024-04-22T19:58:47.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.378+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~2TNCVWI1YQZB.0 size=233175 2024-04-22T19:58:47.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.380+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=2TNCVWI1YQZB tag=PLWK8YKCM4B8 type=write size=1281751 2024-04-22T19:58:47.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.383+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=SXB4DOSYJINI type=del size=1340939 2024-04-22T19:58:47.384 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.384+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~W34O7WH9N9IB.0 size=1048576 2024-04-22T19:58:47.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.385+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~W34O7WH9N9IB.1 size=326555 2024-04-22T19:58:47.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.387+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=W34O7WH9N9IB tag=WZE3SM0AAEY9 type=write size=2423707 2024-04-22T19:58:47.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.390+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~XWFES30XMAVG.0 size=1048576 2024-04-22T19:58:47.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.391+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~XWFES30XMAVG.1 size=471308 2024-04-22T19:58:47.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.391+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=XWFES30XMAVG tag=WNA2T1JGOCS1 type=write size=2568460 2024-04-22T19:58:47.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.392+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=8RCLTW59RFZ4 type=del 2024-04-22T19:58:47.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.393+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=TIGKZ0QPJCL1 type=del size=1223883 2024-04-22T19:58:47.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.394+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=1E697EZ02PJI type=del 2024-04-22T19:58:47.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.396+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~QEGDUEHKG2VH.0 size=1006480 2024-04-22T19:58:47.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.396+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=QEGDUEHKG2VH tag=ZPOUM8NIIBIK type=write size=2055056 2024-04-22T19:58:47.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.399+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=NNHSQI5SKFTO type=del 2024-04-22T19:58:47.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.401+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~KMG34PR2HP4T.0 size=1048576 2024-04-22T19:58:47.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.402+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~KMG34PR2HP4T.1 size=918830 2024-04-22T19:58:47.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.402+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=KMG34PR2HP4T tag=6ZDHI6JNVHVN type=write size=3015982 2024-04-22T19:58:47.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.403+0000 7f5269a8e780 1 < completed write operation key=ZC4ZOHUJ3HQD tag=4OPKPMQLQ5WV type=write size=979344 2024-04-22T19:58:47.404 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.404+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=O5SXBQYGKG0V type=del size=3282285 2024-04-22T19:58:47.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.406+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~DGYPFMENBRD9.0 size=1048576 2024-04-22T19:58:47.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.407+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~DGYPFMENBRD9.1 size=189494 2024-04-22T19:58:47.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.409+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=DGYPFMENBRD9 tag=ALGHA8SAWQ77 type=write size=2286646 2024-04-22T19:58:47.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.411+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~V6QJ2UNHKL4A.0 size=1048576 2024-04-22T19:58:47.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.412+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~V6QJ2UNHKL4A.1 size=90159 2024-04-22T19:58:47.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.412+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=V6QJ2UNHKL4A tag=DG4J7G87B22V type=write size=2187311 2024-04-22T19:58:47.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.413+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=30PMVZ15VYEZ type=del 2024-04-22T19:58:47.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.414+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=A32X0DG0NFB6 type=del size=2697726 2024-04-22T19:58:47.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.415+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=Y1JJP1JJLEIS type=del 2024-04-22T19:58:47.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.417+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=Y5FSKRL3INSX type=del size=3522250 2024-04-22T19:58:47.418 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.418+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=7RJEMYVB5KL2 type=del 2024-04-22T19:58:47.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.419+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=CLXYSC55W7J9 type=write size=16869 2024-04-22T19:58:47.420 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.420+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=UATA98MXDY4Y type=del 2024-04-22T19:58:47.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.422+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=MOMTSZ9H55UG type=write size=570607 2024-04-22T19:58:47.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.422+0000 7f5269a8e780 1 < canceled part key=_multipart_4EA3LUQRL8II.2~TFZJMEV9MTGJ.0 size=1048576 2024-04-22T19:58:47.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.423+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~TFZJMEV9MTGJ.1 size=1048576 2024-04-22T19:58:47.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.424+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~TFZJMEV9MTGJ.2 size=984370 2024-04-22T19:58:47.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.426+0000 7f5269a8e780 1 < canceled multipart upload key=4EA3LUQRL8II upload=TFZJMEV9MTGJ tag=H9O04BL0PFKX type=write size=4130098 2024-04-22T19:58:47.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.427+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=R8RP3HI9Y855 type=del size=3448348 2024-04-22T19:58:47.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.428+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~3Y8ZVSYLO4MO.0 size=154801 2024-04-22T19:58:47.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.430+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=3Y8ZVSYLO4MO tag=IJB26D5W1F1S type=write size=1203377 2024-04-22T19:58:47.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.431+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=2FAF38I0HOS1 type=del size=3355600 2024-04-22T19:58:47.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.432+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=0YF0BY7U9RPS type=write size=728178 2024-04-22T19:58:47.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.433+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=O2UJWW5H92KS type=del size=1233706 2024-04-22T19:58:47.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.435+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~1RAB8W6BYOZF.0 size=1048576 2024-04-22T19:58:47.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.436+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~1RAB8W6BYOZF.1 size=782062 2024-04-22T19:58:47.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.437+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=1RAB8W6BYOZF tag=0X5DSBQCFHOH type=write size=2879214 2024-04-22T19:58:47.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.438+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=9026QDVKLC4Q type=del size=2772588 2024-04-22T19:58:47.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.440+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=05ZRJE6W9PQ5 type=del 2024-04-22T19:58:47.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.441+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=X39MK58Z0XSQ type=del size=2791892 2024-04-22T19:58:47.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.442+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=SXB4DOSYJINI type=del 2024-04-22T19:58:47.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.444+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~Z3XPWMPSVQ1C.0 size=383680 2024-04-22T19:58:47.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.444+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=Z3XPWMPSVQ1C tag=STOLEDV1JUKK type=write size=1432256 2024-04-22T19:58:47.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.445+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~XWFES30XMAVG.0 size=1048576 2024-04-22T19:58:47.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.445+0000 7f5269a8e780 1 < canceled part key=_multipart_HIAXPVKZ1IIO.2~XWFES30XMAVG.1 size=471308 2024-04-22T19:58:47.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.447+0000 7f5269a8e780 1 < canceled multipart upload key=HIAXPVKZ1IIO upload=XWFES30XMAVG tag=WNA2T1JGOCS1 type=write size=2568460 2024-04-22T19:58:47.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.448+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=OBAYUTUMHWFP type=write size=268057 2024-04-22T19:58:47.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.449+0000 7f5269a8e780 1 < canceled operation key=ZC4ZOHUJ3HQD tag=TIGKZ0QPJCL1 type=del size=1223883 2024-04-22T19:58:47.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.450+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=PFR262Q9LWLB type=del size=2401842 2024-04-22T19:58:47.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.452+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~QEGDUEHKG2VH.0 size=1006480 2024-04-22T19:58:47.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.454+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=QEGDUEHKG2VH tag=ZPOUM8NIIBIK type=write size=2055056 2024-04-22T19:58:47.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.456+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~535ZKH7BOGX8.0 size=1048576 2024-04-22T19:58:47.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.456+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~535ZKH7BOGX8.1 size=880678 2024-04-22T19:58:47.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.456+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=535ZKH7BOGX8 tag=1D0YR1S2EZM3 type=write size=2977830 2024-04-22T19:58:47.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.458+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~KMG34PR2HP4T.0 size=1048576 2024-04-22T19:58:47.458 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.458+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~KMG34PR2HP4T.1 size=918830 2024-04-22T19:58:47.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.460+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=KMG34PR2HP4T tag=6ZDHI6JNVHVN type=write size=3015982 2024-04-22T19:58:47.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.461+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=5LLKKR60VXJ0 type=write size=429626 2024-04-22T19:58:47.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.463+0000 7f5269a8e780 1 < suggested update operation key=RY1O7X1CWZMK tag=O5SXBQYGKG0V type=del size=3282285 2024-04-22T19:58:47.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.465+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~A0RNWHRSOQ7G.0 size=1048576 2024-04-22T19:58:47.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.466+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~A0RNWHRSOQ7G.1 size=1039236 2024-04-22T19:58:47.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.466+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=A0RNWHRSOQ7G tag=9P6CJY299N8E type=write size=3136388 2024-04-22T19:58:47.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.467+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~V6QJ2UNHKL4A.0 size=1048576 2024-04-22T19:58:47.468 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.468+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~V6QJ2UNHKL4A.1 size=90159 2024-04-22T19:58:47.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.470+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=V6QJ2UNHKL4A tag=DG4J7G87B22V type=write size=2187311 2024-04-22T19:58:47.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.471+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=Y64M3WNANEJZ type=del size=106035 2024-04-22T19:58:47.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.472+0000 7f5269a8e780 1 < canceled operation key=AMBJHMJGLUUS tag=A32X0DG0NFB6 type=del size=2697726 2024-04-22T19:58:47.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.473+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=9NX7LVYU8I5G type=del size=201969 2024-04-22T19:58:47.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.475+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=Y5FSKRL3INSX type=del 2024-04-22T19:58:47.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.477+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~5MAYE5FGBVFK.0 size=1048576 2024-04-22T19:58:47.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.477+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~5MAYE5FGBVFK.1 size=1048576 2024-04-22T19:58:47.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.478+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~5MAYE5FGBVFK.2 size=753595 2024-04-22T19:58:47.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.478+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=5MAYE5FGBVFK tag=V2U0DZJORFR4 type=write size=3899323 2024-04-22T19:58:47.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.480+0000 7f5269a8e780 1 < completed write operation key=M9M4SEEEZSFV tag=CLXYSC55W7J9 type=write size=16869 2024-04-22T19:58:47.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.481+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=XMB7BZG3Z8OA type=del size=2531803 2024-04-22T19:58:47.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.482+0000 7f5269a8e780 1 < completed write operation key=DX5YUYQRZKWA tag=MOMTSZ9H55UG type=write size=570607 2024-04-22T19:58:47.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.483+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=F38AWNF2QBKQ type=del size=3096103 2024-04-22T19:58:47.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.485+0000 7f5269a8e780 1 < suggested update operation key=AMBJHMJGLUUS tag=R8RP3HI9Y855 type=del size=3448348 2024-04-22T19:58:47.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.486+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=FG0OX3PFJANL type=del size=4018003 2024-04-22T19:58:47.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.487+0000 7f5269a8e780 1 < canceled operation key=IBH6W1NINUHC tag=2FAF38I0HOS1 type=del size=3355600 2024-04-22T19:58:47.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.489+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=9M187MZD8M7L type=del size=2172208 2024-04-22T19:58:47.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.491+0000 7f5269a8e780 1 < suggested remove operation key=86GW6HV9G1IZ tag=O2UJWW5H92KS type=del 2024-04-22T19:58:47.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.493+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=OLL7PLDFOG18 type=del size=312697 2024-04-22T19:58:47.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.494+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=9026QDVKLC4Q type=del 2024-04-22T19:58:47.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.496+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~HR73PCE1BKER.0 size=1048576 2024-04-22T19:58:47.497 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.497+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~HR73PCE1BKER.1 size=1048576 2024-04-22T19:58:47.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.498+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~HR73PCE1BKER.2 size=810442 2024-04-22T19:58:47.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.498+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=HR73PCE1BKER tag=STYQ2WHWJT1Z type=write size=3956170 2024-04-22T19:58:47.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.499+0000 7f5269a8e780 -1 < failed to complete operation key=AMBJHMJGLUUS tag=X39MK58Z0XSQ type=del size=2791892: (22) Invalid argument 2024-04-22T19:58:47.501 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.501+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~FU6BD0VXBKMG.0 size=1048576 2024-04-22T19:58:47.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.502+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~FU6BD0VXBKMG.1 size=76049 2024-04-22T19:58:47.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.502+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=FU6BD0VXBKMG tag=WADQNO2XDX2A type=write size=2173201 2024-04-22T19:58:47.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.503+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~Z3XPWMPSVQ1C.0 size=383680 2024-04-22T19:58:47.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.505+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=Z3XPWMPSVQ1C tag=STOLEDV1JUKK type=write size=1432256 2024-04-22T19:58:47.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.506+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=WJC8IWGB35HB type=del size=3989581 2024-04-22T19:58:47.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.507+0000 7f5269a8e780 1 < completed write operation key=HIAXPVKZ1IIO tag=OBAYUTUMHWFP type=write size=268057 2024-04-22T19:58:47.508 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.508+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=7CLBV868PSIJ type=del size=73275 2024-04-22T19:58:47.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.510+0000 7f5269a8e780 1 < canceled operation key=A853MOZPT9BG tag=PFR262Q9LWLB type=del size=2401842 2024-04-22T19:58:47.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.511+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=100B4PG6T0YV type=del size=1593341 2024-04-22T19:58:47.512 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.512+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~535ZKH7BOGX8.0 size=1048576 2024-04-22T19:58:47.513 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.513+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~535ZKH7BOGX8.1 size=880678 2024-04-22T19:58:47.515 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.515+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=535ZKH7BOGX8 tag=1D0YR1S2EZM3 type=write size=2977830 2024-04-22T19:58:47.516 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.516+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=7A3KC0IQ3RB1 type=del size=2361449 2024-04-22T19:58:47.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.517+0000 7f5269a8e780 1 < canceled operation key=B0Y7ZTPC0975 tag=5LLKKR60VXJ0 type=write size=429626 2024-04-22T19:58:47.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.518+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=4CC0LGC4X4N2 type=del size=2422811 2024-04-22T19:58:47.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.519+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~A0RNWHRSOQ7G.0 size=1048576 2024-04-22T19:58:47.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.520+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~A0RNWHRSOQ7G.1 size=1039236 2024-04-22T19:58:47.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.522+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=A0RNWHRSOQ7G tag=9P6CJY299N8E type=write size=3136388 2024-04-22T19:58:47.523 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.523+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=8XRNBSPHCENI type=del size=4021329 2024-04-22T19:58:47.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.525+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=Y64M3WNANEJZ type=del 2024-04-22T19:58:47.527 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.527+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~XUYTNLJ5236R.0 size=1048576 2024-04-22T19:58:47.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.528+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~XUYTNLJ5236R.1 size=536302 2024-04-22T19:58:47.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.528+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=XUYTNLJ5236R tag=PTO8UAOVITQI type=write size=2633454 2024-04-22T19:58:47.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.529+0000 7f5269a8e780 -1 < failed to complete operation key=86GW6HV9G1IZ tag=9NX7LVYU8I5G type=del size=201969: (22) Invalid argument 2024-04-22T19:58:47.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.531+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~YDXKDSBJZ24N.0 size=686128 2024-04-22T19:58:47.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.531+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=YDXKDSBJZ24N tag=23RZCG3CH179 type=write size=1734704 2024-04-22T19:58:47.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.532+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~5MAYE5FGBVFK.0 size=1048576 2024-04-22T19:58:47.533 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.533+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~5MAYE5FGBVFK.1 size=1048576 2024-04-22T19:58:47.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.534+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~5MAYE5FGBVFK.2 size=753595 2024-04-22T19:58:47.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.537+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=5MAYE5FGBVFK tag=V2U0DZJORFR4 type=write size=3899323 2024-04-22T19:58:47.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.539+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~SE4R4Y2XSRNB.0 size=1048576 2024-04-22T19:58:47.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.539+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~SE4R4Y2XSRNB.1 size=27185 2024-04-22T19:58:47.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.539+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=SE4R4Y2XSRNB tag=WJZGFL59Z6B4 type=write size=2124337 2024-04-22T19:58:47.540 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.541+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=XMB7BZG3Z8OA type=del 2024-04-22T19:58:47.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.543+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~YY11PTMV6BKI.0 size=1048576 2024-04-22T19:58:47.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.543+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~YY11PTMV6BKI.1 size=520424 2024-04-22T19:58:47.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.543+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=YY11PTMV6BKI tag=H2D6WQCSDG3L type=write size=2617576 2024-04-22T19:58:47.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.545+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=F38AWNF2QBKQ type=del 2024-04-22T19:58:47.546 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.546+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=WAGDVNVRVTKX type=del size=3680871 2024-04-22T19:58:47.547 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.547+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=FG0OX3PFJANL type=del 2024-04-22T19:58:47.548 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.548+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=YPGXPZH02DT1 type=del size=3366822 2024-04-22T19:58:47.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.549+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=9M187MZD8M7L type=del 2024-04-22T19:58:47.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.551+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~2UEFWY0OUYYO.0 size=1043806 2024-04-22T19:58:47.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.551+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=2UEFWY0OUYYO tag=DCGXY0OQY6CV type=write size=2092382 2024-04-22T19:58:47.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.553+0000 7f5269a8e780 1 < canceled operation key=M9M4SEEEZSFV tag=OLL7PLDFOG18 type=del size=312697 2024-04-22T19:58:47.554 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.555+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~GYQE0YFQFJH6.0 size=444897 2024-04-22T19:58:47.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.555+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=GYQE0YFQFJH6 tag=AHUY5G835Q7D type=write size=1493473 2024-04-22T19:58:47.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.556+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~HR73PCE1BKER.0 size=1048576 2024-04-22T19:58:47.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.556+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~HR73PCE1BKER.1 size=1048576 2024-04-22T19:58:47.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.557+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~HR73PCE1BKER.2 size=810442 2024-04-22T19:58:47.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.559+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=HR73PCE1BKER tag=STYQ2WHWJT1Z type=write size=3956170 2024-04-22T19:58:47.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.561+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~NY88YGQ2OKDM.0 size=1048576 2024-04-22T19:58:47.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.562+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~NY88YGQ2OKDM.1 size=1048576 2024-04-22T19:58:47.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.563+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~NY88YGQ2OKDM.2 size=209398 2024-04-22T19:58:47.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.563+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=NY88YGQ2OKDM tag=6OEIJ030QVW6 type=write size=3355126 2024-04-22T19:58:47.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.563+0000 7f5269a8e780 1 < canceled part key=_multipart_8HFSDSNZONIN.2~FU6BD0VXBKMG.0 size=1048576 2024-04-22T19:58:47.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.563+0000 7f5269a8e780 1 < canceled part key=_multipart_8HFSDSNZONIN.2~FU6BD0VXBKMG.1 size=76049 2024-04-22T19:58:47.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.565+0000 7f5269a8e780 1 < canceled multipart upload key=8HFSDSNZONIN upload=FU6BD0VXBKMG tag=WADQNO2XDX2A type=write size=2173201 2024-04-22T19:58:47.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.566+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=1QWE4BUFLI6P type=del size=3117514 2024-04-22T19:58:47.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.567+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=WJC8IWGB35HB type=del 2024-04-22T19:58:47.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.569+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=6FW7ABN7YU4U type=del size=4190673 2024-04-22T19:58:47.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.570+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=7CLBV868PSIJ type=del 2024-04-22T19:58:47.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.571+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=RZ5RAZQ7OP9Y type=del size=3836396 2024-04-22T19:58:47.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.572+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=100B4PG6T0YV type=del 2024-04-22T19:58:47.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.574+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=M2X3FD74ZTFC type=del size=2118096 2024-04-22T19:58:47.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.575+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=7A3KC0IQ3RB1 type=del 2024-04-22T19:58:47.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.577+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~NU7KI3ZUU7BR.0 size=1048576 2024-04-22T19:58:47.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.578+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~NU7KI3ZUU7BR.1 size=707105 2024-04-22T19:58:47.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.578+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=NU7KI3ZUU7BR tag=7XDIKLPBD9NZ type=write size=2804257 2024-04-22T19:58:47.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.579+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=4CC0LGC4X4N2 type=del 2024-04-22T19:58:47.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.580+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=JWNIBLRQPAZU type=del size=4153550 2024-04-22T19:58:47.582 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.582+0000 7f5269a8e780 1 < suggested update operation key=HIAXPVKZ1IIO tag=8XRNBSPHCENI type=del size=4021329 2024-04-22T19:58:47.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.583+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=3A3ZG0KQP454 type=del size=3449952 2024-04-22T19:58:47.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.585+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~XUYTNLJ5236R.0 size=1048576 2024-04-22T19:58:47.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.586+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~XUYTNLJ5236R.1 size=536302 2024-04-22T19:58:47.588 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.588+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=XUYTNLJ5236R tag=PTO8UAOVITQI type=write size=2633454 2024-04-22T19:58:47.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.590+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~ITHRTWL96Z2B.0 size=1048576 2024-04-22T19:58:47.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.591+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~ITHRTWL96Z2B.1 size=1048576 2024-04-22T19:58:47.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.594+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~ITHRTWL96Z2B.2 size=114530 2024-04-22T19:58:47.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.594+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=ITHRTWL96Z2B tag=PLM6F6B26JR9 type=write size=3260258 2024-04-22T19:58:47.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.595+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~YDXKDSBJZ24N.0 size=686128 2024-04-22T19:58:47.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.597+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=YDXKDSBJZ24N tag=23RZCG3CH179 type=write size=1734704 2024-04-22T19:58:47.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.598+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=8F6F9Y0E1MYS type=del size=550368 2024-04-22T19:58:47.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.598+0000 7f5269a8e780 1 < canceled part key=_multipart_8V6ESIGMRO5G.2~SE4R4Y2XSRNB.0 size=1048576 2024-04-22T19:58:47.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.599+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~SE4R4Y2XSRNB.1 size=27185 2024-04-22T19:58:47.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.601+0000 7f5269a8e780 1 < canceled multipart upload key=8V6ESIGMRO5G upload=SE4R4Y2XSRNB tag=WJZGFL59Z6B4 type=write size=2124337 2024-04-22T19:58:47.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.604+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~4KSQWXXW3FFX.0 size=368094 2024-04-22T19:58:47.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.604+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=4KSQWXXW3FFX tag=XPNR27343OWN type=write size=1416670 2024-04-22T19:58:47.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.607+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~YY11PTMV6BKI.0 size=1048576 2024-04-22T19:58:47.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.608+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~YY11PTMV6BKI.1 size=520424 2024-04-22T19:58:47.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.610+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=YY11PTMV6BKI tag=H2D6WQCSDG3L type=write size=2617576 2024-04-22T19:58:47.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.612+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~AVLTOFSGW4RL.0 size=1048576 2024-04-22T19:58:47.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.613+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~AVLTOFSGW4RL.1 size=267825 2024-04-22T19:58:47.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.613+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=AVLTOFSGW4RL tag=8ZMJO2L8N23N type=write size=2364977 2024-04-22T19:58:47.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.615+0000 7f5269a8e780 1 < suggested update operation key=5UBYMQT7ZWKX tag=WAGDVNVRVTKX type=del size=3680871 2024-04-22T19:58:47.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.617+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~81Q39BTR79DV.0 size=1048576 2024-04-22T19:58:47.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.618+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~81Q39BTR79DV.1 size=1045335 2024-04-22T19:58:47.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.618+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=81Q39BTR79DV tag=6S0WPNI7BUWR type=write size=3142487 2024-04-22T19:58:47.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.619+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=YPGXPZH02DT1 type=del 2024-04-22T19:58:47.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.621+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=X0048BL5KLSS type=del size=1615226 2024-04-22T19:58:47.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.622+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~2UEFWY0OUYYO.0 size=1043806 2024-04-22T19:58:47.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.624+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=2UEFWY0OUYYO tag=DCGXY0OQY6CV type=write size=2092382 2024-04-22T19:58:47.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.626+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~B6G35BJDGUOW.0 size=1048576 2024-04-22T19:58:47.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.627+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~B6G35BJDGUOW.1 size=1048576 2024-04-22T19:58:47.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.627+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~B6G35BJDGUOW.2 size=649637 2024-04-22T19:58:47.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.627+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=B6G35BJDGUOW tag=TCZP727HMQZL type=write size=3795365 2024-04-22T19:58:47.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.629+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~GYQE0YFQFJH6.0 size=444897 2024-04-22T19:58:47.630 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.630+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=GYQE0YFQFJH6 tag=AHUY5G835Q7D type=write size=1493473 2024-04-22T19:58:47.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.631+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=SCEFK7DLJ0BT type=write size=724069 2024-04-22T19:58:47.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.633+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~NY88YGQ2OKDM.0 size=1048576 2024-04-22T19:58:47.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.633+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~NY88YGQ2OKDM.1 size=1048576 2024-04-22T19:58:47.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.634+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~NY88YGQ2OKDM.2 size=209398 2024-04-22T19:58:47.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.636+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=NY88YGQ2OKDM tag=6OEIJ030QVW6 type=write size=3355126 2024-04-22T19:58:47.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.638+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~UH3T6611YEW2.0 size=812292 2024-04-22T19:58:47.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.638+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=UH3T6611YEW2 tag=QZDSFE4VME0D type=write size=1860868 2024-04-22T19:58:47.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.640+0000 7f5269a8e780 1 < suggested update operation key=5UBYMQT7ZWKX tag=1QWE4BUFLI6P type=del size=3117514 2024-04-22T19:58:47.642 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.642+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~CNI4HXTH1FIQ.0 size=431496 2024-04-22T19:58:47.642 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.642+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=CNI4HXTH1FIQ tag=AIJN063ZW1Y5 type=write size=1480072 2024-04-22T19:58:47.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.645+0000 7f5269a8e780 1 < suggested remove operation key=637M7AP8X9E9 tag=6FW7ABN7YU4U type=del 2024-04-22T19:58:47.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.646+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=A2Y4VJ2LFJV6 type=del size=4096348 2024-04-22T19:58:47.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.647+0000 7f5269a8e780 -1 < failed to complete operation key=5UBYMQT7ZWKX tag=RZ5RAZQ7OP9Y type=del size=3836396: (22) Invalid argument 2024-04-22T19:58:47.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.648+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=UNEWQ8SSUOTO type=del size=3781039 2024-04-22T19:58:47.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.649+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=M2X3FD74ZTFC type=del 2024-04-22T19:58:47.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.650+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=J42XQ7R1QNGG type=del size=69491 2024-04-22T19:58:47.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.651+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~NU7KI3ZUU7BR.0 size=1048576 2024-04-22T19:58:47.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.652+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~NU7KI3ZUU7BR.1 size=707105 2024-04-22T19:58:47.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.655+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=NU7KI3ZUU7BR tag=7XDIKLPBD9NZ type=write size=2804257 2024-04-22T19:58:47.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.656+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=3LLACE2AERMI type=write size=120930 2024-04-22T19:58:47.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.657+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=JWNIBLRQPAZU type=del 2024-04-22T19:58:47.659 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.659+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~NC9A091QTYM6.0 size=875463 2024-04-22T19:58:47.659 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.659+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=NC9A091QTYM6 tag=QRWBPEABM1HH type=write size=1924039 2024-04-22T19:58:47.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.661+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=3A3ZG0KQP454 type=del 2024-04-22T19:58:47.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.662+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=4YRZFDXJZEON type=del size=2132795 2024-04-22T19:58:47.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.663+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~ITHRTWL96Z2B.0 size=1048576 2024-04-22T19:58:47.664 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.664+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~ITHRTWL96Z2B.1 size=1048576 2024-04-22T19:58:47.665 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.665+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~ITHRTWL96Z2B.2 size=114530 2024-04-22T19:58:47.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.667+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=ITHRTWL96Z2B tag=PLM6F6B26JR9 type=write size=3260258 2024-04-22T19:58:47.668 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.668+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=DPMDFUHHHZM1 type=del size=123884 2024-04-22T19:58:47.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.669+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=8F6F9Y0E1MYS type=del 2024-04-22T19:58:47.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.670+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=H1MGER3A2EIO type=del size=715988 2024-04-22T19:58:47.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.671+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~4KSQWXXW3FFX.0 size=368094 2024-04-22T19:58:47.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.673+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=4KSQWXXW3FFX tag=XPNR27343OWN type=write size=1416670 2024-04-22T19:58:47.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.675+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~JJUVF7WBFG9V.0 size=1048576 2024-04-22T19:58:47.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.676+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~JJUVF7WBFG9V.1 size=674208 2024-04-22T19:58:47.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.676+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=JJUVF7WBFG9V tag=W2AZMXPU1WV3 type=write size=2771360 2024-04-22T19:58:47.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.677+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~AVLTOFSGW4RL.0 size=1048576 2024-04-22T19:58:47.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.678+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~AVLTOFSGW4RL.1 size=267825 2024-04-22T19:58:47.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.680+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=AVLTOFSGW4RL tag=8ZMJO2L8N23N type=write size=2364977 2024-04-22T19:58:47.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.681+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=MZG30CVVY9BQ type=write size=281590 2024-04-22T19:58:47.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.682+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~81Q39BTR79DV.0 size=1048576 2024-04-22T19:58:47.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.683+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~81Q39BTR79DV.1 size=1045335 2024-04-22T19:58:47.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.686+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=81Q39BTR79DV tag=6S0WPNI7BUWR type=write size=3142487 2024-04-22T19:58:47.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.687+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=09VKHZ2PHW1C type=del size=1119011 2024-04-22T19:58:47.690 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.689+0000 7f5269a8e780 1 < suggested remove operation key=Q1UKF4UMJ2HO tag=X0048BL5KLSS type=del 2024-04-22T19:58:47.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.691+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~FDEYIHRBB1DZ.0 size=1048576 2024-04-22T19:58:47.692 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.692+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~FDEYIHRBB1DZ.1 size=1048576 2024-04-22T19:58:47.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.693+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~FDEYIHRBB1DZ.2 size=26179 2024-04-22T19:58:47.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.693+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=FDEYIHRBB1DZ tag=GIK8QVFYSWTG type=write size=3171907 2024-04-22T19:58:47.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.694+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~B6G35BJDGUOW.0 size=1048576 2024-04-22T19:58:47.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.695+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~B6G35BJDGUOW.1 size=1048576 2024-04-22T19:58:47.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.696+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~B6G35BJDGUOW.2 size=649637 2024-04-22T19:58:47.698 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.698+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=B6G35BJDGUOW tag=TCZP727HMQZL type=write size=3795365 2024-04-22T19:58:47.699 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.699+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=57MJRYI0M04G type=del size=854365 2024-04-22T19:58:47.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.701+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=SCEFK7DLJ0BT type=write size=724069 2024-04-22T19:58:47.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.702+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=JG45KUWVW3VS type=write size=323874 2024-04-22T19:58:47.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.703+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~UH3T6611YEW2.0 size=812292 2024-04-22T19:58:47.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.705+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=UH3T6611YEW2 tag=QZDSFE4VME0D type=write size=1860868 2024-04-22T19:58:47.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.707+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~YZV4K0DASOX0.0 size=1048576 2024-04-22T19:58:47.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.708+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~YZV4K0DASOX0.1 size=1048576 2024-04-22T19:58:47.708 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.709+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~YZV4K0DASOX0.2 size=789935 2024-04-22T19:58:47.708 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.709+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=YZV4K0DASOX0 tag=MFSSQF3XO4MW type=write size=3935663 2024-04-22T19:58:47.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.711+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~CNI4HXTH1FIQ.0 size=431496 2024-04-22T19:58:47.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.715+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=CNI4HXTH1FIQ tag=AIJN063ZW1Y5 type=write size=1480072 2024-04-22T19:58:47.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.716+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=5TZ8C1KPNVNJ type=del size=3014043 2024-04-22T19:58:47.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.717+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=A2Y4VJ2LFJV6 type=del 2024-04-22T19:58:47.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.718+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=RGT9PXE00BOE type=del size=2204239 2024-04-22T19:58:47.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.719+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=UNEWQ8SSUOTO type=del 2024-04-22T19:58:47.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.721+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=1E71PXDDX94R type=del size=3468562 2024-04-22T19:58:47.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.722+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=J42XQ7R1QNGG type=del 2024-04-22T19:58:47.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.724+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~HLPPELYN04LT.0 size=1048576 2024-04-22T19:58:47.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.725+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~HLPPELYN04LT.1 size=164601 2024-04-22T19:58:47.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.725+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=HLPPELYN04LT tag=WO8345W6SE1U type=write size=2261753 2024-04-22T19:58:47.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.726+0000 7f5269a8e780 1 < completed write operation key=86GW6HV9G1IZ tag=3LLACE2AERMI type=write size=120930 2024-04-22T19:58:47.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.727+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=VQ62TT6Q6PVY type=del size=1125182 2024-04-22T19:58:47.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.729+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~NC9A091QTYM6.0 size=875463 2024-04-22T19:58:47.730 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.731+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=NC9A091QTYM6 tag=QRWBPEABM1HH type=write size=1924039 2024-04-22T19:58:47.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.732+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=T4XFFIZHSUTW type=del size=750896 2024-04-22T19:58:47.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.733+0000 7f5269a8e780 1 < canceled operation key=8HFSDSNZONIN tag=4YRZFDXJZEON type=del size=2132795 2024-04-22T19:58:47.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.734+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=PTY1WLDQRNYW type=del size=252619 2024-04-22T19:58:47.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.736+0000 7f5269a8e780 1 < suggested remove operation key=8HFSDSNZONIN tag=DPMDFUHHHZM1 type=del 2024-04-22T19:58:47.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.737+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=ZML0Z647SY04 type=del size=525987 2024-04-22T19:58:47.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.738+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=H1MGER3A2EIO type=del 2024-04-22T19:58:47.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.739+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=N22312VRVW31 type=del size=2641624 2024-04-22T19:58:47.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.741+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~JJUVF7WBFG9V.0 size=1048576 2024-04-22T19:58:47.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.742+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~JJUVF7WBFG9V.1 size=674208 2024-04-22T19:58:47.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.743+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=JJUVF7WBFG9V tag=W2AZMXPU1WV3 type=write size=2771360 2024-04-22T19:58:47.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.746+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~B38E22426PDH.0 size=918876 2024-04-22T19:58:47.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.746+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=B38E22426PDH tag=6687WPUG2AAK type=write size=1967452 2024-04-22T19:58:47.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.747+0000 7f5269a8e780 1 < completed write operation key=HV8J6YM0YNC1 tag=MZG30CVVY9BQ type=write size=281590 2024-04-22T19:58:47.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.748+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=V771ZDES0N2C type=write size=996174 2024-04-22T19:58:47.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.749+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=09VKHZ2PHW1C type=del 2024-04-22T19:58:47.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.750+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=4VTA1405MNVL type=del size=1252603 2024-04-22T19:58:47.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.752+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~FDEYIHRBB1DZ.0 size=1048576 2024-04-22T19:58:47.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.753+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~FDEYIHRBB1DZ.1 size=1048576 2024-04-22T19:58:47.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.754+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~FDEYIHRBB1DZ.2 size=26179 2024-04-22T19:58:47.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.756+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=FDEYIHRBB1DZ tag=GIK8QVFYSWTG type=write size=3171907 2024-04-22T19:58:47.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.758+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~INEOE1AG5HSJ.0 size=1048576 2024-04-22T19:58:47.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.759+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~INEOE1AG5HSJ.1 size=1048576 2024-04-22T19:58:47.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.760+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~INEOE1AG5HSJ.2 size=397958 2024-04-22T19:58:47.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.760+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=INEOE1AG5HSJ tag=LR3O2UUFVTGK type=write size=3543686 2024-04-22T19:58:47.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.762+0000 7f5269a8e780 1 < suggested update operation key=Q1UKF4UMJ2HO tag=57MJRYI0M04G type=del size=854365 2024-04-22T19:58:47.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.763+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=70Y8ZCZFC1EV type=del size=3316336 2024-04-22T19:58:47.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.764+0000 7f5269a8e780 1 < completed write operation key=O9NFAVE09ZN7 tag=JG45KUWVW3VS type=write size=323874 2024-04-22T19:58:47.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.766+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~T5CKOQ6YU8KH.0 size=1048576 2024-04-22T19:58:47.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.767+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~T5CKOQ6YU8KH.1 size=379827 2024-04-22T19:58:47.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.767+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=T5CKOQ6YU8KH tag=16S0ZFAMEPEZ type=write size=2476979 2024-04-22T19:58:47.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.768+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~YZV4K0DASOX0.0 size=1048576 2024-04-22T19:58:47.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.769+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~YZV4K0DASOX0.1 size=1048576 2024-04-22T19:58:47.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.769+0000 7f5269a8e780 1 < canceled part key=_multipart_VPAVX5FXYWZO.2~YZV4K0DASOX0.2 size=789935 2024-04-22T19:58:47.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.771+0000 7f5269a8e780 1 < canceled multipart upload key=VPAVX5FXYWZO upload=YZV4K0DASOX0 tag=MFSSQF3XO4MW type=write size=3935663 2024-04-22T19:58:47.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.772+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=8WMLR1PNDV3B type=del size=353046 2024-04-22T19:58:47.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.774+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=5TZ8C1KPNVNJ type=del 2024-04-22T19:58:47.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.775+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=W0PDUGQN1SKP type=del size=1997989 2024-04-22T19:58:47.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.776+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=RGT9PXE00BOE type=del 2024-04-22T19:58:47.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.778+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~WKTPPPJK7A6A.0 size=1048576 2024-04-22T19:58:47.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.779+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~WKTPPPJK7A6A.1 size=595854 2024-04-22T19:58:47.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.779+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=WKTPPPJK7A6A tag=ANNCX0EUME5S type=write size=2693006 2024-04-22T19:58:47.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.780+0000 7f5269a8e780 1 < canceled operation key=2YFD2U6HVNE7 tag=1E71PXDDX94R type=del size=3468562 2024-04-22T19:58:47.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.781+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=VGHIQY13IMRV type=del size=2935554 2024-04-22T19:58:47.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.781+0000 7f5269a8e780 1 < canceled part key=_multipart_B0Y7ZTPC0975.2~HLPPELYN04LT.0 size=1048576 2024-04-22T19:58:47.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.781+0000 7f5269a8e780 1 < canceled part key=_multipart_B0Y7ZTPC0975.2~HLPPELYN04LT.1 size=164601 2024-04-22T19:58:47.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.783+0000 7f5269a8e780 1 < canceled multipart upload key=B0Y7ZTPC0975 upload=HLPPELYN04LT tag=WO8345W6SE1U type=write size=2261753 2024-04-22T19:58:47.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.785+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~CCX4GYASRZML.0 size=1048576 2024-04-22T19:58:47.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.785+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~CCX4GYASRZML.1 size=673258 2024-04-22T19:58:47.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.785+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=CCX4GYASRZML tag=BTATGFJ42QUE type=write size=2770410 2024-04-22T19:58:47.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.787+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=VQ62TT6Q6PVY type=del 2024-04-22T19:58:47.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.788+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=CQ7SOQNYQ3XA type=del size=1358296 2024-04-22T19:58:47.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.790+0000 7f5269a8e780 1 < suggested update operation key=HIAXPVKZ1IIO tag=T4XFFIZHSUTW type=del size=750896 2024-04-22T19:58:47.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.791+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=QC7LV7VZ0V4I type=write size=548428 2024-04-22T19:58:47.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.792+0000 7f5269a8e780 -1 < failed to complete operation key=HIAXPVKZ1IIO tag=PTY1WLDQRNYW type=del size=252619: (22) Invalid argument 2024-04-22T19:58:47.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.793+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=EH7SLUZ13UMS type=write size=318706 2024-04-22T19:58:47.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.795+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=ZML0Z647SY04 type=del 2024-04-22T19:58:47.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.796+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=STPT7EES3OYW type=del size=247142 2024-04-22T19:58:47.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.797+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=N22312VRVW31 type=del 2024-04-22T19:58:47.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.798+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=89VHORCOWSQ6 type=del size=445415 2024-04-22T19:58:47.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.800+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~B38E22426PDH.0 size=918876 2024-04-22T19:58:47.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.802+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=B38E22426PDH tag=6687WPUG2AAK type=write size=1967452 2024-04-22T19:58:47.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.803+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=76T4SKAU4A93 type=del size=743868 2024-04-22T19:58:47.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.804+0000 7f5269a8e780 1 < completed write operation key=M9M4SEEEZSFV tag=V771ZDES0N2C type=write size=996174 2024-04-22T19:58:47.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.805+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=FXX86RSBX41A type=del size=501692 2024-04-22T19:58:47.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.806+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=4VTA1405MNVL type=del 2024-04-22T19:58:47.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.808+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=SYVDMC9OIWCV type=del size=320602 2024-04-22T19:58:47.809 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.809+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~INEOE1AG5HSJ.0 size=1048576 2024-04-22T19:58:47.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.810+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~INEOE1AG5HSJ.1 size=1048576 2024-04-22T19:58:47.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.811+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~INEOE1AG5HSJ.2 size=397958 2024-04-22T19:58:47.813 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.813+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=INEOE1AG5HSJ tag=LR3O2UUFVTGK type=write size=3543686 2024-04-22T19:58:47.814 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.814+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=NU0YNHUDGAM4 type=del size=1865408 2024-04-22T19:58:47.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.816+0000 7f5269a8e780 1 < suggested remove operation key=637M7AP8X9E9 tag=70Y8ZCZFC1EV type=del 2024-04-22T19:58:47.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.817+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=SI39OMO2O9ZE type=del size=2322899 2024-04-22T19:58:47.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.818+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~T5CKOQ6YU8KH.0 size=1048576 2024-04-22T19:58:47.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.819+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~T5CKOQ6YU8KH.1 size=379827 2024-04-22T19:58:47.823 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.823+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=T5CKOQ6YU8KH tag=16S0ZFAMEPEZ type=write size=2476979 2024-04-22T19:58:47.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.824+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=AZEU2C75MZKL type=del size=2229971 2024-04-22T19:58:47.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.827+0000 7f5269a8e780 1 < suggested remove operation key=DX5YUYQRZKWA tag=8WMLR1PNDV3B type=del 2024-04-22T19:58:47.828 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.828+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=TUMMBI33VV8R type=del size=3554722 2024-04-22T19:58:47.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.829+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=W0PDUGQN1SKP type=del 2024-04-22T19:58:47.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.831+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~FPPQ9YZOT632.0 size=1048576 2024-04-22T19:58:47.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.832+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~FPPQ9YZOT632.1 size=7875 2024-04-22T19:58:47.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.832+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=FPPQ9YZOT632 tag=VVH0ILSQ1FUD type=write size=2105027 2024-04-22T19:58:47.833 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.833+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~WKTPPPJK7A6A.0 size=1048576 2024-04-22T19:58:47.834 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.834+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~WKTPPPJK7A6A.1 size=595854 2024-04-22T19:58:47.836 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.836+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=WKTPPPJK7A6A tag=ANNCX0EUME5S type=write size=2693006 2024-04-22T19:58:47.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.837+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=DGL11HF5T6GU type=write size=1039538 2024-04-22T19:58:47.838 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.838+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=VGHIQY13IMRV type=del 2024-04-22T19:58:47.840 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.840+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~0ZILOCACK1YR.0 size=1048576 2024-04-22T19:58:47.841 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.841+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~0ZILOCACK1YR.1 size=587604 2024-04-22T19:58:47.841 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.841+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=0ZILOCACK1YR tag=ZIPLK7D2KY59 type=write size=2684756 2024-04-22T19:58:47.841 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.841+0000 7f5269a8e780 1 < canceled part key=_multipart_SN2BZATJGUWA.2~CCX4GYASRZML.0 size=1048576 2024-04-22T19:58:47.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.842+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~CCX4GYASRZML.1 size=673258 2024-04-22T19:58:47.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.844+0000 7f5269a8e780 1 < canceled multipart upload key=SN2BZATJGUWA upload=CCX4GYASRZML tag=BTATGFJ42QUE type=write size=2770410 2024-04-22T19:58:47.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.845+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=PPVCBTGUW3IW type=del size=1728061 2024-04-22T19:58:47.846 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.846+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=CQ7SOQNYQ3XA type=del 2024-04-22T19:58:47.847 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.847+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=I9N4CNLK36BU type=del size=1871204 2024-04-22T19:58:47.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.849+0000 7f5269a8e780 1 < completed write operation key=Q1UKF4UMJ2HO tag=QC7LV7VZ0V4I type=write size=548428 2024-04-22T19:58:47.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.850+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=N3SJZYNO08A3 type=del size=390149 2024-04-22T19:58:47.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.851+0000 7f5269a8e780 1 < completed write operation key=8V6ESIGMRO5G tag=EH7SLUZ13UMS type=write size=318706 2024-04-22T19:58:47.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.853+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~LO2395HDEM3X.0 size=1048576 2024-04-22T19:58:47.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.854+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~LO2395HDEM3X.1 size=1048576 2024-04-22T19:58:47.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.855+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~LO2395HDEM3X.2 size=639324 2024-04-22T19:58:47.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.855+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=LO2395HDEM3X tag=98EIB05QHKD6 type=write size=3785052 2024-04-22T19:58:47.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.856+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=STPT7EES3OYW type=del 2024-04-22T19:58:47.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.857+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=LN8Y2LEUPYLE type=del size=3131291 2024-04-22T19:58:47.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.858+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=89VHORCOWSQ6 type=del 2024-04-22T19:58:47.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.860+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~GXXXTMBN0BD7.0 size=855173 2024-04-22T19:58:47.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.860+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=GXXXTMBN0BD7 tag=TTZ7F7P6TRP0 type=write size=1903749 2024-04-22T19:58:47.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.861+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=76T4SKAU4A93 type=del 2024-04-22T19:58:47.863 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.863+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=7TQZF8OJ6STQ type=del size=3501862 2024-04-22T19:58:47.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.864+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=FXX86RSBX41A type=del 2024-04-22T19:58:47.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.865+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=Y9EI7BVJ6J78 type=del size=2003414 2024-04-22T19:58:47.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.868+0000 7f5269a8e780 1 < suggested remove operation key=B0Y7ZTPC0975 tag=SYVDMC9OIWCV type=del 2024-04-22T19:58:47.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.869+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=CNA2S566ZCUE type=del size=4090316 2024-04-22T19:58:47.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.870+0000 7f5269a8e780 -1 < no bi entry to suggest for operation key=DX5YUYQRZKWA tag=NU0YNHUDGAM4 type=del size=1865408: (2) No such file or directory 2024-04-22T19:58:47.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.871+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=CQ7Z3MZGW80V type=del size=3552155 2024-04-22T19:58:47.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.872+0000 7f5269a8e780 1 < canceled operation key=T2BV2LUCY4E3 tag=SI39OMO2O9ZE type=del size=2322899 2024-04-22T19:58:47.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.875+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~VU092D6TGRMB.0 size=1048576 2024-04-22T19:58:47.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.876+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~VU092D6TGRMB.1 size=628300 2024-04-22T19:58:47.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.876+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=VU092D6TGRMB tag=W8J8T2R155II type=write size=2725452 2024-04-22T19:58:47.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.877+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=AZEU2C75MZKL type=del 2024-04-22T19:58:47.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.878+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=8AQEM2A28BS4 type=del size=3626788 2024-04-22T19:58:47.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.879+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=TUMMBI33VV8R type=del 2024-04-22T19:58:47.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.881+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~CSD2TJM6L9LY.0 size=729537 2024-04-22T19:58:47.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.881+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=CSD2TJM6L9LY tag=NEF7SI9U87FM type=write size=1778113 2024-04-22T19:58:47.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.883+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~FPPQ9YZOT632.0 size=1048576 2024-04-22T19:58:47.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.883+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~FPPQ9YZOT632.1 size=7875 2024-04-22T19:58:47.885 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.885+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=FPPQ9YZOT632 tag=VVH0ILSQ1FUD type=write size=2105027 2024-04-22T19:58:47.886 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.886+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=VH840RDODW0A type=del size=3917786 2024-04-22T19:58:47.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.889+0000 7f5269a8e780 1 < suggested remove operation key=KV5DYQ49ZKCX tag=DGL11HF5T6GU type=write 2024-04-22T19:58:47.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.891+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~5WOBG3A3XDVK.0 size=1048576 2024-04-22T19:58:47.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.892+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~5WOBG3A3XDVK.1 size=406474 2024-04-22T19:58:47.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.892+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=5WOBG3A3XDVK tag=2GMS7D2PH9ZD type=write size=2503626 2024-04-22T19:58:47.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.893+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~0ZILOCACK1YR.0 size=1048576 2024-04-22T19:58:47.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.894+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~0ZILOCACK1YR.1 size=587604 2024-04-22T19:58:47.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.896+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=0ZILOCACK1YR tag=ZIPLK7D2KY59 type=write size=2684756 2024-04-22T19:58:47.897 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.898+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~GHUIH9IS705W.0 size=1048576 2024-04-22T19:58:47.898 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.899+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~GHUIH9IS705W.1 size=904450 2024-04-22T19:58:47.899 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.899+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=GHUIH9IS705W tag=4LV0QWPXU12X type=write size=3001602 2024-04-22T19:58:47.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.901+0000 7f5269a8e780 1 < suggested remove operation key=IBH6W1NINUHC tag=PPVCBTGUW3IW type=del 2024-04-22T19:58:47.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.903+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~3IW3N4KH0L3J.0 size=1048576 2024-04-22T19:58:47.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.904+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~3IW3N4KH0L3J.1 size=345529 2024-04-22T19:58:47.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.904+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=3IW3N4KH0L3J tag=HRR1TC0XE4ZU type=write size=2442681 2024-04-22T19:58:47.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.905+0000 7f5269a8e780 -1 < failed to complete operation key=KV5DYQ49ZKCX tag=I9N4CNLK36BU type=del size=1871204: (22) Invalid argument 2024-04-22T19:58:47.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.907+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~TVINJ8PD3KW8.0 size=1048576 2024-04-22T19:58:47.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.908+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~TVINJ8PD3KW8.1 size=1048576 2024-04-22T19:58:47.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.908+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~TVINJ8PD3KW8.2 size=653041 2024-04-22T19:58:47.909 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.908+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=TVINJ8PD3KW8 tag=IK2S9APC0UR2 type=write size=3798769 2024-04-22T19:58:47.909 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.910+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=N3SJZYNO08A3 type=del 2024-04-22T19:58:47.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.911+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=CKP9KZ0NVGJF type=del size=638243 2024-04-22T19:58:47.912 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.912+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~LO2395HDEM3X.0 size=1048576 2024-04-22T19:58:47.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.913+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~LO2395HDEM3X.1 size=1048576 2024-04-22T19:58:47.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.913+0000 7f5269a8e780 1 < canceled part key=_multipart_T2BV2LUCY4E3.2~LO2395HDEM3X.2 size=639324 2024-04-22T19:58:47.915 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.915+0000 7f5269a8e780 1 < canceled multipart upload key=T2BV2LUCY4E3 upload=LO2395HDEM3X tag=98EIB05QHKD6 type=write size=3785052 2024-04-22T19:58:47.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.916+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=63FYHTO06QBM type=del size=1820834 2024-04-22T19:58:47.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.917+0000 7f5269a8e780 1 < canceled operation key=ZC4ZOHUJ3HQD tag=LN8Y2LEUPYLE type=del size=3131291 2024-04-22T19:58:47.918 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.919+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=NG40Y9YZAU8Z type=del size=2687822 2024-04-22T19:58:47.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.920+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~GXXXTMBN0BD7.0 size=855173 2024-04-22T19:58:47.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.922+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=GXXXTMBN0BD7 tag=TTZ7F7P6TRP0 type=write size=1903749 2024-04-22T19:58:47.923 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.923+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=4FW9R54CEDTJ type=del size=3233831 2024-04-22T19:58:47.924 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.924+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=7TQZF8OJ6STQ type=del 2024-04-22T19:58:47.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.925+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=6FJFFL13EOA4 type=del size=742877 2024-04-22T19:58:47.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.926+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=Y9EI7BVJ6J78 type=del 2024-04-22T19:58:47.928 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.928+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~FW0VVMK1MY3O.0 size=1048576 2024-04-22T19:58:47.929 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.929+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~FW0VVMK1MY3O.1 size=1048576 2024-04-22T19:58:47.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.930+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~FW0VVMK1MY3O.2 size=741589 2024-04-22T19:58:47.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.930+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=FW0VVMK1MY3O tag=IQDC61NGYXFB type=write size=3887317 2024-04-22T19:58:47.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.933+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=CNA2S566ZCUE type=del 2024-04-22T19:58:47.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.934+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=4ORZIQ5X30CZ type=write size=151139 2024-04-22T19:58:47.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.936+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=CQ7Z3MZGW80V type=del 2024-04-22T19:58:47.937 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.938+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=P8R8FUGHZEOY type=del size=2920720 2024-04-22T19:58:47.939 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.939+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~VU092D6TGRMB.0 size=1048576 2024-04-22T19:58:47.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.940+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~VU092D6TGRMB.1 size=628300 2024-04-22T19:58:47.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.942+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=VU092D6TGRMB tag=W8J8T2R155II type=write size=2725452 2024-04-22T19:58:47.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.944+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~CNKW68E3COIR.0 size=97320 2024-04-22T19:58:47.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.944+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=CNKW68E3COIR tag=AE3IHYBQJY5T type=write size=1145896 2024-04-22T19:58:47.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.945+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=8AQEM2A28BS4 type=del 2024-04-22T19:58:47.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.946+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=4G2BGU2B58ZD type=del size=3940832 2024-04-22T19:58:47.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.947+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~CSD2TJM6L9LY.0 size=729537 2024-04-22T19:58:47.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.949+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=CSD2TJM6L9LY tag=NEF7SI9U87FM type=write size=1778113 2024-04-22T19:58:47.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.951+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~IYRGO3I0FS32.0 size=1048576 2024-04-22T19:58:47.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.952+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~IYRGO3I0FS32.1 size=476629 2024-04-22T19:58:47.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.952+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=IYRGO3I0FS32 tag=DQN5Z4HANLX6 type=write size=2573781 2024-04-22T19:58:47.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.954+0000 7f5269a8e780 1 < suggested remove operation key=70MLQGJPQYPY tag=VH840RDODW0A type=del 2024-04-22T19:58:47.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.955+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=DJ0PIZD8L34Q type=write size=698714 2024-04-22T19:58:47.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.956+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~5WOBG3A3XDVK.0 size=1048576 2024-04-22T19:58:47.957 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.957+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~5WOBG3A3XDVK.1 size=406474 2024-04-22T19:58:47.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.959+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=5WOBG3A3XDVK tag=2GMS7D2PH9ZD type=write size=2503626 2024-04-22T19:58:47.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.960+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=AJCHWSQZPNWC type=del size=2665922 2024-04-22T19:58:47.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.961+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~GHUIH9IS705W.0 size=1048576 2024-04-22T19:58:47.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.961+0000 7f5269a8e780 1 < canceled part key=_multipart_2YFD2U6HVNE7.2~GHUIH9IS705W.1 size=904450 2024-04-22T19:58:47.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.963+0000 7f5269a8e780 1 < canceled multipart upload key=2YFD2U6HVNE7 upload=GHUIH9IS705W tag=4LV0QWPXU12X type=write size=3001602 2024-04-22T19:58:47.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.965+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~RD40ZMP6X0OA.0 size=1048576 2024-04-22T19:58:47.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.966+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~RD40ZMP6X0OA.1 size=1048576 2024-04-22T19:58:47.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.967+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~RD40ZMP6X0OA.2 size=31803 2024-04-22T19:58:47.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.967+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=RD40ZMP6X0OA tag=X35PSNPT9J75 type=write size=3177531 2024-04-22T19:58:47.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.969+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~3IW3N4KH0L3J.0 size=1048576 2024-04-22T19:58:47.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.970+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~3IW3N4KH0L3J.1 size=345529 2024-04-22T19:58:47.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.971+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=3IW3N4KH0L3J tag=HRR1TC0XE4ZU type=write size=2442681 2024-04-22T19:58:47.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.972+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=QAB32U0C99FP type=del size=1916991 2024-04-22T19:58:47.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.974+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~TVINJ8PD3KW8.0 size=1048576 2024-04-22T19:58:47.974 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.975+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~TVINJ8PD3KW8.1 size=1048576 2024-04-22T19:58:47.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.975+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~TVINJ8PD3KW8.2 size=653041 2024-04-22T19:58:47.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.977+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=TVINJ8PD3KW8 tag=IK2S9APC0UR2 type=write size=3798769 2024-04-22T19:58:47.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.979+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=TK23W9R6LNN2 type=write size=474673 2024-04-22T19:58:47.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.980+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=CKP9KZ0NVGJF type=del 2024-04-22T19:58:47.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.981+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=C0BN41KSEA4I type=del size=341408 2024-04-22T19:58:47.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.982+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=63FYHTO06QBM type=del 2024-04-22T19:58:47.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.983+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=WA322VLMU20H type=del size=1918326 2024-04-22T19:58:47.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.984+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=NG40Y9YZAU8Z type=del 2024-04-22T19:58:47.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.985+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=N3YA4TOKNQCV type=write size=643693 2024-04-22T19:58:47.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.986+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=4FW9R54CEDTJ type=del 2024-04-22T19:58:47.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.988+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~EXYDOM8ZE07A.0 size=827924 2024-04-22T19:58:47.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.988+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=EXYDOM8ZE07A tag=RCX6PZOF23G7 type=write size=1876500 2024-04-22T19:58:47.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.990+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=6FJFFL13EOA4 type=del 2024-04-22T19:58:47.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.991+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=2B6G68MHMX7J type=del size=2448866 2024-04-22T19:58:47.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.992+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~FW0VVMK1MY3O.0 size=1048576 2024-04-22T19:58:47.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.992+0000 7f5269a8e780 1 < canceled part key=_multipart_637M7AP8X9E9.2~FW0VVMK1MY3O.1 size=1048576 2024-04-22T19:58:47.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.993+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~FW0VVMK1MY3O.2 size=741589 2024-04-22T19:58:47.994 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.994+0000 7f5269a8e780 1 < canceled multipart upload key=637M7AP8X9E9 upload=FW0VVMK1MY3O tag=IQDC61NGYXFB type=write size=3887317 2024-04-22T19:58:47.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.995+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=H5BZ7KBWPJ3T type=del size=1749258 2024-04-22T19:58:47.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.997+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=4ORZIQ5X30CZ type=write size=151139 2024-04-22T19:58:47.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.998+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~7EGD7H0PYCZ3.0 size=482793 2024-04-22T19:58:47.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.998+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=7EGD7H0PYCZ3 tag=M5E15DACXINS type=write size=1531369 2024-04-22T19:58:47.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:47.999+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=P8R8FUGHZEOY type=del 2024-04-22T19:58:48.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.001+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=BFDB8FSSQ9GP type=del size=3610390 2024-04-22T19:58:48.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.002+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~CNKW68E3COIR.0 size=97320 2024-04-22T19:58:48.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.003+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=CNKW68E3COIR tag=AE3IHYBQJY5T type=write size=1145896 2024-04-22T19:58:48.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.005+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~52I0KPDSWSD7.0 size=72251 2024-04-22T19:58:48.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.005+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=52I0KPDSWSD7 tag=S0BIBLDUK1AH type=write size=1120827 2024-04-22T19:58:48.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.007+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=4G2BGU2B58ZD type=del 2024-04-22T19:58:48.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.009+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~YF53OYSKIDO9.0 size=1048576 2024-04-22T19:58:48.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.010+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~YF53OYSKIDO9.1 size=1048576 2024-04-22T19:58:48.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.011+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~YF53OYSKIDO9.2 size=278559 2024-04-22T19:58:48.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.011+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=YF53OYSKIDO9 tag=BIXLWLC6V4UX type=write size=3424287 2024-04-22T19:58:48.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.012+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~IYRGO3I0FS32.0 size=1048576 2024-04-22T19:58:48.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.013+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~IYRGO3I0FS32.1 size=476629 2024-04-22T19:58:48.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.015+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=IYRGO3I0FS32 tag=DQN5Z4HANLX6 type=write size=2573781 2024-04-22T19:58:48.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.018+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~DAAHFPJXAP5S.0 size=1048576 2024-04-22T19:58:48.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.018+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~DAAHFPJXAP5S.1 size=1048576 2024-04-22T19:58:48.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.019+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~DAAHFPJXAP5S.2 size=994169 2024-04-22T19:58:48.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.019+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=DAAHFPJXAP5S tag=JNCGTFTMWTBU type=write size=4139897 2024-04-22T19:58:48.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.021+0000 7f5269a8e780 1 < completed write operation key=O9NFAVE09ZN7 tag=DJ0PIZD8L34Q type=write size=698714 2024-04-22T19:58:48.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.022+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=1A2QHT47VYN9 type=write size=823835 2024-04-22T19:58:48.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.023+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=AJCHWSQZPNWC type=del 2024-04-22T19:58:48.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.024+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=JWB9NEW2Q2PC type=write size=857300 2024-04-22T19:58:48.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.025+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~RD40ZMP6X0OA.0 size=1048576 2024-04-22T19:58:48.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.026+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~RD40ZMP6X0OA.1 size=1048576 2024-04-22T19:58:48.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.027+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~RD40ZMP6X0OA.2 size=31803 2024-04-22T19:58:48.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.029+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=RD40ZMP6X0OA tag=X35PSNPT9J75 type=write size=3177531 2024-04-22T19:58:48.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.030+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=R1ILOJO275BT type=del size=1008339 2024-04-22T19:58:48.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.032+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=QAB32U0C99FP type=del 2024-04-22T19:58:48.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.034+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~V88DU96P6RDV.0 size=531460 2024-04-22T19:58:48.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.034+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=V88DU96P6RDV tag=CZX4MVCOD5G0 type=write size=1580036 2024-04-22T19:58:48.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.035+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=TK23W9R6LNN2 type=write size=474673 2024-04-22T19:58:48.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.037+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=YPDOVC3ZOJUB type=del size=1569473 2024-04-22T19:58:48.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.038+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=C0BN41KSEA4I type=del 2024-04-22T19:58:48.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.039+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=X4IGR1H3KO9I type=write size=1007971 2024-04-22T19:58:48.043 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.043+0000 7f5269a8e780 1 < suggested update operation key=KE7YGPTF1MOR tag=WA322VLMU20H type=del size=1918326 2024-04-22T19:58:48.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.045+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=RL49DHYE07TR type=del size=2504777 2024-04-22T19:58:48.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.048+0000 7f5269a8e780 1 < suggested remove operation key=2YFD2U6HVNE7 tag=N3YA4TOKNQCV type=write 2024-04-22T19:58:48.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.049+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=VE8U97WMNIKW type=del size=3223620 2024-04-22T19:58:48.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.050+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~EXYDOM8ZE07A.0 size=827924 2024-04-22T19:58:48.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.052+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=EXYDOM8ZE07A tag=RCX6PZOF23G7 type=write size=1876500 2024-04-22T19:58:48.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.053+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=QV2MD0J0YK0C type=del size=3347149 2024-04-22T19:58:48.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.055+0000 7f5269a8e780 1 < suggested update operation key=5UBYMQT7ZWKX tag=2B6G68MHMX7J type=del size=2448866 2024-04-22T19:58:48.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.056+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=H7EEAS2CCO8L type=del size=1740429 2024-04-22T19:58:48.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.058+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=H5BZ7KBWPJ3T type=del 2024-04-22T19:58:48.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.059+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=64868IOV3INT type=del size=4012911 2024-04-22T19:58:48.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.060+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~7EGD7H0PYCZ3.0 size=482793 2024-04-22T19:58:48.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.062+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=7EGD7H0PYCZ3 tag=M5E15DACXINS type=write size=1531369 2024-04-22T19:58:48.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.063+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=LO0GHZ2A8Q3T type=del size=1046851 2024-04-22T19:58:48.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.064+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=BFDB8FSSQ9GP type=del 2024-04-22T19:58:48.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.065+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=WVOFWVLDWNE3 type=del size=3482633 2024-04-22T19:58:48.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.065+0000 7f5269a8e780 1 < canceled part key=_multipart_86GW6HV9G1IZ.2~52I0KPDSWSD7.0 size=72251 2024-04-22T19:58:48.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.067+0000 7f5269a8e780 1 < canceled multipart upload key=86GW6HV9G1IZ upload=52I0KPDSWSD7 tag=S0BIBLDUK1AH type=write size=1120827 2024-04-22T19:58:48.069 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.068+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=TAKACO1QUSSI type=del size=72001 2024-04-22T19:58:48.069 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.069+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~YF53OYSKIDO9.0 size=1048576 2024-04-22T19:58:48.070 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.070+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~YF53OYSKIDO9.1 size=1048576 2024-04-22T19:58:48.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.071+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~YF53OYSKIDO9.2 size=278559 2024-04-22T19:58:48.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.073+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=YF53OYSKIDO9 tag=BIXLWLC6V4UX type=write size=3424287 2024-04-22T19:58:48.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.075+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~GAT3TU6UUH18.0 size=1048576 2024-04-22T19:58:48.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.076+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~GAT3TU6UUH18.1 size=1048576 2024-04-22T19:58:48.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.077+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~GAT3TU6UUH18.2 size=301261 2024-04-22T19:58:48.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.077+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=GAT3TU6UUH18 tag=JT358IF1U4NR type=write size=3446989 2024-04-22T19:58:48.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.078+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~DAAHFPJXAP5S.0 size=1048576 2024-04-22T19:58:48.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.079+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~DAAHFPJXAP5S.1 size=1048576 2024-04-22T19:58:48.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.080+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~DAAHFPJXAP5S.2 size=994169 2024-04-22T19:58:48.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.082+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=DAAHFPJXAP5S tag=JNCGTFTMWTBU type=write size=4139897 2024-04-22T19:58:48.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.084+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~O9ANZUOEX51I.0 size=218103 2024-04-22T19:58:48.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.084+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=O9ANZUOEX51I tag=CH58CYG7DJKL type=write size=1266679 2024-04-22T19:58:48.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.085+0000 7f5269a8e780 1 < completed write operation key=VPAVX5FXYWZO tag=1A2QHT47VYN9 type=write size=823835 2024-04-22T19:58:48.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.086+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=ZUTK4GYZERYQ type=write size=965626 2024-04-22T19:58:48.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.087+0000 7f5269a8e780 1 < completed write operation key=O9NFAVE09ZN7 tag=JWB9NEW2Q2PC type=write size=857300 2024-04-22T19:58:48.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.089+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~JLSEXUHGQTU4.0 size=412128 2024-04-22T19:58:48.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.089+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=JLSEXUHGQTU4 tag=XKG5FP500675 type=write size=1460704 2024-04-22T19:58:48.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.091+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=R1ILOJO275BT type=del 2024-04-22T19:58:48.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.092+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=MQNJR25QGOFJ type=del size=2346121 2024-04-22T19:58:48.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.093+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~V88DU96P6RDV.0 size=531460 2024-04-22T19:58:48.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.095+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=V88DU96P6RDV tag=CZX4MVCOD5G0 type=write size=1580036 2024-04-22T19:58:48.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.097+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~N0WGH8SU8QTV.0 size=862885 2024-04-22T19:58:48.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.097+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=N0WGH8SU8QTV tag=66F18LFRXMVT type=write size=1911461 2024-04-22T19:58:48.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.098+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=YPDOVC3ZOJUB type=del 2024-04-22T19:58:48.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.100+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~AH6XK8B4AWIY.0 size=860354 2024-04-22T19:58:48.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.100+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=AH6XK8B4AWIY tag=OTSR1PSEJLMY type=write size=1908930 2024-04-22T19:58:48.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.101+0000 7f5269a8e780 1 < completed write operation key=B0Y7ZTPC0975 tag=X4IGR1H3KO9I type=write size=1007971 2024-04-22T19:58:48.103 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.103+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~BHFC2XMW5O5Z.0 size=1048576 2024-04-22T19:58:48.104 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.104+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~BHFC2XMW5O5Z.1 size=895070 2024-04-22T19:58:48.104 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.104+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=BHFC2XMW5O5Z tag=DWOMRVWYF1EI type=write size=2992222 2024-04-22T19:58:48.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.105+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=RL49DHYE07TR type=del 2024-04-22T19:58:48.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.106+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=XQE236RQTN2W type=del size=3988377 2024-04-22T19:58:48.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.108+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=VE8U97WMNIKW type=del 2024-04-22T19:58:48.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.109+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=YA94O8C3UCCU type=write size=521427 2024-04-22T19:58:48.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.110+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=QV2MD0J0YK0C type=del 2024-04-22T19:58:48.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.111+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=6J57FFBVC2NE type=del size=1551645 2024-04-22T19:58:48.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.112+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=H7EEAS2CCO8L type=del 2024-04-22T19:58:48.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.113+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=5KR1GC6DR3HY type=del size=2894219 2024-04-22T19:58:48.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.114+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=64868IOV3INT type=del 2024-04-22T19:58:48.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.116+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=GJNW76THKA8M type=del size=56275 2024-04-22T19:58:48.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.117+0000 7f5269a8e780 1 < canceled operation key=4GX83ZJWZCD1 tag=LO0GHZ2A8Q3T type=del size=1046851 2024-04-22T19:58:48.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.119+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~OCC07QD7A9ZF.0 size=1048576 2024-04-22T19:58:48.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.120+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~OCC07QD7A9ZF.1 size=803660 2024-04-22T19:58:48.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.120+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=OCC07QD7A9ZF tag=N7832B2PNKA2 type=write size=2900812 2024-04-22T19:58:48.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.122+0000 7f5269a8e780 1 < suggested update operation key=2YFD2U6HVNE7 tag=WVOFWVLDWNE3 type=del size=3482633 2024-04-22T19:58:48.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.123+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=A0AAA1F3SU1Y type=del size=3954128 2024-04-22T19:58:48.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.124+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=TAKACO1QUSSI type=del 2024-04-22T19:58:48.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.125+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=BEYG1TN0PAZ0 type=write size=187294 2024-04-22T19:58:48.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.125+0000 7f5269a8e780 1 < canceled part key=_multipart_4EA3LUQRL8II.2~GAT3TU6UUH18.0 size=1048576 2024-04-22T19:58:48.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.126+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~GAT3TU6UUH18.1 size=1048576 2024-04-22T19:58:48.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.127+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~GAT3TU6UUH18.2 size=301261 2024-04-22T19:58:48.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.129+0000 7f5269a8e780 1 < canceled multipart upload key=4EA3LUQRL8II upload=GAT3TU6UUH18 tag=JT358IF1U4NR type=write size=3446989 2024-04-22T19:58:48.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.130+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=2M8T3MMXKO45 type=del size=4093954 2024-04-22T19:58:48.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.132+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~O9ANZUOEX51I.0 size=218103 2024-04-22T19:58:48.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.134+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=O9ANZUOEX51I tag=CH58CYG7DJKL type=write size=1266679 2024-04-22T19:58:48.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.135+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=G8XU21KWK6SM type=write size=908317 2024-04-22T19:58:48.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.136+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=ZUTK4GYZERYQ type=write size=965626 2024-04-22T19:58:48.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.138+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~A4QRKV8S0NHY.0 size=1048576 2024-04-22T19:58:48.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.139+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~A4QRKV8S0NHY.1 size=788503 2024-04-22T19:58:48.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.139+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=A4QRKV8S0NHY tag=KFD7KMEYBUNN type=write size=2885655 2024-04-22T19:58:48.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.140+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~JLSEXUHGQTU4.0 size=412128 2024-04-22T19:58:48.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.142+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=JLSEXUHGQTU4 tag=XKG5FP500675 type=write size=1460704 2024-04-22T19:58:48.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.143+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=8PKZEDWDGR2Q type=del size=2428678 2024-04-22T19:58:48.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.144+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=MQNJR25QGOFJ type=del 2024-04-22T19:58:48.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.147+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~W737Y03S4KKM.0 size=209587 2024-04-22T19:58:48.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.147+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=W737Y03S4KKM tag=QCIE48Q3EPKV type=write size=1258163 2024-04-22T19:58:48.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.148+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~N0WGH8SU8QTV.0 size=862885 2024-04-22T19:58:48.151 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.151+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=N0WGH8SU8QTV tag=66F18LFRXMVT type=write size=1911461 2024-04-22T19:58:48.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.156+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~ZJAKV7FST99U.0 size=883268 2024-04-22T19:58:48.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.156+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=ZJAKV7FST99U tag=5ZKBKYAJRJ9G type=write size=1931844 2024-04-22T19:58:48.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.159+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~AH6XK8B4AWIY.0 size=860354 2024-04-22T19:58:48.160 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.160+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=AH6XK8B4AWIY tag=OTSR1PSEJLMY type=write size=1908930 2024-04-22T19:58:48.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.162+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~8E1PF03FOYIB.0 size=1048576 2024-04-22T19:58:48.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.163+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~8E1PF03FOYIB.1 size=1048576 2024-04-22T19:58:48.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.164+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~8E1PF03FOYIB.2 size=716204 2024-04-22T19:58:48.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.164+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=8E1PF03FOYIB tag=RM6R0Q7KJ9IN type=write size=3861932 2024-04-22T19:58:48.165 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.165+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~BHFC2XMW5O5Z.0 size=1048576 2024-04-22T19:58:48.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.166+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~BHFC2XMW5O5Z.1 size=895070 2024-04-22T19:58:48.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.168+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=BHFC2XMW5O5Z tag=DWOMRVWYF1EI type=write size=2992222 2024-04-22T19:58:48.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.170+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~ACJYL2Q02EVT.0 size=554775 2024-04-22T19:58:48.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.170+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=ACJYL2Q02EVT tag=DVMXP4VZM31O type=write size=1603351 2024-04-22T19:58:48.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.173+0000 7f5269a8e780 1 < suggested remove operation key=GX6NILQG878S tag=XQE236RQTN2W type=del 2024-04-22T19:58:48.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.175+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~DSO2TQC1P2CF.0 size=1048576 2024-04-22T19:58:48.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.176+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~DSO2TQC1P2CF.1 size=333899 2024-04-22T19:58:48.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.176+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=DSO2TQC1P2CF tag=MBIRYZD6PNT3 type=write size=2431051 2024-04-22T19:58:48.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.177+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=YA94O8C3UCCU type=write size=521427 2024-04-22T19:58:48.178 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.178+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=15G3BXPHB3BI type=write size=907716 2024-04-22T19:58:48.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.180+0000 7f5269a8e780 1 < suggested remove operation key=T2BV2LUCY4E3 tag=6J57FFBVC2NE type=del 2024-04-22T19:58:48.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.181+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=5UIYGW06Q6T5 type=del size=777674 2024-04-22T19:58:48.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.183+0000 7f5269a8e780 1 < suggested update operation key=O9NFAVE09ZN7 tag=5KR1GC6DR3HY type=del size=2894219 2024-04-22T19:58:48.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.184+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=FV9DCIX5F5IM type=write size=747351 2024-04-22T19:58:48.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.186+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=GJNW76THKA8M type=del 2024-04-22T19:58:48.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.188+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~2537HOGSSGSF.0 size=1048576 2024-04-22T19:58:48.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.188+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~2537HOGSSGSF.1 size=647872 2024-04-22T19:58:48.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.188+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=2537HOGSSGSF tag=MNSGPZSQSUBM type=write size=2745024 2024-04-22T19:58:48.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.190+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~OCC07QD7A9ZF.0 size=1048576 2024-04-22T19:58:48.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.190+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~OCC07QD7A9ZF.1 size=803660 2024-04-22T19:58:48.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.192+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=OCC07QD7A9ZF tag=N7832B2PNKA2 type=write size=2900812 2024-04-22T19:58:48.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.194+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=P5V9PGEJ37A0 type=del size=377013 2024-04-22T19:58:48.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.195+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=A0AAA1F3SU1Y type=del 2024-04-22T19:58:48.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.197+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~OASWML6WJGTC.0 size=381286 2024-04-22T19:58:48.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.197+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=OASWML6WJGTC tag=QKZBR4TAOWZX type=write size=1429862 2024-04-22T19:58:48.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.198+0000 7f5269a8e780 1 < completed write operation key=M9M4SEEEZSFV tag=BEYG1TN0PAZ0 type=write size=187294 2024-04-22T19:58:48.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.199+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=KLPS476W7I70 type=write size=593471 2024-04-22T19:58:48.200 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.200+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=2M8T3MMXKO45 type=del 2024-04-22T19:58:48.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.201+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=6HI562RKXFJD type=del size=3967865 2024-04-22T19:58:48.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.202+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=G8XU21KWK6SM type=write size=908317 2024-04-22T19:58:48.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.204+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=9YIPFPVBPF95 type=write size=849580 2024-04-22T19:58:48.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.205+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~A4QRKV8S0NHY.0 size=1048576 2024-04-22T19:58:48.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.205+0000 7f5269a8e780 1 < canceled part key=_multipart_HV8J6YM0YNC1.2~A4QRKV8S0NHY.1 size=788503 2024-04-22T19:58:48.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.207+0000 7f5269a8e780 1 < canceled multipart upload key=HV8J6YM0YNC1 upload=A4QRKV8S0NHY tag=KFD7KMEYBUNN type=write size=2885655 2024-04-22T19:58:48.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.208+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=17HT68XWF95M type=del size=3972643 2024-04-22T19:58:48.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.209+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=8PKZEDWDGR2Q type=del 2024-04-22T19:58:48.211 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.211+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~N4JVKHM46ZG7.0 size=1048576 2024-04-22T19:58:48.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.212+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~N4JVKHM46ZG7.1 size=2922 2024-04-22T19:58:48.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.212+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=N4JVKHM46ZG7 tag=0JZ79HGTKXWY type=write size=2100074 2024-04-22T19:58:48.213 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.213+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~W737Y03S4KKM.0 size=209587 2024-04-22T19:58:48.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.215+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=W737Y03S4KKM tag=QCIE48Q3EPKV type=write size=1258163 2024-04-22T19:58:48.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.216+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=F1EGMLPVR7Y9 type=del size=568220 2024-04-22T19:58:48.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.217+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~ZJAKV7FST99U.0 size=883268 2024-04-22T19:58:48.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.219+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=ZJAKV7FST99U tag=5ZKBKYAJRJ9G type=write size=1931844 2024-04-22T19:58:48.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.221+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~Y9E5HT41VYZO.0 size=48601 2024-04-22T19:58:48.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.221+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=Y9E5HT41VYZO tag=FHW1Q2KEBHBE type=write size=1097177 2024-04-22T19:58:48.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.222+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~8E1PF03FOYIB.0 size=1048576 2024-04-22T19:58:48.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.223+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~8E1PF03FOYIB.1 size=1048576 2024-04-22T19:58:48.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.224+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~8E1PF03FOYIB.2 size=716204 2024-04-22T19:58:48.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.226+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=8E1PF03FOYIB tag=RM6R0Q7KJ9IN type=write size=3861932 2024-04-22T19:58:48.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.227+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=6AT02HR27F8R type=del size=2572220 2024-04-22T19:58:48.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.228+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~ACJYL2Q02EVT.0 size=554775 2024-04-22T19:58:48.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.230+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=ACJYL2Q02EVT tag=DVMXP4VZM31O type=write size=1603351 2024-04-22T19:58:48.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.231+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=GNXBBFP71TQ6 type=del size=3482378 2024-04-22T19:58:48.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.232+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~DSO2TQC1P2CF.0 size=1048576 2024-04-22T19:58:48.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.233+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~DSO2TQC1P2CF.1 size=333899 2024-04-22T19:58:48.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.235+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=DSO2TQC1P2CF tag=MBIRYZD6PNT3 type=write size=2431051 2024-04-22T19:58:48.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.236+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=861ZNGN7TIPX type=write size=521878 2024-04-22T19:58:48.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.238+0000 7f5269a8e780 1 < suggested update operation key=GRSL48RUEBX8 tag=15G3BXPHB3BI type=write size=907716 2024-04-22T19:58:48.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.239+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=7ONOX9TDJVF7 type=del size=878127 2024-04-22T19:58:48.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.240+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=5UIYGW06Q6T5 type=del 2024-04-22T19:58:48.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.242+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=Z81PTM9WMOZZ type=write size=552261 2024-04-22T19:58:48.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.243+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=FV9DCIX5F5IM type=write size=747351 2024-04-22T19:58:48.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.244+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=NQ19IHGFNTMP type=del size=1986902 2024-04-22T19:58:48.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.245+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~2537HOGSSGSF.0 size=1048576 2024-04-22T19:58:48.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.246+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~2537HOGSSGSF.1 size=647872 2024-04-22T19:58:48.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.248+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=2537HOGSSGSF tag=MNSGPZSQSUBM type=write size=2745024 2024-04-22T19:58:48.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.249+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=PYX82JFKMMX3 type=del size=2516681 2024-04-22T19:58:48.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.251+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=P5V9PGEJ37A0 type=del 2024-04-22T19:58:48.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.252+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=6E4C68L34WVD type=del size=2493157 2024-04-22T19:58:48.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.253+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~OASWML6WJGTC.0 size=381286 2024-04-22T19:58:48.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.255+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=OASWML6WJGTC tag=QKZBR4TAOWZX type=write size=1429862 2024-04-22T19:58:48.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.257+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~7T5W89U2RUS7.0 size=429318 2024-04-22T19:58:48.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.257+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=7T5W89U2RUS7 tag=QIGX5S8C382K type=write size=1477894 2024-04-22T19:58:48.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.258+0000 7f5269a8e780 1 < completed write operation key=SN2BZATJGUWA tag=KLPS476W7I70 type=write size=593471 2024-04-22T19:58:48.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.259+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=IH95FNTEI6P2 type=del size=1905719 2024-04-22T19:58:48.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.260+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=6HI562RKXFJD type=del 2024-04-22T19:58:48.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.262+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=QHF2NYMV478Q type=write size=455382 2024-04-22T19:58:48.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.267+0000 7f5269a8e780 1 < suggested update operation key=8V6ESIGMRO5G tag=9YIPFPVBPF95 type=write size=849580 2024-04-22T19:58:48.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.268+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=VDETRO34Q8T4 type=write size=722730 2024-04-22T19:58:48.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.269+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=17HT68XWF95M type=del 2024-04-22T19:58:48.271 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.271+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~CGO8FZ169Q56.0 size=1048576 2024-04-22T19:58:48.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.272+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~CGO8FZ169Q56.1 size=384943 2024-04-22T19:58:48.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.272+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=CGO8FZ169Q56 tag=3PDT4438S3TJ type=write size=2482095 2024-04-22T19:58:48.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.273+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~N4JVKHM46ZG7.0 size=1048576 2024-04-22T19:58:48.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.274+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~N4JVKHM46ZG7.1 size=2922 2024-04-22T19:58:48.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.276+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=N4JVKHM46ZG7 tag=0JZ79HGTKXWY type=write size=2100074 2024-04-22T19:58:48.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.277+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=XXRXN480ZWAD type=write size=1005487 2024-04-22T19:58:48.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.279+0000 7f5269a8e780 1 < suggested remove operation key=GP1C5ULUU143 tag=F1EGMLPVR7Y9 type=del 2024-04-22T19:58:48.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.281+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~08BVQXXL288I.0 size=204723 2024-04-22T19:58:48.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.281+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=08BVQXXL288I tag=9JEURMZBTM54 type=write size=1253299 2024-04-22T19:58:48.283 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.283+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~Y9E5HT41VYZO.0 size=48601 2024-04-22T19:58:48.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.285+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=Y9E5HT41VYZO tag=FHW1Q2KEBHBE type=write size=1097177 2024-04-22T19:58:48.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.286+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=OOCFBQ4WSG6Y type=write size=788013 2024-04-22T19:58:48.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.287+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=6AT02HR27F8R type=del 2024-04-22T19:58:48.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.289+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~MVX3OR50QF68.0 size=394257 2024-04-22T19:58:48.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.289+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=MVX3OR50QF68 tag=ONB1SF5Y2AOV type=write size=1442833 2024-04-22T19:58:48.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.291+0000 7f5269a8e780 1 < suggested remove operation key=B0Y7ZTPC0975 tag=GNXBBFP71TQ6 type=del 2024-04-22T19:58:48.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.293+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=DIO6NOY5RCND type=del size=1661159 2024-04-22T19:58:48.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.294+0000 7f5269a8e780 1 < completed write operation key=EHRS6DQDRXVS tag=861ZNGN7TIPX type=write size=521878 2024-04-22T19:58:48.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.295+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=FT7GZJV8ZG3G type=del size=2435780 2024-04-22T19:58:48.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.297+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=7ONOX9TDJVF7 type=del 2024-04-22T19:58:48.297 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.298+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=BQVXIAZWCGES type=del size=2545848 2024-04-22T19:58:48.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.299+0000 7f5269a8e780 1 < completed write operation key=SN2BZATJGUWA tag=Z81PTM9WMOZZ type=write size=552261 2024-04-22T19:58:48.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.300+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=UNO30LHP2D7S type=del size=3899725 2024-04-22T19:58:48.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.301+0000 7f5269a8e780 1 < canceled operation key=86GW6HV9G1IZ tag=NQ19IHGFNTMP type=del size=1986902 2024-04-22T19:58:48.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.302+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=T79E0BWYI7DB type=del size=2595435 2024-04-22T19:58:48.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.303+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=PYX82JFKMMX3 type=del 2024-04-22T19:58:48.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.304+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=KL2IM6PVWOCC type=del size=1506133 2024-04-22T19:58:48.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.305+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=6E4C68L34WVD type=del 2024-04-22T19:58:48.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.307+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=3D2GEWMEBQKE type=del size=3126729 2024-04-22T19:58:48.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.308+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~7T5W89U2RUS7.0 size=429318 2024-04-22T19:58:48.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.310+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=7T5W89U2RUS7 tag=QIGX5S8C382K type=write size=1477894 2024-04-22T19:58:48.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.311+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=RQVD5DCH3GP8 type=del size=670241 2024-04-22T19:58:48.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.312+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=IH95FNTEI6P2 type=del 2024-04-22T19:58:48.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.313+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=FNLV3QW50RGB type=del size=2503761 2024-04-22T19:58:48.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.315+0000 7f5269a8e780 1 < suggested remove operation key=M9M4SEEEZSFV tag=QHF2NYMV478Q type=write 2024-04-22T19:58:48.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.316+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=UPVHKSDOKF1Q type=del size=2473724 2024-04-22T19:58:48.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.319+0000 7f5269a8e780 1 < suggested update operation key=DX5YUYQRZKWA tag=VDETRO34Q8T4 type=write size=722730 2024-04-22T19:58:48.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.320+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~6DARU51JIV0M.0 size=436846 2024-04-22T19:58:48.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.320+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=6DARU51JIV0M tag=UE9YA6RSFMD9 type=write size=1485422 2024-04-22T19:58:48.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.322+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~CGO8FZ169Q56.0 size=1048576 2024-04-22T19:58:48.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.322+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~CGO8FZ169Q56.1 size=384943 2024-04-22T19:58:48.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.324+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=CGO8FZ169Q56 tag=3PDT4438S3TJ type=write size=2482095 2024-04-22T19:58:48.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.325+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=R5ARRTFI5OR7 type=del size=4154507 2024-04-22T19:58:48.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.326+0000 7f5269a8e780 1 < canceled operation key=GX6NILQG878S tag=XXRXN480ZWAD type=write size=1005487 2024-04-22T19:58:48.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.328+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~1PK2FW3IUAWV.0 size=1048576 2024-04-22T19:58:48.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.329+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~1PK2FW3IUAWV.1 size=1048576 2024-04-22T19:58:48.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.330+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~1PK2FW3IUAWV.2 size=228595 2024-04-22T19:58:48.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.330+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=1PK2FW3IUAWV tag=7LSXAPZVLGTA type=write size=3374323 2024-04-22T19:58:48.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.331+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~08BVQXXL288I.0 size=204723 2024-04-22T19:58:48.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.333+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=08BVQXXL288I tag=9JEURMZBTM54 type=write size=1253299 2024-04-22T19:58:48.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.335+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~AI1I53225F0C.0 size=511896 2024-04-22T19:58:48.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.335+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=AI1I53225F0C tag=DJP686ZA2DIV type=write size=1560472 2024-04-22T19:58:48.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.336+0000 7f5269a8e780 1 < completed write operation key=2YFD2U6HVNE7 tag=OOCFBQ4WSG6Y type=write size=788013 2024-04-22T19:58:48.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.337+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=110EUS3BWIFH type=del size=1941012 2024-04-22T19:58:48.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.339+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~MVX3OR50QF68.0 size=394257 2024-04-22T19:58:48.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.341+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=MVX3OR50QF68 tag=ONB1SF5Y2AOV type=write size=1442833 2024-04-22T19:58:48.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.343+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~IUNZHJD4SJ9C.0 size=956694 2024-04-22T19:58:48.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.343+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=IUNZHJD4SJ9C tag=OLII1SLN0HFP type=write size=2005270 2024-04-22T19:58:48.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.344+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=DIO6NOY5RCND type=del 2024-04-22T19:58:48.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.345+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=NO27A3K6VWZP type=del size=1667303 2024-04-22T19:58:48.346 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.346+0000 7f5269a8e780 1 < canceled operation key=5UBYMQT7ZWKX tag=FT7GZJV8ZG3G type=del size=2435780 2024-04-22T19:58:48.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.347+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=RB4ULU33E0BL type=del size=477494 2024-04-22T19:58:48.348 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.348+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=BQVXIAZWCGES type=del 2024-04-22T19:58:48.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.350+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=DBVP9S4G22H2 type=del size=1448943 2024-04-22T19:58:48.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.351+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=UNO30LHP2D7S type=del 2024-04-22T19:58:48.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.352+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=SWIXRU8IA5UZ type=del size=1944171 2024-04-22T19:58:48.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.354+0000 7f5269a8e780 1 < suggested remove operation key=KV5DYQ49ZKCX tag=T79E0BWYI7DB type=del 2024-04-22T19:58:48.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.356+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~2NXSOU1F5XH2.0 size=1048576 2024-04-22T19:58:48.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.357+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~2NXSOU1F5XH2.1 size=1048576 2024-04-22T19:58:48.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.358+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~2NXSOU1F5XH2.2 size=338302 2024-04-22T19:58:48.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.358+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=2NXSOU1F5XH2 tag=A6Y0NJLHQJI8 type=write size=3484030 2024-04-22T19:58:48.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.359+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=KL2IM6PVWOCC type=del 2024-04-22T19:58:48.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.360+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=CXZJ8A8HR397 type=del size=384722 2024-04-22T19:58:48.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.362+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=3D2GEWMEBQKE type=del 2024-04-22T19:58:48.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.364+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~X1OGFILEENPG.0 size=1048576 2024-04-22T19:58:48.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.365+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~X1OGFILEENPG.1 size=562443 2024-04-22T19:58:48.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.365+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=X1OGFILEENPG tag=514JMH0TAQ1M type=write size=2659595 2024-04-22T19:58:48.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.366+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=RQVD5DCH3GP8 type=del 2024-04-22T19:58:48.367 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.367+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=422XKWWLW68N type=del size=3870925 2024-04-22T19:58:48.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.368+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=FNLV3QW50RGB type=del 2024-04-22T19:58:48.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.369+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=ODV5QUNLRAUB type=del size=2668930 2024-04-22T19:58:48.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.371+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=UPVHKSDOKF1Q type=del 2024-04-22T19:58:48.372 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.372+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=3SKBLX92PIWG type=del size=2017260 2024-04-22T19:58:48.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.374+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~6DARU51JIV0M.0 size=436846 2024-04-22T19:58:48.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.376+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=6DARU51JIV0M tag=UE9YA6RSFMD9 type=write size=1485422 2024-04-22T19:58:48.377 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.377+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=5CGQX0UGJUXM type=del size=2981328 2024-04-22T19:58:48.379 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.379+0000 7f5269a8e780 1 < suggested update operation key=A853MOZPT9BG tag=R5ARRTFI5OR7 type=del size=4154507 2024-04-22T19:58:48.381 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.381+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~OK05SLF196SP.0 size=957510 2024-04-22T19:58:48.381 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.381+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=OK05SLF196SP tag=EVN6F65EDYL2 type=write size=2006086 2024-04-22T19:58:48.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.382+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~1PK2FW3IUAWV.0 size=1048576 2024-04-22T19:58:48.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.382+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~1PK2FW3IUAWV.1 size=1048576 2024-04-22T19:58:48.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.383+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~1PK2FW3IUAWV.2 size=228595 2024-04-22T19:58:48.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.385+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=1PK2FW3IUAWV tag=7LSXAPZVLGTA type=write size=3374323 2024-04-22T19:58:48.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.386+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=CJYM7O2K2LJD type=del size=286426 2024-04-22T19:58:48.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.387+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~AI1I53225F0C.0 size=511896 2024-04-22T19:58:48.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.389+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=AI1I53225F0C tag=DJP686ZA2DIV type=write size=1560472 2024-04-22T19:58:48.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.390+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=CK2BE3JQRKRV type=del size=2293572 2024-04-22T19:58:48.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.392+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=110EUS3BWIFH type=del 2024-04-22T19:58:48.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.394+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~34CK5TCS5M2A.0 size=1048576 2024-04-22T19:58:48.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.394+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~34CK5TCS5M2A.1 size=103243 2024-04-22T19:58:48.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.394+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=34CK5TCS5M2A tag=UUI3SJ33O1UJ type=write size=2200395 2024-04-22T19:58:48.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.396+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~IUNZHJD4SJ9C.0 size=956694 2024-04-22T19:58:48.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.398+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=IUNZHJD4SJ9C tag=OLII1SLN0HFP type=write size=2005270 2024-04-22T19:58:48.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.400+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~HR5YFRTZV5E5.0 size=1048576 2024-04-22T19:58:48.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.401+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~HR5YFRTZV5E5.1 size=1048576 2024-04-22T19:58:48.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.402+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~HR5YFRTZV5E5.2 size=397845 2024-04-22T19:58:48.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.402+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=HR5YFRTZV5E5 tag=XOJOMZREZBRT type=write size=3543573 2024-04-22T19:58:48.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.403+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=NO27A3K6VWZP type=del 2024-04-22T19:58:48.404 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.404+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=BBDOUC270TZQ type=del size=2871106 2024-04-22T19:58:48.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.405+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=RB4ULU33E0BL type=del 2024-04-22T19:58:48.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.407+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=4J8I4YSRFVJW type=del size=3403949 2024-04-22T19:58:48.408 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.408+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=DBVP9S4G22H2 type=del 2024-04-22T19:58:48.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.409+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=4JRHEGO0TTA0 type=write size=1024221 2024-04-22T19:58:48.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.411+0000 7f5269a8e780 1 < suggested update operation key=2YFD2U6HVNE7 tag=SWIXRU8IA5UZ type=del size=1944171 2024-04-22T19:58:48.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.414+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~G4Y6W4R3B92P.0 size=1048576 2024-04-22T19:58:48.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.415+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~G4Y6W4R3B92P.1 size=290518 2024-04-22T19:58:48.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.415+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=G4Y6W4R3B92P tag=EUDGACAVF1CT type=write size=2387670 2024-04-22T19:58:48.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.416+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~2NXSOU1F5XH2.0 size=1048576 2024-04-22T19:58:48.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.417+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~2NXSOU1F5XH2.1 size=1048576 2024-04-22T19:58:48.417 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.418+0000 7f5269a8e780 1 < completed part key=_multipart_KE7YGPTF1MOR.2~2NXSOU1F5XH2.2 size=338302 2024-04-22T19:58:48.420 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.420+0000 7f5269a8e780 1 < completed multipart upload key=KE7YGPTF1MOR upload=2NXSOU1F5XH2 tag=A6Y0NJLHQJI8 type=write size=3484030 2024-04-22T19:58:48.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.422+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~02E1MJ0Q497Z.0 size=1048576 2024-04-22T19:58:48.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.423+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~02E1MJ0Q497Z.1 size=660995 2024-04-22T19:58:48.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.423+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=02E1MJ0Q497Z tag=TZIRQ47OUT39 type=write size=2758147 2024-04-22T19:58:48.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.424+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=CXZJ8A8HR397 type=del 2024-04-22T19:58:48.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.426+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~O8PCY4P9YN6Z.0 size=595836 2024-04-22T19:58:48.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.426+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=O8PCY4P9YN6Z tag=VQSAMCTO65JT type=write size=1644412 2024-04-22T19:58:48.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.427+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~X1OGFILEENPG.0 size=1048576 2024-04-22T19:58:48.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.428+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~X1OGFILEENPG.1 size=562443 2024-04-22T19:58:48.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.430+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=X1OGFILEENPG tag=514JMH0TAQ1M type=write size=2659595 2024-04-22T19:58:48.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.431+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=9ZM9VULVANQG type=del size=3570736 2024-04-22T19:58:48.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.432+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=422XKWWLW68N type=del 2024-04-22T19:58:48.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.433+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=0064ARYI8FWX type=del size=1120616 2024-04-22T19:58:48.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.435+0000 7f5269a8e780 -1 < failed to complete operation key=A853MOZPT9BG tag=ODV5QUNLRAUB type=del size=2668930: (22) Invalid argument 2024-04-22T19:58:48.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.437+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~0Q7IR2FBA4O3.0 size=810226 2024-04-22T19:58:48.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.437+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=0Q7IR2FBA4O3 tag=EQC5C732IQ1W type=write size=1858802 2024-04-22T19:58:48.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.438+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=3SKBLX92PIWG type=del 2024-04-22T19:58:48.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.440+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~KSXGRH62V2N7.0 size=1048576 2024-04-22T19:58:48.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.441+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~KSXGRH62V2N7.1 size=789534 2024-04-22T19:58:48.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.441+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=KSXGRH62V2N7 tag=IMR3Y5SHRPDP type=write size=2886686 2024-04-22T19:58:48.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.442+0000 7f5269a8e780 1 < completed delete operation key=4GX83ZJWZCD1 tag=5CGQX0UGJUXM type=del 2024-04-22T19:58:48.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.444+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~81HF3ISZLKKF.0 size=1048576 2024-04-22T19:58:48.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.445+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~81HF3ISZLKKF.1 size=1048576 2024-04-22T19:58:48.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.446+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~81HF3ISZLKKF.2 size=552365 2024-04-22T19:58:48.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.446+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=81HF3ISZLKKF tag=7DPXC524U4HJ type=write size=3698093 2024-04-22T19:58:48.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.447+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~OK05SLF196SP.0 size=957510 2024-04-22T19:58:48.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.449+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=OK05SLF196SP tag=EVN6F65EDYL2 type=write size=2006086 2024-04-22T19:58:48.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.451+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~JPDDXEHI5LOE.0 size=1048576 2024-04-22T19:58:48.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.452+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~JPDDXEHI5LOE.1 size=1048576 2024-04-22T19:58:48.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.453+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~JPDDXEHI5LOE.2 size=491832 2024-04-22T19:58:48.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.453+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=JPDDXEHI5LOE tag=1F345GP6NGT8 type=write size=3637560 2024-04-22T19:58:48.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.454+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=CJYM7O2K2LJD type=del 2024-04-22T19:58:48.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.456+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~7INQ9NT4UZEE.0 size=1048576 2024-04-22T19:58:48.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.457+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~7INQ9NT4UZEE.1 size=615883 2024-04-22T19:58:48.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.457+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=7INQ9NT4UZEE tag=8U2U8GHIQMCR type=write size=2713035 2024-04-22T19:58:48.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.460+0000 7f5269a8e780 1 < suggested update operation key=DX5YUYQRZKWA tag=CK2BE3JQRKRV type=del size=2293572 2024-04-22T19:58:48.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.461+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=8F6LK07Q5RQQ type=del size=3292894 2024-04-22T19:58:48.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.462+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~34CK5TCS5M2A.0 size=1048576 2024-04-22T19:58:48.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.463+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~34CK5TCS5M2A.1 size=103243 2024-04-22T19:58:48.465 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.465+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=34CK5TCS5M2A tag=UUI3SJ33O1UJ type=write size=2200395 2024-04-22T19:58:48.467 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.467+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~3C2IAA5AYNNY.0 size=1048576 2024-04-22T19:58:48.468 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.468+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~3C2IAA5AYNNY.1 size=1048576 2024-04-22T19:58:48.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.469+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~3C2IAA5AYNNY.2 size=76495 2024-04-22T19:58:48.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.469+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=3C2IAA5AYNNY tag=VEAPO1Z6ZZDH type=write size=3222223 2024-04-22T19:58:48.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.470+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~HR5YFRTZV5E5.0 size=1048576 2024-04-22T19:58:48.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.471+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~HR5YFRTZV5E5.1 size=1048576 2024-04-22T19:58:48.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.472+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~HR5YFRTZV5E5.2 size=397845 2024-04-22T19:58:48.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.474+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=HR5YFRTZV5E5 tag=XOJOMZREZBRT type=write size=3543573 2024-04-22T19:58:48.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.475+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=V3V4E9VDPYL2 type=write size=452680 2024-04-22T19:58:48.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.476+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=BBDOUC270TZQ type=del 2024-04-22T19:58:48.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.477+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=MEC0CSZYMSK1 type=del size=2098969 2024-04-22T19:58:48.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.479+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=4J8I4YSRFVJW type=del 2024-04-22T19:58:48.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.481+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~AZ669SKBSBX5.0 size=1048576 2024-04-22T19:58:48.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.482+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~AZ669SKBSBX5.1 size=274930 2024-04-22T19:58:48.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.482+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=AZ669SKBSBX5 tag=YI5HZV2F9I4I type=write size=2372082 2024-04-22T19:58:48.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.483+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=4JRHEGO0TTA0 type=write size=1024221 2024-04-22T19:58:48.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.485+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~1SBSG11K6OFK.0 size=1048576 2024-04-22T19:58:48.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.486+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~1SBSG11K6OFK.1 size=1048576 2024-04-22T19:58:48.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.487+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~1SBSG11K6OFK.2 size=742703 2024-04-22T19:58:48.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.487+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=1SBSG11K6OFK tag=H2DWN9DVV0WC type=write size=3888431 2024-04-22T19:58:48.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.488+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~G4Y6W4R3B92P.0 size=1048576 2024-04-22T19:58:48.489 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.489+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~G4Y6W4R3B92P.1 size=290518 2024-04-22T19:58:48.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.491+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=G4Y6W4R3B92P tag=EUDGACAVF1CT type=write size=2387670 2024-04-22T19:58:48.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.493+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~RLQVTTENKYQ6.0 size=36114 2024-04-22T19:58:48.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.493+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=RLQVTTENKYQ6 tag=KJC3Z4NPURH7 type=write size=1084690 2024-04-22T19:58:48.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.494+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~02E1MJ0Q497Z.0 size=1048576 2024-04-22T19:58:48.495 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.495+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~02E1MJ0Q497Z.1 size=660995 2024-04-22T19:58:48.497 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.497+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=02E1MJ0Q497Z tag=TZIRQ47OUT39 type=write size=2758147 2024-04-22T19:58:48.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.499+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~NYD66VV1K9I2.0 size=1031907 2024-04-22T19:58:48.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.499+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=NYD66VV1K9I2 tag=IN94U1B3PNZK type=write size=2080483 2024-04-22T19:58:48.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.500+0000 7f5269a8e780 1 < completed part key=_multipart_4EA3LUQRL8II.2~O8PCY4P9YN6Z.0 size=595836 2024-04-22T19:58:48.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.502+0000 7f5269a8e780 1 < completed multipart upload key=4EA3LUQRL8II upload=O8PCY4P9YN6Z tag=VQSAMCTO65JT type=write size=1644412 2024-04-22T19:58:48.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.504+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=K2EEOXHJF4BQ type=write size=893210 2024-04-22T19:58:48.506 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.506+0000 7f5269a8e780 1 < suggested update operation key=EHRS6DQDRXVS tag=9ZM9VULVANQG type=del size=3570736 2024-04-22T19:58:48.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.507+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=3C17FQ1BM12G type=del size=2590351 2024-04-22T19:58:48.508 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.508+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=0064ARYI8FWX type=del 2024-04-22T19:58:48.510 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.510+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~OJT1IR49MQ6A.0 size=1048576 2024-04-22T19:58:48.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.511+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~OJT1IR49MQ6A.1 size=1048576 2024-04-22T19:58:48.512 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.512+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~OJT1IR49MQ6A.2 size=138272 2024-04-22T19:58:48.512 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.512+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=OJT1IR49MQ6A tag=UVL0VCGI61M5 type=write size=3284000 2024-04-22T19:58:48.513 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.513+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~0Q7IR2FBA4O3.0 size=810226 2024-04-22T19:58:48.515 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.516+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=0Q7IR2FBA4O3 tag=EQC5C732IQ1W type=write size=1858802 2024-04-22T19:58:48.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.517+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=GF5BLH12UFUD type=del size=3749485 2024-04-22T19:58:48.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.518+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~KSXGRH62V2N7.0 size=1048576 2024-04-22T19:58:48.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.519+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~KSXGRH62V2N7.1 size=789534 2024-04-22T19:58:48.521 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.521+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=KSXGRH62V2N7 tag=IMR3Y5SHRPDP type=write size=2886686 2024-04-22T19:58:48.523 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.523+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~BGR374QH1NA2.0 size=1048576 2024-04-22T19:58:48.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.524+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~BGR374QH1NA2.1 size=1048576 2024-04-22T19:58:48.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.525+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~BGR374QH1NA2.2 size=66555 2024-04-22T19:58:48.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.525+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=BGR374QH1NA2 tag=8F0GW769YNBE type=write size=3212283 2024-04-22T19:58:48.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.526+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~81HF3ISZLKKF.0 size=1048576 2024-04-22T19:58:48.527 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.527+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~81HF3ISZLKKF.1 size=1048576 2024-04-22T19:58:48.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.528+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~81HF3ISZLKKF.2 size=552365 2024-04-22T19:58:48.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.530+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=81HF3ISZLKKF tag=7DPXC524U4HJ type=write size=3698093 2024-04-22T19:58:48.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.531+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=XSUYE2FR5FYE type=del size=1429319 2024-04-22T19:58:48.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.532+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~JPDDXEHI5LOE.0 size=1048576 2024-04-22T19:58:48.533 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.533+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~JPDDXEHI5LOE.1 size=1048576 2024-04-22T19:58:48.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.534+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~JPDDXEHI5LOE.2 size=491832 2024-04-22T19:58:48.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.536+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=JPDDXEHI5LOE tag=1F345GP6NGT8 type=write size=3637560 2024-04-22T19:58:48.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.537+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=OE8BXRW3GP49 type=del size=2687244 2024-04-22T19:58:48.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.538+0000 7f5269a8e780 1 < canceled part key=_multipart_T2BV2LUCY4E3.2~7INQ9NT4UZEE.0 size=1048576 2024-04-22T19:58:48.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.538+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~7INQ9NT4UZEE.1 size=615883 2024-04-22T19:58:48.540 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.540+0000 7f5269a8e780 1 < canceled multipart upload key=T2BV2LUCY4E3 upload=7INQ9NT4UZEE tag=8U2U8GHIQMCR type=write size=2713035 2024-04-22T19:58:48.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.541+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=B8KIKBKDGHFX type=del size=508498 2024-04-22T19:58:48.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.543+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=8F6LK07Q5RQQ type=del 2024-04-22T19:58:48.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.544+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=BX8T1LNB47F6 type=del size=3826469 2024-04-22T19:58:48.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.544+0000 7f5269a8e780 1 < canceled part key=_multipart_637M7AP8X9E9.2~3C2IAA5AYNNY.0 size=1048576 2024-04-22T19:58:48.545 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.545+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~3C2IAA5AYNNY.1 size=1048576 2024-04-22T19:58:48.545 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.545+0000 7f5269a8e780 1 < canceled part key=_multipart_637M7AP8X9E9.2~3C2IAA5AYNNY.2 size=76495 2024-04-22T19:58:48.547 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.547+0000 7f5269a8e780 1 < canceled multipart upload key=637M7AP8X9E9 upload=3C2IAA5AYNNY tag=VEAPO1Z6ZZDH type=write size=3222223 2024-04-22T19:58:48.548 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.548+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=56AVZWUIDR6I type=del size=1276171 2024-04-22T19:58:48.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.549+0000 7f5269a8e780 1 < completed write operation key=4EA3LUQRL8II tag=V3V4E9VDPYL2 type=write size=452680 2024-04-22T19:58:48.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.551+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~3RS2GTLD6J1C.0 size=1048576 2024-04-22T19:58:48.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.552+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~3RS2GTLD6J1C.1 size=698319 2024-04-22T19:58:48.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.552+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=3RS2GTLD6J1C tag=NOYEGW8HDK28 type=write size=2795471 2024-04-22T19:58:48.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.553+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=MEC0CSZYMSK1 type=del 2024-04-22T19:58:48.554 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.555+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=DGZCBYMNK5TQ type=write size=267912 2024-04-22T19:58:48.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.556+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~AZ669SKBSBX5.0 size=1048576 2024-04-22T19:58:48.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.556+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~AZ669SKBSBX5.1 size=274930 2024-04-22T19:58:48.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.558+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=AZ669SKBSBX5 tag=YI5HZV2F9I4I type=write size=2372082 2024-04-22T19:58:48.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.559+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=MSBI82WG290T type=del size=1913339 2024-04-22T19:58:48.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.560+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~1SBSG11K6OFK.0 size=1048576 2024-04-22T19:58:48.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.561+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~1SBSG11K6OFK.1 size=1048576 2024-04-22T19:58:48.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.562+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~1SBSG11K6OFK.2 size=742703 2024-04-22T19:58:48.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.564+0000 7f5269a8e780 1 < completed multipart upload key=A853MOZPT9BG upload=1SBSG11K6OFK tag=H2DWN9DVV0WC type=write size=3888431 2024-04-22T19:58:48.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.565+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=XAI23LBAC78I type=del size=3709946 2024-04-22T19:58:48.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.565+0000 7f5269a8e780 1 < canceled part key=_multipart_HV8J6YM0YNC1.2~RLQVTTENKYQ6.0 size=36114 2024-04-22T19:58:48.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.567+0000 7f5269a8e780 1 < canceled multipart upload key=HV8J6YM0YNC1 upload=RLQVTTENKYQ6 tag=KJC3Z4NPURH7 type=write size=1084690 2024-04-22T19:58:48.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.569+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~SV0CK6M9X8ND.0 size=1048576 2024-04-22T19:58:48.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.570+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~SV0CK6M9X8ND.1 size=366821 2024-04-22T19:58:48.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.570+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=SV0CK6M9X8ND tag=E1D3BDKIKULE type=write size=2463973 2024-04-22T19:58:48.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.571+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~NYD66VV1K9I2.0 size=1031907 2024-04-22T19:58:48.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.573+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=NYD66VV1K9I2 tag=IN94U1B3PNZK type=write size=2080483 2024-04-22T19:58:48.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.574+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=IBMAJ02NDYLV type=del size=3516899 2024-04-22T19:58:48.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.576+0000 7f5269a8e780 -1 < failed to complete operation key=EHRS6DQDRXVS tag=K2EEOXHJF4BQ type=write size=893210: (22) Invalid argument 2024-04-22T19:58:48.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.577+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=FE8IRML01Q5H type=del size=4169079 2024-04-22T19:58:48.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.579+0000 7f5269a8e780 1 < suggested remove operation key=GP1C5ULUU143 tag=3C17FQ1BM12G type=del 2024-04-22T19:58:48.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.581+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~TPF85YMCVSXG.0 size=1048576 2024-04-22T19:58:48.582 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.582+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~TPF85YMCVSXG.1 size=1048576 2024-04-22T19:58:48.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.583+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~TPF85YMCVSXG.2 size=808560 2024-04-22T19:58:48.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.583+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=TPF85YMCVSXG tag=3J3FO92DKLZ2 type=write size=3954288 2024-04-22T19:58:48.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.586+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~OJT1IR49MQ6A.0 size=1048576 2024-04-22T19:58:48.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.587+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~OJT1IR49MQ6A.1 size=1048576 2024-04-22T19:58:48.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.588+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~OJT1IR49MQ6A.2 size=138272 2024-04-22T19:58:48.589 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.590+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=OJT1IR49MQ6A tag=UVL0VCGI61M5 type=write size=3284000 2024-04-22T19:58:48.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.592+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~HL1VTKA10R9N.0 size=1048576 2024-04-22T19:58:48.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.593+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~HL1VTKA10R9N.1 size=1048576 2024-04-22T19:58:48.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.593+0000 7f5269a8e780 1 > prepared part key=_multipart_DX5YUYQRZKWA.2~HL1VTKA10R9N.2 size=572446 2024-04-22T19:58:48.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.593+0000 7f5269a8e780 1 > prepared multipart upload key=DX5YUYQRZKWA upload=HL1VTKA10R9N tag=SZV6WECF0535 type=write size=3718174 2024-04-22T19:58:48.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.595+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=GF5BLH12UFUD type=del 2024-04-22T19:58:48.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.596+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=LWKTSELD6B3P type=del size=2777950 2024-04-22T19:58:48.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.597+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~BGR374QH1NA2.0 size=1048576 2024-04-22T19:58:48.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.598+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~BGR374QH1NA2.1 size=1048576 2024-04-22T19:58:48.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.598+0000 7f5269a8e780 1 < canceled part key=_multipart_2YFD2U6HVNE7.2~BGR374QH1NA2.2 size=66555 2024-04-22T19:58:48.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.600+0000 7f5269a8e780 1 < canceled multipart upload key=2YFD2U6HVNE7 upload=BGR374QH1NA2 tag=8F0GW769YNBE type=write size=3212283 2024-04-22T19:58:48.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.601+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=CHPI1YXUPI4V type=del size=3784207 2024-04-22T19:58:48.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.602+0000 7f5269a8e780 1 < canceled operation key=O9NFAVE09ZN7 tag=XSUYE2FR5FYE type=del size=1429319 2024-04-22T19:58:48.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.603+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=58QQ4JQZP7MG type=write size=5133 2024-04-22T19:58:48.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.604+0000 7f5269a8e780 -1 < failed to complete operation key=GP1C5ULUU143 tag=OE8BXRW3GP49 type=del size=2687244: (22) Invalid argument 2024-04-22T19:58:48.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.606+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~I1YADYHK1YMR.0 size=1048576 2024-04-22T19:58:48.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.607+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~I1YADYHK1YMR.1 size=880743 2024-04-22T19:58:48.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.607+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=I1YADYHK1YMR tag=N9INXSJ735L2 type=write size=2977895 2024-04-22T19:58:48.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.608+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=B8KIKBKDGHFX type=del 2024-04-22T19:58:48.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.610+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~D5RPENLP5C3A.0 size=1048576 2024-04-22T19:58:48.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.611+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~D5RPENLP5C3A.1 size=1048576 2024-04-22T19:58:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.612+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~D5RPENLP5C3A.2 size=156118 2024-04-22T19:58:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.612+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=D5RPENLP5C3A tag=J7EB405JYBWH type=write size=3301846 2024-04-22T19:58:48.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.613+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=BX8T1LNB47F6 type=del 2024-04-22T19:58:48.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.616+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~V9FHDAWJWPDB.0 size=1048576 2024-04-22T19:58:48.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.616+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~V9FHDAWJWPDB.1 size=1048576 2024-04-22T19:58:48.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.617+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~V9FHDAWJWPDB.2 size=153310 2024-04-22T19:58:48.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.617+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=V9FHDAWJWPDB tag=WGLE090ZUPF4 type=write size=3299038 2024-04-22T19:58:48.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.618+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=56AVZWUIDR6I type=del 2024-04-22T19:58:48.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.620+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~DCUCRTDFW05V.0 size=673886 2024-04-22T19:58:48.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.620+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=DCUCRTDFW05V tag=N9TD56TU8KWL type=write size=1722462 2024-04-22T19:58:48.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.622+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~3RS2GTLD6J1C.0 size=1048576 2024-04-22T19:58:48.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.622+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~3RS2GTLD6J1C.1 size=698319 2024-04-22T19:58:48.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.624+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=3RS2GTLD6J1C tag=NOYEGW8HDK28 type=write size=2795471 2024-04-22T19:58:48.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.626+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=H699C828V28R type=del size=2037689 2024-04-22T19:58:48.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.628+0000 7f5269a8e780 1 < suggested remove operation key=KE7YGPTF1MOR tag=DGZCBYMNK5TQ type=write 2024-04-22T19:58:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.629+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=7A1G233QXS4F type=write size=889850 2024-04-22T19:58:48.630 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.630+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=MSBI82WG290T type=del 2024-04-22T19:58:48.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.632+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=DYG2CKT234AT type=del size=2089480 2024-04-22T19:58:48.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.633+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=XAI23LBAC78I type=del 2024-04-22T19:58:48.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.635+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~PB8NAMPT90GR.0 size=1048576 2024-04-22T19:58:48.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.636+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~PB8NAMPT90GR.1 size=1048576 2024-04-22T19:58:48.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.637+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~PB8NAMPT90GR.2 size=70686 2024-04-22T19:58:48.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.637+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=PB8NAMPT90GR tag=2RC9T9EB3QPY type=write size=3216414 2024-04-22T19:58:48.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.639+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~SV0CK6M9X8ND.0 size=1048576 2024-04-22T19:58:48.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.639+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~SV0CK6M9X8ND.1 size=366821 2024-04-22T19:58:48.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.642+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=SV0CK6M9X8ND tag=E1D3BDKIKULE type=write size=2463973 2024-04-22T19:58:48.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.644+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~IMZV38VM31E0.0 size=1048576 2024-04-22T19:58:48.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.645+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~IMZV38VM31E0.1 size=1048576 2024-04-22T19:58:48.645 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.646+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~IMZV38VM31E0.2 size=785119 2024-04-22T19:58:48.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.646+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=IMZV38VM31E0 tag=NY3ZN9G07WHD type=write size=3930847 2024-04-22T19:58:48.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.647+0000 7f5269a8e780 -1 < failed to complete operation key=GP1C5ULUU143 tag=IBMAJ02NDYLV type=del size=3516899: (22) Invalid argument 2024-04-22T19:58:48.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.648+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=WYLHJLNOP22X type=del size=911360 2024-04-22T19:58:48.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.649+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=FE8IRML01Q5H type=del 2024-04-22T19:58:48.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.650+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=5PFTO0Z2FY3O type=write size=40641 2024-04-22T19:58:48.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.651+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~TPF85YMCVSXG.0 size=1048576 2024-04-22T19:58:48.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.652+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~TPF85YMCVSXG.1 size=1048576 2024-04-22T19:58:48.653 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.653+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~TPF85YMCVSXG.2 size=808560 2024-04-22T19:58:48.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.655+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=TPF85YMCVSXG tag=3J3FO92DKLZ2 type=write size=3954288 2024-04-22T19:58:48.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.657+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~TOFZ2PWAE0KB.0 size=1048576 2024-04-22T19:58:48.658 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.658+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~TOFZ2PWAE0KB.1 size=279958 2024-04-22T19:58:48.658 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.658+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=TOFZ2PWAE0KB tag=IF6OGIQU6S2N type=write size=2377110 2024-04-22T19:58:48.659 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.660+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~HL1VTKA10R9N.0 size=1048576 2024-04-22T19:58:48.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.660+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~HL1VTKA10R9N.1 size=1048576 2024-04-22T19:58:48.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.661+0000 7f5269a8e780 1 < completed part key=_multipart_DX5YUYQRZKWA.2~HL1VTKA10R9N.2 size=572446 2024-04-22T19:58:48.664 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.664+0000 7f5269a8e780 1 < completed multipart upload key=DX5YUYQRZKWA upload=HL1VTKA10R9N tag=SZV6WECF0535 type=write size=3718174 2024-04-22T19:58:48.665 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.665+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=R2SZGGH42GVO type=del size=2885001 2024-04-22T19:58:48.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.666+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=LWKTSELD6B3P type=del 2024-04-22T19:58:48.668 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.668+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~7E1ZFXMR0AT7.0 size=231488 2024-04-22T19:58:48.668 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.668+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=7E1ZFXMR0AT7 tag=B2XH82DKQIYH type=write size=1280064 2024-04-22T19:58:48.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.669+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=CHPI1YXUPI4V type=del 2024-04-22T19:58:48.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.670+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=TVEYRJ7R48IO type=del size=2131284 2024-04-22T19:58:48.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.672+0000 7f5269a8e780 1 < completed write operation key=8HFSDSNZONIN tag=58QQ4JQZP7MG type=write size=5133 2024-04-22T19:58:48.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.674+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~KUL01I61VMLJ.0 size=924661 2024-04-22T19:58:48.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.674+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=KUL01I61VMLJ tag=UA5GKWGX0Y3G type=write size=1973237 2024-04-22T19:58:48.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.675+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~I1YADYHK1YMR.0 size=1048576 2024-04-22T19:58:48.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.676+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~I1YADYHK1YMR.1 size=880743 2024-04-22T19:58:48.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.677+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=I1YADYHK1YMR tag=N9INXSJ735L2 type=write size=2977895 2024-04-22T19:58:48.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.678+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=3P3BSNRKD2SA type=del size=854708 2024-04-22T19:58:48.679 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.680+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~D5RPENLP5C3A.0 size=1048576 2024-04-22T19:58:48.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.680+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~D5RPENLP5C3A.1 size=1048576 2024-04-22T19:58:48.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.682+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~D5RPENLP5C3A.2 size=156118 2024-04-22T19:58:48.684 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.684+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=D5RPENLP5C3A tag=J7EB405JYBWH type=write size=3301846 2024-04-22T19:58:48.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.685+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=TXCW4T4SIV1Z type=del size=1264925 2024-04-22T19:58:48.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.686+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~V9FHDAWJWPDB.0 size=1048576 2024-04-22T19:58:48.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.687+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~V9FHDAWJWPDB.1 size=1048576 2024-04-22T19:58:48.688 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.688+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~V9FHDAWJWPDB.2 size=153310 2024-04-22T19:58:48.690 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.690+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=V9FHDAWJWPDB tag=WGLE090ZUPF4 type=write size=3299038 2024-04-22T19:58:48.692 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.692+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=DN4OUIBK229G type=del size=2954651 2024-04-22T19:58:48.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.693+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~DCUCRTDFW05V.0 size=673886 2024-04-22T19:58:48.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.695+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=DCUCRTDFW05V tag=N9TD56TU8KWL type=write size=1722462 2024-04-22T19:58:48.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.696+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=3O9T51AED94I type=del size=3380268 2024-04-22T19:58:48.697 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.697+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=H699C828V28R type=del 2024-04-22T19:58:48.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.699+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~JD56AQFKSH44.0 size=1048576 2024-04-22T19:58:48.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.700+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~JD56AQFKSH44.1 size=1048576 2024-04-22T19:58:48.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.701+0000 7f5269a8e780 1 > prepared part key=_multipart_SN2BZATJGUWA.2~JD56AQFKSH44.2 size=130652 2024-04-22T19:58:48.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.701+0000 7f5269a8e780 1 > prepared multipart upload key=SN2BZATJGUWA upload=JD56AQFKSH44 tag=GHIVXRKAYQ1W type=write size=3276380 2024-04-22T19:58:48.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.704+0000 7f5269a8e780 1 < suggested update operation key=HIAXPVKZ1IIO tag=7A1G233QXS4F type=write size=889850 2024-04-22T19:58:48.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.706+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~085K4XZ96AA2.0 size=1048576 2024-04-22T19:58:48.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.707+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~085K4XZ96AA2.1 size=1048576 2024-04-22T19:58:48.708 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.708+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~085K4XZ96AA2.2 size=525295 2024-04-22T19:58:48.708 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.708+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=085K4XZ96AA2 tag=S2KY8PO24AYZ type=write size=3671023 2024-04-22T19:58:48.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.709+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=DYG2CKT234AT type=del 2024-04-22T19:58:48.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.711+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~UOIRFCWPET0M.0 size=1048576 2024-04-22T19:58:48.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.712+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~UOIRFCWPET0M.1 size=1048576 2024-04-22T19:58:48.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.713+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~UOIRFCWPET0M.2 size=131454 2024-04-22T19:58:48.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.713+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=UOIRFCWPET0M tag=V7ENFN7VDUXP type=write size=3277182 2024-04-22T19:58:48.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.714+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~PB8NAMPT90GR.0 size=1048576 2024-04-22T19:58:48.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.714+0000 7f5269a8e780 1 < canceled part key=_multipart_EHRS6DQDRXVS.2~PB8NAMPT90GR.1 size=1048576 2024-04-22T19:58:48.715 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.715+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~PB8NAMPT90GR.2 size=70686 2024-04-22T19:58:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.717+0000 7f5269a8e780 1 < canceled multipart upload key=EHRS6DQDRXVS upload=PB8NAMPT90GR tag=2RC9T9EB3QPY type=write size=3216414 2024-04-22T19:58:48.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.718+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=CR574IJZBRNR type=del size=1786767 2024-04-22T19:58:48.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.719+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~IMZV38VM31E0.0 size=1048576 2024-04-22T19:58:48.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.720+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~IMZV38VM31E0.1 size=1048576 2024-04-22T19:58:48.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.720+0000 7f5269a8e780 1 < canceled part key=_multipart_5UBYMQT7ZWKX.2~IMZV38VM31E0.2 size=785119 2024-04-22T19:58:48.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.722+0000 7f5269a8e780 1 < canceled multipart upload key=5UBYMQT7ZWKX upload=IMZV38VM31E0 tag=NY3ZN9G07WHD type=write size=3930847 2024-04-22T19:58:48.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.723+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=HNKJHP6JFCP8 type=del size=978303 2024-04-22T19:58:48.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.724+0000 7f5269a8e780 1 < completed delete operation key=I91AD8FJ3P4T tag=WYLHJLNOP22X type=del 2024-04-22T19:58:48.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.725+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=P1CUR5F5AEDN type=del size=4140929 2024-04-22T19:58:48.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.727+0000 7f5269a8e780 1 < completed write operation key=HV8J6YM0YNC1 tag=5PFTO0Z2FY3O type=write size=40641 2024-04-22T19:58:48.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.728+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=LTJ32BWQZZB4 type=del size=527067 2024-04-22T19:58:48.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.729+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~TOFZ2PWAE0KB.0 size=1048576 2024-04-22T19:58:48.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.729+0000 7f5269a8e780 1 < canceled part key=_multipart_VPAVX5FXYWZO.2~TOFZ2PWAE0KB.1 size=279958 2024-04-22T19:58:48.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.731+0000 7f5269a8e780 1 < canceled multipart upload key=VPAVX5FXYWZO upload=TOFZ2PWAE0KB tag=IF6OGIQU6S2N type=write size=2377110 2024-04-22T19:58:48.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.732+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=D28OG0AH51ZO type=del size=3469390 2024-04-22T19:58:48.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.733+0000 7f5269a8e780 1 < canceled operation key=HV8J6YM0YNC1 tag=R2SZGGH42GVO type=del size=2885001 2024-04-22T19:58:48.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.734+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=9P7CR56GMM7N type=del size=4055828 2024-04-22T19:58:48.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.735+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~7E1ZFXMR0AT7.0 size=231488 2024-04-22T19:58:48.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.737+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=7E1ZFXMR0AT7 tag=B2XH82DKQIYH type=write size=1280064 2024-04-22T19:58:48.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.739+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~GR17WU4CRJ0Z.0 size=1048576 2024-04-22T19:58:48.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.740+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~GR17WU4CRJ0Z.1 size=201965 2024-04-22T19:58:48.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.740+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=GR17WU4CRJ0Z tag=DQ2YT0I94240 type=write size=2299117 2024-04-22T19:58:48.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.741+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=TVEYRJ7R48IO type=del 2024-04-22T19:58:48.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.743+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~FKAXUKQ9JI4E.0 size=1048576 2024-04-22T19:58:48.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.744+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~FKAXUKQ9JI4E.1 size=323041 2024-04-22T19:58:48.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.744+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=FKAXUKQ9JI4E tag=836C23HQB2XQ type=write size=2420193 2024-04-22T19:58:48.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.744+0000 7f5269a8e780 1 < canceled part key=_multipart_86GW6HV9G1IZ.2~KUL01I61VMLJ.0 size=924661 2024-04-22T19:58:48.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.746+0000 7f5269a8e780 1 < canceled multipart upload key=86GW6HV9G1IZ upload=KUL01I61VMLJ tag=UA5GKWGX0Y3G type=write size=1973237 2024-04-22T19:58:48.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.747+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=RCEAOEYXUQP7 type=del size=2763971 2024-04-22T19:58:48.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.748+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=3P3BSNRKD2SA type=del 2024-04-22T19:58:48.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.750+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~52JQD5V5MIIE.0 size=327418 2024-04-22T19:58:48.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.750+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=52JQD5V5MIIE tag=SGRQ1G8JW499 type=write size=1375994 2024-04-22T19:58:48.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.752+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=TXCW4T4SIV1Z type=del 2024-04-22T19:58:48.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.754+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~CURZIX377L1Y.0 size=178402 2024-04-22T19:58:48.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.754+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=CURZIX377L1Y tag=RQH1O9GB2JC5 type=write size=1226978 2024-04-22T19:58:48.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.755+0000 7f5269a8e780 1 < canceled operation key=A853MOZPT9BG tag=DN4OUIBK229G type=del size=2954651 2024-04-22T19:58:48.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.756+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=A305T5YWBMCI type=del size=1340749 2024-04-22T19:58:48.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.757+0000 7f5269a8e780 -1 < failed to complete operation key=HIAXPVKZ1IIO tag=3O9T51AED94I type=del size=3380268: (22) Invalid argument 2024-04-22T19:58:48.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.759+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~2EOGKW80O1V3.0 size=1048576 2024-04-22T19:58:48.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.760+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~2EOGKW80O1V3.1 size=1048576 2024-04-22T19:58:48.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.761+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~2EOGKW80O1V3.2 size=300529 2024-04-22T19:58:48.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.761+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=2EOGKW80O1V3 tag=G7FC3YER93BB type=write size=3446257 2024-04-22T19:58:48.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.762+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~JD56AQFKSH44.0 size=1048576 2024-04-22T19:58:48.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.763+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~JD56AQFKSH44.1 size=1048576 2024-04-22T19:58:48.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.764+0000 7f5269a8e780 1 < completed part key=_multipart_SN2BZATJGUWA.2~JD56AQFKSH44.2 size=130652 2024-04-22T19:58:48.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.766+0000 7f5269a8e780 1 < completed multipart upload key=SN2BZATJGUWA upload=JD56AQFKSH44 tag=GHIVXRKAYQ1W type=write size=3276380 2024-04-22T19:58:48.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.768+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~9WGIK6225LRZ.0 size=1048576 2024-04-22T19:58:48.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.769+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~9WGIK6225LRZ.1 size=953652 2024-04-22T19:58:48.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.769+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=9WGIK6225LRZ tag=YJJRNK7AQMPP type=write size=3050804 2024-04-22T19:58:48.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.769+0000 7f5269a8e780 1 < canceled part key=_multipart_RY1O7X1CWZMK.2~085K4XZ96AA2.0 size=1048576 2024-04-22T19:58:48.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.770+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~085K4XZ96AA2.1 size=1048576 2024-04-22T19:58:48.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.771+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~085K4XZ96AA2.2 size=525295 2024-04-22T19:58:48.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.773+0000 7f5269a8e780 1 < canceled multipart upload key=RY1O7X1CWZMK upload=085K4XZ96AA2 tag=S2KY8PO24AYZ type=write size=3671023 2024-04-22T19:58:48.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.774+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=74MWB8HRZUUA type=write size=1002430 2024-04-22T19:58:48.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.775+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~UOIRFCWPET0M.0 size=1048576 2024-04-22T19:58:48.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.775+0000 7f5269a8e780 1 < canceled part key=_multipart_Y5HR8XW4L0SP.2~UOIRFCWPET0M.1 size=1048576 2024-04-22T19:58:48.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.776+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~UOIRFCWPET0M.2 size=131454 2024-04-22T19:58:48.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.778+0000 7f5269a8e780 1 < canceled multipart upload key=Y5HR8XW4L0SP upload=UOIRFCWPET0M tag=V7ENFN7VDUXP type=write size=3277182 2024-04-22T19:58:48.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.779+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=26BATXSAIX5I type=del size=2449598 2024-04-22T19:58:48.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.780+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=CR574IJZBRNR type=del 2024-04-22T19:58:48.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.782+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~HY1MHMY53V0E.0 size=1048576 2024-04-22T19:58:48.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.783+0000 7f5269a8e780 1 > prepared part key=_multipart_86GW6HV9G1IZ.2~HY1MHMY53V0E.1 size=42332 2024-04-22T19:58:48.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.783+0000 7f5269a8e780 1 > prepared multipart upload key=86GW6HV9G1IZ upload=HY1MHMY53V0E tag=58U25ZZJYNRX type=write size=2139484 2024-04-22T19:58:48.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.784+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=HNKJHP6JFCP8 type=del 2024-04-22T19:58:48.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.785+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=N9Z725DUJ9RK type=del size=433132 2024-04-22T19:58:48.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.787+0000 7f5269a8e780 1 < suggested remove operation key=SN2BZATJGUWA tag=P1CUR5F5AEDN type=del 2024-04-22T19:58:48.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.789+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~U18GU33E8D0P.0 size=1048576 2024-04-22T19:58:48.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.792+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~U18GU33E8D0P.1 size=1048576 2024-04-22T19:58:48.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.793+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~U18GU33E8D0P.2 size=15851 2024-04-22T19:58:48.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.793+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=U18GU33E8D0P tag=T5NZ455XFXW2 type=write size=3161579 2024-04-22T19:58:48.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.794+0000 7f5269a8e780 1 < canceled operation key=KV5DYQ49ZKCX tag=LTJ32BWQZZB4 type=del size=527067 2024-04-22T19:58:48.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.795+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=QTAEUAHX6E0D type=write size=515837 2024-04-22T19:58:48.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.796+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=D28OG0AH51ZO type=del 2024-04-22T19:58:48.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.799+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~4VSUWD4AK03D.0 size=1048576 2024-04-22T19:58:48.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.802+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~4VSUWD4AK03D.1 size=1048576 2024-04-22T19:58:48.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.803+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~4VSUWD4AK03D.2 size=948059 2024-04-22T19:58:48.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.803+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=4VSUWD4AK03D tag=E5GKBGHOBYV6 type=write size=4093787 2024-04-22T19:58:48.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.805+0000 7f5269a8e780 1 < suggested update operation key=A853MOZPT9BG tag=9P7CR56GMM7N type=del size=4055828 2024-04-22T19:58:48.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.806+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=QWQ5IZ8OIZ98 type=del size=969225 2024-04-22T19:58:48.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.807+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~GR17WU4CRJ0Z.0 size=1048576 2024-04-22T19:58:48.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.808+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~GR17WU4CRJ0Z.1 size=201965 2024-04-22T19:58:48.810 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.810+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=GR17WU4CRJ0Z tag=DQ2YT0I94240 type=write size=2299117 2024-04-22T19:58:48.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.811+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=B8UG7ZHI7U1P type=del size=1639084 2024-04-22T19:58:48.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.812+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~FKAXUKQ9JI4E.0 size=1048576 2024-04-22T19:58:48.813 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.813+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~FKAXUKQ9JI4E.1 size=323041 2024-04-22T19:58:48.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.815+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=FKAXUKQ9JI4E tag=836C23HQB2XQ type=write size=2420193 2024-04-22T19:58:48.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.816+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=IN2XTGZPMUG3 type=del size=3986962 2024-04-22T19:58:48.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.817+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=RCEAOEYXUQP7 type=del 2024-04-22T19:58:48.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.818+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=HJJ8N8RW4EVE type=del size=1373354 2024-04-22T19:58:48.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.819+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~52JQD5V5MIIE.0 size=327418 2024-04-22T19:58:48.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.821+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=52JQD5V5MIIE tag=SGRQ1G8JW499 type=write size=1375994 2024-04-22T19:58:48.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.822+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=A5YP92LGVEBF type=write size=444350 2024-04-22T19:58:48.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.823+0000 7f5269a8e780 1 < canceled part key=_multipart_Q1UKF4UMJ2HO.2~CURZIX377L1Y.0 size=178402 2024-04-22T19:58:48.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.825+0000 7f5269a8e780 1 < canceled multipart upload key=Q1UKF4UMJ2HO upload=CURZIX377L1Y tag=RQH1O9GB2JC5 type=write size=1226978 2024-04-22T19:58:48.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.826+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=3Z590BIA2F4H type=del size=2249119 2024-04-22T19:58:48.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.827+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=A305T5YWBMCI type=del 2024-04-22T19:58:48.828 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.828+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=1KQENO208S7V type=del size=2502210 2024-04-22T19:58:48.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.829+0000 7f5269a8e780 1 < canceled part key=_multipart_Q1UKF4UMJ2HO.2~2EOGKW80O1V3.0 size=1048576 2024-04-22T19:58:48.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.830+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~2EOGKW80O1V3.1 size=1048576 2024-04-22T19:58:48.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.831+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~2EOGKW80O1V3.2 size=300529 2024-04-22T19:58:48.833 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.833+0000 7f5269a8e780 1 < canceled multipart upload key=Q1UKF4UMJ2HO upload=2EOGKW80O1V3 tag=G7FC3YER93BB type=write size=3446257 2024-04-22T19:58:48.834 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.834+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=EAS9DNRR73HM type=write size=270807 2024-04-22T19:58:48.834 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.834+0000 7f5269a8e780 1 < canceled part key=_multipart_637M7AP8X9E9.2~9WGIK6225LRZ.0 size=1048576 2024-04-22T19:58:48.835 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.835+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~9WGIK6225LRZ.1 size=953652 2024-04-22T19:58:48.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.837+0000 7f5269a8e780 1 < canceled multipart upload key=637M7AP8X9E9 upload=9WGIK6225LRZ tag=YJJRNK7AQMPP type=write size=3050804 2024-04-22T19:58:48.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.839+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~4FFDF20Y6AIS.0 size=1048576 2024-04-22T19:58:48.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.840+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~4FFDF20Y6AIS.1 size=1048576 2024-04-22T19:58:48.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.841+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~4FFDF20Y6AIS.2 size=956944 2024-04-22T19:58:48.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.841+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=4FFDF20Y6AIS tag=4PBXN1AE0O8V type=write size=4102672 2024-04-22T19:58:48.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.843+0000 7f5269a8e780 1 < completed write operation key=KE7YGPTF1MOR tag=74MWB8HRZUUA type=write size=1002430 2024-04-22T19:58:48.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.844+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=CG2ZX4LVVEUF type=del size=3462153 2024-04-22T19:58:48.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.845+0000 7f5269a8e780 1 < canceled operation key=4GX83ZJWZCD1 tag=26BATXSAIX5I type=del size=2449598 2024-04-22T19:58:48.846 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.846+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=3NGTBJGZHW56 type=del size=955463 2024-04-22T19:58:48.847 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.848+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~HY1MHMY53V0E.0 size=1048576 2024-04-22T19:58:48.848 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.848+0000 7f5269a8e780 1 < completed part key=_multipart_86GW6HV9G1IZ.2~HY1MHMY53V0E.1 size=42332 2024-04-22T19:58:48.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.850+0000 7f5269a8e780 1 < completed multipart upload key=86GW6HV9G1IZ upload=HY1MHMY53V0E tag=58U25ZZJYNRX type=write size=2139484 2024-04-22T19:58:48.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.852+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=BKYJGXCM5B3G type=del size=1572759 2024-04-22T19:58:48.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.853+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=N9Z725DUJ9RK type=del 2024-04-22T19:58:48.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.854+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=UFJS5F3CMIDU type=del size=64013 2024-04-22T19:58:48.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.854+0000 7f5269a8e780 1 < canceled part key=_multipart_B0Y7ZTPC0975.2~U18GU33E8D0P.0 size=1048576 2024-04-22T19:58:48.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.855+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~U18GU33E8D0P.1 size=1048576 2024-04-22T19:58:48.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.856+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~U18GU33E8D0P.2 size=15851 2024-04-22T19:58:48.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.858+0000 7f5269a8e780 1 < canceled multipart upload key=B0Y7ZTPC0975 upload=U18GU33E8D0P tag=T5NZ455XFXW2 type=write size=3161579 2024-04-22T19:58:48.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.859+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=VMJHN88XZHYS type=write size=986033 2024-04-22T19:58:48.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.861+0000 7f5269a8e780 1 < completed write operation key=70MLQGJPQYPY tag=QTAEUAHX6E0D type=write size=515837 2024-04-22T19:58:48.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.862+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=ES7P2NLSBOBY type=del size=692186 2024-04-22T19:58:48.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.864+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~4VSUWD4AK03D.0 size=1048576 2024-04-22T19:58:48.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.865+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~4VSUWD4AK03D.1 size=1048576 2024-04-22T19:58:48.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.865+0000 7f5269a8e780 1 < canceled part key=_multipart_M9M4SEEEZSFV.2~4VSUWD4AK03D.2 size=948059 2024-04-22T19:58:48.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.867+0000 7f5269a8e780 1 < canceled multipart upload key=M9M4SEEEZSFV upload=4VSUWD4AK03D tag=E5GKBGHOBYV6 type=write size=4093787 2024-04-22T19:58:48.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.868+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=V0RXA9TTWRXY type=write size=733973 2024-04-22T19:58:48.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.869+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=QWQ5IZ8OIZ98 type=del 2024-04-22T19:58:48.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.871+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~J0XPEBE64QEX.0 size=1048576 2024-04-22T19:58:48.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.872+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~J0XPEBE64QEX.1 size=38645 2024-04-22T19:58:48.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.872+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=J0XPEBE64QEX tag=0LZAU6X9BELI type=write size=2135797 2024-04-22T19:58:48.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.874+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=B8UG7ZHI7U1P type=del 2024-04-22T19:58:48.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.875+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=0DEXUIVYQUBI type=del size=53047 2024-04-22T19:58:48.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.877+0000 7f5269a8e780 1 < suggested update operation key=4GX83ZJWZCD1 tag=IN2XTGZPMUG3 type=del size=3986962 2024-04-22T19:58:48.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.878+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=V1YMN9BGEQWF type=del size=523828 2024-04-22T19:58:48.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.880+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=HJJ8N8RW4EVE type=del 2024-04-22T19:58:48.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.881+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=MEANJLAJFOD8 type=del size=55169 2024-04-22T19:58:48.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.882+0000 7f5269a8e780 1 < completed write operation key=KV5DYQ49ZKCX tag=A5YP92LGVEBF type=write size=444350 2024-04-22T19:58:48.884 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.884+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=ANLRSI0Z7YH1 type=del size=1473038 2024-04-22T19:58:48.885 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.885+0000 7f5269a8e780 1 < canceled operation key=GP1C5ULUU143 tag=3Z590BIA2F4H type=del size=2249119 2024-04-22T19:58:48.887 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.887+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~DIOXABJVM8NR.0 size=1048576 2024-04-22T19:58:48.888 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.888+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~DIOXABJVM8NR.1 size=1048576 2024-04-22T19:58:48.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.889+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~DIOXABJVM8NR.2 size=175766 2024-04-22T19:58:48.889 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.889+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=DIOXABJVM8NR tag=4EW8L7Y5R7N6 type=write size=3321494 2024-04-22T19:58:48.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.890+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=1KQENO208S7V type=del 2024-04-22T19:58:48.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.892+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~UGJOUH2RPEZ1.0 size=1048576 2024-04-22T19:58:48.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.893+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~UGJOUH2RPEZ1.1 size=1048576 2024-04-22T19:58:48.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.894+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~UGJOUH2RPEZ1.2 size=478363 2024-04-22T19:58:48.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.894+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=UGJOUH2RPEZ1 tag=4NUDJOVQFKYK type=write size=3624091 2024-04-22T19:58:48.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.895+0000 7f5269a8e780 1 < completed write operation key=T2BV2LUCY4E3 tag=EAS9DNRR73HM type=write size=270807 2024-04-22T19:58:48.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.897+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=7W33Y7HJ3N8A type=del size=3439444 2024-04-22T19:58:48.898 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.898+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~4FFDF20Y6AIS.0 size=1048576 2024-04-22T19:58:48.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.901+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~4FFDF20Y6AIS.1 size=1048576 2024-04-22T19:58:48.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.902+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~4FFDF20Y6AIS.2 size=956944 2024-04-22T19:58:48.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.904+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=4FFDF20Y6AIS tag=4PBXN1AE0O8V type=write size=4102672 2024-04-22T19:58:48.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.905+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=IDUUKVE2FGYI type=del size=2809643 2024-04-22T19:58:48.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.906+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=CG2ZX4LVVEUF type=del 2024-04-22T19:58:48.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.907+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=10NW00MM1296 type=del size=2667471 2024-04-22T19:58:48.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.908+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=3NGTBJGZHW56 type=del 2024-04-22T19:58:48.909 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.909+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=EP1AU4Y81LD7 type=write size=987176 2024-04-22T19:58:48.910 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.910+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=BKYJGXCM5B3G type=del 2024-04-22T19:58:48.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.911+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=2PE0PIT1AL52 type=del size=2398971 2024-04-22T19:58:48.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.917+0000 7f5269a8e780 1 < suggested update operation key=ZC4ZOHUJ3HQD tag=UFJS5F3CMIDU type=del size=64013 2024-04-22T19:58:48.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.919+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=1C7FCPJZW17L type=write size=955744 2024-04-22T19:58:48.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.921+0000 7f5269a8e780 1 < suggested remove operation key=I91AD8FJ3P4T tag=VMJHN88XZHYS type=write 2024-04-22T19:58:48.923 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.923+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~KRAKPIBGB2VH.0 size=1048576 2024-04-22T19:58:48.924 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.924+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~KRAKPIBGB2VH.1 size=1048576 2024-04-22T19:58:48.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.925+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~KRAKPIBGB2VH.2 size=404694 2024-04-22T19:58:48.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.925+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=KRAKPIBGB2VH tag=R6UR70201MAE type=write size=3550422 2024-04-22T19:58:48.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.927+0000 7f5269a8e780 1 < suggested update operation key=GX6NILQG878S tag=ES7P2NLSBOBY type=del size=692186 2024-04-22T19:58:48.928 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.928+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=8HQVYM9NYJQH type=del size=135637 2024-04-22T19:58:48.929 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.929+0000 7f5269a8e780 1 < completed write operation key=Y5HR8XW4L0SP tag=V0RXA9TTWRXY type=write size=733973 2024-04-22T19:58:48.930 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.930+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=V0RPPNDAWLQ4 type=del size=3339915 2024-04-22T19:58:48.931 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.932+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~J0XPEBE64QEX.0 size=1048576 2024-04-22T19:58:48.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.932+0000 7f5269a8e780 1 < canceled part key=_multipart_8HFSDSNZONIN.2~J0XPEBE64QEX.1 size=38645 2024-04-22T19:58:48.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.933+0000 7f5269a8e780 1 < canceled multipart upload key=8HFSDSNZONIN upload=J0XPEBE64QEX tag=0LZAU6X9BELI type=write size=2135797 2024-04-22T19:58:48.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.934+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=5P27T3F7XG1Q type=del size=1084108 2024-04-22T19:58:48.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.936+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=0DEXUIVYQUBI type=del 2024-04-22T19:58:48.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.937+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=O0IQKTG8MQX4 type=del size=4058081 2024-04-22T19:58:48.938 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.938+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=V1YMN9BGEQWF type=del 2024-04-22T19:58:48.939 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.939+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=01OU9DWO6204 type=del size=1833816 2024-04-22T19:58:48.940 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.940+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=MEANJLAJFOD8 type=del 2024-04-22T19:58:48.941 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.941+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=C3A1WHIN3FW7 type=del size=973593 2024-04-22T19:58:48.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.942+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=ANLRSI0Z7YH1 type=del 2024-04-22T19:58:48.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.943+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=H7L4HYDJMSAW type=del size=1096284 2024-04-22T19:58:48.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.944+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~DIOXABJVM8NR.0 size=1048576 2024-04-22T19:58:48.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.945+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~DIOXABJVM8NR.1 size=1048576 2024-04-22T19:58:48.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.946+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~DIOXABJVM8NR.2 size=175766 2024-04-22T19:58:48.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.948+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=DIOXABJVM8NR tag=4EW8L7Y5R7N6 type=write size=3321494 2024-04-22T19:58:48.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.950+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~PIGSGMHB3SOT.0 size=1048576 2024-04-22T19:58:48.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.951+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~PIGSGMHB3SOT.1 size=297843 2024-04-22T19:58:48.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.951+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=PIGSGMHB3SOT tag=2G04JF6N213T type=write size=2394995 2024-04-22T19:58:48.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.952+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~UGJOUH2RPEZ1.0 size=1048576 2024-04-22T19:58:48.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.953+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~UGJOUH2RPEZ1.1 size=1048576 2024-04-22T19:58:48.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.954+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~UGJOUH2RPEZ1.2 size=478363 2024-04-22T19:58:48.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.956+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=UGJOUH2RPEZ1 tag=4NUDJOVQFKYK type=write size=3624091 2024-04-22T19:58:48.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.958+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~JTONYAL6OTFR.0 size=1048576 2024-04-22T19:58:48.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.959+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~JTONYAL6OTFR.1 size=108912 2024-04-22T19:58:48.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.959+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=JTONYAL6OTFR tag=X6W0WGJOTQOS type=write size=2206064 2024-04-22T19:58:48.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.961+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=7W33Y7HJ3N8A type=del 2024-04-22T19:58:48.962 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.962+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=I6XN449RTMLT type=write size=114672 2024-04-22T19:58:48.963 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.963+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=IDUUKVE2FGYI type=del 2024-04-22T19:58:48.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.965+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~TDYR1FUOIT4N.0 size=1048576 2024-04-22T19:58:48.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.966+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~TDYR1FUOIT4N.1 size=1048576 2024-04-22T19:58:48.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.967+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~TDYR1FUOIT4N.2 size=389189 2024-04-22T19:58:48.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.967+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=TDYR1FUOIT4N tag=WU2QI2Z7TQOF type=write size=3534917 2024-04-22T19:58:48.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.969+0000 7f5269a8e780 1 < suggested update operation key=RY1O7X1CWZMK tag=10NW00MM1296 type=del size=2667471 2024-04-22T19:58:48.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.970+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=BN3MSN6E75T1 type=write size=657099 2024-04-22T19:58:48.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.971+0000 7f5269a8e780 -1 < failed to complete operation key=GX6NILQG878S tag=EP1AU4Y81LD7 type=write size=987176: (22) Invalid argument 2024-04-22T19:58:48.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.972+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=SUR0I5OFCMNM type=del size=3706655 2024-04-22T19:58:48.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.974+0000 7f5269a8e780 1 < completed delete operation key=A853MOZPT9BG tag=2PE0PIT1AL52 type=del 2024-04-22T19:58:48.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.975+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=RDN5HYUDBFEC type=del size=3913790 2024-04-22T19:58:48.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.976+0000 7f5269a8e780 1 < completed write operation key=M9M4SEEEZSFV tag=1C7FCPJZW17L type=write size=955744 2024-04-22T19:58:48.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.977+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=PO32PQYVX5HE type=del size=1788525 2024-04-22T19:58:48.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.979+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~KRAKPIBGB2VH.0 size=1048576 2024-04-22T19:58:48.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.980+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~KRAKPIBGB2VH.1 size=1048576 2024-04-22T19:58:48.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.981+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~KRAKPIBGB2VH.2 size=404694 2024-04-22T19:58:48.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.983+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=KRAKPIBGB2VH tag=R6UR70201MAE type=write size=3550422 2024-04-22T19:58:48.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.984+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=SY1QT9YT1UQ7 type=del size=3154163 2024-04-22T19:58:48.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.985+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=8HQVYM9NYJQH type=del 2024-04-22T19:58:48.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.987+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~ZTMC06LU0102.0 size=1048576 2024-04-22T19:58:48.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.988+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~ZTMC06LU0102.1 size=1048576 2024-04-22T19:58:48.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.988+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~ZTMC06LU0102.2 size=599823 2024-04-22T19:58:48.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.988+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=ZTMC06LU0102 tag=C12HATDZNMJE type=write size=3745551 2024-04-22T19:58:48.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.990+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=V0RPPNDAWLQ4 type=del 2024-04-22T19:58:48.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.991+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=66BKQUCD43FI type=write size=936800 2024-04-22T19:58:48.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.992+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=5P27T3F7XG1Q type=del 2024-04-22T19:58:48.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.993+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=S5IMJ4AEZOON type=del size=2000853 2024-04-22T19:58:48.994 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.994+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=O0IQKTG8MQX4 type=del 2024-04-22T19:58:48.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.996+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~63FOK383VV9Z.0 size=1048576 2024-04-22T19:58:48.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.996+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~63FOK383VV9Z.1 size=1048576 2024-04-22T19:58:48.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.997+0000 7f5269a8e780 1 > prepared part key=_multipart_B0Y7ZTPC0975.2~63FOK383VV9Z.2 size=94174 2024-04-22T19:58:48.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.997+0000 7f5269a8e780 1 > prepared multipart upload key=B0Y7ZTPC0975 upload=63FOK383VV9Z tag=T8XXG8K2BZK7 type=write size=3239902 2024-04-22T19:58:48.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:48.999+0000 7f5269a8e780 1 < suggested remove operation key=T2BV2LUCY4E3 tag=01OU9DWO6204 type=del 2024-04-22T19:58:49.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.001+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~M7ULVT9HCMRM.0 size=1048576 2024-04-22T19:58:49.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.002+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~M7ULVT9HCMRM.1 size=587548 2024-04-22T19:58:49.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.002+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=M7ULVT9HCMRM tag=R03TPLG3C3HT type=write size=2684700 2024-04-22T19:58:49.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.003+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=C3A1WHIN3FW7 type=del 2024-04-22T19:58:49.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.005+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~E3469NKWXZWX.0 size=1048576 2024-04-22T19:58:49.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.006+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~E3469NKWXZWX.1 size=1048576 2024-04-22T19:58:49.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.007+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~E3469NKWXZWX.2 size=753896 2024-04-22T19:58:49.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.007+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=E3469NKWXZWX tag=UB6R6LKA8NCW type=write size=3899624 2024-04-22T19:58:49.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.008+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=H7L4HYDJMSAW type=del 2024-04-22T19:58:49.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.010+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~B2ZGU9W40LWD.0 size=1048576 2024-04-22T19:58:49.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.011+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~B2ZGU9W40LWD.1 size=1048576 2024-04-22T19:58:49.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.011+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~B2ZGU9W40LWD.2 size=650704 2024-04-22T19:58:49.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.011+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=B2ZGU9W40LWD tag=2N6WB3BABD5Y type=write size=3796432 2024-04-22T19:58:49.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.013+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~PIGSGMHB3SOT.0 size=1048576 2024-04-22T19:58:49.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.014+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~PIGSGMHB3SOT.1 size=297843 2024-04-22T19:58:49.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.015+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=PIGSGMHB3SOT tag=2G04JF6N213T type=write size=2394995 2024-04-22T19:58:49.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.016+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=CWOSFW3ZNRGN type=del size=2941666 2024-04-22T19:58:49.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.018+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~JTONYAL6OTFR.0 size=1048576 2024-04-22T19:58:49.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.018+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~JTONYAL6OTFR.1 size=108912 2024-04-22T19:58:49.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.021+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=JTONYAL6OTFR tag=X6W0WGJOTQOS type=write size=2206064 2024-04-22T19:58:49.023 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.023+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~08SHCSP21ACX.0 size=1048576 2024-04-22T19:58:49.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.024+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~08SHCSP21ACX.1 size=36054 2024-04-22T19:58:49.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.024+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=08SHCSP21ACX tag=AIPH3JSSPD37 type=write size=2133206 2024-04-22T19:58:49.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.026+0000 7f5269a8e780 1 < suggested remove operation key=DX5YUYQRZKWA tag=I6XN449RTMLT type=write 2024-04-22T19:58:49.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.027+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=15GG29H3RHWA type=del size=3622589 2024-04-22T19:58:49.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.028+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~TDYR1FUOIT4N.0 size=1048576 2024-04-22T19:58:49.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.029+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~TDYR1FUOIT4N.1 size=1048576 2024-04-22T19:58:49.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.030+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~TDYR1FUOIT4N.2 size=389189 2024-04-22T19:58:49.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.032+0000 7f5269a8e780 1 < completed multipart upload key=M9M4SEEEZSFV upload=TDYR1FUOIT4N tag=WU2QI2Z7TQOF type=write size=3534917 2024-04-22T19:58:49.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.034+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=4W6SDQUZTUIR type=write size=423512 2024-04-22T19:58:49.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.035+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=BN3MSN6E75T1 type=write size=657099 2024-04-22T19:58:49.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.037+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~2MWITF4X7E90.0 size=1048576 2024-04-22T19:58:49.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.038+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~2MWITF4X7E90.1 size=1048576 2024-04-22T19:58:49.038 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.038+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~2MWITF4X7E90.2 size=257248 2024-04-22T19:58:49.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.038+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=2MWITF4X7E90 tag=45O1EM3SI9TI type=write size=3402976 2024-04-22T19:58:49.039 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.040+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=SUR0I5OFCMNM type=del 2024-04-22T19:58:49.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.042+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~9AAQ38V3OG55.0 size=1048576 2024-04-22T19:58:49.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.042+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~9AAQ38V3OG55.1 size=1048576 2024-04-22T19:58:49.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.043+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~9AAQ38V3OG55.2 size=990207 2024-04-22T19:58:49.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.043+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=9AAQ38V3OG55 tag=QVZAD6K3U9M8 type=write size=4135935 2024-04-22T19:58:49.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.045+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=RDN5HYUDBFEC type=del 2024-04-22T19:58:49.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.046+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=C7VM5MYBJORJ type=del size=2229710 2024-04-22T19:58:49.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.048+0000 7f5269a8e780 1 < suggested remove operation key=SN2BZATJGUWA tag=PO32PQYVX5HE type=del 2024-04-22T19:58:49.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.050+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~8U56O7TA028F.0 size=1048576 2024-04-22T19:58:49.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.051+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~8U56O7TA028F.1 size=463579 2024-04-22T19:58:49.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.051+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=8U56O7TA028F tag=UY8H1AGQ8X3Y type=write size=2560731 2024-04-22T19:58:49.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.052+0000 7f5269a8e780 1 < canceled operation key=ZC4ZOHUJ3HQD tag=SY1QT9YT1UQ7 type=del size=3154163 2024-04-22T19:58:49.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.053+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=PZRZ126JARSF type=del size=2852790 2024-04-22T19:58:49.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.054+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~ZTMC06LU0102.0 size=1048576 2024-04-22T19:58:49.054 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.054+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~ZTMC06LU0102.1 size=1048576 2024-04-22T19:58:49.054 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.054+0000 7f5269a8e780 1 < canceled part key=_multipart_AMBJHMJGLUUS.2~ZTMC06LU0102.2 size=599823 2024-04-22T19:58:49.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.057+0000 7f5269a8e780 1 < canceled multipart upload key=AMBJHMJGLUUS upload=ZTMC06LU0102 tag=C12HATDZNMJE type=write size=3745551 2024-04-22T19:58:49.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.058+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=0ZCCO01QNBSB type=del size=187589 2024-04-22T19:58:49.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.060+0000 7f5269a8e780 1 < suggested remove operation key=KE7YGPTF1MOR tag=66BKQUCD43FI type=write 2024-04-22T19:58:49.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.062+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~KN183V8IVZXI.0 size=1048576 2024-04-22T19:58:49.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.063+0000 7f5269a8e780 1 > prepared part key=_multipart_IBH6W1NINUHC.2~KN183V8IVZXI.1 size=870639 2024-04-22T19:58:49.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.063+0000 7f5269a8e780 1 > prepared multipart upload key=IBH6W1NINUHC upload=KN183V8IVZXI tag=ABIE36Y23SP3 type=write size=2967791 2024-04-22T19:58:49.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.064+0000 7f5269a8e780 1 < canceled operation key=AMBJHMJGLUUS tag=S5IMJ4AEZOON type=del size=2000853 2024-04-22T19:58:49.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.065+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=CZLSPNJJ0FV0 type=write size=805386 2024-04-22T19:58:49.066 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.067+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~63FOK383VV9Z.0 size=1048576 2024-04-22T19:58:49.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.067+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~63FOK383VV9Z.1 size=1048576 2024-04-22T19:58:49.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.068+0000 7f5269a8e780 1 < completed part key=_multipart_B0Y7ZTPC0975.2~63FOK383VV9Z.2 size=94174 2024-04-22T19:58:49.070 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.070+0000 7f5269a8e780 1 < completed multipart upload key=B0Y7ZTPC0975 upload=63FOK383VV9Z tag=T8XXG8K2BZK7 type=write size=3239902 2024-04-22T19:58:49.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.072+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=52CDZGEZCD4E type=write size=912390 2024-04-22T19:58:49.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.073+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~M7ULVT9HCMRM.0 size=1048576 2024-04-22T19:58:49.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.074+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~M7ULVT9HCMRM.1 size=587548 2024-04-22T19:58:49.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.076+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=M7ULVT9HCMRM tag=R03TPLG3C3HT type=write size=2684700 2024-04-22T19:58:49.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.077+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=YTSQSWBCDOS3 type=write size=849231 2024-04-22T19:58:49.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.078+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~E3469NKWXZWX.0 size=1048576 2024-04-22T19:58:49.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.079+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~E3469NKWXZWX.1 size=1048576 2024-04-22T19:58:49.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.080+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~E3469NKWXZWX.2 size=753896 2024-04-22T19:58:49.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.082+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=E3469NKWXZWX tag=UB6R6LKA8NCW type=write size=3899624 2024-04-22T19:58:49.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.083+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=A1V0VYKCBAGG type=del size=725624 2024-04-22T19:58:49.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.084+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~B2ZGU9W40LWD.0 size=1048576 2024-04-22T19:58:49.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.085+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~B2ZGU9W40LWD.1 size=1048576 2024-04-22T19:58:49.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.086+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~B2ZGU9W40LWD.2 size=650704 2024-04-22T19:58:49.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.088+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=B2ZGU9W40LWD tag=2N6WB3BABD5Y type=write size=3796432 2024-04-22T19:58:49.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.089+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=QHYVX4AU7MCU type=write size=554605 2024-04-22T19:58:49.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.091+0000 7f5269a8e780 1 < canceled operation key=637M7AP8X9E9 tag=CWOSFW3ZNRGN type=del size=2941666 2024-04-22T19:58:49.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.092+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=KO0A1VY02WC9 type=del size=1134712 2024-04-22T19:58:49.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.092+0000 7f5269a8e780 1 < canceled part key=_multipart_ZC4ZOHUJ3HQD.2~08SHCSP21ACX.0 size=1048576 2024-04-22T19:58:49.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.093+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~08SHCSP21ACX.1 size=36054 2024-04-22T19:58:49.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.095+0000 7f5269a8e780 1 < canceled multipart upload key=ZC4ZOHUJ3HQD upload=08SHCSP21ACX tag=AIPH3JSSPD37 type=write size=2133206 2024-04-22T19:58:49.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.097+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~XNU3NMOXW910.0 size=78260 2024-04-22T19:58:49.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.097+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=XNU3NMOXW910 tag=JY9U4Y579I59 type=write size=1126836 2024-04-22T19:58:49.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.098+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=15GG29H3RHWA type=del 2024-04-22T19:58:49.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.101+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~I752UOT06VQ8.0 size=1048576 2024-04-22T19:58:49.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.102+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~I752UOT06VQ8.1 size=140896 2024-04-22T19:58:49.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.102+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=I752UOT06VQ8 tag=CL1O6KGVFIUH type=write size=2238048 2024-04-22T19:58:49.103 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.103+0000 7f5269a8e780 1 < completed write operation key=GX6NILQG878S tag=4W6SDQUZTUIR type=write size=423512 2024-04-22T19:58:49.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.105+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~KIWH27YS83KX.0 size=855633 2024-04-22T19:58:49.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.105+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=KIWH27YS83KX tag=BF9GNTOA1YJF type=write size=1904209 2024-04-22T19:58:49.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.106+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~2MWITF4X7E90.0 size=1048576 2024-04-22T19:58:49.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.107+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~2MWITF4X7E90.1 size=1048576 2024-04-22T19:58:49.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.108+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~2MWITF4X7E90.2 size=257248 2024-04-22T19:58:49.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.110+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=2MWITF4X7E90 tag=45O1EM3SI9TI type=write size=3402976 2024-04-22T19:58:49.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.113+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~N2ZP0LDGWPML.0 size=1048576 2024-04-22T19:58:49.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.114+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~N2ZP0LDGWPML.1 size=1048576 2024-04-22T19:58:49.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.115+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~N2ZP0LDGWPML.2 size=907117 2024-04-22T19:58:49.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.115+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=N2ZP0LDGWPML tag=N88O0QIYOD23 type=write size=4052845 2024-04-22T19:58:49.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.116+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~9AAQ38V3OG55.0 size=1048576 2024-04-22T19:58:49.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.117+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~9AAQ38V3OG55.1 size=1048576 2024-04-22T19:58:49.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.118+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~9AAQ38V3OG55.2 size=990207 2024-04-22T19:58:49.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.120+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=9AAQ38V3OG55 tag=QVZAD6K3U9M8 type=write size=4135935 2024-04-22T19:58:49.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.122+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~PCHL3COJ958E.0 size=1048576 2024-04-22T19:58:49.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.123+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~PCHL3COJ958E.1 size=128526 2024-04-22T19:58:49.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.123+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=PCHL3COJ958E tag=RTEBNOUWL3S2 type=write size=2225678 2024-04-22T19:58:49.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.124+0000 7f5269a8e780 1 < canceled operation key=4GX83ZJWZCD1 tag=C7VM5MYBJORJ type=del size=2229710 2024-04-22T19:58:49.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.127+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~0BOF2RP3XKNA.0 size=1048576 2024-04-22T19:58:49.128 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.128+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~0BOF2RP3XKNA.1 size=1048576 2024-04-22T19:58:49.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.129+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~0BOF2RP3XKNA.2 size=577936 2024-04-22T19:58:49.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.129+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=0BOF2RP3XKNA tag=1C1UHOO9D4CM type=write size=3723664 2024-04-22T19:58:49.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.130+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~8U56O7TA028F.0 size=1048576 2024-04-22T19:58:49.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.131+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~8U56O7TA028F.1 size=463579 2024-04-22T19:58:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.133+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=8U56O7TA028F tag=UY8H1AGQ8X3Y type=write size=2560731 2024-04-22T19:58:49.134 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.134+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=LSO510GROAO2 type=del size=3238519 2024-04-22T19:58:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.135+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=PZRZ126JARSF type=del 2024-04-22T19:58:49.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.136+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=UQVZKMHRJXJW type=del size=3746353 2024-04-22T19:58:49.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.138+0000 7f5269a8e780 1 < completed delete operation key=637M7AP8X9E9 tag=0ZCCO01QNBSB type=del 2024-04-22T19:58:49.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.140+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~ZWI6IHTDD1CS.0 size=1048576 2024-04-22T19:58:49.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.141+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~ZWI6IHTDD1CS.1 size=1048576 2024-04-22T19:58:49.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.142+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~ZWI6IHTDD1CS.2 size=718872 2024-04-22T19:58:49.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.142+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=ZWI6IHTDD1CS tag=SA8UZ2JU0SUF type=write size=3864600 2024-04-22T19:58:49.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.143+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~KN183V8IVZXI.0 size=1048576 2024-04-22T19:58:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.144+0000 7f5269a8e780 1 < completed part key=_multipart_IBH6W1NINUHC.2~KN183V8IVZXI.1 size=870639 2024-04-22T19:58:49.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.146+0000 7f5269a8e780 1 < completed multipart upload key=IBH6W1NINUHC upload=KN183V8IVZXI tag=ABIE36Y23SP3 type=write size=2967791 2024-04-22T19:58:49.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.148+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~N0N4KAYV698O.0 size=1048576 2024-04-22T19:58:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.149+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~N0N4KAYV698O.1 size=1048576 2024-04-22T19:58:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.150+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~N0N4KAYV698O.2 size=464998 2024-04-22T19:58:49.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.150+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=N0N4KAYV698O tag=JXKIKJ76CUDZ type=write size=3610726 2024-04-22T19:58:49.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.152+0000 7f5269a8e780 1 < suggested remove operation key=8V6ESIGMRO5G tag=CZLSPNJJ0FV0 type=write 2024-04-22T19:58:49.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.154+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~YL13Z4PH5II2.0 size=1048576 2024-04-22T19:58:49.155 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.155+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~YL13Z4PH5II2.1 size=1048576 2024-04-22T19:58:49.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.156+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~YL13Z4PH5II2.2 size=609565 2024-04-22T19:58:49.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.156+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=YL13Z4PH5II2 tag=S743K9F02KQB type=write size=3755293 2024-04-22T19:58:49.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.157+0000 7f5269a8e780 1 < completed write operation key=RY1O7X1CWZMK tag=52CDZGEZCD4E type=write size=912390 2024-04-22T19:58:49.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.159+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=WXUKQAMDQ1WI type=write size=380607 2024-04-22T19:58:49.161 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.161+0000 7f5269a8e780 1 < suggested update operation key=GX6NILQG878S tag=YTSQSWBCDOS3 type=write size=849231 2024-04-22T19:58:49.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.162+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=IB4P7SMD16TW type=write size=785037 2024-04-22T19:58:49.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.163+0000 7f5269a8e780 -1 < failed to complete operation key=GX6NILQG878S tag=A1V0VYKCBAGG type=del size=725624: (22) Invalid argument 2024-04-22T19:58:49.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.164+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=UBRVX2USXPV5 type=write size=368603 2024-04-22T19:58:49.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.167+0000 7f5269a8e780 1 < suggested update operation key=EHRS6DQDRXVS tag=QHYVX4AU7MCU type=write size=554605 2024-04-22T19:58:49.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.168+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=7TZEZBGJQCIA type=del size=3447471 2024-04-22T19:58:49.169 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.169+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=KO0A1VY02WC9 type=del 2024-04-22T19:58:49.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.170+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=Q4EHOOMJCFQF type=write size=431166 2024-04-22T19:58:49.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.171+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~XNU3NMOXW910.0 size=78260 2024-04-22T19:58:49.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.173+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=XNU3NMOXW910 tag=JY9U4Y579I59 type=write size=1126836 2024-04-22T19:58:49.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.174+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=J7EBS1MJ0UTU type=del size=2192817 2024-04-22T19:58:49.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.176+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~I752UOT06VQ8.0 size=1048576 2024-04-22T19:58:49.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.177+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~I752UOT06VQ8.1 size=140896 2024-04-22T19:58:49.178 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.179+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=I752UOT06VQ8 tag=CL1O6KGVFIUH type=write size=2238048 2024-04-22T19:58:49.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.180+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=H0HV6GYONF6H type=del size=615435 2024-04-22T19:58:49.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.181+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~KIWH27YS83KX.0 size=855633 2024-04-22T19:58:49.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.183+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=KIWH27YS83KX tag=BF9GNTOA1YJF type=write size=1904209 2024-04-22T19:58:49.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.184+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=8NZAPLNDNSPQ type=del size=3639974 2024-04-22T19:58:49.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.185+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~N2ZP0LDGWPML.0 size=1048576 2024-04-22T19:58:49.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.186+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~N2ZP0LDGWPML.1 size=1048576 2024-04-22T19:58:49.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.187+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~N2ZP0LDGWPML.2 size=907117 2024-04-22T19:58:49.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.189+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=N2ZP0LDGWPML tag=N88O0QIYOD23 type=write size=4052845 2024-04-22T19:58:49.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.190+0000 7f5269a8e780 1 > prepared operation key=GX6NILQG878S tag=11H5K8HNWYJD type=del size=2942580 2024-04-22T19:58:49.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.191+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~PCHL3COJ958E.0 size=1048576 2024-04-22T19:58:49.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.192+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~PCHL3COJ958E.1 size=128526 2024-04-22T19:58:49.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.194+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=PCHL3COJ958E tag=RTEBNOUWL3S2 type=write size=2225678 2024-04-22T19:58:49.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.195+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=7SULLVIRNIAC type=del size=1562536 2024-04-22T19:58:49.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.195+0000 7f5269a8e780 1 < canceled part key=_multipart_Q1UKF4UMJ2HO.2~0BOF2RP3XKNA.0 size=1048576 2024-04-22T19:58:49.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.196+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~0BOF2RP3XKNA.1 size=1048576 2024-04-22T19:58:49.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.197+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~0BOF2RP3XKNA.2 size=577936 2024-04-22T19:58:49.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.199+0000 7f5269a8e780 1 < canceled multipart upload key=Q1UKF4UMJ2HO upload=0BOF2RP3XKNA tag=1C1UHOO9D4CM type=write size=3723664 2024-04-22T19:58:49.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.201+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~RL7E0OEBK3CP.0 size=425926 2024-04-22T19:58:49.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.201+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=RL7E0OEBK3CP tag=CKG75YKOFBQC type=write size=1474502 2024-04-22T19:58:49.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.202+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=LSO510GROAO2 type=del 2024-04-22T19:58:49.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.203+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=TL4HUZ8P32DE type=del size=220316 2024-04-22T19:58:49.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.205+0000 7f5269a8e780 1 < suggested update operation key=A853MOZPT9BG tag=UQVZKMHRJXJW type=del size=3746353 2024-04-22T19:58:49.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.207+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=FFG7JJLG2A7X type=del size=310299 2024-04-22T19:58:49.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.208+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~ZWI6IHTDD1CS.0 size=1048576 2024-04-22T19:58:49.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.209+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~ZWI6IHTDD1CS.1 size=1048576 2024-04-22T19:58:49.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.210+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~ZWI6IHTDD1CS.2 size=718872 2024-04-22T19:58:49.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.212+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=ZWI6IHTDD1CS tag=SA8UZ2JU0SUF type=write size=3864600 2024-04-22T19:58:49.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.214+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~8QJXTZBW6FD1.0 size=1048576 2024-04-22T19:58:49.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.215+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~8QJXTZBW6FD1.1 size=902347 2024-04-22T19:58:49.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.215+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=8QJXTZBW6FD1 tag=K3B894B91PM7 type=write size=2999499 2024-04-22T19:58:49.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.216+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~N0N4KAYV698O.0 size=1048576 2024-04-22T19:58:49.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.217+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~N0N4KAYV698O.1 size=1048576 2024-04-22T19:58:49.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.218+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~N0N4KAYV698O.2 size=464998 2024-04-22T19:58:49.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.220+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=N0N4KAYV698O tag=JXKIKJ76CUDZ type=write size=3610726 2024-04-22T19:58:49.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.222+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~LSU1N07MJGFZ.0 size=1048576 2024-04-22T19:58:49.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.223+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~LSU1N07MJGFZ.1 size=1048576 2024-04-22T19:58:49.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.224+0000 7f5269a8e780 1 > prepared part key=_multipart_Y5HR8XW4L0SP.2~LSU1N07MJGFZ.2 size=163265 2024-04-22T19:58:49.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.224+0000 7f5269a8e780 1 > prepared multipart upload key=Y5HR8XW4L0SP upload=LSU1N07MJGFZ tag=58O5NNLS83KJ type=write size=3308993 2024-04-22T19:58:49.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.225+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~YL13Z4PH5II2.0 size=1048576 2024-04-22T19:58:49.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.226+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~YL13Z4PH5II2.1 size=1048576 2024-04-22T19:58:49.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.227+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~YL13Z4PH5II2.2 size=609565 2024-04-22T19:58:49.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.229+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=YL13Z4PH5II2 tag=S743K9F02KQB type=write size=3755293 2024-04-22T19:58:49.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.231+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~T4JP26024PGH.0 size=1048576 2024-04-22T19:58:49.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.234+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~T4JP26024PGH.1 size=1048576 2024-04-22T19:58:49.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.235+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~T4JP26024PGH.2 size=946110 2024-04-22T19:58:49.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.235+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=T4JP26024PGH tag=ZFU06HHB2P6Z type=write size=4091838 2024-04-22T19:58:49.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.236+0000 7f5269a8e780 1 < completed write operation key=B0Y7ZTPC0975 tag=WXUKQAMDQ1WI type=write size=380607 2024-04-22T19:58:49.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.238+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~OYHPCQE73Y8X.0 size=1048576 2024-04-22T19:58:49.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.239+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~OYHPCQE73Y8X.1 size=1048576 2024-04-22T19:58:49.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.239+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~OYHPCQE73Y8X.2 size=80383 2024-04-22T19:58:49.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.239+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=OYHPCQE73Y8X tag=CRAK8BOIS6U3 type=write size=3226111 2024-04-22T19:58:49.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.242+0000 7f5269a8e780 1 < suggested update operation key=KV5DYQ49ZKCX tag=IB4P7SMD16TW type=write size=785037 2024-04-22T19:58:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.244+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~H5HRH337QHTI.0 size=1048576 2024-04-22T19:58:49.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.245+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~H5HRH337QHTI.1 size=1048576 2024-04-22T19:58:49.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.245+0000 7f5269a8e780 1 > prepared part key=_multipart_Q1UKF4UMJ2HO.2~H5HRH337QHTI.2 size=746953 2024-04-22T19:58:49.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.245+0000 7f5269a8e780 1 > prepared multipart upload key=Q1UKF4UMJ2HO upload=H5HRH337QHTI tag=XG5Y1RYWFSGL type=write size=3892681 2024-04-22T19:58:49.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.247+0000 7f5269a8e780 1 < completed write operation key=5UBYMQT7ZWKX tag=UBRVX2USXPV5 type=write size=368603 2024-04-22T19:58:49.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.249+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~LFQHEE71JVY0.0 size=1048576 2024-04-22T19:58:49.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.250+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~LFQHEE71JVY0.1 size=1048576 2024-04-22T19:58:49.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.251+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~LFQHEE71JVY0.2 size=749373 2024-04-22T19:58:49.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.251+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=LFQHEE71JVY0 tag=ZFRMQ689PDAL type=write size=3895101 2024-04-22T19:58:49.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.252+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=7TZEZBGJQCIA type=del 2024-04-22T19:58:49.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.253+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=Z344EZ0371VB type=del size=3269106 2024-04-22T19:58:49.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.254+0000 7f5269a8e780 1 < canceled operation key=EHRS6DQDRXVS tag=Q4EHOOMJCFQF type=write size=431166 2024-04-22T19:58:49.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.255+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=VR8RUGBVOD4L type=write size=481396 2024-04-22T19:58:49.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.257+0000 7f5269a8e780 1 < suggested remove operation key=IBH6W1NINUHC tag=J7EBS1MJ0UTU type=del 2024-04-22T19:58:49.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.258+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=228NEKTD2UXU type=del size=1233025 2024-04-22T19:58:49.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.261+0000 7f5269a8e780 1 < suggested remove operation key=5UBYMQT7ZWKX tag=H0HV6GYONF6H type=del 2024-04-22T19:58:49.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.262+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=61KFEYYTWORL type=del size=3860802 2024-04-22T19:58:49.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.263+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=8NZAPLNDNSPQ type=del 2024-04-22T19:58:49.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.264+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=VV5ZQ7JN223U type=write size=438521 2024-04-22T19:58:49.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.266+0000 7f5269a8e780 1 < completed delete operation key=GX6NILQG878S tag=11H5K8HNWYJD type=del 2024-04-22T19:58:49.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.267+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=IKH4JEOOGDL6 type=del size=2831680 2024-04-22T19:58:49.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.268+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=7SULLVIRNIAC type=del 2024-04-22T19:58:49.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.270+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=77XDD07TCP20 type=del size=3962371 2024-04-22T19:58:49.271 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.271+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~RL7E0OEBK3CP.0 size=425926 2024-04-22T19:58:49.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.273+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=RL7E0OEBK3CP tag=CKG75YKOFBQC type=write size=1474502 2024-04-22T19:58:49.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.274+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=HDTK6NDSADK1 type=write size=419589 2024-04-22T19:58:49.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.276+0000 7f5269a8e780 1 < canceled operation key=T2BV2LUCY4E3 tag=TL4HUZ8P32DE type=del size=220316 2024-04-22T19:58:49.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.278+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~BX6Y0SUUSQ71.0 size=1048576 2024-04-22T19:58:49.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.279+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~BX6Y0SUUSQ71.1 size=603590 2024-04-22T19:58:49.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.279+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=BX6Y0SUUSQ71 tag=KTYFIGLAM2AU type=write size=2700742 2024-04-22T19:58:49.280 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.280+0000 7f5269a8e780 1 < canceled operation key=B0Y7ZTPC0975 tag=FFG7JJLG2A7X type=del size=310299 2024-04-22T19:58:49.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.281+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=2KI6WRL209GQ type=write size=1044104 2024-04-22T19:58:49.282 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.283+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~8QJXTZBW6FD1.0 size=1048576 2024-04-22T19:58:49.283 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.284+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~8QJXTZBW6FD1.1 size=902347 2024-04-22T19:58:49.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.286+0000 7f5269a8e780 1 < completed multipart upload key=HIAXPVKZ1IIO upload=8QJXTZBW6FD1 tag=K3B894B91PM7 type=write size=2999499 2024-04-22T19:58:49.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.287+0000 7f5269a8e780 1 > prepared operation key=A853MOZPT9BG tag=YC0S4AIWZMGH type=del size=2079940 2024-04-22T19:58:49.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.288+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~LSU1N07MJGFZ.0 size=1048576 2024-04-22T19:58:49.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.289+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~LSU1N07MJGFZ.1 size=1048576 2024-04-22T19:58:49.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.290+0000 7f5269a8e780 1 < completed part key=_multipart_Y5HR8XW4L0SP.2~LSU1N07MJGFZ.2 size=163265 2024-04-22T19:58:49.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.292+0000 7f5269a8e780 1 < completed multipart upload key=Y5HR8XW4L0SP upload=LSU1N07MJGFZ tag=58O5NNLS83KJ type=write size=3308993 2024-04-22T19:58:49.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.293+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=WN3RTM2VQEMF type=del size=3461462 2024-04-22T19:58:49.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.294+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~T4JP26024PGH.0 size=1048576 2024-04-22T19:58:49.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.295+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~T4JP26024PGH.1 size=1048576 2024-04-22T19:58:49.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.296+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~T4JP26024PGH.2 size=946110 2024-04-22T19:58:49.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.298+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=T4JP26024PGH tag=ZFU06HHB2P6Z type=write size=4091838 2024-04-22T19:58:49.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.300+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=2W9D7NYT0QON type=del size=3204123 2024-04-22T19:58:49.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.301+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~OYHPCQE73Y8X.0 size=1048576 2024-04-22T19:58:49.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.302+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~OYHPCQE73Y8X.1 size=1048576 2024-04-22T19:58:49.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.303+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~OYHPCQE73Y8X.2 size=80383 2024-04-22T19:58:49.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.305+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=OYHPCQE73Y8X tag=CRAK8BOIS6U3 type=write size=3226111 2024-04-22T19:58:49.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.307+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~I53GIVWZR22Q.0 size=535773 2024-04-22T19:58:49.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.307+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=I53GIVWZR22Q tag=3V4RXOCIJ6IN type=write size=1584349 2024-04-22T19:58:49.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.308+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~H5HRH337QHTI.0 size=1048576 2024-04-22T19:58:49.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.309+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~H5HRH337QHTI.1 size=1048576 2024-04-22T19:58:49.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.310+0000 7f5269a8e780 1 < completed part key=_multipart_Q1UKF4UMJ2HO.2~H5HRH337QHTI.2 size=746953 2024-04-22T19:58:49.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.312+0000 7f5269a8e780 1 < completed multipart upload key=Q1UKF4UMJ2HO upload=H5HRH337QHTI tag=XG5Y1RYWFSGL type=write size=3892681 2024-04-22T19:58:49.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.313+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=W5NOZBIHCPLT type=del size=1178259 2024-04-22T19:58:49.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.314+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~LFQHEE71JVY0.0 size=1048576 2024-04-22T19:58:49.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.315+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~LFQHEE71JVY0.1 size=1048576 2024-04-22T19:58:49.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.316+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~LFQHEE71JVY0.2 size=749373 2024-04-22T19:58:49.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.317+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=LFQHEE71JVY0 tag=ZFRMQ689PDAL type=write size=3895101 2024-04-22T19:58:49.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.319+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=72FD6GWKNOQD type=del size=1423168 2024-04-22T19:58:49.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.320+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=Z344EZ0371VB type=del 2024-04-22T19:58:49.321 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.322+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~SIQ96EVQV53K.0 size=1048576 2024-04-22T19:58:49.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.322+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~SIQ96EVQV53K.1 size=1048576 2024-04-22T19:58:49.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.323+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~SIQ96EVQV53K.2 size=657214 2024-04-22T19:58:49.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.323+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=SIQ96EVQV53K tag=PKQZZ1SSR2QU type=write size=3802942 2024-04-22T19:58:49.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.325+0000 7f5269a8e780 1 < canceled operation key=70MLQGJPQYPY tag=VR8RUGBVOD4L type=write size=481396 2024-04-22T19:58:49.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.326+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=S8QV0TMUA7T2 type=del size=2665360 2024-04-22T19:58:49.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.327+0000 7f5269a8e780 1 < completed delete operation key=Q1UKF4UMJ2HO tag=228NEKTD2UXU type=del 2024-04-22T19:58:49.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.329+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=ECJAZIJ789XQ type=del size=3088648 2024-04-22T19:58:49.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.330+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=61KFEYYTWORL type=del 2024-04-22T19:58:49.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.331+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=OD831PM0XI05 type=write size=1044694 2024-04-22T19:58:49.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.332+0000 7f5269a8e780 1 < canceled operation key=B0Y7ZTPC0975 tag=VV5ZQ7JN223U type=write size=438521 2024-04-22T19:58:49.334 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.334+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~REZLI40OOKZB.0 size=1035735 2024-04-22T19:58:49.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.334+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=REZLI40OOKZB tag=7PVY5OZBCVMD type=write size=2084311 2024-04-22T19:58:49.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.335+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=IKH4JEOOGDL6 type=del 2024-04-22T19:58:49.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.337+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=DVHPE8X5ITQS type=del size=2619930 2024-04-22T19:58:49.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.338+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=77XDD07TCP20 type=del 2024-04-22T19:58:49.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.340+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~N5BNOG4LSB15.0 size=1048576 2024-04-22T19:58:49.341 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.341+0000 7f5269a8e780 1 > prepared part key=_multipart_5UBYMQT7ZWKX.2~N5BNOG4LSB15.1 size=650362 2024-04-22T19:58:49.341 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.341+0000 7f5269a8e780 1 > prepared multipart upload key=5UBYMQT7ZWKX upload=N5BNOG4LSB15 tag=II6OPJSHJRLZ type=write size=2747514 2024-04-22T19:58:49.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.343+0000 7f5269a8e780 1 < canceled operation key=KE7YGPTF1MOR tag=HDTK6NDSADK1 type=write size=419589 2024-04-22T19:58:49.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.344+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=T9285628IZY9 type=del size=2044335 2024-04-22T19:58:49.346 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.346+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~BX6Y0SUUSQ71.0 size=1048576 2024-04-22T19:58:49.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.347+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~BX6Y0SUUSQ71.1 size=603590 2024-04-22T19:58:49.348 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.349+0000 7f5269a8e780 1 < completed multipart upload key=VPAVX5FXYWZO upload=BX6Y0SUUSQ71 tag=KTYFIGLAM2AU type=write size=2700742 2024-04-22T19:58:49.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.350+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=TU7CX096LQ7J type=del size=4067779 2024-04-22T19:58:49.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.352+0000 7f5269a8e780 1 < suggested update operation key=VPAVX5FXYWZO tag=2KI6WRL209GQ type=write size=1044104 2024-04-22T19:58:49.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.353+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=AGLX6SQW3FBF type=del size=3725934 2024-04-22T19:58:49.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.354+0000 7f5269a8e780 1 < canceled operation key=A853MOZPT9BG tag=YC0S4AIWZMGH type=del size=2079940 2024-04-22T19:58:49.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.356+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=OMTNOONGPEMQ type=write size=1001398 2024-04-22T19:58:49.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.357+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=WN3RTM2VQEMF type=del 2024-04-22T19:58:49.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.358+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=6MWV6L5TSJQV type=del size=1450594 2024-04-22T19:58:49.359 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.359+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=2W9D7NYT0QON type=del 2024-04-22T19:58:49.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.360+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=VNXWUWJ05DZJ type=del size=345694 2024-04-22T19:58:49.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.361+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~I53GIVWZR22Q.0 size=535773 2024-04-22T19:58:49.363 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.363+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=I53GIVWZR22Q tag=3V4RXOCIJ6IN type=write size=1584349 2024-04-22T19:58:49.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.365+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=BSPJWNRAOM7P type=del size=3563841 2024-04-22T19:58:49.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.366+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=W5NOZBIHCPLT type=del 2024-04-22T19:58:49.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.368+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~0YW1O55MVQQ6.0 size=1048576 2024-04-22T19:58:49.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.369+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~0YW1O55MVQQ6.1 size=1048576 2024-04-22T19:58:49.369 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.370+0000 7f5269a8e780 1 > prepared part key=_multipart_EHRS6DQDRXVS.2~0YW1O55MVQQ6.2 size=364540 2024-04-22T19:58:49.370 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.370+0000 7f5269a8e780 1 > prepared multipart upload key=EHRS6DQDRXVS upload=0YW1O55MVQQ6 tag=IG3ICH67UYCP type=write size=3510268 2024-04-22T19:58:49.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.371+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=72FD6GWKNOQD type=del 2024-04-22T19:58:49.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.373+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~YED1FDDMXVH0.0 size=1048576 2024-04-22T19:58:49.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.374+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~YED1FDDMXVH0.1 size=391794 2024-04-22T19:58:49.374 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.374+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=YED1FDDMXVH0 tag=8VRPXHESMZM9 type=write size=2488946 2024-04-22T19:58:49.375 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.375+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~SIQ96EVQV53K.0 size=1048576 2024-04-22T19:58:49.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.376+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~SIQ96EVQV53K.1 size=1048576 2024-04-22T19:58:49.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.376+0000 7f5269a8e780 1 < canceled part key=_multipart_I91AD8FJ3P4T.2~SIQ96EVQV53K.2 size=657214 2024-04-22T19:58:49.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.378+0000 7f5269a8e780 1 < canceled multipart upload key=I91AD8FJ3P4T upload=SIQ96EVQV53K tag=PKQZZ1SSR2QU type=write size=3802942 2024-04-22T19:58:49.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.380+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~5MXI7BA20JW0.0 size=1048576 2024-04-22T19:58:49.381 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.381+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~5MXI7BA20JW0.1 size=455046 2024-04-22T19:58:49.381 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.381+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=5MXI7BA20JW0 tag=D49WTHWSVL7G type=write size=2552198 2024-04-22T19:58:49.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.382+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=S8QV0TMUA7T2 type=del 2024-04-22T19:58:49.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.383+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=K4DJ4GD1HQ0U type=write size=717240 2024-04-22T19:58:49.384 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.384+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=ECJAZIJ789XQ type=del 2024-04-22T19:58:49.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.385+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=7POOIMMZ3JZN type=del size=471654 2024-04-22T19:58:49.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.388+0000 7f5269a8e780 1 < suggested remove operation key=T2BV2LUCY4E3 tag=OD831PM0XI05 type=write 2024-04-22T19:58:49.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.389+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=O00SA5ELZKPF type=del size=1877216 2024-04-22T19:58:49.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.390+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~REZLI40OOKZB.0 size=1035735 2024-04-22T19:58:49.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.392+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=REZLI40OOKZB tag=7PVY5OZBCVMD type=write size=2084311 2024-04-22T19:58:49.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.394+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~AQXCC6FIM3K6.0 size=1048576 2024-04-22T19:58:49.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.394+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~AQXCC6FIM3K6.1 size=1048576 2024-04-22T19:58:49.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.395+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~AQXCC6FIM3K6.2 size=763559 2024-04-22T19:58:49.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.395+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=AQXCC6FIM3K6 tag=L9YRGY5WHROU type=write size=3909287 2024-04-22T19:58:49.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.396+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=DVHPE8X5ITQS type=del 2024-04-22T19:58:49.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.397+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=G2E490KA07A6 type=del size=3597561 2024-04-22T19:58:49.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.398+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~N5BNOG4LSB15.0 size=1048576 2024-04-22T19:58:49.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.399+0000 7f5269a8e780 1 < completed part key=_multipart_5UBYMQT7ZWKX.2~N5BNOG4LSB15.1 size=650362 2024-04-22T19:58:49.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.401+0000 7f5269a8e780 1 < completed multipart upload key=5UBYMQT7ZWKX upload=N5BNOG4LSB15 tag=II6OPJSHJRLZ type=write size=2747514 2024-04-22T19:58:49.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.402+0000 7f5269a8e780 1 > prepared operation key=EHRS6DQDRXVS tag=4C4A6TCQUMVL type=del size=1701046 2024-04-22T19:58:49.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.403+0000 7f5269a8e780 1 < completed delete operation key=M9M4SEEEZSFV tag=T9285628IZY9 type=del 2024-04-22T19:58:49.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.406+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~HX6DU3LLATWS.0 size=1048576 2024-04-22T19:58:49.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.407+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~HX6DU3LLATWS.1 size=1048576 2024-04-22T19:58:49.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.408+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~HX6DU3LLATWS.2 size=545441 2024-04-22T19:58:49.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.408+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=HX6DU3LLATWS tag=D57EKDB1X6D4 type=write size=3691169 2024-04-22T19:58:49.408 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.409+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=TU7CX096LQ7J type=del 2024-04-22T19:58:49.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.410+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=1KXWVE1LHNQU type=write size=470068 2024-04-22T19:58:49.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.411+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=AGLX6SQW3FBF type=del 2024-04-22T19:58:49.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.412+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=16GL227EKIJJ type=del size=838990 2024-04-22T19:58:49.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.413+0000 7f5269a8e780 -1 < failed to complete operation key=T2BV2LUCY4E3 tag=OMTNOONGPEMQ type=write size=1001398: (22) Invalid argument 2024-04-22T19:58:49.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.414+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=64SRB856AAGU type=del size=1608033 2024-04-22T19:58:49.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.415+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=6MWV6L5TSJQV type=del 2024-04-22T19:58:49.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.417+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=4ZNTWV4MG7F0 type=del size=236111 2024-04-22T19:58:49.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.419+0000 7f5269a8e780 1 < suggested remove operation key=B0Y7ZTPC0975 tag=VNXWUWJ05DZJ type=del 2024-04-22T19:58:49.420 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.420+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=ZJSO1CXWLQTV type=del size=3937133 2024-04-22T19:58:49.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.421+0000 7f5269a8e780 -1 < failed to complete operation key=B0Y7ZTPC0975 tag=BSPJWNRAOM7P type=del size=3563841: (22) Invalid argument 2024-04-22T19:58:49.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.422+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=DUJVYDWXSUZJ type=del size=85866 2024-04-22T19:58:49.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.423+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~0YW1O55MVQQ6.0 size=1048576 2024-04-22T19:58:49.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.424+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~0YW1O55MVQQ6.1 size=1048576 2024-04-22T19:58:49.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.425+0000 7f5269a8e780 1 < completed part key=_multipart_EHRS6DQDRXVS.2~0YW1O55MVQQ6.2 size=364540 2024-04-22T19:58:49.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.427+0000 7f5269a8e780 1 < completed multipart upload key=EHRS6DQDRXVS upload=0YW1O55MVQQ6 tag=IG3ICH67UYCP type=write size=3510268 2024-04-22T19:58:49.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.428+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=V08PXVQR6FPI type=del size=3124390 2024-04-22T19:58:49.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.429+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~YED1FDDMXVH0.0 size=1048576 2024-04-22T19:58:49.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.430+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~YED1FDDMXVH0.1 size=391794 2024-04-22T19:58:49.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.432+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=YED1FDDMXVH0 tag=8VRPXHESMZM9 type=write size=2488946 2024-04-22T19:58:49.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.433+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=B27M5TTCDC1F type=del size=4025549 2024-04-22T19:58:49.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.434+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~5MXI7BA20JW0.0 size=1048576 2024-04-22T19:58:49.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.435+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~5MXI7BA20JW0.1 size=455046 2024-04-22T19:58:49.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.437+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=5MXI7BA20JW0 tag=D49WTHWSVL7G type=write size=2552198 2024-04-22T19:58:49.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.438+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=FHXLFCBDH9C3 type=del size=3352125 2024-04-22T19:58:49.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.439+0000 7f5269a8e780 1 < canceled operation key=86GW6HV9G1IZ tag=K4DJ4GD1HQ0U type=write size=717240 2024-04-22T19:58:49.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.440+0000 7f5269a8e780 1 > prepared operation key=KV5DYQ49ZKCX tag=QM7BYK6ZVSIV type=del size=3343759 2024-04-22T19:58:49.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.441+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=7POOIMMZ3JZN type=del 2024-04-22T19:58:49.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.442+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=NMG4OK3Q9OLE type=write size=6840 2024-04-22T19:58:49.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.443+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=O00SA5ELZKPF type=del 2024-04-22T19:58:49.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.444+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=SCVIAK4DI6O2 type=del size=2718661 2024-04-22T19:58:49.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.445+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~AQXCC6FIM3K6.0 size=1048576 2024-04-22T19:58:49.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.446+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~AQXCC6FIM3K6.1 size=1048576 2024-04-22T19:58:49.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.447+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~AQXCC6FIM3K6.2 size=763559 2024-04-22T19:58:49.450 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.450+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=AQXCC6FIM3K6 tag=L9YRGY5WHROU type=write size=3909287 2024-04-22T19:58:49.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.453+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=WG7XJU6T9Z3K type=del size=1294868 2024-04-22T19:58:49.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.454+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=G2E490KA07A6 type=del 2024-04-22T19:58:49.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.456+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=RUKH375CZ587 type=del size=301210 2024-04-22T19:58:49.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.457+0000 7f5269a8e780 1 < completed delete operation key=EHRS6DQDRXVS tag=4C4A6TCQUMVL type=del 2024-04-22T19:58:49.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.459+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~QS29EU7ZFZR2.0 size=1048576 2024-04-22T19:58:49.459 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.460+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~QS29EU7ZFZR2.1 size=484186 2024-04-22T19:58:49.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.460+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=QS29EU7ZFZR2 tag=NEQ8496HD69H type=write size=2581338 2024-04-22T19:58:49.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.461+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~HX6DU3LLATWS.0 size=1048576 2024-04-22T19:58:49.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.462+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~HX6DU3LLATWS.1 size=1048576 2024-04-22T19:58:49.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.463+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~HX6DU3LLATWS.2 size=545441 2024-04-22T19:58:49.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.465+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=HX6DU3LLATWS tag=D57EKDB1X6D4 type=write size=3691169 2024-04-22T19:58:49.466 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.466+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=755OJ8T66P55 type=del size=2051174 2024-04-22T19:58:49.468 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.468+0000 7f5269a8e780 1 < suggested remove operation key=Q1UKF4UMJ2HO tag=1KXWVE1LHNQU type=write 2024-04-22T19:58:49.469 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.469+0000 7f5269a8e780 1 > prepared operation key=VPAVX5FXYWZO tag=O8OZRME63APC type=del size=1449817 2024-04-22T19:58:49.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.471+0000 7f5269a8e780 1 < suggested remove operation key=4GX83ZJWZCD1 tag=16GL227EKIJJ type=del 2024-04-22T19:58:49.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.472+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=82X0KSFAY0A9 type=del size=137170 2024-04-22T19:58:49.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.474+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=64SRB856AAGU type=del 2024-04-22T19:58:49.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.475+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=9QX0G46KJEAM type=del size=1326710 2024-04-22T19:58:49.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.476+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=4ZNTWV4MG7F0 type=del 2024-04-22T19:58:49.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.478+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~9PVNFLTSAUGH.0 size=1048576 2024-04-22T19:58:49.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.479+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~9PVNFLTSAUGH.1 size=908235 2024-04-22T19:58:49.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.479+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=9PVNFLTSAUGH tag=GRCTHWYQD7VW type=write size=3005387 2024-04-22T19:58:49.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.480+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=ZJSO1CXWLQTV type=del 2024-04-22T19:58:49.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.481+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=GAWLHY8CB4YW type=del size=651409 2024-04-22T19:58:49.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.483+0000 7f5269a8e780 1 < suggested update operation key=M9M4SEEEZSFV tag=DUJVYDWXSUZJ type=del size=85866 2024-04-22T19:58:49.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.485+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~CV9D5PAFG81Q.0 size=1048576 2024-04-22T19:58:49.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.486+0000 7f5269a8e780 1 > prepared part key=_multipart_M9M4SEEEZSFV.2~CV9D5PAFG81Q.1 size=425049 2024-04-22T19:58:49.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.486+0000 7f5269a8e780 1 > prepared multipart upload key=M9M4SEEEZSFV upload=CV9D5PAFG81Q tag=WL6YTU6KZMZX type=write size=2522201 2024-04-22T19:58:49.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.487+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=V08PXVQR6FPI type=del 2024-04-22T19:58:49.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.489+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=FHE126CZSFOC type=del size=2520398 2024-04-22T19:58:49.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.490+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=B27M5TTCDC1F type=del 2024-04-22T19:58:49.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.491+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=YJNSM0HL6036 type=del size=3544571 2024-04-22T19:58:49.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.492+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=FHXLFCBDH9C3 type=del 2024-04-22T19:58:49.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.494+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=JRY2R836BMNR type=del size=103851 2024-04-22T19:58:49.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.495+0000 7f5269a8e780 1 < completed delete operation key=KV5DYQ49ZKCX tag=QM7BYK6ZVSIV type=del 2024-04-22T19:58:49.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.496+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=LAW1V8AHAL0A type=del size=2008897 2024-04-22T19:58:49.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.498+0000 7f5269a8e780 1 < suggested update operation key=T2BV2LUCY4E3 tag=NMG4OK3Q9OLE type=write size=6840 2024-04-22T19:58:49.499 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.499+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=G6ZGEPOOUCEK type=write size=107461 2024-04-22T19:58:49.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.500+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=SCVIAK4DI6O2 type=del 2024-04-22T19:58:49.501 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.501+0000 7f5269a8e780 1 > prepared operation key=5UBYMQT7ZWKX tag=NWWMV94SMWE0 type=del size=2240962 2024-04-22T19:58:49.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.504+0000 7f5269a8e780 1 < suggested remove operation key=AMBJHMJGLUUS tag=WG7XJU6T9Z3K type=del 2024-04-22T19:58:49.505 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.505+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=O4EB4VN121A5 type=del size=767172 2024-04-22T19:58:49.507 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.507+0000 7f5269a8e780 1 < suggested update operation key=86GW6HV9G1IZ tag=RUKH375CZ587 type=del size=301210 2024-04-22T19:58:49.508 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.508+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=UW0JBHKKGQRK type=write size=867330 2024-04-22T19:58:49.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.509+0000 7f5269a8e780 1 < completed part key=_multipart_A853MOZPT9BG.2~QS29EU7ZFZR2.0 size=1048576 2024-04-22T19:58:49.509 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.509+0000 7f5269a8e780 1 < canceled part key=_multipart_A853MOZPT9BG.2~QS29EU7ZFZR2.1 size=484186 2024-04-22T19:58:49.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.511+0000 7f5269a8e780 1 < canceled multipart upload key=A853MOZPT9BG upload=QS29EU7ZFZR2 tag=NEQ8496HD69H type=write size=2581338 2024-04-22T19:58:49.512 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.512+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=YV4YWHR641FZ type=write size=335948 2024-04-22T19:58:49.514 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.514+0000 7f5269a8e780 1 < completed delete operation key=KE7YGPTF1MOR tag=755OJ8T66P55 type=del 2024-04-22T19:58:49.516 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.516+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~CRJP5VW0PGH3.0 size=319622 2024-04-22T19:58:49.516 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.516+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=CRJP5VW0PGH3 tag=ERCQNK696NWO type=write size=1368198 2024-04-22T19:58:49.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.517+0000 7f5269a8e780 1 < completed delete operation key=VPAVX5FXYWZO tag=O8OZRME63APC type=del 2024-04-22T19:58:49.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.520+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~DXN35KOPXI9W.0 size=505493 2024-04-22T19:58:49.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.520+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=DXN35KOPXI9W tag=I0D0J4ANS0PB type=write size=1554069 2024-04-22T19:58:49.522 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.522+0000 7f5269a8e780 1 < suggested update operation key=86GW6HV9G1IZ tag=82X0KSFAY0A9 type=del size=137170 2024-04-22T19:58:49.524 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.524+0000 7f5269a8e780 1 > prepared operation key=M9M4SEEEZSFV tag=I2D1VK4U25PJ type=write size=148512 2024-04-22T19:58:49.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.525+0000 7f5269a8e780 1 < completed delete operation key=SN2BZATJGUWA tag=9QX0G46KJEAM type=del 2024-04-22T19:58:49.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.526+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=7Y3ZTL64467K type=del size=1033933 2024-04-22T19:58:49.527 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.527+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~9PVNFLTSAUGH.0 size=1048576 2024-04-22T19:58:49.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.528+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~9PVNFLTSAUGH.1 size=908235 2024-04-22T19:58:49.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.530+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=9PVNFLTSAUGH tag=GRCTHWYQD7VW type=write size=3005387 2024-04-22T19:58:49.532 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.532+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~3ICVV7N28JZK.0 size=1048576 2024-04-22T19:58:49.533 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.533+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~3ICVV7N28JZK.1 size=1048576 2024-04-22T19:58:49.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.534+0000 7f5269a8e780 1 > prepared part key=_multipart_VPAVX5FXYWZO.2~3ICVV7N28JZK.2 size=957528 2024-04-22T19:58:49.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.534+0000 7f5269a8e780 1 > prepared multipart upload key=VPAVX5FXYWZO upload=3ICVV7N28JZK tag=UKPRR4LX6OAI type=write size=4103256 2024-04-22T19:58:49.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.535+0000 7f5269a8e780 1 < canceled operation key=I91AD8FJ3P4T tag=GAWLHY8CB4YW type=del size=651409 2024-04-22T19:58:49.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.537+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~0P7012W27F5R.0 size=1048576 2024-04-22T19:58:49.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.538+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~0P7012W27F5R.1 size=525080 2024-04-22T19:58:49.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.538+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=0P7012W27F5R tag=1Y72G65XRQK1 type=write size=2622232 2024-04-22T19:58:49.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.538+0000 7f5269a8e780 1 < canceled part key=_multipart_M9M4SEEEZSFV.2~CV9D5PAFG81Q.0 size=1048576 2024-04-22T19:58:49.539 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.539+0000 7f5269a8e780 1 < completed part key=_multipart_M9M4SEEEZSFV.2~CV9D5PAFG81Q.1 size=425049 2024-04-22T19:58:49.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.541+0000 7f5269a8e780 1 < canceled multipart upload key=M9M4SEEEZSFV upload=CV9D5PAFG81Q tag=WL6YTU6KZMZX type=write size=2522201 2024-04-22T19:58:49.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.543+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~2RTOGXOPGQBZ.0 size=1048576 2024-04-22T19:58:49.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.544+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~2RTOGXOPGQBZ.1 size=1048576 2024-04-22T19:58:49.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.545+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~2RTOGXOPGQBZ.2 size=674130 2024-04-22T19:58:49.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.545+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=2RTOGXOPGQBZ tag=DP788UJ2HRUV type=write size=3819858 2024-04-22T19:58:49.546 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.546+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=FHE126CZSFOC type=del 2024-04-22T19:58:49.547 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.548+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~9351GZACA4CM.0 size=1048576 2024-04-22T19:58:49.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.549+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~9351GZACA4CM.1 size=393805 2024-04-22T19:58:49.549 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.549+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=9351GZACA4CM tag=I1ZLC66XOBH4 type=write size=2490957 2024-04-22T19:58:49.550 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.550+0000 7f5269a8e780 1 < canceled operation key=IBH6W1NINUHC tag=YJNSM0HL6036 type=del size=3544571 2024-04-22T19:58:49.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.552+0000 7f5269a8e780 1 > prepared part key=_multipart_4EA3LUQRL8II.2~D3Q7I2VB0E8J.0 size=600359 2024-04-22T19:58:49.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.552+0000 7f5269a8e780 1 > prepared multipart upload key=4EA3LUQRL8II upload=D3Q7I2VB0E8J tag=ZR7PRQ06MQ2X type=write size=1648935 2024-04-22T19:58:49.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.554+0000 7f5269a8e780 -1 < failed to complete operation key=AMBJHMJGLUUS tag=JRY2R836BMNR type=del size=103851: (22) Invalid argument 2024-04-22T19:58:49.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.556+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~E1F70J0B5SC9.0 size=929340 2024-04-22T19:58:49.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.556+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=E1F70J0B5SC9 tag=DA3ASX3NOSVN type=write size=1977916 2024-04-22T19:58:49.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.558+0000 7f5269a8e780 1 < suggested remove operation key=8V6ESIGMRO5G tag=LAW1V8AHAL0A type=del 2024-04-22T19:58:49.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.562+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~RILV4PQ6AQWQ.0 size=581423 2024-04-22T19:58:49.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.562+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=RILV4PQ6AQWQ tag=0UAGG8VQQ3KO type=write size=1629999 2024-04-22T19:58:49.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.563+0000 7f5269a8e780 1 < completed write operation key=4GX83ZJWZCD1 tag=G6ZGEPOOUCEK type=write size=107461 2024-04-22T19:58:49.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.564+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=NS7SR7FCMJQ8 type=del size=4188391 2024-04-22T19:58:49.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.565+0000 7f5269a8e780 1 < completed delete operation key=5UBYMQT7ZWKX tag=NWWMV94SMWE0 type=del 2024-04-22T19:58:49.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.568+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~3V95BHQKK2P1.0 size=1048576 2024-04-22T19:58:49.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.569+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~3V95BHQKK2P1.1 size=246747 2024-04-22T19:58:49.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.569+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=3V95BHQKK2P1 tag=IHM730JO0W2T type=write size=2343899 2024-04-22T19:58:49.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.570+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=O4EB4VN121A5 type=del 2024-04-22T19:58:49.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.572+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~EUVZPFEE68Y9.0 size=77927 2024-04-22T19:58:49.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.572+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=EUVZPFEE68Y9 tag=NDS2L42YS298 type=write size=1126503 2024-04-22T19:58:49.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.575+0000 7f5269a8e780 1 < suggested update operation key=8HFSDSNZONIN tag=UW0JBHKKGQRK type=write size=867330 2024-04-22T19:58:49.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.576+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=FX1GFKUHW8Q9 type=del size=134470 2024-04-22T19:58:49.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.577+0000 7f5269a8e780 1 < completed write operation key=I91AD8FJ3P4T tag=YV4YWHR641FZ type=write size=335948 2024-04-22T19:58:49.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.578+0000 7f5269a8e780 1 > prepared operation key=IBH6W1NINUHC tag=3UXO9HNT0GJ8 type=del size=3158143 2024-04-22T19:58:49.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.579+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~CRJP5VW0PGH3.0 size=319622 2024-04-22T19:58:49.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.583+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=CRJP5VW0PGH3 tag=ERCQNK696NWO type=write size=1368198 2024-04-22T19:58:49.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.584+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~VH8T1BGLGA8L.0 size=1048576 2024-04-22T19:58:49.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.585+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~VH8T1BGLGA8L.1 size=1048576 2024-04-22T19:58:49.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.586+0000 7f5269a8e780 1 > prepared part key=_multipart_2YFD2U6HVNE7.2~VH8T1BGLGA8L.2 size=819497 2024-04-22T19:58:49.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.586+0000 7f5269a8e780 1 > prepared multipart upload key=2YFD2U6HVNE7 upload=VH8T1BGLGA8L tag=XO6L7M8G6XL1 type=write size=3965225 2024-04-22T19:58:49.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.586+0000 7f5269a8e780 1 < canceled part key=_multipart_KV5DYQ49ZKCX.2~DXN35KOPXI9W.0 size=505493 2024-04-22T19:58:49.588 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.589+0000 7f5269a8e780 1 < canceled multipart upload key=KV5DYQ49ZKCX upload=DXN35KOPXI9W tag=I0D0J4ANS0PB type=write size=1554069 2024-04-22T19:58:49.589 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.590+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=4K8ZA998Q0EM type=del size=3296624 2024-04-22T19:58:49.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.593+0000 7f5269a8e780 1 < suggested update operation key=M9M4SEEEZSFV tag=I2D1VK4U25PJ type=write size=148512 2024-04-22T19:58:49.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.595+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~QAA3AZOAI5OI.0 size=556088 2024-04-22T19:58:49.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.595+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=QAA3AZOAI5OI tag=7SHEC1HVE12C type=write size=1604664 2024-04-22T19:58:49.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.595+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=7Y3ZTL64467K type=del 2024-04-22T19:58:49.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.596+0000 7f5269a8e780 1 > prepared operation key=B0Y7ZTPC0975 tag=5665FKB8A0O8 type=del size=744992 2024-04-22T19:58:49.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.599+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~3ICVV7N28JZK.0 size=1048576 2024-04-22T19:58:49.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.599+0000 7f5269a8e780 1 < canceled part key=_multipart_VPAVX5FXYWZO.2~3ICVV7N28JZK.1 size=1048576 2024-04-22T19:58:49.599 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.599+0000 7f5269a8e780 1 < completed part key=_multipart_VPAVX5FXYWZO.2~3ICVV7N28JZK.2 size=957528 2024-04-22T19:58:49.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.601+0000 7f5269a8e780 1 < canceled multipart upload key=VPAVX5FXYWZO upload=3ICVV7N28JZK tag=UKPRR4LX6OAI type=write size=4103256 2024-04-22T19:58:49.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.604+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~597RSJX2RZMA.0 size=180003 2024-04-22T19:58:49.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.604+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=597RSJX2RZMA tag=04RN2YFU7EOQ type=write size=1228579 2024-04-22T19:58:49.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.605+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~0P7012W27F5R.0 size=1048576 2024-04-22T19:58:49.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.606+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~0P7012W27F5R.1 size=525080 2024-04-22T19:58:49.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.608+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=0P7012W27F5R tag=1Y72G65XRQK1 type=write size=2622232 2024-04-22T19:58:49.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.609+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=TV4QGJYOF17S type=del size=2729426 2024-04-22T19:58:49.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.610+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~2RTOGXOPGQBZ.0 size=1048576 2024-04-22T19:58:49.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.610+0000 7f5269a8e780 1 < completed part key=_multipart_HIAXPVKZ1IIO.2~2RTOGXOPGQBZ.1 size=1048576 2024-04-22T19:58:49.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.610+0000 7f5269a8e780 1 < canceled part key=_multipart_HIAXPVKZ1IIO.2~2RTOGXOPGQBZ.2 size=674130 2024-04-22T19:58:49.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.613+0000 7f5269a8e780 1 < canceled multipart upload key=HIAXPVKZ1IIO upload=2RTOGXOPGQBZ tag=DP788UJ2HRUV type=write size=3819858 2024-04-22T19:58:49.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.614+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=QJO9W6DL80UB type=del size=3699095 2024-04-22T19:58:49.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.615+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~9351GZACA4CM.0 size=1048576 2024-04-22T19:58:49.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.616+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~9351GZACA4CM.1 size=393805 2024-04-22T19:58:49.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.618+0000 7f5269a8e780 1 < completed multipart upload key=RY1O7X1CWZMK upload=9351GZACA4CM tag=I1ZLC66XOBH4 type=write size=2490957 2024-04-22T19:58:49.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.619+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=V5HDPSAUKSKH type=write size=808649 2024-04-22T19:58:49.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.619+0000 7f5269a8e780 1 < canceled part key=_multipart_4EA3LUQRL8II.2~D3Q7I2VB0E8J.0 size=600359 2024-04-22T19:58:49.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.621+0000 7f5269a8e780 1 < canceled multipart upload key=4EA3LUQRL8II upload=D3Q7I2VB0E8J tag=ZR7PRQ06MQ2X type=write size=1648935 2024-04-22T19:58:49.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.622+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=G1GJXUL4U98G type=del size=2049956 2024-04-22T19:58:49.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.624+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~E1F70J0B5SC9.0 size=929340 2024-04-22T19:58:49.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.625+0000 7f5269a8e780 1 < completed multipart upload key=2YFD2U6HVNE7 upload=E1F70J0B5SC9 tag=DA3ASX3NOSVN type=write size=1977916 2024-04-22T19:58:49.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.627+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~U832ONAK13Q9.0 size=1048576 2024-04-22T19:58:49.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.628+0000 7f5269a8e780 1 > prepared part key=_multipart_I91AD8FJ3P4T.2~U832ONAK13Q9.1 size=811084 2024-04-22T19:58:49.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.628+0000 7f5269a8e780 1 > prepared multipart upload key=I91AD8FJ3P4T upload=U832ONAK13Q9 tag=YVAK9BKEPR0W type=write size=2908236 2024-04-22T19:58:49.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.630+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~RILV4PQ6AQWQ.0 size=581423 2024-04-22T19:58:49.630 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.632+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=RILV4PQ6AQWQ tag=0UAGG8VQQ3KO type=write size=1629999 2024-04-22T19:58:49.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.633+0000 7f5269a8e780 1 > prepared operation key=DX5YUYQRZKWA tag=3ZP7A1VK9BNQ type=del size=1570380 2024-04-22T19:58:49.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.634+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=NS7SR7FCMJQ8 type=del 2024-04-22T19:58:49.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.635+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=0O7M5Z789M3L type=write size=213436 2024-04-22T19:58:49.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.636+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~3V95BHQKK2P1.0 size=1048576 2024-04-22T19:58:49.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.637+0000 7f5269a8e780 1 < completed part key=_multipart_HV8J6YM0YNC1.2~3V95BHQKK2P1.1 size=246747 2024-04-22T19:58:49.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.640+0000 7f5269a8e780 1 < completed multipart upload key=HV8J6YM0YNC1 upload=3V95BHQKK2P1 tag=IHM730JO0W2T type=write size=2343899 2024-04-22T19:58:49.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.641+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=5LZRGLAEBY4E type=write size=641544 2024-04-22T19:58:49.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.642+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~EUVZPFEE68Y9.0 size=77927 2024-04-22T19:58:49.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.644+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=EUVZPFEE68Y9 tag=NDS2L42YS298 type=write size=1126503 2024-04-22T19:58:49.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.645+0000 7f5269a8e780 1 > prepared operation key=8HFSDSNZONIN tag=VF9BGM3XYATX type=del size=1390014 2024-04-22T19:58:49.645 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.646+0000 7f5269a8e780 1 < canceled operation key=I91AD8FJ3P4T tag=FX1GFKUHW8Q9 type=del size=134470 2024-04-22T19:58:49.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.647+0000 7f5269a8e780 1 > prepared operation key=RY1O7X1CWZMK tag=Y163JUMLGDF7 type=del size=1338593 2024-04-22T19:58:49.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.649+0000 7f5269a8e780 1 < completed delete operation key=IBH6W1NINUHC tag=3UXO9HNT0GJ8 type=del 2024-04-22T19:58:49.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.650+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=UXCT3GRDSUE5 type=del size=2974555 2024-04-22T19:58:49.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.650+0000 7f5269a8e780 1 < canceled part key=_multipart_2YFD2U6HVNE7.2~VH8T1BGLGA8L.0 size=1048576 2024-04-22T19:58:49.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.651+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~VH8T1BGLGA8L.1 size=1048576 2024-04-22T19:58:49.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.652+0000 7f5269a8e780 1 < completed part key=_multipart_2YFD2U6HVNE7.2~VH8T1BGLGA8L.2 size=819497 2024-04-22T19:58:49.653 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.654+0000 7f5269a8e780 1 < canceled multipart upload key=2YFD2U6HVNE7 upload=VH8T1BGLGA8L tag=XO6L7M8G6XL1 type=write size=3965225 2024-04-22T19:58:49.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.655+0000 7f5269a8e780 1 > prepared operation key=O9NFAVE09ZN7 tag=0Q4MWT8THZNJ type=del size=1041608 2024-04-22T19:58:49.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.657+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=4K8ZA998Q0EM type=del 2024-04-22T19:58:49.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.658+0000 7f5269a8e780 1 > prepared operation key=GRSL48RUEBX8 tag=PF2FHW3YHT4O type=write size=394750 2024-04-22T19:58:49.658 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.659+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~QAA3AZOAI5OI.0 size=556088 2024-04-22T19:58:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.661+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=QAA3AZOAI5OI tag=7SHEC1HVE12C type=write size=1604664 2024-04-22T19:58:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.663+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~CRP7MQYZ7R06.0 size=1048576 2024-04-22T19:58:49.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.664+0000 7f5269a8e780 1 > prepared part key=_multipart_O9NFAVE09ZN7.2~CRP7MQYZ7R06.1 size=809369 2024-04-22T19:58:49.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.664+0000 7f5269a8e780 1 > prepared multipart upload key=O9NFAVE09ZN7 upload=CRP7MQYZ7R06 tag=G4E33YBYDTL6 type=write size=2906521 2024-04-22T19:58:49.664 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.665+0000 7f5269a8e780 1 < completed delete operation key=B0Y7ZTPC0975 tag=5665FKB8A0O8 type=del 2024-04-22T19:58:49.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.667+0000 7f5269a8e780 1 > prepared part key=_multipart_GX6NILQG878S.2~HY0INC53JOFS.0 size=58952 2024-04-22T19:58:49.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.667+0000 7f5269a8e780 1 > prepared multipart upload key=GX6NILQG878S upload=HY0INC53JOFS tag=74CK3BVDL03O type=write size=1107528 2024-04-22T19:58:49.667 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.668+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~597RSJX2RZMA.0 size=180003 2024-04-22T19:58:49.669 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.670+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=597RSJX2RZMA tag=04RN2YFU7EOQ type=write size=1228579 2024-04-22T19:58:49.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.671+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=FLYT30P6YRAN type=write size=791768 2024-04-22T19:58:49.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.674+0000 7f5269a8e780 1 < completed delete operation key=8HFSDSNZONIN tag=TV4QGJYOF17S type=del 2024-04-22T19:58:49.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.675+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=TXOEXABOB7KP type=del size=3152757 2024-04-22T19:58:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.676+0000 7f5269a8e780 1 < completed delete operation key=AMBJHMJGLUUS tag=QJO9W6DL80UB type=del 2024-04-22T19:58:49.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.677+0000 7f5269a8e780 1 > prepared operation key=4EA3LUQRL8II tag=YU1HZ6ZPEY58 type=del size=178130 2024-04-22T19:58:49.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.678+0000 7f5269a8e780 1 < completed write operation key=GP1C5ULUU143 tag=V5HDPSAUKSKH type=write size=808649 2024-04-22T19:58:49.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.681+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~JEVYRHYXUXGU.0 size=1048576 2024-04-22T19:58:49.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.682+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~JEVYRHYXUXGU.1 size=966125 2024-04-22T19:58:49.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.682+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=JEVYRHYXUXGU tag=OVVXVQQIL0CD type=write size=3063277 2024-04-22T19:58:49.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.683+0000 7f5269a8e780 1 < completed delete operation key=GRSL48RUEBX8 tag=G1GJXUL4U98G type=del 2024-04-22T19:58:49.683 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.684+0000 7f5269a8e780 1 > prepared operation key=Q1UKF4UMJ2HO tag=3NSQX8C0UQD6 type=del size=396123 2024-04-22T19:58:49.684 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.685+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~U832ONAK13Q9.0 size=1048576 2024-04-22T19:58:49.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.686+0000 7f5269a8e780 1 < completed part key=_multipart_I91AD8FJ3P4T.2~U832ONAK13Q9.1 size=811084 2024-04-22T19:58:49.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.688+0000 7f5269a8e780 1 < completed multipart upload key=I91AD8FJ3P4T upload=U832ONAK13Q9 tag=YVAK9BKEPR0W type=write size=2908236 2024-04-22T19:58:49.689 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.690+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~CMN5LYNN7AH8.0 size=1048576 2024-04-22T19:58:49.690 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.691+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~CMN5LYNN7AH8.1 size=922113 2024-04-22T19:58:49.690 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.691+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=CMN5LYNN7AH8 tag=ZHD203H2WIBJ type=write size=3019265 2024-04-22T19:58:49.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.693+0000 7f5269a8e780 1 < completed delete operation key=DX5YUYQRZKWA tag=3ZP7A1VK9BNQ type=del 2024-04-22T19:58:49.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.694+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=KY7A719IIRYO type=del size=3075711 2024-04-22T19:58:49.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.695+0000 7f5269a8e780 1 < completed write operation key=HV8J6YM0YNC1 tag=0O7M5Z789M3L type=write size=213436 2024-04-22T19:58:49.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.696+0000 7f5269a8e780 1 > prepared operation key=HV8J6YM0YNC1 tag=QK2B1UVAEZ2I type=del size=2531562 2024-04-22T19:58:49.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.697+0000 7f5269a8e780 1 < completed write operation key=O9NFAVE09ZN7 tag=5LZRGLAEBY4E type=write size=641544 2024-04-22T19:58:49.698 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.699+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~M5BU8BQ0LN2Q.0 size=1048576 2024-04-22T19:58:49.699 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.700+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~M5BU8BQ0LN2Q.1 size=1048576 2024-04-22T19:58:49.699 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.701+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~M5BU8BQ0LN2Q.2 size=591757 2024-04-22T19:58:49.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.701+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=M5BU8BQ0LN2Q tag=AINZ1VAZEB2R type=write size=3737485 2024-04-22T19:58:49.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.703+0000 7f5269a8e780 1 < suggested remove operation key=8HFSDSNZONIN tag=VF9BGM3XYATX type=del 2024-04-22T19:58:49.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.704+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=XKDBCTS3Y24I type=del size=2294269 2024-04-22T19:58:49.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.705+0000 7f5269a8e780 1 < completed delete operation key=RY1O7X1CWZMK tag=Y163JUMLGDF7 type=del 2024-04-22T19:58:49.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.707+0000 7f5269a8e780 1 > prepared part key=_multipart_8V6ESIGMRO5G.2~VVOJT01SZWG9.0 size=64000 2024-04-22T19:58:49.706 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.707+0000 7f5269a8e780 1 > prepared multipart upload key=8V6ESIGMRO5G upload=VVOJT01SZWG9 tag=F0QTVA0NY9AA type=write size=1112576 2024-04-22T19:58:49.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.708+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=UXCT3GRDSUE5 type=del 2024-04-22T19:58:49.708 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.709+0000 7f5269a8e780 1 > prepared operation key=86GW6HV9G1IZ tag=TXI8P8532F0K type=del size=2751582 2024-04-22T19:58:49.709 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.710+0000 7f5269a8e780 1 < completed delete operation key=O9NFAVE09ZN7 tag=0Q4MWT8THZNJ type=del 2024-04-22T19:58:49.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.712+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~TNW3KJLY5W3H.0 size=1048576 2024-04-22T19:58:49.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.713+0000 7f5269a8e780 1 > prepared part key=_multipart_ZC4ZOHUJ3HQD.2~TNW3KJLY5W3H.1 size=733160 2024-04-22T19:58:49.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.713+0000 7f5269a8e780 1 > prepared multipart upload key=ZC4ZOHUJ3HQD upload=TNW3KJLY5W3H tag=ZQA4DEXDHVZF type=write size=2830312 2024-04-22T19:58:49.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.715+0000 7f5269a8e780 1 < suggested update operation key=GRSL48RUEBX8 tag=PF2FHW3YHT4O type=write size=394750 2024-04-22T19:58:49.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.717+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~U10ZHS9LPSH2.0 size=1048576 2024-04-22T19:58:49.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.718+0000 7f5269a8e780 1 > prepared part key=_multipart_70MLQGJPQYPY.2~U10ZHS9LPSH2.1 size=540173 2024-04-22T19:58:49.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.718+0000 7f5269a8e780 1 > prepared multipart upload key=70MLQGJPQYPY upload=U10ZHS9LPSH2 tag=OLT8FXT4ZWBZ type=write size=2637325 2024-04-22T19:58:49.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.719+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~CRP7MQYZ7R06.0 size=1048576 2024-04-22T19:58:49.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.720+0000 7f5269a8e780 1 < completed part key=_multipart_O9NFAVE09ZN7.2~CRP7MQYZ7R06.1 size=809369 2024-04-22T19:58:49.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.722+0000 7f5269a8e780 1 < completed multipart upload key=O9NFAVE09ZN7 upload=CRP7MQYZ7R06 tag=G4E33YBYDTL6 type=write size=2906521 2024-04-22T19:58:49.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.724+0000 7f5269a8e780 1 > prepared part key=_multipart_GP1C5ULUU143.2~RJ5F9ZVDQGHR.0 size=147168 2024-04-22T19:58:49.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.724+0000 7f5269a8e780 1 > prepared multipart upload key=GP1C5ULUU143 upload=RJ5F9ZVDQGHR tag=60IURY31TFB8 type=write size=1195744 2024-04-22T19:58:49.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.725+0000 7f5269a8e780 1 < completed part key=_multipart_GX6NILQG878S.2~HY0INC53JOFS.0 size=58952 2024-04-22T19:58:49.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.727+0000 7f5269a8e780 1 < completed multipart upload key=GX6NILQG878S upload=HY0INC53JOFS tag=74CK3BVDL03O type=write size=1107528 2024-04-22T19:58:49.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.729+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~7O282PK4CV2O.0 size=1048576 2024-04-22T19:58:49.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.730+0000 7f5269a8e780 1 > prepared part key=_multipart_KV5DYQ49ZKCX.2~7O282PK4CV2O.1 size=721827 2024-04-22T19:58:49.729 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.730+0000 7f5269a8e780 1 > prepared multipart upload key=KV5DYQ49ZKCX upload=7O282PK4CV2O tag=96CN9YLS5VN5 type=write size=2818979 2024-04-22T19:58:49.730 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.731+0000 7f5269a8e780 1 < completed write operation key=70MLQGJPQYPY tag=FLYT30P6YRAN type=write size=791768 2024-04-22T19:58:49.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.732+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=OO473NYJEDWU type=del size=3154705 2024-04-22T19:58:49.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.733+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=TXOEXABOB7KP type=del 2024-04-22T19:58:49.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.734+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=3W0BNM83DUEU type=del size=2121791 2024-04-22T19:58:49.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.736+0000 7f5269a8e780 1 < completed delete operation key=4EA3LUQRL8II tag=YU1HZ6ZPEY58 type=del 2024-04-22T19:58:49.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.737+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=CK1JSDF6LXGG type=write size=265306 2024-04-22T19:58:49.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.737+0000 7f5269a8e780 1 < canceled part key=_multipart_RY1O7X1CWZMK.2~JEVYRHYXUXGU.0 size=1048576 2024-04-22T19:58:49.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.738+0000 7f5269a8e780 1 < completed part key=_multipart_RY1O7X1CWZMK.2~JEVYRHYXUXGU.1 size=966125 2024-04-22T19:58:49.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.740+0000 7f5269a8e780 1 < canceled multipart upload key=RY1O7X1CWZMK upload=JEVYRHYXUXGU tag=OVVXVQQIL0CD type=write size=3063277 2024-04-22T19:58:49.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.741+0000 7f5269a8e780 1 > prepared operation key=ZC4ZOHUJ3HQD tag=IIZZIQMZGEYY type=del size=2856321 2024-04-22T19:58:49.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.743+0000 7f5269a8e780 1 < suggested remove operation key=Q1UKF4UMJ2HO tag=3NSQX8C0UQD6 type=del 2024-04-22T19:58:49.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.745+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=R6MRQI6Y7EZA type=del size=1150853 2024-04-22T19:58:49.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.746+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~CMN5LYNN7AH8.0 size=1048576 2024-04-22T19:58:49.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.747+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~CMN5LYNN7AH8.1 size=922113 2024-04-22T19:58:49.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.749+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=CMN5LYNN7AH8 tag=ZHD203H2WIBJ type=write size=3019265 2024-04-22T19:58:49.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.751+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~8GPAI3K1GYJC.0 size=1048576 2024-04-22T19:58:49.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.752+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~8GPAI3K1GYJC.1 size=205421 2024-04-22T19:58:49.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.752+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=8GPAI3K1GYJC tag=31E3OXAX3R81 type=write size=2302573 2024-04-22T19:58:49.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.753+0000 7f5269a8e780 1 < completed delete operation key=HIAXPVKZ1IIO tag=KY7A719IIRYO type=del 2024-04-22T19:58:49.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.754+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=T0NCJGQT0LCM type=del size=2409781 2024-04-22T19:58:49.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.755+0000 7f5269a8e780 1 < completed delete operation key=HV8J6YM0YNC1 tag=QK2B1UVAEZ2I type=del 2024-04-22T19:58:49.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.757+0000 7f5269a8e780 1 > prepared part key=_multipart_T2BV2LUCY4E3.2~6VOIUQZZEI7D.0 size=401916 2024-04-22T19:58:49.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.757+0000 7f5269a8e780 1 > prepared multipart upload key=T2BV2LUCY4E3 upload=6VOIUQZZEI7D tag=Y0GMVKM0OPG3 type=write size=1450492 2024-04-22T19:58:49.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.758+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~M5BU8BQ0LN2Q.0 size=1048576 2024-04-22T19:58:49.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.758+0000 7f5269a8e780 1 < canceled part key=_multipart_4GX83ZJWZCD1.2~M5BU8BQ0LN2Q.1 size=1048576 2024-04-22T19:58:49.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.759+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~M5BU8BQ0LN2Q.2 size=591757 2024-04-22T19:58:49.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.761+0000 7f5269a8e780 1 < canceled multipart upload key=4GX83ZJWZCD1 upload=M5BU8BQ0LN2Q tag=AINZ1VAZEB2R type=write size=3737485 2024-04-22T19:58:49.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.763+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=ROBQFA75JDW3 type=write size=164652 2024-04-22T19:58:49.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.764+0000 7f5269a8e780 1 < completed delete operation key=GP1C5ULUU143 tag=XKDBCTS3Y24I type=del 2024-04-22T19:58:49.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.765+0000 7f5269a8e780 1 > prepared operation key=T2BV2LUCY4E3 tag=VTAKQG91H3RL type=del size=3125609 2024-04-22T19:58:49.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.766+0000 7f5269a8e780 1 < completed part key=_multipart_8V6ESIGMRO5G.2~VVOJT01SZWG9.0 size=64000 2024-04-22T19:58:49.767 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.768+0000 7f5269a8e780 1 < completed multipart upload key=8V6ESIGMRO5G upload=VVOJT01SZWG9 tag=F0QTVA0NY9AA type=write size=1112576 2024-04-22T19:58:49.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.770+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~27VO1JBFR1Q2.0 size=1048576 2024-04-22T19:58:49.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.771+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~27VO1JBFR1Q2.1 size=1048576 2024-04-22T19:58:49.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.772+0000 7f5269a8e780 1 > prepared part key=_multipart_637M7AP8X9E9.2~27VO1JBFR1Q2.2 size=67304 2024-04-22T19:58:49.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.772+0000 7f5269a8e780 1 > prepared multipart upload key=637M7AP8X9E9 upload=27VO1JBFR1Q2 tag=B3S7BSM180U7 type=write size=3213032 2024-04-22T19:58:49.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.773+0000 7f5269a8e780 1 < completed delete operation key=86GW6HV9G1IZ tag=TXI8P8532F0K type=del 2024-04-22T19:58:49.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.775+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~HK9KRG886UKV.0 size=1048576 2024-04-22T19:58:49.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.776+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~HK9KRG886UKV.1 size=1048576 2024-04-22T19:58:49.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.777+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~HK9KRG886UKV.2 size=875396 2024-04-22T19:58:49.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.777+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=HK9KRG886UKV tag=KSA6YL4Q0FXM type=write size=4021124 2024-04-22T19:58:49.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.778+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~TNW3KJLY5W3H.0 size=1048576 2024-04-22T19:58:49.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.780+0000 7f5269a8e780 1 < completed part key=_multipart_ZC4ZOHUJ3HQD.2~TNW3KJLY5W3H.1 size=733160 2024-04-22T19:58:49.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.784+0000 7f5269a8e780 1 < completed multipart upload key=ZC4ZOHUJ3HQD upload=TNW3KJLY5W3H tag=ZQA4DEXDHVZF type=write size=2830312 2024-04-22T19:58:49.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.786+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~O3LXEC4K756D.0 size=1048576 2024-04-22T19:58:49.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.787+0000 7f5269a8e780 1 > prepared part key=_multipart_GRSL48RUEBX8.2~O3LXEC4K756D.1 size=202404 2024-04-22T19:58:49.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.787+0000 7f5269a8e780 1 > prepared multipart upload key=GRSL48RUEBX8 upload=O3LXEC4K756D tag=YFF018H43IAE type=write size=2299556 2024-04-22T19:58:49.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.788+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~U10ZHS9LPSH2.0 size=1048576 2024-04-22T19:58:49.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.789+0000 7f5269a8e780 1 < completed part key=_multipart_70MLQGJPQYPY.2~U10ZHS9LPSH2.1 size=540173 2024-04-22T19:58:49.790 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.791+0000 7f5269a8e780 1 < completed multipart upload key=70MLQGJPQYPY upload=U10ZHS9LPSH2 tag=OLT8FXT4ZWBZ type=write size=2637325 2024-04-22T19:58:49.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.792+0000 7f5269a8e780 1 > prepared operation key=70MLQGJPQYPY tag=1L0B7SDRJ995 type=del size=210745 2024-04-22T19:58:49.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.793+0000 7f5269a8e780 1 < completed part key=_multipart_GP1C5ULUU143.2~RJ5F9ZVDQGHR.0 size=147168 2024-04-22T19:58:49.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.795+0000 7f5269a8e780 1 < completed multipart upload key=GP1C5ULUU143 upload=RJ5F9ZVDQGHR tag=60IURY31TFB8 type=write size=1195744 2024-04-22T19:58:49.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.797+0000 7f5269a8e780 1 > prepared part key=_multipart_AMBJHMJGLUUS.2~9WIE4F5QRFYV.0 size=949072 2024-04-22T19:58:49.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.797+0000 7f5269a8e780 1 > prepared multipart upload key=AMBJHMJGLUUS upload=9WIE4F5QRFYV tag=TVKIV573XZ7B type=write size=1997648 2024-04-22T19:58:49.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.798+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~7O282PK4CV2O.0 size=1048576 2024-04-22T19:58:49.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.799+0000 7f5269a8e780 1 < completed part key=_multipart_KV5DYQ49ZKCX.2~7O282PK4CV2O.1 size=721827 2024-04-22T19:58:49.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.801+0000 7f5269a8e780 1 < completed multipart upload key=KV5DYQ49ZKCX upload=7O282PK4CV2O tag=96CN9YLS5VN5 type=write size=2818979 2024-04-22T19:58:49.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.802+0000 7f5269a8e780 1 > prepared operation key=Y5HR8XW4L0SP tag=W4W31GUNISA7 type=del size=2798103 2024-04-22T19:58:49.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.803+0000 7f5269a8e780 1 < completed delete operation key=Y5HR8XW4L0SP tag=OO473NYJEDWU type=del 2024-04-22T19:58:49.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.805+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~NHBBS9UASF2J.0 size=1048576 2024-04-22T19:58:49.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.806+0000 7f5269a8e780 1 > prepared part key=_multipart_4GX83ZJWZCD1.2~NHBBS9UASF2J.1 size=214829 2024-04-22T19:58:49.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.806+0000 7f5269a8e780 1 > prepared multipart upload key=4GX83ZJWZCD1 upload=NHBBS9UASF2J tag=JWB10L2H9K9T type=write size=2311981 2024-04-22T19:58:49.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.808+0000 7f5269a8e780 1 < suggested remove operation key=I91AD8FJ3P4T tag=3W0BNM83DUEU type=del 2024-04-22T19:58:49.808 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.809+0000 7f5269a8e780 1 > prepared operation key=8V6ESIGMRO5G tag=KMTMYIC9EZF1 type=del size=848694 2024-04-22T19:58:49.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.812+0000 7f5269a8e780 1 < suggested update operation key=GP1C5ULUU143 tag=CK1JSDF6LXGG type=write size=265306 2024-04-22T19:58:49.812 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.813+0000 7f5269a8e780 1 > prepared operation key=SN2BZATJGUWA tag=MHX8RMUJ3DOE type=del size=2985917 2024-04-22T19:58:49.813 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.814+0000 7f5269a8e780 1 < completed delete operation key=ZC4ZOHUJ3HQD tag=IIZZIQMZGEYY type=del 2024-04-22T19:58:49.814 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.815+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=POAH3ARY7O1S type=del size=3162263 2024-04-22T19:58:49.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.817+0000 7f5269a8e780 -1 < failed to complete operation key=I91AD8FJ3P4T tag=R6MRQI6Y7EZA type=del size=1150853: (22) Invalid argument 2024-04-22T19:58:49.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.818+0000 7f5269a8e780 1 > prepared operation key=2YFD2U6HVNE7 tag=M8N57ANA0KI9 type=write size=785832 2024-04-22T19:58:49.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.819+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~8GPAI3K1GYJC.0 size=1048576 2024-04-22T19:58:49.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.820+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~8GPAI3K1GYJC.1 size=205421 2024-04-22T19:58:49.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.822+0000 7f5269a8e780 1 < completed multipart upload key=4GX83ZJWZCD1 upload=8GPAI3K1GYJC tag=31E3OXAX3R81 type=write size=2302573 2024-04-22T19:58:49.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.823+0000 7f5269a8e780 1 > prepared operation key=HIAXPVKZ1IIO tag=JSPXCUTRLUU7 type=del size=4129398 2024-04-22T19:58:49.823 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.824+0000 7f5269a8e780 1 < completed delete operation key=2YFD2U6HVNE7 tag=T0NCJGQT0LCM type=del 2024-04-22T19:58:49.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.825+0000 7f5269a8e780 1 > prepared operation key=637M7AP8X9E9 tag=POS2GNOJ66YL type=del size=3723673 2024-04-22T19:58:49.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.827+0000 7f5269a8e780 1 < completed part key=_multipart_T2BV2LUCY4E3.2~6VOIUQZZEI7D.0 size=401916 2024-04-22T19:58:49.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.828+0000 7f5269a8e780 1 < completed multipart upload key=T2BV2LUCY4E3 upload=6VOIUQZZEI7D tag=Y0GMVKM0OPG3 type=write size=1450492 2024-04-22T19:58:49.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.830+0000 7f5269a8e780 1 > prepared part key=_multipart_8HFSDSNZONIN.2~H2G3QA9N28UV.0 size=706724 2024-04-22T19:58:49.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.830+0000 7f5269a8e780 1 > prepared multipart upload key=8HFSDSNZONIN upload=H2G3QA9N28UV tag=GC00LBUNMZAA type=write size=1755300 2024-04-22T19:58:49.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.831+0000 7f5269a8e780 -1 < failed to complete operation key=I91AD8FJ3P4T tag=ROBQFA75JDW3 type=write size=164652: (22) Invalid argument 2024-04-22T19:58:49.832 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.833+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~WMQGR1ZKJHCL.0 size=1048576 2024-04-22T19:58:49.833 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.834+0000 7f5269a8e780 1 > prepared part key=_multipart_RY1O7X1CWZMK.2~WMQGR1ZKJHCL.1 size=682583 2024-04-22T19:58:49.833 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.834+0000 7f5269a8e780 1 > prepared multipart upload key=RY1O7X1CWZMK upload=WMQGR1ZKJHCL tag=T00BUQL6FDUA type=write size=2779735 2024-04-22T19:58:49.834 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.835+0000 7f5269a8e780 1 < completed delete operation key=T2BV2LUCY4E3 tag=VTAKQG91H3RL type=del 2024-04-22T19:58:49.835 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.836+0000 7f5269a8e780 1 > prepared operation key=4GX83ZJWZCD1 tag=D85H8EMRKY3Y type=del size=4071952 2024-04-22T19:58:49.836 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.838+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~27VO1JBFR1Q2.0 size=1048576 2024-04-22T19:58:49.837 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.838+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~27VO1JBFR1Q2.1 size=1048576 2024-04-22T19:58:49.838 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.839+0000 7f5269a8e780 1 < completed part key=_multipart_637M7AP8X9E9.2~27VO1JBFR1Q2.2 size=67304 2024-04-22T19:58:49.840 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.841+0000 7f5269a8e780 1 < completed multipart upload key=637M7AP8X9E9 upload=27VO1JBFR1Q2 tag=B3S7BSM180U7 type=write size=3213032 2024-04-22T19:58:49.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.843+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~29U3I69R2PN1.0 size=1048576 2024-04-22T19:58:49.843 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.844+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~29U3I69R2PN1.1 size=1048576 2024-04-22T19:58:49.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.845+0000 7f5269a8e780 1 > prepared part key=_multipart_HIAXPVKZ1IIO.2~29U3I69R2PN1.2 size=386505 2024-04-22T19:58:49.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.845+0000 7f5269a8e780 1 > prepared multipart upload key=HIAXPVKZ1IIO upload=29U3I69R2PN1 tag=GVD757NG3HRE type=write size=3532233 2024-04-22T19:58:49.845 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.846+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~HK9KRG886UKV.0 size=1048576 2024-04-22T19:58:49.846 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.847+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~HK9KRG886UKV.1 size=1048576 2024-04-22T19:58:49.847 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.848+0000 7f5269a8e780 1 < completed part key=_multipart_8HFSDSNZONIN.2~HK9KRG886UKV.2 size=875396 2024-04-22T19:58:49.849 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.850+0000 7f5269a8e780 1 < completed multipart upload key=8HFSDSNZONIN upload=HK9KRG886UKV tag=KSA6YL4Q0FXM type=write size=4021124 2024-04-22T19:58:49.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.851+0000 7f5269a8e780 1 > prepared operation key=KE7YGPTF1MOR tag=8I0C48MLMGPC type=write size=352426 2024-04-22T19:58:49.851 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.852+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~O3LXEC4K756D.0 size=1048576 2024-04-22T19:58:49.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.853+0000 7f5269a8e780 1 < completed part key=_multipart_GRSL48RUEBX8.2~O3LXEC4K756D.1 size=202404 2024-04-22T19:58:49.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.855+0000 7f5269a8e780 1 < completed multipart upload key=GRSL48RUEBX8 upload=O3LXEC4K756D tag=YFF018H43IAE type=write size=2299556 2024-04-22T19:58:49.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.856+0000 7f5269a8e780 1 > prepared operation key=AMBJHMJGLUUS tag=5Q2ERCWAK726 type=del size=2295805 2024-04-22T19:58:49.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.857+0000 7f5269a8e780 1 < completed delete operation key=70MLQGJPQYPY tag=1L0B7SDRJ995 type=del 2024-04-22T19:58:49.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.860+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~OSEOB1WOCUWS.0 size=1048576 2024-04-22T19:58:49.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.860+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~OSEOB1WOCUWS.1 size=1048576 2024-04-22T19:58:49.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.861+0000 7f5269a8e780 1 > prepared part key=_multipart_HV8J6YM0YNC1.2~OSEOB1WOCUWS.2 size=461411 2024-04-22T19:58:49.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.861+0000 7f5269a8e780 1 > prepared multipart upload key=HV8J6YM0YNC1 upload=OSEOB1WOCUWS tag=KR2J9Y4B02B7 type=write size=3607139 2024-04-22T19:58:49.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.863+0000 7f5269a8e780 1 < completed part key=_multipart_AMBJHMJGLUUS.2~9WIE4F5QRFYV.0 size=949072 2024-04-22T19:58:49.863 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.864+0000 7f5269a8e780 1 < completed multipart upload key=AMBJHMJGLUUS upload=9WIE4F5QRFYV tag=TVKIV573XZ7B type=write size=1997648 2024-04-22T19:58:49.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.866+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~HLCIX7J0NZCQ.0 size=1048576 2024-04-22T19:58:49.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.867+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~HLCIX7J0NZCQ.1 size=1048576 2024-04-22T19:58:49.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.868+0000 7f5269a8e780 1 > prepared part key=_multipart_A853MOZPT9BG.2~HLCIX7J0NZCQ.2 size=226955 2024-04-22T19:58:49.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.868+0000 7f5269a8e780 1 > prepared multipart upload key=A853MOZPT9BG upload=HLCIX7J0NZCQ tag=NPY03YZTOY98 type=write size=3372683 2024-04-22T19:58:49.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.870+0000 7f5269a8e780 1 < suggested remove operation key=Y5HR8XW4L0SP tag=W4W31GUNISA7 type=del 2024-04-22T19:58:49.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.871+0000 7f5269a8e780 1 > prepared operation key=I91AD8FJ3P4T tag=F1HSK20TRW34 type=del size=3617042 2024-04-22T19:58:49.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.873+0000 7f5269a8e780 1 < completed part key=_multipart_4GX83ZJWZCD1.2~NHBBS9UASF2J.0 size=1048576 2024-04-22T19:58:49.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.873+0000 7f5269a8e780 1 < canceled part key=_multipart_4GX83ZJWZCD1.2~NHBBS9UASF2J.1 size=214829 2024-04-22T19:58:49.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.874+0000 7f5269a8e780 1 < canceled multipart upload key=4GX83ZJWZCD1 upload=NHBBS9UASF2J tag=JWB10L2H9K9T type=write size=2311981 2024-04-22T19:58:49.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.876+0000 7f5269a8e780 1 > prepared operation key=GP1C5ULUU143 tag=1VPRQ67YISIL type=del size=1074179 2024-04-22T19:58:49.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.877+0000 7f5269a8e780 1 < completed delete operation key=8V6ESIGMRO5G tag=KMTMYIC9EZF1 type=del 2024-04-22T19:58:49.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.879+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~4CWE2NFS0Y5P.0 size=1048576 2024-04-22T19:58:49.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.880+0000 7f5269a8e780 1 > prepared part key=_multipart_KE7YGPTF1MOR.2~4CWE2NFS0Y5P.1 size=946586 2024-04-22T19:58:49.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T19:58:49.880+0000 7f5269a8e780 1 > prepared multipart upload key=KE7YGPTF1MOR upload=4CWE2NFS0Y5P tag=9HLV10E063JH type=write size=3043738 2024-04-22T19:58:49.879 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] cls_rgw_stats.simulate (8641 ms) 2024-04-22T19:58:49.879 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 1 test from cls_rgw_stats (8641 ms total) 2024-04-22T19:58:49.879 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T19:58:49.879 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T19:58:50.168 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 1 test from 1 test suite ran. (10759 ms total) 2024-04-22T19:58:50.169 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 1 test. 2024-04-22T19:58:50.171 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T19:58:50.171 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T19:58:50.248 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2024-04-22T19:58:50.249 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2024-04-22T19:58:50.249 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2024-04-22T19:58:50.329 INFO:tasks.workunit.client.0.smithi136.stdout:Running main() from gmock_main.cc 2024-04-22T19:58:50.329 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 31 tests from 1 test suite. 2024-04-22T19:58:50.329 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T19:58:52.191 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 31 tests from CmpOmap 2024-04-22T19:58:52.191 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2024-04-22T19:58:52.209 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (18 ms) 2024-04-22T19:58:52.209 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2024-04-22T19:58:52.213 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (5 ms) 2024-04-22T19:58:52.214 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2024-04-22T19:58:52.217 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (3 ms) 2024-04-22T19:58:52.217 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2024-04-22T19:58:52.220 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (3 ms) 2024-04-22T19:58:52.220 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_str 2024-04-22T19:58:52.226 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_str (6 ms) 2024-04-22T19:58:52.226 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2024-04-22T19:58:52.232 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_u64 (7 ms) 2024-04-22T19:58:52.232 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2024-04-22T19:58:52.236 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (3 ms) 2024-04-22T19:58:52.236 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2024-04-22T19:58:52.239 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (3 ms) 2024-04-22T19:58:52.239 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2024-04-22T19:58:52.444 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (205 ms) 2024-04-22T19:58:52.444 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2024-04-22T19:58:52.449 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (5 ms) 2024-04-22T19:58:52.449 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2024-04-22T19:58:52.449 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (1 ms) 2024-04-22T19:58:52.449 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2024-04-22T19:58:52.450 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (0 ms) 2024-04-22T19:58:52.450 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2024-04-22T19:58:52.458 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (9 ms) 2024-04-22T19:58:52.458 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2024-04-22T19:58:52.466 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (8 ms) 2024-04-22T19:58:52.466 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2024-04-22T19:58:52.473 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (7 ms) 2024-04-22T19:58:52.473 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2024-04-22T19:58:52.480 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (7 ms) 2024-04-22T19:58:52.480 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2024-04-22T19:58:52.489 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_str (8 ms) 2024-04-22T19:58:52.489 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2024-04-22T19:58:52.499 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (11 ms) 2024-04-22T19:58:52.499 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2024-04-22T19:58:52.503 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (3 ms) 2024-04-22T19:58:52.503 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2024-04-22T19:58:52.505 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (2 ms) 2024-04-22T19:58:52.505 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2024-04-22T19:58:52.506 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (2 ms) 2024-04-22T19:58:52.506 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2024-04-22T19:58:52.507 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (0 ms) 2024-04-22T19:58:52.507 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2024-04-22T19:58:52.513 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (7 ms) 2024-04-22T19:58:52.513 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2024-04-22T19:58:52.519 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (6 ms) 2024-04-22T19:58:52.519 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2024-04-22T19:58:52.527 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (7 ms) 2024-04-22T19:58:52.527 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2024-04-22T19:58:52.536 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (9 ms) 2024-04-22T19:58:52.536 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2024-04-22T19:58:52.538 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (3 ms) 2024-04-22T19:58:52.539 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2024-04-22T19:58:52.541 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (3 ms) 2024-04-22T19:58:52.541 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2024-04-22T19:58:52.542 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (1 ms) 2024-04-22T19:58:52.542 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2024-04-22T19:58:52.543 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (0 ms) 2024-04-22T19:58:52.543 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2024-04-22T19:58:52.550 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (7 ms) 2024-04-22T19:58:52.550 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 31 tests from CmpOmap (359 ms total) 2024-04-22T19:58:52.550 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T19:58:52.550 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T19:58:53.192 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 31 tests from 1 test suite ran. (2863 ms total) 2024-04-22T19:58:53.192 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 31 tests. 2024-04-22T19:58:53.195 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T19:58:53.195 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T19:58:53.276 INFO:tasks.workunit:Running workunits matching cls/test_cls_2pc_queue.sh on client.0... 2024-04-22T19:58:53.277 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2024-04-22T19:58:53.278 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-04-22T19:58:53.363 INFO:tasks.workunit.client.0.smithi136.stdout:Running main() from gmock_main.cc 2024-04-22T19:58:53.363 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 24 tests from 1 test suite. 2024-04-22T19:58:53.363 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T19:58:53.363 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 24 tests from TestCls2PCQueue 2024-04-22T19:58:53.363 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2024-04-22T19:58:56.213 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2851 ms) 2024-04-22T19:58:56.213 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2024-04-22T19:58:59.250 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3037 ms) 2024-04-22T19:58:59.251 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.Reserve 2024-04-22T19:59:02.260 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.Reserve (3009 ms) 2024-04-22T19:59:02.260 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2024-04-22T19:59:05.290 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3030 ms) 2024-04-22T19:59:05.290 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.Commit 2024-04-22T19:59:09.317 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.Commit (4027 ms) 2024-04-22T19:59:09.317 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.Stats 2024-04-22T19:59:12.324 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.Stats (3006 ms) 2024-04-22T19:59:12.324 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.UpgradeFromReef 2024-04-22T19:59:16.353 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.UpgradeFromReef (4030 ms) 2024-04-22T19:59:16.353 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.Abort 2024-04-22T19:59:19.374 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.Abort (3020 ms) 2024-04-22T19:59:19.374 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2024-04-22T19:59:22.377 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.ReserveError (3004 ms) 2024-04-22T19:59:22.377 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.CommitError 2024-04-22T19:59:25.406 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.CommitError (3028 ms) 2024-04-22T19:59:25.406 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.AbortError 2024-04-22T19:59:28.413 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.AbortError (3007 ms) 2024-04-22T19:59:28.413 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2024-04-22T19:59:31.445 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3033 ms) 2024-04-22T19:59:31.445 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2024-04-22T19:59:34.459 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3014 ms) 2024-04-22T19:59:34.459 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2024-04-22T19:59:37.481 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3022 ms) 2024-04-22T19:59:37.481 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2024-04-22T19:59:40.489 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3008 ms) 2024-04-22T19:59:40.489 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2024-04-22T19:59:43.517 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3028 ms) 2024-04-22T19:59:43.517 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2024-04-22T19:59:46.538 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3022 ms) 2024-04-22T19:59:46.538 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2024-04-22T19:59:49.559 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.Cleanup (3020 ms) 2024-04-22T19:59:49.559 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2024-04-22T19:59:59.598 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.MultiProducer (10039 ms) 2024-04-22T19:59:59.598 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2024-04-22T20:00:02.615 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (3017 ms) 2024-04-22T20:00:02.615 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2024-04-22T20:00:15.642 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (13028 ms) 2024-04-22T20:00:15.643 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2024-04-22T20:00:19.670 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (4027 ms) 2024-04-22T20:00:19.670 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2024-04-22T20:00:25.692 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (6023 ms) 2024-04-22T20:00:25.693 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2024-04-22T20:00:30.727 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (5035 ms) 2024-04-22T20:00:30.728 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 24 tests from TestCls2PCQueue (97365 ms total) 2024-04-22T20:00:30.728 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:30.728 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:30.728 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 24 tests from 1 test suite ran. (97365 ms total) 2024-04-22T20:00:30.728 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 24 tests. 2024-04-22T20:00:30.730 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T20:00:30.731 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T20:00:30.770 INFO:tasks.workunit:Running workunits matching cls/test_cls_user.sh on client.0... 2024-04-22T20:00:30.771 INFO:tasks.workunit:Running workunit cls/test_cls_user.sh... 2024-04-22T20:00:30.771 DEBUG:teuthology.orchestra.run.smithi136:workunit test cls/test_cls_user.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh 2024-04-22T20:00:30.863 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:30.863 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:30.863 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 4 tests from 1 test suite. 2024-04-22T20:00:30.863 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:32.733 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 4 tests from ClsAccount 2024-04-22T20:00:32.734 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsAccount.add 2024-04-22T20:00:32.753 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsAccount.add (19 ms) 2024-04-22T20:00:32.753 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsAccount.get 2024-04-22T20:00:32.758 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsAccount.get (5 ms) 2024-04-22T20:00:32.758 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsAccount.rm 2024-04-22T20:00:33.166 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsAccount.rm (408 ms) 2024-04-22T20:00:33.166 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] ClsAccount.list 2024-04-22T20:00:33.185 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] ClsAccount.list (19 ms) 2024-04-22T20:00:33.185 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 4 tests from ClsAccount (451 ms total) 2024-04-22T20:00:33.185 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:33.185 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:33.746 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 4 tests from 1 test suite ran. (2883 ms total) 2024-04-22T20:00:33.746 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 4 tests. 2024-04-22T20:00:33.749 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T20:00:33.749 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T20:00:33.792 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2024-04-22T20:00:33.793 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2024-04-22T20:00:33.793 DEBUG:teuthology.orchestra.run.smithi136:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2024-04-22T20:00:33.885 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:33.885 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:33.885 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 6 tests from 1 test suite. 2024-04-22T20:00:33.886 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:35.748 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 6 tests from rgw_gc_log 2024-04-22T20:00:35.748 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2024-04-22T20:00:35.770 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] rgw_gc_log.init_existing_queue (22 ms) 2024-04-22T20:00:35.770 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2024-04-22T20:00:35.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] rgw_gc_log.init_existing_omap (3 ms) 2024-04-22T20:00:35.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2024-04-22T20:00:35.776 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (4 ms) 2024-04-22T20:00:35.776 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2024-04-22T20:00:35.778 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (2 ms) 2024-04-22T20:00:35.778 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2024-04-22T20:00:35.781 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] rgw_gc_log.defer1_after_init (2 ms) 2024-04-22T20:00:35.781 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2024-04-22T20:00:35.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] rgw_gc_log.defer2_before_init (2 ms) 2024-04-22T20:00:35.782 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 6 tests from rgw_gc_log (35 ms total) 2024-04-22T20:00:35.782 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:35.783 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:36.762 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 6 tests from 1 test suite ran. (2876 ms total) 2024-04-22T20:00:36.762 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 6 tests. 2024-04-22T20:00:36.767 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T20:00:36.767 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T20:00:36.808 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_obj.sh on client.0... 2024-04-22T20:00:36.809 INFO:tasks.workunit:Running workunit rgw/test_rgw_obj.sh... 2024-04-22T20:00:36.809 DEBUG:teuthology.orchestra.run.smithi136:workunit test rgw/test_rgw_obj.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_obj.sh 2024-04-22T20:00:36.902 INFO:tasks.workunit.client.0.smithi136.stdout:Running main() from gmock_main.cc 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 4 tests from 1 test suite. 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 4 tests from TestRGWObj 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestRGWObj.underscore 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns= i= 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns= i= 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.903 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns= i= 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "obj1": { 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.904 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "obj2": { 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.905 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3": { 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.906 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.907 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3-2": { 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.908 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns=ns i= 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns=ns i= 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns=ns i= 2024-04-22T20:00:36.909 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "obj1": { 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.910 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "obj2": { 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.911 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3": { 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.912 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3-2": { 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.913 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.914 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns= i=v1 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns= i=v1 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns= i=v1 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout: "obj1": { 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.915 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "obj2": { 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.916 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3": { 2024-04-22T20:00:36.917 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.918 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3-2": { 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.919 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns=ns i=v1 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns=ns i=v1 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: _obj[v1] 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: _obj ns=ns i=v1 2024-04-22T20:00:36.920 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "obj1": { 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.921 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "obj2": { 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.922 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3": { 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.923 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3-2": { 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.924 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_obj", 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestRGWObj.underscore (1 ms) 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestRGWObj.no_underscore 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns= i= 2024-04-22T20:00:36.925 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns= i= 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns= i= 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "obj1": { 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.926 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "obj2": { 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.927 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3": { 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.928 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3-2": { 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.929 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.930 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns=ns i= 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns=ns i= 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns=ns i= 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout: "obj1": { 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.931 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.932 INFO:tasks.workunit.client.0.smithi136.stdout: "obj2": { 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.933 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3": { 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.934 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3-2": { 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.935 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns= i=v1 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns= i=v1 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns= i=v1 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout: "obj1": { 2024-04-22T20:00:36.936 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.937 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "obj2": { 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.938 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3": { 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.939 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3-2": { 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.940 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns=ns i=v1 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.941 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns=ns i=v1 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout:parsed: obj[v1] 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout:stripped: obj ns=ns i=v1 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "obj1": { 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.942 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "obj2": { 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.943 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3": { 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.944 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "obj3-2": { 2024-04-22T20:00:36.945 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "obj", 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "v1", 2024-04-22T20:00:36.946 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestRGWObj.no_underscore (0 ms) 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestRGWObj.obj_to_raw 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.947 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.948 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker_myobj", 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.949 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.950 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.951 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.952 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker_myobj", 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.953 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.954 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.955 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.956 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns_myobj", 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.957 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.958 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.959 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.960 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns_myobj", 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.961 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.962 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.963 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.964 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst_myobj", 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.965 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.966 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.967 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.968 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst_myobj", 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.969 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.970 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.971 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.972 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst_myobj", 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.973 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.974 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.975 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.976 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst_myobj", 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.977 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.978 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.979 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.980 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker___myobj", 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj" 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.981 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.982 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.983 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.984 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker___myobj", 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj" 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.985 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.986 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.987 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.988 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns__myobj", 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.989 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.990 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.991 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.992 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns__myobj", 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.993 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.994 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.995 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.996 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst__myobj", 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj" 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.997 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.998 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:36.999 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.000 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst__myobj", 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj" 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.001 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.002 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.003 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.004 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst__myobj", 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.005 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.006 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.007 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst__myobj", 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.008 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.009 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.010 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.011 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker___myobj_", 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj_" 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.012 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.013 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.014 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.015 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker___myobj_", 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj_" 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.016 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.017 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.018 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns__myobj_", 2024-04-22T20:00:37.019 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.020 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.021 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.022 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns__myobj_", 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.023 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.024 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.025 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.026 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst__myobj_", 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj_" 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.027 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.028 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.029 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.030 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst__myobj_", 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj_" 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.031 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.032 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.033 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": "default-placement.data", 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst__myobj_", 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.034 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "test", 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": "", 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": "" 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.035 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.036 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "obj": { 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.037 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst__myobj_", 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.038 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.039 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestRGWObj.obj_to_raw (0 ms) 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] TestRGWObj.old_to_raw 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker_myobj", 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.040 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.041 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker_myobj 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker_myobj", 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.042 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker_myobj 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.043 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns_myobj", 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.044 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__ns_myobj 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns_myobj", 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.045 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.046 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__ns_myobj 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst_myobj", 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.047 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.048 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__:inst_myobj 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst_myobj", 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.049 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.050 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__:inst_myobj 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst_myobj", 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.051 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__ns:inst_myobj 2024-04-22T20:00:37.052 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst_myobj", 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.053 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "myobj", 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__ns:inst_myobj 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker___myobj", 2024-04-22T20:00:37.054 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj" 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.055 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker___myobj 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker___myobj", 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj" 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.056 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.057 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker___myobj 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns__myobj", 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.058 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__ns__myobj 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.059 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns__myobj", 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.060 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__ns__myobj 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst__myobj", 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj" 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.061 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.062 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__:inst__myobj 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst__myobj", 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj" 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.063 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.064 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__:inst__myobj 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst__myobj", 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.065 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.066 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__ns:inst__myobj 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst__myobj", 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.067 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj", 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__ns:inst__myobj 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.068 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker___myobj_", 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj_" 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.069 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker___myobj_ 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker___myobj_", 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj_" 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.070 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.071 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker___myobj_ 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns__myobj_", 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.072 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__ns__myobj_ 2024-04-22T20:00:37.073 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns__myobj_", 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.074 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "", 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__ns__myobj_ 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.075 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst__myobj_", 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj_" 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.076 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__:inst__myobj_ 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__:inst__myobj_", 2024-04-22T20:00:37.077 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "marker__myobj_" 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.078 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "" 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__:inst__myobj_ 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj": { 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst__myobj_", 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.079 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj": { 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.080 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout:raw=.data-pool:marker__ns:inst__myobj_ 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: "raw_obj2": { 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: "pool": ".data-pool", 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: "oid": "marker__ns:inst__myobj_", 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: "loc": "" 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.081 INFO:tasks.workunit.client.0.smithi136.stdout: "new_obj2": { 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket": { 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "ebtest", 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "marker": "marker", 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_id": "bucket-id", 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "tenant": "", 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "explicit_placement": { 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "data_pool": ".data-pool", 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "data_extra_pool": "", 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "index_pool": ".index-pool" 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "key": { 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "name": "_myobj_", 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "instance": "inst", 2024-04-22T20:00:37.082 INFO:tasks.workunit.client.0.smithi136.stdout: "ns": "ns" 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout:raw2=.data-pool:marker__ns:inst__myobj_ 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] TestRGWObj.old_to_raw (0 ms) 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 4 tests from TestRGWObj (1 ms total) 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 4 tests from 1 test suite ran. (1 ms total) 2024-04-22T20:00:37.083 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 4 tests. 2024-04-22T20:00:37.083 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T20:00:37.083 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T20:00:37.124 INFO:tasks.workunit:Running workunits matching rgw/test_librgw_file.sh on client.0... 2024-04-22T20:00:37.125 INFO:tasks.workunit:Running workunit rgw/test_librgw_file.sh... 2024-04-22T20:00:37.125 DEBUG:teuthology.orchestra.run.smithi136:workunit test rgw/test_librgw_file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2024-04-22T20:00:37.240 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:37.240 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:37.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.286+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.288+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.288+0000 7fa912c13800 20 realm 2024-04-22T20:00:37.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.288+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.288+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.288+0000 7fa912c13800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:00:37.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.288+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.288+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.288+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.289+0000 7fa912c13800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:00:37.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.289+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.290+0000 7fa912c13800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:00:37.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.290+0000 7fa912c13800 20 searching for the correct realm 2024-04-22T20:00:37.297 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.297+0000 7fa90ea6f640 0 -- 172.21.15.136:0/3181767336 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fa90805df60 msgr2=0x7fa9080771f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:00:37.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.503+0000 7fa905a6d640 0 -- 172.21.15.136:0/3181767336 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fa9080777d0 msgr2=0x7fa908073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.709+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:00:37.710 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 20 zone default found 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 4 Realm: () 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 20 zonegroup default 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.710+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.711+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.711+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.711+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.711+0000 7fa912c13800 20 started sync module instance, tier type = 2024-04-22T20:00:37.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.711+0000 7fa912c13800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:00:37.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.721+0000 7fa912c13800 20 add_watcher() i=0 2024-04-22T20:00:37.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.721+0000 7fa912c13800 20 add_watcher() i=1 2024-04-22T20:00:37.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.721+0000 7fa912c13800 20 add_watcher() i=2 2024-04-22T20:00:37.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.721+0000 7fa912c13800 20 add_watcher() i=3 2024-04-22T20:00:37.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.721+0000 7fa912c13800 20 add_watcher() i=4 2024-04-22T20:00:37.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.721+0000 7fa912c13800 20 add_watcher() i=5 2024-04-22T20:00:37.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.721+0000 7fa912c13800 20 add_watcher() i=6 2024-04-22T20:00:37.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.722+0000 7fa912c13800 20 add_watcher() i=7 2024-04-22T20:00:37.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.722+0000 7fa912c13800 2 all 8 watchers are set, enabling cache 2024-04-22T20:00:37.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.724+0000 7fa7c4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:00:37.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.724+0000 7fa912c13800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:00:37.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.724+0000 7fa912c13800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:00:37.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.724+0000 7fa912c13800 5 note: GC not initialized 2024-04-22T20:00:37.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.724+0000 7fa73f7fe640 20 reqs_thread_entry: start 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.724+0000 7fa912c13800 20 init_complete bucket index max shards: 1 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.724+0000 7fa912c13800 20 Filter name: none 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.724+0000 7fa73d7fa640 20 reqs_thread_entry: start 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 10 cache get: name=default.rgw.meta+users.uid+ceph-test-librgw-file : miss 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x0 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 10 adding default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 10 cache get: name=default.rgw.meta+users.email+librgw@example.com : miss 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x0 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 10 adding default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 10 cache get: name=default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= : miss 2024-04-22T20:00:37.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.725+0000 7fa912c13800 20 rados->read ofs=0 len=0 2024-04-22T20:00:37.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.726+0000 7fa912c13800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:37.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.726+0000 7fa912c13800 10 cache put: name=default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= info.flags=0x0 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.726+0000 7fa912c13800 10 adding default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= to cache LRU end 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.726+0000 7fa912c13800 10 cache get: name=default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= : hit (negative entry) 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.726+0000 7fa912c13800 10 cache get: name=default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= : hit (negative entry) 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.727+0000 7fa912c13800 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.727+0000 7fa912c13800 10 moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.727+0000 7fa912c13800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-librgw-file, ofs0, ns] 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.727+0000 7fa7f1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922116 cookie 94397049478464 notifier 5538 bl.length()=697 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.727+0000 7fa7f1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-04-22T20:00:37.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.727+0000 7fa7f1ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-04-22T20:00:37.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.730+0000 7fa912c13800 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-04-22T20:00:37.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.730+0000 7fa912c13800 10 moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-04-22T20:00:37.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.730+0000 7fa912c13800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:librgw@example.com, ofs0, ns] 2024-04-22T20:00:37.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.731+0000 7fa7f1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922112 cookie 94397049484720 notifier 5538 bl.length()=194 2024-04-22T20:00:37.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.731+0000 7fa7f1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-04-22T20:00:37.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.731+0000 7fa7f1ffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-04-22T20:00:37.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.734+0000 7fa912c13800 10 cache put: name=default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= info.flags=0x7 2024-04-22T20:00:37.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.734+0000 7fa912c13800 10 moving default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= to cache LRU end 2024-04-22T20:00:37.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.734+0000 7fa912c13800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:Bca3w+sQe3mTPpHg0dUjQYXGRa4=, ofs0, ns] 2024-04-22T20:00:37.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.734+0000 7fa7f17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922112 cookie 94397049499648 notifier 5538 bl.length()=203 2024-04-22T20:00:37.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.734+0000 7fa7f17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= info.flags=0x7 2024-04-22T20:00:37.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.734+0000 7fa7f17fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+Bca3w+sQe3mTPpHg0dUjQYXGRa4= to cache LRU end 2024-04-22T20:00:37.735 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "user_id": "ceph-test-librgw-file", 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "display_name": "librgw test user", 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "email": "librgw@example.com", 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "suspended": 0, 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "subusers": [], 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "keys": [ 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: { 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "user": "ceph-test-librgw-file", 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "access_key": "Bca3w+sQe3mTPpHg0dUjQYXGRa4=", 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "secret_key": "qJgbPJAfl/oPK3N+Ag5SfkUOz865xYilqdbLpaoMvEMc8LpTKC/T6w==", 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "active": true, 2024-04-22T20:00:37.736 INFO:tasks.workunit.client.0.smithi136.stdout: "create_date": "2024-04-22T20:00:37.727442Z" 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: ], 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "swift_keys": [], 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "caps": [], 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "default_placement": "", 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "placement_tags": [], 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_quota": { 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "enabled": false, 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size": -1, 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:00:37.737 INFO:tasks.workunit.client.0.smithi136.stdout: "max_objects": -1 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "user_quota": { 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "enabled": false, 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size": -1, 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "max_objects": -1 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "type": "rgw", 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "account_id": "", 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "path": "/", 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "create_date": "2024-04-22T20:00:37.727429Z", 2024-04-22T20:00:37.738 INFO:tasks.workunit.client.0.smithi136.stdout: "tags": [], 2024-04-22T20:00:37.739 INFO:tasks.workunit.client.0.smithi136.stdout: "group_ids": [] 2024-04-22T20:00:37.739 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:37.739 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:37.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.736+0000 7fa7c4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:00:37.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.738+0000 7fa912c13800 20 remove_watcher() i=0 2024-04-22T20:00:37.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.738+0000 7fa912c13800 2 removed watcher, disabling cache 2024-04-22T20:00:37.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.740+0000 7fa912c13800 20 remove_watcher() i=1 2024-04-22T20:00:37.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.741+0000 7fa912c13800 20 remove_watcher() i=2 2024-04-22T20:00:37.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.742+0000 7fa912c13800 20 remove_watcher() i=3 2024-04-22T20:00:37.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.743+0000 7fa912c13800 20 remove_watcher() i=4 2024-04-22T20:00:37.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.744+0000 7fa912c13800 20 remove_watcher() i=5 2024-04-22T20:00:37.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.745+0000 7fa912c13800 20 remove_watcher() i=6 2024-04-22T20:00:37.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:37.746+0000 7fa912c13800 20 remove_watcher() i=7 2024-04-22T20:00:37.755 INFO:tasks.workunit.client.0.smithi136.stdout:phase 1.1 2024-04-22T20:00:37.776 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 24 tests from 1 test suite. 2024-04-22T20:00:37.776 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:37.776 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 24 tests from LibRGW 2024-04-22T20:00:37.776 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:37.787 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:37.787 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:38.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:38.289+0000 7ff7275268c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:38.361 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (585 ms) 2024-04-22T20:00:38.361 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-04-22T20:00:38.362 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT_NOROOT (1 ms) 2024-04-22T20:00:38.362 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:38.362 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-22T20:00:38.362 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-04-22T20:00:38.373 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/baz/quux 2024-04-22T20:00:38.377 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/f1 2024-04-22T20:00:38.380 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/f2 2024-04-22T20:00:38.385 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/f1 2024-04-22T20:00:38.388 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/d1/ 2024-04-22T20:00:38.392 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/baz/hungry 2024-04-22T20:00:38.394 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/baz/hungry/ 2024-04-22T20:00:38.397 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/baz/momma 2024-04-22T20:00:38.400 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/baz/bear/ 2024-04-22T20:00:38.403 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/baz/sasquatch 2024-04-22T20:00:38.407 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/baz/sasquatch/ 2024-04-22T20:00:38.410 INFO:tasks.workunit.client.0.smithi136.stdout:creating: nfsroot:foo/bar/baz/frobozz 2024-04-22T20:00:38.412 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETUP_HIER1 (51 ms) 2024-04-22T20:00:38.412 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-04-22T20:00:38.509 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETUP_DIRS1 (96 ms) 2024-04-22T20:00:38.509 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETATTR 2024-04-22T20:00:38.509 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-04-22T20:00:38.509 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-04-22T20:00:38.526 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (17 ms) 2024-04-22T20:00:38.526 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-04-22T20:00:38.565 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (39 ms) 2024-04-22T20:00:38.565 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-04-22T20:00:38.565 INFO:tasks.workunit.client.0.smithi136.stdout:rename file rfile_0 to rfile_r0 (bucket brename0) 2024-04-22T20:00:38.572 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (7 ms) 2024-04-22T20:00:38.572 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-04-22T20:00:38.572 INFO:tasks.workunit.client.0.smithi136.stdout:rename file rfile_1 (bucket brename0) to rfile_rhilldog (bucket brename1) 2024-04-22T20:00:38.578 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (6 ms) 2024-04-22T20:00:38.578 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-04-22T20:00:38.578 INFO:tasks.workunit.client.0.smithi136.stdout:scanning objects in /bdirs1/dir_0 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout:scanning objects in /bdirs1/dir_1 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout:scanning objects in /bdirs1/dir_2 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-22T20:00:38.579 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-04-22T20:00:38.580 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-04-22T20:00:38.580 INFO:tasks.workunit.client.0.smithi136.stdout:read back objects in /bdirs1/dir_0 2024-04-22T20:00:38.580 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_0/sfile_0 2024-04-22T20:00:38.582 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-22T20:00:38.583 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_0/sfile_1 2024-04-22T20:00:38.584 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-22T20:00:38.584 INFO:tasks.workunit.client.0.smithi136.stdout:read back objects in /bdirs1/dir_1 2024-04-22T20:00:38.584 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_1/sfile_0 2024-04-22T20:00:38.585 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-22T20:00:38.585 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_1/sfile_1 2024-04-22T20:00:38.587 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-22T20:00:38.587 INFO:tasks.workunit.client.0.smithi136.stdout:read back objects in /bdirs1/dir_2 2024-04-22T20:00:38.587 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_2/sfile_0 2024-04-22T20:00:38.588 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-22T20:00:38.588 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_2/sfile_1 2024-04-22T20:00:38.590 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-22T20:00:38.590 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READ_DIRS1 (11 ms) 2024-04-22T20:00:38.590 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READF_SETUP1 (172 ms) 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_0 type: DIR refs: 2 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_0 type: FILE refs: 2 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_1 type: DIR refs: 2 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_1 type: FILE refs: 2 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_0 type: DIR refs: 2 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_0 type: FILE refs: 2 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_1 type: DIR refs: 2 2024-04-22T20:00:38.762 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_1 type: FILE refs: 2 2024-04-22T20:00:38.763 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_0 type: DIR refs: 2 2024-04-22T20:00:38.763 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_0 type: FILE refs: 2 2024-04-22T20:00:38.763 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_1 type: DIR refs: 2 2024-04-22T20:00:38.763 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_1 type: FILE refs: 2 2024-04-22T20:00:38.763 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-04-22T20:00:38.763 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.HIER1 2024-04-22T20:00:38.769 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/f2 (f2): FILE 2024-04-22T20:00:38.770 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/f1 (f1): FILE 2024-04-22T20:00:38.772 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-04-22T20:00:38.773 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-04-22T20:00:38.776 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-04-22T20:00:38.777 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-04-22T20:00:38.777 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-04-22T20:00:38.778 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-04-22T20:00:38.778 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-04-22T20:00:38.778 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-04-22T20:00:38.779 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-04-22T20:00:38.781 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-04-22T20:00:38.781 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-04-22T20:00:38.781 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar (bar): DIR 2024-04-22T20:00:38.781 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo (foo): DIR 2024-04-22T20:00:38.781 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot (nfsroot): DIR 2024-04-22T20:00:38.781 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.HIER1 (19 ms) 2024-04-22T20:00:38.781 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-22T20:00:38.781 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:38.782 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:38.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:38.780+0000 7ff7275268c0 -1 shutting down 2024-04-22T20:00:38.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:38.782+0000 7ff600b79640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 63032, si_uid : 1000, si_addr0x3e80000f638, si_status0 2024-04-22T20:00:38.826 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (45 ms) 2024-04-22T20:00:38.826 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 24 tests from LibRGW (1050 ms total) 2024-04-22T20:00:38.826 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:38.826 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:38.826 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 24 tests from 1 test suite ran. (1050 ms total) 2024-04-22T20:00:38.826 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 24 tests. 2024-04-22T20:00:38.832 INFO:tasks.workunit.client.0.smithi136.stdout:phase 1.2 2024-04-22T20:00:38.852 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 24 tests from 1 test suite. 2024-04-22T20:00:38.852 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:38.852 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 24 tests from LibRGW 2024-04-22T20:00:38.852 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:38.860 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:38.860 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:39.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:39.354+0000 7f6e234128c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:39.425 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (573 ms) 2024-04-22T20:00:39.425 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-04-22T20:00:39.426 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT_NOROOT (1 ms) 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETUP_HIER1 (0 ms) 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETUP_DIRS1 (0 ms) 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETATTR 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-04-22T20:00:39.427 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READ_DIRS1 (0 ms) 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-04-22T20:00:39.428 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.HIER1 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.HIER1 (0 ms) 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:39.429 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:39.430 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:39.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:39.426+0000 7f6e234128c0 -1 shutting down 2024-04-22T20:00:39.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:39.427+0000 7f6cfa86e640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 63650, si_uid : 1000, si_addr0x3e80000f8a2, si_status0 2024-04-22T20:00:39.499 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (73 ms) 2024-04-22T20:00:39.499 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 24 tests from LibRGW (647 ms total) 2024-04-22T20:00:39.499 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:39.499 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:39.499 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 24 tests from 1 test suite ran. (647 ms total) 2024-04-22T20:00:39.500 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 24 tests. 2024-04-22T20:00:39.504 INFO:tasks.workunit.client.0.smithi136.stdout:phase 1.3 2024-04-22T20:00:39.524 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 24 tests from 1 test suite. 2024-04-22T20:00:39.524 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:39.524 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 24 tests from LibRGW 2024-04-22T20:00:39.524 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:39.534 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:39.534 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:40.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:40.029+0000 7efd7e8c58c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:40.099 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (576 ms) 2024-04-22T20:00:40.100 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-04-22T20:00:40.101 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT_NOROOT (1 ms) 2024-04-22T20:00:40.101 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:40.101 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-22T20:00:40.101 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-04-22T20:00:40.106 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETUP_HIER1 (5 ms) 2024-04-22T20:00:40.106 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-04-22T20:00:40.166 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETUP_DIRS1 (60 ms) 2024-04-22T20:00:40.166 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SETATTR 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout:scanning objects in /bdirs1/dir_0 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:40.167 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout:scanning objects in /bdirs1/dir_1 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout:scanning objects in /bdirs1/dir_2 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-04-22T20:00:40.168 INFO:tasks.workunit.client.0.smithi136.stdout:read back objects in /bdirs1/dir_0 2024-04-22T20:00:40.169 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_0/sfile_0 2024-04-22T20:00:40.169 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-22T20:00:40.169 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_0/sfile_1 2024-04-22T20:00:40.169 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-22T20:00:40.169 INFO:tasks.workunit.client.0.smithi136.stdout:read back objects in /bdirs1/dir_1 2024-04-22T20:00:40.169 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_1/sfile_0 2024-04-22T20:00:40.171 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-22T20:00:40.171 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_1/sfile_1 2024-04-22T20:00:40.172 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-22T20:00:40.172 INFO:tasks.workunit.client.0.smithi136.stdout:read back objects in /bdirs1/dir_2 2024-04-22T20:00:40.173 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_2/sfile_0 2024-04-22T20:00:40.174 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_0 : "data for sfile_0" 2024-04-22T20:00:40.174 INFO:tasks.workunit.client.0.smithi136.stdout:reading 0,256 dir_2/sfile_1 2024-04-22T20:00:40.175 INFO:tasks.workunit.client.0.smithi136.stdout: read back from sfile_1 : "data for sfile_1" 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READ_DIRS1 (9 ms) 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_0 type: DIR refs: 2 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_0 type: FILE refs: 2 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_1 type: DIR refs: 2 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_1 type: FILE refs: 2 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_0 type: DIR refs: 2 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_0 type: FILE refs: 2 2024-04-22T20:00:40.176 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_1 type: DIR refs: 2 2024-04-22T20:00:40.177 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_1 type: FILE refs: 2 2024-04-22T20:00:40.177 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_0 type: DIR refs: 2 2024-04-22T20:00:40.177 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_0 type: FILE refs: 2 2024-04-22T20:00:40.177 INFO:tasks.workunit.client.0.smithi136.stdout:release sdir_1 type: DIR refs: 2 2024-04-22T20:00:40.177 INFO:tasks.workunit.client.0.smithi136.stdout:release sfile_1 type: FILE refs: 2 2024-04-22T20:00:40.177 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (1 ms) 2024-04-22T20:00:40.177 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.HIER1 2024-04-22T20:00:40.182 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/f2 (f2): FILE 2024-04-22T20:00:40.182 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/f1 (f1): FILE 2024-04-22T20:00:40.185 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-04-22T20:00:40.187 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-04-22T20:00:40.189 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-04-22T20:00:40.190 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-04-22T20:00:40.190 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-04-22T20:00:40.191 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-04-22T20:00:40.191 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-04-22T20:00:40.192 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-04-22T20:00:40.192 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-04-22T20:00:40.194 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-04-22T20:00:40.194 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo/bar (bar): DIR 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot/foo (foo): DIR 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:/nfsroot (nfsroot): DIR 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.HIER1 (18 ms) 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:40.195 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:40.196 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:40.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:40.194+0000 7efd7e8c58c0 -1 shutting down 2024-04-22T20:00:40.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:40.195+0000 7efc5744d640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 64268, si_uid : 1000, si_addr0x3e80000fb0c, si_status0 2024-04-22T20:00:40.264 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (70 ms) 2024-04-22T20:00:40.264 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 24 tests from LibRGW (740 ms total) 2024-04-22T20:00:40.264 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:40.264 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:40.264 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 24 tests from 1 test suite ran. (740 ms total) 2024-04-22T20:00:40.264 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 24 tests. 2024-04-22T20:00:40.270 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2.1 2024-04-22T20:00:40.292 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 9 tests from 1 test suite. 2024-04-22T20:00:40.292 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:40.292 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 9 tests from LibRGW 2024-04-22T20:00:40.292 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:40.301 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:40.301 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:40.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:40.592+0000 7f1135f408c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:40.661 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (369 ms) 2024-04-22T20:00:40.661 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:40.662 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-22T20:00:40.662 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-22T20:00:40.671 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKET (9 ms) 2024-04-22T20:00:40.671 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-22T20:00:40.671 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-04-22T20:00:40.671 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-04-22T20:00:40.680 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave0 2024-04-22T20:00:40.688 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave1 2024-04-22T20:00:40.699 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave2 2024-04-22T20:00:40.708 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave3 2024-04-22T20:00:40.717 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave4 2024-04-22T20:00:40.725 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave5 2024-04-22T20:00:40.734 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave6 2024-04-22T20:00:40.743 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave7 2024-04-22T20:00:40.752 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave8 2024-04-22T20:00:40.761 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave9 2024-04-22T20:00:40.761 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (90 ms) 2024-04-22T20:00:40.761 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-04-22T20:00:40.933 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (172 ms) 2024-04-22T20:00:40.933 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:40.933 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:40.933 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:40.933 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:40.933 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:40.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:40.933+0000 7f1135f408c0 -1 shutting down 2024-04-22T20:00:40.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:40.934+0000 7f100f642640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 64886, si_uid : 1000, si_addr0x3e80000fd76, si_status0 2024-04-22T20:00:41.037 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (104 ms) 2024-04-22T20:00:41.037 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 9 tests from LibRGW (745 ms total) 2024-04-22T20:00:41.037 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:41.037 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:41.037 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 9 tests from 1 test suite ran. (745 ms total) 2024-04-22T20:00:41.037 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 9 tests. 2024-04-22T20:00:41.043 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2.2 2024-04-22T20:00:41.064 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 9 tests from 1 test suite. 2024-04-22T20:00:41.064 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:41.064 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 9 tests from LibRGW 2024-04-22T20:00:41.064 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:41.073 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:41.073 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:41.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:41.567+0000 7fe6f25a28c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:41.637 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (573 ms) 2024-04-22T20:00:41.637 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:41.637 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-22T20:00:41.638 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-22T20:00:41.638 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-04-22T20:00:41.638 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-22T20:00:41.658 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_BUCKET (20 ms) 2024-04-22T20:00:41.658 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-04-22T20:00:41.666 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave0 2024-04-22T20:00:41.675 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave1 2024-04-22T20:00:41.684 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave2 2024-04-22T20:00:41.693 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave3 2024-04-22T20:00:41.701 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave4 2024-04-22T20:00:41.710 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave5 2024-04-22T20:00:41.719 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave6 2024-04-22T20:00:41.728 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave7 2024-04-22T20:00:41.737 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave8 2024-04-22T20:00:41.747 INFO:tasks.workunit.client.0.smithi136.stdout:created: sorrydave9 2024-04-22T20:00:41.747 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (89 ms) 2024-04-22T20:00:41.747 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-04-22T20:00:41.921 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (174 ms) 2024-04-22T20:00:41.921 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:41.921 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:41.921 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:41.921 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:41.921 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:41.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:41.920+0000 7fe6f25a28c0 -1 shutting down 2024-04-22T20:00:41.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:41.922+0000 7fe5cdae4640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 65504, si_uid : 1000, si_addr0x3e80000ffe0, si_status0 2024-04-22T20:00:42.013 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (92 ms) 2024-04-22T20:00:42.013 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 9 tests from LibRGW (949 ms total) 2024-04-22T20:00:42.013 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:42.013 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:42.013 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 9 tests from 1 test suite ran. (949 ms total) 2024-04-22T20:00:42.013 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 9 tests. 2024-04-22T20:00:42.018 INFO:tasks.workunit.client.0.smithi136.stdout:phase 3.1 2024-04-22T20:00:42.329 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 13 tests from 1 test suite. 2024-04-22T20:00:42.329 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:42.329 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 13 tests from LibRGW 2024-04-22T20:00:42.329 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:42.338 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:42.338 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:42.838 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:42.838+0000 7f3612ac38c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:42.910 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (580 ms) 2024-04-22T20:00:42.910 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:42.910 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-22T20:00:42.911 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-22T20:00:42.920 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKET (9 ms) 2024-04-22T20:00:42.920 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-04-22T20:00:42.923 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (4 ms) 2024-04-22T20:00:42.923 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-04-22T20:00:42.925 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (1 ms) 2024-04-22T20:00:42.925 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LARGE1 2024-04-22T20:00:42.979 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LARGE1 (54 ms) 2024-04-22T20:00:42.979 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LARGE2 2024-04-22T20:00:43.032 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LARGE2 (53 ms) 2024-04-22T20:00:43.032 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-04-22T20:00:43.032 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-04-22T20:00:43.032 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-04-22T20:00:43.032 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-04-22T20:00:43.032 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-22T20:00:43.032 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-04-22T20:00:43.033 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:43.033 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:43.033 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:43.033 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:43.033 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:43.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:43.032+0000 7f3612ac38c0 -1 shutting down 2024-04-22T20:00:43.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:43.034+0000 7f34eb3e9640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 66122, si_uid : 1000, si_addr0x3e80001024a, si_status0 2024-04-22T20:00:43.072 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (41 ms) 2024-04-22T20:00:43.072 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 13 tests from LibRGW (743 ms total) 2024-04-22T20:00:43.073 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:43.073 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:43.073 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 13 tests from 1 test suite ran. (743 ms total) 2024-04-22T20:00:43.073 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 13 tests. 2024-04-22T20:00:43.079 INFO:tasks.workunit.client.0.smithi136.stdout:phase 3.2 2024-04-22T20:00:43.391 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 13 tests from 1 test suite. 2024-04-22T20:00:43.392 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:43.392 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 13 tests from LibRGW 2024-04-22T20:00:43.392 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:43.400 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:43.400 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:43.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:43.491+0000 7f4a747318c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:43.561 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (170 ms) 2024-04-22T20:00:43.561 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:43.562 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-22T20:00:43.562 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-22T20:00:43.563 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-04-22T20:00:43.563 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-04-22T20:00:43.568 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (5 ms) 2024-04-22T20:00:43.568 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-04-22T20:00:43.571 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (4 ms) 2024-04-22T20:00:43.572 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LARGE1 2024-04-22T20:00:43.625 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LARGE1 (54 ms) 2024-04-22T20:00:43.626 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LARGE2 2024-04-22T20:00:43.626 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LARGE2 (0 ms) 2024-04-22T20:00:43.626 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-04-22T20:00:43.626 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-04-22T20:00:43.626 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-04-22T20:00:43.630 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_OBJECT (4 ms) 2024-04-22T20:00:43.630 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-22T20:00:43.647 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_BUCKET (17 ms) 2024-04-22T20:00:43.647 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:43.647 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:43.647 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:43.647 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:43.647 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:43.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:43.647+0000 7f4a747318c0 -1 shutting down 2024-04-22T20:00:43.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:43.648+0000 7f494dea7640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 66740, si_uid : 1000, si_addr0x3e8000104b4, si_status0 2024-04-22T20:00:43.725 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (78 ms) 2024-04-22T20:00:43.725 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 13 tests from LibRGW (333 ms total) 2024-04-22T20:00:43.725 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:43.725 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:43.725 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 13 tests from 1 test suite ran. (333 ms total) 2024-04-22T20:00:43.725 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 13 tests. 2024-04-22T20:00:43.732 INFO:tasks.workunit.client.0.smithi136.stdout:phase 4.1 2024-04-22T20:00:43.753 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 11 tests from 1 test suite. 2024-04-22T20:00:43.753 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:43.753 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 11 tests from LibRGW 2024-04-22T20:00:43.753 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.TVAR 2024-04-22T20:00:43.753 INFO:tasks.workunit.client.0.smithi136.stdout:read i1: 64001 2024-04-22T20:00:43.753 INFO:tasks.workunit.client.0.smithi136.stdout:read s1: blunderbuss 2024-04-22T20:00:43.754 INFO:tasks.workunit.client.0.smithi136.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-04-22T20:00:43.754 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:43.764 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:43.764 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:44.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:44.263+0000 7f391c7478c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:44.334 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (581 ms) 2024-04-22T20:00:44.334 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:44.335 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-22T20:00:44.335 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-22T20:00:44.351 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (17 ms) 2024-04-22T20:00:44.352 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-22T20:00:44.896 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_0 2024-04-22T20:00:44.896 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_1 2024-04-22T20:00:44.896 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_2 2024-04-22T20:00:44.896 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_3 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_4 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_5 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_6 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_7 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_8 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_9 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_10 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_11 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_12 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_13 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_14 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_15 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_16 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_17 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_18 2024-04-22T20:00:44.897 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_19 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_20 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_21 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_22 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_23 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_24 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_25 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_26 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_27 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_28 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_29 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_30 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_31 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_32 2024-04-22T20:00:44.898 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_33 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_34 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_35 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_36 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_37 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_38 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_39 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_40 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_41 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_42 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_43 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_44 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_45 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_46 2024-04-22T20:00:44.899 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_47 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_48 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_49 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_50 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_51 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_52 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_53 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_54 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_55 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_56 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_57 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_58 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_59 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_60 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_61 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_62 2024-04-22T20:00:44.900 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_63 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_64 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_65 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_66 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_67 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_68 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_69 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_70 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_71 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_72 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_73 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_74 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_75 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_76 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_77 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_78 2024-04-22T20:00:44.901 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_79 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_80 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_81 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_82 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_83 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_84 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_85 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_86 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_87 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_88 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_89 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_90 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_91 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_92 2024-04-22T20:00:44.902 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_93 2024-04-22T20:00:44.903 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_94 2024-04-22T20:00:44.903 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_95 2024-04-22T20:00:44.903 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_96 2024-04-22T20:00:44.903 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_97 2024-04-22T20:00:44.903 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_98 2024-04-22T20:00:44.903 INFO:tasks.workunit.client.0.smithi136.stdout:created: dmarker:f_99 2024-04-22T20:00:44.903 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (544 ms) 2024-04-22T20:00:44.903 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-22T20:00:44.904 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=. flags=4 2024-04-22T20:00:44.904 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=.. flags=4 2024-04-22T20:00:44.904 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-04-22T20:00:44.904 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-04-22T20:00:44.904 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-04-22T20:00:44.905 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-04-22T20:00:44.906 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-04-22T20:00:44.907 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-04-22T20:00:44.908 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-04-22T20:00:44.909 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-04-22T20:00:44.910 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_READDIR (3 ms) 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-04-22T20:00:44.911 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-04-22T20:00:44.912 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-04-22T20:00:44.913 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-04-22T20:00:44.914 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-04-22T20:00:44.915 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-04-22T20:00:44.916 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-04-22T20:00:44.917 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER2_READDIR (2 ms) 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:44.918 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:44.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:44.901+0000 7f391c7478c0 -1 shutting down 2024-04-22T20:00:44.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:44.902+0000 7f37f5f4a640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 67358, si_uid : 1000, si_addr0x3e80001071e, si_status0 2024-04-22T20:00:45.010 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (109 ms) 2024-04-22T20:00:45.010 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 11 tests from LibRGW (1257 ms total) 2024-04-22T20:00:45.010 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:45.010 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:45.011 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 11 tests from 1 test suite ran. (1257 ms total) 2024-04-22T20:00:45.011 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 11 tests. 2024-04-22T20:00:45.016 INFO:tasks.workunit.client.0.smithi136.stdout:phase 4.2 2024-04-22T20:00:45.037 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 11 tests from 1 test suite. 2024-04-22T20:00:45.037 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:45.037 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 11 tests from LibRGW 2024-04-22T20:00:45.037 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.TVAR 2024-04-22T20:00:45.037 INFO:tasks.workunit.client.0.smithi136.stdout:read i1: 64001 2024-04-22T20:00:45.038 INFO:tasks.workunit.client.0.smithi136.stdout:read s1: blunderbuss 2024-04-22T20:00:45.038 INFO:tasks.workunit.client.0.smithi136.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-04-22T20:00:45.038 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:45.045 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:45.045 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:45.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.135+0000 7fbe799dc8c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:45.206 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (170 ms) 2024-04-22T20:00:45.206 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:45.207 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-22T20:00:45.207 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-04-22T20:00:45.217 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (9 ms) 2024-04-22T20:00:45.217 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-04-22T20:00:45.217 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-04-22T20:00:45.217 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-04-22T20:00:45.217 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-04-22T20:00:45.217 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-04-22T20:00:45.220 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-04-22T20:00:45.220 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-04-22T20:00:45.220 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-04-22T20:00:45.220 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-04-22T20:00:45.220 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-04-22T20:00:45.220 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-04-22T20:00:45.220 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-04-22T20:00:45.221 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-04-22T20:00:45.222 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-04-22T20:00:45.223 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-04-22T20:00:45.224 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-04-22T20:00:45.225 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-04-22T20:00:45.226 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER2_READDIR (3 ms) 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.220+0000 7fbe799dc8c0 -1 shutting down 2024-04-22T20:00:45.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.221+0000 7fbd54f1e640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 67976, si_uid : 1000, si_addr0x3e800010988, si_status0 2024-04-22T20:00:45.280 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (60 ms) 2024-04-22T20:00:45.280 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 11 tests from LibRGW (243 ms total) 2024-04-22T20:00:45.280 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:45.280 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:45.280 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 11 tests from 1 test suite ran. (243 ms total) 2024-04-22T20:00:45.280 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 11 tests. 2024-04-22T20:00:45.285 INFO:tasks.workunit.client.0.smithi136.stdout:phase 5.1 2024-04-22T20:00:45.315 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 19 tests from 1 test suite. 2024-04-22T20:00:45.315 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:45.315 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 19 tests from LibRGW 2024-04-22T20:00:45.315 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:45.325 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:45.325 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:45.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.415+0000 7fd8a3e408c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:45.486 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (172 ms) 2024-04-22T20:00:45.486 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:45.487 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-04-22T20:00:45.487 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-22T20:00:45.497 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKET (10 ms) 2024-04-22T20:00:45.497 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-04-22T20:00:45.501 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (4 ms) 2024-04-22T20:00:45.501 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-04-22T20:00:45.502 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-04-22T20:00:45.502 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-04-22T20:00:45.503 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (1 ms) 2024-04-22T20:00:45.503 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-04-22T20:00:45.503 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-04-22T20:00:45.503 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-04-22T20:00:45.507 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.PUT_OBJECT (5 ms) 2024-04-22T20:00:45.507 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-04-22T20:00:45.509 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.GET_OBJECT (1 ms) 2024-04-22T20:00:45.509 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-04-22T20:00:45.509 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.WRITEV 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READV 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READV (0 ms) 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-04-22T20:00:45.510 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:45.511 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:45.511 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:45.511 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:45.511 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:45.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.509+0000 7fd8a3e408c0 -1 shutting down 2024-04-22T20:00:45.511 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.510+0000 7fd77ce83640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 68594, si_uid : 1000, si_addr0x3e800010bf2, si_status0 2024-04-22T20:00:45.560 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (51 ms) 2024-04-22T20:00:45.560 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 19 tests from LibRGW (245 ms total) 2024-04-22T20:00:45.560 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:45.560 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:45.561 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 19 tests from 1 test suite ran. (245 ms total) 2024-04-22T20:00:45.561 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 19 tests. 2024-04-22T20:00:45.566 INFO:tasks.workunit.client.0.smithi136.stdout:phase 5.2 2024-04-22T20:00:45.595 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 19 tests from 1 test suite. 2024-04-22T20:00:45.595 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:45.595 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 19 tests from LibRGW 2024-04-22T20:00:45.595 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:45.602 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:45.602 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:45.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.694+0000 7f874332f8c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:45.764 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (170 ms) 2024-04-22T20:00:45.764 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:45.765 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-22T20:00:45.765 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-04-22T20:00:45.765 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-04-22T20:00:45.765 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-04-22T20:00:45.771 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (6 ms) 2024-04-22T20:00:45.771 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (0 ms) 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.PUT_OBJECT (0 ms) 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-04-22T20:00:45.772 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-04-22T20:00:45.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.WRITEV 2024-04-22T20:00:45.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-04-22T20:00:45.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READV 2024-04-22T20:00:45.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READV (0 ms) 2024-04-22T20:00:45.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-04-22T20:00:45.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-04-22T20:00:45.773 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-04-22T20:00:45.776 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_OBJECT (5 ms) 2024-04-22T20:00:45.776 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-04-22T20:00:45.793 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.DELETE_BUCKET (17 ms) 2024-04-22T20:00:45.793 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:45.793 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:45.793 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:45.793 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:45.793 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:45.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.793+0000 7f874332f8c0 -1 shutting down 2024-04-22T20:00:45.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:45.794+0000 7f861c994640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 69212, si_uid : 1000, si_addr0x3e800010e5c, si_status0 2024-04-22T20:00:45.929 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (137 ms) 2024-04-22T20:00:45.930 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 19 tests from LibRGW (335 ms total) 2024-04-22T20:00:45.930 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:45.930 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:45.930 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 19 tests from 1 test suite ran. (335 ms total) 2024-04-22T20:00:45.930 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 19 tests. 2024-04-22T20:00:45.935 INFO:tasks.workunit.client.0.smithi136.stdout:phase 6.1 2024-04-22T20:00:45.957 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] Running 10 tests from 1 test suite. 2024-04-22T20:00:45.957 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment set-up. 2024-04-22T20:00:45.957 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 10 tests from LibRGW 2024-04-22T20:00:45.957 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.INIT 2024-04-22T20:00:45.963 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:45.963 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:46.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.255+0000 7f4d9d52e8c0 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-22T20:00:46.325 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.INIT (370 ms) 2024-04-22T20:00:46.326 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.MOUNT 2024-04-22T20:00:46.326 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-04-22T20:00:46.326 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CREATE_BUCKETS 2024-04-22T20:00:46.344 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CREATE_BUCKETS (19 ms) 2024-04-22T20:00:46.345 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.LOOKUP_BUCKETS 2024-04-22T20:00:46.351 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.LOOKUP_BUCKETS (7 ms) 2024-04-22T20:00:46.351 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.TOPDIR_RENAME 2024-04-22T20:00:46.369 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.TOPDIR_RENAME (18 ms) 2024-04-22T20:00:46.369 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SUBDIR_RENAME 2024-04-22T20:00:46.392 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SUBDIR_RENAME (22 ms) 2024-04-22T20:00:46.392 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CROSS_BUCKET_RENAME 2024-04-22T20:00:46.413 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CROSS_BUCKET_RENAME (22 ms) 2024-04-22T20:00:46.414 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.CLEANUP 2024-04-22T20:00:46.414 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-04-22T20:00:46.414 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.UMOUNT 2024-04-22T20:00:46.414 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-04-22T20:00:46.414 INFO:tasks.workunit.client.0.smithi136.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-04-22T20:00:46.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.413+0000 7f4d9d52e8c0 -1 shutting down 2024-04-22T20:00:46.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.414+0000 7f4c78a70640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 69830, si_uid : 1000, si_addr0x3e8000110c6, si_status0 2024-04-22T20:00:46.490 INFO:tasks.workunit.client.0.smithi136.stdout:[ OK ] LibRGW.SHUTDOWN (76 ms) 2024-04-22T20:00:46.490 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] 10 tests from LibRGW (534 ms total) 2024-04-22T20:00:46.490 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:46.490 INFO:tasks.workunit.client.0.smithi136.stdout:[----------] Global test environment tear-down 2024-04-22T20:00:46.490 INFO:tasks.workunit.client.0.smithi136.stdout:[==========] 10 tests from 1 test suite ran. (534 ms total) 2024-04-22T20:00:46.490 INFO:tasks.workunit.client.0.smithi136.stdout:[ PASSED ] 10 tests. 2024-04-22T20:00:46.496 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T20:00:46.496 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T20:00:46.534 INFO:tasks.workunit:Running workunits matching rgw/test_awssdkv4_sig.sh on client.0... 2024-04-22T20:00:46.535 INFO:tasks.workunit:Running workunit rgw/test_awssdkv4_sig.sh... 2024-04-22T20:00:46.535 DEBUG:teuthology.orchestra.run.smithi136:workunit test rgw/test_awssdkv4_sig.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_awssdkv4_sig.sh 2024-04-22T20:00:46.605 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -z ']' 2024-04-22T20:00:46.605 INFO:tasks.workunit.client.0.smithi136.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-04-22T20:00:46.606 INFO:tasks.workunit.client.0.smithi136.stderr:+ AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-04-22T20:00:46.606 INFO:tasks.workunit.client.0.smithi136.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-04-22T20:00:46.606 INFO:tasks.workunit.client.0.smithi136.stderr:+ AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-04-22T20:00:46.606 INFO:tasks.workunit.client.0.smithi136.stderr:+ radosgw-admin user create --uid ceph-test-maven --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name 'maven test user' --email sigv4@example.com 2024-04-22T20:00:46.640 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:00:46.641 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:00:46.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.674+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:46.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.675+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:46.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.675+0000 7f8e76f3f800 20 realm 2024-04-22T20:00:46.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.675+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:46.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.675+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:46.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.675+0000 7f8e76f3f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:00:46.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.675+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:46.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.676+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:46.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.676+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:46.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.676+0000 7f8e76f3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:00:46.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.676+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:46.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.677+0000 7f8e76f3f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:00:46.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.677+0000 7f8e76f3f800 20 searching for the correct realm 2024-04-22T20:00:46.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.679+0000 7f8e7349c640 0 -- 172.21.15.136:0/3954946571 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f8e6c035070 msgr2=0x7f8e6c041400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:00:46.884 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:46.884+0000 7f8e6bfff640 0 -- 172.21.15.136:0/3954946571 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f8e6c01a040 msgr2=0x7f8e6c0283b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:00:47.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.090+0000 7f8e76f3f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:00:47.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.090+0000 7f8e76f3f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:00:47.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.090+0000 7f8e76f3f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:00:47.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.090+0000 7f8e76f3f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:00:47.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.090+0000 7f8e76f3f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:00:47.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.090+0000 7f8e76f3f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:00:47.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.090+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.091+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:47.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.091+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.091+0000 7f8e76f3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:00:47.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.091+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:00:47.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 20 zone default found 2024-04-22T20:00:47.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 4 Realm: () 2024-04-22T20:00:47.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:00:47.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:00:47.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:00:47.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 20 zonegroup default 2024-04-22T20:00:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.092+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.093+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.093+0000 7f8e76f3f800 20 started sync module instance, tier type = 2024-04-22T20:00:47.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.093+0000 7f8e76f3f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:00:47.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.102+0000 7f8e76f3f800 20 add_watcher() i=0 2024-04-22T20:00:47.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.102+0000 7f8e76f3f800 20 add_watcher() i=1 2024-04-22T20:00:47.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.102+0000 7f8e76f3f800 20 add_watcher() i=2 2024-04-22T20:00:47.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.102+0000 7f8e76f3f800 20 add_watcher() i=3 2024-04-22T20:00:47.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.102+0000 7f8e76f3f800 20 add_watcher() i=4 2024-04-22T20:00:47.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.102+0000 7f8e76f3f800 20 add_watcher() i=5 2024-04-22T20:00:47.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.102+0000 7f8e76f3f800 20 add_watcher() i=6 2024-04-22T20:00:47.103 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.103+0000 7f8e76f3f800 20 add_watcher() i=7 2024-04-22T20:00:47.103 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.103+0000 7f8e76f3f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:00:47.104 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.104+0000 7f8d297e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:00:47.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8e76f3f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:00:47.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8e76f3f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:00:47.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8e76f3f800 5 note: GC not initialized 2024-04-22T20:00:47.106 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8ca3fff640 20 reqs_thread_entry: start 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8e76f3f800 20 init_complete bucket index max shards: 1 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8e76f3f800 20 Filter name: none 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8ca1ffb640 20 reqs_thread_entry: start 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8e76f3f800 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.105+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x0 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 cache get: name=default.rgw.meta+users.email+sigv4@example.com : miss 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x0 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 adding default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2024-04-22T20:00:47.107 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 20 rados->read ofs=0 len=0 2024-04-22T20:00:47.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:00:47.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x0 2024-04-22T20:00:47.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-04-22T20:00:47.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-04-22T20:00:47.108 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.106+0000 7f8e76f3f800 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-04-22T20:00:47.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.107+0000 7f8e76f3f800 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-04-22T20:00:47.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.107+0000 7f8e76f3f800 10 moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-04-22T20:00:47.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.107+0000 7f8e76f3f800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-maven, ofs0, ns] 2024-04-22T20:00:47.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.108+0000 7f8d567fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922112 cookie 94801816141328 notifier 5778 bl.length()=677 2024-04-22T20:00:47.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.108+0000 7f8d567fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-04-22T20:00:47.109 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.108+0000 7f8d567fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-04-22T20:00:47.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.111+0000 7f8e76f3f800 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-04-22T20:00:47.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.111+0000 7f8e76f3f800 10 moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-04-22T20:00:47.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.111+0000 7f8e76f3f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:sigv4@example.com, ofs0, ns] 2024-04-22T20:00:47.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.111+0000 7f8d55ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922122 cookie 94801816127440 notifier 5778 bl.length()=187 2024-04-22T20:00:47.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.111+0000 7f8d55ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-04-22T20:00:47.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.111+0000 7f8d55ffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-04-22T20:00:47.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.113+0000 7f8e76f3f800 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-04-22T20:00:47.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.113+0000 7f8e76f3f800 10 moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-04-22T20:00:47.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.113+0000 7f8e76f3f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:lNCnR47C2g+ZidCWBAUuwfSAA7Q=, ofs0, ns] 2024-04-22T20:00:47.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.113+0000 7f8d55ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1060856922123 cookie 94801816127440 notifier 5778 bl.length()=197 2024-04-22T20:00:47.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.113+0000 7f8d55ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-04-22T20:00:47.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.113+0000 7f8d55ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout: "user_id": "ceph-test-maven", 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout: "display_name": "maven test user", 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout: "email": "sigv4@example.com", 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout: "suspended": 0, 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout: "subusers": [], 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout: "keys": [ 2024-04-22T20:00:47.115 INFO:tasks.workunit.client.0.smithi136.stdout: { 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "user": "ceph-test-maven", 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "access_key": "lNCnR47C2g+ZidCWBAUuwfSAA7Q=", 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "secret_key": "tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g==", 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "active": true, 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "create_date": "2024-04-22T20:00:47.108002Z" 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: ], 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "swift_keys": [], 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "caps": [], 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "default_placement": "", 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "placement_tags": [], 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_quota": { 2024-04-22T20:00:47.116 INFO:tasks.workunit.client.0.smithi136.stdout: "enabled": false, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size": -1, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "max_objects": -1 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "user_quota": { 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "enabled": false, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size": -1, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "max_objects": -1 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "type": "rgw", 2024-04-22T20:00:47.117 INFO:tasks.workunit.client.0.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:00:47.118 INFO:tasks.workunit.client.0.smithi136.stdout: "account_id": "", 2024-04-22T20:00:47.118 INFO:tasks.workunit.client.0.smithi136.stdout: "path": "/", 2024-04-22T20:00:47.118 INFO:tasks.workunit.client.0.smithi136.stdout: "create_date": "2024-04-22T20:00:47.107988Z", 2024-04-22T20:00:47.118 INFO:tasks.workunit.client.0.smithi136.stdout: "tags": [], 2024-04-22T20:00:47.118 INFO:tasks.workunit.client.0.smithi136.stdout: "group_ids": [] 2024-04-22T20:00:47.118 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:00:47.118 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:00:47.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.116+0000 7f8d297e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:00:47.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.118+0000 7f8e76f3f800 20 remove_watcher() i=0 2024-04-22T20:00:47.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.118+0000 7f8e76f3f800 2 removed watcher, disabling cache 2024-04-22T20:00:47.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.119+0000 7f8e76f3f800 20 remove_watcher() i=1 2024-04-22T20:00:47.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.120+0000 7f8e76f3f800 20 remove_watcher() i=2 2024-04-22T20:00:47.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.121+0000 7f8e76f3f800 20 remove_watcher() i=3 2024-04-22T20:00:47.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.122+0000 7f8e76f3f800 20 remove_watcher() i=4 2024-04-22T20:00:47.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.123+0000 7f8e76f3f800 20 remove_watcher() i=5 2024-04-22T20:00:47.124 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.124+0000 7f8e76f3f800 20 remove_watcher() i=6 2024-04-22T20:00:47.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:00:47.125+0000 7f8e76f3f800 20 remove_watcher() i=7 2024-04-22T20:00:47.135 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -z ']' 2024-04-22T20:00:47.135 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -z /home/ubuntu/cephtest ']' 2024-04-22T20:00:47.135 INFO:tasks.workunit.client.0.smithi136.stderr:++ cat /home/ubuntu/cephtest/url_file 2024-04-22T20:00:47.136 INFO:tasks.workunit.client.0.smithi136.stderr:+ export RGW_HTTP_ENDPOINT_URL=https://smithi136.front.sepia.ceph.com:443 2024-04-22T20:00:47.136 INFO:tasks.workunit.client.0.smithi136.stderr:+ RGW_HTTP_ENDPOINT_URL=https://smithi136.front.sepia.ceph.com:443 2024-04-22T20:00:47.136 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -z /home/ubuntu/cephtest/clone.client.0 ']' 2024-04-22T20:00:47.136 INFO:tasks.workunit.client.0.smithi136.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum 2024-04-22T20:00:47.136 INFO:tasks.workunit.client.0.smithi136.stderr:+ ./mvnw clean package 2024-04-22T20:00:47.145 INFO:tasks.workunit.client.0.smithi136.stdout:Warning: JAVA_HOME environment variable is not set. 2024-04-22T20:00:48.651 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Scanning for projects... 2024-04-22T20:00:48.763 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom 2024-04-22T20:00:48.854 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/5.6 kB Progress (1): 2.8/5.6 kB Progress (1): 4.1/5.6 kB Progress (1): 5.5/5.6 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom (5.6 kB at 59 kB/s) 2024-04-22T20:00:48.865 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom 2024-04-22T20:00:48.927 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/83 kB Progress (1): 33/83 kB Progress (1): 49/83 kB Progress (1): 66/83 kB Progress (1): 82/83 kB Progress (1): 83 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom (83 kB at 1.3 MB/s) 2024-04-22T20:00:48.933 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom 2024-04-22T20:00:48.961 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/77 kB Progress (1): 33/77 kB Progress (1): 49/77 kB Progress (1): 66/77 kB Progress (1): 77 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom (77 kB at 2.8 MB/s) 2024-04-22T20:00:48.983 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:48.983 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-04-22T20:00:48.983 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Building jcksum 1.0 2024-04-22T20:00:48.983 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] from pom.xml 2024-04-22T20:00:48.983 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-04-22T20:00:48.986 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom 2024-04-22T20:00:49.004 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom (5.3 kB at 279 kB/s) 2024-04-22T20:00:49.011 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom 2024-04-22T20:00:49.028 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom (9.9 kB at 550 kB/s) 2024-04-22T20:00:49.031 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom 2024-04-22T20:00:49.051 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom (45 kB at 2.4 MB/s) 2024-04-22T20:00:49.056 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom 2024-04-22T20:00:49.074 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom (21 kB at 1.1 MB/s) 2024-04-22T20:00:49.085 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar 2024-04-22T20:00:49.104 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar (36 kB at 1.9 MB/s) 2024-04-22T20:00:49.111 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom 2024-04-22T20:00:49.129 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom (8.2 kB at 453 kB/s) 2024-04-22T20:00:49.131 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom 2024-04-22T20:00:49.149 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom (8.1 kB at 476 kB/s) 2024-04-22T20:00:49.151 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom 2024-04-22T20:00:49.170 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/48 kB Progress (1): 33/48 kB Progress (1): 48 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom (48 kB at 2.7 MB/s) 2024-04-22T20:00:49.174 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom 2024-04-22T20:00:49.191 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom (21 kB at 1.2 MB/s) 2024-04-22T20:00:49.196 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar 2024-04-22T20:00:49.215 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar (31 kB at 1.6 MB/s) 2024-04-22T20:00:49.220 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom 2024-04-22T20:00:49.238 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 9.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom (9.8 kB at 545 kB/s) 2024-04-22T20:00:49.243 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar 2024-04-22T20:00:49.263 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/66 kB Progress (1): 32/66 kB Progress (1): 49/66 kB Progress (1): 65/66 kB Progress (1): 66 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar (66 kB at 3.3 MB/s) 2024-04-22T20:00:49.268 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom 2024-04-22T20:00:49.286 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom (5.3 kB at 280 kB/s) 2024-04-22T20:00:49.289 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom 2024-04-22T20:00:49.308 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom (22 kB at 1.2 MB/s) 2024-04-22T20:00:49.310 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom 2024-04-22T20:00:49.329 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/50 kB Progress (1): 33/50 kB Progress (1): 49/50 kB Progress (1): 50 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom (50 kB at 2.6 MB/s) 2024-04-22T20:00:49.335 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom 2024-04-22T20:00:49.354 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom (24 kB at 1.2 MB/s) 2024-04-22T20:00:49.362 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom 2024-04-22T20:00:49.380 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom (5.6 kB at 313 kB/s) 2024-04-22T20:00:49.384 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar 2024-04-22T20:00:49.404 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar (45 kB at 2.2 MB/s) 2024-04-22T20:00:49.411 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom 2024-04-22T20:00:49.429 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom (6.8 kB at 376 kB/s) 2024-04-22T20:00:49.432 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom 2024-04-22T20:00:49.450 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom (9.9 kB at 550 kB/s) 2024-04-22T20:00:49.453 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom 2024-04-22T20:00:49.472 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/46 kB Progress (1): 33/46 kB Progress (1): 46 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom (46 kB at 2.4 MB/s) 2024-04-22T20:00:49.475 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom 2024-04-22T20:00:49.493 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom (20 kB at 1.1 MB/s) 2024-04-22T20:00:49.499 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar 2024-04-22T20:00:49.517 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar (27 kB at 1.5 MB/s) 2024-04-22T20:00:49.531 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom 2024-04-22T20:00:49.549 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom (6.1 kB at 320 kB/s) 2024-04-22T20:00:49.551 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom 2024-04-22T20:00:49.570 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom (21 kB at 1.1 MB/s) 2024-04-22T20:00:49.574 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom 2024-04-22T20:00:49.593 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom (20 kB at 1.1 MB/s) 2024-04-22T20:00:49.600 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom 2024-04-22T20:00:49.619 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom (4.0 kB at 211 kB/s) 2024-04-22T20:00:49.622 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom 2024-04-22T20:00:49.640 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom (1.6 kB at 82 kB/s) 2024-04-22T20:00:49.642 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom 2024-04-22T20:00:49.660 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom (2.2 kB at 124 kB/s) 2024-04-22T20:00:49.664 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom 2024-04-22T20:00:49.684 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom (3.6 kB at 182 kB/s) 2024-04-22T20:00:49.691 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom 2024-04-22T20:00:49.712 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom (3.2 kB at 153 kB/s) 2024-04-22T20:00:49.716 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom 2024-04-22T20:00:49.735 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom (11 kB at 609 kB/s) 2024-04-22T20:00:49.740 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom 2024-04-22T20:00:49.757 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom (1.7 kB at 94 kB/s) 2024-04-22T20:00:49.760 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom 2024-04-22T20:00:49.778 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom (3.8 kB at 212 kB/s) 2024-04-22T20:00:49.781 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom 2024-04-22T20:00:49.800 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom (4.9 kB at 273 kB/s) 2024-04-22T20:00:49.804 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom 2024-04-22T20:00:49.821 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom (1.2 kB at 68 kB/s) 2024-04-22T20:00:49.823 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom 2024-04-22T20:00:49.841 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom (3.8 kB at 202 kB/s) 2024-04-22T20:00:49.843 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom 2024-04-22T20:00:49.860 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom (14 kB at 812 kB/s) 2024-04-22T20:00:49.863 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom 2024-04-22T20:00:49.881 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom (3.7 kB at 204 kB/s) 2024-04-22T20:00:49.885 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom 2024-04-22T20:00:49.903 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom (2.5 kB at 136 kB/s) 2024-04-22T20:00:49.907 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom 2024-04-22T20:00:49.924 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom (2.9 kB at 162 kB/s) 2024-04-22T20:00:49.928 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom 2024-04-22T20:00:49.947 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 7.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom (7.4 kB at 412 kB/s) 2024-04-22T20:00:49.950 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom 2024-04-22T20:00:49.968 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom (5.5 kB at 303 kB/s) 2024-04-22T20:00:49.972 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom 2024-04-22T20:00:49.990 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom (3.5 kB at 194 kB/s) 2024-04-22T20:00:49.996 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom 2024-04-22T20:00:50.014 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom (4.4 kB at 244 kB/s) 2024-04-22T20:00:50.017 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom 2024-04-22T20:00:50.035 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom (2.4 kB at 127 kB/s) 2024-04-22T20:00:50.038 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom 2024-04-22T20:00:50.056 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom (6.4 kB at 358 kB/s) 2024-04-22T20:00:50.060 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom 2024-04-22T20:00:50.078 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom (6.1 kB at 319 kB/s) 2024-04-22T20:00:50.082 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom 2024-04-22T20:00:50.099 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom (3.0 kB at 165 kB/s) 2024-04-22T20:00:50.102 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom 2024-04-22T20:00:50.120 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom (3.5 kB at 193 kB/s) 2024-04-22T20:00:50.124 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom 2024-04-22T20:00:50.142 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom (4.1 kB at 227 kB/s) 2024-04-22T20:00:50.144 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom 2024-04-22T20:00:50.162 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom (1.8 kB at 99 kB/s) 2024-04-22T20:00:50.165 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom 2024-04-22T20:00:50.183 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (6.6 kB at 367 kB/s) 2024-04-22T20:00:50.185 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom 2024-04-22T20:00:50.203 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom (16 kB at 964 kB/s) 2024-04-22T20:00:50.205 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom 2024-04-22T20:00:50.223 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/35 kB Progress (1): 33/35 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom (35 kB at 1.9 MB/s) 2024-04-22T20:00:50.227 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom 2024-04-22T20:00:50.245 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom (17 kB at 950 kB/s) 2024-04-22T20:00:50.250 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom 2024-04-22T20:00:50.268 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom (5.0 kB at 276 kB/s) 2024-04-22T20:00:50.270 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom 2024-04-22T20:00:50.288 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom (13 kB at 732 kB/s) 2024-04-22T20:00:50.292 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom 2024-04-22T20:00:50.310 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom (14 kB at 776 kB/s) 2024-04-22T20:00:50.313 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom 2024-04-22T20:00:50.333 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/68 kB Progress (1): 33/68 kB Progress (1): 49/68 kB Progress (1): 66/68 kB Progress (1): 68 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom (68 kB at 3.4 MB/s) 2024-04-22T20:00:50.339 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2024-04-22T20:00:50.358 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom (16 kB at 825 kB/s) 2024-04-22T20:00:50.364 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom 2024-04-22T20:00:50.382 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom (15 kB at 858 kB/s) 2024-04-22T20:00:50.384 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom 2024-04-22T20:00:50.403 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/79 kB Progress (1): 32/79 kB Progress (1): 49/79 kB Progress (1): 65/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom (79 kB at 4.0 MB/s) 2024-04-22T20:00:50.408 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom 2024-04-22T20:00:50.427 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 1.0 MB/s) 2024-04-22T20:00:50.431 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom 2024-04-22T20:00:50.450 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom (20 kB at 1.1 MB/s) 2024-04-22T20:00:50.453 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom 2024-04-22T20:00:50.475 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 66/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom (78 kB at 3.7 MB/s) 2024-04-22T20:00:50.483 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom 2024-04-22T20:00:50.503 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom (3.8 kB at 192 kB/s) 2024-04-22T20:00:50.505 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom 2024-04-22T20:00:50.524 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom (14 kB at 767 kB/s) 2024-04-22T20:00:50.526 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom 2024-04-22T20:00:50.547 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom (3.2 kB at 160 kB/s) 2024-04-22T20:00:50.550 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom 2024-04-22T20:00:50.574 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom (3.2 kB at 133 kB/s) 2024-04-22T20:00:50.577 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom 2024-04-22T20:00:50.594 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom (2.0 kB at 120 kB/s) 2024-04-22T20:00:50.596 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom 2024-04-22T20:00:50.615 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom (2.8 kB at 157 kB/s) 2024-04-22T20:00:50.618 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom 2024-04-22T20:00:50.636 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom (1.5 kB at 85 kB/s) 2024-04-22T20:00:50.638 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom 2024-04-22T20:00:50.656 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom (3.0 kB at 177 kB/s) 2024-04-22T20:00:50.658 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom 2024-04-22T20:00:50.675 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom (3.2 kB at 178 kB/s) 2024-04-22T20:00:50.678 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom 2024-04-22T20:00:50.697 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom (3.2 kB at 160 kB/s) 2024-04-22T20:00:50.700 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom 2024-04-22T20:00:50.718 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom (1.0 kB at 57 kB/s) 2024-04-22T20:00:50.721 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom 2024-04-22T20:00:50.739 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 964 B Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom (964 B at 57 kB/s) 2024-04-22T20:00:50.741 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom 2024-04-22T20:00:50.758 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom (3.1 kB at 180 kB/s) 2024-04-22T20:00:50.761 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom 2024-04-22T20:00:50.778 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom (3.0 kB at 168 kB/s) 2024-04-22T20:00:50.780 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom 2024-04-22T20:00:50.798 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom (3.4 kB at 202 kB/s) 2024-04-22T20:00:50.800 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom 2024-04-22T20:00:50.817 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom (3.4 kB at 191 kB/s) 2024-04-22T20:00:50.820 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom 2024-04-22T20:00:50.838 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom (3.0 kB at 168 kB/s) 2024-04-22T20:00:50.856 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar 2024-04-22T20:00:51.137 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3 MB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar (3.3 MB at 12 MB/s) 2024-04-22T20:00:51.139 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar 2024-04-22T20:00:51.139 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar 2024-04-22T20:00:51.139 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar 2024-04-22T20:00:51.139 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar 2024-04-22T20:00:51.139 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar 2024-04-22T20:00:51.163 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/98 kB Progress (1): 33/98 kB Progress (1): 49/98 kB Progress (1): 66/98 kB Progress (1): 82/98 kB Progress (1): 98 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar (98 kB at 3.6 MB/s) 2024-04-22T20:00:51.164 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar 2024-04-22T20:00:51.182 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar (16 kB at 382 kB/s) 2024-04-22T20:00:51.182 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar 2024-04-22T20:00:51.201 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/16 kB Progress (2): 1.4/16 kB | 1.4/35 kB Progress (2): 2.8/16 kB | 1.4/35 kB Progress (3): 2.8/16 kB | 1.4/35 kB | 1.4/49 kB Progress (3): 4.1/16 kB | 1.4/35 kB | 1.4/49 kB Progress (3): 4.1/16 kB | 2.8/35 kB | 1.4/49 kB Progress (3): 4.1/16 kB | 2.8/35 kB | 2.8/49 kB Progress (3): 5.5/16 kB | 2.8/35 kB | 2.8/49 kB Progress (3): 5.5/16 kB | 4.1/35 kB | 2.8/49 kB Progress (3): 6.9/16 kB | 4.1/35 kB | 2.8/49 kB Progress (3): 6.9/16 kB | 5.5/35 kB | 2.8/49 kB Progress (3): 8.3/16 kB | 5.5/35 kB | 2.8/49 kB Progress (3): 8.3/16 kB | 5.5/35 kB | 4.1/49 kB Progress (3): 9.7/16 kB | 5.5/35 kB | 4.1/49 kB Progress (3): 9.7/16 kB | 5.5/35 kB | 5.5/49 kB Progress (3): 9.7/16 kB | 6.9/35 kB | 5.5/49 kB Progress (3): 11/16 kB | 6.9/35 kB | 5.5/49 kB Progress (3): 11/16 kB | 8.3/35 kB | 5.5/49 kB Progress (3): 12/16 kB | 8.3/35 kB | 5.5/49 kB Progress (3): 12/16 kB | 9.7/35 kB | 5.5/49 kB Progress (3): 12/16 kB | 9.7/35 kB | 6.9/49 kB Progress (3): 12/16 kB | 11/35 kB | 6.9/49 kB Progress (3): 14/16 kB | 11/35 kB | 6.9/49 kB Progress (3): 14/16 kB | 11/35 kB | 8.3/49 kB Progress (3): 14/16 kB | 12/35 kB | 8.3/49 kB Progress (3): 15/16 kB | 12/35 kB | 8.3/49 kB Progress (3): 15/16 kB | 14/35 kB | 8.3/49 kB Progress (3): 15/16 kB | 14/35 kB | 9.7/49 kB Progress (3): 15/16 kB | 15/35 kB | 9.7/49 kB Progress (3): 16 kB | 15/35 kB | 9.7/49 kB Progress (3): 16 kB | 15/35 kB | 11/49 kB Progress (3): 16 kB | 15/35 kB | 12/49 kB Progress (3): 16 kB | 15/35 kB | 14/49 kB Progress (3): 16 kB | 15/35 kB | 15/49 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar (16 kB at 268 kB/s) 2024-04-22T20:00:51.201 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar 2024-04-22T20:00:51.213 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 15/35 kB | 17/49 kB Progress (2): 17/35 kB | 17/49 kB Progress (3): 17/35 kB | 17/49 kB | 16/71 kB Progress (3): 17/35 kB | 18/49 kB | 16/71 kB Progress (3): 18/35 kB | 18/49 kB | 16/71 kB Progress (3): 18/35 kB | 19/49 kB | 16/71 kB Progress (3): 19/35 kB | 19/49 kB | 16/71 kB Progress (3): 19/35 kB | 21/49 kB | 16/71 kB Progress (3): 21/35 kB | 21/49 kB | 16/71 kB Progress (3): 21/35 kB | 22/49 kB | 16/71 kB Progress (3): 22/35 kB | 22/49 kB | 16/71 kB Progress (3): 22/35 kB | 23/49 kB | 16/71 kB Progress (3): 23/35 kB | 23/49 kB | 16/71 kB Progress (3): 25/35 kB | 23/49 kB | 16/71 kB Progress (3): 25/35 kB | 25/49 kB | 16/71 kB Progress (4): 25/35 kB | 25/49 kB | 16/71 kB | 16/782 kB Progress (4): 25/35 kB | 26/49 kB | 16/71 kB | 16/782 kB Progress (4): 26/35 kB | 26/49 kB | 16/71 kB | 16/782 kB Progress (4): 26/35 kB | 28/49 kB | 16/71 kB | 16/782 kB Progress (4): 28/35 kB | 28/49 kB | 16/71 kB | 16/782 kB Progress (4): 28/35 kB | 28/49 kB | 16/71 kB | 32/782 kB Progress (4): 28/35 kB | 29/49 kB | 16/71 kB | 32/782 kB Progress (4): 29/35 kB | 29/49 kB | 16/71 kB | 32/782 kB Progress (4): 29/35 kB | 30/49 kB | 16/71 kB | 32/782 kB Progress (4): 29/35 kB | 30/49 kB | 16/71 kB | 49/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 49/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 65/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 81/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 98/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 114/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 130/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 147/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 163/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 180/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 196/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 212/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 229/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 245/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 262/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 278/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 294/782 kB Progress (4): 30/35 kB | 30/49 kB | 16/71 kB | 311/782 kB Progress (4): 30/35 kB | 32/49 kB | 16/71 kB | 311/782 kB Progress (4): 32/35 kB | 32/49 kB | 16/71 kB | 311/782 kB Progress (4): 32/35 kB | 33/49 kB | 16/71 kB | 311/782 kB Progress (4): 32/35 kB | 33/49 kB | 16/71 kB | 327/782 kB Progress (4): 33/35 kB | 33/49 kB | 16/71 kB | 327/782 kB Progress (4): 33/35 kB | 34/49 kB | 16/71 kB | 327/782 kB Progress (4): 34/35 kB | 34/49 kB | 16/71 kB | 327/782 kB Progress (4): 34/35 kB | 36/49 kB | 16/71 kB | 327/782 kB Progress (4): 35 kB | 36/49 kB | 16/71 kB | 327/782 kB Progress (4): 35 kB | 37/49 kB | 16/71 kB | 327/782 kB Progress (4): 35 kB | 37/49 kB | 33/71 kB | 327/782 kB Progress (4): 35 kB | 39/49 kB | 33/71 kB | 327/782 kB Progress (4): 35 kB | 39/49 kB | 33/71 kB | 343/782 kB Progress (4): 35 kB | 40/49 kB | 33/71 kB | 343/782 kB Progress (4): 35 kB | 41/49 kB | 33/71 kB | 343/782 kB Progress (4): 35 kB | 43/49 kB | 33/71 kB | 343/782 kB Progress (4): 35 kB | 43/49 kB | 33/71 kB | 360/782 kB Progress (4): 35 kB | 44/49 kB | 33/71 kB | 360/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar (35 kB at 475 kB/s) 2024-04-22T20:00:51.213 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 46/49 kB | 33/71 kB | 360/782 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar 2024-04-22T20:00:51.215 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 46/49 kB | 33/71 kB | 376/782 kB Progress (3): 47/49 kB | 33/71 kB | 376/782 kB Progress (3): 48/49 kB | 33/71 kB | 376/782 kB Progress (3): 49 kB | 33/71 kB | 376/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar (49 kB at 648 kB/s) 2024-04-22T20:00:51.215 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar 2024-04-22T20:00:51.217 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 33/71 kB | 376/782 kB | 1.4/11 kB Progress (3): 33/71 kB | 376/782 kB | 2.8/11 kB Progress (3): 49/71 kB | 376/782 kB | 2.8/11 kB Progress (3): 49/71 kB | 376/782 kB | 4.1/11 kB Progress (3): 49/71 kB | 376/782 kB | 5.5/11 kB Progress (3): 49/71 kB | 376/782 kB | 6.9/11 kB Progress (3): 49/71 kB | 393/782 kB | 6.9/11 kB Progress (3): 49/71 kB | 393/782 kB | 8.3/11 kB Progress (3): 49/71 kB | 393/782 kB | 9.7/11 kB Progress (3): 49/71 kB | 402/782 kB | 9.7/11 kB Progress (3): 49/71 kB | 402/782 kB | 11/11 kB Progress (3): 49/71 kB | 402/782 kB | 11 kB Progress (3): 49/71 kB | 418/782 kB | 11 kB Progress (3): 49/71 kB | 435/782 kB | 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar (11 kB at 146 kB/s) 2024-04-22T20:00:51.217 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar 2024-04-22T20:00:51.228 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 49/71 kB | 451/782 kB Progress (2): 49/71 kB | 468/782 kB Progress (2): 49/71 kB | 484/782 kB Progress (2): 49/71 kB | 500/782 kB Progress (2): 49/71 kB | 517/782 kB Progress (2): 49/71 kB | 533/782 kB Progress (2): 49/71 kB | 549/782 kB Progress (2): 49/71 kB | 566/782 kB Progress (2): 49/71 kB | 582/782 kB Progress (2): 49/71 kB | 599/782 kB Progress (2): 49/71 kB | 615/782 kB Progress (2): 49/71 kB | 631/782 kB Progress (2): 49/71 kB | 648/782 kB Progress (2): 49/71 kB | 664/782 kB Progress (2): 49/71 kB | 681/782 kB Progress (2): 49/71 kB | 697/782 kB Progress (2): 66/71 kB | 697/782 kB Progress (2): 66/71 kB | 713/782 kB Progress (2): 71 kB | 713/782 kB Progress (2): 71 kB | 730/782 kB Progress (2): 71 kB | 746/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar (71 kB at 784 kB/s) 2024-04-22T20:00:51.228 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar 2024-04-22T20:00:51.232 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 762/782 kB Progress (1): 779/782 kB Progress (1): 782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar (782 kB at 8.6 MB/s) 2024-04-22T20:00:51.232 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar 2024-04-22T20:00:51.235 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/205 kB Progress (1): 2.8/205 kB Progress (1): 4.1/205 kB Progress (1): 5.5/205 kB Progress (1): 6.9/205 kB Progress (1): 8.3/205 kB Progress (1): 9.7/205 kB Progress (1): 11/205 kB Progress (1): 12/205 kB Progress (1): 14/205 kB Progress (1): 15/205 kB Progress (1): 16/205 kB Progress (2): 16/205 kB | 16/30 kB Progress (2): 16/205 kB | 30 kB Progress (2): 32/205 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar (30 kB at 318 kB/s) 2024-04-22T20:00:51.235 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar 2024-04-22T20:00:51.250 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 32/205 kB | 16/81 kB Progress (3): 32/205 kB | 16/81 kB | 16/798 kB Progress (3): 49/205 kB | 16/81 kB | 16/798 kB Progress (3): 49/205 kB | 16/81 kB | 33/798 kB Progress (3): 65/205 kB | 16/81 kB | 33/798 kB Progress (4): 65/205 kB | 16/81 kB | 33/798 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar (13 kB at 123 kB/s) 2024-04-22T20:00:51.250 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar 2024-04-22T20:00:51.257 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 65/205 kB | 33/81 kB | 33/798 kB Progress (3): 65/205 kB | 33/81 kB | 49/798 kB Progress (3): 81/205 kB | 33/81 kB | 49/798 kB Progress (3): 81/205 kB | 33/81 kB | 66/798 kB Progress (3): 98/205 kB | 33/81 kB | 66/798 kB Progress (3): 98/205 kB | 49/81 kB | 66/798 kB Progress (3): 114/205 kB | 49/81 kB | 66/798 kB Progress (4): 114/205 kB | 49/81 kB | 66/798 kB | 16/182 kB Progress (4): 130/205 kB | 49/81 kB | 66/798 kB | 16/182 kB Progress (4): 130/205 kB | 66/81 kB | 66/798 kB | 16/182 kB Progress (4): 130/205 kB | 66/81 kB | 66/798 kB | 33/182 kB Progress (4): 130/205 kB | 81 kB | 66/798 kB | 33/182 kB Progress (4): 130/205 kB | 81 kB | 66/798 kB | 49/182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar (81 kB at 695 kB/s) 2024-04-22T20:00:51.257 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar 2024-04-22T20:00:51.266 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 130/205 kB | 66/798 kB | 66/182 kB Progress (3): 130/205 kB | 82/798 kB | 66/182 kB Progress (3): 130/205 kB | 98/798 kB | 66/182 kB Progress (3): 147/205 kB | 98/798 kB | 66/182 kB Progress (3): 163/205 kB | 98/798 kB | 66/182 kB Progress (3): 180/205 kB | 98/798 kB | 66/182 kB Progress (3): 180/205 kB | 115/798 kB | 66/182 kB Progress (3): 196/205 kB | 115/798 kB | 66/182 kB Progress (3): 205 kB | 115/798 kB | 66/182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar (205 kB at 1.6 MB/s) 2024-04-22T20:00:51.266 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar 2024-04-22T20:00:51.272 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 131/798 kB | 66/182 kB Progress (2): 147/798 kB | 66/182 kB Progress (2): 164/798 kB | 66/182 kB Progress (3): 164/798 kB | 66/182 kB | 16/151 kB Progress (3): 164/798 kB | 82/182 kB | 16/151 kB Progress (3): 180/798 kB | 82/182 kB | 16/151 kB Progress (3): 180/798 kB | 82/182 kB | 32/151 kB Progress (3): 180/798 kB | 82/182 kB | 49/151 kB Progress (3): 197/798 kB | 82/182 kB | 49/151 kB Progress (3): 197/798 kB | 82/182 kB | 65/151 kB Progress (3): 197/798 kB | 82/182 kB | 81/151 kB Progress (3): 213/798 kB | 82/182 kB | 81/151 kB Progress (3): 213/798 kB | 82/182 kB | 98/151 kB Progress (3): 213/798 kB | 98/182 kB | 98/151 kB Progress (3): 213/798 kB | 98/182 kB | 114/151 kB Progress (3): 213/798 kB | 98/182 kB | 130/151 kB Progress (3): 213/798 kB | 115/182 kB | 130/151 kB Progress (3): 213/798 kB | 115/182 kB | 147/151 kB Progress (3): 213/798 kB | 115/182 kB | 151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar (151 kB at 1.2 MB/s) 2024-04-22T20:00:51.272 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar 2024-04-22T20:00:51.276 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 213/798 kB | 131/182 kB Progress (2): 213/798 kB | 147/182 kB Progress (2): 213/798 kB | 164/182 kB Progress (3): 213/798 kB | 164/182 kB | 16/27 kB Progress (3): 213/798 kB | 164/182 kB | 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar (27 kB at 202 kB/s) 2024-04-22T20:00:51.276 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 213/798 kB | 180/182 kB Progress (2): 229/798 kB | 180/182 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar 2024-04-22T20:00:51.276 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 229/798 kB | 182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar (182 kB at 1.3 MB/s) 2024-04-22T20:00:51.277 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar 2024-04-22T20:00:51.282 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 246/798 kB Progress (1): 256/798 kB Progress (1): 272/798 kB Progress (1): 289/798 kB Progress (1): 305/798 kB Progress (1): 321/798 kB Progress (1): 338/798 kB Progress (1): 354/798 kB Progress (2): 354/798 kB | 16/30 kB Progress (2): 370/798 kB | 16/30 kB Progress (2): 370/798 kB | 30 kB Progress (2): 387/798 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar (30 kB at 212 kB/s) 2024-04-22T20:00:51.283 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar 2024-04-22T20:00:51.294 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 403/798 kB Progress (1): 420/798 kB Progress (1): 436/798 kB Progress (1): 452/798 kB Progress (1): 469/798 kB Progress (1): 485/798 kB Progress (1): 502/798 kB Progress (2): 502/798 kB | 16/385 kB Progress (2): 518/798 kB | 16/385 kB Progress (2): 518/798 kB | 33/385 kB Progress (2): 518/798 kB | 49/385 kB Progress (2): 534/798 kB | 49/385 kB Progress (2): 534/798 kB | 66/385 kB Progress (2): 551/798 kB | 66/385 kB Progress (2): 551/798 kB | 82/385 kB Progress (2): 551/798 kB | 98/385 kB Progress (2): 567/798 kB | 98/385 kB Progress (3): 567/798 kB | 98/385 kB | 13 kB Progress (3): 567/798 kB | 115/385 kB | 13 kB Progress (3): 583/798 kB | 115/385 kB | 13 kB Progress (3): 600/798 kB | 115/385 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar (13 kB at 86 kB/s) 2024-04-22T20:00:51.294 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar 2024-04-22T20:00:51.300 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 616/798 kB | 115/385 kB Progress (2): 616/798 kB | 131/385 kB Progress (2): 616/798 kB | 147/385 kB Progress (2): 633/798 kB | 147/385 kB Progress (2): 649/798 kB | 147/385 kB Progress (2): 665/798 kB | 147/385 kB Progress (2): 665/798 kB | 164/385 kB Progress (3): 665/798 kB | 164/385 kB | 16/75 kB Progress (3): 665/798 kB | 180/385 kB | 16/75 kB Progress (3): 682/798 kB | 180/385 kB | 16/75 kB Progress (3): 682/798 kB | 180/385 kB | 33/75 kB Progress (3): 682/798 kB | 197/385 kB | 33/75 kB Progress (3): 698/798 kB | 197/385 kB | 33/75 kB Progress (3): 698/798 kB | 213/385 kB | 33/75 kB Progress (3): 715/798 kB | 213/385 kB | 33/75 kB Progress (3): 731/798 kB | 213/385 kB | 33/75 kB Progress (3): 747/798 kB | 213/385 kB | 33/75 kB Progress (3): 747/798 kB | 213/385 kB | 49/75 kB Progress (3): 747/798 kB | 229/385 kB | 49/75 kB Progress (3): 764/798 kB | 229/385 kB | 49/75 kB Progress (3): 764/798 kB | 246/385 kB | 49/75 kB Progress (3): 780/798 kB | 246/385 kB | 49/75 kB Progress (3): 780/798 kB | 246/385 kB | 66/75 kB Progress (3): 796/798 kB | 246/385 kB | 66/75 kB Progress (3): 798 kB | 246/385 kB | 66/75 kB Progress (3): 798 kB | 262/385 kB | 66/75 kB Progress (3): 798 kB | 262/385 kB | 75 kB Progress (3): 798 kB | 279/385 kB | 75 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar (798 kB at 5.0 MB/s) 2024-04-22T20:00:51.300 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar 2024-04-22T20:00:51.300 INFO:tasks.workunit.client.0.smithi136.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar (75 kB at 469 kB/s) 2024-04-22T20:00:51.300 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar 2024-04-22T20:00:51.304 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 295/385 kB Progress (1): 311/385 kB Progress (1): 328/385 kB Progress (1): 344/385 kB Progress (1): 360/385 kB Progress (1): 377/385 kB Progress (2): 377/385 kB | 16/780 kB Progress (2): 385 kB | 16/780 kB Progress (2): 385 kB | 33/780 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar (385 kB at 2.4 MB/s) 2024-04-22T20:00:51.304 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 49/780 kB Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar 2024-04-22T20:00:51.325 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 66/780 kB Progress (1): 82/780 kB Progress (1): 98/780 kB Progress (1): 115/780 kB Progress (1): 131/780 kB Progress (1): 147/780 kB Progress (1): 164/780 kB Progress (1): 180/780 kB Progress (1): 197/780 kB Progress (1): 213/780 kB Progress (1): 229/780 kB Progress (1): 246/780 kB Progress (1): 262/780 kB Progress (2): 262/780 kB | 16/329 kB Progress (2): 262/780 kB | 33/329 kB Progress (2): 262/780 kB | 49/329 kB Progress (2): 262/780 kB | 66/329 kB Progress (2): 262/780 kB | 82/329 kB Progress (2): 262/780 kB | 98/329 kB Progress (2): 262/780 kB | 115/329 kB Progress (2): 262/780 kB | 131/329 kB Progress (2): 262/780 kB | 147/329 kB Progress (2): 262/780 kB | 164/329 kB Progress (2): 262/780 kB | 180/329 kB Progress (3): 262/780 kB | 180/329 kB | 16/501 kB Progress (3): 262/780 kB | 197/329 kB | 16/501 kB Progress (3): 262/780 kB | 197/329 kB | 33/501 kB Progress (3): 262/780 kB | 213/329 kB | 33/501 kB Progress (3): 279/780 kB | 213/329 kB | 33/501 kB Progress (4): 279/780 kB | 213/329 kB | 33/501 kB | 16/354 kB Progress (4): 279/780 kB | 213/329 kB | 33/501 kB | 33/354 kB Progress (4): 279/780 kB | 229/329 kB | 33/501 kB | 33/354 kB Progress (4): 279/780 kB | 229/329 kB | 33/501 kB | 49/354 kB Progress (4): 279/780 kB | 246/329 kB | 33/501 kB | 49/354 kB Progress (4): 295/780 kB | 246/329 kB | 33/501 kB | 49/354 kB Progress (4): 295/780 kB | 246/329 kB | 33/501 kB | 66/354 kB Progress (4): 311/780 kB | 246/329 kB | 33/501 kB | 66/354 kB Progress (4): 311/780 kB | 246/329 kB | 33/501 kB | 82/354 kB Progress (4): 311/780 kB | 246/329 kB | 33/501 kB | 98/354 kB Progress (4): 328/780 kB | 246/329 kB | 33/501 kB | 98/354 kB Progress (4): 328/780 kB | 246/329 kB | 49/501 kB | 98/354 kB Progress (4): 328/780 kB | 246/329 kB | 49/501 kB | 115/354 kB Progress (4): 328/780 kB | 246/329 kB | 49/501 kB | 131/354 kB Progress (4): 328/780 kB | 246/329 kB | 49/501 kB | 147/354 kB Progress (4): 344/780 kB | 246/329 kB | 49/501 kB | 147/354 kB Progress (4): 344/780 kB | 246/329 kB | 49/501 kB | 164/354 kB Progress (4): 360/780 kB | 246/329 kB | 49/501 kB | 164/354 kB Progress (4): 360/780 kB | 246/329 kB | 64/501 kB | 164/354 kB Progress (4): 360/780 kB | 246/329 kB | 64/501 kB | 180/354 kB Progress (4): 360/780 kB | 246/329 kB | 64/501 kB | 197/354 kB Progress (4): 360/780 kB | 246/329 kB | 64/501 kB | 213/354 kB Progress (4): 360/780 kB | 246/329 kB | 64/501 kB | 229/354 kB Progress (4): 360/780 kB | 246/329 kB | 64/501 kB | 246/354 kB Progress (4): 377/780 kB | 246/329 kB | 64/501 kB | 246/354 kB Progress (4): 377/780 kB | 246/329 kB | 64/501 kB | 262/354 kB Progress (4): 393/780 kB | 246/329 kB | 64/501 kB | 262/354 kB Progress (4): 393/780 kB | 246/329 kB | 64/501 kB | 279/354 kB Progress (4): 393/780 kB | 246/329 kB | 64/501 kB | 295/354 kB Progress (4): 393/780 kB | 246/329 kB | 81/501 kB | 295/354 kB Progress (4): 393/780 kB | 246/329 kB | 81/501 kB | 311/354 kB Progress (4): 393/780 kB | 246/329 kB | 81/501 kB | 328/354 kB Progress (4): 410/780 kB | 246/329 kB | 81/501 kB | 328/354 kB Progress (4): 410/780 kB | 246/329 kB | 81/501 kB | 344/354 kB Progress (4): 410/780 kB | 246/329 kB | 97/501 kB | 344/354 kB Progress (4): 426/780 kB | 246/329 kB | 97/501 kB | 344/354 kB Progress (4): 426/780 kB | 246/329 kB | 97/501 kB | 354 kB Progress (4): 442/780 kB | 246/329 kB | 97/501 kB | 354 kB Progress (5): 442/780 kB | 246/329 kB | 97/501 kB | 354 kB | 16/41 kB Progress (5): 459/780 kB | 246/329 kB | 97/501 kB | 354 kB | 16/41 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar (354 kB at 1.9 MB/s) 2024-04-22T20:00:51.325 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar 2024-04-22T20:00:51.326 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (4): 459/780 kB | 246/329 kB | 97/501 kB | 33/41 kB Progress (4): 459/780 kB | 246/329 kB | 114/501 kB | 33/41 kB Progress (4): 459/780 kB | 246/329 kB | 114/501 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar (41 kB at 221 kB/s) 2024-04-22T20:00:51.327 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 475/780 kB | 246/329 kB | 114/501 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar 2024-04-22T20:00:51.333 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 492/780 kB | 246/329 kB | 114/501 kB Progress (3): 508/780 kB | 246/329 kB | 114/501 kB Progress (3): 508/780 kB | 246/329 kB | 130/501 kB Progress (3): 524/780 kB | 246/329 kB | 130/501 kB Progress (3): 541/780 kB | 246/329 kB | 130/501 kB Progress (3): 541/780 kB | 246/329 kB | 146/501 kB Progress (3): 557/780 kB | 246/329 kB | 146/501 kB Progress (3): 573/780 kB | 246/329 kB | 146/501 kB Progress (3): 585/780 kB | 246/329 kB | 146/501 kB Progress (3): 601/780 kB | 246/329 kB | 146/501 kB Progress (3): 617/780 kB | 246/329 kB | 146/501 kB Progress (3): 634/780 kB | 246/329 kB | 146/501 kB Progress (3): 650/780 kB | 246/329 kB | 146/501 kB Progress (3): 667/780 kB | 246/329 kB | 146/501 kB Progress (3): 683/780 kB | 246/329 kB | 146/501 kB Progress (3): 699/780 kB | 246/329 kB | 146/501 kB Progress (3): 716/780 kB | 246/329 kB | 146/501 kB Progress (3): 732/780 kB | 246/329 kB | 146/501 kB Progress (3): 732/780 kB | 262/329 kB | 146/501 kB Progress (3): 749/780 kB | 262/329 kB | 146/501 kB Progress (3): 749/780 kB | 279/329 kB | 146/501 kB Progress (3): 765/780 kB | 279/329 kB | 146/501 kB Progress (3): 780 kB | 279/329 kB | 146/501 kB Progress (3): 780 kB | 295/329 kB | 146/501 kB Progress (3): 780 kB | 311/329 kB | 146/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar (780 kB at 4.0 MB/s) 2024-04-22T20:00:51.333 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar 2024-04-22T20:00:51.334 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 328/329 kB | 146/501 kB Progress (2): 329 kB | 146/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar (329 kB at 1.7 MB/s) 2024-04-22T20:00:51.334 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar 2024-04-22T20:00:51.343 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 163/501 kB Progress (1): 179/501 kB Progress (1): 196/501 kB Progress (1): 212/501 kB Progress (1): 228/501 kB Progress (1): 245/501 kB Progress (1): 261/501 kB Progress (1): 277/501 kB Progress (1): 294/501 kB Progress (1): 310/501 kB Progress (1): 327/501 kB Progress (1): 343/501 kB Progress (1): 359/501 kB Progress (1): 376/501 kB Progress (1): 392/501 kB Progress (2): 392/501 kB | 6.4 kB Progress (2): 409/501 kB | 6.4 kB Progress (2): 425/501 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar (6.4 kB at 31 kB/s) 2024-04-22T20:00:51.343 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar 2024-04-22T20:00:51.351 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 441/501 kB Progress (1): 458/501 kB Progress (2): 458/501 kB | 16/586 kB Progress (2): 458/501 kB | 32/586 kB Progress (2): 458/501 kB | 49/586 kB Progress (2): 458/501 kB | 65/586 kB Progress (2): 458/501 kB | 81/586 kB Progress (2): 458/501 kB | 98/586 kB Progress (2): 458/501 kB | 114/586 kB Progress (2): 458/501 kB | 130/586 kB Progress (2): 458/501 kB | 147/586 kB Progress (2): 458/501 kB | 163/586 kB Progress (2): 458/501 kB | 180/586 kB Progress (2): 474/501 kB | 180/586 kB Progress (2): 490/501 kB | 180/586 kB Progress (2): 501 kB | 180/586 kB Progress (3): 501 kB | 180/586 kB | 16/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar (501 kB at 2.4 MB/s) 2024-04-22T20:00:51.351 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 196/586 kB | 16/211 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar 2024-04-22T20:00:51.352 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 212/586 kB | 16/211 kB Progress (2): 212/586 kB | 33/211 kB Progress (3): 212/586 kB | 33/211 kB | 14 kB Progress (3): 212/586 kB | 49/211 kB | 14 kB Progress (3): 229/586 kB | 49/211 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar (14 kB at 68 kB/s) 2024-04-22T20:00:51.352 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar 2024-04-22T20:00:51.372 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 245/586 kB | 49/211 kB Progress (2): 245/586 kB | 66/211 kB Progress (2): 262/586 kB | 66/211 kB Progress (2): 262/586 kB | 82/211 kB Progress (2): 278/586 kB | 82/211 kB Progress (2): 278/586 kB | 98/211 kB Progress (2): 294/586 kB | 98/211 kB Progress (2): 294/586 kB | 115/211 kB Progress (2): 311/586 kB | 115/211 kB Progress (2): 311/586 kB | 131/211 kB Progress (2): 327/586 kB | 131/211 kB Progress (2): 327/586 kB | 147/211 kB Progress (2): 343/586 kB | 147/211 kB Progress (2): 343/586 kB | 164/211 kB Progress (2): 360/586 kB | 164/211 kB Progress (2): 376/586 kB | 164/211 kB Progress (2): 376/586 kB | 180/211 kB Progress (2): 376/586 kB | 197/211 kB Progress (2): 393/586 kB | 197/211 kB Progress (2): 393/586 kB | 211 kB Progress (2): 402/586 kB | 211 kB Progress (2): 418/586 kB | 211 kB Progress (3): 418/586 kB | 211 kB | 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar (211 kB at 921 kB/s) 2024-04-22T20:00:51.372 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 418/586 kB | 6.8 kB | 16/245 kB Progress (4): 418/586 kB | 6.8 kB | 16/245 kB | 16/106 kB Progress (4): 435/586 kB | 6.8 kB | 16/245 kB | 16/106 kB Progress (4): 435/586 kB | 6.8 kB | 32/245 kB | 16/106 kB Progress (4): 435/586 kB | 6.8 kB | 32/245 kB | 32/106 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar 2024-04-22T20:00:51.372 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (4): 435/586 kB | 6.8 kB | 49/245 kB | 32/106 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar (6.8 kB at 30 kB/s) 2024-04-22T20:00:51.372 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 451/586 kB | 49/245 kB | 32/106 kB Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar 2024-04-22T20:00:51.375 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 451/586 kB | 65/245 kB | 32/106 kB Progress (3): 468/586 kB | 65/245 kB | 32/106 kB Progress (3): 468/586 kB | 65/245 kB | 49/106 kB Progress (3): 468/586 kB | 81/245 kB | 49/106 kB Progress (3): 484/586 kB | 81/245 kB | 49/106 kB Progress (3): 484/586 kB | 98/245 kB | 49/106 kB Progress (3): 484/586 kB | 98/245 kB | 65/106 kB Progress (3): 500/586 kB | 98/245 kB | 65/106 kB Progress (3): 500/586 kB | 114/245 kB | 65/106 kB Progress (3): 500/586 kB | 114/245 kB | 81/106 kB Progress (3): 517/586 kB | 114/245 kB | 81/106 kB Progress (3): 517/586 kB | 130/245 kB | 81/106 kB Progress (3): 517/586 kB | 130/245 kB | 98/106 kB Progress (3): 533/586 kB | 130/245 kB | 98/106 kB Progress (3): 533/586 kB | 130/245 kB | 106 kB Progress (3): 533/586 kB | 147/245 kB | 106 kB Progress (3): 549/586 kB | 147/245 kB | 106 kB Progress (3): 549/586 kB | 163/245 kB | 106 kB Progress (3): 566/586 kB | 163/245 kB | 106 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar (106 kB at 458 kB/s) 2024-04-22T20:00:51.375 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 566/586 kB | 180/245 kB Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar 2024-04-22T20:00:51.375 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 582/586 kB | 180/245 kB Progress (2): 586 kB | 180/245 kB Progress (2): 586 kB | 196/245 kB Progress (2): 586 kB | 212/245 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar (586 kB at 2.5 MB/s) 2024-04-22T20:00:51.375 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar 2024-04-22T20:00:51.375 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 229/245 kB Progress (1): 245 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar (245 kB at 1.0 MB/s) 2024-04-22T20:00:51.376 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar 2024-04-22T20:00:51.390 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/205 kB Progress (2): 16/205 kB | 15 kB Progress (2): 33/205 kB | 15 kB Progress (2): 49/205 kB | 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar (15 kB at 61 kB/s) 2024-04-22T20:00:51.390 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar 2024-04-22T20:00:51.392 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 66/205 kB Progress (1): 82/205 kB Progress (1): 98/205 kB Progress (1): 115/205 kB Progress (2): 115/205 kB | 16/16 kB Progress (2): 115/205 kB | 16 kB Progress (2): 131/205 kB | 16 kB Progress (2): 147/205 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar (16 kB at 66 kB/s) 2024-04-22T20:00:51.393 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 147/205 kB | 6.4 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar 2024-04-22T20:00:51.393 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 164/205 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar (6.4 kB at 25 kB/s) 2024-04-22T20:00:51.393 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 180/205 kB Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar 2024-04-22T20:00:51.393 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 180/205 kB | 16/23 kB Progress (2): 197/205 kB | 16/23 kB Progress (2): 197/205 kB | 23 kB Progress (2): 205 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar (23 kB at 90 kB/s) 2024-04-22T20:00:51.394 INFO:tasks.workunit.client.0.smithi136.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar (205 kB at 810 kB/s) 2024-04-22T20:00:51.408 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar (24 kB at 90 kB/s) 2024-04-22T20:00:51.410 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar (17 kB at 65 kB/s) 2024-04-22T20:00:51.416 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/184 kB Progress (1): 33/184 kB Progress (1): 49/184 kB Progress (1): 66/184 kB Progress (1): 82/184 kB Progress (1): 98/184 kB Progress (1): 115/184 kB Progress (1): 131/184 kB Progress (1): 147/184 kB Progress (1): 164/184 kB Progress (1): 180/184 kB Progress (1): 184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar (184 kB at 671 kB/s) 2024-04-22T20:00:51.438 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:51.438 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- clean:3.2.0:clean (default-clean) @ jcksum --- 2024-04-22T20:00:51.445 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom 2024-04-22T20:00:51.472 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom (5.8 kB at 208 kB/s) 2024-04-22T20:00:51.475 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom 2024-04-22T20:00:51.520 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom (5.1 kB at 113 kB/s) 2024-04-22T20:00:51.522 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom 2024-04-22T20:00:51.569 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/43 kB Progress (1): 33/43 kB Progress (1): 43 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 911 kB/s) 2024-04-22T20:00:51.574 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom 2024-04-22T20:00:51.611 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom (14 kB at 385 kB/s) 2024-04-22T20:00:51.616 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar 2024-04-22T20:00:51.647 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/153 kB Progress (1): 33/153 kB Progress (1): 49/153 kB Progress (1): 66/153 kB Progress (1): 82/153 kB Progress (1): 98/153 kB Progress (1): 115/153 kB Progress (1): 131/153 kB Progress (1): 147/153 kB Progress (1): 153 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar (153 kB at 4.8 MB/s) 2024-04-22T20:00:51.648 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar 2024-04-22T20:00:51.676 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/215 kB Progress (1): 32/215 kB Progress (1): 49/215 kB Progress (1): 65/215 kB Progress (1): 81/215 kB Progress (1): 98/215 kB Progress (1): 114/215 kB Progress (1): 130/215 kB Progress (1): 147/215 kB Progress (1): 163/215 kB Progress (1): 180/215 kB Progress (1): 196/215 kB Progress (1): 212/215 kB Progress (1): 215 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar (215 kB at 7.7 MB/s) 2024-04-22T20:00:51.719 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:51.719 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-04-22T20:00:51.723 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom 2024-04-22T20:00:51.744 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom (2.7 kB at 121 kB/s) 2024-04-22T20:00:51.746 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom 2024-04-22T20:00:51.769 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom (23 kB at 1.0 MB/s) 2024-04-22T20:00:51.772 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom 2024-04-22T20:00:51.795 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom (8.8 kB at 365 kB/s) 2024-04-22T20:00:51.798 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom 2024-04-22T20:00:51.822 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom (25 kB at 1.0 MB/s) 2024-04-22T20:00:51.826 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom 2024-04-22T20:00:51.850 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom (6.0 kB at 252 kB/s) 2024-04-22T20:00:51.851 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom 2024-04-22T20:00:51.877 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom (3.2 kB at 124 kB/s) 2024-04-22T20:00:51.881 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom 2024-04-22T20:00:51.908 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 612 B Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom (612 B at 22 kB/s) 2024-04-22T20:00:51.910 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom 2024-04-22T20:00:51.939 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom (2.7 kB at 91 kB/s) 2024-04-22T20:00:51.942 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom 2024-04-22T20:00:51.969 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom (14 kB at 522 kB/s) 2024-04-22T20:00:51.971 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom 2024-04-22T20:00:51.995 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom (3.2 kB at 128 kB/s) 2024-04-22T20:00:51.997 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom 2024-04-22T20:00:52.024 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom (8.4 kB at 310 kB/s) 2024-04-22T20:00:52.026 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom 2024-04-22T20:00:52.047 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom (8.4 kB at 398 kB/s) 2024-04-22T20:00:52.049 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom 2024-04-22T20:00:52.073 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom (8.0 kB at 334 kB/s) 2024-04-22T20:00:52.077 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom 2024-04-22T20:00:52.102 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom (20 kB at 759 kB/s) 2024-04-22T20:00:52.108 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom 2024-04-22T20:00:52.136 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom (5.1 kB at 182 kB/s) 2024-04-22T20:00:52.139 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom 2024-04-22T20:00:52.168 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom (31 kB at 1.1 MB/s) 2024-04-22T20:00:52.173 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom 2024-04-22T20:00:52.203 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom (5.1 kB at 176 kB/s) 2024-04-22T20:00:52.207 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar 2024-04-22T20:00:52.239 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/85 kB Progress (1): 33/85 kB Progress (1): 49/85 kB Progress (1): 66/85 kB Progress (1): 82/85 kB Progress (1): 85 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar (85 kB at 2.7 MB/s) 2024-04-22T20:00:52.240 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar 2024-04-22T20:00:52.240 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar 2024-04-22T20:00:52.240 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar 2024-04-22T20:00:52.240 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar 2024-04-22T20:00:52.240 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar 2024-04-22T20:00:52.273 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.5 kB Progress (2): 2.5 kB | 16/269 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar (2.5 kB at 76 kB/s) 2024-04-22T20:00:52.273 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar 2024-04-22T20:00:52.274 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 33/269 kB Progress (1): 49/269 kB Progress (2): 49/269 kB | 8.5 kB Progress (3): 49/269 kB | 8.5 kB | 16/41 kB Progress (3): 66/269 kB | 8.5 kB | 16/41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar (8.5 kB at 249 kB/s) 2024-04-22T20:00:52.274 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar 2024-04-22T20:00:52.275 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 66/269 kB | 33/41 kB Progress (2): 82/269 kB | 33/41 kB Progress (2): 82/269 kB | 41 kB Progress (2): 98/269 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar (41 kB at 1.2 MB/s) 2024-04-22T20:00:52.277 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 115/269 kB Progress (2): 115/269 kB | 16/55 kB Progress (2): 131/269 kB | 16/55 kB Progress (2): 131/269 kB | 33/55 kB Progress (2): 147/269 kB | 33/55 kB Progress (2): 164/269 kB | 33/55 kB Progress (2): 180/269 kB | 33/55 kB Progress (2): 197/269 kB | 33/55 kB Progress (2): 213/269 kB | 33/55 kB Progress (2): 213/269 kB | 49/55 kB Progress (2): 229/269 kB | 49/55 kB Progress (2): 229/269 kB | 55 kB Progress (2): 246/269 kB | 55 kB Progress (2): 256/269 kB | 55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar (55 kB at 1.5 MB/s) 2024-04-22T20:00:52.277 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 269 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar (269 kB at 7.1 MB/s) 2024-04-22T20:00:52.312 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/327 kB Progress (1): 32/327 kB Progress (1): 49/327 kB Progress (1): 65/327 kB Progress (1): 81/327 kB Progress (2): 81/327 kB | 16/587 kB Progress (2): 98/327 kB | 16/587 kB Progress (2): 114/327 kB | 16/587 kB Progress (2): 114/327 kB | 33/587 kB Progress (2): 130/327 kB | 33/587 kB Progress (2): 147/327 kB | 33/587 kB Progress (2): 147/327 kB | 49/587 kB Progress (2): 147/327 kB | 66/587 kB Progress (2): 163/327 kB | 66/587 kB Progress (2): 180/327 kB | 66/587 kB Progress (2): 180/327 kB | 82/587 kB Progress (2): 196/327 kB | 82/587 kB Progress (2): 212/327 kB | 82/587 kB Progress (2): 212/327 kB | 98/587 kB Progress (2): 229/327 kB | 98/587 kB Progress (2): 245/327 kB | 98/587 kB Progress (2): 245/327 kB | 115/587 kB Progress (2): 245/327 kB | 131/587 kB Progress (2): 262/327 kB | 131/587 kB Progress (2): 278/327 kB | 131/587 kB Progress (2): 278/327 kB | 147/587 kB Progress (2): 294/327 kB | 147/587 kB Progress (2): 311/327 kB | 147/587 kB Progress (2): 311/327 kB | 164/587 kB Progress (2): 311/327 kB | 180/587 kB Progress (2): 327/327 kB | 180/587 kB Progress (2): 327 kB | 180/587 kB Progress (2): 327 kB | 197/587 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar (327 kB at 4.5 MB/s) 2024-04-22T20:00:52.341 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 213/587 kB Progress (1): 229/587 kB Progress (1): 246/587 kB Progress (1): 262/587 kB Progress (1): 279/587 kB Progress (1): 295/587 kB Progress (1): 311/587 kB Progress (1): 328/587 kB Progress (1): 344/587 kB Progress (1): 360/587 kB Progress (1): 377/587 kB Progress (1): 393/587 kB Progress (1): 410/587 kB Progress (1): 426/587 kB Progress (1): 442/587 kB Progress (1): 459/587 kB Progress (1): 475/587 kB Progress (1): 492/587 kB Progress (1): 508/587 kB Progress (1): 524/587 kB Progress (1): 541/587 kB Progress (1): 557/587 kB Progress (1): 573/587 kB Progress (1): 585/587 kB Progress (1): 587 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar (587 kB at 5.8 MB/s) 2024-04-22T20:00:52.370 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-04-22T20:00:52.370 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:52.370 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-04-22T20:00:52.374 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom 2024-04-22T20:00:52.409 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom (4.7 kB at 135 kB/s) 2024-04-22T20:00:52.410 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom 2024-04-22T20:00:52.442 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom (6.4 kB at 199 kB/s) 2024-04-22T20:00:52.444 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom 2024-04-22T20:00:52.475 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/33 kB Progress (1): 32/33 kB Progress (1): 33 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 1.0 MB/s) 2024-04-22T20:00:52.477 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2024-04-22T20:00:52.507 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 451 kB/s) 2024-04-22T20:00:52.510 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom 2024-04-22T20:00:52.541 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 815 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom (815 B at 26 kB/s) 2024-04-22T20:00:52.543 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom 2024-04-22T20:00:52.578 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom (4.2 kB at 121 kB/s) 2024-04-22T20:00:52.580 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom 2024-04-22T20:00:52.615 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom (17 kB at 494 kB/s) 2024-04-22T20:00:52.617 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom 2024-04-22T20:00:52.651 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom (5.0 kB at 146 kB/s) 2024-04-22T20:00:52.653 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom 2024-04-22T20:00:52.683 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom (4.1 kB at 134 kB/s) 2024-04-22T20:00:52.686 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom 2024-04-22T20:00:52.717 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom (2.4 kB at 74 kB/s) 2024-04-22T20:00:52.719 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom 2024-04-22T20:00:52.753 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom (11 kB at 323 kB/s) 2024-04-22T20:00:52.756 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom 2024-04-22T20:00:52.793 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom (17 kB at 454 kB/s) 2024-04-22T20:00:52.795 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2024-04-22T20:00:52.832 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom (6.6 kB at 182 kB/s) 2024-04-22T20:00:52.834 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom 2024-04-22T20:00:52.873 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom (1.1 kB at 28 kB/s) 2024-04-22T20:00:52.875 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom 2024-04-22T20:00:52.911 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom (8.4 kB at 233 kB/s) 2024-04-22T20:00:52.912 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom 2024-04-22T20:00:52.942 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom (2.7 kB at 92 kB/s) 2024-04-22T20:00:52.944 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom 2024-04-22T20:00:52.969 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom (5.6 kB at 235 kB/s) 2024-04-22T20:00:52.971 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom 2024-04-22T20:00:52.995 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom (1.1 kB at 49 kB/s) 2024-04-22T20:00:52.997 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom 2024-04-22T20:00:53.025 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 770 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom (770 B at 28 kB/s) 2024-04-22T20:00:53.027 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom 2024-04-22T20:00:53.047 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom (6.0 kB at 287 kB/s) 2024-04-22T20:00:53.049 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom 2024-04-22T20:00:53.070 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom (26 kB at 1.2 MB/s) 2024-04-22T20:00:53.072 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom 2024-04-22T20:00:53.101 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom (1.2 kB at 43 kB/s) 2024-04-22T20:00:53.102 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom 2024-04-22T20:00:53.139 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom (1.3 kB at 35 kB/s) 2024-04-22T20:00:53.144 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar 2024-04-22T20:00:53.184 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar (14 kB at 338 kB/s) 2024-04-22T20:00:53.184 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar 2024-04-22T20:00:53.184 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar 2024-04-22T20:00:53.184 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar 2024-04-22T20:00:53.184 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar 2024-04-22T20:00:53.185 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar 2024-04-22T20:00:53.221 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.2 kB Progress (2): 4.2 kB | 16/55 kB Progress (2): 4.2 kB | 32/55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar (4.2 kB at 114 kB/s) 2024-04-22T20:00:53.221 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 32/55 kB | 16/122 kB Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar 2024-04-22T20:00:53.223 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 32/55 kB | 16/122 kB | 16/27 kB Progress (3): 32/55 kB | 16/122 kB | 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar (27 kB at 730 kB/s) 2024-04-22T20:00:53.223 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 32/55 kB | 33/122 kB Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar 2024-04-22T20:00:53.223 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 32/55 kB | 49/122 kB Progress (2): 49/55 kB | 49/122 kB Progress (2): 55 kB | 49/122 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar (55 kB at 1.4 MB/s) 2024-04-22T20:00:53.224 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar 2024-04-22T20:00:53.228 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 49/122 kB | 16/334 kB Progress (2): 49/122 kB | 33/334 kB Progress (2): 66/122 kB | 33/334 kB Progress (2): 82/122 kB | 33/334 kB Progress (2): 82/122 kB | 49/334 kB Progress (2): 82/122 kB | 66/334 kB Progress (2): 98/122 kB | 66/334 kB Progress (2): 115/122 kB | 66/334 kB Progress (2): 115/122 kB | 82/334 kB Progress (2): 115/122 kB | 98/334 kB Progress (2): 122 kB | 98/334 kB Progress (2): 122 kB | 115/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar (122 kB at 2.7 MB/s) 2024-04-22T20:00:53.237 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 131/334 kB Progress (1): 147/334 kB Progress (1): 164/334 kB Progress (1): 180/334 kB Progress (1): 197/334 kB Progress (1): 213/334 kB Progress (1): 229/334 kB Progress (1): 246/334 kB Progress (1): 262/334 kB Progress (1): 279/334 kB Progress (1): 295/334 kB Progress (1): 311/334 kB Progress (1): 328/334 kB Progress (1): 334 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar (334 kB at 6.2 MB/s) 2024-04-22T20:00:53.271 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.7 kB Progress (2): 4.7 kB | 16/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar (4.7 kB at 54 kB/s) 2024-04-22T20:00:53.273 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 32/267 kB Progress (2): 32/267 kB | 16/23 kB Progress (2): 49/267 kB | 16/23 kB Progress (2): 49/267 kB | 23 kB Progress (2): 65/267 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar (23 kB at 256 kB/s) 2024-04-22T20:00:53.282 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 81/267 kB Progress (1): 98/267 kB Progress (1): 114/267 kB Progress (1): 130/267 kB Progress (1): 147/267 kB Progress (1): 163/267 kB Progress (1): 180/267 kB Progress (1): 196/267 kB Progress (1): 212/267 kB Progress (1): 229/267 kB Progress (1): 245/267 kB Progress (1): 262/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar (267 kB at 2.7 MB/s) 2024-04-22T20:00:53.328 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Changes detected - recompiling the module! :source 2024-04-22T20:00:53.329 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/classes 2024-04-22T20:00:53.953 INFO:tasks.workunit.client.0.smithi136.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-04-22T20:00:53.953 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:53.954 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-04-22T20:00:53.956 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-04-22T20:00:53.956 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:53.956 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-04-22T20:00:53.961 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Changes detected - recompiling the module! :dependency 2024-04-22T20:00:53.961 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/test-classes 2024-04-22T20:00:54.107 INFO:tasks.workunit.client.0.smithi136.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-04-22T20:00:54.107 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:54.108 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-04-22T20:00:54.111 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom 2024-04-22T20:00:54.130 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom (6.2 kB at 347 kB/s) 2024-04-22T20:00:54.133 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom 2024-04-22T20:00:54.153 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom (3.5 kB at 176 kB/s) 2024-04-22T20:00:54.156 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom 2024-04-22T20:00:54.175 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom (3.3 kB at 172 kB/s) 2024-04-22T20:00:54.179 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom 2024-04-22T20:00:54.200 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom (4.1 kB at 193 kB/s) 2024-04-22T20:00:54.203 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom 2024-04-22T20:00:54.223 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom (3.3 kB at 165 kB/s) 2024-04-22T20:00:54.226 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom 2024-04-22T20:00:54.248 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom (4.5 kB at 212 kB/s) 2024-04-22T20:00:54.251 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom 2024-04-22T20:00:54.272 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom (1.8 kB at 80 kB/s) 2024-04-22T20:00:54.276 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom 2024-04-22T20:00:54.293 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom (2.2 kB at 122 kB/s) 2024-04-22T20:00:54.295 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom 2024-04-22T20:00:54.314 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom (30 kB at 1.6 MB/s) 2024-04-22T20:00:54.316 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom 2024-04-22T20:00:54.337 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom (1.9 kB at 90 kB/s) 2024-04-22T20:00:54.340 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom 2024-04-22T20:00:54.365 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom (5.8 kB at 232 kB/s) 2024-04-22T20:00:54.368 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom 2024-04-22T20:00:54.386 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom (2.3 kB at 130 kB/s) 2024-04-22T20:00:54.387 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom 2024-04-22T20:00:54.408 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom (22 kB at 1.1 MB/s) 2024-04-22T20:00:54.410 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom 2024-04-22T20:00:54.435 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/37 kB Progress (1): 33/37 kB Progress (1): 37 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom (37 kB at 1.5 MB/s) 2024-04-22T20:00:54.437 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2024-04-22T20:00:54.463 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom (15 kB at 586 kB/s) 2024-04-22T20:00:54.466 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom 2024-04-22T20:00:54.494 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom (8.7 kB at 309 kB/s) 2024-04-22T20:00:54.496 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom 2024-04-22T20:00:54.526 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom (27 kB at 913 kB/s) 2024-04-22T20:00:54.528 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom 2024-04-22T20:00:54.559 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom (8.1 kB at 260 kB/s) 2024-04-22T20:00:54.562 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom 2024-04-22T20:00:54.595 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom (2.2 kB at 66 kB/s) 2024-04-22T20:00:54.598 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom 2024-04-22T20:00:54.627 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom (2.6 kB at 90 kB/s) 2024-04-22T20:00:54.630 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom 2024-04-22T20:00:54.655 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom (1.5 kB at 62 kB/s) 2024-04-22T20:00:54.656 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom 2024-04-22T20:00:54.672 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom (3.1 kB at 180 kB/s) 2024-04-22T20:00:54.673 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom 2024-04-22T20:00:54.694 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom (20 kB at 1.0 MB/s) 2024-04-22T20:00:54.695 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom 2024-04-22T20:00:54.719 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom (6.8 kB at 282 kB/s) 2024-04-22T20:00:54.721 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom 2024-04-22T20:00:54.744 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom (14 kB at 590 kB/s) 2024-04-22T20:00:54.746 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom 2024-04-22T20:00:54.764 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom (3.0 kB at 164 kB/s) 2024-04-22T20:00:54.766 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom 2024-04-22T20:00:54.786 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom (6.8 kB at 340 kB/s) 2024-04-22T20:00:54.787 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom 2024-04-22T20:00:54.807 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom (8.4 kB at 420 kB/s) 2024-04-22T20:00:54.808 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom 2024-04-22T20:00:54.826 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom (2.1 kB at 115 kB/s) 2024-04-22T20:00:54.828 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom 2024-04-22T20:00:54.849 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom (2.2 kB at 106 kB/s) 2024-04-22T20:00:54.853 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom 2024-04-22T20:00:54.871 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom (3.0 kB at 168 kB/s) 2024-04-22T20:00:54.874 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom 2024-04-22T20:00:54.899 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom (15 kB at 599 kB/s) 2024-04-22T20:00:54.900 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom 2024-04-22T20:00:54.925 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom (15 kB at 604 kB/s) 2024-04-22T20:00:54.928 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom 2024-04-22T20:00:54.957 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom (13 kB at 448 kB/s) 2024-04-22T20:00:54.959 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom 2024-04-22T20:00:54.977 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom (4.8 kB at 266 kB/s) 2024-04-22T20:00:54.979 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom 2024-04-22T20:00:55.002 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom (6.3 kB at 273 kB/s) 2024-04-22T20:00:55.004 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom 2024-04-22T20:00:55.032 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/32 kB Progress (1): 32 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom (32 kB at 1.2 MB/s) 2024-04-22T20:00:55.035 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom 2024-04-22T20:00:55.068 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom (17 kB at 510 kB/s) 2024-04-22T20:00:55.070 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom 2024-04-22T20:00:55.107 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom (15 kB at 400 kB/s) 2024-04-22T20:00:55.110 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom 2024-04-22T20:00:55.144 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 750 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom (750 B at 22 kB/s) 2024-04-22T20:00:55.145 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom 2024-04-22T20:00:55.178 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom (4.8 kB at 141 kB/s) 2024-04-22T20:00:55.181 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom 2024-04-22T20:00:55.211 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 7.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom (7.9 kB at 264 kB/s) 2024-04-22T20:00:55.213 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom 2024-04-22T20:00:55.242 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom (3.0 kB at 103 kB/s) 2024-04-22T20:00:55.245 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom 2024-04-22T20:00:55.278 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom (4.2 kB at 129 kB/s) 2024-04-22T20:00:55.280 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom 2024-04-22T20:00:55.316 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom (2.1 kB at 59 kB/s) 2024-04-22T20:00:55.318 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom 2024-04-22T20:00:55.351 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom (3.5 kB at 102 kB/s) 2024-04-22T20:00:55.353 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom 2024-04-22T20:00:55.387 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom (11 kB at 331 kB/s) 2024-04-22T20:00:55.388 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom 2024-04-22T20:00:55.424 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom (13 kB at 374 kB/s) 2024-04-22T20:00:55.426 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom 2024-04-22T20:00:55.454 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/19 kB Progress (1): 19 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom (19 kB at 669 kB/s) 2024-04-22T20:00:55.457 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom 2024-04-22T20:00:55.481 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 363 B Downloaded from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom (363 B at 15 kB/s) 2024-04-22T20:00:55.483 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom 2024-04-22T20:00:55.506 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom (6.1 kB at 254 kB/s) 2024-04-22T20:00:55.508 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom 2024-04-22T20:00:55.532 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom (7.3 kB at 293 kB/s) 2024-04-22T20:00:55.534 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2024-04-22T20:00:55.561 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom (4.8 kB at 179 kB/s) 2024-04-22T20:00:55.564 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom 2024-04-22T20:00:55.597 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom (7.3 kB at 221 kB/s) 2024-04-22T20:00:55.600 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom 2024-04-22T20:00:55.637 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom (22 kB at 570 kB/s) 2024-04-22T20:00:55.639 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom 2024-04-22T20:00:55.678 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 66/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom (78 kB at 2.1 MB/s) 2024-04-22T20:00:55.682 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom 2024-04-22T20:00:55.712 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom (23 kB at 801 kB/s) 2024-04-22T20:00:55.715 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom 2024-04-22T20:00:55.736 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom (5.6 kB at 269 kB/s) 2024-04-22T20:00:55.739 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom 2024-04-22T20:00:55.761 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom (4.3 kB at 186 kB/s) 2024-04-22T20:00:55.764 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom 2024-04-22T20:00:55.785 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom (3.2 kB at 145 kB/s) 2024-04-22T20:00:55.787 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom 2024-04-22T20:00:55.811 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/28 kB Progress (1): 28 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom (28 kB at 1.2 MB/s) 2024-04-22T20:00:55.814 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom 2024-04-22T20:00:55.837 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom (2.4 kB at 103 kB/s) 2024-04-22T20:00:55.844 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar 2024-04-22T20:00:55.889 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/308 kB Progress (1): 33/308 kB Progress (1): 49/308 kB Progress (1): 66/308 kB Progress (1): 82/308 kB Progress (1): 98/308 kB Progress (1): 115/308 kB Progress (1): 131/308 kB Progress (1): 147/308 kB Progress (1): 164/308 kB Progress (1): 180/308 kB Progress (1): 197/308 kB Progress (1): 213/308 kB Progress (1): 229/308 kB Progress (1): 246/308 kB Progress (1): 262/308 kB Progress (1): 279/308 kB Progress (1): 295/308 kB Progress (1): 308 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar (308 kB at 6.8 MB/s) 2024-04-22T20:00:55.890 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar 2024-04-22T20:00:55.890 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar 2024-04-22T20:00:55.890 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar 2024-04-22T20:00:55.890 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar 2024-04-22T20:00:55.890 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar 2024-04-22T20:00:55.917 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar (14 kB at 503 kB/s) 2024-04-22T20:00:55.917 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/26 kB Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar 2024-04-22T20:00:55.918 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 26 kB Progress (2): 26 kB | 8.2 kB Progress (3): 26 kB | 8.2 kB | 16/118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar (26 kB at 923 kB/s) 2024-04-22T20:00:55.919 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar 2024-04-22T20:00:55.919 INFO:tasks.workunit.client.0.smithi136.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar (8.2 kB at 282 kB/s) 2024-04-22T20:00:55.919 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar 2024-04-22T20:00:55.924 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 16/118 kB | 16/171 kB Progress (2): 33/118 kB | 16/171 kB Progress (2): 49/118 kB | 16/171 kB Progress (2): 49/118 kB | 33/171 kB Progress (2): 66/118 kB | 33/171 kB Progress (2): 82/118 kB | 33/171 kB Progress (2): 82/118 kB | 49/171 kB Progress (2): 98/118 kB | 49/171 kB Progress (2): 115/118 kB | 49/171 kB Progress (2): 118 kB | 49/171 kB Progress (2): 118 kB | 66/171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar (118 kB at 3.5 MB/s) 2024-04-22T20:00:55.924 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar 2024-04-22T20:00:55.931 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 82/171 kB Progress (1): 98/171 kB Progress (1): 115/171 kB Progress (1): 131/171 kB Progress (1): 147/171 kB Progress (1): 164/171 kB Progress (1): 171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar (171 kB at 4.2 MB/s) 2024-04-22T20:00:55.931 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar 2024-04-22T20:00:55.950 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/146 kB Progress (2): 16/146 kB | 16/136 kB Progress (2): 33/146 kB | 16/136 kB Progress (3): 33/146 kB | 16/136 kB | 16/61 kB Progress (3): 33/146 kB | 33/136 kB | 16/61 kB Progress (3): 33/146 kB | 49/136 kB | 16/61 kB Progress (3): 33/146 kB | 49/136 kB | 33/61 kB Progress (3): 33/146 kB | 49/136 kB | 49/61 kB Progress (3): 49/146 kB | 49/136 kB | 49/61 kB Progress (3): 49/146 kB | 66/136 kB | 49/61 kB Progress (3): 49/146 kB | 66/136 kB | 61 kB Progress (3): 49/146 kB | 82/136 kB | 61 kB Progress (3): 66/146 kB | 82/136 kB | 61 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar (61 kB at 1.0 MB/s) 2024-04-22T20:00:55.950 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar 2024-04-22T20:00:55.952 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 82/146 kB | 82/136 kB Progress (2): 82/146 kB | 98/136 kB Progress (2): 82/146 kB | 115/136 kB Progress (2): 98/146 kB | 115/136 kB Progress (2): 98/146 kB | 131/136 kB Progress (2): 98/146 kB | 136 kB Progress (2): 115/146 kB | 136 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar (136 kB at 2.2 MB/s) 2024-04-22T20:00:55.952 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar 2024-04-22T20:00:55.954 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 131/146 kB Progress (1): 146 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar (146 kB at 2.2 MB/s) 2024-04-22T20:00:55.967 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/501 kB Progress (1): 32/501 kB Progress (1): 49/501 kB Progress (1): 65/501 kB Progress (1): 81/501 kB Progress (1): 98/501 kB Progress (1): 114/501 kB Progress (1): 130/501 kB Progress (1): 147/501 kB Progress (1): 163/501 kB Progress (1): 180/501 kB Progress (1): 196/501 kB Progress (1): 212/501 kB Progress (1): 229/501 kB Progress (1): 245/501 kB Progress (1): 262/501 kB Progress (1): 278/501 kB Progress (2): 278/501 kB | 16/58 kB Progress (2): 294/501 kB | 16/58 kB Progress (2): 294/501 kB | 33/58 kB Progress (2): 311/501 kB | 33/58 kB Progress (2): 327/501 kB | 33/58 kB Progress (2): 343/501 kB | 33/58 kB Progress (2): 343/501 kB | 49/58 kB Progress (2): 360/501 kB | 49/58 kB Progress (2): 360/501 kB | 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar (58 kB at 738 kB/s) 2024-04-22T20:00:55.972 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 376/501 kB Progress (1): 393/501 kB Progress (1): 402/501 kB Progress (1): 418/501 kB Progress (1): 435/501 kB Progress (1): 451/501 kB Progress (1): 468/501 kB Progress (1): 484/501 kB Progress (1): 500/501 kB Progress (1): 501 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar (501 kB at 6.1 MB/s) 2024-04-22T20:00:55.988 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/124 kB Progress (1): 33/124 kB Progress (1): 49/124 kB Progress (1): 66/124 kB Progress (1): 82/124 kB Progress (1): 98/124 kB Progress (1): 115/124 kB Progress (1): 124 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar (124 kB at 1.3 MB/s) 2024-04-22T20:00:56.084 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 0/2.4 MB Progress (1): 0/2.4 MB Progress (1): 0/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.1/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar (2.4 MB at 13 MB/s) 2024-04-22T20:00:56.146 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:56.146 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- jar:3.3.0:jar (default-jar) @ jcksum --- 2024-04-22T20:00:56.149 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom 2024-04-22T20:00:56.171 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom (4.5 kB at 195 kB/s) 2024-04-22T20:00:56.173 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom 2024-04-22T20:00:56.192 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom (4.9 kB at 272 kB/s) 2024-04-22T20:00:56.193 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom 2024-04-22T20:00:56.210 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom (45 kB at 2.5 MB/s) 2024-04-22T20:00:56.212 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom 2024-04-22T20:00:56.230 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom (21 kB at 1.2 MB/s) 2024-04-22T20:00:56.233 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom 2024-04-22T20:00:56.249 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 5.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom (5.2 kB at 305 kB/s) 2024-04-22T20:00:56.251 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom 2024-04-22T20:00:56.269 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 3.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom (3.9 kB at 231 kB/s) 2024-04-22T20:00:56.271 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom 2024-04-22T20:00:56.288 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom (6.0 kB at 354 kB/s) 2024-04-22T20:00:56.290 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom 2024-04-22T20:00:56.310 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom (6.3 kB at 313 kB/s) 2024-04-22T20:00:56.312 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom 2024-04-22T20:00:56.330 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom (20 kB at 1.1 MB/s) 2024-04-22T20:00:56.333 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom 2024-04-22T20:00:56.355 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom (15 kB at 632 kB/s) 2024-04-22T20:00:56.357 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom 2024-04-22T20:00:56.377 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom (2.0 kB at 108 kB/s) 2024-04-22T20:00:56.379 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom 2024-04-22T20:00:56.398 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom (8.2 kB at 432 kB/s) 2024-04-22T20:00:56.400 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom 2024-04-22T20:00:56.420 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom (25 kB at 1.2 MB/s) 2024-04-22T20:00:56.425 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar 2024-04-22T20:00:56.445 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar (36 kB at 1.8 MB/s) 2024-04-22T20:00:56.445 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar 2024-04-22T20:00:56.445 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar 2024-04-22T20:00:56.445 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar 2024-04-22T20:00:56.446 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar 2024-04-22T20:00:56.446 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar 2024-04-22T20:00:56.559 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 0/1.0 MB Progress (2): 0/1.0 MB | 16/26 kB Progress (3): 0/1.0 MB | 16/26 kB | 16/58 kB Progress (4): 0/1.0 MB | 16/26 kB | 16/58 kB | 16/79 kB Progress (5): 0/1.0 MB | 16/26 kB | 16/58 kB | 16/79 kB | 16/211 kB Progress (5): 0/1.0 MB | 26 kB | 16/58 kB | 16/79 kB | 16/211 kB Progress (5): 0/1.0 MB | 26 kB | 16/58 kB | 16/79 kB | 16/211 kB Progress (5): 0/1.0 MB | 26 kB | 33/58 kB | 16/79 kB | 16/211 kB Progress (5): 0/1.0 MB | 26 kB | 33/58 kB | 33/79 kB | 16/211 kB Progress (5): 0/1.0 MB | 26 kB | 33/58 kB | 33/79 kB | 32/211 kB Progress (5): 0/1.0 MB | 26 kB | 33/58 kB | 33/79 kB | 32/211 kB Progress (5): 0/1.0 MB | 26 kB | 49/58 kB | 33/79 kB | 32/211 kB Progress (5): 0/1.0 MB | 26 kB | 49/58 kB | 49/79 kB | 32/211 kB Progress (5): 0/1.0 MB | 26 kB | 49/58 kB | 49/79 kB | 49/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar (26 kB at 241 kB/s) 2024-04-22T20:00:56.559 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (4): 0/1.0 MB | 58 kB | 49/79 kB | 49/211 kB Progress (4): 0.1/1.0 MB | 58 kB | 49/79 kB | 49/211 kB Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar 2024-04-22T20:00:56.559 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (4): 0.1/1.0 MB | 58 kB | 62/79 kB | 49/211 kB Progress (4): 0.1/1.0 MB | 58 kB | 62/79 kB | 65/211 kB Progress (4): 0.1/1.0 MB | 58 kB | 62/79 kB | 65/211 kB Progress (4): 0.1/1.0 MB | 58 kB | 78/79 kB | 65/211 kB Progress (4): 0.1/1.0 MB | 58 kB | 78/79 kB | 78/211 kB Progress (4): 0.1/1.0 MB | 58 kB | 79 kB | 78/211 kB Progress (4): 0.1/1.0 MB | 58 kB | 79 kB | 78/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar (58 kB at 532 kB/s) 2024-04-22T20:00:56.559 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar 2024-04-22T20:00:56.559 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 0.1/1.0 MB | 79 kB | 95/211 kB Progress (3): 0.1/1.0 MB | 79 kB | 95/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar (79 kB at 721 kB/s) 2024-04-22T20:00:56.559 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 0.1/1.0 MB | 95/211 kB Progress (2): 0.1/1.0 MB | 111/211 kB Progress (2): 0.1/1.0 MB | 111/211 kB Progress (2): 0.1/1.0 MB | 127/211 kB Progress (2): 0.1/1.0 MB | 144/211 kB Progress (2): 0.2/1.0 MB | 144/211 kB Progress (2): 0.2/1.0 MB | 160/211 kB Progress (2): 0.2/1.0 MB | 160/211 kB Progress (2): 0.2/1.0 MB | 176/211 kB Progress (2): 0.2/1.0 MB | 176/211 kB Progress (2): 0.2/1.0 MB | 193/211 kB Progress (2): 0.2/1.0 MB | 193/211 kB Progress (2): 0.2/1.0 MB | 209/211 kB Progress (2): 0.2/1.0 MB | 211 kB Progress (2): 0.2/1.0 MB | 211 kB Progress (2): 0.2/1.0 MB | 211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar (211 kB at 1.9 MB/s) 2024-04-22T20:00:56.569 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar (1.0 MB at 8.3 MB/s) 2024-04-22T20:00:56.586 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/116 kB Progress (1): 32/116 kB Progress (2): 32/116 kB | 16/267 kB Progress (2): 32/116 kB | 32/267 kB Progress (2): 49/116 kB | 32/267 kB Progress (2): 49/116 kB | 49/267 kB Progress (2): 49/116 kB | 65/267 kB Progress (2): 65/116 kB | 65/267 kB Progress (2): 65/116 kB | 81/267 kB Progress (2): 81/116 kB | 81/267 kB Progress (2): 81/116 kB | 98/267 kB Progress (2): 81/116 kB | 114/267 kB Progress (2): 98/116 kB | 114/267 kB Progress (2): 98/116 kB | 131/267 kB Progress (2): 98/116 kB | 147/267 kB Progress (2): 114/116 kB | 147/267 kB Progress (2): 116 kB | 147/267 kB Progress (2): 116 kB | 163/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar (116 kB at 829 kB/s) 2024-04-22T20:00:56.590 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 180/267 kB Progress (1): 196/267 kB Progress (1): 212/267 kB Progress (1): 229/267 kB Progress (1): 245/267 kB Progress (1): 262/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar (267 kB at 1.8 MB/s) 2024-04-22T20:00:56.714 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Building jar: /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/target/jcksum-1.0.jar 2024-04-22T20:00:56.753 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-22T20:00:56.753 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] BUILD SUCCESS 2024-04-22T20:00:56.753 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-22T20:00:56.754 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Total time: 8.131 s 2024-04-22T20:00:56.755 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Finished at: 2024-04-22T20:00:56Z 2024-04-22T20:00:56.755 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-22T20:00:56.779 INFO:tasks.workunit.client.0.smithi136.stderr:+ ./mvnw test -Dtest=PutObjects 2024-04-22T20:00:56.788 INFO:tasks.workunit.client.0.smithi136.stdout:Warning: JAVA_HOME environment variable is not set. 2024-04-22T20:00:57.833 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Scanning for projects... 2024-04-22T20:00:57.937 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:57.938 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-04-22T20:00:57.938 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Building jcksum 1.0 2024-04-22T20:00:57.938 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] from pom.xml 2024-04-22T20:00:57.938 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-04-22T20:00:58.258 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:58.258 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-04-22T20:00:58.368 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-04-22T20:00:58.369 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:58.369 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-04-22T20:00:58.476 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Nothing to compile - all classes are up to date 2024-04-22T20:00:58.476 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:58.476 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-04-22T20:00:58.479 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-04-22T20:00:58.479 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:58.479 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-04-22T20:00:58.485 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Nothing to compile - all classes are up to date 2024-04-22T20:00:58.485 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:58.485 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-04-22T20:00:58.712 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider 2024-04-22T20:00:58.902 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom 2024-04-22T20:00:59.111 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/4.7 kB Progress (1): 2.8/4.7 kB Progress (1): 4.1/4.7 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom (4.7 kB at 22 kB/s) 2024-04-22T20:00:59.119 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom 2024-04-22T20:00:59.140 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/2.6 kB Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom (2.6 kB at 131 kB/s) 2024-04-22T20:00:59.144 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom 2024-04-22T20:00:59.164 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom (2.8 kB at 141 kB/s) 2024-04-22T20:00:59.168 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom 2024-04-22T20:00:59.188 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/3.0 kB Progress (1): 2.8/3.0 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom (3.0 kB at 159 kB/s) 2024-04-22T20:00:59.190 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom 2024-04-22T20:00:59.210 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/3.2 kB Progress (1): 2.8/3.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom (3.2 kB at 160 kB/s) 2024-04-22T20:00:59.213 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom 2024-04-22T20:00:59.231 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/1.7 kB Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom (1.7 kB at 89 kB/s) 2024-04-22T20:00:59.234 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom 2024-04-22T20:00:59.253 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom (2.8 kB at 149 kB/s) 2024-04-22T20:00:59.257 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar 2024-04-22T20:00:59.284 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/27 kB Progress (1): 2.8/27 kB Progress (1): 4.1/27 kB Progress (1): 5.5/27 kB Progress (1): 6.9/27 kB Progress (1): 8.3/27 kB Progress (1): 9.7/27 kB Progress (1): 11/27 kB Progress (1): 12/27 kB Progress (1): 14/27 kB Progress (1): 15/27 kB Progress (1): 17/27 kB Progress (1): 18/27 kB Progress (1): 19/27 kB Progress (1): 21/27 kB Progress (1): 22/27 kB Progress (1): 23/27 kB Progress (1): 25/27 kB Progress (1): 26/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar (27 kB at 1.0 MB/s) 2024-04-22T20:00:59.287 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar 2024-04-22T20:00:59.287 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar 2024-04-22T20:00:59.287 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar 2024-04-22T20:00:59.287 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 2024-04-22T20:00:59.288 INFO:tasks.workunit.client.0.smithi136.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar 2024-04-22T20:00:59.309 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar (18 kB at 702 kB/s) 2024-04-22T20:00:59.351 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (1): 1.4/7.7 kB Progress (2): 1.4/7.7 kB | 1.4/189 kB Progress (2): 2.8/7.7 kB | 1.4/189 kB Progress (2): 2.8/7.7 kB | 2.8/189 kB Progress (2): 4.1/7.7 kB | 2.8/189 kB Progress (2): 4.1/7.7 kB | 4.1/189 kB Progress (2): 5.5/7.7 kB | 4.1/189 kB Progress (2): 5.5/7.7 kB | 5.5/189 kB Progress (2): 6.9/7.7 kB | 5.5/189 kB Progress (3): 6.9/7.7 kB | 5.5/189 kB | 16/103 kB Progress (3): 6.9/7.7 kB | 6.9/189 kB | 16/103 kB Progress (3): 7.7 kB | 6.9/189 kB | 16/103 kB Progress (4): 7.7 kB | 6.9/189 kB | 16/103 kB | 16/169 kB Progress (4): 7.7 kB | 8.3/189 kB | 16/103 kB | 16/169 kB Progress (4): 7.7 kB | 9.7/189 kB | 16/103 kB | 16/169 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar (7.7 kB at 120 kB/s) 2024-04-22T20:00:59.380 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (3): 11/189 kB | 16/103 kB | 16/169 kB Progress (3): 12/189 kB | 16/103 kB | 16/169 kB Progress (3): 14/189 kB | 16/103 kB | 16/169 kB Progress (3): 15/189 kB | 16/103 kB | 16/169 kB Progress (3): 17/189 kB | 16/103 kB | 16/169 kB Progress (3): 18/189 kB | 16/103 kB | 16/169 kB Progress (3): 19/189 kB | 16/103 kB | 16/169 kB Progress (3): 21/189 kB | 16/103 kB | 16/169 kB Progress (3): 22/189 kB | 16/103 kB | 16/169 kB Progress (3): 23/189 kB | 16/103 kB | 16/169 kB Progress (3): 25/189 kB | 16/103 kB | 16/169 kB Progress (3): 26/189 kB | 16/103 kB | 16/169 kB Progress (3): 28/189 kB | 16/103 kB | 16/169 kB Progress (3): 29/189 kB | 16/103 kB | 16/169 kB Progress (3): 30/189 kB | 16/103 kB | 16/169 kB Progress (3): 32/189 kB | 16/103 kB | 16/169 kB Progress (3): 32/189 kB | 33/103 kB | 16/169 kB Progress (3): 33/189 kB | 33/103 kB | 16/169 kB Progress (3): 33/189 kB | 33/103 kB | 33/169 kB Progress (3): 34/189 kB | 33/103 kB | 33/169 kB Progress (3): 36/189 kB | 33/103 kB | 33/169 kB Progress (3): 37/189 kB | 33/103 kB | 33/169 kB Progress (3): 39/189 kB | 33/103 kB | 33/169 kB Progress (3): 39/189 kB | 49/103 kB | 33/169 kB Progress (3): 40/189 kB | 49/103 kB | 33/169 kB Progress (3): 41/189 kB | 49/103 kB | 33/169 kB Progress (3): 41/189 kB | 49/103 kB | 49/169 kB Progress (3): 43/189 kB | 49/103 kB | 49/169 kB Progress (3): 44/189 kB | 49/103 kB | 49/169 kB Progress (3): 46/189 kB | 49/103 kB | 49/169 kB Progress (3): 47/189 kB | 49/103 kB | 49/169 kB Progress (3): 48/189 kB | 49/103 kB | 49/169 kB Progress (3): 50/189 kB | 49/103 kB | 49/169 kB Progress (3): 51/189 kB | 49/103 kB | 49/169 kB Progress (3): 52/189 kB | 49/103 kB | 49/169 kB Progress (3): 54/189 kB | 49/103 kB | 49/169 kB Progress (3): 55/189 kB | 49/103 kB | 49/169 kB Progress (3): 57/189 kB | 49/103 kB | 49/169 kB Progress (3): 58/189 kB | 49/103 kB | 49/169 kB Progress (3): 59/189 kB | 49/103 kB | 49/169 kB Progress (3): 61/189 kB | 49/103 kB | 49/169 kB Progress (3): 62/189 kB | 49/103 kB | 49/169 kB Progress (3): 62/189 kB | 66/103 kB | 49/169 kB Progress (3): 63/189 kB | 66/103 kB | 49/169 kB Progress (3): 63/189 kB | 66/103 kB | 66/169 kB Progress (3): 63/189 kB | 82/103 kB | 66/169 kB Progress (3): 79/189 kB | 82/103 kB | 66/169 kB Progress (3): 79/189 kB | 82/103 kB | 82/169 kB Progress (3): 79/189 kB | 98/103 kB | 82/169 kB Progress (3): 79/189 kB | 103 kB | 82/169 kB Progress (3): 96/189 kB | 103 kB | 82/169 kB Progress (3): 96/189 kB | 103 kB | 98/169 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar (103 kB at 1.1 MB/s) 2024-04-22T20:00:59.392 INFO:tasks.workunit.client.0.smithi136.stdout:Progress (2): 96/189 kB | 115/169 kB Progress (2): 112/189 kB | 115/169 kB Progress (2): 112/189 kB | 131/169 kB Progress (2): 128/189 kB | 131/169 kB Progress (2): 128/189 kB | 147/169 kB Progress (2): 145/189 kB | 147/169 kB Progress (2): 145/189 kB | 164/169 kB Progress (2): 161/189 kB | 164/169 kB Progress (2): 161/189 kB | 169 kB Progress (2): 177/189 kB | 169 kB Progress (2): 189 kB | 169 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar (169 kB at 1.6 MB/s) 2024-04-22T20:00:59.393 INFO:tasks.workunit.client.0.smithi136.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar (189 kB at 1.8 MB/s) 2024-04-22T20:00:59.412 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:00:59.413 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ------------------------------------------------------- 2024-04-22T20:00:59.413 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] T E S T S 2024-04-22T20:00:59.413 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ------------------------------------------------------- 2024-04-22T20:00:59.865 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Running io.ceph.jcksum.PutObjects 2024-04-22T20:00:59.899 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjects.java: starting test run: 2024-04-22T20:00:59.997 INFO:tasks.workunit.client.0.smithi136.stdout: AccessKey=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-04-22T20:00:59.997 INFO:tasks.workunit.client.0.smithi136.stdout: SecretKey=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-04-22T20:00:59.997 INFO:tasks.workunit.client.0.smithi136.stdout: EndpointUrl=https://smithi136.front.sepia.ceph.com:443 2024-04-22T20:00:59.997 INFO:tasks.workunit.client.0.smithi136.stderr:[main] WARN software.amazon.awssdk.http.apache.ApacheHttpClient - SSL Certificate verification is disabled. This is not a safe setting and should only be used for testing. 2024-04-22T20:01:00.097 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Environment Variables Read 2024-04-22T20:01:00.398 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: S3 Client Initialized 2024-04-22T20:01:00.398 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Generating File 2024-04-22T20:01:00.398 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: File Generated 2024-04-22T20:01:00.398 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Generating File 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: File Generated 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Generating File 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: File Generated 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Generating File 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: File Generated 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Generating File 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: File Generated 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Generating File 2024-04-22T20:01:00.498 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: File Generated 2024-04-22T20:01:00.499 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Generated Big Files 2024-04-22T20:01:00.887 INFO:tasks.workunit.client.0.smithi136.stdout:HeadBucketResponse() 2024-04-22T20:01:00.888 INFO:tasks.workunit.client.0.smithi136.stdout:sheik is ready 2024-04-22T20:01:00.888 INFO:tasks.workunit.client.0.smithi136.stdout:DEBUG: Test Bucket Created 2024-04-22T20:01:00.898 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-0b 2024-04-22T20:01:01.098 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-8b 2024-04-22T20:01:01.299 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-200b 2024-04-22T20:01:01.399 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-21983b 2024-04-22T20:01:01.499 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-5519b 2024-04-22T20:01:01.599 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-204329b 2024-04-22T20:01:01.800 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-256k 2024-04-22T20:01:02.100 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-1m 2024-04-22T20:01:02.601 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileNoCksum called with file-1038757b 2024-04-22T20:01:03.101 INFO:tasks.workunit.client.0.smithi136.stdout:mpuObjectFromFileNoCksum called with file-5m 2024-04-22T20:01:07.108 INFO:tasks.workunit.client.0.smithi136.stdout:mpuObjectFromFileNoCksum called with file-10m 2024-04-22T20:01:11.013 INFO:tasks.workunit.client.0.smithi136.stdout:mpuObjectFromFileNoCksum called with file-100m 2024-04-22T20:01:46.105 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-0b 2024-04-22T20:01:46.106 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-8b 2024-04-22T20:01:46.107 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-200b 2024-04-22T20:01:46.108 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-21983b 2024-04-22T20:01:46.109 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-5519b 2024-04-22T20:01:46.110 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-204329b 2024-04-22T20:01:46.111 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-256k 2024-04-22T20:01:46.112 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-1m 2024-04-22T20:01:46.113 INFO:tasks.workunit.client.0.smithi136.stdout:arg: file-1038757b 2024-04-22T20:01:46.154 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-0b 2024-04-22T20:01:46.236 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:46.254 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-8b 2024-04-22T20:01:46.354 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:46.455 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-200b 2024-04-22T20:01:46.455 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:46.554 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-21983b 2024-04-22T20:01:46.654 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:46.755 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-5519b 2024-04-22T20:01:46.755 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:46.855 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-204329b 2024-04-22T20:01:46.954 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:47.055 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-256k 2024-04-22T20:01:47.154 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:47.255 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-1m 2024-04-22T20:01:47.555 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:47.655 INFO:tasks.workunit.client.0.smithi136.stdout:putObjectFromFileCksum called with file-1038757b 2024-04-22T20:01:47.956 INFO:tasks.workunit.client.0.smithi136.stdout:PutObjectResponse 2024-04-22T20:01:48.156 INFO:tasks.workunit.client.0.smithi136.stdout:mpuObjectFromFileCksum called with file-5m 2024-04-22T20:01:50.359 INFO:tasks.workunit.client.0.smithi136.stdout:mpuObjectFromFileCksum called with file-10m 2024-04-22T20:01:55.163 INFO:tasks.workunit.client.0.smithi136.stdout:mpuObjectFromFileCksum called with file-100m 2024-04-22T20:02:33.960 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 94.09 s -- in io.ceph.jcksum.PutObjects 2024-04-22T20:02:33.973 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:02:33.974 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Results: 2024-04-22T20:02:33.974 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:02:33.974 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0 2024-04-22T20:02:33.974 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] 2024-04-22T20:02:33.976 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-22T20:02:33.976 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] BUILD SUCCESS 2024-04-22T20:02:33.976 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-22T20:02:33.976 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Total time: 01:36 min 2024-04-22T20:02:33.977 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] Finished at: 2024-04-22T20:02:33Z 2024-04-22T20:02:33.977 INFO:tasks.workunit.client.0.smithi136.stdout:[INFO] ------------------------------------------------------------------------ 2024-04-22T20:02:33.999 INFO:tasks.workunit.client.0.smithi136.stderr:+ exit 2024-04-22T20:02:34.000 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T20:02:34.000 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T20:02:34.077 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh'] on client.0... 2024-04-22T20:02:34.077 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T20:02:34.420 DEBUG:teuthology.parallel:result is None 2024-04-22T20:02:34.420 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:02:34.457 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:02:34.457 DEBUG:teuthology.orchestra.run.smithi136:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T20:02:34.524 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:02:34.524 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T20:02:34.533 INFO:tasks.workunit:Pulling workunits from ref 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T20:02:34.533 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T20:02:34.533 DEBUG:teuthology.orchestra.run.smithi136:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T20:02:34.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T20:02:34.591 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-22T20:02:34.591 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T20:02:34.658 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-22T20:02:34.658 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-22T20:02:34.726 INFO:tasks.workunit:timeout=3h 2024-04-22T20:02:34.726 INFO:tasks.workunit:cleanup=True 2024-04-22T20:02:34.726 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T20:02:34.793 INFO:tasks.workunit.client.0.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T20:03:58.342 INFO:tasks.workunit.client.0.smithi136.stderr:Note: switching to '9ed8263edff7e87907bf24b054e5c7144c194dbe'. 2024-04-22T20:03:58.342 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:03:58.342 INFO:tasks.workunit.client.0.smithi136.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr: git switch -c 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr:Or undo this operation with: 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr: git switch - 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:03:58.343 INFO:tasks.workunit.client.0.smithi136.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T20:03:58.344 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:03:58.344 INFO:tasks.workunit.client.0.smithi136.stderr:HEAD is now at 9ed8263edff rgw: apply default quota config on account creation 2024-04-22T20:03:58.348 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T20:03:58.415 INFO:tasks.workunit.client.0.smithi136.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T20:03:58.417 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T20:03:58.417 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T20:03:58.465 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T20:03:58.515 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T20:03:58.555 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T20:03:58.556 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T20:03:58.556 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T20:03:58.595 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T20:03:58.598 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T20:03:58.599 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T20:03:58.668 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_s3_mp_reupload.sh on client.0... 2024-04-22T20:03:58.669 INFO:tasks.workunit:Running workunit rgw/test_rgw_s3_mp_reupload.sh... 2024-04-22T20:03:58.669 DEBUG:teuthology.orchestra.run.smithi136:workunit test rgw/test_rgw_s3_mp_reupload.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh 2024-04-22T20:03:58.741 INFO:tasks.workunit.client.0.smithi136.stdout:INFO: fully qualified domain name: smithi136 2024-04-22T20:03:58.741 INFO:tasks.workunit.client.0.smithi136.stdout:== TESTING *ABORTING* MULTIPART UPLOAD WITH RE-UPLOADS == 2024-04-22T20:03:58.741 INFO:tasks.workunit.client.0.smithi136.stdout:== TESTING NON-VERSIONED BUCKET == 2024-04-22T20:04:01.395 INFO:tasks.workunit.client.0.smithi136.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.9/site-packages (21.3.1) 2024-04-22T20:04:01.607 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting pip 2024-04-22T20:04:01.613 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-04-22T20:04:01.661 INFO:tasks.workunit.client.0.smithi136.stdout:Installing collected packages: pip 2024-04-22T20:04:01.661 INFO:tasks.workunit.client.0.smithi136.stdout: Attempting uninstall: pip 2024-04-22T20:04:01.661 INFO:tasks.workunit.client.0.smithi136.stdout: Found existing installation: pip 21.3.1 2024-04-22T20:04:01.816 INFO:tasks.workunit.client.0.smithi136.stdout: Uninstalling pip-21.3.1: 2024-04-22T20:04:01.821 INFO:tasks.workunit.client.0.smithi136.stdout: Successfully uninstalled pip-21.3.1 2024-04-22T20:04:02.916 INFO:tasks.workunit.client.0.smithi136.stdout:Successfully installed pip-24.0 2024-04-22T20:04:03.791 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting boto3 2024-04-22T20:04:03.793 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached boto3-1.34.89-py3-none-any.whl.metadata (6.6 kB) 2024-04-22T20:04:04.332 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting botocore<1.35.0,>=1.34.89 (from boto3) 2024-04-22T20:04:04.334 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached botocore-1.34.89-py3-none-any.whl.metadata (5.7 kB) 2024-04-22T20:04:04.398 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-04-22T20:04:04.399 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-04-22T20:04:04.454 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-04-22T20:04:04.455 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-04-22T20:04:04.513 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T20:04:04.514 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-04-22T20:04:04.588 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting urllib3<1.27,>=1.25.4 (from botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T20:04:04.589 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB) 2024-04-22T20:04:04.760 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T20:04:04.761 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-04-22T20:04:04.783 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached boto3-1.34.89-py3-none-any.whl (139 kB) 2024-04-22T20:04:04.785 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached botocore-1.34.89-py3-none-any.whl (12.2 MB) 2024-04-22T20:04:04.805 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-04-22T20:04:04.806 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-04-22T20:04:04.808 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-04-22T20:04:04.809 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB) 2024-04-22T20:04:04.811 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-22T20:04:04.913 INFO:tasks.workunit.client.0.smithi136.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-04-22T20:04:05.951 INFO:tasks.workunit.client.0.smithi136.stdout:Successfully installed boto3-1.34.89 botocore-1.34.89 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-1.26.18 2024-04-22T20:04:06.608 INFO:tasks.workunit.client.0.smithi136.stdout:INFO: creating user 2024-04-22T20:04:06.919 INFO:tasks.workunit.client.0.smithi136.stdout:{ 2024-04-22T20:04:06.919 INFO:tasks.workunit.client.0.smithi136.stdout: "user_id": "testid", 2024-04-22T20:04:06.919 INFO:tasks.workunit.client.0.smithi136.stdout: "display_name": "M. Tester", 2024-04-22T20:04:06.919 INFO:tasks.workunit.client.0.smithi136.stdout: "email": "tester@ceph.com", 2024-04-22T20:04:06.919 INFO:tasks.workunit.client.0.smithi136.stdout: "suspended": 0, 2024-04-22T20:04:06.919 INFO:tasks.workunit.client.0.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:04:06.919 INFO:tasks.workunit.client.0.smithi136.stdout: "subusers": [], 2024-04-22T20:04:06.919 INFO:tasks.workunit.client.0.smithi136.stdout: "keys": [ 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: { 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "user": "testid", 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "access_key": "0555b35654ad1656d804", 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==", 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "active": true, 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "create_date": "2024-04-22T20:04:06.911263Z" 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: } 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: ], 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "swift_keys": [], 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "caps": [], 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "default_placement": "", 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "placement_tags": [], 2024-04-22T20:04:06.920 INFO:tasks.workunit.client.0.smithi136.stdout: "bucket_quota": { 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "enabled": false, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size": -1, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "max_objects": -1 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "user_quota": { 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "enabled": false, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size": -1, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "max_objects": -1 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: }, 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "type": "rgw", 2024-04-22T20:04:06.921 INFO:tasks.workunit.client.0.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:04:06.922 INFO:tasks.workunit.client.0.smithi136.stdout: "account_id": "", 2024-04-22T20:04:06.922 INFO:tasks.workunit.client.0.smithi136.stdout: "path": "/", 2024-04-22T20:04:06.922 INFO:tasks.workunit.client.0.smithi136.stdout: "create_date": "2024-04-22T20:04:06.911251Z", 2024-04-22T20:04:06.922 INFO:tasks.workunit.client.0.smithi136.stdout: "tags": [], 2024-04-22T20:04:06.922 INFO:tasks.workunit.client.0.smithi136.stdout: "group_ids": [] 2024-04-22T20:04:06.922 INFO:tasks.workunit.client.0.smithi136.stdout:} 2024-04-22T20:04:06.922 INFO:tasks.workunit.client.0.smithi136.stdout: 2024-04-22T20:04:14.049 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:14.049 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:14.115 INFO:tasks.workunit.client.0.smithi136.stderr:5+0 records in 2024-04-22T20:04:14.115 INFO:tasks.workunit.client.0.smithi136.stderr:5+0 records out 2024-04-22T20:04:14.115 INFO:tasks.workunit.client.0.smithi136.stderr:5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0145649 s, 360 MB/s 2024-04-22T20:04:14.118 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:14.118 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:14.196 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:14.196 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:15.234 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:15.234 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:16.278 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:16.279 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:17.327 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:17.327 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:18.370 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:18.370 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:19.422 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:19.422 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:20.469 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:20.469 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:21.512 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:21.513 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:22.558 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:22.558 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:23.603 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:23.603 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:24.655 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:24.655 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:25.699 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:25.699 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:26.742 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:26.742 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:27.778 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:04:27.778 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:04:27.856 INFO:tasks.workunit.client.0.smithi136.stdout:bucket nams is reupload-bkt-108604 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:endpoint is https://smithi136:443 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:start UploadId=2~EkkJ9k7wIqXADOxRzp2M_yBshj8Qb_I 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 1 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 1 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 2} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.857 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.858 INFO:tasks.workunit.client.0.smithi136.stdout:phase 2 uploaded part {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1} 2024-04-22T20:04:27.858 INFO:tasks.workunit.client.0.smithi136.stdout:aborting multipart upload, parts=[{'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 1}, {'ETag': '"a080246ff6f1d9872e3cdf4879f132f9"', 'PartNumber': 2}] 2024-04-22T20:04:29.304 INFO:tasks.workunit.client.0.smithi136.stdout:radosgw-admin bucket radoslist: 2024-04-22T20:04:29.367 INFO:tasks.workunit.client.0.smithi136.stdout:rados ls: 2024-04-22T20:04:29.396 INFO:tasks.workunit.client.0.smithi136.stdout:== PASSED == 2024-04-22T20:04:29.396 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T20:04:29.396 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T20:04:29.472 INFO:tasks.workunit:Stopping ['rgw/test_rgw_s3_mp_reupload.sh'] on client.0... 2024-04-22T20:04:29.472 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T20:04:29.851 DEBUG:teuthology.parallel:result is None 2024-04-22T20:04:29.851 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:04:29.886 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:04:29.887 DEBUG:teuthology.orchestra.run.smithi136:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T20:04:29.955 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:04:29.955 INFO:teuthology.run_tasks:Running task ragweed... 2024-04-22T20:04:29.968 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}} 2024-04-22T20:04:29.968 INFO:tasks.ragweed:Downloading ragweed... 2024-04-22T20:04:29.968 INFO:tasks.ragweed:Using branch 'wip-65551' for ragweed 2024-04-22T20:04:29.969 DEBUG:teuthology.orchestra.run.smithi136:> git clone -b wip-65551 https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-04-22T20:04:30.022 INFO:teuthology.orchestra.run.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-04-22T20:04:30.067 INFO:teuthology.orchestra.run.smithi136.stderr:fatal: Remote branch wip-65551 not found in upstream origin 2024-04-22T20:04:30.067 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-04-22T20:04:30.069 INFO:tasks.ragweed:Using branch 'ceph-master' for ragweed 2024-04-22T20:04:30.069 DEBUG:teuthology.orchestra.run.smithi136:> git clone -b ceph-master https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-04-22T20:04:30.136 INFO:teuthology.orchestra.run.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-04-22T20:04:30.360 INFO:tasks.ragweed:Creating rgw users... 2024-04-22T20:04:30.361 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2024-04-22T20:04:30.361 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key NWEWLXEAWQBCKWKCBKIA --secret i9ra6MBZUXDoARhxSauJk0y5krRcoz6AT40TKMbZz/syQ8DnfmbRsw== --email ragweed.client.0+test@test.test --system false 2024-04-22T20:04:30.464 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:04:30.464 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:04:30.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.494+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.496+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.496+0000 7fb165baf800 20 realm 2024-04-22T20:04:30.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.496+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.496+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.496+0000 7fb165baf800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:04:30.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.496+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.496+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.496+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.497 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.497+0000 7fb165baf800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:04:30.497 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.497+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.497 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.497+0000 7fb165baf800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:04:30.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.498+0000 7fb165baf800 20 searching for the correct realm 2024-04-22T20:04:30.499 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.500+0000 7fb163eb6640 0 -- 172.21.15.136:0/1748924295 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fb15c092a60 msgr2=0x7fb15c060260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:04:30.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.706+0000 7fb15bfff640 0 -- 172.21.15.136:0/1748924295 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fb15c0897f0 msgr2=0x7fb15c043220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:04:30.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:04:30.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:04:30.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:04:30.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:04:30.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:04:30.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:04:30.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:04:30.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.913+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:04:30.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 20 zone default found 2024-04-22T20:04:30.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 4 Realm: () 2024-04-22T20:04:30.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:04:30.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 20 zonegroup default 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.914+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.915+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.915+0000 7fb165baf800 20 started sync module instance, tier type = 2024-04-22T20:04:30.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.915+0000 7fb165baf800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:04:30.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.924+0000 7fb165baf800 20 add_watcher() i=0 2024-04-22T20:04:30.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.924+0000 7fb165baf800 20 add_watcher() i=1 2024-04-22T20:04:30.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.924+0000 7fb165baf800 20 add_watcher() i=2 2024-04-22T20:04:30.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.924+0000 7fb165baf800 20 add_watcher() i=3 2024-04-22T20:04:30.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.924+0000 7fb165baf800 20 add_watcher() i=4 2024-04-22T20:04:30.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.924+0000 7fb165baf800 20 add_watcher() i=5 2024-04-22T20:04:30.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.924+0000 7fb165baf800 20 add_watcher() i=6 2024-04-22T20:04:30.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.925+0000 7fb165baf800 20 add_watcher() i=7 2024-04-22T20:04:30.926 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.925+0000 7fb165baf800 2 all 8 watchers are set, enabling cache 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.927+0000 7fb019fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.927+0000 7fb165baf800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.927+0000 7fb165baf800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.927+0000 7fb165baf800 5 note: GC not initialized 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb018fe1640 20 reqs_thread_entry: start 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 20 init_complete bucket index max shards: 1 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 20 Filter name: none 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7faf927fc640 20 reqs_thread_entry: start 2024-04-22T20:04:30.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-04-22T20:04:30.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.928+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 10 cache get: name=default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA : miss 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 20 rados->read ofs=0 len=0 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 10 cache put: name=default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA info.flags=0x0 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 10 adding default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA to cache LRU end 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 10 cache get: name=default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA : hit (negative entry) 2024-04-22T20:04:30.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.929+0000 7fb165baf800 10 cache get: name=default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA : hit (negative entry) 2024-04-22T20:04:30.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.930+0000 7fb165baf800 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-04-22T20:04:30.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.930+0000 7fb165baf800 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-04-22T20:04:30.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.930+0000 7fb165baf800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-04-22T20:04:30.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.931+0000 7fb046ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824065 cookie 94463279066944 notifier 5895 bl.length()=675 2024-04-22T20:04:30.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.931+0000 7fb046ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-04-22T20:04:30.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.931+0000 7fb046ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-04-22T20:04:30.934 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.934+0000 7fb165baf800 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-04-22T20:04:30.934 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.934+0000 7fb165baf800 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-04-22T20:04:30.934 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.934+0000 7fb165baf800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-04-22T20:04:30.934 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.934+0000 7fb046ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824013 cookie 94463279060080 notifier 5895 bl.length()=202 2024-04-22T20:04:30.934 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.934+0000 7fb046ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-04-22T20:04:30.934 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.934+0000 7fb046ffd640 10 rgw watcher librados: moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-04-22T20:04:30.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.937+0000 7fb165baf800 10 cache put: name=default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA info.flags=0x7 2024-04-22T20:04:30.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.937+0000 7fb165baf800 10 moving default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA to cache LRU end 2024-04-22T20:04:30.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.937+0000 7fb165baf800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NWEWLXEAWQBCKWKCBKIA, ofs0, ns] 2024-04-22T20:04:30.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.937+0000 7fb0467fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824010 cookie 94463279070496 notifier 5895 bl.length()=190 2024-04-22T20:04:30.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.937+0000 7fb0467fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA info.flags=0x7 2024-04-22T20:04:30.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.937+0000 7fb0467fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NWEWLXEAWQBCKWKCBKIA to cache LRU end 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "ragweed.client.0", 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr. ragweed.client.0", 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "ragweed.client.0+test@test.test", 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "ragweed.client.0", 2024-04-22T20:04:30.939 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "NWEWLXEAWQBCKWKCBKIA", 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "i9ra6MBZUXDoARhxSauJk0y5krRcoz6AT40TKMbZz/syQ8DnfmbRsw==", 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:04:30.930811Z" 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "rgw", 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "", 2024-04-22T20:04:30.941 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:04:30.942 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:04:30.930798Z", 2024-04-22T20:04:30.942 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:04:30.942 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:04:30.942 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:04:30.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:04:30.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.939+0000 7fb019fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:04:30.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.942+0000 7fb165baf800 20 remove_watcher() i=0 2024-04-22T20:04:30.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.942+0000 7fb165baf800 2 removed watcher, disabling cache 2024-04-22T20:04:30.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.944+0000 7fb165baf800 20 remove_watcher() i=1 2024-04-22T20:04:30.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.945+0000 7fb165baf800 20 remove_watcher() i=2 2024-04-22T20:04:30.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.946+0000 7fb165baf800 20 remove_watcher() i=3 2024-04-22T20:04:30.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.947+0000 7fb165baf800 20 remove_watcher() i=4 2024-04-22T20:04:30.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.948+0000 7fb165baf800 20 remove_watcher() i=5 2024-04-22T20:04:30.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.949+0000 7fb165baf800 20 remove_watcher() i=6 2024-04-22T20:04:30.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:30.950+0000 7fb165baf800 20 remove_watcher() i=7 2024-04-22T20:04:30.960 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2024-04-22T20:04:30.960 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key ZGJGGKNMIAGKWDMXZXCS --secret 1RQLAb4l2LqUDJyLqt2qjjbNh20afIL/9tiLjIx9wY90cge7ivSgLw== --email sysuser.client.0+test@test.test --system true 2024-04-22T20:04:31.023 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:04:31.024 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:04:31.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.055+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.056+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.056+0000 7fcf7440e800 20 realm 2024-04-22T20:04:31.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.056+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.056+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.056+0000 7fcf7440e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:04:31.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.056+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.056+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.056+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.057+0000 7fcf7440e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:04:31.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.057+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.058+0000 7fcf7440e800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:04:31.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.058+0000 7fcf7440e800 20 searching for the correct realm 2024-04-22T20:04:31.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.059+0000 7fcf7270e640 0 -- 172.21.15.136:0/1542028764 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55d4722585f0 msgr2=0x55d47225a9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:04:31.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.063+0000 7fcf7270e640 0 -- 172.21.15.136:0/1542028764 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7fce3c079e00 msgr2=0x7fce3c07c230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:04:31.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:04:31.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:04:31.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:04:31.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:04:31.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:04:31.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:04:31.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.069+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:04:31.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:04:31.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 20 zone default found 2024-04-22T20:04:31.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 4 Realm: () 2024-04-22T20:04:31.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:04:31.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:04:31.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:04:31.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 20 zonegroup default 2024-04-22T20:04:31.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.070+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.071+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.071+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.071+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.071+0000 7fcf7440e800 20 started sync module instance, tier type = 2024-04-22T20:04:31.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.071+0000 7fcf7440e800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:04:31.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.080+0000 7fcf7440e800 20 add_watcher() i=0 2024-04-22T20:04:31.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.080+0000 7fcf7440e800 20 add_watcher() i=1 2024-04-22T20:04:31.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.080+0000 7fcf7440e800 20 add_watcher() i=2 2024-04-22T20:04:31.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.080+0000 7fcf7440e800 20 add_watcher() i=3 2024-04-22T20:04:31.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.080+0000 7fcf7440e800 20 add_watcher() i=4 2024-04-22T20:04:31.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.080+0000 7fcf7440e800 20 add_watcher() i=5 2024-04-22T20:04:31.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.080+0000 7fcf7440e800 20 add_watcher() i=6 2024-04-22T20:04:31.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.081+0000 7fcf7440e800 20 add_watcher() i=7 2024-04-22T20:04:31.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.081+0000 7fcf7440e800 2 all 8 watchers are set, enabling cache 2024-04-22T20:04:31.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fce237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:04:31.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fcf7440e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:04:31.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fcf7440e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:04:31.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fcf7440e800 5 note: GC not initialized 2024-04-22T20:04:31.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fce227fc640 20 reqs_thread_entry: start 2024-04-22T20:04:31.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fcf7440e800 20 init_complete bucket index max shards: 1 2024-04-22T20:04:31.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fcf7440e800 20 Filter name: none 2024-04-22T20:04:31.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fcd93fff640 20 reqs_thread_entry: start 2024-04-22T20:04:31.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fcf7440e800 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-04-22T20:04:31.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.083+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2024-04-22T20:04:31.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-04-22T20:04:31.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2024-04-22T20:04:31.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2024-04-22T20:04:31.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-04-22T20:04:31.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 cache get: name=default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS : miss 2024-04-22T20:04:31.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 20 rados->read ofs=0 len=0 2024-04-22T20:04:31.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:04:31.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 cache put: name=default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS info.flags=0x0 2024-04-22T20:04:31.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 adding default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS to cache LRU end 2024-04-22T20:04:31.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 cache get: name=default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS : hit (negative entry) 2024-04-22T20:04:31.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.084+0000 7fcf7440e800 10 cache get: name=default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS : hit (negative entry) 2024-04-22T20:04:31.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.086+0000 7fcf7440e800 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-04-22T20:04:31.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.086+0000 7fcf7440e800 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-04-22T20:04:31.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.086+0000 7fcf7440e800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-04-22T20:04:31.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.086+0000 7fce557fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824014 cookie 94370936526176 notifier 5657 bl.length()=675 2024-04-22T20:04:31.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.086+0000 7fce557fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-04-22T20:04:31.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.086+0000 7fce557fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-04-22T20:04:31.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.089+0000 7fcf7440e800 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-04-22T20:04:31.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.089+0000 7fcf7440e800 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-04-22T20:04:31.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.089+0000 7fcf7440e800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-04-22T20:04:31.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.089+0000 7fce557fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824066 cookie 94370936531632 notifier 5657 bl.length()=202 2024-04-22T20:04:31.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.089+0000 7fce557fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-04-22T20:04:31.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.089+0000 7fce557fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-04-22T20:04:31.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.092+0000 7fcf7440e800 10 cache put: name=default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS info.flags=0x7 2024-04-22T20:04:31.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.092+0000 7fcf7440e800 10 moving default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS to cache LRU end 2024-04-22T20:04:31.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.092+0000 7fcf7440e800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:ZGJGGKNMIAGKWDMXZXCS, ofs0, ns] 2024-04-22T20:04:31.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.092+0000 7fce557fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1073741824062 cookie 94370936497312 notifier 5657 bl.length()=190 2024-04-22T20:04:31.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.092+0000 7fce557fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS info.flags=0x7 2024-04-22T20:04:31.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.092+0000 7fce557fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ZGJGGKNMIAGKWDMXZXCS to cache LRU end 2024-04-22T20:04:31.093 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:04:31.093 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "sysuser.client.0", 2024-04-22T20:04:31.093 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr. sysuser.client.0", 2024-04-22T20:04:31.093 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "sysuser.client.0+test@test.test", 2024-04-22T20:04:31.093 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:04:31.093 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "sysuser.client.0", 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "ZGJGGKNMIAGKWDMXZXCS", 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "1RQLAb4l2LqUDJyLqt2qjjbNh20afIL/9tiLjIx9wY90cge7ivSgLw==", 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:04:31.086277Z" 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:04:31.094 INFO:teuthology.orchestra.run.smithi136.stdout: "system": true, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:04:31.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "rgw", 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "", 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:04:31.086263Z", 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:04:31.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:04:31.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.094+0000 7fce237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:04:31.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.096+0000 7fcf7440e800 20 remove_watcher() i=0 2024-04-22T20:04:31.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.096+0000 7fcf7440e800 2 removed watcher, disabling cache 2024-04-22T20:04:31.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.097+0000 7fcf7440e800 20 remove_watcher() i=1 2024-04-22T20:04:31.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.098+0000 7fcf7440e800 20 remove_watcher() i=2 2024-04-22T20:04:31.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.099+0000 7fcf7440e800 20 remove_watcher() i=3 2024-04-22T20:04:31.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.100+0000 7fcf7440e800 20 remove_watcher() i=4 2024-04-22T20:04:31.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.102+0000 7fcf7440e800 20 remove_watcher() i=5 2024-04-22T20:04:31.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.103+0000 7fcf7440e800 20 remove_watcher() i=6 2024-04-22T20:04:31.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:04:31.103+0000 7fcf7440e800 20 remove_watcher() i=7 2024-04-22T20:04:31.114 INFO:tasks.ragweed:Configuring ragweed... 2024-04-22T20:04:31.114 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T20:04:31.115 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/archive/ragweed.client.0.conf 2024-04-22T20:04:31.141 INFO:tasks.ragweed:Configuring boto... 2024-04-22T20:04:31.141 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T20:04:31.142 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/boto.cfg 2024-04-22T20:04:31.209 DEBUG:teuthology.orchestra.run.smithi136:ragweed tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw' 2024-04-22T20:04:31.393 INFO:teuthology.orchestra.run.smithi136.stdout:GLOB sdist-make: /home/ubuntu/cephtest/ragweed.client.0/setup.py 2024-04-22T20:04:31.568 INFO:teuthology.orchestra.run.smithi136.stdout:py create: /home/ubuntu/cephtest/ragweed.client.0/.tox/py 2024-04-22T20:04:32.060 INFO:teuthology.orchestra.run.smithi136.stdout:py installdeps: -rrequirements.txt 2024-04-22T20:04:39.307 INFO:teuthology.orchestra.run.smithi136.stdout:py inst: /home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip 2024-04-22T20:04:41.439 INFO:teuthology.orchestra.run.smithi136.stdout:py installed: asyncssh==2.13.2,Babel==2.9.1,bcrypt==3.2.2,boto==2.49.0,cachetools==4.2.4,ceph==1.0.0,ceph-volume==1.0.0,cephfs==2.0.0,certifi==2021.10.8,cffi==1.14.5,chardet==4.0.0,cheroot==8.6.0,CherryPy==18.6.1,cockpit @ file:///builddir/build/BUILD/cockpit-314/tmp/wheel/cockpit-314-py3-none-any.whl,cryptography==36.0.1,dasbus==1.4,dbus-python==1.2.18,distro==1.5.0,docopt==0.6.2,exceptiongroup==1.2.1,execnet==2.1.1,file-magic==0.4.0,gevent==24.2.1,google-auth==2.28.2,gpg==1.15.1,greenlet==3.0.3,grpcio==1.46.7,grpcio-tools==1.46.7,httplib2==0.22.0,idna==2.10,iniconfig==2.0.0,isodate==0.6.1,jaraco.classes==3.2.1,jaraco.collections==3.0.0,jaraco.functools==3.5.0,jaraco.text==3.2.0,Jinja2==2.11.3,jsonpatch==1.21,jsonpointer==2.0,kubernetes==26.1.0,libcomps==0.1.18,logutils==0.3.5,lxml==4.6.5,Mako==1.1.4.dev0,MarkupSafe==1.1.1,more-itertools==8.12.0,munch==4.0.0,natsort==7.1.1,nftables==0.1,numpy==1.20.1,oauthlib==3.1.1,packaging==20.9,pecan==1.4.2,perf==0.1,pexpect==4.8.0,pluggy==1.5.0,ply==3.11,portend==3.1.0,prettytable==0.7.2,protobuf==3.14.0,psutil==5.8.0,ptyprocess==0.6.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pycparser==2.20,PyGObject==3.40.1,pyOpenSSL==21.0.0,pyparsing==2.4.7,PySocks==1.7.1,pytest==8.1.1,python-dateutil==2.8.1,python-linux-procfs==0.7.3,pytz==2021.1,pyudev==0.22.0,PyYAML==5.4.1,rados==2.0.0,ragweed @ file:///home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip#sha256=4a8820ad437621b7ca57ebcd5eeabea4537ea7a21911811f568bac17353c3d2b,rbd==2.0.0,registries-conf-ctl==1.0,remoto==1.2.1,repoze.lru==0.7,requests==2.25.1,requests-oauthlib==1.3.0,rgw==2.0.0,Routes==2.5.1,rpm==4.16.1.3,rsa==4.9,s3cmd==2.4.0,scipy==1.6.2,selinux==3.6,sepolicy==3.6,setools==4.4.4,setroubleshoot @ file:///builddir/build/BUILD/setroubleshoot-3.3.32/src,six==1.15.0,systemd-python==234,tempora==5.0.0,toml==0.10.2,tomli==2.0.1,typing_extensions==4.4.0,urllib3==1.26.5,WebOb==1.8.7,websocket-client==1.2.3,Werkzeug==2.0.3,zc.lockfile==2.0,zope.event==5.0,zope.interface==6.3 2024-04-22T20:04:41.439 INFO:teuthology.orchestra.run.smithi136.stdout:py run-test-pre: PYTHONHASHSEED='2286725132' 2024-04-22T20:04:41.440 INFO:teuthology.orchestra.run.smithi136.stdout:py run-test: commands[0] | python -m pytest -v -m 'not fails_on_rgw' 2024-04-22T20:04:41.638 INFO:teuthology.orchestra.run.smithi136.stdout:============================= test session starts ============================== 2024-04-22T20:04:41.639 INFO:teuthology.orchestra.run.smithi136.stdout:platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.5.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python 2024-04-22T20:04:41.639 INFO:teuthology.orchestra.run.smithi136.stdout:cachedir: .tox/py/.pytest_cache 2024-04-22T20:04:41.639 INFO:teuthology.orchestra.run.smithi136.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0 2024-04-22T20:04:41.639 INFO:teuthology.orchestra.run.smithi136.stdout:configfile: pytest.ini 2024-04-22T20:04:41.736 INFO:teuthology.orchestra.run.smithi136.stdout:collecting ... collected 9 items 2024-04-22T20:04:41.736 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:04:47.980 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_small_obj_data::test PASSED [ 11%] 2024-04-22T20:04:54.709 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_multipart_simple::test PASSED [ 22%] 2024-04-22T20:04:59.256 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED [ 33%] 2024-04-22T20:05:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED [ 44%] 2024-04-22T20:05:12.159 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED [ 55%] 2024-04-22T20:05:26.817 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED [ 66%] 2024-04-22T20:05:38.837 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED [ 77%] 2024-04-22T20:05:52.119 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED [ 88%] 2024-04-22T20:06:18.831 INFO:teuthology.orchestra.run.smithi136.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED [100%] 2024-04-22T20:06:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:06:18.831 INFO:teuthology.orchestra.run.smithi136.stdout:=============================== warnings summary =============================== 2024-04-22T20:06:18.831 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/boto/plugin.py:40 2024-04-22T20:06:18.831 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/ragweed.client.0/.tox/py/lib/python3.9/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses 2024-04-22T20:06:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: import imp 2024-04-22T20:06:18.832 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:06:18.832 INFO:teuthology.orchestra.run.smithi136.stdout:-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html 2024-04-22T20:06:18.832 INFO:teuthology.orchestra.run.smithi136.stdout:=================== 9 passed, 1 warning in 97.19s (0:01:37) ==================== 2024-04-22T20:06:18.865 INFO:teuthology.orchestra.run.smithi136.stdout:___________________________________ summary ____________________________________ 2024-04-22T20:06:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: py: commands succeeded 2024-04-22T20:06:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: congratulations :) 2024-04-22T20:06:18.879 INFO:tasks.ragweed:Cleaning up boto... 2024-04-22T20:06:18.879 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/boto.cfg 2024-04-22T20:06:18.907 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2024-04-22T20:06:19.001 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:06:19.001 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:06:19.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.033+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.034+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.034+0000 7fbb46d2d800 20 realm 2024-04-22T20:06:19.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.034+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.034+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.034+0000 7fbb46d2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:06:19.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.034+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.035+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.035+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.035+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:06:19.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.035+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.036+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:06:19.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.036+0000 7fbb46d2d800 20 searching for the correct realm 2024-04-22T20:06:19.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.038+0000 7fbb431af640 0 -- 172.21.15.136:0/713730586 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fbb3c033040 msgr2=0x7fbb3c033420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:06:19.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.244+0000 7fbb3bfff640 0 -- 172.21.15.136:0/713730586 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fbb3c033b80 msgr2=0x7fbb3c055900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:06:19.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.450+0000 7fbb46d2d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:06:19.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.450+0000 7fbb46d2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:06:19.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.450+0000 7fbb46d2d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:06:19.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.450+0000 7fbb46d2d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:06:19.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.450+0000 7fbb46d2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:06:19.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.450+0000 7fbb46d2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:06:19.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.450+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:06:19.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:06:19.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 20 zone default found 2024-04-22T20:06:19.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 4 Realm: () 2024-04-22T20:06:19.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:06:19.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:06:19.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:06:19.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 20 zonegroup default 2024-04-22T20:06:19.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.451+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.452+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.452+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.452+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.452+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.452+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.452+0000 7fbb46d2d800 20 started sync module instance, tier type = 2024-04-22T20:06:19.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.452+0000 7fbb46d2d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:06:19.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.462+0000 7fbb46d2d800 20 add_watcher() i=0 2024-04-22T20:06:19.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.462+0000 7fbb46d2d800 20 add_watcher() i=1 2024-04-22T20:06:19.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.462+0000 7fbb46d2d800 20 add_watcher() i=2 2024-04-22T20:06:19.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.462+0000 7fbb46d2d800 20 add_watcher() i=3 2024-04-22T20:06:19.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.462+0000 7fbb46d2d800 20 add_watcher() i=4 2024-04-22T20:06:19.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.462+0000 7fbb46d2d800 20 add_watcher() i=5 2024-04-22T20:06:19.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.462+0000 7fbb46d2d800 20 add_watcher() i=6 2024-04-22T20:06:19.463 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.463+0000 7fbb46d2d800 20 add_watcher() i=7 2024-04-22T20:06:19.463 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.463+0000 7fbb46d2d800 2 all 8 watchers are set, enabling cache 2024-04-22T20:06:19.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.465+0000 7fb9f97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:06:19.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.465+0000 7fbb46d2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:06:19.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.465+0000 7fbb46d2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:06:19.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.503+0000 7fb973fff640 20 reqs_thread_entry: start 2024-04-22T20:06:19.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.503+0000 7fbb46d2d800 20 init_complete bucket index max shards: 1 2024-04-22T20:06:19.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.503+0000 7fbb46d2d800 20 Filter name: none 2024-04-22T20:06:19.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.504+0000 7fb971ffb640 20 reqs_thread_entry: start 2024-04-22T20:06:19.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.504+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-04-22T20:06:19.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.504+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.504 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.504+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=486 2024-04-22T20:06:19.504 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.504+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x13 2024-04-22T20:06:19.504 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.504+0000 7fbb46d2d800 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-04-22T20:06:19.504 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.504+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.504 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.505+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4 : miss 2024-04-22T20:06:19.504 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.505+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=418 2024-04-22T20:06:19.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4 info.flags=0x17 2024-04-22T20:06:19.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4 to cache LRU end 2024-04-22T20:06:19.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_multipart_simple 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got tests/r_test_small_obj_data 2024-04-22T20:06:19.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.506+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x560c83d24858 s->prefetch_data=0 2024-04-22T20:06:19.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.507+0000 7fbb46d2d800 10 manifest: total_size = 5191 2024-04-22T20:06:19.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.507+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7868511335926160054 2024-04-22T20:06:19.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.507+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x560c83d24858 s->prefetch_data=0 2024-04-22T20:06:19.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.507+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83d24858 2024-04-22T20:06:19.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.507+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.509+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.509+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5191 stripe_ofs=5191 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.509+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7868511335926160054 2024-04-22T20:06:19.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.509+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.509+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.2e1ZY4wuTILZpJrEndGeJvP1BW4ampx_0 2024-04-22T20:06:19.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.509+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.509+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.509+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7868511335926160054 2024-04-22T20:06:19.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.511+0000 7fba267fc640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_complete 2024-04-22T20:06:19.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.511+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x560c83ca9fc8 s->prefetch_data=0 2024-04-22T20:06:19.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.511+0000 7fbb46d2d800 10 manifest: total_size = 5202 2024-04-22T20:06:19.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.511+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15723389894443242038 2024-04-22T20:06:19.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.511+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x560c83ca9fc8 s->prefetch_data=0 2024-04-22T20:06:19.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.511+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83ca9fc8 2024-04-22T20:06:19.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.511+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.514+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.514+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5202 stripe_ofs=5202 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.514+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15723389894443242038 2024-04-22T20:06:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.514+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.514+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.m7uCOIrMFkCmPl-gNeqU_IroBn2oDiK_0 2024-04-22T20:06:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.514+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-22T20:06:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.514+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.514+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15723389894443242038 2024-04-22T20:06:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.515+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_update_complete 2024-04-22T20:06:19.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.515+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.515+0000 7fbb46d2d800 10 manifest: total_size = 7407 2024-04-22T20:06:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.515+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5288031347509135434 2024-04-22T20:06:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.515+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.515+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cafc38 2024-04-22T20:06:19.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.515+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.518+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.518+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=7407 stripe_ofs=7407 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.518+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5288031347509135434 2024-04-22T20:06:19.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.518+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.518+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.QcMBS1inwWBH_6hy7pxjBEBwZ-dPZE4_0 2024-04-22T20:06:19.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.518+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.518+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.518+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5288031347509135434 2024-04-22T20:06:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.519+0000 7fba267fc640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_index_versioning 2024-04-22T20:06:19.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.519+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.520+0000 7fbb46d2d800 10 manifest: total_size = 5082 2024-04-22T20:06:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.520+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1941564247898136352 2024-04-22T20:06:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.520+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.520+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cafc38 2024-04-22T20:06:19.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.520+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.522+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.522+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5082 stripe_ofs=5082 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.522+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1941564247898136352 2024-04-22T20:06:19.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.522+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.522+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.NdmkIMb4RQ0j7NZZbGdF8pEJCwgh99A_0 2024-04-22T20:06:19.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.522+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.522+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.522+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1941564247898136352 2024-04-22T20:06:19.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.524+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_simple 2024-04-22T20:06:19.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.524+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.524+0000 7fbb46d2d800 10 manifest: total_size = 7538 2024-04-22T20:06:19.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.524+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9061042827645264942 2024-04-22T20:06:19.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.524+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.524+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cafc38 2024-04-22T20:06:19.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.524+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.527+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.527+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=7538 stripe_ofs=7538 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.527+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9061042827645264942 2024-04-22T20:06:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.527+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.527+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.LnhnG9lxQ8Y_3PRp9qYtH0w_7V8TIsO_0 2024-04-22T20:06:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.527+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.527+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.527+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9061042827645264942 2024-04-22T20:06:19.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.528+0000 7fba267fc640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class 2024-04-22T20:06:19.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.528+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.528+0000 7fbb46d2d800 10 manifest: total_size = 5235 2024-04-22T20:06:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.528+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3379786376601497935 2024-04-22T20:06:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.528+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.528+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cafc38 2024-04-22T20:06:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.528+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.530 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.531+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.530 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.531+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5235 stripe_ofs=5235 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.530 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.531+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3379786376601497935 2024-04-22T20:06:19.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.531+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.531+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.f5JvrGljUf27eqP98Og8vVifu89mhse_0 2024-04-22T20:06:19.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.531+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.531+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.531+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3379786376601497935 2024-04-22T20:06:19.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.533+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.533+0000 7fbb46d2d800 10 manifest: total_size = 5367 2024-04-22T20:06:19.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.533+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15625124244436283087 2024-04-22T20:06:19.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.533+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.533+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cafc38 2024-04-22T20:06:19.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.533+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.534+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_copy 2024-04-22T20:06:19.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.535+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.535+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5367 stripe_ofs=5367 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.535+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15625124244436283087 2024-04-22T20:06:19.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.535+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.535+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.rivaAMaP8BOaOkle9zHSr1qEnqPQsbg_0 2024-04-22T20:06:19.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.535+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-22T20:06:19.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.535+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.535+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15625124244436283087 2024-04-22T20:06:19.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.537+0000 7fba267fc640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart 2024-04-22T20:06:19.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.537+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.537+0000 7fbb46d2d800 10 manifest: total_size = 5280 2024-04-22T20:06:19.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.537+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.941870224416043255 2024-04-22T20:06:19.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.537+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.537+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cafc38 2024-04-22T20:06:19.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.537+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.540+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.540+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5280 stripe_ofs=5280 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.540+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.941870224416043255 2024-04-22T20:06:19.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.540+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.540+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.cDInpDvqWPWqzxV6dzfd-HYizr-LM4D_0 2024-04-22T20:06:19.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.540+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 327 2024-04-22T20:06:19.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.540+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.540+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.941870224416043255 2024-04-22T20:06:19.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.541+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart_copy 2024-04-22T20:06:19.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.541+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.541+0000 7fbb46d2d800 10 manifest: total_size = 5385 2024-04-22T20:06:19.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.541+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5167284803758613191 2024-04-22T20:06:19.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.541+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c027b20 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x560c83cafc38 s->prefetch_data=0 2024-04-22T20:06:19.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.541+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cafc38 2024-04-22T20:06:19.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.541+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4.0 2024-04-22T20:06:19.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.544+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.544+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5385 stripe_ofs=5385 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.544+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5167284803758613191 2024-04-22T20:06:19.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.544+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.544+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.SYRR4aq-6lZ-bpcY8Yd2vOy7RkWbzex_0 2024-04-22T20:06:19.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.544+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.544+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.544+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5167284803758613191 2024-04-22T20:06:19.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.545+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.545+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.545+0000 7fba267fc640 20 handle_completion(): completion ok for obj=tests/r_test_small_obj_data 2024-04-22T20:06:19.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.546+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.546+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.546+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.546+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.546+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.547+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2024-04-22T20:06:19.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.547+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.547+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=204 2024-04-22T20:06:19.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.547+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x11 2024-04-22T20:06:19.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.547+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2024-04-22T20:06:19.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.548+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-conf from cache 2024-04-22T20:06:19.548 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.548+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-conf, ofs0, ns] 2024-04-22T20:06:19.548 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.549+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562439 cookie 94611750410480 notifier 5952 bl.length()=162 2024-04-22T20:06:19.550 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.551+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4 from cache 2024-04-22T20:06:19.550 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.551+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-conf:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4, ofs0, ns] 2024-04-22T20:06:19.550 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.551+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562374 cookie 94611750432432 notifier 5952 bl.length()=219 2024-04-22T20:06:19.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.554+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-conf/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4 : miss 2024-04-22T20:06:19.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.554+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.554+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.554+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-conf/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4 info.flags=0x0 2024-04-22T20:06:19.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.554+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-conf/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4 to cache LRU end 2024-04-22T20:06:19.555 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.556+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11 : miss 2024-04-22T20:06:19.555 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.556+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.556+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=449 2024-04-22T20:06:19.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.556+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11 info.flags=0x17 2024-04-22T20:06:19.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.556+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11 to cache LRU end 2024-04-22T20:06:19.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.556+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.556+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.557+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11 2024-04-22T20:06:19.557 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.557+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo 2024-04-22T20:06:19.557 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.557+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83ca9b30 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x560c83cb3618 s->prefetch_data=0 2024-04-22T20:06:19.557 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.557+0000 7fbb46d2d800 10 manifest: total_size = 1667 2024-04-22T20:06:19.557 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.557+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6960362847265545089 2024-04-22T20:06:19.557 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.557+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83ca9b30 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x560c83cb3618 s->prefetch_data=0 2024-04-22T20:06:19.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.557+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb3618 2024-04-22T20:06:19.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.557+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11.0 2024-04-22T20:06:19.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-04-22T20:06:19.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=516 2024-04-22T20:06:19.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194820 part_ofs=516 rule->part_size=516 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4195336 part_ofs=1032 rule->part_size=516 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4195852 part_ofs=1548 rule->part_size=119 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6960362847265545089 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11__multipart_foo.2~lRRu1mPsTn6E7sRY658QyGFcN5djD_9.1 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11__multipart_foo.2~lRRu1mPsTn6E7sRY658QyGFcN5djD_9.2 2024-04-22T20:06:19.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-04-22T20:06:19.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11__multipart_foo.2~lRRu1mPsTn6E7sRY658QyGFcN5djD_9.3 2024-04-22T20:06:19.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-04-22T20:06:19.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11__multipart_foo.2~lRRu1mPsTn6E7sRY658QyGFcN5djD_9.4 2024-04-22T20:06:19.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-04-22T20:06:19.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.560+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6960362847265545089 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.561+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.561+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.561+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.562+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.562+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.562+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.562+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.562+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.563+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2024-04-22T20:06:19.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.563+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.564+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=235 2024-04-22T20:06:19.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.564+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x11 2024-04-22T20:06:19.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.564+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2024-04-22T20:06:19.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.565+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2024-04-22T20:06:19.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.565+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1, ofs0, ns] 2024-04-22T20:06:19.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.565+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562435 cookie 94611750415504 notifier 5952 bl.length()=191 2024-04-22T20:06:19.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.568+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11 from cache 2024-04-22T20:06:19.567 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.568+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11, ofs0, ns] 2024-04-22T20:06:19.567 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.568+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562375 cookie 94611750432432 notifier 5952 bl.length()=249 2024-04-22T20:06:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.570+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11 : miss 2024-04-22T20:06:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.570+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.570+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.570+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11 info.flags=0x0 2024-04-22T20:06:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.570+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11 to cache LRU end 2024-04-22T20:06:19.572 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.573+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12 : miss 2024-04-22T20:06:19.572 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.573+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.573+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=456 2024-04-22T20:06:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.573+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12 info.flags=0x17 2024-04-22T20:06:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.573+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12 to cache LRU end 2024-04-22T20:06:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.573+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.573+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.573+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12 2024-04-22T20:06:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.574+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo 2024-04-22T20:06:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.574+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x560c83cb3b28 s->prefetch_data=0 2024-04-22T20:06:19.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.574+0000 7fbb46d2d800 10 manifest: total_size = 886297 2024-04-22T20:06:19.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.574+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1781355800613501990 2024-04-22T20:06:19.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.574+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x560c83cb3b28 s->prefetch_data=0 2024-04-22T20:06:19.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.574+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb3b28 2024-04-22T20:06:19.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.574+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12.0 2024-04-22T20:06:19.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-22T20:06:19.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2024-04-22T20:06:19.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2024-04-22T20:06:19.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=246247 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4933045 part_ofs=738741 rule->part_size=147556 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1781355800613501990 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12__multipart_foo.2~-LTG0TDmGoM7WCJ_xnQAU3_VmkBqwCg.1 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12__multipart_foo.2~-LTG0TDmGoM7WCJ_xnQAU3_VmkBqwCg.2 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12__multipart_foo.2~-LTG0TDmGoM7WCJ_xnQAU3_VmkBqwCg.3 2024-04-22T20:06:19.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-04-22T20:06:19.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12__multipart_foo.2~-LTG0TDmGoM7WCJ_xnQAU3_VmkBqwCg.4 2024-04-22T20:06:19.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-04-22T20:06:19.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.577+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1781355800613501990 2024-04-22T20:06:19.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.578+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo 2024-04-22T20:06:19.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.579+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.579+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.579+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.579+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.579+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.579+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.579+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.580+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2024-04-22T20:06:19.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.580+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.581+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=242 2024-04-22T20:06:19.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.581+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x11 2024-04-22T20:06:19.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.581+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2024-04-22T20:06:19.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.582+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2024-04-22T20:06:19.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.582+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1, ofs0, ns] 2024-04-22T20:06:19.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.582+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562393 cookie 94611750419312 notifier 5952 bl.length()=198 2024-04-22T20:06:19.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.585+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12 from cache 2024-04-22T20:06:19.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.585+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12, ofs0, ns] 2024-04-22T20:06:19.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.586+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562440 cookie 94611750425664 notifier 5952 bl.length()=256 2024-04-22T20:06:19.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.588+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12 : miss 2024-04-22T20:06:19.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.588+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.588+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.588+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12 info.flags=0x0 2024-04-22T20:06:19.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.588+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12 to cache LRU end 2024-04-22T20:06:19.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.591+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10 : miss 2024-04-22T20:06:19.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.591+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.591+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=451 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.591+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10 info.flags=0x17 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.591+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10 to cache LRU end 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.591+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.591+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.591+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.592+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo[IbGqtxhW2ZSrh76tavKhWP9pXGpRhL6] 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.592+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10.0 2024-04-22T20:06:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.592+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34003f30 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.593+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 04rukyq1oj86g0f9vfwpmuhjqgx2lyy0 2024-04-22T20:06:19.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.593+0000 7fbb46d2d800 20 get_obj_state_impl: setting s->olh_tag to 6qwwc7qk382uux4iq1bzbg3c309rixx6 2024-04-22T20:06:19.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.593+0000 7fbb46d2d800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=6qwwc7qk382uux4iq1bzbg3c309rixx6 2024-04-22T20:06:19.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.594+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10.0 2024-04-22T20:06:19.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.595+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10.0 2024-04-22T20:06:19.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.595+0000 7fbb46d2d800 20 olh_log_entry: epoch=3 op=2 key=foo[] 2024-04-22T20:06:19.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.595+0000 7fbb46d2d800 20 olh_log_entry: epoch=3 op=3 key=foo[IbGqtxhW2ZSrh76tavKhWP9pXGpRhL6] 2024-04-22T20:06:19.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.595+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34003f30 obj=test-client.0-r-test-multipart-index-versioning.1:_:IbGqtxhW2ZSrh76tavKhWP9pXGpRhL6_foo state=0x560c83cc0aa8 s->prefetch_data=0 2024-04-22T20:06:19.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.596+0000 7fbb46d2d800 10 manifest: total_size = 18874368 2024-04-22T20:06:19.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.596+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13745958173434414860 2024-04-22T20:06:19.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.596+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34003f30 obj=test-client.0-r-test-multipart-index-versioning.1:_:IbGqtxhW2ZSrh76tavKhWP9pXGpRhL6_foo state=0x560c83cc0aa8 s->prefetch_data=0 2024-04-22T20:06:19.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.596+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc0aa8 2024-04-22T20:06:19.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.596+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10.0 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=5242880 2024-04-22T20:06:19.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=14680064 stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=18874368 part_ofs=10485760 rule->part_size=5242880 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=15728640 stripe_ofs=15728640 part_ofs=15728640 rule->part_size=3145728 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=3145728 rules.size()=2 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=19922944 part_ofs=15728640 rule->part_size=3145728 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=18874368 stripe_ofs=18874368 part_ofs=18874368 rule->part_size=3145728 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13745958173434414860 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__multipart_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.1 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__shadow_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.1_1 2024-04-22T20:06:19.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 591 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__multipart_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.2 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 837 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__shadow_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.2_1 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1081 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__multipart_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.3 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1327 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__shadow_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.3_1 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1571 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__multipart_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.4 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1817 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.598+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13745958173434414860 2024-04-22T20:06:19.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.599+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo[IbGqtxhW2ZSrh76tavKhWP9pXGpRhL6] 2024-04-22T20:06:19.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.601+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34003f30 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.602+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10.0 2024-04-22T20:06:19.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.603+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10.0 2024-04-22T20:06:19.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.603+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.603+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.603+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.603+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.603+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.603+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.603+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.605+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2024-04-22T20:06:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.605+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.605+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=237 2024-04-22T20:06:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.605+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x11 2024-04-22T20:06:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.605+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2024-04-22T20:06:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.606+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2024-04-22T20:06:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.606+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1, ofs0, ns] 2024-04-22T20:06:19.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.606+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562394 cookie 94611750419312 notifier 5952 bl.length()=193 2024-04-22T20:06:19.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.609+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10 from cache 2024-04-22T20:06:19.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.609+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10, ofs0, ns] 2024-04-22T20:06:19.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.609+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562395 cookie 94611750419312 notifier 5952 bl.length()=251 2024-04-22T20:06:19.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.611+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10 : miss 2024-04-22T20:06:19.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.611+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.611+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.611+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10 info.flags=0x0 2024-04-22T20:06:19.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.611+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10 to cache LRU end 2024-04-22T20:06:19.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.613+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9 : miss 2024-04-22T20:06:19.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.613+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.614+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=439 2024-04-22T20:06:19.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.614+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9 info.flags=0x17 2024-04-22T20:06:19.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.614+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9 to cache LRU end 2024-04-22T20:06:19.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.614+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.614+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.614+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9 2024-04-22T20:06:19.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.614+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo 2024-04-22T20:06:19.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.614+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf520 obj=test-client.0-r-test-multipart-simple.1:foo state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.615+0000 7fbb46d2d800 10 manifest: total_size = 886297 2024-04-22T20:06:19.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.615+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3546291817057341720 2024-04-22T20:06:19.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.615+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf520 obj=test-client.0-r-test-multipart-simple.1:foo state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.615+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.615+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9.0 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=246247 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2024-04-22T20:06:19.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4933045 part_ofs=738741 rule->part_size=147556 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3546291817057341720 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9__multipart_foo.2~JKgiZvhRhypZ9CtQWmdlGR8okhMlwqF.1 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 344 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9__multipart_foo.2~JKgiZvhRhypZ9CtQWmdlGR8okhMlwqF.2 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 588 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9__multipart_foo.2~JKgiZvhRhypZ9CtQWmdlGR8okhMlwqF.3 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 832 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9__multipart_foo.2~JKgiZvhRhypZ9CtQWmdlGR8okhMlwqF.4 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1076 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.617+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3546291817057341720 2024-04-22T20:06:19.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.618+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo 2024-04-22T20:06:19.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.619+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.619+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.619+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.619+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.619+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.619+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.619+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.620+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2024-04-22T20:06:19.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.620+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.621+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=225 2024-04-22T20:06:19.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.621+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x11 2024-04-22T20:06:19.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.621+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2024-04-22T20:06:19.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.622+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2024-04-22T20:06:19.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.622+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-simple.1, ofs0, ns] 2024-04-22T20:06:19.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.622+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562441 cookie 94611750425664 notifier 5952 bl.length()=183 2024-04-22T20:06:19.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.625+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9 from cache 2024-04-22T20:06:19.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.625+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9, ofs0, ns] 2024-04-22T20:06:19.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.625+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562383 cookie 94611750428320 notifier 5952 bl.length()=240 2024-04-22T20:06:19.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.628+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9 : miss 2024-04-22T20:06:19.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.628+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.628+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.628+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9 info.flags=0x0 2024-04-22T20:06:19.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.628+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9 to cache LRU end 2024-04-22T20:06:19.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.630+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15 : miss 2024-04-22T20:06:19.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.630+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=447 2024-04-22T20:06:19.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15 info.flags=0x17 2024-04-22T20:06:19.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15 to cache LRU end 2024-04-22T20:06:19.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD 2024-04-22T20:06:19.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-04-22T20:06:19.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-04-22T20:06:19.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-04-22T20:06:19.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.631+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf520 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.632+0000 7fbb46d2d800 10 manifest: total_size = 5242880 2024-04-22T20:06:19.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.632+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14254305980892923405 2024-04-22T20:06:19.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.632+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf520 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.632+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.632+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14254305980892923405 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_0 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 575 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.634+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14254305980892923405 2024-04-22T20:06:19.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.635+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-04-22T20:06:19.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34097a80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 10 manifest: total_size = 5242880 2024-04-22T20:06:19.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 10 generate_fake_tag new tag=19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1_ff389ce5235714e61b9ab067ddc64c64 2024-04-22T20:06:19.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1_ff389ce5235714e61b9ab067ddc64c64 2024-04-22T20:06:19.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34097a80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.636+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1_ff389ce5235714e61b9ab067ddc64c64 2024-04-22T20:06:19.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_0 2024-04-22T20:06:19.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 396 2024-04-22T20:06:19.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1 2024-04-22T20:06:19.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 633 2024-04-22T20:06:19.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.638+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1_ff389ce5235714e61b9ab067ddc64c64 2024-04-22T20:06:19.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.639+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-04-22T20:06:19.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.639+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34003f30 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.640+0000 7fbb46d2d800 10 manifest: total_size = 5242880 2024-04-22T20:06:19.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.640+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _ZK4INnDQKVXSiHlgtMHLmJnKJsQBNZN 2024-04-22T20:06:19.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.640+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34003f30 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.640+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.640+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _ZK4INnDQKVXSiHlgtMHLmJnKJsQBNZN 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.A_dg0zF3R_4Vkdi09XtyLIYFFoOpt4t_0 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.A_dg0zF3R_4Vkdi09XtyLIYFFoOpt4t_1 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.642+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _ZK4INnDQKVXSiHlgtMHLmJnKJsQBNZN 2024-04-22T20:06:19.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.643+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-04-22T20:06:19.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.643+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c00ba60 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.644+0000 7fbb46d2d800 10 manifest: total_size = 5242880 2024-04-22T20:06:19.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.644+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _eXtGZCCfrb-zKwh_gM2lUzoSpbvaUS0 2024-04-22T20:06:19.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.644+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb3c00ba60 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.644+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.644+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _eXtGZCCfrb-zKwh_gM2lUzoSpbvaUS0 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.X9ye873tkmd1aAm4_K9pS1FBSzTsRTz_0 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.X9ye873tkmd1aAm4_K9pS1FBSzTsRTz_1 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.646+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: _eXtGZCCfrb-zKwh_gM2lUzoSpbvaUS0 2024-04-22T20:06:19.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.647+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-04-22T20:06:19.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.648+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.648+0000 7fbb46d2d800 10 manifest: total_size = 5242880 2024-04-22T20:06:19.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.648+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2269165753410918401 2024-04-22T20:06:19.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.648+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.648+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.648+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2269165753410918401 2024-04-22T20:06:19.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_0 2024-04-22T20:06:19.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 339 2024-04-22T20:06:19.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1 2024-04-22T20:06:19.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 578 2024-04-22T20:06:19.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.650+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2269165753410918401 2024-04-22T20:06:19.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.651+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-04-22T20:06:19.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.651+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.652+0000 7fbb46d2d800 10 manifest: total_size = 5242880 2024-04-22T20:06:19.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.652+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _Rpx47fHzqjW40yD-UQjGSmuqb_Vkc7l 2024-04-22T20:06:19.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.652+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.652+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.652+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _Rpx47fHzqjW40yD-UQjGSmuqb_Vkc7l 2024-04-22T20:06:19.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.QfMAtLs_3laUIorPAVnrMFfVe0UNULP_0 2024-04-22T20:06:19.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-04-22T20:06:19.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.QfMAtLs_3laUIorPAVnrMFfVe0UNULP_1 2024-04-22T20:06:19.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-04-22T20:06:19.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.654+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: _Rpx47fHzqjW40yD-UQjGSmuqb_Vkc7l 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 10 manifest: total_size = 5242880 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 10 generate_fake_tag new tag=19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1_bbab19ceef56354f88e20599ba4eb0c4 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1_bbab19ceef56354f88e20599ba4eb0c4 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.655+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1_bbab19ceef56354f88e20599ba4eb0c4 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_0 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 398 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 637 2024-04-22T20:06:19.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.658+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1_bbab19ceef56354f88e20599ba4eb0c4 2024-04-22T20:06:19.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.659+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-04-22T20:06:19.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.659+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.659+0000 7fbb46d2d800 10 manifest: total_size = 5242880 2024-04-22T20:06:19.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.659+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _6ZPjoRbO_uWpCX0a5tGcOaw3v4F84sp 2024-04-22T20:06:19.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.659+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.659+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.659+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _6ZPjoRbO_uWpCX0a5tGcOaw3v4F84sp 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.lHF_y9q7KsXveMvFfDmCcXxYx783WkW_0 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.lHF_y9q7KsXveMvFfDmCcXxYx783WkW_1 2024-04-22T20:06:19.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-04-22T20:06:19.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.662+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: _6ZPjoRbO_uWpCX0a5tGcOaw3v4F84sp 2024-04-22T20:06:19.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.663+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-04-22T20:06:19.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.663+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.663+0000 7fbb46d2d800 10 manifest: total_size = 324647 2024-04-22T20:06:19.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.663+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5138276915250340076 2024-04-22T20:06:19.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.663+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.663+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.663+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.665+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.665+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.665+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5138276915250340076 2024-04-22T20:06:19.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.665+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.665+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_0 2024-04-22T20:06:19.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.665+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 330 2024-04-22T20:06:19.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.665+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.665+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5138276915250340076 2024-04-22T20:06:19.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.667+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-04-22T20:06:19.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.667+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.667+0000 7fbb46d2d800 10 manifest: total_size = 324647 2024-04-22T20:06:19.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.667+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _M8lXXSD9nblSW6RCjyDJC6PsBf6YIR- 2024-04-22T20:06:19.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.667+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.667+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.667+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.670+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.670+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.670+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _M8lXXSD9nblSW6RCjyDJC6PsBf6YIR- 2024-04-22T20:06:19.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.670+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.670+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.hi4eBSriCAUiCvTkbFVudNOAff5R-Yx_0 2024-04-22T20:06:19.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.670+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-04-22T20:06:19.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.670+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.670+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _M8lXXSD9nblSW6RCjyDJC6PsBf6YIR- 2024-04-22T20:06:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.671+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-04-22T20:06:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.671+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc5970 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.671+0000 7fbb46d2d800 10 manifest: total_size = 324647 2024-04-22T20:06:19.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.671+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _7Tq7Ujhj4ovtHzIpHFfHuRqC9V-vz4Z 2024-04-22T20:06:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.671+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc5970 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.671+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.671+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.673+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.673+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.673+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _7Tq7Ujhj4ovtHzIpHFfHuRqC9V-vz4Z 2024-04-22T20:06:19.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.673+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.673+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.gHghNPhtFNO_WaXqx_dgFfRtHUJc2N7_0 2024-04-22T20:06:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.673+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-04-22T20:06:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.673+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.673+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: _7Tq7Ujhj4ovtHzIpHFfHuRqC9V-vz4Z 2024-04-22T20:06:19.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.674+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-04-22T20:06:19.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 10 manifest: total_size = 324647 2024-04-22T20:06:19.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 10 generate_fake_tag new tag=19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_1_99992276c73b41acc0150aa4ff2209a0 2024-04-22T20:06:19.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_1_99992276c73b41acc0150aa4ff2209a0 2024-04-22T20:06:19.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.675+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15.0 2024-04-22T20:06:19.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.677+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.677+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.677+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_1_99992276c73b41acc0150aa4ff2209a0 2024-04-22T20:06:19.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.677+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.677+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_0 2024-04-22T20:06:19.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.677+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 389 2024-04-22T20:06:19.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.677+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.677+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_1_99992276c73b41acc0150aa4ff2209a0 2024-04-22T20:06:19.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.678+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-04-22T20:06:19.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.679+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.679+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.679+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.679+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.679+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.679+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.679+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.680+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2024-04-22T20:06:19.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.680+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.681+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=233 2024-04-22T20:06:19.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.681+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x11 2024-04-22T20:06:19.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.681+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2024-04-22T20:06:19.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.682+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2024-04-22T20:06:19.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.682+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1, ofs0, ns] 2024-04-22T20:06:19.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.682+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562396 cookie 94611750419312 notifier 5952 bl.length()=189 2024-04-22T20:06:19.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.685+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15 from cache 2024-04-22T20:06:19.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.685+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15, ofs0, ns] 2024-04-22T20:06:19.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.685+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562397 cookie 94611750419312 notifier 5952 bl.length()=247 2024-04-22T20:06:19.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.688+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15 : miss 2024-04-22T20:06:19.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.688+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.688+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.688+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15 info.flags=0x0 2024-04-22T20:06:19.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.688+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15 to cache LRU end 2024-04-22T20:06:19.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.690+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16 : miss 2024-04-22T20:06:19.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.690+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=457 2024-04-22T20:06:19.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16 info.flags=0x17 2024-04-22T20:06:19.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16 to cache LRU end 2024-04-22T20:06:19.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16 2024-04-22T20:06:19.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN 2024-04-22T20:06:19.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-04-22T20:06:19.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-04-22T20:06:19.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-04-22T20:06:19.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-04-22T20:06:19.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.691+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cad6c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.692+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.692+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4464368107970261222 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.692+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cad6c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.692+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.692+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4464368107970261222 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 367 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 632 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.2 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 899 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.2_1 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1164 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.3 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1431 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.694+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4464368107970261222 2024-04-22T20:06:19.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.695+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-04-22T20:06:19.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.695+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 10 generate_fake_tag new tag=19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1_1f0c5ea57d715e24c2f266da123be979 2024-04-22T20:06:19.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1_1f0c5ea57d715e24c2f266da123be979 2024-04-22T20:06:19.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.696+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1_1f0c5ea57d715e24c2f266da123be979 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 705 2024-04-22T20:06:19.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.2 2024-04-22T20:06:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 972 2024-04-22T20:06:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.2_1 2024-04-22T20:06:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1237 2024-04-22T20:06:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.3 2024-04-22T20:06:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1504 2024-04-22T20:06:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.698+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1_1f0c5ea57d715e24c2f266da123be979 2024-04-22T20:06:19.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.699+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-04-22T20:06:19.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.699+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.700+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.700+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _Icc7jkR0y1f3HcPlZDGMma3uWpIUDGl 2024-04-22T20:06:19.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.700+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.700+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.700+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _Icc7jkR0y1f3HcPlZDGMma3uWpIUDGl 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_._GMSN2cCpZMxK-PNVzix3jlIdiMNCIN_0 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_._GMSN2cCpZMxK-PNVzix3jlIdiMNCIN_1 2024-04-22T20:06:19.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-04-22T20:06:19.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_._GMSN2cCpZMxK-PNVzix3jlIdiMNCIN_2 2024-04-22T20:06:19.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 788 2024-04-22T20:06:19.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.702+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: _Icc7jkR0y1f3HcPlZDGMma3uWpIUDGl 2024-04-22T20:06:19.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.703+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-04-22T20:06:19.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.703+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.703+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.703+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _WqZ4Z-g4nmIIq5kxPHNKvdcggYXBkRo 2024-04-22T20:06:19.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.703+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.703+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.703+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.705+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.705+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.705+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.705+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.705+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _WqZ4Z-g4nmIIq5kxPHNKvdcggYXBkRo 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.9ESM-r6fMpvpG23M5kYZLhyRx602i2o_0 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-04-22T20:06:19.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.9ESM-r6fMpvpG23M5kYZLhyRx602i2o_1 2024-04-22T20:06:19.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-04-22T20:06:19.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.9ESM-r6fMpvpG23M5kYZLhyRx602i2o_2 2024-04-22T20:06:19.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 761 2024-04-22T20:06:19.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.706+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: _WqZ4Z-g4nmIIq5kxPHNKvdcggYXBkRo 2024-04-22T20:06:19.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.707+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c066f80 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.707+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-04-22T20:06:19.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.707+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.707+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10290313714166553581 2024-04-22T20:06:19.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.707+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c066f80 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.707+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.707+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10290313714166553581 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 374 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 645 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.2 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 918 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.2_1 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1189 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.3 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1462 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.709+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10290313714166553581 2024-04-22T20:06:19.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.710+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-04-22T20:06:19.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.711+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cb6f40 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.711+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.711+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _tbzq519WG5reY-jETQte14-lG29QV04 2024-04-22T20:06:19.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.711+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cb6f40 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.711+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.711+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _tbzq519WG5reY-jETQte14-lG29QV04 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.PkJhzRsAMqhbTakknTyEjIdaznR3ieb_0 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.PkJhzRsAMqhbTakknTyEjIdaznR3ieb_1 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.PkJhzRsAMqhbTakknTyEjIdaznR3ieb_2 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 782 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.714+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _tbzq519WG5reY-jETQte14-lG29QV04 2024-04-22T20:06:19.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-04-22T20:06:19.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 10 generate_fake_tag new tag=19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1_0e2e20e50dc96ed1fcb7c15ef4ac7841 2024-04-22T20:06:19.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1_0e2e20e50dc96ed1fcb7c15ef4ac7841 2024-04-22T20:06:19.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.715+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1_0e2e20e50dc96ed1fcb7c15ef4ac7841 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 448 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 719 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.2 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 992 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.2_1 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1263 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.3 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1536 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.718+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1_0e2e20e50dc96ed1fcb7c15ef4ac7841 2024-04-22T20:06:19.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.719+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-04-22T20:06:19.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.719+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cad8f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.719+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.719+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _yIwytN-8WsngtTzPsXRyGdEzEw3Vugx 2024-04-22T20:06:19.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.719+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cad8f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.719+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.719+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _yIwytN-8WsngtTzPsXRyGdEzEw3Vugx 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.HPk3nza-Vr8B2GmMK5-QJguftEJzdxi_0 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.HPk3nza-Vr8B2GmMK5-QJguftEJzdxi_1 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.HPk3nza-Vr8B2GmMK5-QJguftEJzdxi_2 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 761 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.721+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: _yIwytN-8WsngtTzPsXRyGdEzEw3Vugx 2024-04-22T20:06:19.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.723+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-04-22T20:06:19.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.723+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.723+0000 7fbb46d2d800 10 manifest: total_size = 16382 2024-04-22T20:06:19.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.723+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15524819901381658645 2024-04-22T20:06:19.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.723+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb2c05fa90 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.723+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.723+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=7674 rules.size()=3 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7674 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=7674 stripe_ofs=7674 part_ofs=7674 rule->part_size=7673 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=7673 rules.size()=3 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4201978 part_ofs=7674 rule->part_size=7673 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=15347 stripe_ofs=15347 part_ofs=15347 rule->part_size=1035 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=3 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4209651 part_ofs=15347 rule->part_size=1035 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=16382 stripe_ofs=16382 part_ofs=16382 rule->part_size=1035 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15524819901381658645 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.1 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2 2024-04-22T20:06:19.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-04-22T20:06:19.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.3 2024-04-22T20:06:19.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-04-22T20:06:19.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.725+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15524819901381658645 2024-04-22T20:06:19.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.727+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-04-22T20:06:19.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.727+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cad6c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.727+0000 7fbb46d2d800 10 manifest: total_size = 16382 2024-04-22T20:06:19.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.727+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _VFhY0mFB_acH2IZTC-APyWDQTsvMV15 2024-04-22T20:06:19.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.727+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cad6c0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.727+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.727+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.729+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.729+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=16382 stripe_ofs=16382 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.729+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _VFhY0mFB_acH2IZTC-APyWDQTsvMV15 2024-04-22T20:06:19.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.729+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.729+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.XAjAhvOYfhKID58FNR8ZFXo8VB07YAo_0 2024-04-22T20:06:19.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.729+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-04-22T20:06:19.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.729+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.729+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _VFhY0mFB_acH2IZTC-APyWDQTsvMV15 2024-04-22T20:06:19.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.730+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-04-22T20:06:19.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.731+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc5940 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.731+0000 7fbb46d2d800 10 manifest: total_size = 16382 2024-04-22T20:06:19.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.731+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to _VnrFze2BWDJaxSsxARBaGgwL4i3zZkA 2024-04-22T20:06:19.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.731+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc5940 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.731+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.731+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.733+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.733+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=16382 stripe_ofs=16382 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.733+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: _VnrFze2BWDJaxSsxARBaGgwL4i3zZkA 2024-04-22T20:06:19.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.733+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.733+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.K3vdaO7MbBEkMt_gxtcunVtumba-y_H_0 2024-04-22T20:06:19.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.733+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-04-22T20:06:19.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.733+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.733+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _VnrFze2BWDJaxSsxARBaGgwL4i3zZkA 2024-04-22T20:06:19.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-04-22T20:06:19.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc5940 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 10 manifest: total_size = 16382 2024-04-22T20:06:19.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=7674 rules.size()=3 2024-04-22T20:06:19.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7674 2024-04-22T20:06:19.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=7674 stripe_ofs=7674 part_ofs=7674 rule->part_size=7673 2024-04-22T20:06:19.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 10 generate_fake_tag new tag=19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2_8aac982f249899867ff058f048594b83 2024-04-22T20:06:19.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2_8aac982f249899867ff058f048594b83 2024-04-22T20:06:19.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc5940 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x560c83cb0b68 s->prefetch_data=0 2024-04-22T20:06:19.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb0b68 2024-04-22T20:06:19.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.735+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16.0 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=7674 rules.size()=3 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7674 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=7674 stripe_ofs=7674 part_ofs=7674 rule->part_size=7673 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=7673 rules.size()=3 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4201978 part_ofs=7674 rule->part_size=7673 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=15347 stripe_ofs=15347 part_ofs=15347 rule->part_size=1035 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=3 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4209651 part_ofs=15347 rule->part_size=1035 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=16382 stripe_ofs=16382 part_ofs=16382 rule->part_size=1035 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2_8aac982f249899867ff058f048594b83 2024-04-22T20:06:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.1 2024-04-22T20:06:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-04-22T20:06:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2 2024-04-22T20:06:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 704 2024-04-22T20:06:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.3 2024-04-22T20:06:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 968 2024-04-22T20:06:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.738+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2_8aac982f249899867ff058f048594b83 2024-04-22T20:06:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.739+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-04-22T20:06:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.739+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.739+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.739+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.739+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.739+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.739+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.739+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.740+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2024-04-22T20:06:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.741+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.741+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=243 2024-04-22T20:06:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.741+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x11 2024-04-22T20:06:19.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.741+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2024-04-22T20:06:19.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.742+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2024-04-22T20:06:19.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.742+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1, ofs0, ns] 2024-04-22T20:06:19.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.743+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562378 cookie 94611750422912 notifier 5952 bl.length()=199 2024-04-22T20:06:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.745+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16 from cache 2024-04-22T20:06:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.745+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16, ofs0, ns] 2024-04-22T20:06:19.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.745+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562384 cookie 94611750428320 notifier 5952 bl.length()=257 2024-04-22T20:06:19.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.748+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16 : miss 2024-04-22T20:06:19.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.748+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.748+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.748+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16 info.flags=0x0 2024-04-22T20:06:19.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.748+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16 to cache LRU end 2024-04-22T20:06:19.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.750+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14 : miss 2024-04-22T20:06:19.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.750+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=452 2024-04-22T20:06:19.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14 info.flags=0x17 2024-04-22T20:06:19.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14 to cache LRU end 2024-04-22T20:06:19.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14 2024-04-22T20:06:19.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN 2024-04-22T20:06:19.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-04-22T20:06:19.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD 2024-04-22T20:06:19.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.751+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83ca94f0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x560c83ca7bd8 s->prefetch_data=0 2024-04-22T20:06:19.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.752+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.752+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16469392166702829532 2024-04-22T20:06:19.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.752+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83ca94f0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x560c83ca7bd8 s->prefetch_data=0 2024-04-22T20:06:19.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.752+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83ca7bd8 2024-04-22T20:06:19.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.752+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14.0 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16469392166702829532 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.1 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__shadow_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.1_1 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 633 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.2 2024-04-22T20:06:19.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 900 2024-04-22T20:06:19.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__shadow_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.2_1 2024-04-22T20:06:19.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1165 2024-04-22T20:06:19.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.3 2024-04-22T20:06:19.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1432 2024-04-22T20:06:19.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.754+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16469392166702829532 2024-04-22T20:06:19.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.756+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-04-22T20:06:19.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.756+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc8220 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x560c83ca7bd8 s->prefetch_data=0 2024-04-22T20:06:19.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.756+0000 7fbb46d2d800 10 manifest: total_size = 11534336 2024-04-22T20:06:19.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.756+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5296583134551112091 2024-04-22T20:06:19.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.756+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc8220 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x560c83ca7bd8 s->prefetch_data=0 2024-04-22T20:06:19.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.756+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83ca7bd8 2024-04-22T20:06:19.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.756+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14.0 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2024-04-22T20:06:19.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5296583134551112091 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.1 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 373 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__shadow_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.1_1 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 644 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.2 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 917 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__shadow_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.2_1 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1188 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.3 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1461 2024-04-22T20:06:19.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.758+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5296583134551112091 2024-04-22T20:06:19.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.760+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-04-22T20:06:19.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.760+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x560c83ca7bd8 s->prefetch_data=0 2024-04-22T20:06:19.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.760+0000 7fbb46d2d800 10 manifest: total_size = 1087 2024-04-22T20:06:19.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.760+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17415478986783585847 2024-04-22T20:06:19.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.760+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x560c83ca7bd8 s->prefetch_data=0 2024-04-22T20:06:19.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.760+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83ca7bd8 2024-04-22T20:06:19.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.760+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14.0 2024-04-22T20:06:19.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-04-22T20:06:19.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=516 2024-04-22T20:06:19.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-04-22T20:06:19.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-04-22T20:06:19.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4194820 part_ofs=516 rule->part_size=516 2024-04-22T20:06:19.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=55 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=55 rules.size()=2 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): stripe_ofs=4195336 part_ofs=1032 rule->part_size=55 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=1087 rule->part_size=55 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17415478986783585847 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-STANDARD.2~dHoKXv965g23tcG9lyjbpj1RNE3-FrA.1 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-STANDARD.2~dHoKXv965g23tcG9lyjbpj1RNE3-FrA.2 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-STANDARD.2~dHoKXv965g23tcG9lyjbpj1RNE3-FrA.3 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-04-22T20:06:19.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.763+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17415478986783585847 2024-04-22T20:06:19.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.764+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-04-22T20:06:19.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.764+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.764+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.765+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.765+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.765+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.765+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.765+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.766+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2024-04-22T20:06:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.766+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.766+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=238 2024-04-22T20:06:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.766+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x11 2024-04-22T20:06:19.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.766+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2024-04-22T20:06:19.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.767+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2024-04-22T20:06:19.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.767+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1, ofs0, ns] 2024-04-22T20:06:19.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.768+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562385 cookie 94611750428320 notifier 5952 bl.length()=194 2024-04-22T20:06:19.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.770+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14 from cache 2024-04-22T20:06:19.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.770+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14, ofs0, ns] 2024-04-22T20:06:19.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.770+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562386 cookie 94611750428320 notifier 5952 bl.length()=252 2024-04-22T20:06:19.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.772+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14 : miss 2024-04-22T20:06:19.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.772+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.773+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.773+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14 info.flags=0x0 2024-04-22T20:06:19.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.773+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14 to cache LRU end 2024-04-22T20:06:19.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.775+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13 : miss 2024-04-22T20:06:19.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.775+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.775+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=442 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.775+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13 info.flags=0x17 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.775+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13 to cache LRU end 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.775+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.775+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.775+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.776+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-FROZEN 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.776+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.776+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got foo-STANDARD 2024-04-22T20:06:19.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.776+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.777+0000 7fbb46d2d800 10 manifest: total_size = 10485760 2024-04-22T20:06:19.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.777+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6906545616172387648 2024-04-22T20:06:19.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.777+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83caf8c0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.777+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.777+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13.0 2024-04-22T20:06:19.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6906545616172387648 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.SnH2lIpXBu0QqES7HyPPx_3RaWcu64h_0 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 337 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.SnH2lIpXBu0QqES7HyPPx_3RaWcu64h_1 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 574 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.SnH2lIpXBu0QqES7HyPPx_3RaWcu64h_2 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 811 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.779+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6906545616172387648 2024-04-22T20:06:19.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.780+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-04-22T20:06:19.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.780+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34003f30 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x560c83cb4dd8 s->prefetch_data=0 2024-04-22T20:06:19.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.780+0000 7fbb46d2d800 10 manifest: total_size = 10485760 2024-04-22T20:06:19.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.780+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5510026797356098222 2024-04-22T20:06:19.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.780+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34003f30 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x560c83cb4dd8 s->prefetch_data=0 2024-04-22T20:06:19.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.780+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cb4dd8 2024-04-22T20:06:19.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.780+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13.0 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5510026797356098222 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.GnhA_1A4gaaM1Xk0gMoLKTZOKcOQXBb_0 2024-04-22T20:06:19.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 339 2024-04-22T20:06:19.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.GnhA_1A4gaaM1Xk0gMoLKTZOKcOQXBb_1 2024-04-22T20:06:19.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 578 2024-04-22T20:06:19.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.GnhA_1A4gaaM1Xk0gMoLKTZOKcOQXBb_2 2024-04-22T20:06:19.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 817 2024-04-22T20:06:19.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.783+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5510026797356098222 2024-04-22T20:06:19.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.784+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-04-22T20:06:19.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.784+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34097a80 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x560c83ca7bd8 s->prefetch_data=0 2024-04-22T20:06:19.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.784+0000 7fbb46d2d800 10 manifest: total_size = 3309 2024-04-22T20:06:19.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.784+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1020013301252061178 2024-04-22T20:06:19.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.784+0000 7fbb46d2d800 20 get_obj_state: octx=0x7fbb34097a80 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x560c83ca7bd8 s->prefetch_data=0 2024-04-22T20:06:19.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.784+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83ca7bd8 2024-04-22T20:06:19.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.784+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13.0 2024-04-22T20:06:19.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.787+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.787+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=3309 stripe_ofs=3309 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.787+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1020013301252061178 2024-04-22T20:06:19.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.787+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.787+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.RbjQZkc1FXL1vx7YqvANq2tS2jJWbSz_0 2024-04-22T20:06:19.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.787+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 330 2024-04-22T20:06:19.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.787+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.787+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1020013301252061178 2024-04-22T20:06:19.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.788+0000 7fba267fc640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-04-22T20:06:19.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.788+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.788+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.788+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.788+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.788+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.788+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.788+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.790+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2024-04-22T20:06:19.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.790+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.790+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=228 2024-04-22T20:06:19.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.790+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x11 2024-04-22T20:06:19.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.790+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2024-04-22T20:06:19.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.791+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2024-04-22T20:06:19.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.791+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1, ofs0, ns] 2024-04-22T20:06:19.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.792+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562398 cookie 94611750419312 notifier 5952 bl.length()=184 2024-04-22T20:06:19.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.794+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13 from cache 2024-04-22T20:06:19.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.794+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13, ofs0, ns] 2024-04-22T20:06:19.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.795+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562442 cookie 94611750425664 notifier 5952 bl.length()=242 2024-04-22T20:06:19.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.797+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13 : miss 2024-04-22T20:06:19.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.797+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.797+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.797+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13 info.flags=0x0 2024-04-22T20:06:19.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.797+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13 to cache LRU end 2024-04-22T20:06:19.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.799+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6 : miss 2024-04-22T20:06:19.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.799+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=437 2024-04-22T20:06:19.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6 info.flags=0x17 2024-04-22T20:06:19.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6 to cache LRU end 2024-04-22T20:06:19.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got __ 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got __ _ 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got ___ 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got obj 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83c6c0c0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 10 manifest: total_size = 0 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16525252022916998713 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83c6c0c0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.800+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6.0 2024-04-22T20:06:19.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.803+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83c6c0c0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.803+0000 7fbb46d2d800 10 manifest: total_size = 0 2024-04-22T20:06:19.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.803+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13268694951847378469 2024-04-22T20:06:19.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.803+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83c6c0c0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.803+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.803+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6.0 2024-04-22T20:06:19.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.804+0000 7fba267fc640 20 handle_completion(): completion ok for obj=__ 2024-04-22T20:06:19.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.805+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83c6c0c0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.806+0000 7fbb46d2d800 10 manifest: total_size = 0 2024-04-22T20:06:19.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.806+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12680241965108859562 2024-04-22T20:06:19.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.806+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83c6c0c0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.806+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.806+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6.0 2024-04-22T20:06:19.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.807+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=__ _ 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.808+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83c6c0c0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.809+0000 7fbb46d2d800 10 manifest: total_size = 0 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.809+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1972258388122229957 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.809+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83c6c0c0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.809+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.809+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6.0 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.810+0000 7fba267fc640 20 handle_completion(): completion ok for obj=___ 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.811+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.811+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.812+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=obj 2024-04-22T20:06:19.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.812+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.812+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.812+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.812+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.812+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.814+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2024-04-22T20:06:19.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.814+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.814+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=223 2024-04-22T20:06:19.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.814+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x11 2024-04-22T20:06:19.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.814+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2024-04-22T20:06:19.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.815+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2024-04-22T20:06:19.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.815+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.1, ofs0, ns] 2024-04-22T20:06:19.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.815+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562436 cookie 94611750415504 notifier 5952 bl.length()=181 2024-04-22T20:06:19.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.818+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6 from cache 2024-04-22T20:06:19.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.818+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6, ofs0, ns] 2024-04-22T20:06:19.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.819+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562399 cookie 94611750419312 notifier 5952 bl.length()=238 2024-04-22T20:06:19.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.821+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6 : miss 2024-04-22T20:06:19.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.821+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.821+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.821+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6 info.flags=0x0 2024-04-22T20:06:19.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.821+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6 to cache LRU end 2024-04-22T20:06:19.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.823+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7 : miss 2024-04-22T20:06:19.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.823+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=437 2024-04-22T20:06:19.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7 info.flags=0x17 2024-04-22T20:06:19.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7 to cache LRU end 2024-04-22T20:06:19.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7 2024-04-22T20:06:19.824 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got __ 2024-04-22T20:06:19.824 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got __ _ 2024-04-22T20:06:19.824 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got ___ 2024-04-22T20:06:19.824 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got obj 2024-04-22T20:06:19.824 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 10 manifest: total_size = 38 2024-04-22T20:06:19.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16270238826618142801 2024-04-22T20:06:19.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.824+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.825+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7.0 2024-04-22T20:06:19.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.827+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.827+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=38 stripe_ofs=38 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.827+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16270238826618142801 2024-04-22T20:06:19.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.827+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.827+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7__shadow_.8UnRz0Vygc5QVIYmXBDPp8kg_Zwfx4w_0 2024-04-22T20:06:19.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.827+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-22T20:06:19.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.827+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.827+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16270238826618142801 2024-04-22T20:06:19.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.828+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.828+0000 7fba267fc640 20 handle_completion(): completion ok for obj=__ 2024-04-22T20:06:19.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.829+0000 7fbb46d2d800 10 manifest: total_size = 2078 2024-04-22T20:06:19.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.829+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4590915619532309436 2024-04-22T20:06:19.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.829+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.829+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.829+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7.0 2024-04-22T20:06:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.832+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.832+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=2078 stripe_ofs=2078 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.832+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4590915619532309436 2024-04-22T20:06:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.832+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.832+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7__shadow_.FgVJ4iwDdbLemdkJk9E5hvmRuYtj1P8_0 2024-04-22T20:06:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.832+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.833+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.833+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4590915619532309436 2024-04-22T20:06:19.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.834+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=__ _ 2024-04-22T20:06:19.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.834+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.834+0000 7fbb46d2d800 10 manifest: total_size = 38 2024-04-22T20:06:19.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.834+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1657659686465603911 2024-04-22T20:06:19.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.834+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.834+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.834+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7.0 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.836+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.836+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=38 stripe_ofs=38 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.836+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1657659686465603911 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.836+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.836+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7__shadow_.RKqyAG_gaeu_MaoqxB_xK62RwcNOr49_0 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.836+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.836+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.836+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1657659686465603911 2024-04-22T20:06:19.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.838+0000 7fba267fc640 20 handle_completion(): completion ok for obj=___ 2024-04-22T20:06:19.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.838+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.838+0000 7fbb46d2d800 10 manifest: total_size = 38 2024-04-22T20:06:19.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.838+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4826913521811364234 2024-04-22T20:06:19.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.838+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cae3f0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.838+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.838+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7.0 2024-04-22T20:06:19.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.840+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.840+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=38 stripe_ofs=38 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.840+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4826913521811364234 2024-04-22T20:06:19.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.840+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.840+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7__shadow_.Cn2J41Okq5qm_n1UonStT2ILRs_pJjx_0 2024-04-22T20:06:19.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.840+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.840+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.840+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4826913521811364234 2024-04-22T20:06:19.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.842+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=obj 2024-04-22T20:06:19.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.842+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.842+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.842+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.842+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.842+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.842+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.842+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.844+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2024-04-22T20:06:19.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.844+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.844+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=223 2024-04-22T20:06:19.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.844+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x11 2024-04-22T20:06:19.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.844+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2024-04-22T20:06:19.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.845+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2024-04-22T20:06:19.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.845+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.2, ofs0, ns] 2024-04-22T20:06:19.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.846+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562437 cookie 94611750415504 notifier 5952 bl.length()=181 2024-04-22T20:06:19.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.848+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7 from cache 2024-04-22T20:06:19.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.848+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7, ofs0, ns] 2024-04-22T20:06:19.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.848+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562400 cookie 94611750419312 notifier 5952 bl.length()=238 2024-04-22T20:06:19.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.851+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7 : miss 2024-04-22T20:06:19.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.851+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.851+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.851+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7 info.flags=0x0 2024-04-22T20:06:19.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.851+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7 to cache LRU end 2024-04-22T20:06:19.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.853+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8 : miss 2024-04-22T20:06:19.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.853+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=437 2024-04-22T20:06:19.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8 info.flags=0x17 2024-04-22T20:06:19.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8 to cache LRU end 2024-04-22T20:06:19.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:19.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-04-22T20:06:19.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8 2024-04-22T20:06:19.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got __ 2024-04-22T20:06:19.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got __ _ 2024-04-22T20:06:19.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got ___ 2024-04-22T20:06:19.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 10 cls_bucket_list_unordered: got obj 2024-04-22T20:06:19.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.854+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc6690 obj=test-client.0-r-test-small-obj-data.3:__ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.855+0000 7fbb46d2d800 10 manifest: total_size = 55 2024-04-22T20:06:19.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.855+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8515233154692155770 2024-04-22T20:06:19.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.855+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc6690 obj=test-client.0-r-test-small-obj-data.3:__ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.855+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.855+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8.0 2024-04-22T20:06:19.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.857+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.857+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=55 stripe_ofs=55 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.857+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8515233154692155770 2024-04-22T20:06:19.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.857+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.857+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8__shadow_.8iXGndIFhlyXbcS_Qxi03egpRmPWLsi_0 2024-04-22T20:06:19.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.857+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.857+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.857+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8515233154692155770 2024-04-22T20:06:19.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.858+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc6690 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.859+0000 7fba267fc640 20 handle_completion(): completion ok for obj=__ 2024-04-22T20:06:19.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.859+0000 7fbb46d2d800 10 manifest: total_size = 49187 2024-04-22T20:06:19.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.859+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10697927379977224298 2024-04-22T20:06:19.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.859+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc6690 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.859+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.859+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8.0 2024-04-22T20:06:19.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.861+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.861+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=49187 stripe_ofs=49187 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.861+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10697927379977224298 2024-04-22T20:06:19.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.861+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.861+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8__shadow_.NJjgqNtQld1kigVXQ6wsdN1918t0-6S_0 2024-04-22T20:06:19.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.861+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-22T20:06:19.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.861+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.861+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10697927379977224298 2024-04-22T20:06:19.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.862+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=__ _ 2024-04-22T20:06:19.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.862+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc6690 obj=test-client.0-r-test-small-obj-data.3:___ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.863+0000 7fbb46d2d800 10 manifest: total_size = 4134 2024-04-22T20:06:19.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.863+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13250830488287904107 2024-04-22T20:06:19.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.863+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc6690 obj=test-client.0-r-test-small-obj-data.3:___ state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.863+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.863+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8.0 2024-04-22T20:06:19.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.865+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.865+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=4134 stripe_ofs=4134 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.865+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13250830488287904107 2024-04-22T20:06:19.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.865+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.865+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8__shadow_.ZKUWNsr4y1YGwrEfcLoxr8Er7wQXTuT_0 2024-04-22T20:06:19.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.865+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-04-22T20:06:19.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.865+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.865+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13250830488287904107 2024-04-22T20:06:19.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.866+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc6690 obj=test-client.0-r-test-small-obj-data.3:obj state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.866+0000 7fba267fc640 20 handle_completion(): completion ok for obj=___ 2024-04-22T20:06:19.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.866+0000 7fbb46d2d800 10 manifest: total_size = 1035 2024-04-22T20:06:19.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.866+0000 7fbb46d2d800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5739523068998276773 2024-04-22T20:06:19.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.866+0000 7fbb46d2d800 20 get_obj_state: octx=0x560c83cc6690 obj=test-client.0-r-test-small-obj-data.3:obj state=0x560c83cc70b8 s->prefetch_data=0 2024-04-22T20:06:19.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.866+0000 7fbb46d2d800 20 prepare_atomic_modification: state is not atomic. state=0x560c83cc70b8 2024-04-22T20:06:19.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.866+0000 7fbb46d2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8.0 2024-04-22T20:06:19.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.869+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:06:19.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.869+0000 7fbb46d2d800 20 RGWObjManifest::operator++(): result: ofs=1035 stripe_ofs=1035 part_ofs=0 rule->part_size=0 2024-04-22T20:06:19.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.869+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5739523068998276773 2024-04-22T20:06:19.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.869+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-04-22T20:06:19.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.869+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8__shadow_.oMYKmlV3J3w8kgf6x2jb6iW79FqnSfq_0 2024-04-22T20:06:19.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.869+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-04-22T20:06:19.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.869+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-04-22T20:06:19.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.869+0000 7fbb46d2d800 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5739523068998276773 2024-04-22T20:06:19.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.870+0000 7fba25ffb640 20 handle_completion(): completion ok for obj=obj 2024-04-22T20:06:19.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.870+0000 7fbb46d2d800 20 list_objects_ordered: starting attempt 1 2024-04-22T20:06:19.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.870+0000 7fbb46d2d800 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-04-22T20:06:19.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.870+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-22T20:06:19.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.870+0000 7fbb46d2d800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-22T20:06:19.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.870+0000 7fbb46d2d800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-22T20:06:19.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.870+0000 7fbb46d2d800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-22T20:06:19.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.870+0000 7fbb46d2d800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8])"; uploads.size()=0; is_truncated=0 2024-04-22T20:06:19.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.872+0000 7fbb46d2d800 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2024-04-22T20:06:19.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.872+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.872+0000 7fbb46d2d800 20 rados_obj.operate() r=0 bl.length=223 2024-04-22T20:06:19.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.872+0000 7fbb46d2d800 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x11 2024-04-22T20:06:19.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.872+0000 7fbb46d2d800 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2024-04-22T20:06:19.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.873+0000 7fbb46d2d800 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2024-04-22T20:06:19.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.873+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.3, ofs0, ns] 2024-04-22T20:06:19.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.874+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562438 cookie 94611750415504 notifier 5952 bl.length()=181 2024-04-22T20:06:19.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.876+0000 7fbb46d2d800 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8 from cache 2024-04-22T20:06:19.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.876+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8, ofs0, ns] 2024-04-22T20:06:19.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.876+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562401 cookie 94611750419312 notifier 5952 bl.length()=238 2024-04-22T20:06:19.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.878+0000 7fbb46d2d800 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8 : miss 2024-04-22T20:06:19.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.878+0000 7fbb46d2d800 20 rados->read ofs=0 len=0 2024-04-22T20:06:19.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.879+0000 7fbb46d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:19.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.879+0000 7fbb46d2d800 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8 info.flags=0x0 2024-04-22T20:06:19.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.879+0000 7fbb46d2d800 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8 to cache LRU end 2024-04-22T20:06:19.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.881+0000 7fbb46d2d800 10 removing key index: NWEWLXEAWQBCKWKCBKIA 2024-04-22T20:06:19.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.882+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NWEWLXEAWQBCKWKCBKIA, ofs0, ns] 2024-04-22T20:06:19.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.882+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562387 cookie 94611750428320 notifier 5952 bl.length()=170 2024-04-22T20:06:19.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.883+0000 7fbb46d2d800 10 removing email index: ragweed.client.0+test@test.test 2024-04-22T20:06:19.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.884+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-04-22T20:06:19.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.885+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562402 cookie 94611750419312 notifier 5952 bl.length()=182 2024-04-22T20:06:19.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.886+0000 7fbb46d2d800 10 removing user buckets index 2024-04-22T20:06:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.887+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0.buckets, ofs0, ns] 2024-04-22T20:06:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.887+0000 7fba267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562388 cookie 94611750428320 notifier 5952 bl.length()=173 2024-04-22T20:06:19.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.888+0000 7fbb46d2d800 10 removing user index: ragweed.client.0 2024-04-22T20:06:19.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.889+0000 7fbb46d2d800 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2024-04-22T20:06:19.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.889+0000 7fbb46d2d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-04-22T20:06:19.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.890+0000 7fba25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562443 cookie 94611750425664 notifier 5952 bl.length()=165 2024-04-22T20:06:19.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.893+0000 7fb9f97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:06:19.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.895+0000 7fbb46d2d800 20 remove_watcher() i=0 2024-04-22T20:06:19.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.895+0000 7fbb46d2d800 2 removed watcher, disabling cache 2024-04-22T20:06:19.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.896+0000 7fbb46d2d800 20 remove_watcher() i=1 2024-04-22T20:06:19.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.898+0000 7fbb46d2d800 20 remove_watcher() i=2 2024-04-22T20:06:19.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.899+0000 7fbb46d2d800 20 remove_watcher() i=3 2024-04-22T20:06:19.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.900+0000 7fbb46d2d800 20 remove_watcher() i=4 2024-04-22T20:06:19.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.901+0000 7fbb46d2d800 20 remove_watcher() i=5 2024-04-22T20:06:19.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.901+0000 7fbb46d2d800 20 remove_watcher() i=6 2024-04-22T20:06:19.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:19.902+0000 7fbb46d2d800 20 remove_watcher() i=7 2024-04-22T20:06:19.912 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2024-04-22T20:06:20.006 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:06:20.006 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:06:20.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.038+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.039+0000 7f2707211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.039+0000 7f2707211800 20 realm 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.039+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.039+0000 7f2707211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.039+0000 7f2707211800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.039+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.040+0000 7f2707211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.040+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.040+0000 7f2707211800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:06:20.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.040+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.041+0000 7f2707211800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:06:20.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.041+0000 7f2707211800 20 searching for the correct realm 2024-04-22T20:06:20.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.042+0000 7f2703015640 0 -- 172.21.15.136:0/2227925372 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55de4993c970 msgr2=0x55de4993ed60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:06:20.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.046+0000 7f2703015640 0 -- 172.21.15.136:0/2227925372 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f25cc07ba00 msgr2=0x7f25cc07de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:06:20.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:06:20.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:06:20.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.051+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:06:20.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 20 zone default found 2024-04-22T20:06:20.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 4 Realm: () 2024-04-22T20:06:20.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:06:20.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 20 zonegroup default 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.052+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.053+0000 7f2707211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.053+0000 7f2707211800 20 started sync module instance, tier type = 2024-04-22T20:06:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.053+0000 7f2707211800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:06:20.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.061+0000 7f2707211800 20 add_watcher() i=0 2024-04-22T20:06:20.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.061+0000 7f2707211800 20 add_watcher() i=1 2024-04-22T20:06:20.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.061+0000 7f2707211800 20 add_watcher() i=2 2024-04-22T20:06:20.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.061+0000 7f2707211800 20 add_watcher() i=3 2024-04-22T20:06:20.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.061+0000 7f2707211800 20 add_watcher() i=4 2024-04-22T20:06:20.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.061+0000 7f2707211800 20 add_watcher() i=5 2024-04-22T20:06:20.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.061+0000 7f2707211800 20 add_watcher() i=6 2024-04-22T20:06:20.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.062+0000 7f2707211800 20 add_watcher() i=7 2024-04-22T20:06:20.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.062+0000 7f2707211800 2 all 8 watchers are set, enabling cache 2024-04-22T20:06:20.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.064+0000 7f25b97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:06:20.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.064+0000 7f2707211800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:06:20.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.064+0000 7f2707211800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:06:20.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2533fff640 20 reqs_thread_entry: start 2024-04-22T20:06:20.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2707211800 20 init_complete bucket index max shards: 1 2024-04-22T20:06:20.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2707211800 20 Filter name: none 2024-04-22T20:06:20.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2531ffb640 20 reqs_thread_entry: start 2024-04-22T20:06:20.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2707211800 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-04-22T20:06:20.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2707211800 20 rados->read ofs=0 len=0 2024-04-22T20:06:20.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2707211800 20 rados_obj.operate() r=0 bl.length=486 2024-04-22T20:06:20.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2707211800 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x13 2024-04-22T20:06:20.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.099+0000 7f2707211800 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-04-22T20:06:20.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.100+0000 7f2707211800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:06:20.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.100+0000 7f2707211800 10 removing key index: ZGJGGKNMIAGKWDMXZXCS 2024-04-22T20:06:20.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.101+0000 7f2707211800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.keys:ZGJGGKNMIAGKWDMXZXCS, ofs0, ns] 2024-04-22T20:06:20.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.102+0000 7f25e67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562439 cookie 94413205528688 notifier 5699 bl.length()=170 2024-04-22T20:06:20.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.103+0000 7f2707211800 10 removing email index: sysuser.client.0+test@test.test 2024-04-22T20:06:20.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.104+0000 7f2707211800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-04-22T20:06:20.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.105+0000 7f25e67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562444 cookie 94413205569904 notifier 5699 bl.length()=182 2024-04-22T20:06:20.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.106+0000 7f2707211800 10 removing user buckets index 2024-04-22T20:06:20.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.107+0000 7f2707211800 10 removing user index: sysuser.client.0 2024-04-22T20:06:20.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.108+0000 7f2707211800 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2024-04-22T20:06:20.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.108+0000 7f2707211800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-04-22T20:06:20.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.109+0000 7f25e5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562403 cookie 94413205563072 notifier 5699 bl.length()=165 2024-04-22T20:06:20.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.110+0000 7f25b97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:06:20.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.112+0000 7f2707211800 20 remove_watcher() i=0 2024-04-22T20:06:20.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.112+0000 7f2707211800 2 removed watcher, disabling cache 2024-04-22T20:06:20.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.113+0000 7f2707211800 20 remove_watcher() i=1 2024-04-22T20:06:20.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.114+0000 7f2707211800 20 remove_watcher() i=2 2024-04-22T20:06:20.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.115+0000 7f2707211800 20 remove_watcher() i=3 2024-04-22T20:06:20.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.116+0000 7f2707211800 20 remove_watcher() i=4 2024-04-22T20:06:20.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.117+0000 7f2707211800 20 remove_watcher() i=5 2024-04-22T20:06:20.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.118+0000 7f2707211800 20 remove_watcher() i=6 2024-04-22T20:06:20.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:06:20.119+0000 7f2707211800 20 remove_watcher() i=7 2024-04-22T20:06:20.128 INFO:tasks.ragweed:Removing ragweed... 2024-04-22T20:06:20.128 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/ragweed.client.0 2024-04-22T20:06:20.207 INFO:teuthology.run_tasks:Running task workunit... 2024-04-22T20:06:20.225 INFO:tasks.workunit:Pulling workunits from ref 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T20:06:20.225 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-22T20:06:20.225 DEBUG:teuthology.orchestra.run.smithi136:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-22T20:06:20.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T20:06:20.253 INFO:teuthology.orchestra.run.smithi136.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-22T20:06:20.254 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T20:06:20.321 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-04-22T20:06:20.321 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-04-22T20:06:20.388 INFO:tasks.workunit:timeout=3h 2024-04-22T20:06:20.388 INFO:tasks.workunit:cleanup=True 2024-04-22T20:06:20.388 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9ed8263edff7e87907bf24b054e5c7144c194dbe 2024-04-22T20:06:20.456 INFO:tasks.workunit.client.0.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-22T20:07:42.824 INFO:tasks.workunit.client.0.smithi136.stderr:Note: switching to '9ed8263edff7e87907bf24b054e5c7144c194dbe'. 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr:state without impacting any branches by switching back to a branch. 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:42.825 INFO:tasks.workunit.client.0.smithi136.stderr: git switch -c 2024-04-22T20:07:42.826 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:42.826 INFO:tasks.workunit.client.0.smithi136.stderr:Or undo this operation with: 2024-04-22T20:07:42.826 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:42.826 INFO:tasks.workunit.client.0.smithi136.stderr: git switch - 2024-04-22T20:07:42.826 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:42.826 INFO:tasks.workunit.client.0.smithi136.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-22T20:07:42.826 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:42.826 INFO:tasks.workunit.client.0.smithi136.stderr:HEAD is now at 9ed8263edff rgw: apply default quota config on account creation 2024-04-22T20:07:42.830 DEBUG:teuthology.orchestra.run.smithi136:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-22T20:07:42.897 INFO:tasks.workunit.client.0.smithi136.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-22T20:07:42.899 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T20:07:42.900 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-22T20:07:42.948 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-22T20:07:42.996 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-22T20:07:43.034 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-22T20:07:43.036 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T20:07:43.036 INFO:tasks.workunit.client.0.smithi136.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-22T20:07:43.074 INFO:tasks.workunit.client.0.smithi136.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-22T20:07:43.077 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T20:07:43.077 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-22T20:07:43.145 INFO:tasks.workunit:Running workunits matching rgw/run-reshard.sh on client.0... 2024-04-22T20:07:43.146 INFO:tasks.workunit:Running workunit rgw/run-reshard.sh... 2024-04-22T20:07:43.147 DEBUG:teuthology.orchestra.run.smithi136:workunit test rgw/run-reshard.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ed8263edff7e87907bf24b054e5c7144c194dbe TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-04-22T20:07:43.217 INFO:tasks.workunit.client.0.smithi136.stderr:+ ulimit -c 0 2024-04-22T20:07:43.217 INFO:tasks.workunit.client.0.smithi136.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-04-22T20:07:43.217 INFO:tasks.workunit.client.0.smithi136.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-22T20:07:43.218 INFO:tasks.workunit.client.0.smithi136.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:++ deactivate nondestructive 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n '' ']' 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n '' ']' 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:++ hash -r 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n '' ']' 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:++ unset VIRTUAL_ENV 2024-04-22T20:07:45.619 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-04-22T20:07:45.620 INFO:tasks.workunit.client.0.smithi136.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-04-22T20:07:45.620 INFO:tasks.workunit.client.0.smithi136.stderr:++ export VIRTUAL_ENV 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ _OLD_VIRTUAL_PATH=/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ export PATH 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n '' ']' 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -z '' ']' 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ _OLD_VIRTUAL_PS1= 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ PS1='(rgw) ' 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ export PS1 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-22T20:07:45.621 INFO:tasks.workunit.client.0.smithi136.stderr:++ hash -r 2024-04-22T20:07:45.622 INFO:tasks.workunit.client.0.smithi136.stderr:+ pip install pip --upgrade 2024-04-22T20:07:45.852 INFO:tasks.workunit.client.0.smithi136.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.9/site-packages (21.3.1) 2024-04-22T20:07:46.063 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting pip 2024-04-22T20:07:46.069 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-04-22T20:07:46.118 INFO:tasks.workunit.client.0.smithi136.stdout:Installing collected packages: pip 2024-04-22T20:07:46.118 INFO:tasks.workunit.client.0.smithi136.stdout: Attempting uninstall: pip 2024-04-22T20:07:46.118 INFO:tasks.workunit.client.0.smithi136.stdout: Found existing installation: pip 21.3.1 2024-04-22T20:07:46.270 INFO:tasks.workunit.client.0.smithi136.stdout: Uninstalling pip-21.3.1: 2024-04-22T20:07:46.275 INFO:tasks.workunit.client.0.smithi136.stdout: Successfully uninstalled pip-21.3.1 2024-04-22T20:07:47.237 INFO:tasks.workunit.client.0.smithi136.stdout:Successfully installed pip-24.0 2024-04-22T20:07:47.286 INFO:tasks.workunit.client.0.smithi136.stderr:+ pip install boto3 2024-04-22T20:07:48.110 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting boto3 2024-04-22T20:07:48.111 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached boto3-1.34.89-py3-none-any.whl.metadata (6.6 kB) 2024-04-22T20:07:48.665 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting botocore<1.35.0,>=1.34.89 (from boto3) 2024-04-22T20:07:48.666 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached botocore-1.34.89-py3-none-any.whl.metadata (5.7 kB) 2024-04-22T20:07:48.721 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-04-22T20:07:48.722 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-04-22T20:07:48.784 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-04-22T20:07:48.785 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-04-22T20:07:48.852 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T20:07:48.853 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-04-22T20:07:48.935 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting urllib3<1.27,>=1.25.4 (from botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T20:07:48.936 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached urllib3-1.26.18-py2.py3-none-any.whl.metadata (48 kB) 2024-04-22T20:07:49.181 INFO:tasks.workunit.client.0.smithi136.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.89->boto3) 2024-04-22T20:07:49.183 INFO:tasks.workunit.client.0.smithi136.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-04-22T20:07:49.205 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached boto3-1.34.89-py3-none-any.whl (139 kB) 2024-04-22T20:07:49.206 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached botocore-1.34.89-py3-none-any.whl (12.2 MB) 2024-04-22T20:07:49.227 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-04-22T20:07:49.228 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-04-22T20:07:49.230 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-04-22T20:07:49.232 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached urllib3-1.26.18-py2.py3-none-any.whl (143 kB) 2024-04-22T20:07:49.233 INFO:tasks.workunit.client.0.smithi136.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-04-22T20:07:49.336 INFO:tasks.workunit.client.0.smithi136.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-04-22T20:07:50.417 INFO:tasks.workunit.client.0.smithi136.stdout:Successfully installed boto3-1.34.89 botocore-1.34.89 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-1.26.18 2024-04-22T20:07:50.560 INFO:tasks.workunit.client.0.smithi136.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_reshard.py 2024-04-22T20:07:50.686 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin user create --uid tester --display-name "Testing" --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn 2024-04-22T20:07:50.721 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:07:50.721 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:07:50.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.756+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.758+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.758+0000 7f5320927800 20 realm 2024-04-22T20:07:50.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.758+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.758+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.758+0000 7f5320927800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:07:50.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.758+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.758+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.758+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.759+0000 7f5320927800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:07:50.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.759+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.760+0000 7f5320927800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:07:50.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.760+0000 7f5320927800 20 searching for the correct realm 2024-04-22T20:07:50.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.761+0000 7f531cd72640 0 -- 172.21.15.136:0/1455425853 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x563bbafecc20 msgr2=0x563bbafef010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:07:50.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.765+0000 7f5313d70640 0 -- 172.21.15.136:0/1455425853 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f530c02b060 msgr2=0x7f530c058770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:07:50.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.771+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:07:50.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 20 zone default found 2024-04-22T20:07:50.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 4 Realm: () 2024-04-22T20:07:50.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 20 zonegroup default 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.772+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.773+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.773+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.773+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.773+0000 7f5320927800 20 started sync module instance, tier type = 2024-04-22T20:07:50.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.773+0000 7f5320927800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:07:50.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.782+0000 7f5320927800 20 add_watcher() i=0 2024-04-22T20:07:50.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.782+0000 7f5320927800 20 add_watcher() i=1 2024-04-22T20:07:50.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.782+0000 7f5320927800 20 add_watcher() i=2 2024-04-22T20:07:50.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.782+0000 7f5320927800 20 add_watcher() i=3 2024-04-22T20:07:50.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.782+0000 7f5320927800 20 add_watcher() i=4 2024-04-22T20:07:50.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.782+0000 7f5320927800 20 add_watcher() i=5 2024-04-22T20:07:50.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.782+0000 7f5320927800 20 add_watcher() i=6 2024-04-22T20:07:50.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.783+0000 7f5320927800 20 add_watcher() i=7 2024-04-22T20:07:50.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.783+0000 7f5320927800 2 all 8 watchers are set, enabling cache 2024-04-22T20:07:50.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.784+0000 7f51d2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:07:50.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.784+0000 7f5320927800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:07:50.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.784+0000 7f5320927800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:07:50.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.784+0000 7f5320927800 5 note: GC not initialized 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f51d1fdb640 20 reqs_thread_entry: start 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 20 init_complete bucket index max shards: 1 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 20 Filter name: none 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f51437fe640 20 reqs_thread_entry: start 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 10 cache get: name=default.rgw.meta+users.uid+tester : miss 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x0 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 10 adding default.rgw.meta+users.uid+tester to cache LRU end 2024-04-22T20:07:50.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-04-22T20:07:50.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 20 rados->read ofs=0 len=0 2024-04-22T20:07:50.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:50.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-04-22T20:07:50.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-22T20:07:50.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-22T20:07:50.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.785+0000 7f5320927800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-22T20:07:50.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.787+0000 7f5320927800 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x17 2024-04-22T20:07:50.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.787+0000 7f5320927800 10 moving default.rgw.meta+users.uid+tester to cache LRU end 2024-04-22T20:07:50.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.787+0000 7f5320927800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester, ofs0, ns] 2024-04-22T20:07:50.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.787+0000 7f51f37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562389 cookie 94814540368336 notifier 5988 bl.length()=569 2024-04-22T20:07:50.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.787+0000 7f51f37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester info.flags=0x17 2024-04-22T20:07:50.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.787+0000 7f51f37fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester to cache LRU end 2024-04-22T20:07:50.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.790+0000 7f5320927800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-22T20:07:50.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.790+0000 7f5320927800 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-22T20:07:50.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.790+0000 7f5320927800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-22T20:07:50.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.790+0000 7f51f37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562390 cookie 94814540365888 notifier 5988 bl.length()=180 2024-04-22T20:07:50.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.790+0000 7f51f37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-22T20:07:50.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.790+0000 7f51f37fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-22T20:07:50.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.792+0000 7f51d2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:07:50.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.794+0000 7f5320927800 20 remove_watcher() i=0 2024-04-22T20:07:50.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.794+0000 7f5320927800 2 removed watcher, disabling cache 2024-04-22T20:07:50.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.796+0000 7f5320927800 20 remove_watcher() i=1 2024-04-22T20:07:50.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.797+0000 7f5320927800 20 remove_watcher() i=2 2024-04-22T20:07:50.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.798+0000 7f5320927800 20 remove_watcher() i=3 2024-04-22T20:07:50.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.799+0000 7f5320927800 20 remove_watcher() i=4 2024-04-22T20:07:50.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.800+0000 7f5320927800 20 remove_watcher() i=5 2024-04-22T20:07:50.799 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.800+0000 7f5320927800 20 remove_watcher() i=6 2024-04-22T20:07:50.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:50.801+0000 7f5320927800 20 remove_watcher() i=7 2024-04-22T20:07:59.044 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:07:59.044 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:07:59.094 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:07:59.094 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:07:59.135 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:07:59.135 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:07:59.175 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:07:59.175 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:07:59.205 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:07:59.205 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:07:59.265 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:07:59.265 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:07:59.320 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard add 2024-04-22T20:07:59.320 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:59.322 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-22T20:07:59.349 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:07:59.349 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:07:59.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.383+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.384+0000 7fad27010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.384+0000 7fad27010800 20 realm 2024-04-22T20:07:59.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.384+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.385+0000 7fad27010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.385+0000 7fad27010800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:07:59.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.385+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.385+0000 7fad27010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.385+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.386+0000 7fad27010800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:07:59.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.386+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.387+0000 7fad27010800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:07:59.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.387+0000 7fad27010800 20 searching for the correct realm 2024-04-22T20:07:59.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.389+0000 7fad22dbf640 0 -- 172.21.15.136:0/4284564476 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fad1c063aa0 msgr2=0x7fad1c062400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:07:59.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.394+0000 7fad21dbd640 0 -- 172.21.15.136:0/4284564476 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fad1c07e040 msgr2=0x7fad1c0960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:07:59.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:07:59.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:07:59.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:07:59.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:07:59.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:07:59.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:07:59.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.600+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:07:59.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 20 zone default found 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 4 Realm: () 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 20 zonegroup default 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.601+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.602+0000 7fad27010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.602+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.602+0000 7fad27010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.602+0000 7fad27010800 20 started sync module instance, tier type = 2024-04-22T20:07:59.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.602+0000 7fad27010800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:07:59.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.612+0000 7fad27010800 20 add_watcher() i=0 2024-04-22T20:07:59.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.612+0000 7fad27010800 20 add_watcher() i=1 2024-04-22T20:07:59.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.612+0000 7fad27010800 20 add_watcher() i=2 2024-04-22T20:07:59.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.612+0000 7fad27010800 20 add_watcher() i=3 2024-04-22T20:07:59.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.612+0000 7fad27010800 20 add_watcher() i=4 2024-04-22T20:07:59.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.612+0000 7fad27010800 20 add_watcher() i=5 2024-04-22T20:07:59.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.612+0000 7fad27010800 20 add_watcher() i=6 2024-04-22T20:07:59.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.613+0000 7fad27010800 20 add_watcher() i=7 2024-04-22T20:07:59.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.613+0000 7fad27010800 2 all 8 watchers are set, enabling cache 2024-04-22T20:07:59.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.614+0000 7fabd8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:07:59.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.615+0000 7fad27010800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:07:59.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.615+0000 7fad27010800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:07:59.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.615+0000 7fad27010800 5 note: GC not initialized 2024-04-22T20:07:59.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.615+0000 7fab537fe640 20 reqs_thread_entry: start 2024-04-22T20:07:59.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.615+0000 7fad27010800 20 init_complete bucket index max shards: 1 2024-04-22T20:07:59.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.615+0000 7fad27010800 20 Filter name: none 2024-04-22T20:07:59.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.615+0000 7fab517fa640 20 reqs_thread_entry: start 2024-04-22T20:07:59.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.615+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.616+0000 7fad27010800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:07:59.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.616+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.617+0000 7fad27010800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:07:59.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.617+0000 7fad27010800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:07:59.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.617+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.617+0000 7fad27010800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:07:59.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.617+0000 7fad27010800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.617+0000 7fad27010800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:07:59.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.617+0000 7fad27010800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:07:59.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.619+0000 7fabd8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:07:59.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.621+0000 7fad27010800 20 remove_watcher() i=0 2024-04-22T20:07:59.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.621+0000 7fad27010800 2 removed watcher, disabling cache 2024-04-22T20:07:59.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.622+0000 7fad27010800 20 remove_watcher() i=1 2024-04-22T20:07:59.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.623+0000 7fad27010800 20 remove_watcher() i=2 2024-04-22T20:07:59.624 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.624+0000 7fad27010800 20 remove_watcher() i=3 2024-04-22T20:07:59.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.625+0000 7fad27010800 20 remove_watcher() i=4 2024-04-22T20:07:59.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.626+0000 7fad27010800 20 remove_watcher() i=5 2024-04-22T20:07:59.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.627+0000 7fad27010800 20 remove_watcher() i=6 2024-04-22T20:07:59.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.628+0000 7fad27010800 20 remove_watcher() i=7 2024-04-22T20:07:59.637 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:59.637 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:07:59.638 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: a-bucket id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 num_objects: 0 size_kb: 0 num_shards: 1 2024-04-22T20:07:59.638 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:07:59.639 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 2 2024-04-22T20:07:59.668 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:07:59.668 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:07:59.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.702+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.703+0000 7f3d4552e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.703+0000 7f3d4552e800 20 realm 2024-04-22T20:07:59.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.703+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.704+0000 7f3d4552e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.704+0000 7f3d4552e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:07:59.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.704+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.704+0000 7f3d4552e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.704+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.705+0000 7f3d4552e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:07:59.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.705+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.705+0000 7f3d4552e800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:07:59.705 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.705+0000 7f3d4552e800 20 searching for the correct realm 2024-04-22T20:07:59.707 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.707+0000 7f3d41a68640 0 -- 172.21.15.136:0/2112633156 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x5567e0575560 msgr2=0x5567e0577950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:07:59.711 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.711+0000 7f3d41a68640 0 -- 172.21.15.136:0/2112633156 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f3c0c07ba30 msgr2=0x7f3c0c07de60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:07:59.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:07:59.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:07:59.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:07:59.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:07:59.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:07:59.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:07:59.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.717+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:07:59.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:07:59.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 zone default found 2024-04-22T20:07:59.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 4 Realm: () 2024-04-22T20:07:59.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:07:59.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:07:59.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 zonegroup default 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.718+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.719+0000 7f3d4552e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.719+0000 7f3d4552e800 20 started sync module instance, tier type = 2024-04-22T20:07:59.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.719+0000 7f3d4552e800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:07:59.730 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.730+0000 7f3d4552e800 20 add_watcher() i=0 2024-04-22T20:07:59.730 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.730+0000 7f3d4552e800 20 add_watcher() i=1 2024-04-22T20:07:59.730 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.730+0000 7f3d4552e800 20 add_watcher() i=2 2024-04-22T20:07:59.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.730+0000 7f3d4552e800 20 add_watcher() i=3 2024-04-22T20:07:59.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.730+0000 7f3d4552e800 20 add_watcher() i=4 2024-04-22T20:07:59.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.730+0000 7f3d4552e800 20 add_watcher() i=5 2024-04-22T20:07:59.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.730+0000 7f3d4552e800 20 add_watcher() i=6 2024-04-22T20:07:59.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.731+0000 7f3d4552e800 20 add_watcher() i=7 2024-04-22T20:07:59.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.731+0000 7f3d4552e800 2 all 8 watchers are set, enabling cache 2024-04-22T20:07:59.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.733+0000 7f3bf77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:07:59.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.733+0000 7f3d4552e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:07:59.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.733+0000 7f3d4552e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:07:59.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.733+0000 7f3d4552e800 5 note: GC not initialized 2024-04-22T20:07:59.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.733+0000 7f3bf67e4640 20 reqs_thread_entry: start 2024-04-22T20:07:59.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 20 init_complete bucket index max shards: 1 2024-04-22T20:07:59.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 20 Filter name: none 2024-04-22T20:07:59.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3b67fff640 20 reqs_thread_entry: start 2024-04-22T20:07:59.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-22T20:07:59.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:07:59.737 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-04-22T20:07:59.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-22T20:07:59.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 : miss 2024-04-22T20:07:59.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.734+0000 7f3d4552e800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.735+0000 7f3d4552e800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:07:59.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.735+0000 7f3d4552e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:07:59.738 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.735+0000 7f3d4552e800 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:07:59.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.735+0000 7f3d4552e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:07:59.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.735+0000 7f3d4552e800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:07:59.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.735+0000 7f3d4552e800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 2024-04-22T20:07:59.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.737+0000 7f3bf77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:07:59.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.739+0000 7f3d4552e800 20 remove_watcher() i=0 2024-04-22T20:07:59.740 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.739+0000 7f3d4552e800 2 removed watcher, disabling cache 2024-04-22T20:07:59.741 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.741+0000 7f3d4552e800 20 remove_watcher() i=1 2024-04-22T20:07:59.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.742+0000 7f3d4552e800 20 remove_watcher() i=2 2024-04-22T20:07:59.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.743+0000 7f3d4552e800 20 remove_watcher() i=3 2024-04-22T20:07:59.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.744+0000 7f3d4552e800 20 remove_watcher() i=4 2024-04-22T20:07:59.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.745+0000 7f3d4552e800 20 remove_watcher() i=5 2024-04-22T20:07:59.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.746+0000 7f3d4552e800 20 remove_watcher() i=6 2024-04-22T20:07:59.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.747+0000 7f3d4552e800 20 remove_watcher() i=7 2024-04-22T20:07:59.758 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin reshard list 2024-04-22T20:07:59.783 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:07:59.783 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:07:59.817 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.817+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.818+0000 7f9e44c1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.818+0000 7f9e44c1f800 20 realm 2024-04-22T20:07:59.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.818+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.819+0000 7f9e44c1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.819+0000 7f9e44c1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:07:59.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.819+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.819+0000 7f9e44c1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:07:59.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.819+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.820+0000 7f9e44c1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:07:59.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.820+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:07:59.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.820+0000 7f9e44c1f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:07:59.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.820+0000 7f9e44c1f800 20 searching for the correct realm 2024-04-22T20:07:59.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:07:59.822+0000 7f9e42f1f640 0 -- 172.21.15.136:0/798076487 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f9e3c03a080 msgr2=0x7f9e3c035400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:00.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.028+0000 7f9e41f1d640 0 -- 172.21.15.136:0/798076487 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f9e3c0183c0 msgr2=0x7f9e3c018840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:00.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.234+0000 7f9e44c1f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:00.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.234+0000 7f9e44c1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:00.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.234+0000 7f9e44c1f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:00.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.234+0000 7f9e44c1f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:00.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.234+0000 7f9e44c1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:00.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.234+0000 7f9e44c1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:00.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.234+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.235+0000 7f9e44c1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.235+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.235+0000 7f9e44c1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:00.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.235+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.235+0000 7f9e44c1f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 zone default found 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 4 Realm: () 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 zonegroup default 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 started sync module instance, tier type = 2024-04-22T20:08:00.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.236+0000 7f9e44c1f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:00.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.246+0000 7f9e44c1f800 20 add_watcher() i=0 2024-04-22T20:08:00.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.246+0000 7f9e44c1f800 20 add_watcher() i=1 2024-04-22T20:08:00.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.246+0000 7f9e44c1f800 20 add_watcher() i=2 2024-04-22T20:08:00.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.246+0000 7f9e44c1f800 20 add_watcher() i=3 2024-04-22T20:08:00.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.246+0000 7f9e44c1f800 20 add_watcher() i=4 2024-04-22T20:08:00.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.246+0000 7f9e44c1f800 20 add_watcher() i=5 2024-04-22T20:08:00.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.246+0000 7f9e44c1f800 20 add_watcher() i=6 2024-04-22T20:08:00.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.247+0000 7f9e44c1f800 20 add_watcher() i=7 2024-04-22T20:08:00.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.247+0000 7f9e44c1f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:00.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.249+0000 7f9cf8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:00.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.249+0000 7f9e44c1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:00.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.249+0000 7f9e44c1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:00.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.249+0000 7f9e44c1f800 5 note: GC not initialized 2024-04-22T20:08:00.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.249+0000 7f9c737fe640 20 reqs_thread_entry: start 2024-04-22T20:08:00.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.249+0000 7f9e44c1f800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:00.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.249+0000 7f9e44c1f800 20 Filter name: none 2024-04-22T20:08:00.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.249+0000 7f9c717fa640 20 reqs_thread_entry: start 2024-04-22T20:08:00.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.255+0000 7f9cf8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:00.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.258+0000 7f9e44c1f800 20 remove_watcher() i=0 2024-04-22T20:08:00.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.258+0000 7f9e44c1f800 2 removed watcher, disabling cache 2024-04-22T20:08:00.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.259+0000 7f9e44c1f800 20 remove_watcher() i=1 2024-04-22T20:08:00.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.260+0000 7f9e44c1f800 20 remove_watcher() i=2 2024-04-22T20:08:00.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.261+0000 7f9e44c1f800 20 remove_watcher() i=3 2024-04-22T20:08:00.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.262+0000 7f9e44c1f800 20 remove_watcher() i=4 2024-04-22T20:08:00.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.263+0000 7f9e44c1f800 20 remove_watcher() i=5 2024-04-22T20:08:00.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.264+0000 7f9e44c1f800 20 remove_watcher() i=6 2024-04-22T20:08:00.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.264+0000 7f9e44c1f800 20 remove_watcher() i=7 2024-04-22T20:08:00.274 INFO:tasks.workunit.client.0.smithi136.stderr:bucket name a-bucket 2024-04-22T20:08:00.274 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard process 2024-04-22T20:08:00.274 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:00.276 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin reshard process 2024-04-22T20:08:00.301 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:00.302 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:00.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.335+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.337+0000 7f628011f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.337+0000 7f628011f800 20 realm 2024-04-22T20:08:00.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.337+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.337+0000 7f628011f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.337+0000 7f628011f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:00.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.337+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.337+0000 7f628011f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.337+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.338+0000 7f628011f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:00.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.338+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.338+0000 7f628011f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:00.338 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.338+0000 7f628011f800 20 searching for the correct realm 2024-04-22T20:08:00.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.340+0000 7f627c5c4640 0 -- 172.21.15.136:0/625668330 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55944cc76150 msgr2=0x55944cc78540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:00.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.344+0000 7f627c5c4640 0 -- 172.21.15.136:0/625668330 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f614407b980 msgr2=0x7f614407ddb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:00.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:00.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:00.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:00.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:00.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:00.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:00.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.349+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 20 zone default found 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 4 Realm: () 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 20 zonegroup default 2024-04-22T20:08:00.351 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.350+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.351+0000 7f628011f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.351+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.351+0000 7f628011f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.351+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.352+0000 7f628011f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:00.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.352+0000 7f628011f800 20 started sync module instance, tier type = 2024-04-22T20:08:00.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.352+0000 7f628011f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:00.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.361+0000 7f628011f800 20 add_watcher() i=0 2024-04-22T20:08:00.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.361+0000 7f628011f800 20 add_watcher() i=1 2024-04-22T20:08:00.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.361+0000 7f628011f800 20 add_watcher() i=2 2024-04-22T20:08:00.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.361+0000 7f628011f800 20 add_watcher() i=3 2024-04-22T20:08:00.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.361+0000 7f628011f800 20 add_watcher() i=4 2024-04-22T20:08:00.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.361+0000 7f628011f800 20 add_watcher() i=5 2024-04-22T20:08:00.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.361+0000 7f628011f800 20 add_watcher() i=6 2024-04-22T20:08:00.362 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.362+0000 7f628011f800 20 add_watcher() i=7 2024-04-22T20:08:00.362 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.362+0000 7f628011f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:00.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.364+0000 7f6131fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:00.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.364+0000 7f628011f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:00.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.364+0000 7f628011f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:00.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.364+0000 7f628011f800 5 note: GC not initialized 2024-04-22T20:08:00.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.365+0000 7f6130fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:00.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.365+0000 7f628011f800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:00.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.365+0000 7f628011f800 20 Filter name: none 2024-04-22T20:08:00.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.365+0000 7f60aa7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:00.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.365+0000 7f628011f800 20 processing logshard = reshard.0000000000 2024-04-22T20:08:00.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.368+0000 7f628011f800 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-04-22T20:08:00.368 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.368+0000 7f628011f800 20 processing logshard = reshard.0000000001 2024-04-22T20:08:00.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.371+0000 7f628011f800 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-04-22T20:08:00.371 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.371+0000 7f628011f800 20 processing logshard = reshard.0000000002 2024-04-22T20:08:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.373+0000 7f628011f800 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-04-22T20:08:00.373 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.373+0000 7f628011f800 20 processing logshard = reshard.0000000003 2024-04-22T20:08:00.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.376+0000 7f628011f800 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-04-22T20:08:00.376 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.376+0000 7f628011f800 20 processing logshard = reshard.0000000004 2024-04-22T20:08:00.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.378+0000 7f628011f800 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-04-22T20:08:00.378 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.378+0000 7f628011f800 20 processing logshard = reshard.0000000005 2024-04-22T20:08:00.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.380+0000 7f628011f800 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-04-22T20:08:00.380 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.380+0000 7f628011f800 20 processing logshard = reshard.0000000006 2024-04-22T20:08:00.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.382+0000 7f628011f800 20 process_entry resharding a-bucket 2024-04-22T20:08:00.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.382+0000 7f628011f800 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-22T20:08:00.382 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.382+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.382+0000 7f628011f800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.382+0000 7f628011f800 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.382+0000 7f628011f800 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.382+0000 7f628011f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 : miss 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.382+0000 7f628011f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.383+0000 7f628011f800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.383+0000 7f628011f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.383+0000 7f628011f800 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:00.383 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.383+0000 7f628011f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:00.384 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.383+0000 7f628011f800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:00.384 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.383+0000 7f628011f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 2024-04-22T20:08:00.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.388+0000 7f628011f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:00.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.388+0000 7f628011f800 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:00.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.388+0000 7f628011f800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:00.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.388+0000 7f628011f800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17, ofs0, ns] 2024-04-22T20:08:00.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.388+0000 7f615affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562377 cookie 94095431708016 notifier 6063 bl.length()=819 2024-04-22T20:08:00.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.388+0000 7f615affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:00.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.388+0000 7f615affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:00.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.388+0000 7f615affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:00.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.390+0000 7f628011f800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_sources= new_sources= 2024-04-22T20:08:00.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.390+0000 7f628011f800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential sources added=[] removed=[] 2024-04-22T20:08:00.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.390+0000 7f628011f800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_dests= new_dests= 2024-04-22T20:08:00.390 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.390+0000 7f628011f800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential dests added=[] removed=[] 2024-04-22T20:08:00.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.391+0000 7f628011f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.1.0 2024-04-22T20:08:00.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.391+0000 7f628011f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.1.0 2024-04-22T20:08:00.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.391+0000 7f628011f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.1.1 2024-04-22T20:08:00.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.391+0000 7f628011f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.1.1 2024-04-22T20:08:00.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.391+0000 7f628011f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.0 2024-04-22T20:08:00.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.391+0000 7f628011f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 : hit (requested=0x17, cached=0x17) 2024-04-22T20:08:00.392 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.391+0000 7f628011f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 2024-04-22T20:08:00.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.392+0000 7f628011f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:00.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.392+0000 7f628011f800 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:00.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.392+0000 7f628011f800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:00.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.392+0000 7f628011f800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17, ofs0, ns] 2024-04-22T20:08:00.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.393+0000 7f615a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562378 cookie 94095431708016 notifier 6063 bl.length()=787 2024-04-22T20:08:00.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.393+0000 7f615a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:00.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.393+0000 7f615a7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:00.393 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.393+0000 7f615a7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:00.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.394+0000 7f628011f800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_sources= new_sources= 2024-04-22T20:08:00.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.394+0000 7f628011f800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential sources added=[] removed=[] 2024-04-22T20:08:00.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.394+0000 7f628011f800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_dests= new_dests= 2024-04-22T20:08:00.394 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.394+0000 7f628011f800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential dests added=[] removed=[] 2024-04-22T20:08:00.395 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.395+0000 7f628011f800 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-04-22T20:08:00.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.396+0000 7f628011f800 20 process_entry removing reshard queue entry for bucket a-bucket 2024-04-22T20:08:00.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.398+0000 7f628011f800 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-04-22T20:08:00.398 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.398+0000 7f628011f800 20 processing logshard = reshard.0000000007 2024-04-22T20:08:00.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.401+0000 7f628011f800 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-04-22T20:08:00.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.401+0000 7f628011f800 20 processing logshard = reshard.0000000008 2024-04-22T20:08:00.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.403+0000 7f628011f800 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-04-22T20:08:00.403 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.403+0000 7f628011f800 20 processing logshard = reshard.0000000009 2024-04-22T20:08:00.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.405+0000 7f628011f800 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-04-22T20:08:00.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.405+0000 7f628011f800 20 processing logshard = reshard.0000000010 2024-04-22T20:08:00.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.407+0000 7f628011f800 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-04-22T20:08:00.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.407+0000 7f628011f800 20 processing logshard = reshard.0000000011 2024-04-22T20:08:00.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.409+0000 7f628011f800 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-04-22T20:08:00.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.409+0000 7f628011f800 20 processing logshard = reshard.0000000012 2024-04-22T20:08:00.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.412+0000 7f628011f800 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-04-22T20:08:00.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.412+0000 7f628011f800 20 processing logshard = reshard.0000000013 2024-04-22T20:08:00.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.414+0000 7f628011f800 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-04-22T20:08:00.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.414+0000 7f628011f800 20 processing logshard = reshard.0000000014 2024-04-22T20:08:00.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.416+0000 7f628011f800 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-04-22T20:08:00.416 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.416+0000 7f628011f800 20 processing logshard = reshard.0000000015 2024-04-22T20:08:00.418 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.418+0000 7f628011f800 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-04-22T20:08:00.419 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.419+0000 7f6131fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:00.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.421+0000 7f628011f800 20 remove_watcher() i=0 2024-04-22T20:08:00.421 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.421+0000 7f628011f800 2 removed watcher, disabling cache 2024-04-22T20:08:00.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.422+0000 7f628011f800 20 remove_watcher() i=1 2024-04-22T20:08:00.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.423+0000 7f628011f800 20 remove_watcher() i=2 2024-04-22T20:08:00.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.424+0000 7f628011f800 20 remove_watcher() i=3 2024-04-22T20:08:00.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.425+0000 7f628011f800 20 remove_watcher() i=4 2024-04-22T20:08:00.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.426+0000 7f628011f800 20 remove_watcher() i=5 2024-04-22T20:08:00.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.427+0000 7f628011f800 20 remove_watcher() i=6 2024-04-22T20:08:00.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:00.428+0000 7f628011f800 20 remove_watcher() i=7 2024-04-22T20:08:05.445 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-22T20:08:05.478 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:05.479 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:05.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.719+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:05.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.720+0000 7fbc3e611800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:05.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.720+0000 7fbc3e611800 20 realm 2024-04-22T20:08:05.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.720+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:05.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.721+0000 7fbc3e611800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:05.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.721+0000 7fbc3e611800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:05.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.721+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:05.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.721+0000 7fbc3e611800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:05.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.721+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:05.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.722+0000 7fbc3e611800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:05.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.722+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:05.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.722+0000 7fbc3e611800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:05.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.723+0000 7fbc3e611800 20 searching for the correct realm 2024-04-22T20:08:05.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.725+0000 7fbc3a415640 0 -- 172.21.15.136:0/2048540895 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fbc34007b20 msgr2=0x7fbc34077f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:05.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:05.932+0000 7fbc31413640 0 -- 172.21.15.136:0/2048540895 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fbc340c5b60 msgr2=0x7fbc340819d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:06.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:06.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:06.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:06.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:06.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:06.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:06.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.139+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 20 zone default found 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 4 Realm: () 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 20 zonegroup default 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.140+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.141+0000 7fbc3e611800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.141+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.141+0000 7fbc3e611800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.141+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.141+0000 7fbc3e611800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.141+0000 7fbc3e611800 20 started sync module instance, tier type = 2024-04-22T20:08:06.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.141+0000 7fbc3e611800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:06.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.152+0000 7fbc3e611800 20 add_watcher() i=0 2024-04-22T20:08:06.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.152+0000 7fbc3e611800 20 add_watcher() i=1 2024-04-22T20:08:06.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.152+0000 7fbc3e611800 20 add_watcher() i=2 2024-04-22T20:08:06.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.152+0000 7fbc3e611800 20 add_watcher() i=3 2024-04-22T20:08:06.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.152+0000 7fbc3e611800 20 add_watcher() i=4 2024-04-22T20:08:06.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.152+0000 7fbc3e611800 20 add_watcher() i=5 2024-04-22T20:08:06.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.152+0000 7fbc3e611800 20 add_watcher() i=6 2024-04-22T20:08:06.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.153+0000 7fbc3e611800 20 add_watcher() i=7 2024-04-22T20:08:06.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.153+0000 7fbc3e611800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:06.155 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.155+0000 7fbaeb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:06.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.156+0000 7fbc3e611800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:06.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.156+0000 7fbc3e611800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:06.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.156+0000 7fbc3e611800 5 note: GC not initialized 2024-04-22T20:08:06.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.156+0000 7fbaea7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:06.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.156+0000 7fbc3e611800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:06.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.156+0000 7fbc3e611800 20 Filter name: none 2024-04-22T20:08:06.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.156+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.156+0000 7fba5bfff640 20 reqs_thread_entry: start 2024-04-22T20:08:06.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.157+0000 7fbc3e611800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:06.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.157+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.158+0000 7fbc3e611800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:06.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.158+0000 7fbc3e611800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:06.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.158+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.158+0000 7fbc3e611800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:06.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.158+0000 7fbc3e611800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.159+0000 7fbc3e611800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:06.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.159+0000 7fbc3e611800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:06.160 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.160+0000 7fbaeb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:06.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.162+0000 7fbc3e611800 20 remove_watcher() i=0 2024-04-22T20:08:06.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.162+0000 7fbc3e611800 2 removed watcher, disabling cache 2024-04-22T20:08:06.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.164+0000 7fbc3e611800 20 remove_watcher() i=1 2024-04-22T20:08:06.165 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.165+0000 7fbc3e611800 20 remove_watcher() i=2 2024-04-22T20:08:06.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.166+0000 7fbc3e611800 20 remove_watcher() i=3 2024-04-22T20:08:06.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.167+0000 7fbc3e611800 20 remove_watcher() i=4 2024-04-22T20:08:06.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.168+0000 7fbc3e611800 20 remove_watcher() i=5 2024-04-22T20:08:06.169 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.169+0000 7fbc3e611800 20 remove_watcher() i=6 2024-04-22T20:08:06.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.170+0000 7fbc3e611800 20 remove_watcher() i=7 2024-04-22T20:08:06.182 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:06.182 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:06.182 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: a-bucket id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 num_objects: 0 size_kb: 0 num_shards: 2 2024-04-22T20:08:06.182 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:06.182 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard add non empty bucket 2024-04-22T20:08:06.182 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:06.187 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:06.187 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:06.226 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:06.227 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:06.265 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:06.265 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:06.299 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:06.300 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:06.334 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:06.335 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:06.370 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:06.370 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:06.404 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:06.405 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:06.438 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:06.438 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:06.473 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-22T20:08:06.500 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:06.500 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:06.534 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.534+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.536+0000 7f89bc21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.536+0000 7f89bc21f800 20 realm 2024-04-22T20:08:06.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.536+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.536+0000 7f89bc21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.536+0000 7f89bc21f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:06.536 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.536+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.536+0000 7f89bc21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.536+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.537+0000 7f89bc21f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:06.537 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.537+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.538+0000 7f89bc21f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:06.538 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.538+0000 7f89bc21f800 20 searching for the correct realm 2024-04-22T20:08:06.739 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.739+0000 7f89b9d1e640 0 -- 172.21.15.136:0/1116518691 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x55939afe8e80 msgr2=0x55939afeb320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:06.744 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.743+0000 7f89b9d1e640 0 -- 172.21.15.136:0/1116518691 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x55939ae2ab20 msgr2=0x7f89b404d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.952+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.953+0000 7f89bc21f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:06.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.953+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:06.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 20 zone default found 2024-04-22T20:08:06.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 4 Realm: () 2024-04-22T20:08:06.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:06.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:06.955 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 20 zonegroup default 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.954+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.955+0000 7f89bc21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.955+0000 7f89bc21f800 20 started sync module instance, tier type = 2024-04-22T20:08:06.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.955+0000 7f89bc21f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:06.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.965+0000 7f89bc21f800 20 add_watcher() i=0 2024-04-22T20:08:06.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.965+0000 7f89bc21f800 20 add_watcher() i=1 2024-04-22T20:08:06.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.965+0000 7f89bc21f800 20 add_watcher() i=2 2024-04-22T20:08:06.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.965+0000 7f89bc21f800 20 add_watcher() i=3 2024-04-22T20:08:06.965 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.965+0000 7f89bc21f800 20 add_watcher() i=4 2024-04-22T20:08:06.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.965+0000 7f89bc21f800 20 add_watcher() i=5 2024-04-22T20:08:06.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.965+0000 7f89bc21f800 20 add_watcher() i=6 2024-04-22T20:08:06.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.966+0000 7f89bc21f800 20 add_watcher() i=7 2024-04-22T20:08:06.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.966+0000 7f89bc21f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:06.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.967+0000 7f886b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:06.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.968+0000 7f89bc21f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:06.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.968+0000 7f89bc21f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:06.968 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.968+0000 7f89bc21f800 5 note: GC not initialized 2024-04-22T20:08:06.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.968+0000 7f886a7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:06.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.968+0000 7f89bc21f800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:06.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.968+0000 7f89bc21f800 20 Filter name: none 2024-04-22T20:08:06.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.968+0000 7f87dbfff640 20 reqs_thread_entry: start 2024-04-22T20:08:06.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.968+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.969+0000 7f89bc21f800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:06.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.969+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.970+0000 7f89bc21f800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:06.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.970+0000 7f89bc21f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:06.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.970+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.970+0000 7f89bc21f800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:06.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.970+0000 7f89bc21f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:06.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.971+0000 7f89bc21f800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:06.971 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.971+0000 7f89bc21f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:06.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.972+0000 7f886b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:06.974 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.974+0000 7f89bc21f800 20 remove_watcher() i=0 2024-04-22T20:08:06.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.974+0000 7f89bc21f800 2 removed watcher, disabling cache 2024-04-22T20:08:06.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.975+0000 7f89bc21f800 20 remove_watcher() i=1 2024-04-22T20:08:06.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.976+0000 7f89bc21f800 20 remove_watcher() i=2 2024-04-22T20:08:06.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.978+0000 7f89bc21f800 20 remove_watcher() i=3 2024-04-22T20:08:06.979 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.979+0000 7f89bc21f800 20 remove_watcher() i=4 2024-04-22T20:08:06.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.980+0000 7f89bc21f800 20 remove_watcher() i=5 2024-04-22T20:08:06.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.981+0000 7f89bc21f800 20 remove_watcher() i=6 2024-04-22T20:08:06.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:06.982+0000 7f89bc21f800 20 remove_watcher() i=7 2024-04-22T20:08:06.992 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:06.992 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:06.992 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: a-bucket id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 num_objects: 8 size_kb: 1 num_shards: 2 2024-04-22T20:08:06.992 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:06.993 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 3 2024-04-22T20:08:07.023 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:07.023 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:07.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.057+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.058+0000 7efd8c010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.058+0000 7efd8c010800 20 realm 2024-04-22T20:08:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.058+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.058+0000 7efd8c010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.058+0000 7efd8c010800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.058+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.059+0000 7efd8c010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.059+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.059+0000 7efd8c010800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:07.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.059+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.060+0000 7efd8c010800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:07.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.060+0000 7efd8c010800 20 searching for the correct realm 2024-04-22T20:08:07.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.062+0000 7efd87f5d640 0 -- 172.21.15.136:0/1901127538 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7efd80037070 msgr2=0x7efd800601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:07.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.268+0000 7efd7ef5b640 0 -- 172.21.15.136:0/1901127538 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7efd80069a50 msgr2=0x7efd80037830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.474+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 20 zone default found 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 4 Realm: () 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 20 zonegroup default 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.475+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.476+0000 7efd8c010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.476+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.476+0000 7efd8c010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.476+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.476+0000 7efd8c010800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.476+0000 7efd8c010800 20 started sync module instance, tier type = 2024-04-22T20:08:07.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.476+0000 7efd8c010800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:07.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.486+0000 7efd8c010800 20 add_watcher() i=0 2024-04-22T20:08:07.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.486+0000 7efd8c010800 20 add_watcher() i=1 2024-04-22T20:08:07.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.486+0000 7efd8c010800 20 add_watcher() i=2 2024-04-22T20:08:07.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.486+0000 7efd8c010800 20 add_watcher() i=3 2024-04-22T20:08:07.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.486+0000 7efd8c010800 20 add_watcher() i=4 2024-04-22T20:08:07.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.486+0000 7efd8c010800 20 add_watcher() i=5 2024-04-22T20:08:07.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.486+0000 7efd8c010800 20 add_watcher() i=6 2024-04-22T20:08:07.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.487+0000 7efd8c010800 20 add_watcher() i=7 2024-04-22T20:08:07.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.488+0000 7efd8c010800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:07.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.490+0000 7efc3dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.490+0000 7efd8c010800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.490+0000 7efd8c010800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.490+0000 7efd8c010800 5 note: GC not initialized 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.490+0000 7efc3cfe1640 20 reqs_thread_entry: start 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.490+0000 7efd8c010800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.490+0000 7efd8c010800 20 Filter name: none 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.491+0000 7efbb67fc640 20 reqs_thread_entry: start 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.491+0000 7efd8c010800 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-22T20:08:07.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.491+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.491+0000 7efd8c010800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:07.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.491+0000 7efd8c010800 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-04-22T20:08:07.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.491+0000 7efd8c010800 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-22T20:08:07.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.491+0000 7efd8c010800 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 : miss 2024-04-22T20:08:07.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.491+0000 7efd8c010800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.492+0000 7efd8c010800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:07.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.492+0000 7efd8c010800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:07.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.492+0000 7efd8c010800 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:07.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.492+0000 7efd8c010800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:07.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.492+0000 7efd8c010800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:07.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.492+0000 7efd8c010800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 2024-04-22T20:08:07.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.494+0000 7efc3dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:07.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.496+0000 7efd8c010800 20 remove_watcher() i=0 2024-04-22T20:08:07.496 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.496+0000 7efd8c010800 2 removed watcher, disabling cache 2024-04-22T20:08:07.497 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.497+0000 7efd8c010800 20 remove_watcher() i=1 2024-04-22T20:08:07.498 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.498+0000 7efd8c010800 20 remove_watcher() i=2 2024-04-22T20:08:07.500 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.500+0000 7efd8c010800 20 remove_watcher() i=3 2024-04-22T20:08:07.501 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.501+0000 7efd8c010800 20 remove_watcher() i=4 2024-04-22T20:08:07.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.502+0000 7efd8c010800 20 remove_watcher() i=5 2024-04-22T20:08:07.502 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.502+0000 7efd8c010800 20 remove_watcher() i=6 2024-04-22T20:08:07.503 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.503+0000 7efd8c010800 20 remove_watcher() i=7 2024-04-22T20:08:07.515 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin reshard list 2024-04-22T20:08:07.541 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:07.541 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:07.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.575+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.576+0000 7fe43223d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.576+0000 7fe43223d800 20 realm 2024-04-22T20:08:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.576+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.577+0000 7fe43223d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.577+0000 7fe43223d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.577+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.577+0000 7fe43223d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.577+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.578+0000 7fe43223d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:07.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.578+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.578+0000 7fe43223d800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:07.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.578+0000 7fe43223d800 20 searching for the correct realm 2024-04-22T20:08:07.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.580+0000 7fe43053d640 0 -- 172.21.15.136:0/3761898788 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fe428089bc0 msgr2=0x7fe42800cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:07.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.786+0000 7fe427fff640 0 -- 172.21.15.136:0/3761898788 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fe428012c60 msgr2=0x7fe42807e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.992+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.994 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 20 zone default found 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 4 Realm: () 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 20 zonegroup default 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.993+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.994+0000 7fe43223d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.994+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.994+0000 7fe43223d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.994+0000 7fe43223d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:07.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.994+0000 7fe43223d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:07.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.994+0000 7fe43223d800 20 started sync module instance, tier type = 2024-04-22T20:08:07.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:07.994+0000 7fe43223d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:08.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.004+0000 7fe43223d800 20 add_watcher() i=0 2024-04-22T20:08:08.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.004+0000 7fe43223d800 20 add_watcher() i=1 2024-04-22T20:08:08.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.004+0000 7fe43223d800 20 add_watcher() i=2 2024-04-22T20:08:08.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.004+0000 7fe43223d800 20 add_watcher() i=3 2024-04-22T20:08:08.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.004+0000 7fe43223d800 20 add_watcher() i=4 2024-04-22T20:08:08.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.004+0000 7fe43223d800 20 add_watcher() i=5 2024-04-22T20:08:08.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.004+0000 7fe43223d800 20 add_watcher() i=6 2024-04-22T20:08:08.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.005+0000 7fe43223d800 20 add_watcher() i=7 2024-04-22T20:08:08.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.005+0000 7fe43223d800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:08.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.007+0000 7fe2e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:08.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.008+0000 7fe43223d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:08.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.008+0000 7fe43223d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:08.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.008+0000 7fe43223d800 5 note: GC not initialized 2024-04-22T20:08:08.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.008+0000 7fe2e4fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:08.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.008+0000 7fe43223d800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:08.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.008+0000 7fe43223d800 20 Filter name: none 2024-04-22T20:08:08.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.008+0000 7fe25e7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:08.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.015+0000 7fe2e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:08.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.017+0000 7fe43223d800 20 remove_watcher() i=0 2024-04-22T20:08:08.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.017+0000 7fe43223d800 2 removed watcher, disabling cache 2024-04-22T20:08:08.018 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.018+0000 7fe43223d800 20 remove_watcher() i=1 2024-04-22T20:08:08.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.019+0000 7fe43223d800 20 remove_watcher() i=2 2024-04-22T20:08:08.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.021+0000 7fe43223d800 20 remove_watcher() i=3 2024-04-22T20:08:08.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.022+0000 7fe43223d800 20 remove_watcher() i=4 2024-04-22T20:08:08.023 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.023+0000 7fe43223d800 20 remove_watcher() i=5 2024-04-22T20:08:08.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.024+0000 7fe43223d800 20 remove_watcher() i=6 2024-04-22T20:08:08.025 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.025+0000 7fe43223d800 20 remove_watcher() i=7 2024-04-22T20:08:08.035 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard process non empty bucket 2024-04-22T20:08:08.035 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:08.036 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin reshard process 2024-04-22T20:08:08.062 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:08.062 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:08.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.096+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.097+0000 7f2430487800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.097+0000 7f2430487800 20 realm 2024-04-22T20:08:08.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.097+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.098+0000 7f2430487800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.098+0000 7f2430487800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:08.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.098+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.098+0000 7f2430487800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.098+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.099 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.099+0000 7f2430487800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:08.099 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.099+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.099+0000 7f2430487800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:08.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.099+0000 7f2430487800 20 searching for the correct realm 2024-04-22T20:08:08.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.102+0000 7f242e787640 0 -- 172.21.15.136:0/1045192959 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f2428026070 msgr2=0x7f242805d8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:08.105 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.105+0000 7f242e787640 0 -- 172.21.15.136:0/1045192959 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x5619f2a42ee0 msgr2=0x5619f2a452d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:08.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:08.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:08.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:08.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:08.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:08.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:08.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:08.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:08.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 zone default found 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 4 Realm: () 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 zonegroup default 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.111+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.112+0000 7f2430487800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.112+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.112+0000 7f2430487800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.112+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.114 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.112+0000 7f2430487800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.112+0000 7f2430487800 20 started sync module instance, tier type = 2024-04-22T20:08:08.115 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.112+0000 7f2430487800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:08.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.121+0000 7f2430487800 20 add_watcher() i=0 2024-04-22T20:08:08.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.121+0000 7f2430487800 20 add_watcher() i=1 2024-04-22T20:08:08.121 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.121+0000 7f2430487800 20 add_watcher() i=2 2024-04-22T20:08:08.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.121+0000 7f2430487800 20 add_watcher() i=3 2024-04-22T20:08:08.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.121+0000 7f2430487800 20 add_watcher() i=4 2024-04-22T20:08:08.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.121+0000 7f2430487800 20 add_watcher() i=5 2024-04-22T20:08:08.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.121+0000 7f2430487800 20 add_watcher() i=6 2024-04-22T20:08:08.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.122+0000 7f2430487800 20 add_watcher() i=7 2024-04-22T20:08:08.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.122+0000 7f2430487800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:08.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.123+0000 7f22d77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.124+0000 7f2430487800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.124+0000 7f2430487800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.124+0000 7f2430487800 5 note: GC not initialized 2024-04-22T20:08:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.124+0000 7f22d67fc640 20 reqs_thread_entry: start 2024-04-22T20:08:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.124+0000 7f2430487800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.124+0000 7f2430487800 20 Filter name: none 2024-04-22T20:08:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.124+0000 7f224ffff640 20 reqs_thread_entry: start 2024-04-22T20:08:08.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.124+0000 7f2430487800 20 processing logshard = reshard.0000000000 2024-04-22T20:08:08.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.127+0000 7f2430487800 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-04-22T20:08:08.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.127+0000 7f2430487800 20 processing logshard = reshard.0000000001 2024-04-22T20:08:08.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.130+0000 7f2430487800 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-04-22T20:08:08.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.130+0000 7f2430487800 20 processing logshard = reshard.0000000002 2024-04-22T20:08:08.132 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.132+0000 7f2430487800 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-04-22T20:08:08.132 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.132+0000 7f2430487800 20 processing logshard = reshard.0000000003 2024-04-22T20:08:08.134 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.134+0000 7f2430487800 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-04-22T20:08:08.134 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.134+0000 7f2430487800 20 processing logshard = reshard.0000000004 2024-04-22T20:08:08.137 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.137+0000 7f2430487800 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-04-22T20:08:08.137 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.137+0000 7f2430487800 20 processing logshard = reshard.0000000005 2024-04-22T20:08:08.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.139+0000 7f2430487800 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-04-22T20:08:08.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.139+0000 7f2430487800 20 processing logshard = reshard.0000000006 2024-04-22T20:08:08.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 20 process_entry resharding a-bucket 2024-04-22T20:08:08.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-22T20:08:08.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:08.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-04-22T20:08:08.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-22T20:08:08.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 : miss 2024-04-22T20:08:08.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:08.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:08.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:08.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:08.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:08.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.141+0000 7f2430487800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 2024-04-22T20:08:08.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.147+0000 7f2430487800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:08.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.147+0000 7f2430487800 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:08.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.147+0000 7f2430487800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:08.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.147+0000 7f2430487800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17, ofs0, ns] 2024-04-22T20:08:08.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.147+0000 7f2310ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562379 cookie 94669445066208 notifier 5870 bl.length()=819 2024-04-22T20:08:08.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.147+0000 7f2310ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:08.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.147+0000 7f2310ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:08.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.147+0000 7f2310ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:08.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.149+0000 7f2430487800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_sources= new_sources= 2024-04-22T20:08:08.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.149+0000 7f2430487800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential sources added=[] removed=[] 2024-04-22T20:08:08.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.149+0000 7f2430487800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_dests= new_dests= 2024-04-22T20:08:08.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.149+0000 7f2430487800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential dests added=[] removed=[] 2024-04-22T20:08:08.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.0 2024-04-22T20:08:08.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.0 2024-04-22T20:08:08.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.1 2024-04-22T20:08:08.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.1 2024-04-22T20:08:08.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.2 2024-04-22T20:08:08.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.2 2024-04-22T20:08:08.150 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.1.0 2024-04-22T20:08:08.151 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.1.0 2024-04-22T20:08:08.151 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.1.1 2024-04-22T20:08:08.151 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.150+0000 7f2430487800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.1.1 2024-04-22T20:08:08.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.152+0000 7f2430487800 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 : hit (requested=0x17, cached=0x17) 2024-04-22T20:08:08.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.152+0000 7f2430487800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 2024-04-22T20:08:08.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.154+0000 7f2430487800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:08.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.154+0000 7f2430487800 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:08.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.154+0000 7f2430487800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:08.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.154+0000 7f2430487800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17, ofs0, ns] 2024-04-22T20:08:08.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.154+0000 7f2310ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562380 cookie 94669445066208 notifier 5870 bl.length()=787 2024-04-22T20:08:08.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.154+0000 7f2310ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:08:08.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.154+0000 7f2310ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:08:08.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.154+0000 7f2310ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:08:08.155 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.155+0000 7f2430487800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_sources= new_sources= 2024-04-22T20:08:08.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.155+0000 7f2430487800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential sources added=[] removed=[] 2024-04-22T20:08:08.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.155+0000 7f2430487800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_dests= new_dests= 2024-04-22T20:08:08.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.155+0000 7f2430487800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential dests added=[] removed=[] 2024-04-22T20:08:08.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.157+0000 7f2430487800 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-04-22T20:08:08.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.158+0000 7f2430487800 20 process_entry removing reshard queue entry for bucket a-bucket 2024-04-22T20:08:08.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.160+0000 7f2430487800 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-04-22T20:08:08.160 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.160+0000 7f2430487800 20 processing logshard = reshard.0000000007 2024-04-22T20:08:08.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.162+0000 7f2430487800 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-04-22T20:08:08.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.162+0000 7f2430487800 20 processing logshard = reshard.0000000008 2024-04-22T20:08:08.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.164+0000 7f2430487800 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-04-22T20:08:08.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.164+0000 7f2430487800 20 processing logshard = reshard.0000000009 2024-04-22T20:08:08.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.166+0000 7f2430487800 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-04-22T20:08:08.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.166+0000 7f2430487800 20 processing logshard = reshard.0000000010 2024-04-22T20:08:08.169 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.169+0000 7f2430487800 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-04-22T20:08:08.169 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.169+0000 7f2430487800 20 processing logshard = reshard.0000000011 2024-04-22T20:08:08.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.171+0000 7f2430487800 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-04-22T20:08:08.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.171+0000 7f2430487800 20 processing logshard = reshard.0000000012 2024-04-22T20:08:08.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.174+0000 7f2430487800 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-04-22T20:08:08.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.174+0000 7f2430487800 20 processing logshard = reshard.0000000013 2024-04-22T20:08:08.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.177+0000 7f2430487800 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-04-22T20:08:08.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.177+0000 7f2430487800 20 processing logshard = reshard.0000000014 2024-04-22T20:08:08.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.179+0000 7f2430487800 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-04-22T20:08:08.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.179+0000 7f2430487800 20 processing logshard = reshard.0000000015 2024-04-22T20:08:08.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.181+0000 7f2430487800 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-04-22T20:08:08.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.182+0000 7f22d77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:08.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.184+0000 7f2430487800 20 remove_watcher() i=0 2024-04-22T20:08:08.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.184+0000 7f2430487800 2 removed watcher, disabling cache 2024-04-22T20:08:08.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.186+0000 7f2430487800 20 remove_watcher() i=1 2024-04-22T20:08:08.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.187+0000 7f2430487800 20 remove_watcher() i=2 2024-04-22T20:08:08.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.188+0000 7f2430487800 20 remove_watcher() i=3 2024-04-22T20:08:08.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.189+0000 7f2430487800 20 remove_watcher() i=4 2024-04-22T20:08:08.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.190+0000 7f2430487800 20 remove_watcher() i=5 2024-04-22T20:08:08.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.190+0000 7f2430487800 20 remove_watcher() i=6 2024-04-22T20:08:08.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.191+0000 7f2430487800 20 remove_watcher() i=7 2024-04-22T20:08:08.203 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-22T20:08:08.230 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:08.230 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:08.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.263+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.265+0000 7f03c412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.265+0000 7f03c412b800 20 realm 2024-04-22T20:08:08.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.265+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.265+0000 7f03c412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.265+0000 7f03c412b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:08.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.265+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.265+0000 7f03c412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.265+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.266+0000 7f03c412b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:08.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.266+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.266+0000 7f03c412b800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:08.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.266+0000 7f03c412b800 20 searching for the correct realm 2024-04-22T20:08:08.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.268+0000 7f03c06f0640 0 -- 172.21.15.136:0/568333549 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x563e742a8bc0 msgr2=0x563e742aafb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:08.272 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.271+0000 7f03c06f0640 0 -- 172.21.15.136:0/568333549 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f028807ba00 msgr2=0x7f028807de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.276+0000 7f03c412b800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.276+0000 7f03c412b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.276+0000 7f03c412b800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.276+0000 7f03c412b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.276+0000 7f03c412b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.276+0000 7f03c412b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.276+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:08.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 zone default found 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 4 Realm: () 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 zonegroup default 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.277+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.278+0000 7f03c412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.278+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.278+0000 7f03c412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.278+0000 7f03c412b800 20 started sync module instance, tier type = 2024-04-22T20:08:08.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.278+0000 7f03c412b800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:08.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.287+0000 7f03c412b800 20 add_watcher() i=0 2024-04-22T20:08:08.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.287+0000 7f03c412b800 20 add_watcher() i=1 2024-04-22T20:08:08.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.287+0000 7f03c412b800 20 add_watcher() i=2 2024-04-22T20:08:08.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.287+0000 7f03c412b800 20 add_watcher() i=3 2024-04-22T20:08:08.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.287+0000 7f03c412b800 20 add_watcher() i=4 2024-04-22T20:08:08.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.287+0000 7f03c412b800 20 add_watcher() i=5 2024-04-22T20:08:08.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.287+0000 7f03c412b800 20 add_watcher() i=6 2024-04-22T20:08:08.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.288+0000 7f03c412b800 20 add_watcher() i=7 2024-04-22T20:08:08.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.288+0000 7f03c412b800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:08.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.290+0000 7f0275fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:08.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.290+0000 7f03c412b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:08.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.290+0000 7f03c412b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:08.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.290+0000 7f03c412b800 5 note: GC not initialized 2024-04-22T20:08:08.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.290+0000 7f0274fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:08.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.290+0000 7f03c412b800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:08.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.291+0000 7f03c412b800 20 Filter name: none 2024-04-22T20:08:08.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.291+0000 7f01ee7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:08.291 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.291+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.291+0000 7f03c412b800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:08.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.291+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.292+0000 7f03c412b800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:08.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.292+0000 7f03c412b800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:08.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.292+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.292+0000 7f03c412b800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:08:08.292 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.292+0000 7f03c412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:08.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.293+0000 7f03c412b800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:08:08.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.293+0000 7f03c412b800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:08.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.294+0000 7f0275fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:08.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.296+0000 7f03c412b800 20 remove_watcher() i=0 2024-04-22T20:08:08.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.296+0000 7f03c412b800 2 removed watcher, disabling cache 2024-04-22T20:08:08.297 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.297+0000 7f03c412b800 20 remove_watcher() i=1 2024-04-22T20:08:08.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.298+0000 7f03c412b800 20 remove_watcher() i=2 2024-04-22T20:08:08.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.299+0000 7f03c412b800 20 remove_watcher() i=3 2024-04-22T20:08:08.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.300+0000 7f03c412b800 20 remove_watcher() i=4 2024-04-22T20:08:08.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.301+0000 7f03c412b800 20 remove_watcher() i=5 2024-04-22T20:08:08.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.302+0000 7f03c412b800 20 remove_watcher() i=6 2024-04-22T20:08:08.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:08.303+0000 7f03c412b800 20 remove_watcher() i=7 2024-04-22T20:08:08.313 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:08.313 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:08.313 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: a-bucket id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 num_objects: 8 size_kb: 1 num_shards: 3 2024-04-22T20:08:08.313 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:08.313 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with EIO injected at set_target_layout 2024-04-22T20:08:08.313 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:08.315 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.315 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.361 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.361 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.421 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.421 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.454 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.454 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.489 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.489 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.523 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.524 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.558 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.558 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.593 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.593 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.629 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.629 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.661 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.662 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.697 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.697 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.730 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.730 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.766 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.766 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.800 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.801 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.835 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.835 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.869 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.869 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.904 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.904 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.939 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.939 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:08.977 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:08.977 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:09.011 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:09.011 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:09.046 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:09.046 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:09.081 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:09.081 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:09.116 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-22T20:08:09.144 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:09.144 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:09.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.177+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.179+0000 7f8dfb333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.179+0000 7f8dfb333800 20 realm 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.179+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.179+0000 7f8dfb333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.179+0000 7f8dfb333800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.179+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.180+0000 7f8dfb333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.180+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.180+0000 7f8dfb333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.180+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.181+0000 7f8dfb333800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:09.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.181+0000 7f8dfb333800 20 searching for the correct realm 2024-04-22T20:08:09.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.181+0000 7f8df6ff5640 0 -- 172.21.15.136:0/294921910 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5649a64ba7b0 msgr2=0x5649a64bcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:09.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.385+0000 7f8df6ff5640 0 -- 172.21.15.136:0/294921910 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x7f8df0073060 msgr2=0x7f8df005a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.594+0000 7f8dfb333800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.594+0000 7f8dfb333800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.594+0000 7f8dfb333800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.594+0000 7f8dfb333800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.594+0000 7f8dfb333800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.594+0000 7f8dfb333800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.594+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.596 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 zone default found 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 4 Realm: () 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 zonegroup default 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.595+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.596+0000 7f8dfb333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.596+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.596+0000 7f8dfb333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.596+0000 7f8dfb333800 20 started sync module instance, tier type = 2024-04-22T20:08:09.598 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.596+0000 7f8dfb333800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:09.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.605+0000 7f8dfb333800 20 add_watcher() i=0 2024-04-22T20:08:09.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.605+0000 7f8dfb333800 20 add_watcher() i=1 2024-04-22T20:08:09.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.605+0000 7f8dfb333800 20 add_watcher() i=2 2024-04-22T20:08:09.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.605+0000 7f8dfb333800 20 add_watcher() i=3 2024-04-22T20:08:09.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.605+0000 7f8dfb333800 20 add_watcher() i=4 2024-04-22T20:08:09.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.605+0000 7f8dfb333800 20 add_watcher() i=5 2024-04-22T20:08:09.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.605+0000 7f8dfb333800 20 add_watcher() i=6 2024-04-22T20:08:09.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.606+0000 7f8dfb333800 20 add_watcher() i=7 2024-04-22T20:08:09.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.606+0000 7f8dfb333800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:09.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8cad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8dfb333800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8dfb333800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8dfb333800 5 note: GC not initialized 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8c27fff640 20 reqs_thread_entry: start 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8dfb333800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8dfb333800 20 Filter name: none 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8c25ffb640 20 reqs_thread_entry: start 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.608+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.609+0000 7f8dfb333800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:09.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.609+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.610+0000 7f8dfb333800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:09.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.610+0000 7f8dfb333800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:09.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.610+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.610+0000 7f8dfb333800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:09.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.610+0000 7f8dfb333800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.611+0000 7f8dfb333800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:09.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.611+0000 7f8dfb333800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:09.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.612+0000 7f8cad7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:09.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.614+0000 7f8dfb333800 20 remove_watcher() i=0 2024-04-22T20:08:09.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.614+0000 7f8dfb333800 2 removed watcher, disabling cache 2024-04-22T20:08:09.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.615+0000 7f8dfb333800 20 remove_watcher() i=1 2024-04-22T20:08:09.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.616+0000 7f8dfb333800 20 remove_watcher() i=2 2024-04-22T20:08:09.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.617+0000 7f8dfb333800 20 remove_watcher() i=3 2024-04-22T20:08:09.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.618+0000 7f8dfb333800 20 remove_watcher() i=4 2024-04-22T20:08:09.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.619+0000 7f8dfb333800 20 remove_watcher() i=5 2024-04-22T20:08:09.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.620+0000 7f8dfb333800 20 remove_watcher() i=6 2024-04-22T20:08:09.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.621+0000 7f8dfb333800 20 remove_watcher() i=7 2024-04-22T20:08:09.631 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:09.631 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:09.631 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-set-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:09.631 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:09.633 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout 2024-04-22T20:08:09.666 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:09.666 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:09.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.700+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.701+0000 7fc26312c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.701+0000 7fc26312c800 20 realm 2024-04-22T20:08:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.701+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.701+0000 7fc26312c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.701+0000 7fc26312c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.701+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.701+0000 7fc26312c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.701+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.702+0000 7fc26312c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.702+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.703+0000 7fc26312c800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.703+0000 7fc26312c800 20 searching for the correct realm 2024-04-22T20:08:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.703+0000 7fc25edf2640 0 -- 172.21.15.136:0/582440753 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x55d07be77d10 msgr2=0x55d07be7a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:09.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:09.908+0000 7fc25edf2640 0 -- 172.21.15.136:0/582440753 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x55d07bc278f0 msgr2=0x7fc258021980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:10.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:10.116 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:10.117 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.116+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:10.118 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 zone default found 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 4 Realm: () 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 zonegroup default 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.117+0000 7fc26312c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.118+0000 7fc26312c800 20 started sync module instance, tier type = 2024-04-22T20:08:10.119 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.118+0000 7fc26312c800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:10.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.126+0000 7fc26312c800 20 add_watcher() i=0 2024-04-22T20:08:10.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.126+0000 7fc26312c800 20 add_watcher() i=1 2024-04-22T20:08:10.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.126+0000 7fc26312c800 20 add_watcher() i=2 2024-04-22T20:08:10.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.126+0000 7fc26312c800 20 add_watcher() i=3 2024-04-22T20:08:10.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.126+0000 7fc26312c800 20 add_watcher() i=4 2024-04-22T20:08:10.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.126+0000 7fc26312c800 20 add_watcher() i=5 2024-04-22T20:08:10.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.126+0000 7fc26312c800 20 add_watcher() i=6 2024-04-22T20:08:10.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.127+0000 7fc26312c800 20 add_watcher() i=7 2024-04-22T20:08:10.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.127+0000 7fc26312c800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:10.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.128+0000 7fc1157e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:10.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:10.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:10.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 5 note: GC not initialized 2024-04-22T20:08:10.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc08ffff640 20 reqs_thread_entry: start 2024-04-22T20:08:10.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:10.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 20 Filter name: none 2024-04-22T20:08:10.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc08dffb640 20 reqs_thread_entry: start 2024-04-22T20:08:10.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-04-22T20:08:10.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 : miss 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.129+0000 7fc26312c800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.130+0000 7fc26312c800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.130+0000 7fc26312c800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 info.flags=0x17 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.130+0000 7fc26312c800 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 to cache LRU end 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.130+0000 7fc26312c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.130+0000 7fc26312c800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:10.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.130+0000 7fc26312c800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 2024-04-22T20:08:10.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.133+0000 7fc26312c800 -1 Injecting error=-5 at location=set_target_layout 2024-04-22T20:08:10.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.133+0000 7fc26312c800 0 ERROR: init_target_layout failed to write target index layout to bucket info: (5) Input/output error 2024-04-22T20:08:10.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.136+0000 7fc1157e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:10.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.138+0000 7fc26312c800 20 remove_watcher() i=0 2024-04-22T20:08:10.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.138+0000 7fc26312c800 2 removed watcher, disabling cache 2024-04-22T20:08:10.139 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.139+0000 7fc26312c800 20 remove_watcher() i=1 2024-04-22T20:08:10.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.140+0000 7fc26312c800 20 remove_watcher() i=2 2024-04-22T20:08:10.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.141+0000 7fc26312c800 20 remove_watcher() i=3 2024-04-22T20:08:10.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.142+0000 7fc26312c800 20 remove_watcher() i=4 2024-04-22T20:08:10.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.143+0000 7fc26312c800 20 remove_watcher() i=5 2024-04-22T20:08:10.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.144+0000 7fc26312c800 20 remove_watcher() i=6 2024-04-22T20:08:10.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.145+0000 7fc26312c800 20 remove_watcher() i=7 2024-04-22T20:08:10.156 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-22T20:08:10.183 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:10.183 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.216+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.218+0000 7f4308733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.218+0000 7f4308733800 20 realm 2024-04-22T20:08:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.218+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.218+0000 7f4308733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.218+0000 7f4308733800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:10.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.218+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.218+0000 7f4308733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.218+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.219+0000 7f4308733800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:10.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.219+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.219+0000 7f4308733800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:10.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.219+0000 7f4308733800 20 searching for the correct realm 2024-04-22T20:08:10.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.221+0000 7f4304cf3640 0 -- 172.21.15.136:0/1055685818 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f42fc083a90 msgr2=0x7f42fc06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:10.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.426+0000 7f42fbfff640 0 -- 172.21.15.136:0/1055685818 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f42fc0a0970 msgr2=0x7f42fc09d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.632+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:10.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 20 zone default found 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 4 Realm: () 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 20 zonegroup default 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.633+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.634+0000 7f4308733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.634+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.634+0000 7f4308733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.634+0000 7f4308733800 20 started sync module instance, tier type = 2024-04-22T20:08:10.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.634+0000 7f4308733800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:10.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.643+0000 7f4308733800 20 add_watcher() i=0 2024-04-22T20:08:10.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.643+0000 7f4308733800 20 add_watcher() i=1 2024-04-22T20:08:10.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.643+0000 7f4308733800 20 add_watcher() i=2 2024-04-22T20:08:10.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.643+0000 7f4308733800 20 add_watcher() i=3 2024-04-22T20:08:10.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.643+0000 7f4308733800 20 add_watcher() i=4 2024-04-22T20:08:10.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.643+0000 7f4308733800 20 add_watcher() i=5 2024-04-22T20:08:10.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.643+0000 7f4308733800 20 add_watcher() i=6 2024-04-22T20:08:10.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.644+0000 7f4308733800 20 add_watcher() i=7 2024-04-22T20:08:10.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.644+0000 7f4308733800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:10.645 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.645+0000 7f41bafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:10.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f4308733800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:10.646 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f4308733800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:10.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f4308733800 5 note: GC not initialized 2024-04-22T20:08:10.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f41b9fdb640 20 reqs_thread_entry: start 2024-04-22T20:08:10.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f4308733800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:10.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f4308733800 20 Filter name: none 2024-04-22T20:08:10.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f412b7fe640 20 reqs_thread_entry: start 2024-04-22T20:08:10.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f4308733800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:10.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.646+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.647+0000 7f4308733800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:10.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.647+0000 7f4308733800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:10.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.647+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.647+0000 7f4308733800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:10.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.647+0000 7f4308733800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.648+0000 7f4308733800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:10.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.648+0000 7f4308733800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:10.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.650+0000 7f41bafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:10.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.652+0000 7f4308733800 20 remove_watcher() i=0 2024-04-22T20:08:10.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.652+0000 7f4308733800 2 removed watcher, disabling cache 2024-04-22T20:08:10.653 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.653+0000 7f4308733800 20 remove_watcher() i=1 2024-04-22T20:08:10.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.654+0000 7f4308733800 20 remove_watcher() i=2 2024-04-22T20:08:10.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.655+0000 7f4308733800 20 remove_watcher() i=3 2024-04-22T20:08:10.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.656+0000 7f4308733800 20 remove_watcher() i=4 2024-04-22T20:08:10.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.657+0000 7f4308733800 20 remove_watcher() i=5 2024-04-22T20:08:10.658 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.658+0000 7f4308733800 20 remove_watcher() i=6 2024-04-22T20:08:10.659 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.659+0000 7f4308733800 20 remove_watcher() i=7 2024-04-22T20:08:10.669 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:10.669 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:10.670 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-set-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:10.670 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:10.672 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:10.672 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:10.705 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:10.705 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:10.761 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:08:10.791 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:10.791 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:10.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.824+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.825+0000 7f3802d22800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.825+0000 7f3802d22800 20 realm 2024-04-22T20:08:10.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.825+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.826+0000 7f3802d22800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.826+0000 7f3802d22800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:10.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.826+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.826+0000 7f3802d22800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:10.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.826+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.827+0000 7f3802d22800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:10.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.827+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:10.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.827+0000 7f3802d22800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:10.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.827+0000 7f3802d22800 20 searching for the correct realm 2024-04-22T20:08:10.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:10.829+0000 7f37ff16b640 0 -- 172.21.15.136:0/3950105954 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f37f8076040 msgr2=0x7f37f8076420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:11.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.034+0000 7f37fe169640 0 -- 172.21.15.136:0/3950105954 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f37f808e900 msgr2=0x7f37f805c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:11.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:11.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.240+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:11.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 20 zone default found 2024-04-22T20:08:11.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 4 Realm: () 2024-04-22T20:08:11.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 20 zonegroup default 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.241+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.242+0000 7f3802d22800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.242+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.242+0000 7f3802d22800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.242+0000 7f3802d22800 20 started sync module instance, tier type = 2024-04-22T20:08:11.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.242+0000 7f3802d22800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:11.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.251+0000 7f3802d22800 20 add_watcher() i=0 2024-04-22T20:08:11.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.251+0000 7f3802d22800 20 add_watcher() i=1 2024-04-22T20:08:11.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.251+0000 7f3802d22800 20 add_watcher() i=2 2024-04-22T20:08:11.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.251+0000 7f3802d22800 20 add_watcher() i=3 2024-04-22T20:08:11.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.251+0000 7f3802d22800 20 add_watcher() i=4 2024-04-22T20:08:11.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.251+0000 7f3802d22800 20 add_watcher() i=5 2024-04-22T20:08:11.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.251+0000 7f3802d22800 20 add_watcher() i=6 2024-04-22T20:08:11.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.252+0000 7f3802d22800 20 add_watcher() i=7 2024-04-22T20:08:11.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.252+0000 7f3802d22800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:11.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.254+0000 7f36b57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:11.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.254+0000 7f3802d22800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:11.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.254+0000 7f3802d22800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:11.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.254+0000 7f3802d22800 5 note: GC not initialized 2024-04-22T20:08:11.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.254+0000 7f362ffff640 20 reqs_thread_entry: start 2024-04-22T20:08:11.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.254+0000 7f3802d22800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:11.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.254+0000 7f3802d22800 20 Filter name: none 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.255+0000 7f362dffb640 20 reqs_thread_entry: start 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.255+0000 7f3802d22800 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.255+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.255+0000 7f3802d22800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.255+0000 7f3802d22800 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.255+0000 7f3802d22800 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.255+0000 7f3802d22800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 : miss 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.255+0000 7f3802d22800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.256+0000 7f3802d22800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:11.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.256+0000 7f3802d22800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 info.flags=0x17 2024-04-22T20:08:11.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.256+0000 7f3802d22800 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 to cache LRU end 2024-04-22T20:08:11.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.256+0000 7f3802d22800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:11.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.256+0000 7f3802d22800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:11.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.256+0000 7f3802d22800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 2024-04-22T20:08:11.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.259+0000 7f3802d22800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 info.flags=0x17 2024-04-22T20:08:11.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.259+0000 7f3802d22800 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 to cache LRU end 2024-04-22T20:08:11.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.259+0000 7f3802d22800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:11.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.259+0000 7f3802d22800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19, ofs0, ns] 2024-04-22T20:08:11.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.260+0000 7f36e27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562382 cookie 94269580865952 notifier 6252 bl.length()=917 2024-04-22T20:08:11.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.260+0000 7f36e27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 info.flags=0x17 2024-04-22T20:08:11.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.260+0000 7f36e27fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 to cache LRU end 2024-04-22T20:08:11.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.260+0000 7f36e27fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:11.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.261+0000 7f3802d22800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19]): orig_sources= new_sources= 2024-04-22T20:08:11.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.262+0000 7f3802d22800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19]): potential sources added=[] removed=[] 2024-04-22T20:08:11.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.262+0000 7f3802d22800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19]): orig_dests= new_dests= 2024-04-22T20:08:11.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.262+0000 7f3802d22800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19]): potential dests added=[] removed=[] 2024-04-22T20:08:11.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.263+0000 7f3802d22800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19.1.0 2024-04-22T20:08:11.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.263+0000 7f3802d22800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19.1.0 2024-04-22T20:08:11.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.263+0000 7f3802d22800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19.1.1 2024-04-22T20:08:11.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.263+0000 7f3802d22800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19.1.1 2024-04-22T20:08:11.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.263+0000 7f3802d22800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19.0 2024-04-22T20:08:11.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.265+0000 7f3802d22800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 : hit (requested=0x17, cached=0x17) 2024-04-22T20:08:11.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.265+0000 7f3802d22800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 2024-04-22T20:08:11.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.267+0000 7f3802d22800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 info.flags=0x17 2024-04-22T20:08:11.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.267+0000 7f3802d22800 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 to cache LRU end 2024-04-22T20:08:11.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.267+0000 7f3802d22800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:11.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.267+0000 7f3802d22800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19, ofs0, ns] 2024-04-22T20:08:11.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.267+0000 7f36e1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562383 cookie 94269580865952 notifier 6252 bl.length()=885 2024-04-22T20:08:11.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.267+0000 7f36e1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 info.flags=0x17 2024-04-22T20:08:11.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.267+0000 7f36e1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 to cache LRU end 2024-04-22T20:08:11.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.267+0000 7f36e1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:11.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.269+0000 7f3802d22800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19]): orig_sources= new_sources= 2024-04-22T20:08:11.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.269+0000 7f3802d22800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19]): potential sources added=[] removed=[] 2024-04-22T20:08:11.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.269+0000 7f3802d22800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19]): orig_dests= new_dests= 2024-04-22T20:08:11.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.269+0000 7f3802d22800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19]): potential dests added=[] removed=[] 2024-04-22T20:08:11.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.270+0000 7f3802d22800 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-04-22T20:08:11.271 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.271+0000 7f36b57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:11.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.274+0000 7f3802d22800 20 remove_watcher() i=0 2024-04-22T20:08:11.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.274+0000 7f3802d22800 2 removed watcher, disabling cache 2024-04-22T20:08:11.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.275+0000 7f3802d22800 20 remove_watcher() i=1 2024-04-22T20:08:11.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.276+0000 7f3802d22800 20 remove_watcher() i=2 2024-04-22T20:08:11.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.277+0000 7f3802d22800 20 remove_watcher() i=3 2024-04-22T20:08:11.278 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.278+0000 7f3802d22800 20 remove_watcher() i=4 2024-04-22T20:08:11.279 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.279+0000 7f3802d22800 20 remove_watcher() i=5 2024-04-22T20:08:11.280 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.280+0000 7f3802d22800 20 remove_watcher() i=6 2024-04-22T20:08:11.281 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.281+0000 7f3802d22800 20 remove_watcher() i=7 2024-04-22T20:08:11.292 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-22T20:08:11.320 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:11.320 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:11.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.353+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.355+0000 7f505a92d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.355+0000 7f505a92d800 20 realm 2024-04-22T20:08:11.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.355+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.355+0000 7f505a92d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.355+0000 7f505a92d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:11.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.355+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.355+0000 7f505a92d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.355+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.356+0000 7f505a92d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:11.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.356+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.356+0000 7f505a92d800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:11.357 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.356+0000 7f505a92d800 20 searching for the correct realm 2024-04-22T20:08:11.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.358+0000 7f5056d80640 0 -- 172.21.15.136:0/3239718705 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f5050033030 msgr2=0x7f5050040430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:11.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.563+0000 7f5055d7e640 0 -- 172.21.15.136:0/3239718705 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f5050018040 msgr2=0x7f50500323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:11.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:11.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:11.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:11.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:11.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:11.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:11.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.769+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:11.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 20 zone default found 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 4 Realm: () 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 20 zonegroup default 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.770+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.771+0000 7f505a92d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.771+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.771+0000 7f505a92d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.771+0000 7f505a92d800 20 started sync module instance, tier type = 2024-04-22T20:08:11.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.771+0000 7f505a92d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:11.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.780+0000 7f505a92d800 20 add_watcher() i=0 2024-04-22T20:08:11.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.780+0000 7f505a92d800 20 add_watcher() i=1 2024-04-22T20:08:11.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.780+0000 7f505a92d800 20 add_watcher() i=2 2024-04-22T20:08:11.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.780+0000 7f505a92d800 20 add_watcher() i=3 2024-04-22T20:08:11.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.780+0000 7f505a92d800 20 add_watcher() i=4 2024-04-22T20:08:11.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.780+0000 7f505a92d800 20 add_watcher() i=5 2024-04-22T20:08:11.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.780+0000 7f505a92d800 20 add_watcher() i=6 2024-04-22T20:08:11.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.781+0000 7f505a92d800 20 add_watcher() i=7 2024-04-22T20:08:11.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.781+0000 7f505a92d800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:11.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.782+0000 7f4f0cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:11.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.783+0000 7f505a92d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:11.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.783+0000 7f505a92d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:11.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.783+0000 7f505a92d800 5 note: GC not initialized 2024-04-22T20:08:11.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.783+0000 7f4e877fe640 20 reqs_thread_entry: start 2024-04-22T20:08:11.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.783+0000 7f505a92d800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:11.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.783+0000 7f505a92d800 20 Filter name: none 2024-04-22T20:08:11.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.783+0000 7f4e857fa640 20 reqs_thread_entry: start 2024-04-22T20:08:11.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.783+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.784+0000 7f505a92d800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:11.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.784+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.785+0000 7f505a92d800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:11.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.785+0000 7f505a92d800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:11.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.785+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.785+0000 7f505a92d800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:11.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.785+0000 7f505a92d800 20 rados->read ofs=0 len=0 2024-04-22T20:08:11.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.785+0000 7f505a92d800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:11.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.785+0000 7f505a92d800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:11.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.787+0000 7f4f0cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:11.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.789+0000 7f505a92d800 20 remove_watcher() i=0 2024-04-22T20:08:11.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.789+0000 7f505a92d800 2 removed watcher, disabling cache 2024-04-22T20:08:11.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.789+0000 7f505a92d800 20 remove_watcher() i=1 2024-04-22T20:08:11.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.791+0000 7f505a92d800 20 remove_watcher() i=2 2024-04-22T20:08:11.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.792+0000 7f505a92d800 20 remove_watcher() i=3 2024-04-22T20:08:11.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.793+0000 7f505a92d800 20 remove_watcher() i=4 2024-04-22T20:08:11.794 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.794+0000 7f505a92d800 20 remove_watcher() i=5 2024-04-22T20:08:11.795 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.795+0000 7f505a92d800 20 remove_watcher() i=6 2024-04-22T20:08:11.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:11.796+0000 7f505a92d800 20 remove_watcher() i=7 2024-04-22T20:08:11.806 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:11.806 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:11.806 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-set-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-22T20:08:11.806 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:11.808 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:11.809 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:11.831 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:11.831 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.057 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.058 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.128 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with ECANCELED injected at set_target_layout 2024-04-22T20:08:12.128 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:12.129 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.129 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.169 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.169 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.229 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.229 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.264 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.264 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.299 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.299 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.334 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.334 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.369 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.369 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.403 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.403 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.437 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.437 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.475 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.475 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.509 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.509 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.544 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.544 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.578 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.579 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.613 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.613 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.649 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.649 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.684 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.684 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.719 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.719 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.757 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.757 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.792 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.792 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.828 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.828 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.863 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.863 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.898 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:12.898 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:12.933 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-22T20:08:12.960 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:12.960 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:12.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.993+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:12.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.995+0000 7fac11919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:12.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.995+0000 7fac11919800 20 realm 2024-04-22T20:08:12.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.995+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:12.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.995+0000 7fac11919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:12.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.995+0000 7fac11919800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:12.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.995+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:12.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.995+0000 7fac11919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:12.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.995+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:12.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.996+0000 7fac11919800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:12.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.996+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:12.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.997+0000 7fac11919800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:12.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.997+0000 7fac11919800 20 searching for the correct realm 2024-04-22T20:08:12.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:12.999+0000 7fac0fc19640 0 -- 172.21.15.136:0/2726110302 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fac0801b040 msgr2=0x7fac0801c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:13.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.204+0000 7fac0ec17640 0 -- 172.21.15.136:0/2726110302 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fac08008410 msgr2=0x7fac08008890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.410+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:13.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 20 zone default found 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 4 Realm: () 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 20 zonegroup default 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.411+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.412+0000 7fac11919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.412+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.412+0000 7fac11919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.412+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.412+0000 7fac11919800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.412+0000 7fac11919800 20 started sync module instance, tier type = 2024-04-22T20:08:13.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.412+0000 7fac11919800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:13.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 20 add_watcher() i=0 2024-04-22T20:08:13.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 20 add_watcher() i=1 2024-04-22T20:08:13.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 20 add_watcher() i=2 2024-04-22T20:08:13.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 20 add_watcher() i=3 2024-04-22T20:08:13.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 20 add_watcher() i=4 2024-04-22T20:08:13.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 20 add_watcher() i=5 2024-04-22T20:08:13.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 20 add_watcher() i=6 2024-04-22T20:08:13.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 20 add_watcher() i=7 2024-04-22T20:08:13.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.422+0000 7fac11919800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.424+0000 7faac5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.424+0000 7fac11919800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.424+0000 7fac11919800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.424+0000 7fac11919800 5 note: GC not initialized 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.424+0000 7faac4fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.425+0000 7fac11919800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.425+0000 7fac11919800 20 Filter name: none 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.425+0000 7faa3e7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:13.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.425+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.425+0000 7fac11919800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:13.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.425+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.426+0000 7fac11919800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:13.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.426+0000 7fac11919800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:13.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.426+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.426+0000 7fac11919800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:13.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.426+0000 7fac11919800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.427+0000 7fac11919800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:13.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.427+0000 7fac11919800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:13.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.428+0000 7faac5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:13.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.430+0000 7fac11919800 20 remove_watcher() i=0 2024-04-22T20:08:13.430 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.430+0000 7fac11919800 2 removed watcher, disabling cache 2024-04-22T20:08:13.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.431+0000 7fac11919800 20 remove_watcher() i=1 2024-04-22T20:08:13.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.432+0000 7fac11919800 20 remove_watcher() i=2 2024-04-22T20:08:13.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.433+0000 7fac11919800 20 remove_watcher() i=3 2024-04-22T20:08:13.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.434+0000 7fac11919800 20 remove_watcher() i=4 2024-04-22T20:08:13.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.435+0000 7fac11919800 20 remove_watcher() i=5 2024-04-22T20:08:13.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.436+0000 7fac11919800 20 remove_watcher() i=6 2024-04-22T20:08:13.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.436+0000 7fac11919800 20 remove_watcher() i=7 2024-04-22T20:08:13.446 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:13.446 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:13.446 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-set-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:13.446 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:13.448 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout --inject-error-code 125 2024-04-22T20:08:13.482 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:13.483 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:13.516 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.516+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.517+0000 7f012c746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.517+0000 7f012c746800 20 realm 2024-04-22T20:08:13.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.517+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.518+0000 7f012c746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.518+0000 7f012c746800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:13.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.518+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.518+0000 7f012c746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.518+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.518+0000 7f012c746800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:13.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.518+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.519+0000 7f012c746800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:13.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.519+0000 7f012c746800 20 searching for the correct realm 2024-04-22T20:08:13.521 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.521+0000 7f012aa46640 0 -- 172.21.15.136:0/388500949 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f012405f3d0 msgr2=0x7f012405e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:13.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.726+0000 7f0121a44640 0 -- 172.21.15.136:0/388500949 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f012407e220 msgr2=0x7f01240798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:13.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:13.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:13.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:13.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:13.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:13.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:13.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.932+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 20 zone default found 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 4 Realm: () 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 20 zonegroup default 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.933+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.934+0000 7f012c746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:13.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.934+0000 7f012c746800 20 started sync module instance, tier type = 2024-04-22T20:08:13.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.934+0000 7f012c746800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:13.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.943+0000 7f012c746800 20 add_watcher() i=0 2024-04-22T20:08:13.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.943+0000 7f012c746800 20 add_watcher() i=1 2024-04-22T20:08:13.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.943+0000 7f012c746800 20 add_watcher() i=2 2024-04-22T20:08:13.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.943+0000 7f012c746800 20 add_watcher() i=3 2024-04-22T20:08:13.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.943+0000 7f012c746800 20 add_watcher() i=4 2024-04-22T20:08:13.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.943+0000 7f012c746800 20 add_watcher() i=5 2024-04-22T20:08:13.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.944+0000 7f012c746800 20 add_watcher() i=6 2024-04-22T20:08:13.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.944+0000 7f012c746800 20 add_watcher() i=7 2024-04-22T20:08:13.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.944+0000 7f012c746800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:13.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.946+0000 7effe0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:13.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.946+0000 7f012c746800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:13.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.946+0000 7f012c746800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:13.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.946+0000 7f012c746800 5 note: GC not initialized 2024-04-22T20:08:13.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7eff5b7fe640 20 reqs_thread_entry: start 2024-04-22T20:08:13.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:13.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 20 Filter name: none 2024-04-22T20:08:13.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7eff597fa640 20 reqs_thread_entry: start 2024-04-22T20:08:13.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 : miss 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.947+0000 7f012c746800 20 rados->read ofs=0 len=0 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.948+0000 7f012c746800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.948+0000 7f012c746800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 info.flags=0x17 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.948+0000 7f012c746800 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 to cache LRU end 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.948+0000 7f012c746800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.948+0000 7f012c746800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:13.949 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.948+0000 7f012c746800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 2024-04-22T20:08:13.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.951+0000 7f012c746800 -1 Injecting error=-125 at location=set_target_layout 2024-04-22T20:08:13.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.952+0000 7f012c746800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 info.flags=0x17 2024-04-22T20:08:13.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.952+0000 7f012c746800 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 to cache LRU end 2024-04-22T20:08:13.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.952+0000 7f012c746800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:13.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.952+0000 7f012c746800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20, ofs0, ns] 2024-04-22T20:08:13.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.952+0000 7f000dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562386 cookie 94263938275824 notifier 6002 bl.length()=917 2024-04-22T20:08:13.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.952+0000 7f000dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 info.flags=0x17 2024-04-22T20:08:13.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.952+0000 7f000dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 to cache LRU end 2024-04-22T20:08:13.953 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.952+0000 7f000dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:13.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.954+0000 7f012c746800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20]): orig_sources= new_sources= 2024-04-22T20:08:13.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.954+0000 7f012c746800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20]): potential sources added=[] removed=[] 2024-04-22T20:08:13.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.954+0000 7f012c746800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20]): orig_dests= new_dests= 2024-04-22T20:08:13.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.954+0000 7f012c746800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20]): potential dests added=[] removed=[] 2024-04-22T20:08:13.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.955+0000 7f012c746800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20.1.0 2024-04-22T20:08:13.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.955+0000 7f012c746800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20.1.0 2024-04-22T20:08:13.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.955+0000 7f012c746800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20.1.1 2024-04-22T20:08:13.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.955+0000 7f012c746800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20.1.1 2024-04-22T20:08:13.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.955+0000 7f012c746800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20.0 2024-04-22T20:08:13.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.958+0000 7f012c746800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 : hit (requested=0x17, cached=0x17) 2024-04-22T20:08:13.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.958+0000 7f012c746800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 2024-04-22T20:08:13.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.959+0000 7f012c746800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 info.flags=0x17 2024-04-22T20:08:13.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.959+0000 7f012c746800 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 to cache LRU end 2024-04-22T20:08:13.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.959+0000 7f012c746800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.959+0000 7f012c746800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20, ofs0, ns] 2024-04-22T20:08:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.959+0000 7f000d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562387 cookie 94263938275824 notifier 6002 bl.length()=885 2024-04-22T20:08:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.959+0000 7f000d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 info.flags=0x17 2024-04-22T20:08:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.959+0000 7f000d7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 to cache LRU end 2024-04-22T20:08:13.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.959+0000 7f000d7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:13.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.961+0000 7f012c746800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20]): orig_sources= new_sources= 2024-04-22T20:08:13.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.961+0000 7f012c746800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20]): potential sources added=[] removed=[] 2024-04-22T20:08:13.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.961+0000 7f012c746800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20]): orig_dests= new_dests= 2024-04-22T20:08:13.961 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.961+0000 7f012c746800 20 handle_bi_update(): bucket=:error-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20]): potential dests added=[] removed=[] 2024-04-22T20:08:13.962 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.962+0000 7f012c746800 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-04-22T20:08:13.964 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.964+0000 7effe0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:13.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.967+0000 7f012c746800 20 remove_watcher() i=0 2024-04-22T20:08:13.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.967+0000 7f012c746800 2 removed watcher, disabling cache 2024-04-22T20:08:13.967 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.968+0000 7f012c746800 20 remove_watcher() i=1 2024-04-22T20:08:13.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.969+0000 7f012c746800 20 remove_watcher() i=2 2024-04-22T20:08:13.969 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.970+0000 7f012c746800 20 remove_watcher() i=3 2024-04-22T20:08:13.970 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.971+0000 7f012c746800 20 remove_watcher() i=4 2024-04-22T20:08:13.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.972+0000 7f012c746800 20 remove_watcher() i=5 2024-04-22T20:08:13.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.972+0000 7f012c746800 20 remove_watcher() i=6 2024-04-22T20:08:13.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:13.973+0000 7f012c746800 20 remove_watcher() i=7 2024-04-22T20:08:13.985 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-04-22T20:08:14.013 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:14.013 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:14.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.046+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.047+0000 7fe6a4613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:14.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.047+0000 7fe6a4613800 20 realm 2024-04-22T20:08:14.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.047+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.048+0000 7fe6a4613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:14.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.048+0000 7fe6a4613800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:14.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.048+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.048+0000 7fe6a4613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:14.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.048+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.048+0000 7fe6a4613800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:14.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.048+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.049+0000 7fe6a4613800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:14.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.049+0000 7fe6a4613800 20 searching for the correct realm 2024-04-22T20:08:14.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.051+0000 7fe6a0422640 0 -- 172.21.15.136:0/997966573 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55f658f87ee0 msgr2=0x55f658f8a2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:14.054 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.054+0000 7fe6a0422640 0 -- 172.21.15.136:0/997966573 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7fe56807ba00 msgr2=0x7fe56807de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:14.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:14.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:14.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:14.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:14.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:14.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:14.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:14.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:14.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.059+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:14.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 zone default found 2024-04-22T20:08:14.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 4 Realm: () 2024-04-22T20:08:14.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:14.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:14.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:14.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 zonegroup default 2024-04-22T20:08:14.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:14.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:14.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:14.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 started sync module instance, tier type = 2024-04-22T20:08:14.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.060+0000 7fe6a4613800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:14.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.072+0000 7fe6a4613800 20 add_watcher() i=0 2024-04-22T20:08:14.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.072+0000 7fe6a4613800 20 add_watcher() i=1 2024-04-22T20:08:14.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.072+0000 7fe6a4613800 20 add_watcher() i=2 2024-04-22T20:08:14.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.072+0000 7fe6a4613800 20 add_watcher() i=3 2024-04-22T20:08:14.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.072+0000 7fe6a4613800 20 add_watcher() i=4 2024-04-22T20:08:14.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.072+0000 7fe6a4613800 20 add_watcher() i=5 2024-04-22T20:08:14.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.072+0000 7fe6a4613800 20 add_watcher() i=6 2024-04-22T20:08:14.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.073+0000 7fe6a4613800 20 add_watcher() i=7 2024-04-22T20:08:14.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.073+0000 7fe6a4613800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:14.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.074+0000 7fe555fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:14.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe6a4613800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:14.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe6a4613800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:14.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe6a4613800 5 note: GC not initialized 2024-04-22T20:08:14.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe554fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:14.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe6a4613800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:14.075 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe6a4613800 20 Filter name: none 2024-04-22T20:08:14.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe4ce7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:14.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe6a4613800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.075+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.076+0000 7fe6a4613800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.076+0000 7fe6a4613800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.076+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.076+0000 7fe6a4613800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.076+0000 7fe6a4613800 20 rados->read ofs=0 len=0 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.076+0000 7fe6a4613800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.076+0000 7fe6a4613800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:14.077 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.078+0000 7fe555fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:14.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.080+0000 7fe6a4613800 20 remove_watcher() i=0 2024-04-22T20:08:14.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.080+0000 7fe6a4613800 2 removed watcher, disabling cache 2024-04-22T20:08:14.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.081+0000 7fe6a4613800 20 remove_watcher() i=1 2024-04-22T20:08:14.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.082+0000 7fe6a4613800 20 remove_watcher() i=2 2024-04-22T20:08:14.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.083+0000 7fe6a4613800 20 remove_watcher() i=3 2024-04-22T20:08:14.084 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.084+0000 7fe6a4613800 20 remove_watcher() i=4 2024-04-22T20:08:14.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.085+0000 7fe6a4613800 20 remove_watcher() i=5 2024-04-22T20:08:14.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.086+0000 7fe6a4613800 20 remove_watcher() i=6 2024-04-22T20:08:14.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:14.086+0000 7fe6a4613800 20 remove_watcher() i=7 2024-04-22T20:08:14.096 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:14.096 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:14.096 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-set-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20 num_objects: 20 size_kb: 1 num_shards: 2 2024-04-22T20:08:14.096 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:14.098 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.099 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.118 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.118 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.329 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.330 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.393 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with abort at set_target_layout 2024-04-22T20:08:14.394 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:14.395 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.395 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.437 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.437 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.493 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.493 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.528 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.528 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.562 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.562 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.596 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.597 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.631 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.631 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.666 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.666 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.700 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.700 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.734 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.734 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.769 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.770 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.804 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.804 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.838 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.838 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.873 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.873 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.908 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.908 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.943 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.943 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:14.978 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:14.978 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:15.014 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:15.014 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:15.050 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:15.050 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:15.084 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:15.084 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:15.118 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:15.118 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:15.152 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:15.152 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:15.187 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-04-22T20:08:15.214 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:15.214 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:15.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.247+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.248+0000 7f7e27b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.248+0000 7f7e27b25800 20 realm 2024-04-22T20:08:15.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.248+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.249+0000 7f7e27b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.249+0000 7f7e27b25800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:15.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.249+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.249+0000 7f7e27b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.249+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.249+0000 7f7e27b25800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.249+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.250+0000 7f7e27b25800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:15.250 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.250+0000 7f7e27b25800 20 searching for the correct realm 2024-04-22T20:08:15.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.452+0000 7f7e240b8640 0 -- 172.21.15.136:0/1188890569 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55896302b800 msgr2=0x55896302dbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.455+0000 7f7e230b6640 0 -- 172.21.15.136:0/1188890569 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f7e1408c410 msgr2=0x7f7e140586d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:15.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:15.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:15.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:15.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:15.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:15.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.462 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.461+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:15.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 zone default found 2024-04-22T20:08:15.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 4 Realm: () 2024-04-22T20:08:15.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:15.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 zonegroup default 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 started sync module instance, tier type = 2024-04-22T20:08:15.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.462+0000 7f7e27b25800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:15.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.471+0000 7f7e27b25800 20 add_watcher() i=0 2024-04-22T20:08:15.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.471+0000 7f7e27b25800 20 add_watcher() i=1 2024-04-22T20:08:15.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.471+0000 7f7e27b25800 20 add_watcher() i=2 2024-04-22T20:08:15.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.471+0000 7f7e27b25800 20 add_watcher() i=3 2024-04-22T20:08:15.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.471+0000 7f7e27b25800 20 add_watcher() i=4 2024-04-22T20:08:15.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.471+0000 7f7e27b25800 20 add_watcher() i=5 2024-04-22T20:08:15.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.471+0000 7f7e27b25800 20 add_watcher() i=6 2024-04-22T20:08:15.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.472+0000 7f7e27b25800 20 add_watcher() i=7 2024-04-22T20:08:15.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.472+0000 7f7e27b25800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:15.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7cd9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7e27b25800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:15.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7e27b25800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:15.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7e27b25800 5 note: GC not initialized 2024-04-22T20:08:15.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7cd8fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:15.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7e27b25800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:15.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7e27b25800 20 Filter name: none 2024-04-22T20:08:15.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7c527fc640 20 reqs_thread_entry: start 2024-04-22T20:08:15.475 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.474+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.475+0000 7f7e27b25800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:15.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.475+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.475+0000 7f7e27b25800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:15.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.475+0000 7f7e27b25800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:15.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.476+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.476+0000 7f7e27b25800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:15.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.476+0000 7f7e27b25800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.476+0000 7f7e27b25800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:15.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.476+0000 7f7e27b25800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:15.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.477+0000 7f7cd9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.479+0000 7f7e27b25800 20 remove_watcher() i=0 2024-04-22T20:08:15.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.479+0000 7f7e27b25800 2 removed watcher, disabling cache 2024-04-22T20:08:15.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.480+0000 7f7e27b25800 20 remove_watcher() i=1 2024-04-22T20:08:15.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.481+0000 7f7e27b25800 20 remove_watcher() i=2 2024-04-22T20:08:15.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.482+0000 7f7e27b25800 20 remove_watcher() i=3 2024-04-22T20:08:15.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.483+0000 7f7e27b25800 20 remove_watcher() i=4 2024-04-22T20:08:15.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.484+0000 7f7e27b25800 20 remove_watcher() i=5 2024-04-22T20:08:15.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.485+0000 7f7e27b25800 20 remove_watcher() i=6 2024-04-22T20:08:15.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.486+0000 7f7e27b25800 20 remove_watcher() i=7 2024-04-22T20:08:15.495 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.495 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:15.495 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-set-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:15.496 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.497 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at set_target_layout 2024-04-22T20:08:15.529 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:15.530 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:15.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.563+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.564+0000 7f6a4b922800 20 realm 2024-04-22T20:08:15.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.564+0000 7f6a4b922800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:15.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.566+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:15.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.566+0000 7f6a4b922800 20 searching for the correct realm 2024-04-22T20:08:15.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.567+0000 7f6a47d6b640 0 -- 172.21.15.136:0/991704454 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x557b8e7521b0 msgr2=0x557b8e7545c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.571+0000 7f6a47d6b640 0 -- 172.21.15.136:0/991704454 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f691007ba00 msgr2=0x7f691007de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 zone default found 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 4 Realm: () 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 zonegroup default 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.578 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 started sync module instance, tier type = 2024-04-22T20:08:15.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.577+0000 7f6a4b922800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:15.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=0 2024-04-22T20:08:15.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=1 2024-04-22T20:08:15.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=2 2024-04-22T20:08:15.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=3 2024-04-22T20:08:15.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=4 2024-04-22T20:08:15.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=5 2024-04-22T20:08:15.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=6 2024-04-22T20:08:15.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.587+0000 7f6a4b922800 20 add_watcher() i=7 2024-04-22T20:08:15.587 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.587+0000 7f6a4b922800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:15.588 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.588+0000 7f68fdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.588+0000 7f6a4b922800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:15.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.588+0000 7f6a4b922800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:15.590 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 5 note: GC not initialized 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f68fcfe1640 20 reqs_thread_entry: start 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 20 Filter name: none 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f68767fc640 20 reqs_thread_entry: start 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 : miss 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.590+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:15.591 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.590+0000 7f6a4b922800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:15.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.590+0000 7f6a4b922800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:15.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.590+0000 7f6a4b922800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:15.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.590+0000 7f6a4b922800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:15.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.590+0000 7f6a4b922800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 2024-04-22T20:08:15.592 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6a4b922800 time 2024-04-22T20:08:15.593950+0000 2024-04-22T20:08:15.592 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:08:15.593 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:15.593 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f6a4cd81f5c] 2024-04-22T20:08:15.593 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f6a4cd8211a] 2024-04-22T20:08:15.593 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557b8cfe4dc1] 2024-04-22T20:08:15.593 INFO:tasks.workunit.client.0.smithi136.stderr: 4: radosgw-admin(+0x83755a) [0x557b8cfee55a] 2024-04-22T20:08:15.593 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x25b) [0x557b8cff2dbb] 2024-04-22T20:08:15.593 INFO:tasks.workunit.client.0.smithi136.stderr: 6: main() 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: 7: /lib64/libc.so.6(+0x29590) [0x7f6a4bc29590] 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: 8: __libc_start_main() 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: 9: _start() 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.593+0000 7f6a4b922800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6a4b922800 time 2024-04-22T20:08:15.593950+0000 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f6a4cd81f5c] 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f6a4cd8211a] 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557b8cfe4dc1] 2024-04-22T20:08:15.594 INFO:tasks.workunit.client.0.smithi136.stderr: 4: radosgw-admin(+0x83755a) [0x557b8cfee55a] 2024-04-22T20:08:15.595 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x25b) [0x557b8cff2dbb] 2024-04-22T20:08:15.595 INFO:tasks.workunit.client.0.smithi136.stderr: 6: main() 2024-04-22T20:08:15.595 INFO:tasks.workunit.client.0.smithi136.stderr: 7: /lib64/libc.so.6(+0x29590) [0x7f6a4bc29590] 2024-04-22T20:08:15.595 INFO:tasks.workunit.client.0.smithi136.stderr: 8: __libc_start_main() 2024-04-22T20:08:15.595 INFO:tasks.workunit.client.0.smithi136.stderr: 9: _start() 2024-04-22T20:08:15.595 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.595 INFO:tasks.workunit.client.0.smithi136.stderr:*** Caught signal (Aborted) ** 2024-04-22T20:08:15.595 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7f6a4b922800 thread_name:radosgw-admin 2024-04-22T20:08:15.611 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:15.611 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6a4bc3e6f0] 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f6a4bc8b94c] 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f6a4cd81fb6] 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f6a4cd8211a] 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557b8cfe4dc1] 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 8: radosgw-admin(+0x83755a) [0x557b8cfee55a] 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x25b) [0x557b8cff2dbb] 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 10: main() 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 11: /lib64/libc.so.6(+0x29590) [0x7f6a4bc29590] 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 12: __libc_start_main() 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr: 13: _start() 2024-04-22T20:08:15.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.594+0000 7f6a4b922800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7f6a4b922800 thread_name:radosgw-admin 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6a4bc3e6f0] 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f6a4bc8b94c] 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f6a4cd81fb6] 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f6a4cd8211a] 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557b8cfe4dc1] 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 8: radosgw-admin(+0x83755a) [0x557b8cfee55a] 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x25b) [0x557b8cff2dbb] 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 10: main() 2024-04-22T20:08:15.613 INFO:tasks.workunit.client.0.smithi136.stderr: 11: /lib64/libc.so.6(+0x29590) [0x7f6a4bc29590] 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr: 12: __libc_start_main() 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr: 13: _start() 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr:--- begin dump of recent events --- 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr: -499> 2024-04-22T20:08:15.547+0000 7f693560a640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr: -498> 2024-04-22T20:08:15.547+0000 7f693560a640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr: -497> 2024-04-22T20:08:15.547+0000 7f693560a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr: -496> 2024-04-22T20:08:15.547+0000 7f693560a640 10 monclient: dump: 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.614 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr: -495> 2024-04-22T20:08:15.547+0000 7f6a4b922800 10 monclient: get_monmap_and_config success 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr: -494> 2024-04-22T20:08:15.547+0000 7f6a4b922800 4 set_mon_vals no callback set 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr: -493> 2024-04-22T20:08:15.548+0000 7f6a4b922800 10 monclient: shutdown 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr: -492> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 asok(0x557b8e43ad00) unregister_commands rotate-key 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr: -491> 2024-04-22T20:08:15.548+0000 7f6a4b922800 10 monclient: build_initial_monmap 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr: -490> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 build_initial for_mkfs: 0 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr: -489> 2024-04-22T20:08:15.548+0000 7f6a4b922800 10 monclient: monmap: 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:15.615 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr: -488> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding auth protocol: cephx 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr: -487> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding auth protocol: cephx 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr: -486> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding auth protocol: cephx 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr: -485> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr: -484> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr: -483> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.616 INFO:tasks.workunit.client.0.smithi136.stderr: -482> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -481> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -480> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -479> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -478> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -477> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -476> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -475> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -474> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -473> 2024-04-22T20:08:15.548+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -472> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 librados: starting msgr at 2024-04-22T20:08:15.617 INFO:tasks.workunit.client.0.smithi136.stderr: -471> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 librados: starting objecter 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -470> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command objecter_requests hook 0x557b8e65e870 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -469> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 librados: setting wanted keys 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -468> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 librados: calling monclient init 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -467> 2024-04-22T20:08:15.548+0000 7f6a4b922800 10 monclient: init 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -466> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding auth protocol: cephx 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -465> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding auth protocol: cephx 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -464> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding auth protocol: cephx 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -463> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -462> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -461> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.618 INFO:tasks.workunit.client.0.smithi136.stderr: -460> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -459> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -458> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -457> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -456> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -455> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -454> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -453> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -452> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -451> 2024-04-22T20:08:15.548+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -450> 2024-04-22T20:08:15.549+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.619 INFO:tasks.workunit.client.0.smithi136.stderr: -449> 2024-04-22T20:08:15.549+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command rotate-key hook 0x557b8e65ae88 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -448> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -447> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -446> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-a con 0x557b8e512480 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -445> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-c con 0x557b8e59c870 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -444> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-b con 0x557b8e5a2680 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -443> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -442> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -441> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -440> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -439> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): authenticate will time out at 1884.901012s 2024-04-22T20:08:15.620 INFO:tasks.workunit.client.0.smithi136.stderr: -438> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): get_auth_request con 0x557b8e59c870 auth_method 0 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -437> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -436> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -435> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -434> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): get_auth_request con 0x557b8e512480 auth_method 0 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -433> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -432> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -431> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -430> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -429> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -428> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -427> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -426> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -425> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -424> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request con 0x557b8e5a2680 auth_method 0 2024-04-22T20:08:15.621 INFO:tasks.workunit.client.0.smithi136.stderr: -423> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -422> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -421> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -420> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -419> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -418> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -417> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): handle_auth_done global_id 6044 payload 824 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -416> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient: _finish_hunting 0 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -415> 2024-04-22T20:08:15.549+0000 7f6a46d69640 1 monclient: found mon.noname-c 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -414> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -413> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -412> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr: -411> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: dump: 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.622 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr: -410> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: _finish_auth 0 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr: -409> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: _check_auth_tickets 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr: -408> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr: -407> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr: -406> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr: -405> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: dump: 2024-04-22T20:08:15.623 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr: -404> 2024-04-22T20:08:15.549+0000 7f693560a640 4 set_mon_vals no callback set 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr: -403> 2024-04-22T20:08:15.549+0000 7f6a4b922800 5 monclient: authenticate success, global_id 6044 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr: -402> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr: -401> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr: -400> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.624 INFO:tasks.workunit.client.0.smithi136.stderr: -399> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -398> 2024-04-22T20:08:15.549+0000 7f6a4b922800 1 librados: init done 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -397> 2024-04-22T20:08:15.551+0000 7f6933e07640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -396> 2024-04-22T20:08:15.551+0000 7f6933e07640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -395> 2024-04-22T20:08:15.551+0000 7f6933e07640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -394> 2024-04-22T20:08:15.551+0000 7f6a4756a640 10 monclient: get_auth_request con 0x7f692c06b2d0 auth_method 0 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -393> 2024-04-22T20:08:15.551+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x557b8e680ac0 auth_method 0 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -392> 2024-04-22T20:08:15.552+0000 7f6a4756a640 10 monclient: get_auth_request con 0x557b8e6a6570 auth_method 0 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -391> 2024-04-22T20:08:15.553+0000 7f6a46d69640 10 monclient: get_auth_request con 0x557b8e6aa760 auth_method 0 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -390> 2024-04-22T20:08:15.554+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x557b8e6a9e50 auth_method 0 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -389> 2024-04-22T20:08:15.554+0000 7f6a4b922800 10 monclient: build_initial_monmap 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -388> 2024-04-22T20:08:15.554+0000 7f6a4b922800 1 build_initial for_mkfs: 0 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr: -387> 2024-04-22T20:08:15.554+0000 7f6a4b922800 10 monclient: monmap: 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:15.625 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr: -386> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding auth protocol: cephx 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr: -385> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding auth protocol: cephx 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr: -384> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding auth protocol: cephx 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr: -383> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr: -382> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.626 INFO:tasks.workunit.client.0.smithi136.stderr: -381> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -380> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -379> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -378> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -377> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -376> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -375> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -374> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -373> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -372> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -371> 2024-04-22T20:08:15.555+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -370> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command objecter_requests cmddesc objecter_requests hook 0x557b8e4e4680 EEXIST 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -369> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient: init 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -368> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding auth protocol: cephx 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -367> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding auth protocol: cephx 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -366> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding auth protocol: cephx 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -365> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.627 INFO:tasks.workunit.client.0.smithi136.stderr: -364> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -363> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -362> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -361> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -360> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -359> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -358> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -357> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -356> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -355> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -354> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -353> 2024-04-22T20:08:15.555+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -352> 2024-04-22T20:08:15.555+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -351> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command rotate-key cmddesc rotate-key hook 0x557b8e6b1930 EEXIST 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -350> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:15.628 INFO:tasks.workunit.client.0.smithi136.stderr: -349> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -348> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-a con 0x557b8e6dde00 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -347> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-c con 0x557b8e6de1e0 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -346> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-b con 0x557b8e6d92e0 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -345> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -344> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -343> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -342> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -341> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): authenticate will time out at 1884.907699s 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -340> 2024-04-22T20:08:15.555+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request con 0x557b8e6de1e0 auth_method 0 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -339> 2024-04-22T20:08:15.555+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -338> 2024-04-22T20:08:15.555+0000 7f6a47d6b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -337> 2024-04-22T20:08:15.555+0000 7f6a47d6b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -336> 2024-04-22T20:08:15.555+0000 7f6a46d69640 10 monclient(hunting): get_auth_request con 0x557b8e6dde00 auth_method 0 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -335> 2024-04-22T20:08:15.555+0000 7f6a46d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.629 INFO:tasks.workunit.client.0.smithi136.stderr: -334> 2024-04-22T20:08:15.555+0000 7f6a46d69640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -333> 2024-04-22T20:08:15.555+0000 7f6a46d69640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -332> 2024-04-22T20:08:15.555+0000 7f6a4756a640 10 monclient(hunting): get_auth_request con 0x557b8e6d92e0 auth_method 0 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -331> 2024-04-22T20:08:15.555+0000 7f6a4756a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -330> 2024-04-22T20:08:15.555+0000 7f6a4756a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -329> 2024-04-22T20:08:15.555+0000 7f6a4756a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -328> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -327> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -326> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -325> 2024-04-22T20:08:15.556+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -324> 2024-04-22T20:08:15.556+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -323> 2024-04-22T20:08:15.556+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -322> 2024-04-22T20:08:15.556+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -321> 2024-04-22T20:08:15.556+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -320> 2024-04-22T20:08:15.556+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.630 INFO:tasks.workunit.client.0.smithi136.stderr: -319> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_done global_id 6047 payload 824 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr: -318> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient: _finish_hunting 0 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr: -317> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 1 monclient: found mon.noname-c 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr: -316> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr: -315> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr: -314> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr: -313> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: dump: 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.631 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -312> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: _finish_auth 0 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -311> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: _check_auth_tickets 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -310> 2024-04-22T20:08:15.556+0000 7f6a4b922800 5 monclient: authenticate success, global_id 6047 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -309> 2024-04-22T20:08:15.556+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -308> 2024-04-22T20:08:15.556+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -307> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -306> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -305> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr: -304> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: dump: 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.632 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr: -303> 2024-04-22T20:08:15.556+0000 7f6a4856c640 4 set_mon_vals no callback set 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr: -302> 2024-04-22T20:08:15.556+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr: -301> 2024-04-22T20:08:15.556+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr: -300> 2024-04-22T20:08:15.558+0000 7f6931602640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:15.633 INFO:tasks.workunit.client.0.smithi136.stderr: -299> 2024-04-22T20:08:15.558+0000 7f6931602640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr: -298> 2024-04-22T20:08:15.558+0000 7f6931602640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr: -297> 2024-04-22T20:08:15.558+0000 7f6a46d69640 10 monclient: get_auth_request con 0x7f692006b1e0 auth_method 0 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr: -296> 2024-04-22T20:08:15.558+0000 7f6a4b922800 10 monclient: get_monmap_and_config 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr: -295> 2024-04-22T20:08:15.558+0000 7f6a4b922800 10 monclient: build_initial_monmap 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr: -294> 2024-04-22T20:08:15.558+0000 7f6a4b922800 1 build_initial for_mkfs: 0 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr: -293> 2024-04-22T20:08:15.558+0000 7f6a4b922800 10 monclient: monmap: 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:15.634 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -292> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -291> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -290> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -289> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -288> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -287> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -286> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -285> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -284> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.635 INFO:tasks.workunit.client.0.smithi136.stderr: -283> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -282> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -281> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -280> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -279> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -278> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -277> 2024-04-22T20:08:15.558+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -276> 2024-04-22T20:08:15.558+0000 7f6a4b922800 10 monclient: init 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -275> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding auth protocol: cephx 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -274> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding auth protocol: cephx 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -273> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding auth protocol: cephx 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -272> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.636 INFO:tasks.workunit.client.0.smithi136.stderr: -271> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -270> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -269> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -268> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -267> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -266> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -265> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -264> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -263> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -262> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -261> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -260> 2024-04-22T20:08:15.558+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.637 INFO:tasks.workunit.client.0.smithi136.stderr: -259> 2024-04-22T20:08:15.559+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -258> 2024-04-22T20:08:15.559+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command rotate-key cmddesc rotate-key hook 0x7fffc13d31b8 EEXIST 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -257> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -256> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient: _add_conns ranks=[1,2,0] 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -255> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-b con 0x557b8e72a470 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -254> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-c con 0x557b8e72a850 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -253> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-a con 0x557b8e7354a0 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -252> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -251> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -250> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -249> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:15.638 INFO:tasks.workunit.client.0.smithi136.stderr: -248> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): authenticate will time out at 1884.911053s 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -247> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): get_auth_request con 0x557b8e7354a0 auth_method 0 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -246> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -245> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -244> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -243> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): get_auth_request con 0x557b8e72a470 auth_method 0 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -242> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -241> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -240> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -239> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request con 0x557b8e72a850 auth_method 0 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -238> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -237> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.639 INFO:tasks.workunit.client.0.smithi136.stderr: -236> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -235> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -234> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -233> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -232> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -231> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -230> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -229> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -228> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -227> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -226> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): handle_auth_done global_id 6366 payload 274 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -225> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient: _finish_hunting 0 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -224> 2024-04-22T20:08:15.559+0000 7f6a4756a640 1 monclient: found mon.noname-a 2024-04-22T20:08:15.640 INFO:tasks.workunit.client.0.smithi136.stderr: -223> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr: -222> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr: -221> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr: -220> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: dump: 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.641 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -219> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: _finish_auth 0 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -218> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: _check_auth_tickets 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -217> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -216> 2024-04-22T20:08:15.559+0000 7f6a4b922800 5 monclient: authenticate success, global_id 6366 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -215> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient: get_monmap_and_config success 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -214> 2024-04-22T20:08:15.559+0000 7f6a4b922800 4 set_mon_vals no callback set 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -213> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -212> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr: -211> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: dump: 2024-04-22T20:08:15.642 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr: -210> 2024-04-22T20:08:15.560+0000 7f6a4b922800 10 monclient: shutdown 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr: -209> 2024-04-22T20:08:15.560+0000 7f6a4b922800 10 monclient: build_initial_monmap 2024-04-22T20:08:15.643 INFO:tasks.workunit.client.0.smithi136.stderr: -208> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 build_initial for_mkfs: 0 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr: -207> 2024-04-22T20:08:15.560+0000 7f6a4b922800 10 monclient: monmap: 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.644 INFO:tasks.workunit.client.0.smithi136.stderr: -206> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -205> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -204> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -203> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -202> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -201> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -200> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -199> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -198> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -197> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -196> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -195> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.645 INFO:tasks.workunit.client.0.smithi136.stderr: -194> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -193> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -192> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -191> 2024-04-22T20:08:15.560+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -190> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 librados: starting msgr at 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -189> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 librados: starting objecter 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -188> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command objecter_requests cmddesc objecter_requests hook 0x557b8e6f4740 EEXIST 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -187> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 librados: setting wanted keys 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -186> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 librados: calling monclient init 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -185> 2024-04-22T20:08:15.560+0000 7f6a4b922800 10 monclient: init 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -184> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding auth protocol: cephx 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -183> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding auth protocol: cephx 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -182> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding auth protocol: cephx 2024-04-22T20:08:15.646 INFO:tasks.workunit.client.0.smithi136.stderr: -181> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -180> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -179> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -178> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -177> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -176> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -175> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -174> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -173> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -172> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -171> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -170> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -169> 2024-04-22T20:08:15.561+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -168> 2024-04-22T20:08:15.561+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -167> 2024-04-22T20:08:15.561+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command rotate-key cmddesc rotate-key hook 0x557b8e6fac48 EEXIST 2024-04-22T20:08:15.647 INFO:tasks.workunit.client.0.smithi136.stderr: -166> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -165> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _add_conns ranks=[2,0,1] 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -164> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-c con 0x557b8e72a470 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -163> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-a con 0x557b8e7354a0 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -162> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-b con 0x557b8e728ff0 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -161> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -160> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -159> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -158> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -157> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): authenticate will time out at 1884.913190s 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -156> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): get_auth_request con 0x557b8e72a470 auth_method 0 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -155> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -154> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -153> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -152> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request con 0x557b8e7354a0 auth_method 0 2024-04-22T20:08:15.648 INFO:tasks.workunit.client.0.smithi136.stderr: -151> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -150> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -149> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -148> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -147> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -146> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -145> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -144> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -143> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -142> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): get_auth_request con 0x557b8e728ff0 auth_method 0 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -141> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -140> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -139> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -138> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -137> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.649 INFO:tasks.workunit.client.0.smithi136.stderr: -136> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr: -135> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): handle_auth_done global_id 6053 payload 824 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr: -134> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient: _finish_hunting 0 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr: -133> 2024-04-22T20:08:15.561+0000 7f6a46d69640 1 monclient: found mon.noname-c 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr: -132> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr: -131> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr: -130> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr: -129> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: dump: 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.650 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr: -128> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: _finish_auth 0 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr: -127> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: _check_auth_tickets 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr: -126> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr: -125> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr: -124> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr: -123> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: dump: 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.651 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -122> 2024-04-22T20:08:15.561+0000 7f692affd640 4 set_mon_vals no callback set 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -121> 2024-04-22T20:08:15.561+0000 7f6a4b922800 5 monclient: authenticate success, global_id 6053 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -120> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -119> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -118> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -117> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -116> 2024-04-22T20:08:15.561+0000 7f6a4b922800 1 librados: init done 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -115> 2024-04-22T20:08:15.561+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cr dump hook 0x557b8e72fc28 2024-04-22T20:08:15.652 INFO:tasks.workunit.client.0.smithi136.stderr: -114> 2024-04-22T20:08:15.563+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -113> 2024-04-22T20:08:15.563+0000 7f69297fa640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -112> 2024-04-22T20:08:15.563+0000 7f69297fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -111> 2024-04-22T20:08:15.563+0000 7f69297fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -110> 2024-04-22T20:08:15.564+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x7f691006b2d0 auth_method 0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -109> 2024-04-22T20:08:15.564+0000 7f6a4756a640 10 monclient: get_auth_request con 0x557b8e74a670 auth_method 0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -108> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -107> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 realm 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -106> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -105> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -104> 2024-04-22T20:08:15.564+0000 7f6a4b922800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -103> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -102> 2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -101> 2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.653 INFO:tasks.workunit.client.0.smithi136.stderr: -100> 2024-04-22T20:08:15.565+0000 7f6a46d69640 10 monclient: get_auth_request con 0x557b8e74e950 auth_method 0 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -99> 2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -98> 2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -97> 2024-04-22T20:08:15.565+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x557b8e7521b0 auth_method 0 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -96> 2024-04-22T20:08:15.566+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -95> 2024-04-22T20:08:15.566+0000 7f6a4b922800 20 searching for the correct realm 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -94> 2024-04-22T20:08:15.566+0000 7f6a4756a640 10 monclient: get_auth_request con 0x557b8e756f50 auth_method 0 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -93> 2024-04-22T20:08:15.567+0000 7f6a46d69640 10 monclient: get_auth_request con 0x7f6a38037930 auth_method 0 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -92> 2024-04-22T20:08:15.567+0000 7f6a47d6b640 0 -- 172.21.15.136:0/991704454 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x557b8e7521b0 msgr2=0x557b8e7545c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -91> 2024-04-22T20:08:15.568+0000 7f69297fa640 1 client.6053.objecter ms_handle_reset 0x557b8e7521b0 session 0x557b8e74dce0 osd.2 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -90> 2024-04-22T20:08:15.568+0000 7f69297fa640 10 monclient: _renew_subs 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -89> 2024-04-22T20:08:15.568+0000 7f69297fa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -88> 2024-04-22T20:08:15.568+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x7f691007ba00 auth_method 0 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -87> 2024-04-22T20:08:15.568+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x7f6a4003c070 auth_method 0 2024-04-22T20:08:15.654 INFO:tasks.workunit.client.0.smithi136.stderr: -86> 2024-04-22T20:08:15.569+0000 7f6a4756a640 10 monclient: get_auth_request con 0x7f6a40060550 auth_method 0 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -85> 2024-04-22T20:08:15.571+0000 7f6a47d6b640 0 -- 172.21.15.136:0/991704454 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f691007ba00 msgr2=0x7f691007de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -84> 2024-04-22T20:08:15.571+0000 7f69297fa640 1 client.6053.objecter ms_handle_reset 0x7f691007ba00 session 0x557b8e74dce0 osd.2 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -83> 2024-04-22T20:08:15.571+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x557b8e7521b0 auth_method 0 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -82> 2024-04-22T20:08:15.572+0000 7f6a46d69640 10 monclient: get_auth_request con 0x7f6a4005fb70 auth_method 0 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -81> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -80> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -79> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -78> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -77> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -76> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -75> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -74> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -73> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.655 INFO:tasks.workunit.client.0.smithi136.stderr: -72> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -71> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -70> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -69> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 zone default found 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -68> 2024-04-22T20:08:15.577+0000 7f6a4b922800 4 Realm: () 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -67> 2024-04-22T20:08:15.577+0000 7f6a4b922800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -66> 2024-04-22T20:08:15.577+0000 7f6a4b922800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -65> 2024-04-22T20:08:15.577+0000 7f6a4b922800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -64> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 zonegroup default 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -63> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -62> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -61> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -60> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -59> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -58> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -57> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 started sync module instance, tier type = 2024-04-22T20:08:15.656 INFO:tasks.workunit.client.0.smithi136.stderr: -56> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -55> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=0 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -54> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=1 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -53> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=2 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -52> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=3 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -51> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=4 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -50> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=5 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -49> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=6 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -48> 2024-04-22T20:08:15.587+0000 7f6a4b922800 20 add_watcher() i=7 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -47> 2024-04-22T20:08:15.587+0000 7f6a4b922800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -46> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding auth protocol: cephx 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -45> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding auth protocol: cephx 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -44> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding auth protocol: cephx 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -43> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -42> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.657 INFO:tasks.workunit.client.0.smithi136.stderr: -41> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -40> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -39> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -38> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -37> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -36> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -35> 2024-04-22T20:08:15.588+0000 7f68fdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -34> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -33> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -32> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -31> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -30> 2024-04-22T20:08:15.588+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -29> 2024-04-22T20:08:15.588+0000 7f6a4b922800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -28> 2024-04-22T20:08:15.588+0000 7f6a4b922800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:15.658 INFO:tasks.workunit.client.0.smithi136.stderr: -27> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cache list hook 0x557b8e729410 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -26> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cache inspect hook 0x557b8e729410 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -25> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cache erase hook 0x557b8e729410 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -24> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cache zap hook 0x557b8e729410 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -23> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 note: GC not initialized 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -22> 2024-04-22T20:08:15.589+0000 7f68fcfe1640 20 reqs_thread_entry: start 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -21> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command sync trace show hook 0x7f6a38012070 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -20> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command sync trace history hook 0x7f6a38012070 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -19> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command sync trace active hook 0x7f6a38012070 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -18> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command sync trace active_short hook 0x7f6a38012070 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -17> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -16> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 Filter name: none 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -15> 2024-04-22T20:08:15.589+0000 7f68767fc640 20 reqs_thread_entry: start 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -14> 2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -13> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.659 INFO:tasks.workunit.client.0.smithi136.stderr: -12> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -11> 2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -10> 2024-04-22T20:08:15.589+0000 7f6a4b922800 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -9> 2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 : miss 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -8> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -7> 2024-04-22T20:08:15.590+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -6> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -5> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -4> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -3> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -2> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: -1> 2024-04-22T20:08:15.593+0000 7f6a4b922800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6a4b922800 time 2024-04-22T20:08:15.593950+0000 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.660 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f6a4cd81f5c] 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f6a4cd8211a] 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557b8cfe4dc1] 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 4: radosgw-admin(+0x83755a) [0x557b8cfee55a] 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x25b) [0x557b8cff2dbb] 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 6: main() 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 7: /lib64/libc.so.6(+0x29590) [0x7f6a4bc29590] 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 8: __libc_start_main() 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 9: _start() 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 0> 2024-04-22T20:08:15.594+0000 7f6a4b922800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7f6a4b922800 thread_name:radosgw-admin 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:15.661 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6a4bc3e6f0] 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f6a4bc8b94c] 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f6a4cd81fb6] 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f6a4cd8211a] 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557b8cfe4dc1] 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 8: radosgw-admin(+0x83755a) [0x557b8cfee55a] 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x25b) [0x557b8cff2dbb] 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 10: main() 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 11: /lib64/libc.so.6(+0x29590) [0x7f6a4bc29590] 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 12: __libc_start_main() 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 13: _start() 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:08:15.662 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr:--- logging levels --- 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 none 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 lockdep 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 context 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 crush 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_balancer 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_locker 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log_expire 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_migrator 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 3/ 5 mds_quiesce 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 buffer 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 timer 2024-04-22T20:08:15.663 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 filer 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 striper 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 objecter 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rados 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_mirror 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_replay 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_pwl 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 journaler 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objectcacher 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 immutable_obj_cache 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 client 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 osd 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 optracker 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objclass 2024-04-22T20:08:15.664 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 filestore 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 journal 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 ms 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mon 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 0/10 monc 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 paxos 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 tp 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 auth 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 crypto 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 finisher 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 reserver 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 heartbeatmap 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 perfcounter 2024-04-22T20:08:15.665 INFO:tasks.workunit.client.0.smithi136.stderr: 20/20 rgw 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_sync 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_datacache 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_access 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_dbstore 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_flight 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_lifecycle 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_notification 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 javaclient 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 asok 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 throttle 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 refs 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 compressor 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluestore 2024-04-22T20:08:15.666 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluefs 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 bdev 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 kstore 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 4/ 5 rocksdb 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 fuse 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 2/ 5 mgr 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mgrc 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 dpdk 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 eventtrace 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 prioritycache 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 test 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephfs_mirror 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephsqlite 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 crimson_interrupt 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore 2024-04-22T20:08:15.667 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_onode 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_odata 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_omap 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_tm 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_t 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cleaner 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_epm 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_lba 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cache 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_journal 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_device 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_backref 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 alienstore 2024-04-22T20:08:15.668 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mclock 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cyanstore 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 ceph_exporter 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 memstore 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 trace 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: -2/-2 (syslog threshold) 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 99/99 (stderr threshold) 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 7f68767fc640 / http_manager 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 7f68fcfe1640 / http_manager 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 7f68fdfe3640 / rgw_dt_lg_renew 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 7f69297fa640 / ms_dispatch 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 7f692affd640 / io_context_pool 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6931602640 / ms_dispatch 2024-04-22T20:08:15.669 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6933e07640 / ms_dispatch 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: 7f693560a640 / io_context_pool 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a46d69640 / msgr-worker-2 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a4756a640 / msgr-worker-1 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a47d6b640 / msgr-worker-0 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a4856c640 / io_context_pool 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a4b922800 / radosgw-admin 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: max_recent 500 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: max_new 1000 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: log_file /var/log/ceph/ceph-client.admin.89350.log 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr:--- end dump of recent events --- 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr:--- begin dump of recent events --- 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: -499> 2024-04-22T20:08:15.547+0000 7f693560a640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.670 INFO:tasks.workunit.client.0.smithi136.stderr: -498> 2024-04-22T20:08:15.547+0000 7f693560a640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr: -497> 2024-04-22T20:08:15.547+0000 7f693560a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr: -496> 2024-04-22T20:08:15.547+0000 7f693560a640 10 monclient: dump: 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr: -495> 2024-04-22T20:08:15.547+0000 7f6a4b922800 10 monclient: get_monmap_and_config success 2024-04-22T20:08:15.671 INFO:tasks.workunit.client.0.smithi136.stderr: -494> 2024-04-22T20:08:15.547+0000 7f6a4b922800 4 set_mon_vals no callback set 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr: -493> 2024-04-22T20:08:15.548+0000 7f6a4b922800 10 monclient: shutdown 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr: -492> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 asok(0x557b8e43ad00) unregister_commands rotate-key 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr: -491> 2024-04-22T20:08:15.548+0000 7f6a4b922800 10 monclient: build_initial_monmap 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr: -490> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 build_initial for_mkfs: 0 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr: -489> 2024-04-22T20:08:15.548+0000 7f6a4b922800 10 monclient: monmap: 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:15.672 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -488> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding auth protocol: cephx 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -487> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding auth protocol: cephx 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -486> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding auth protocol: cephx 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -485> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -484> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -483> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -482> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -481> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -480> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -479> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -478> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -477> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -476> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.673 INFO:tasks.workunit.client.0.smithi136.stderr: -475> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: crc 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -474> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65dbc0) adding con mode: secure 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -473> 2024-04-22T20:08:15.548+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -472> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 librados: starting msgr at 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -471> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 librados: starting objecter 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -470> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command objecter_requests hook 0x557b8e65e870 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -469> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 librados: setting wanted keys 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -468> 2024-04-22T20:08:15.548+0000 7f6a4b922800 1 librados: calling monclient init 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -467> 2024-04-22T20:08:15.548+0000 7f6a4b922800 10 monclient: init 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -466> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding auth protocol: cephx 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -465> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding auth protocol: cephx 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -464> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding auth protocol: cephx 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -463> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -462> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.674 INFO:tasks.workunit.client.0.smithi136.stderr: -461> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -460> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -459> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -458> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -457> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -456> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -455> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -454> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -453> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: crc 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -452> 2024-04-22T20:08:15.548+0000 7f6a4b922800 5 AuthRegistry(0x557b8e65ad40) adding con mode: secure 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -451> 2024-04-22T20:08:15.548+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -450> 2024-04-22T20:08:15.549+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -449> 2024-04-22T20:08:15.549+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command rotate-key hook 0x557b8e65ae88 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -448> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -447> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:08:15.675 INFO:tasks.workunit.client.0.smithi136.stderr: -446> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-a con 0x557b8e512480 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -445> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-c con 0x557b8e59c870 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -444> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-b con 0x557b8e5a2680 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -443> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -442> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -441> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -440> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -439> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient(hunting): authenticate will time out at 1884.901012s 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -438> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): get_auth_request con 0x557b8e59c870 auth_method 0 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -437> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -436> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -435> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -434> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): get_auth_request con 0x557b8e512480 auth_method 0 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -433> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.676 INFO:tasks.workunit.client.0.smithi136.stderr: -432> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -431> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -430> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -429> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -428> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -427> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -426> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -425> 2024-04-22T20:08:15.549+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -424> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request con 0x557b8e5a2680 auth_method 0 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -423> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -422> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -421> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.677 INFO:tasks.workunit.client.0.smithi136.stderr: -420> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -419> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -418> 2024-04-22T20:08:15.549+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -417> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient(hunting): handle_auth_done global_id 6044 payload 824 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -416> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient: _finish_hunting 0 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -415> 2024-04-22T20:08:15.549+0000 7f6a46d69640 1 monclient: found mon.noname-c 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -414> 2024-04-22T20:08:15.549+0000 7f6a46d69640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -413> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -412> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr: -411> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: dump: 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.678 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr: -410> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: _finish_auth 0 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr: -409> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: _check_auth_tickets 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr: -408> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr: -407> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr: -406> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr: -405> 2024-04-22T20:08:15.549+0000 7f6933e07640 10 monclient: dump: 2024-04-22T20:08:15.679 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr: -404> 2024-04-22T20:08:15.549+0000 7f693560a640 4 set_mon_vals no callback set 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr: -403> 2024-04-22T20:08:15.549+0000 7f6a4b922800 5 monclient: authenticate success, global_id 6044 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr: -402> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr: -401> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr: -400> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr: -399> 2024-04-22T20:08:15.549+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.680 INFO:tasks.workunit.client.0.smithi136.stderr: -398> 2024-04-22T20:08:15.549+0000 7f6a4b922800 1 librados: init done 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -397> 2024-04-22T20:08:15.551+0000 7f6933e07640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -396> 2024-04-22T20:08:15.551+0000 7f6933e07640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -395> 2024-04-22T20:08:15.551+0000 7f6933e07640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -394> 2024-04-22T20:08:15.551+0000 7f6a4756a640 10 monclient: get_auth_request con 0x7f692c06b2d0 auth_method 0 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -393> 2024-04-22T20:08:15.551+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x557b8e680ac0 auth_method 0 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -392> 2024-04-22T20:08:15.552+0000 7f6a4756a640 10 monclient: get_auth_request con 0x557b8e6a6570 auth_method 0 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -391> 2024-04-22T20:08:15.553+0000 7f6a46d69640 10 monclient: get_auth_request con 0x557b8e6aa760 auth_method 0 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -390> 2024-04-22T20:08:15.554+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x557b8e6a9e50 auth_method 0 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -389> 2024-04-22T20:08:15.554+0000 7f6a4b922800 10 monclient: build_initial_monmap 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -388> 2024-04-22T20:08:15.554+0000 7f6a4b922800 1 build_initial for_mkfs: 0 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr: -387> 2024-04-22T20:08:15.554+0000 7f6a4b922800 10 monclient: monmap: 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:15.681 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: -386> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding auth protocol: cephx 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: -385> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding auth protocol: cephx 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: -384> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding auth protocol: cephx 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: -383> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: -382> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: -381> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: -380> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.682 INFO:tasks.workunit.client.0.smithi136.stderr: -379> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -378> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -377> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -376> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -375> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -374> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -373> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: crc 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -372> 2024-04-22T20:08:15.554+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b2a00) adding con mode: secure 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -371> 2024-04-22T20:08:15.555+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -370> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command objecter_requests cmddesc objecter_requests hook 0x557b8e4e4680 EEXIST 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -369> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient: init 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -368> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding auth protocol: cephx 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -367> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding auth protocol: cephx 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -366> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding auth protocol: cephx 2024-04-22T20:08:15.683 INFO:tasks.workunit.client.0.smithi136.stderr: -365> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -364> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -363> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -362> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -361> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -360> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -359> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -358> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -357> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -356> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -355> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: crc 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -354> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6b17e8) adding con mode: secure 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -353> 2024-04-22T20:08:15.555+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.684 INFO:tasks.workunit.client.0.smithi136.stderr: -352> 2024-04-22T20:08:15.555+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -351> 2024-04-22T20:08:15.555+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command rotate-key cmddesc rotate-key hook 0x557b8e6b1930 EEXIST 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -350> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -349> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -348> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-a con 0x557b8e6dde00 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -347> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-c con 0x557b8e6de1e0 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -346> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-b con 0x557b8e6d92e0 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -345> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -344> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -343> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -342> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -341> 2024-04-22T20:08:15.555+0000 7f6a4b922800 10 monclient(hunting): authenticate will time out at 1884.907699s 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -340> 2024-04-22T20:08:15.555+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request con 0x557b8e6de1e0 auth_method 0 2024-04-22T20:08:15.685 INFO:tasks.workunit.client.0.smithi136.stderr: -339> 2024-04-22T20:08:15.555+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -338> 2024-04-22T20:08:15.555+0000 7f6a47d6b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -337> 2024-04-22T20:08:15.555+0000 7f6a47d6b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -336> 2024-04-22T20:08:15.555+0000 7f6a46d69640 10 monclient(hunting): get_auth_request con 0x557b8e6dde00 auth_method 0 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -335> 2024-04-22T20:08:15.555+0000 7f6a46d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -334> 2024-04-22T20:08:15.555+0000 7f6a46d69640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -333> 2024-04-22T20:08:15.555+0000 7f6a46d69640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -332> 2024-04-22T20:08:15.555+0000 7f6a4756a640 10 monclient(hunting): get_auth_request con 0x557b8e6d92e0 auth_method 0 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -331> 2024-04-22T20:08:15.555+0000 7f6a4756a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -330> 2024-04-22T20:08:15.555+0000 7f6a4756a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -329> 2024-04-22T20:08:15.555+0000 7f6a4756a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -328> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -327> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.686 INFO:tasks.workunit.client.0.smithi136.stderr: -326> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -325> 2024-04-22T20:08:15.556+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -324> 2024-04-22T20:08:15.556+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -323> 2024-04-22T20:08:15.556+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -322> 2024-04-22T20:08:15.556+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -321> 2024-04-22T20:08:15.556+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -320> 2024-04-22T20:08:15.556+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -319> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_done global_id 6047 payload 824 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -318> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient: _finish_hunting 0 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -317> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 1 monclient: found mon.noname-c 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -316> 2024-04-22T20:08:15.556+0000 7f6a47d6b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -315> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -314> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:15.687 INFO:tasks.workunit.client.0.smithi136.stderr: -313> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: dump: 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr: -312> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: _finish_auth 0 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr: -311> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: _check_auth_tickets 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr: -310> 2024-04-22T20:08:15.556+0000 7f6a4b922800 5 monclient: authenticate success, global_id 6047 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr: -309> 2024-04-22T20:08:15.556+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.688 INFO:tasks.workunit.client.0.smithi136.stderr: -308> 2024-04-22T20:08:15.556+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr: -307> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr: -306> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr: -305> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr: -304> 2024-04-22T20:08:15.556+0000 7f6931602640 10 monclient: dump: 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.689 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -303> 2024-04-22T20:08:15.556+0000 7f6a4856c640 4 set_mon_vals no callback set 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -302> 2024-04-22T20:08:15.556+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -301> 2024-04-22T20:08:15.556+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -300> 2024-04-22T20:08:15.558+0000 7f6931602640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -299> 2024-04-22T20:08:15.558+0000 7f6931602640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -298> 2024-04-22T20:08:15.558+0000 7f6931602640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -297> 2024-04-22T20:08:15.558+0000 7f6a46d69640 10 monclient: get_auth_request con 0x7f692006b1e0 auth_method 0 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -296> 2024-04-22T20:08:15.558+0000 7f6a4b922800 10 monclient: get_monmap_and_config 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -295> 2024-04-22T20:08:15.558+0000 7f6a4b922800 10 monclient: build_initial_monmap 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -294> 2024-04-22T20:08:15.558+0000 7f6a4b922800 1 build_initial for_mkfs: 0 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr: -293> 2024-04-22T20:08:15.558+0000 7f6a4b922800 10 monclient: monmap: 2024-04-22T20:08:15.690 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr: -292> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr: -291> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr: -290> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.691 INFO:tasks.workunit.client.0.smithi136.stderr: -289> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -288> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -287> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -286> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -285> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -284> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -283> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -282> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -281> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -280> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -279> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -278> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -277> 2024-04-22T20:08:15.558+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -276> 2024-04-22T20:08:15.558+0000 7f6a4b922800 10 monclient: init 2024-04-22T20:08:15.692 INFO:tasks.workunit.client.0.smithi136.stderr: -275> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding auth protocol: cephx 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -274> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding auth protocol: cephx 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -273> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding auth protocol: cephx 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -272> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -271> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -270> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -269> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -268> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -267> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -266> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -265> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -264> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -263> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -262> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: crc 2024-04-22T20:08:15.693 INFO:tasks.workunit.client.0.smithi136.stderr: -261> 2024-04-22T20:08:15.558+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3070) adding con mode: secure 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -260> 2024-04-22T20:08:15.558+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -259> 2024-04-22T20:08:15.559+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -258> 2024-04-22T20:08:15.559+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command rotate-key cmddesc rotate-key hook 0x7fffc13d31b8 EEXIST 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -257> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -256> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient: _add_conns ranks=[1,2,0] 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -255> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-b con 0x557b8e72a470 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -254> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-c con 0x557b8e72a850 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -253> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-a con 0x557b8e7354a0 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -252> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -251> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -250> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -249> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -248> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient(hunting): authenticate will time out at 1884.911053s 2024-04-22T20:08:15.694 INFO:tasks.workunit.client.0.smithi136.stderr: -247> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): get_auth_request con 0x557b8e7354a0 auth_method 0 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -246> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -245> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -244> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -243> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): get_auth_request con 0x557b8e72a470 auth_method 0 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -242> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -241> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -240> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -239> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request con 0x557b8e72a850 auth_method 0 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -238> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -237> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -236> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -235> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.695 INFO:tasks.workunit.client.0.smithi136.stderr: -234> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -233> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -232> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -231> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -230> 2024-04-22T20:08:15.559+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -229> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -228> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -227> 2024-04-22T20:08:15.559+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -226> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient(hunting): handle_auth_done global_id 6366 payload 274 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -225> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient: _finish_hunting 0 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -224> 2024-04-22T20:08:15.559+0000 7f6a4756a640 1 monclient: found mon.noname-a 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -223> 2024-04-22T20:08:15.559+0000 7f6a4756a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -222> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.696 INFO:tasks.workunit.client.0.smithi136.stderr: -221> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr: -220> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: dump: 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr: -219> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: _finish_auth 0 2024-04-22T20:08:15.697 INFO:tasks.workunit.client.0.smithi136.stderr: -218> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: _check_auth_tickets 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr: -217> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr: -216> 2024-04-22T20:08:15.559+0000 7f6a4b922800 5 monclient: authenticate success, global_id 6366 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr: -215> 2024-04-22T20:08:15.559+0000 7f6a4b922800 10 monclient: get_monmap_and_config success 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr: -214> 2024-04-22T20:08:15.559+0000 7f6a4b922800 4 set_mon_vals no callback set 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr: -213> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr: -212> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr: -211> 2024-04-22T20:08:15.559+0000 7f692affd640 10 monclient: dump: 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.698 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr: -210> 2024-04-22T20:08:15.560+0000 7f6a4b922800 10 monclient: shutdown 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr: -209> 2024-04-22T20:08:15.560+0000 7f6a4b922800 10 monclient: build_initial_monmap 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr: -208> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 build_initial for_mkfs: 0 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr: -207> 2024-04-22T20:08:15.560+0000 7f6a4b922800 10 monclient: monmap: 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:15.699 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -206> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -205> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -204> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding auth protocol: cephx 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -203> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -202> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -201> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -200> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -199> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.700 INFO:tasks.workunit.client.0.smithi136.stderr: -198> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -197> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -196> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -195> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -194> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -193> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: crc 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -192> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fb930) adding con mode: secure 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -191> 2024-04-22T20:08:15.560+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -190> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 librados: starting msgr at 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -189> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 librados: starting objecter 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -188> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command objecter_requests cmddesc objecter_requests hook 0x557b8e6f4740 EEXIST 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -187> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 librados: setting wanted keys 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -186> 2024-04-22T20:08:15.560+0000 7f6a4b922800 1 librados: calling monclient init 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -185> 2024-04-22T20:08:15.560+0000 7f6a4b922800 10 monclient: init 2024-04-22T20:08:15.701 INFO:tasks.workunit.client.0.smithi136.stderr: -184> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding auth protocol: cephx 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -183> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding auth protocol: cephx 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -182> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding auth protocol: cephx 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -181> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -180> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -179> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -178> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -177> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -176> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -175> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -174> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -173> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -172> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -171> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: crc 2024-04-22T20:08:15.702 INFO:tasks.workunit.client.0.smithi136.stderr: -170> 2024-04-22T20:08:15.560+0000 7f6a4b922800 5 AuthRegistry(0x557b8e6fab00) adding con mode: secure 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -169> 2024-04-22T20:08:15.561+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -168> 2024-04-22T20:08:15.561+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -167> 2024-04-22T20:08:15.561+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command rotate-key cmddesc rotate-key hook 0x557b8e6fac48 EEXIST 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -166> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -165> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _add_conns ranks=[2,0,1] 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -164> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-c con 0x557b8e72a470 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -163> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-a con 0x557b8e7354a0 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -162> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): picked mon.noname-b con 0x557b8e728ff0 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -161> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -160> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -159> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -158> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -157> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient(hunting): authenticate will time out at 1884.913190s 2024-04-22T20:08:15.703 INFO:tasks.workunit.client.0.smithi136.stderr: -156> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): get_auth_request con 0x557b8e72a470 auth_method 0 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -155> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -154> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -153> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -152> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request con 0x557b8e7354a0 auth_method 0 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -151> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -150> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -149> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -148> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -147> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -146> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -145> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -144> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -143> 2024-04-22T20:08:15.561+0000 7f6a47d6b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -142> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): get_auth_request con 0x557b8e728ff0 auth_method 0 2024-04-22T20:08:15.704 INFO:tasks.workunit.client.0.smithi136.stderr: -141> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -140> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -139> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -138> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -137> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -136> 2024-04-22T20:08:15.561+0000 7f6a4756a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -135> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient(hunting): handle_auth_done global_id 6053 payload 824 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -134> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient: _finish_hunting 0 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -133> 2024-04-22T20:08:15.561+0000 7f6a46d69640 1 monclient: found mon.noname-c 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -132> 2024-04-22T20:08:15.561+0000 7f6a46d69640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -131> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -130> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr: -129> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: dump: 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.705 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr: -128> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: _finish_auth 0 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr: -127> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: _check_auth_tickets 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr: -126> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr: -125> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:15.706 INFO:tasks.workunit.client.0.smithi136.stderr: -124> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr: -123> 2024-04-22T20:08:15.561+0000 7f69297fa640 10 monclient: dump: 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr: -122> 2024-04-22T20:08:15.561+0000 7f692affd640 4 set_mon_vals no callback set 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr: -121> 2024-04-22T20:08:15.561+0000 7f6a4b922800 5 monclient: authenticate success, global_id 6053 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr: -120> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.707 INFO:tasks.workunit.client.0.smithi136.stderr: -119> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -118> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _renew_subs 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -117> 2024-04-22T20:08:15.561+0000 7f6a4b922800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -116> 2024-04-22T20:08:15.561+0000 7f6a4b922800 1 librados: init done 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -115> 2024-04-22T20:08:15.561+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cr dump hook 0x557b8e72fc28 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -114> 2024-04-22T20:08:15.563+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -113> 2024-04-22T20:08:15.563+0000 7f69297fa640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -112> 2024-04-22T20:08:15.563+0000 7f69297fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -111> 2024-04-22T20:08:15.563+0000 7f69297fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -110> 2024-04-22T20:08:15.564+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x7f691006b2d0 auth_method 0 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -109> 2024-04-22T20:08:15.564+0000 7f6a4756a640 10 monclient: get_auth_request con 0x557b8e74a670 auth_method 0 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -108> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -107> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 realm 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -106> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -105> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.708 INFO:tasks.workunit.client.0.smithi136.stderr: -104> 2024-04-22T20:08:15.564+0000 7f6a4b922800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -103> 2024-04-22T20:08:15.564+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -102> 2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -101> 2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -100> 2024-04-22T20:08:15.565+0000 7f6a46d69640 10 monclient: get_auth_request con 0x557b8e74e950 auth_method 0 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -99> 2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -98> 2024-04-22T20:08:15.565+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -97> 2024-04-22T20:08:15.565+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x557b8e7521b0 auth_method 0 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -96> 2024-04-22T20:08:15.566+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -95> 2024-04-22T20:08:15.566+0000 7f6a4b922800 20 searching for the correct realm 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -94> 2024-04-22T20:08:15.566+0000 7f6a4756a640 10 monclient: get_auth_request con 0x557b8e756f50 auth_method 0 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -93> 2024-04-22T20:08:15.567+0000 7f6a46d69640 10 monclient: get_auth_request con 0x7f6a38037930 auth_method 0 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -92> 2024-04-22T20:08:15.567+0000 7f6a47d6b640 0 -- 172.21.15.136:0/991704454 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x557b8e7521b0 msgr2=0x557b8e7545c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -91> 2024-04-22T20:08:15.568+0000 7f69297fa640 1 client.6053.objecter ms_handle_reset 0x557b8e7521b0 session 0x557b8e74dce0 osd.2 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -90> 2024-04-22T20:08:15.568+0000 7f69297fa640 10 monclient: _renew_subs 2024-04-22T20:08:15.709 INFO:tasks.workunit.client.0.smithi136.stderr: -89> 2024-04-22T20:08:15.568+0000 7f69297fa640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -88> 2024-04-22T20:08:15.568+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x7f691007ba00 auth_method 0 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -87> 2024-04-22T20:08:15.568+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x7f6a4003c070 auth_method 0 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -86> 2024-04-22T20:08:15.569+0000 7f6a4756a640 10 monclient: get_auth_request con 0x7f6a40060550 auth_method 0 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -85> 2024-04-22T20:08:15.571+0000 7f6a47d6b640 0 -- 172.21.15.136:0/991704454 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f691007ba00 msgr2=0x7f691007de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -84> 2024-04-22T20:08:15.571+0000 7f69297fa640 1 client.6053.objecter ms_handle_reset 0x7f691007ba00 session 0x557b8e74dce0 osd.2 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -83> 2024-04-22T20:08:15.571+0000 7f6a47d6b640 10 monclient: get_auth_request con 0x557b8e7521b0 auth_method 0 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -82> 2024-04-22T20:08:15.572+0000 7f6a46d69640 10 monclient: get_auth_request con 0x7f6a4005fb70 auth_method 0 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -81> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -80> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -79> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -78> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -77> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:15.710 INFO:tasks.workunit.client.0.smithi136.stderr: -76> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -75> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -74> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -73> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -72> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -71> 2024-04-22T20:08:15.576+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -70> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -69> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 zone default found 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -68> 2024-04-22T20:08:15.577+0000 7f6a4b922800 4 Realm: () 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -67> 2024-04-22T20:08:15.577+0000 7f6a4b922800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -66> 2024-04-22T20:08:15.577+0000 7f6a4b922800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -65> 2024-04-22T20:08:15.577+0000 7f6a4b922800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -64> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 zonegroup default 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -63> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -62> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.711 INFO:tasks.workunit.client.0.smithi136.stderr: -61> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -60> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -59> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -58> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -57> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 started sync module instance, tier type = 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -56> 2024-04-22T20:08:15.577+0000 7f6a4b922800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -55> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=0 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -54> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=1 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -53> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=2 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -52> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=3 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -51> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=4 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -50> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=5 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -49> 2024-04-22T20:08:15.586+0000 7f6a4b922800 20 add_watcher() i=6 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -48> 2024-04-22T20:08:15.587+0000 7f6a4b922800 20 add_watcher() i=7 2024-04-22T20:08:15.712 INFO:tasks.workunit.client.0.smithi136.stderr: -47> 2024-04-22T20:08:15.587+0000 7f6a4b922800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -46> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding auth protocol: cephx 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -45> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding auth protocol: cephx 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -44> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding auth protocol: cephx 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -43> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -42> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -41> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -40> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -39> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -38> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -37> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -36> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -35> 2024-04-22T20:08:15.588+0000 7f68fdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -34> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.713 INFO:tasks.workunit.client.0.smithi136.stderr: -33> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -32> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: crc 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -31> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 AuthRegistry(0x7fffc13d3500) adding con mode: secure 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -30> 2024-04-22T20:08:15.588+0000 7f6a4b922800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -29> 2024-04-22T20:08:15.588+0000 7f6a4b922800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -28> 2024-04-22T20:08:15.588+0000 7f6a4b922800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -27> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cache list hook 0x557b8e729410 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -26> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cache inspect hook 0x557b8e729410 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -25> 2024-04-22T20:08:15.588+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cache erase hook 0x557b8e729410 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -24> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command cache zap hook 0x557b8e729410 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -23> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 note: GC not initialized 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -22> 2024-04-22T20:08:15.589+0000 7f68fcfe1640 20 reqs_thread_entry: start 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -21> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command sync trace show hook 0x7f6a38012070 2024-04-22T20:08:15.714 INFO:tasks.workunit.client.0.smithi136.stderr: -20> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command sync trace history hook 0x7f6a38012070 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -19> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command sync trace active hook 0x7f6a38012070 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -18> 2024-04-22T20:08:15.589+0000 7f6a4b922800 5 asok(0x557b8e43ad00) register_command sync trace active_short hook 0x7f6a38012070 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -17> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -16> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 Filter name: none 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -15> 2024-04-22T20:08:15.589+0000 7f68767fc640 20 reqs_thread_entry: start 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -14> 2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -13> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -12> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -11> 2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -10> 2024-04-22T20:08:15.589+0000 7f6a4b922800 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -9> 2024-04-22T20:08:15.589+0000 7f6a4b922800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 : miss 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -8> 2024-04-22T20:08:15.589+0000 7f6a4b922800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.715 INFO:tasks.workunit.client.0.smithi136.stderr: -7> 2024-04-22T20:08:15.590+0000 7f6a4b922800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: -6> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: -5> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: -4> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: -3> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: -2> 2024-04-22T20:08:15.590+0000 7f6a4b922800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: -1> 2024-04-22T20:08:15.593+0000 7f6a4b922800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f6a4b922800 time 2024-04-22T20:08:15.593950+0000 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f6a4cd81f5c] 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f6a4cd8211a] 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557b8cfe4dc1] 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: 4: radosgw-admin(+0x83755a) [0x557b8cfee55a] 2024-04-22T20:08:15.716 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x25b) [0x557b8cff2dbb] 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 6: main() 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 7: /lib64/libc.so.6(+0x29590) [0x7f6a4bc29590] 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 8: __libc_start_main() 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 9: _start() 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 0> 2024-04-22T20:08:15.594+0000 7f6a4b922800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7f6a4b922800 thread_name:radosgw-admin 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f6a4bc3e6f0] 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7f6a4bc8b94c] 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f6a4cd81fb6] 2024-04-22T20:08:15.717 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7f6a4cd8211a] 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x557b8cfe4dc1] 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 8: radosgw-admin(+0x83755a) [0x557b8cfee55a] 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x25b) [0x557b8cff2dbb] 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 10: main() 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 11: /lib64/libc.so.6(+0x29590) [0x7f6a4bc29590] 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 12: __libc_start_main() 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 13: _start() 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr:--- logging levels --- 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 none 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 lockdep 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 context 2024-04-22T20:08:15.718 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 crush 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_balancer 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_locker 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log_expire 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_migrator 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 3/ 5 mds_quiesce 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 buffer 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 timer 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 filer 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 striper 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 objecter 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rados 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd 2024-04-22T20:08:15.719 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_mirror 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_replay 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_pwl 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 journaler 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objectcacher 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 immutable_obj_cache 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 client 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 osd 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 optracker 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objclass 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 filestore 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 journal 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 ms 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mon 2024-04-22T20:08:15.720 INFO:tasks.workunit.client.0.smithi136.stderr: 0/10 monc 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 paxos 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 tp 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 auth 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 crypto 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 finisher 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 reserver 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 heartbeatmap 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 perfcounter 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 20/20 rgw 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_sync 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_datacache 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_access 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_dbstore 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_flight 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_lifecycle 2024-04-22T20:08:15.721 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_notification 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 javaclient 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 asok 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 throttle 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 refs 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 compressor 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluestore 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluefs 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 bdev 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 kstore 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 4/ 5 rocksdb 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 fuse 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 2/ 5 mgr 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mgrc 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 dpdk 2024-04-22T20:08:15.722 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 eventtrace 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 prioritycache 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 test 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephfs_mirror 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephsqlite 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 crimson_interrupt 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_onode 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_odata 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_omap 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_tm 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_t 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cleaner 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_epm 2024-04-22T20:08:15.723 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_lba 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cache 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_journal 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_device 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_backref 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 alienstore 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mclock 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cyanstore 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 ceph_exporter 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 memstore 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 trace 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: -2/-2 (syslog threshold) 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 99/99 (stderr threshold) 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 7f68767fc640 / http_manager 2024-04-22T20:08:15.724 INFO:tasks.workunit.client.0.smithi136.stderr: 7f68fcfe1640 / http_manager 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f68fdfe3640 / rgw_dt_lg_renew 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f69297fa640 / ms_dispatch 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f692affd640 / io_context_pool 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6931602640 / ms_dispatch 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6933e07640 / ms_dispatch 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f693560a640 / io_context_pool 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a46d69640 / msgr-worker-2 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a4756a640 / msgr-worker-1 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a47d6b640 / msgr-worker-0 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a4856c640 / io_context_pool 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7f6a4b922800 / radosgw-admin 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: max_recent 500 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: max_new 1000 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr: log_file /var/lib/ceph/crash/2024-04-22T20:08:15.595564Z_8607ce2a-5024-497d-9e14-79849fc0c854/log 2024-04-22T20:08:15.725 INFO:tasks.workunit.client.0.smithi136.stderr:--- end dump of recent events --- 2024-04-22T20:08:15.726 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.664+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.665+0000 7fa178e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.665+0000 7fa178e98800 20 realm 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.665+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.665+0000 7fa178e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.665+0000 7fa178e98800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.665+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.666+0000 7fa178e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.666+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.666+0000 7fa178e98800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.666+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.667+0000 7fa178e98800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.667+0000 7fa178e98800 20 searching for the correct realm 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.669+0000 7fa177198640 0 -- 172.21.15.136:0/1123080289 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x563c579bcd40 msgr2=0x563c579bf130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.672+0000 7fa177198640 0 -- 172.21.15.136:0/1123080289 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7fa04007ba00 msgr2=0x7fa04007de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.677+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 zone default found 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 4 Realm: () 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 zonegroup default 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.678+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.679+0000 7fa178e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.679+0000 7fa178e98800 20 started sync module instance, tier type = 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.679+0000 7fa178e98800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 20 add_watcher() i=0 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 20 add_watcher() i=1 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 20 add_watcher() i=2 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 20 add_watcher() i=3 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 20 add_watcher() i=4 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 20 add_watcher() i=5 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 20 add_watcher() i=6 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 20 add_watcher() i=7 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.687+0000 7fa178e98800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.689+0000 7fa02d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.690+0000 7fa178e98800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.690+0000 7fa178e98800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:15.734 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.690+0000 7fa178e98800 5 note: GC not initialized 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.690+0000 7f9fa7fff640 20 reqs_thread_entry: start 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.690+0000 7fa178e98800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.690+0000 7fa178e98800 20 Filter name: none 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.690+0000 7f9fa5ffb640 20 reqs_thread_entry: start 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.690+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.691+0000 7fa178e98800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.691+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.691+0000 7fa178e98800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.691+0000 7fa178e98800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.691+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.692+0000 7fa178e98800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.692+0000 7fa178e98800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.692+0000 7fa178e98800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:15.735 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.692+0000 7fa178e98800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.693+0000 7fa02d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.695+0000 7fa178e98800 20 remove_watcher() i=0 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.695+0000 7fa178e98800 2 removed watcher, disabling cache 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.696+0000 7fa178e98800 20 remove_watcher() i=1 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.697+0000 7fa178e98800 20 remove_watcher() i=2 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.698+0000 7fa178e98800 20 remove_watcher() i=3 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.699+0000 7fa178e98800 20 remove_watcher() i=4 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.700+0000 7fa178e98800 20 remove_watcher() i=5 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.701+0000 7fa178e98800 20 remove_watcher() i=6 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.702+0000 7fa178e98800 20 remove_watcher() i=7 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-set-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:15.736 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:15.737 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:15.737 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:15.745 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:15.745 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:15.804 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:08:15.834 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:15.834 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:15.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.867+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.868+0000 7feb35742800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.868+0000 7feb35742800 20 realm 2024-04-22T20:08:15.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.868+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.869+0000 7feb35742800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.869+0000 7feb35742800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:15.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.869+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.869+0000 7feb35742800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.869+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.870+0000 7feb35742800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.870+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.870+0000 7feb35742800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:15.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.870+0000 7feb35742800 20 searching for the correct realm 2024-04-22T20:08:15.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.872+0000 7feb33a42640 0 -- 172.21.15.136:0/2050568550 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x5610924162d0 msgr2=0x5610924186e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.875+0000 7feb33a42640 0 -- 172.21.15.136:0/2050568550 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7fe9fc07ba00 msgr2=0x7fe9fc07de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:15.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:15.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:15.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:15.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:15.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:15.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:15.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:15.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.880+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:15.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 zone default found 2024-04-22T20:08:15.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 4 Realm: () 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 zonegroup default 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.881+0000 7feb35742800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.882+0000 7feb35742800 20 started sync module instance, tier type = 2024-04-22T20:08:15.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.882+0000 7feb35742800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:15.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.890+0000 7feb35742800 20 add_watcher() i=0 2024-04-22T20:08:15.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.890+0000 7feb35742800 20 add_watcher() i=1 2024-04-22T20:08:15.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.890+0000 7feb35742800 20 add_watcher() i=2 2024-04-22T20:08:15.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.890+0000 7feb35742800 20 add_watcher() i=3 2024-04-22T20:08:15.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.890+0000 7feb35742800 20 add_watcher() i=4 2024-04-22T20:08:15.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.890+0000 7feb35742800 20 add_watcher() i=5 2024-04-22T20:08:15.890 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.890+0000 7feb35742800 20 add_watcher() i=6 2024-04-22T20:08:15.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.891+0000 7feb35742800 20 add_watcher() i=7 2024-04-22T20:08:15.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.891+0000 7feb35742800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:15.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.892+0000 7fe9e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7feb35742800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:15.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7feb35742800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:15.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7feb35742800 5 note: GC not initialized 2024-04-22T20:08:15.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7fe9e8fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:15.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7feb35742800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:15.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7feb35742800 20 Filter name: none 2024-04-22T20:08:15.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7fe9627fc640 20 reqs_thread_entry: start 2024-04-22T20:08:15.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7feb35742800 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-22T20:08:15.894 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.893+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:15.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-22T20:08:15.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-22T20:08:15.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 : miss 2024-04-22T20:08:15.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 20 rados->read ofs=0 len=0 2024-04-22T20:08:15.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:15.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:15.895 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:15.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:15.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:15.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.894+0000 7feb35742800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 2024-04-22T20:08:15.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.895+0000 7feb35742800 0 INFO: RGWReshardLock::lock found lock on abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to be held by another RGW process; skipping for now 2024-04-22T20:08:15.896 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.896+0000 7fe9e9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:15.898 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.898+0000 7feb35742800 20 remove_watcher() i=0 2024-04-22T20:08:15.898 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.898+0000 7feb35742800 2 removed watcher, disabling cache 2024-04-22T20:08:15.899 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.899+0000 7feb35742800 20 remove_watcher() i=1 2024-04-22T20:08:15.900 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.900+0000 7feb35742800 20 remove_watcher() i=2 2024-04-22T20:08:15.901 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.901+0000 7feb35742800 20 remove_watcher() i=3 2024-04-22T20:08:15.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.902+0000 7feb35742800 20 remove_watcher() i=4 2024-04-22T20:08:15.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.903+0000 7feb35742800 20 remove_watcher() i=5 2024-04-22T20:08:15.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.904+0000 7feb35742800 20 remove_watcher() i=6 2024-04-22T20:08:15.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:15.905+0000 7feb35742800 20 remove_watcher() i=7 2024-04-22T20:08:15.914 INFO:tasks.workunit.client.0.smithi136.stderr:waiting 30 seconds for reshard lock to expire... 2024-04-22T20:08:45.944 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:08:45.978 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:45.979 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:46.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.018+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.020+0000 7fd398c62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.020+0000 7fd398c62800 20 realm 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.020+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.020+0000 7fd398c62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.020+0000 7fd398c62800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.020+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.020+0000 7fd398c62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.020+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.021+0000 7fd398c62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:46.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.021+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.022+0000 7fd398c62800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:46.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.022+0000 7fd398c62800 20 searching for the correct realm 2024-04-22T20:08:46.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.024+0000 7fd396f62640 0 -- 172.21.15.136:0/1866700861 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fd390034070 msgr2=0x7fd390066120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:46.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.230+0000 7fd38df60640 0 -- 172.21.15.136:0/1866700861 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fd39006aa60 msgr2=0x7fd3900921a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.437+0000 7fd398c62800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.437+0000 7fd398c62800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.437+0000 7fd398c62800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.437+0000 7fd398c62800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.437+0000 7fd398c62800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.437+0000 7fd398c62800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.437+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.438+0000 7fd398c62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.438+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.438+0000 7fd398c62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.438+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 20 zone default found 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 4 Realm: () 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:46.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 20 zonegroup default 2024-04-22T20:08:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.439+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.440+0000 7fd398c62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.440+0000 7fd398c62800 20 started sync module instance, tier type = 2024-04-22T20:08:46.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.440+0000 7fd398c62800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:46.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.451+0000 7fd398c62800 20 add_watcher() i=0 2024-04-22T20:08:46.451 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.451+0000 7fd398c62800 20 add_watcher() i=1 2024-04-22T20:08:46.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.451+0000 7fd398c62800 20 add_watcher() i=2 2024-04-22T20:08:46.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.451+0000 7fd398c62800 20 add_watcher() i=3 2024-04-22T20:08:46.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.451+0000 7fd398c62800 20 add_watcher() i=4 2024-04-22T20:08:46.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.451+0000 7fd398c62800 20 add_watcher() i=5 2024-04-22T20:08:46.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.451+0000 7fd398c62800 20 add_watcher() i=6 2024-04-22T20:08:46.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.452+0000 7fd398c62800 20 add_watcher() i=7 2024-04-22T20:08:46.452 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.452+0000 7fd398c62800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:46.454 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.454+0000 7fd24cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:46.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.454+0000 7fd398c62800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:46.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.454+0000 7fd398c62800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:46.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.454+0000 7fd398c62800 5 note: GC not initialized 2024-04-22T20:08:46.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.455+0000 7fd1c77fe640 20 reqs_thread_entry: start 2024-04-22T20:08:46.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.455+0000 7fd398c62800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:46.455 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.455+0000 7fd398c62800 20 Filter name: none 2024-04-22T20:08:46.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.455+0000 7fd1c57fa640 20 reqs_thread_entry: start 2024-04-22T20:08:46.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.455+0000 7fd398c62800 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-04-22T20:08:46.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.455+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.456 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 : miss 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:46.457 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.456+0000 7fd398c62800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 2024-04-22T20:08:46.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.460+0000 7fd398c62800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:46.460 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.460+0000 7fd398c62800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:46.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.460+0000 7fd398c62800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:46.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.460+0000 7fd398c62800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21, ofs0, ns] 2024-04-22T20:08:46.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.461+0000 7fd279ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562449 cookie 93875856622896 notifier 6101 bl.length()=917 2024-04-22T20:08:46.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.461+0000 7fd279ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:46.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.461+0000 7fd279ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:46.461 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.461+0000 7fd279ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:46.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.463+0000 7fd398c62800 20 handle_bi_update(): bucket=:abort-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21]): orig_sources= new_sources= 2024-04-22T20:08:46.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.463+0000 7fd398c62800 20 handle_bi_update(): bucket=:abort-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21]): potential sources added=[] removed=[] 2024-04-22T20:08:46.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.463+0000 7fd398c62800 20 handle_bi_update(): bucket=:abort-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21]): orig_dests= new_dests= 2024-04-22T20:08:46.463 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.463+0000 7fd398c62800 20 handle_bi_update(): bucket=:abort-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21]): potential dests added=[] removed=[] 2024-04-22T20:08:46.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.464+0000 7fd398c62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21.1.0 2024-04-22T20:08:46.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.464+0000 7fd398c62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21.1.0 2024-04-22T20:08:46.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.464+0000 7fd398c62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21.1.1 2024-04-22T20:08:46.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.464+0000 7fd398c62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21.1.1 2024-04-22T20:08:46.464 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.464+0000 7fd398c62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21.0 2024-04-22T20:08:46.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.467+0000 7fd398c62800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 : hit (requested=0x17, cached=0x17) 2024-04-22T20:08:46.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.467+0000 7fd398c62800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 2024-04-22T20:08:46.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.468+0000 7fd398c62800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:46.470 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.468+0000 7fd398c62800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.468+0000 7fd398c62800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.468+0000 7fd398c62800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21, ofs0, ns] 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.468+0000 7fd2797fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562450 cookie 93875856622896 notifier 6101 bl.length()=885 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.468+0000 7fd2797fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 info.flags=0x17 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.468+0000 7fd2797fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 to cache LRU end 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.468+0000 7fd2797fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.470+0000 7fd398c62800 20 handle_bi_update(): bucket=:abort-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21]): orig_sources= new_sources= 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.470+0000 7fd398c62800 20 handle_bi_update(): bucket=:abort-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21]): potential sources added=[] removed=[] 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.470+0000 7fd398c62800 20 handle_bi_update(): bucket=:abort-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21]): orig_dests= new_dests= 2024-04-22T20:08:46.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.470+0000 7fd398c62800 20 handle_bi_update(): bucket=:abort-at-set-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21]): potential dests added=[] removed=[] 2024-04-22T20:08:46.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.471+0000 7fd398c62800 1 execute INFO: reshard of bucket "abort-at-set-target-layout" completed successfully 2024-04-22T20:08:46.474 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.473+0000 7fd24cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:46.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.476+0000 7fd398c62800 20 remove_watcher() i=0 2024-04-22T20:08:46.476 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.476+0000 7fd398c62800 2 removed watcher, disabling cache 2024-04-22T20:08:46.477 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.477+0000 7fd398c62800 20 remove_watcher() i=1 2024-04-22T20:08:46.478 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.478+0000 7fd398c62800 20 remove_watcher() i=2 2024-04-22T20:08:46.479 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.479+0000 7fd398c62800 20 remove_watcher() i=3 2024-04-22T20:08:46.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.480+0000 7fd398c62800 20 remove_watcher() i=4 2024-04-22T20:08:46.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.481+0000 7fd398c62800 20 remove_watcher() i=5 2024-04-22T20:08:46.482 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.482+0000 7fd398c62800 20 remove_watcher() i=6 2024-04-22T20:08:46.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.483+0000 7fd398c62800 20 remove_watcher() i=7 2024-04-22T20:08:46.496 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-04-22T20:08:46.527 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:46.527 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:46.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.562+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.564+0000 7f1711213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.564+0000 7f1711213800 20 realm 2024-04-22T20:08:46.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.564+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.564+0000 7f1711213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.564+0000 7f1711213800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:46.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.564+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.564+0000 7f1711213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.564+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.565+0000 7f1711213800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:46.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.565+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.565+0000 7f1711213800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:46.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.565+0000 7f1711213800 20 searching for the correct realm 2024-04-22T20:08:46.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.567+0000 7f170d015640 0 -- 172.21.15.136:0/1073311121 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f170806c840 msgr2=0x7f1708057580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:46.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.773+0000 7f1707fff640 0 -- 172.21.15.136:0/1073311121 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f17080599f0 msgr2=0x7f17080768e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:46.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:46.981 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.980+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.982 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 20 zone default found 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 4 Realm: () 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 20 zonegroup default 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.981+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.982+0000 7f1711213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.982+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.982+0000 7f1711213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:46.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.982+0000 7f1711213800 20 started sync module instance, tier type = 2024-04-22T20:08:46.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.982+0000 7f1711213800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:46.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.992+0000 7f1711213800 20 add_watcher() i=0 2024-04-22T20:08:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.992+0000 7f1711213800 20 add_watcher() i=1 2024-04-22T20:08:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.992+0000 7f1711213800 20 add_watcher() i=2 2024-04-22T20:08:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.992+0000 7f1711213800 20 add_watcher() i=3 2024-04-22T20:08:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.992+0000 7f1711213800 20 add_watcher() i=4 2024-04-22T20:08:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.992+0000 7f1711213800 20 add_watcher() i=5 2024-04-22T20:08:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.993+0000 7f1711213800 20 add_watcher() i=6 2024-04-22T20:08:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.993+0000 7f1711213800 20 add_watcher() i=7 2024-04-22T20:08:46.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.993+0000 7f1711213800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:46.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.996+0000 7f15c2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.996+0000 7f1711213800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.996+0000 7f1711213800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.996+0000 7f1711213800 5 note: GC not initialized 2024-04-22T20:08:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.996+0000 7f15c1fe3640 20 reqs_thread_entry: start 2024-04-22T20:08:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.996+0000 7f1711213800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.996+0000 7f1711213800 20 Filter name: none 2024-04-22T20:08:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.997+0000 7f15337fe640 20 reqs_thread_entry: start 2024-04-22T20:08:46.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.997+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.997+0000 7f1711213800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.997+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.998+0000 7f1711213800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.998+0000 7f1711213800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.998+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.998+0000 7f1711213800 20 rados_obj.operate() r=0 bl.length=194 2024-04-22T20:08:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.998+0000 7f1711213800 20 rados->read ofs=0 len=0 2024-04-22T20:08:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.998+0000 7f1711213800 20 rados_obj.operate() r=0 bl.length=408 2024-04-22T20:08:46.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:46.998+0000 7f1711213800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:47.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.000+0000 7f15c2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:47.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.002+0000 7f1711213800 20 remove_watcher() i=0 2024-04-22T20:08:47.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.002+0000 7f1711213800 2 removed watcher, disabling cache 2024-04-22T20:08:47.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.004+0000 7f1711213800 20 remove_watcher() i=1 2024-04-22T20:08:47.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.005+0000 7f1711213800 20 remove_watcher() i=2 2024-04-22T20:08:47.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.006+0000 7f1711213800 20 remove_watcher() i=3 2024-04-22T20:08:47.007 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.007+0000 7f1711213800 20 remove_watcher() i=4 2024-04-22T20:08:47.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.008+0000 7f1711213800 20 remove_watcher() i=5 2024-04-22T20:08:47.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.009+0000 7f1711213800 20 remove_watcher() i=6 2024-04-22T20:08:47.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:47.010+0000 7f1711213800 20 remove_watcher() i=7 2024-04-22T20:08:47.020 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:47.020 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:47.020 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-set-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-22T20:08:47.020 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:47.022 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.023 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.043 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.043 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.272 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.273 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.337 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with EIO injected at block_writes 2024-04-22T20:08:47.337 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:47.338 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.338 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.379 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.380 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.436 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.436 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.470 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.470 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.504 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.504 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.540 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.540 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.574 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.574 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.608 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.609 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.643 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.643 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.678 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.679 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.713 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.713 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.748 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.748 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.781 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.782 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.816 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.817 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.851 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.851 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.886 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.886 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.922 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.922 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.956 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.957 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:47.992 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:47.992 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:48.027 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:48.027 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:48.062 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:48.062 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:48.097 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:48.098 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:48.133 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-04-22T20:08:48.160 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:48.160 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:48.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.193+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.195+0000 7f1a25d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.195+0000 7f1a25d35800 20 realm 2024-04-22T20:08:48.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.195+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.195+0000 7f1a25d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.195+0000 7f1a25d35800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:48.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.195+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.195+0000 7f1a25d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.195+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.196+0000 7f1a25d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:48.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.196+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.196+0000 7f1a25d35800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:48.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.196+0000 7f1a25d35800 20 searching for the correct realm 2024-04-22T20:08:48.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.197+0000 7f1a219b5640 0 -- 172.21.15.136:0/1270912742 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x55864c704960 msgr2=0x55864c706e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:48.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.402+0000 7f1a219b5640 0 -- 172.21.15.136:0/1270912742 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x7f1a10003ab0 msgr2=0x7f1a1c032710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:48.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:48.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.611+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 20 zone default found 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 4 Realm: () 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 20 zonegroup default 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.612+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.613+0000 7f1a25d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.613+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.613+0000 7f1a25d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.613+0000 7f1a25d35800 20 started sync module instance, tier type = 2024-04-22T20:08:48.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.613+0000 7f1a25d35800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:48.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.622+0000 7f1a25d35800 20 add_watcher() i=0 2024-04-22T20:08:48.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.622+0000 7f1a25d35800 20 add_watcher() i=1 2024-04-22T20:08:48.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.622+0000 7f1a25d35800 20 add_watcher() i=2 2024-04-22T20:08:48.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.622+0000 7f1a25d35800 20 add_watcher() i=3 2024-04-22T20:08:48.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.622+0000 7f1a25d35800 20 add_watcher() i=4 2024-04-22T20:08:48.622 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.622+0000 7f1a25d35800 20 add_watcher() i=5 2024-04-22T20:08:48.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.622+0000 7f1a25d35800 20 add_watcher() i=6 2024-04-22T20:08:48.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.623+0000 7f1a25d35800 20 add_watcher() i=7 2024-04-22T20:08:48.623 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.623+0000 7f1a25d35800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:48.625 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.625+0000 7f18cb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:48.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.625+0000 7f1a25d35800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:48.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.625+0000 7f1a25d35800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:48.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.625+0000 7f1a25d35800 5 note: GC not initialized 2024-04-22T20:08:48.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.626+0000 7f18ca7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:48.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.626+0000 7f1a25d35800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:48.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.626+0000 7f1a25d35800 20 Filter name: none 2024-04-22T20:08:48.626 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.626+0000 7f1843fff640 20 reqs_thread_entry: start 2024-04-22T20:08:48.627 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.626+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.626+0000 7f1a25d35800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.626+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.627+0000 7f1a25d35800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.627+0000 7f1a25d35800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.627+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.627+0000 7f1a25d35800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.627+0000 7f1a25d35800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.628+0000 7f1a25d35800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.628+0000 7f1a25d35800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:48.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.629+0000 7f18cb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:48.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.631+0000 7f1a25d35800 20 remove_watcher() i=0 2024-04-22T20:08:48.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.631+0000 7f1a25d35800 2 removed watcher, disabling cache 2024-04-22T20:08:48.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.632+0000 7f1a25d35800 20 remove_watcher() i=1 2024-04-22T20:08:48.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.633+0000 7f1a25d35800 20 remove_watcher() i=2 2024-04-22T20:08:48.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.634+0000 7f1a25d35800 20 remove_watcher() i=3 2024-04-22T20:08:48.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.635+0000 7f1a25d35800 20 remove_watcher() i=4 2024-04-22T20:08:48.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.636+0000 7f1a25d35800 20 remove_watcher() i=5 2024-04-22T20:08:48.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.637+0000 7f1a25d35800 20 remove_watcher() i=6 2024-04-22T20:08:48.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.638+0000 7f1a25d35800 20 remove_watcher() i=7 2024-04-22T20:08:48.647 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:48.648 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:48.648 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-block-writes id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:48.648 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:48.649 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at block_writes 2024-04-22T20:08:48.682 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:48.682 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:48.715 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.715+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.717+0000 7f2f5e062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.717+0000 7f2f5e062800 20 realm 2024-04-22T20:08:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.717+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.717+0000 7f2f5e062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.717+0000 7f2f5e062800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.717+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.717+0000 7f2f5e062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:48.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.717+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.718+0000 7f2f5e062800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:48.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.718+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:48.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.718+0000 7f2f5e062800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:48.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.718+0000 7f2f5e062800 20 searching for the correct realm 2024-04-22T20:08:48.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.921+0000 7f2f5c362640 0 -- 172.21.15.136:0/2568388229 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f2f540a1ca0 msgr2=0x7f2f540a0900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:48.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:48.926+0000 7f2f5b360640 0 -- 172.21.15.136:0/2568388229 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f2f54004040 msgr2=0x7f2f540aa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.132+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.134 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 20 zone default found 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 4 Realm: () 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 20 zonegroup default 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.133+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.134+0000 7f2f5e062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.134+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.134+0000 7f2f5e062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.134+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.134+0000 7f2f5e062800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.134+0000 7f2f5e062800 20 started sync module instance, tier type = 2024-04-22T20:08:49.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.134+0000 7f2f5e062800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:49.143 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.143+0000 7f2f5e062800 20 add_watcher() i=0 2024-04-22T20:08:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.143+0000 7f2f5e062800 20 add_watcher() i=1 2024-04-22T20:08:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.143+0000 7f2f5e062800 20 add_watcher() i=2 2024-04-22T20:08:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.143+0000 7f2f5e062800 20 add_watcher() i=3 2024-04-22T20:08:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.143+0000 7f2f5e062800 20 add_watcher() i=4 2024-04-22T20:08:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.143+0000 7f2f5e062800 20 add_watcher() i=5 2024-04-22T20:08:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.143+0000 7f2f5e062800 20 add_watcher() i=6 2024-04-22T20:08:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.144+0000 7f2f5e062800 20 add_watcher() i=7 2024-04-22T20:08:49.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.144+0000 7f2f5e062800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:49.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.146+0000 7f2e11fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:49.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.146+0000 7f2f5e062800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:49.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.146+0000 7f2f5e062800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:49.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 5 note: GC not initialized 2024-04-22T20:08:49.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2e10fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:49.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:49.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 20 Filter name: none 2024-04-22T20:08:49.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2d8a7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:49.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-04-22T20:08:49.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.148 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 : miss 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.147+0000 7f2f5e062800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.148+0000 7f2f5e062800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.148+0000 7f2f5e062800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.148+0000 7f2f5e062800 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.148+0000 7f2f5e062800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.148+0000 7f2f5e062800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:49.149 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.148+0000 7f2f5e062800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 2024-04-22T20:08:49.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.151+0000 7f2f5e062800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:49.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.151+0000 7f2f5e062800 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:49.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.151+0000 7f2f5e062800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:49.152 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.151+0000 7f2f5e062800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22, ofs0, ns] 2024-04-22T20:08:49.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.152+0000 7f2e3affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562392 cookie 94419172185296 notifier 6152 bl.length()=907 2024-04-22T20:08:49.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.152+0000 7f2e3affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:49.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.152+0000 7f2e3affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:49.153 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.152+0000 7f2e3affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:49.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.154+0000 7f2f5e062800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): orig_sources= new_sources= 2024-04-22T20:08:49.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.154+0000 7f2f5e062800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): potential sources added=[] removed=[] 2024-04-22T20:08:49.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.154+0000 7f2f5e062800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): orig_dests= new_dests= 2024-04-22T20:08:49.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.154+0000 7f2f5e062800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): potential dests added=[] removed=[] 2024-04-22T20:08:49.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.154+0000 7f2f5e062800 -1 Injecting error=-5 at location=block_writes 2024-04-22T20:08:49.154 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.154+0000 7f2f5e062800 0 ERROR: init_reshard failed to pause writes to the current index: (5) Input/output error 2024-04-22T20:08:49.155 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.155+0000 7f2f5e062800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 : hit (requested=0x17, cached=0x17) 2024-04-22T20:08:49.155 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.155+0000 7f2f5e062800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 2024-04-22T20:08:49.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.156+0000 7f2f5e062800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:49.156 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.156+0000 7f2f5e062800 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:49.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.156+0000 7f2f5e062800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:49.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.156+0000 7f2f5e062800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22, ofs0, ns] 2024-04-22T20:08:49.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.156+0000 7f2e3affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562393 cookie 94419172185296 notifier 6152 bl.length()=875 2024-04-22T20:08:49.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.156+0000 7f2e3affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:49.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.156+0000 7f2e3affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:49.157 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.156+0000 7f2e3affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:49.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.158+0000 7f2f5e062800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): orig_sources= new_sources= 2024-04-22T20:08:49.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.158+0000 7f2f5e062800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): potential sources added=[] removed=[] 2024-04-22T20:08:49.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.158+0000 7f2f5e062800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): orig_dests= new_dests= 2024-04-22T20:08:49.158 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.158+0000 7f2f5e062800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): potential dests added=[] removed=[] 2024-04-22T20:08:49.159 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.159+0000 7f2e11fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:49.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.162+0000 7f2f5e062800 20 remove_watcher() i=0 2024-04-22T20:08:49.162 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.162+0000 7f2f5e062800 2 removed watcher, disabling cache 2024-04-22T20:08:49.163 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.163+0000 7f2f5e062800 20 remove_watcher() i=1 2024-04-22T20:08:49.164 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.164+0000 7f2f5e062800 20 remove_watcher() i=2 2024-04-22T20:08:49.165 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.165+0000 7f2f5e062800 20 remove_watcher() i=3 2024-04-22T20:08:49.166 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.166+0000 7f2f5e062800 20 remove_watcher() i=4 2024-04-22T20:08:49.167 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.167+0000 7f2f5e062800 20 remove_watcher() i=5 2024-04-22T20:08:49.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.168+0000 7f2f5e062800 20 remove_watcher() i=6 2024-04-22T20:08:49.169 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.169+0000 7f2f5e062800 20 remove_watcher() i=7 2024-04-22T20:08:49.180 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-04-22T20:08:49.207 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:49.207 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:49.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.241+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.242+0000 7fb0dbb31800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.242+0000 7fb0dbb31800 20 realm 2024-04-22T20:08:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.242+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.243+0000 7fb0dbb31800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.243+0000 7fb0dbb31800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.243+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.243+0000 7fb0dbb31800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.243+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.244+0000 7fb0dbb31800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:49.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.244+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.244+0000 7fb0dbb31800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:49.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.244+0000 7fb0dbb31800 20 searching for the correct realm 2024-04-22T20:08:49.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.247+0000 7fb0d8030640 0 -- 172.21.15.136:0/119290707 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fb0d007c040 msgr2=0x7fb0d0087920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:49.453 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.452+0000 7fb0cf02e640 0 -- 172.21.15.136:0/119290707 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fb0d007b410 msgr2=0x7fb0d007b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.659+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 20 zone default found 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 4 Realm: () 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 20 zonegroup default 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.660+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.661+0000 7fb0dbb31800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.661+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.661+0000 7fb0dbb31800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.661+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.661+0000 7fb0dbb31800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.661+0000 7fb0dbb31800 20 started sync module instance, tier type = 2024-04-22T20:08:49.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.661+0000 7fb0dbb31800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:49.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.671+0000 7fb0dbb31800 20 add_watcher() i=0 2024-04-22T20:08:49.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.671+0000 7fb0dbb31800 20 add_watcher() i=1 2024-04-22T20:08:49.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.671+0000 7fb0dbb31800 20 add_watcher() i=2 2024-04-22T20:08:49.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.671+0000 7fb0dbb31800 20 add_watcher() i=3 2024-04-22T20:08:49.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.671+0000 7fb0dbb31800 20 add_watcher() i=4 2024-04-22T20:08:49.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.671+0000 7fb0dbb31800 20 add_watcher() i=5 2024-04-22T20:08:49.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.671+0000 7fb0dbb31800 20 add_watcher() i=6 2024-04-22T20:08:49.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.672+0000 7fb0dbb31800 20 add_watcher() i=7 2024-04-22T20:08:49.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.672+0000 7fb0dbb31800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:49.674 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7faf8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7fb0dbb31800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7fb0dbb31800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7fb0dbb31800 5 note: GC not initialized 2024-04-22T20:08:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7faf8cfe1640 20 reqs_thread_entry: start 2024-04-22T20:08:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7fb0dbb31800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7fb0dbb31800 20 Filter name: none 2024-04-22T20:08:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7faf067fc640 20 reqs_thread_entry: start 2024-04-22T20:08:49.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.674+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.675+0000 7fb0dbb31800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:49.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.675+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.676+0000 7fb0dbb31800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:49.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.676+0000 7fb0dbb31800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:49.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.676+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.676+0000 7fb0dbb31800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:49.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.676+0000 7fb0dbb31800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.677+0000 7fb0dbb31800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:49.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.677+0000 7fb0dbb31800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:49.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.678+0000 7faf8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:49.680 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.680+0000 7fb0dbb31800 20 remove_watcher() i=0 2024-04-22T20:08:49.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.680+0000 7fb0dbb31800 2 removed watcher, disabling cache 2024-04-22T20:08:49.681 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.681+0000 7fb0dbb31800 20 remove_watcher() i=1 2024-04-22T20:08:49.682 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.682+0000 7fb0dbb31800 20 remove_watcher() i=2 2024-04-22T20:08:49.684 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.684+0000 7fb0dbb31800 20 remove_watcher() i=3 2024-04-22T20:08:49.685 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.685+0000 7fb0dbb31800 20 remove_watcher() i=4 2024-04-22T20:08:49.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.686+0000 7fb0dbb31800 20 remove_watcher() i=5 2024-04-22T20:08:49.686 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.686+0000 7fb0dbb31800 20 remove_watcher() i=6 2024-04-22T20:08:49.687 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.687+0000 7fb0dbb31800 20 remove_watcher() i=7 2024-04-22T20:08:49.697 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:49.698 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:49.698 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-block-writes id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:49.698 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:49.700 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:49.700 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:49.733 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:49.733 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:49.790 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:08:49.821 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:49.821 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:49.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.855+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.856+0000 7f790412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.856+0000 7f790412b800 20 realm 2024-04-22T20:08:49.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.856+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.856+0000 7f790412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.856+0000 7f790412b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:49.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.856+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.856+0000 7f790412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:49.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.856+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.857+0000 7f790412b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:49.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.857+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:49.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.858+0000 7f790412b800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:49.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.858+0000 7f790412b800 20 searching for the correct realm 2024-04-22T20:08:49.860 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:49.860+0000 7f7900553640 0 -- 172.21.15.136:0/4059186825 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f78f8034070 msgr2=0x7f78f8017400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:50.066 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.066+0000 7f78f7fff640 0 -- 172.21.15.136:0/4059186825 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f78f805e970 msgr2=0x7f78f8024440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:50.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.272+0000 7f790412b800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:50.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.272+0000 7f790412b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:50.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.272+0000 7f790412b800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:50.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.272+0000 7f790412b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:50.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.272+0000 7f790412b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:50.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.272+0000 7f790412b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:50.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.272+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.273 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 20 zone default found 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 4 Realm: () 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 20 zonegroup default 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.273+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.274+0000 7f790412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.274+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.275 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.274+0000 7f790412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.274+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.275+0000 7f790412b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.275+0000 7f790412b800 20 started sync module instance, tier type = 2024-04-22T20:08:50.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.275+0000 7f790412b800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:50.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.285+0000 7f790412b800 20 add_watcher() i=0 2024-04-22T20:08:50.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.285+0000 7f790412b800 20 add_watcher() i=1 2024-04-22T20:08:50.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.285+0000 7f790412b800 20 add_watcher() i=2 2024-04-22T20:08:50.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.285+0000 7f790412b800 20 add_watcher() i=3 2024-04-22T20:08:50.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.285+0000 7f790412b800 20 add_watcher() i=4 2024-04-22T20:08:50.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.285+0000 7f790412b800 20 add_watcher() i=5 2024-04-22T20:08:50.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.285+0000 7f790412b800 20 add_watcher() i=6 2024-04-22T20:08:50.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.286+0000 7f790412b800 20 add_watcher() i=7 2024-04-22T20:08:50.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.286+0000 7f790412b800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:50.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.287+0000 7f77b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:50.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f790412b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:50.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f790412b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:50.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f790412b800 5 note: GC not initialized 2024-04-22T20:08:50.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f77b4fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:50.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f790412b800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:50.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f790412b800 20 Filter name: none 2024-04-22T20:08:50.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f772e7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:50.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f790412b800 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-04-22T20:08:50.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.288+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 : miss 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:50.290 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.289+0000 7f790412b800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 2024-04-22T20:08:50.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.292+0000 7f790412b800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:50.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.292+0000 7f790412b800 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:50.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.292+0000 7f790412b800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:50.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.292+0000 7f790412b800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22, ofs0, ns] 2024-04-22T20:08:50.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.293+0000 7f77deffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562394 cookie 94445995520992 notifier 6185 bl.length()=907 2024-04-22T20:08:50.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.293+0000 7f77deffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:50.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.293+0000 7f77deffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:50.293 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.293+0000 7f77deffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:50.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.294+0000 7f790412b800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): orig_sources= new_sources= 2024-04-22T20:08:50.294 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.294+0000 7f790412b800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): potential sources added=[] removed=[] 2024-04-22T20:08:50.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.294+0000 7f790412b800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): orig_dests= new_dests= 2024-04-22T20:08:50.295 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.294+0000 7f790412b800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): potential dests added=[] removed=[] 2024-04-22T20:08:50.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.296+0000 7f790412b800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22.1.0 2024-04-22T20:08:50.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.296+0000 7f790412b800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22.1.0 2024-04-22T20:08:50.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.296+0000 7f790412b800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22.1.1 2024-04-22T20:08:50.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.296+0000 7f790412b800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22.1.1 2024-04-22T20:08:50.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.296+0000 7f790412b800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22.0 2024-04-22T20:08:50.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.298+0000 7f790412b800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 : hit (requested=0x17, cached=0x17) 2024-04-22T20:08:50.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.298+0000 7f790412b800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 2024-04-22T20:08:50.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.300+0000 7f790412b800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:50.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.300+0000 7f790412b800 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:50.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.300+0000 7f790412b800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:50.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.300+0000 7f790412b800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22, ofs0, ns] 2024-04-22T20:08:50.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.300+0000 7f77de7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562395 cookie 94445995520992 notifier 6185 bl.length()=875 2024-04-22T20:08:50.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.300+0000 7f77de7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 info.flags=0x17 2024-04-22T20:08:50.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.300+0000 7f77de7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 to cache LRU end 2024-04-22T20:08:50.300 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.300+0000 7f77de7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:50.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.301+0000 7f790412b800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): orig_sources= new_sources= 2024-04-22T20:08:50.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.301+0000 7f790412b800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): potential sources added=[] removed=[] 2024-04-22T20:08:50.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.301+0000 7f790412b800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): orig_dests= new_dests= 2024-04-22T20:08:50.301 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.301+0000 7f790412b800 20 handle_bi_update(): bucket=:error-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22]): potential dests added=[] removed=[] 2024-04-22T20:08:50.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.303+0000 7f790412b800 1 execute INFO: reshard of bucket "error-at-block-writes" completed successfully 2024-04-22T20:08:50.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.304+0000 7f77b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:50.306 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.306+0000 7f790412b800 20 remove_watcher() i=0 2024-04-22T20:08:50.307 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.306+0000 7f790412b800 2 removed watcher, disabling cache 2024-04-22T20:08:50.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.308+0000 7f790412b800 20 remove_watcher() i=1 2024-04-22T20:08:50.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.309+0000 7f790412b800 20 remove_watcher() i=2 2024-04-22T20:08:50.310 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.310+0000 7f790412b800 20 remove_watcher() i=3 2024-04-22T20:08:50.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.311+0000 7f790412b800 20 remove_watcher() i=4 2024-04-22T20:08:50.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.312+0000 7f790412b800 20 remove_watcher() i=5 2024-04-22T20:08:50.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.313+0000 7f790412b800 20 remove_watcher() i=6 2024-04-22T20:08:50.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.314+0000 7f790412b800 20 remove_watcher() i=7 2024-04-22T20:08:50.325 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-04-22T20:08:50.352 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:50.352 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:50.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.386+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.387+0000 7f113212e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.387+0000 7f113212e800 20 realm 2024-04-22T20:08:50.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.387+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.387+0000 7f113212e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.387+0000 7f113212e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:50.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.387+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.387+0000 7f113212e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.387+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.388+0000 7f113212e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:50.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.388+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.389+0000 7f113212e800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:50.389 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.389+0000 7f113212e800 20 searching for the correct realm 2024-04-22T20:08:50.391 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.391+0000 7f112e68c640 0 -- 172.21.15.136:0/387491266 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f112805c040 msgr2=0x7f112809a710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:50.597 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.597+0000 7f112d68a640 0 -- 172.21.15.136:0/387491266 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f112809dbd0 msgr2=0x7f112809e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.803+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:50.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 20 zone default found 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 4 Realm: () 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 20 zonegroup default 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.804+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.805+0000 7f113212e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.805+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.805+0000 7f113212e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:50.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.805+0000 7f113212e800 20 started sync module instance, tier type = 2024-04-22T20:08:50.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.805+0000 7f113212e800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:50.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.815+0000 7f113212e800 20 add_watcher() i=0 2024-04-22T20:08:50.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.815+0000 7f113212e800 20 add_watcher() i=1 2024-04-22T20:08:50.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.815+0000 7f113212e800 20 add_watcher() i=2 2024-04-22T20:08:50.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.815+0000 7f113212e800 20 add_watcher() i=3 2024-04-22T20:08:50.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.815+0000 7f113212e800 20 add_watcher() i=4 2024-04-22T20:08:50.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.815+0000 7f113212e800 20 add_watcher() i=5 2024-04-22T20:08:50.815 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.815+0000 7f113212e800 20 add_watcher() i=6 2024-04-22T20:08:50.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.816+0000 7f113212e800 20 add_watcher() i=7 2024-04-22T20:08:50.816 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.816+0000 7f113212e800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:50.818 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f0fd77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:50.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f113212e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:50.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f113212e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:50.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f113212e800 5 note: GC not initialized 2024-04-22T20:08:50.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f0fd67fc640 20 reqs_thread_entry: start 2024-04-22T20:08:50.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f113212e800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:50.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f113212e800 20 Filter name: none 2024-04-22T20:08:50.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f0f4ffff640 20 reqs_thread_entry: start 2024-04-22T20:08:50.819 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.818+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.819+0000 7f113212e800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:50.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.819+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.820+0000 7f113212e800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:50.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.820+0000 7f113212e800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:50.820 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.820+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.820+0000 7f113212e800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:50.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.820+0000 7f113212e800 20 rados->read ofs=0 len=0 2024-04-22T20:08:50.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.820+0000 7f113212e800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:50.821 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.820+0000 7f113212e800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:50.822 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.822+0000 7f0fd77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:50.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.824+0000 7f113212e800 20 remove_watcher() i=0 2024-04-22T20:08:50.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.824+0000 7f113212e800 2 removed watcher, disabling cache 2024-04-22T20:08:50.825 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.825+0000 7f113212e800 20 remove_watcher() i=1 2024-04-22T20:08:50.826 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.826+0000 7f113212e800 20 remove_watcher() i=2 2024-04-22T20:08:50.827 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.827+0000 7f113212e800 20 remove_watcher() i=3 2024-04-22T20:08:50.828 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.828+0000 7f113212e800 20 remove_watcher() i=4 2024-04-22T20:08:50.829 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.829+0000 7f113212e800 20 remove_watcher() i=5 2024-04-22T20:08:50.830 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.830+0000 7f113212e800 20 remove_watcher() i=6 2024-04-22T20:08:50.831 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:50.831+0000 7f113212e800 20 remove_watcher() i=7 2024-04-22T20:08:50.840 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:50.840 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:50.841 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-block-writes id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-22T20:08:50.841 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:50.842 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:50.842 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:50.863 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:50.863 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.076 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.076 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.142 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with abort at block_writes 2024-04-22T20:08:51.142 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:51.144 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.144 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.184 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.185 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.240 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.241 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.274 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.274 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.311 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.311 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.345 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.345 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.381 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.381 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.414 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.414 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.449 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.449 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.484 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.484 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.519 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.519 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.554 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.554 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.589 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.589 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.623 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.623 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.657 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.657 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.692 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.692 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.726 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.726 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.761 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.761 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.797 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.797 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.833 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.833 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.866 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.866 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.900 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:51.900 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:51.936 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-04-22T20:08:51.963 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:51.963 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:52.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.195+0000 7f196829a640 0 --2- 172.21.15.136:0/2305988802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5560cd343150 0x5560cd35e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-04-22T20:08:52.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.197+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.198+0000 7f1974d3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.198+0000 7f1974d3f800 20 realm 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.198+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.198+0000 7f1974d3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.198+0000 7f1974d3f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.198+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.199+0000 7f1974d3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.199+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.199+0000 7f1974d3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:52.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.199+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.200+0000 7f1974d3f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:52.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.200+0000 7f1974d3f800 20 searching for the correct realm 2024-04-22T20:08:52.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.201+0000 7f1970a9b640 0 -- 172.21.15.136:0/2305988802 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5560cd370e10 msgr2=0x5560cd3732d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:52.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.204+0000 7f1970a9b640 0 -- 172.21.15.136:0/2305988802 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x7f196c04bbe0 msgr2=0x7f196c00c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:52.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:52.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.412+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:52.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 20 zone default found 2024-04-22T20:08:52.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 4 Realm: () 2024-04-22T20:08:52.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:52.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 20 zonegroup default 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.413+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.414+0000 7f1974d3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.414+0000 7f1974d3f800 20 started sync module instance, tier type = 2024-04-22T20:08:52.415 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.414+0000 7f1974d3f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.422+0000 7f1974d3f800 20 add_watcher() i=0 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.422+0000 7f1974d3f800 20 add_watcher() i=1 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.422+0000 7f1974d3f800 20 add_watcher() i=2 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.422+0000 7f1974d3f800 20 add_watcher() i=3 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.422+0000 7f1974d3f800 20 add_watcher() i=4 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.422+0000 7f1974d3f800 20 add_watcher() i=5 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.423+0000 7f1974d3f800 20 add_watcher() i=6 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.423+0000 7f1974d3f800 20 add_watcher() i=7 2024-04-22T20:08:52.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.423+0000 7f1974d3f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.425+0000 7f1826fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.425+0000 7f1974d3f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.425+0000 7f1974d3f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.425+0000 7f1974d3f800 5 note: GC not initialized 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.426+0000 7f1825fe3640 20 reqs_thread_entry: start 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.426+0000 7f1974d3f800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.426+0000 7f1974d3f800 20 Filter name: none 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.426+0000 7f17977fe640 20 reqs_thread_entry: start 2024-04-22T20:08:52.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.426+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.426+0000 7f1974d3f800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:52.427 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.426+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.427+0000 7f1974d3f800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:52.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.427+0000 7f1974d3f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:52.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.427+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.427+0000 7f1974d3f800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:52.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.427+0000 7f1974d3f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.428+0000 7f1974d3f800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:52.428 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.428+0000 7f1974d3f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:52.429 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.429+0000 7f1826fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:52.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.431+0000 7f1974d3f800 20 remove_watcher() i=0 2024-04-22T20:08:52.431 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.431+0000 7f1974d3f800 2 removed watcher, disabling cache 2024-04-22T20:08:52.432 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.432+0000 7f1974d3f800 20 remove_watcher() i=1 2024-04-22T20:08:52.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.433+0000 7f1974d3f800 20 remove_watcher() i=2 2024-04-22T20:08:52.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.434+0000 7f1974d3f800 20 remove_watcher() i=3 2024-04-22T20:08:52.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.435+0000 7f1974d3f800 20 remove_watcher() i=4 2024-04-22T20:08:52.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.436+0000 7f1974d3f800 20 remove_watcher() i=5 2024-04-22T20:08:52.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.437+0000 7f1974d3f800 20 remove_watcher() i=6 2024-04-22T20:08:52.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.438+0000 7f1974d3f800 20 remove_watcher() i=7 2024-04-22T20:08:52.448 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.448 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:52.448 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-block-writes id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:52.448 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.449 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at block_writes 2024-04-22T20:08:52.483 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:52.483 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:52.517 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.516+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.518+0000 7fd182262800 20 realm 2024-04-22T20:08:52.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.518+0000 7fd182262800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:52.518 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.519+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:52.519 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.519+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.519+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:52.520 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.519+0000 7fd182262800 20 searching for the correct realm 2024-04-22T20:08:52.521 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.521+0000 7fd180562640 0 -- 172.21.15.136:0/1735850960 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fd178028070 msgr2=0x7fd178037960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:52.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.726+0000 7fd17f560640 0 -- 172.21.15.136:0/1735850960 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fd17800a940 msgr2=0x7fd178051410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.932+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 zone default found 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 4 Realm: () 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 zonegroup default 2024-04-22T20:08:52.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.934+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:52.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.934+0000 7fd182262800 20 started sync module instance, tier type = 2024-04-22T20:08:52.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.934+0000 7fd182262800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:52.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=0 2024-04-22T20:08:52.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=1 2024-04-22T20:08:52.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=2 2024-04-22T20:08:52.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=3 2024-04-22T20:08:52.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=4 2024-04-22T20:08:52.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=5 2024-04-22T20:08:52.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=6 2024-04-22T20:08:52.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.943+0000 7fd182262800 20 add_watcher() i=7 2024-04-22T20:08:52.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.943+0000 7fd182262800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:52.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd035fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:52.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd182262800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:52.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd182262800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd182262800 5 note: GC not initialized 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd034fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd182262800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd182262800 20 Filter name: none 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fcfae7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd182262800 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.945+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 : miss 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:52.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:52.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:52.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:52.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.946+0000 7fd182262800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 2024-04-22T20:08:52.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.950+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:52.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.950+0000 7fd182262800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:52.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.950+0000 7fd182262800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:52.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.950+0000 7fd182262800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23, ofs0, ns] 2024-04-22T20:08:52.950 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562454 cookie 94609171139344 notifier 6236 bl.length()=907 2024-04-22T20:08:52.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:52.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:52.951 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:52.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_sources= new_sources= 2024-04-22T20:08:52.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential sources added=[] removed=[] 2024-04-22T20:08:52.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_dests= new_dests= 2024-04-22T20:08:52.952 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential dests added=[] removed=[] 2024-04-22T20:08:52.952 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fd182262800 time 2024-04-22T20:08:52.953562+0000 2024-04-22T20:08:52.952 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:08:52.953 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:52.953 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd185581f5c] 2024-04-22T20:08:52.953 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd18558211a] 2024-04-22T20:08:52.953 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560be8f25dc1] 2024-04-22T20:08:52.953 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2b4) [0x560be8f33e14] 2024-04-22T20:08:52.953 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:08:52.954 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fd184429590] 2024-04-22T20:08:52.954 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:08:52.954 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:08:52.954 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.952+0000 7fd182262800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fd182262800 time 2024-04-22T20:08:52.953562+0000 2024-04-22T20:08:52.954 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:08:52.954 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.954 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:52.954 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd185581f5c] 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd18558211a] 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560be8f25dc1] 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2b4) [0x560be8f33e14] 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fd184429590] 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.955 INFO:tasks.workunit.client.0.smithi136.stderr:*** Caught signal (Aborted) ** 2024-04-22T20:08:52.956 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fd182262800 thread_name:radosgw-admin 2024-04-22T20:08:52.971 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:52.971 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd18443e6f0] 2024-04-22T20:08:52.971 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd18448b94c] 2024-04-22T20:08:52.971 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd185581fb6] 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd18558211a] 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560be8f25dc1] 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2b4) [0x560be8f33e14] 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fd184429590] 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:52.953+0000 7fd182262800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fd182262800 thread_name:radosgw-admin 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd18443e6f0] 2024-04-22T20:08:52.972 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd18448b94c] 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd185581fb6] 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd18558211a] 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560be8f25dc1] 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2b4) [0x560be8f33e14] 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fd184429590] 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr:--- begin dump of recent events --- 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr: -499> 2024-04-22T20:08:52.500+0000 7fd06de0b640 10 monclient: dump: 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:52.973 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr: -498> 2024-04-22T20:08:52.500+0000 7fd182262800 5 monclient: authenticate success, global_id 6555 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr: -497> 2024-04-22T20:08:52.500+0000 7fd182262800 10 monclient: get_monmap_and_config success 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr: -496> 2024-04-22T20:08:52.500+0000 7fd182262800 4 set_mon_vals no callback set 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr: -495> 2024-04-22T20:08:52.501+0000 7fd182262800 10 monclient: shutdown 2024-04-22T20:08:52.974 INFO:tasks.workunit.client.0.smithi136.stderr: -494> 2024-04-22T20:08:52.501+0000 7fd182262800 5 asok(0x560be9d77d00) unregister_commands rotate-key 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr: -493> 2024-04-22T20:08:52.501+0000 7fd182262800 10 monclient: build_initial_monmap 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr: -492> 2024-04-22T20:08:52.501+0000 7fd182262800 1 build_initial for_mkfs: 0 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr: -491> 2024-04-22T20:08:52.501+0000 7fd182262800 10 monclient: monmap: 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.975 INFO:tasks.workunit.client.0.smithi136.stderr: -490> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding auth protocol: cephx 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -489> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding auth protocol: cephx 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -488> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding auth protocol: cephx 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -487> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -486> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -485> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -484> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -483> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -482> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -481> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -480> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -479> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -478> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -477> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -476> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:52.976 INFO:tasks.workunit.client.0.smithi136.stderr: -475> 2024-04-22T20:08:52.501+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -474> 2024-04-22T20:08:52.501+0000 7fd182262800 1 librados: starting msgr at 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -473> 2024-04-22T20:08:52.501+0000 7fd182262800 1 librados: starting objecter 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -472> 2024-04-22T20:08:52.501+0000 7fd182262800 5 asok(0x560be9d77d00) register_command objecter_requests hook 0x560be9fa0520 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -471> 2024-04-22T20:08:52.501+0000 7fd182262800 1 librados: setting wanted keys 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -470> 2024-04-22T20:08:52.501+0000 7fd182262800 1 librados: calling monclient init 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -469> 2024-04-22T20:08:52.501+0000 7fd182262800 10 monclient: init 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -468> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding auth protocol: cephx 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -467> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding auth protocol: cephx 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -466> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding auth protocol: cephx 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -465> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -464> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -463> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -462> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:52.977 INFO:tasks.workunit.client.0.smithi136.stderr: -461> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -460> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -459> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -458> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -457> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -456> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -455> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -454> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -453> 2024-04-22T20:08:52.502+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -452> 2024-04-22T20:08:52.502+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -451> 2024-04-22T20:08:52.502+0000 7fd182262800 5 asok(0x560be9d77d00) register_command rotate-key hook 0x560be9f98b68 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -450> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -449> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient: _add_conns ranks=[0,1,2] 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -448> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): picked mon.noname-a con 0x560be9e43f40 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:52.978 INFO:tasks.workunit.client.0.smithi136.stderr: -447> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): picked mon.noname-b con 0x560be9e4ed30 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -446> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): picked mon.noname-c con 0x560be9ee0960 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -445> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -444> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -443> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -442> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -441> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): authenticate will time out at 1921.854257s 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -440> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): get_auth_request con 0x560be9e43f40 auth_method 0 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -439> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -438> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -437> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -436> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): get_auth_request con 0x560be9ee0960 auth_method 0 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -435> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -434> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -433> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.979 INFO:tasks.workunit.client.0.smithi136.stderr: -432> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -431> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -430> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -429> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -428> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -427> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -426> 2024-04-22T20:08:52.502+0000 7fd17f560640 10 monclient(hunting): get_auth_request con 0x560be9e4ed30 auth_method 0 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -425> 2024-04-22T20:08:52.502+0000 7fd17f560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -424> 2024-04-22T20:08:52.502+0000 7fd17f560640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -423> 2024-04-22T20:08:52.502+0000 7fd17f560640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -422> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): handle_auth_done global_id 6558 payload 824 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -421> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient: _finish_hunting 0 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -420> 2024-04-22T20:08:52.502+0000 7fd17fd61640 1 monclient: found mon.noname-a 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -419> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:08:52.980 INFO:tasks.workunit.client.0.smithi136.stderr: -418> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr: -417> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr: -416> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: dump: 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.981 INFO:tasks.workunit.client.0.smithi136.stderr: -415> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: _finish_auth 0 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr: -414> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: _check_auth_tickets 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr: -413> 2024-04-22T20:08:52.502+0000 7fd182262800 5 monclient: authenticate success, global_id 6558 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr: -412> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr: -411> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr: -410> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr: -409> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr: -408> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr: -407> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: dump: 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:52.982 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -406> 2024-04-22T20:08:52.502+0000 7fd06de0b640 4 set_mon_vals no callback set 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -405> 2024-04-22T20:08:52.503+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -404> 2024-04-22T20:08:52.503+0000 7fd182262800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -403> 2024-04-22T20:08:52.503+0000 7fd182262800 1 librados: init done 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -402> 2024-04-22T20:08:52.504+0000 7fd067fff640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -401> 2024-04-22T20:08:52.504+0000 7fd067fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -400> 2024-04-22T20:08:52.504+0000 7fd067fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -399> 2024-04-22T20:08:52.504+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd06806b470 auth_method 0 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -398> 2024-04-22T20:08:52.505+0000 7fd180562640 10 monclient: get_auth_request con 0x560be9fc3840 auth_method 0 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -397> 2024-04-22T20:08:52.505+0000 7fd17fd61640 10 monclient: get_auth_request con 0x560be9fe65e0 auth_method 0 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -396> 2024-04-22T20:08:52.506+0000 7fd17f560640 10 monclient: get_auth_request con 0x560be9fea940 auth_method 0 2024-04-22T20:08:52.983 INFO:tasks.workunit.client.0.smithi136.stderr: -395> 2024-04-22T20:08:52.507+0000 7fd180562640 10 monclient: get_auth_request con 0x560be9fe9e90 auth_method 0 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr: -394> 2024-04-22T20:08:52.508+0000 7fd182262800 10 monclient: build_initial_monmap 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr: -393> 2024-04-22T20:08:52.508+0000 7fd182262800 1 build_initial for_mkfs: 0 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr: -392> 2024-04-22T20:08:52.508+0000 7fd182262800 10 monclient: monmap: 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr: -391> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding auth protocol: cephx 2024-04-22T20:08:52.984 INFO:tasks.workunit.client.0.smithi136.stderr: -390> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding auth protocol: cephx 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -389> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding auth protocol: cephx 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -388> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -387> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -386> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -385> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -384> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -383> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -382> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -381> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -380> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -379> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -378> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -377> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -376> 2024-04-22T20:08:52.508+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.985 INFO:tasks.workunit.client.0.smithi136.stderr: -375> 2024-04-22T20:08:52.508+0000 7fd182262800 5 asok(0x560be9d77d00) register_command objecter_requests cmddesc objecter_requests hook 0x560be9ff8010 EEXIST 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -374> 2024-04-22T20:08:52.508+0000 7fd182262800 10 monclient: init 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -373> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding auth protocol: cephx 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -372> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding auth protocol: cephx 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -371> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding auth protocol: cephx 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -370> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -369> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -368> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -367> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -366> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -365> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -364> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -363> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -362> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -361> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:52.986 INFO:tasks.workunit.client.0.smithi136.stderr: -360> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -359> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -358> 2024-04-22T20:08:52.508+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -357> 2024-04-22T20:08:52.509+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -356> 2024-04-22T20:08:52.509+0000 7fd182262800 5 asok(0x560be9d77d00) register_command rotate-key cmddesc rotate-key hook 0x560be9ff1aa0 EEXIST 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -355> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -354> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _add_conns ranks=[2,1,0] 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -353> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): picked mon.noname-c con 0x560bea01cd70 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -352> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): picked mon.noname-b con 0x560bea01d150 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -351> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): picked mon.noname-a con 0x560bea018210 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -350> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -349> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -348> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -347> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -346> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): authenticate will time out at 1921.861010s 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -345> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): get_auth_request con 0x560bea01cd70 auth_method 0 2024-04-22T20:08:52.987 INFO:tasks.workunit.client.0.smithi136.stderr: -344> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -343> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -342> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -341> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): get_auth_request con 0x560bea018210 auth_method 0 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -340> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -339> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -338> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -337> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): get_auth_request con 0x560bea01d150 auth_method 0 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -336> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -335> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -334> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -333> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -332> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -331> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:52.988 INFO:tasks.workunit.client.0.smithi136.stderr: -330> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -329> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -328> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -327> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -326> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -325> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -324> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): handle_auth_done global_id 6230 payload 824 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -323> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient: _finish_hunting 0 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -322> 2024-04-22T20:08:52.509+0000 7fd17f560640 1 monclient: found mon.noname-c 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -321> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -320> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -319> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr: -318> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: dump: 2024-04-22T20:08:52.989 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr: -317> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: _finish_auth 0 2024-04-22T20:08:52.990 INFO:tasks.workunit.client.0.smithi136.stderr: -316> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: _check_auth_tickets 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr: -315> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr: -314> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr: -313> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr: -312> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: dump: 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:52.991 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -311> 2024-04-22T20:08:52.509+0000 7fd182262800 5 monclient: authenticate success, global_id 6230 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -310> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -309> 2024-04-22T20:08:52.509+0000 7fd180d63640 4 set_mon_vals no callback set 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -308> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -307> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -306> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -305> 2024-04-22T20:08:52.511+0000 7fd0657fa640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -304> 2024-04-22T20:08:52.511+0000 7fd0657fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -303> 2024-04-22T20:08:52.511+0000 7fd0657fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -302> 2024-04-22T20:08:52.511+0000 7fd180562640 10 monclient: get_auth_request con 0x7fd05806b1e0 auth_method 0 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -301> 2024-04-22T20:08:52.511+0000 7fd182262800 10 monclient: get_monmap_and_config 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -300> 2024-04-22T20:08:52.511+0000 7fd182262800 10 monclient: build_initial_monmap 2024-04-22T20:08:52.992 INFO:tasks.workunit.client.0.smithi136.stderr: -299> 2024-04-22T20:08:52.511+0000 7fd182262800 1 build_initial for_mkfs: 0 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr: -298> 2024-04-22T20:08:52.511+0000 7fd182262800 10 monclient: monmap: 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr: -297> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr: -296> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr: -295> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr: -294> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:52.993 INFO:tasks.workunit.client.0.smithi136.stderr: -293> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -292> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -291> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -290> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -289> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -288> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -287> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -286> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -285> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -284> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -283> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -282> 2024-04-22T20:08:52.511+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -281> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient: init 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -280> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding auth protocol: cephx 2024-04-22T20:08:52.994 INFO:tasks.workunit.client.0.smithi136.stderr: -279> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding auth protocol: cephx 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -278> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding auth protocol: cephx 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -277> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -276> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -275> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -274> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -273> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -272> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -271> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -270> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -269> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -268> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -267> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -266> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -265> 2024-04-22T20:08:52.512+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -264> 2024-04-22T20:08:52.512+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:52.995 INFO:tasks.workunit.client.0.smithi136.stderr: -263> 2024-04-22T20:08:52.512+0000 7fd182262800 5 asok(0x560be9d77d00) register_command rotate-key cmddesc rotate-key hook 0x7fff0ce23a08 EEXIST 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -262> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -261> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient: _add_conns ranks=[0,1,2] 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -260> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): picked mon.noname-a con 0x560bea06f2c0 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -259> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): picked mon.noname-b con 0x560bea073f20 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -258> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): picked mon.noname-c con 0x560bea074c30 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -257> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -256> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -255> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -254> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -253> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): authenticate will time out at 1921.864473s 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -252> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): get_auth_request con 0x560bea074c30 auth_method 0 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -251> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -250> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -249> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -248> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): get_auth_request con 0x560bea06f2c0 auth_method 0 2024-04-22T20:08:52.996 INFO:tasks.workunit.client.0.smithi136.stderr: -247> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -246> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -245> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -244> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -243> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -242> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -241> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -240> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -239> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -238> 2024-04-22T20:08:52.512+0000 7fd180562640 10 monclient(hunting): get_auth_request con 0x560bea073f20 auth_method 0 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -237> 2024-04-22T20:08:52.512+0000 7fd180562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -236> 2024-04-22T20:08:52.512+0000 7fd180562640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -235> 2024-04-22T20:08:52.512+0000 7fd180562640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -234> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): handle_auth_done global_id 6233 payload 274 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -233> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient: _finish_hunting 0 2024-04-22T20:08:52.997 INFO:tasks.workunit.client.0.smithi136.stderr: -232> 2024-04-22T20:08:52.512+0000 7fd17fd61640 1 monclient: found mon.noname-c 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr: -231> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr: -230> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr: -229> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr: -228> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: dump: 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:52.998 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr: -227> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: _finish_auth 0 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr: -226> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: _check_auth_tickets 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr: -225> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr: -224> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr: -223> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr: -222> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: dump: 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:52.999 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr: -221> 2024-04-22T20:08:52.513+0000 7fd182262800 5 monclient: authenticate success, global_id 6233 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr: -220> 2024-04-22T20:08:52.513+0000 7fd182262800 10 monclient: get_monmap_and_config success 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr: -219> 2024-04-22T20:08:52.513+0000 7fd182262800 4 set_mon_vals no callback set 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr: -218> 2024-04-22T20:08:52.513+0000 7fd182262800 10 monclient: shutdown 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr: -217> 2024-04-22T20:08:52.513+0000 7fd182262800 10 monclient: build_initial_monmap 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr: -216> 2024-04-22T20:08:52.513+0000 7fd182262800 1 build_initial for_mkfs: 0 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr: -215> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient: monmap: 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:53.000 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -214> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -213> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -212> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -211> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -210> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -209> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -208> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -207> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -206> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -205> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -204> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -203> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.001 INFO:tasks.workunit.client.0.smithi136.stderr: -202> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -201> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -200> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -199> 2024-04-22T20:08:52.514+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -198> 2024-04-22T20:08:52.514+0000 7fd182262800 1 librados: starting msgr at 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -197> 2024-04-22T20:08:52.514+0000 7fd182262800 1 librados: starting objecter 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -196> 2024-04-22T20:08:52.514+0000 7fd182262800 5 asok(0x560be9d77d00) register_command objecter_requests cmddesc objecter_requests hook 0x560bea06c550 EEXIST 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -195> 2024-04-22T20:08:52.514+0000 7fd182262800 1 librados: setting wanted keys 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -194> 2024-04-22T20:08:52.514+0000 7fd182262800 1 librados: calling monclient init 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -193> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient: init 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -192> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding auth protocol: cephx 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -191> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding auth protocol: cephx 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -190> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding auth protocol: cephx 2024-04-22T20:08:53.002 INFO:tasks.workunit.client.0.smithi136.stderr: -189> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -188> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -187> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -186> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -185> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -184> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -183> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -182> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -181> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -180> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -179> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -178> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -177> 2024-04-22T20:08:52.514+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -176> 2024-04-22T20:08:52.514+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -175> 2024-04-22T20:08:52.514+0000 7fd182262800 5 asok(0x560be9d77d00) register_command rotate-key cmddesc rotate-key hook 0x560bea039bd8 EEXIST 2024-04-22T20:08:53.003 INFO:tasks.workunit.client.0.smithi136.stderr: -174> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -173> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient: _add_conns ranks=[1,0,2] 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -172> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): picked mon.noname-b con 0x560bea06f2c0 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -171> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): picked mon.noname-a con 0x560bea073f20 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -170> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): picked mon.noname-c con 0x560bea074c30 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -169> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -168> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -167> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -166> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -165> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): authenticate will time out at 1921.866801s 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -164> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): get_auth_request con 0x560bea074c30 auth_method 0 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -163> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -162> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -161> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -160> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): get_auth_request con 0x560bea073f20 auth_method 0 2024-04-22T20:08:53.004 INFO:tasks.workunit.client.0.smithi136.stderr: -159> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -158> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -157> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -156> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -155> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -154> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -153> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): get_auth_request con 0x560bea06f2c0 auth_method 0 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -152> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -151> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -150> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -149> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -148> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -147> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -146> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -145> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -144> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.005 INFO:tasks.workunit.client.0.smithi136.stderr: -143> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): handle_auth_done global_id 6236 payload 824 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr: -142> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient: _finish_hunting 0 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr: -141> 2024-04-22T20:08:52.515+0000 7fd17fd61640 1 monclient: found mon.noname-c 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr: -140> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr: -139> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr: -138> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr: -137> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: dump: 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.006 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -136> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: _finish_auth 0 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -135> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: _check_auth_tickets 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -134> 2024-04-22T20:08:52.515+0000 7fd182262800 5 monclient: authenticate success, global_id 6236 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -133> 2024-04-22T20:08:52.515+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -132> 2024-04-22T20:08:52.515+0000 7fd182262800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -131> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -130> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -129> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr: -128> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: dump: 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.007 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -127> 2024-04-22T20:08:52.515+0000 7fd05effd640 4 set_mon_vals no callback set 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -126> 2024-04-22T20:08:52.515+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -125> 2024-04-22T20:08:52.515+0000 7fd182262800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -124> 2024-04-22T20:08:52.515+0000 7fd182262800 1 librados: init done 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -123> 2024-04-22T20:08:52.515+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cr dump hook 0x7fd170055f38 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -122> 2024-04-22T20:08:52.516+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -121> 2024-04-22T20:08:52.517+0000 7fd05d7fa640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -120> 2024-04-22T20:08:52.517+0000 7fd05d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -119> 2024-04-22T20:08:52.517+0000 7fd05d7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -118> 2024-04-22T20:08:52.517+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd04806b2d0 auth_method 0 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -117> 2024-04-22T20:08:52.517+0000 7fd180562640 10 monclient: get_auth_request con 0x560bea088e80 auth_method 0 2024-04-22T20:08:53.008 INFO:tasks.workunit.client.0.smithi136.stderr: -116> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -115> 2024-04-22T20:08:52.518+0000 7fd182262800 20 realm 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -114> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -113> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -112> 2024-04-22T20:08:52.518+0000 7fd182262800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -111> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -110> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -109> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -108> 2024-04-22T20:08:52.518+0000 7fd17f560640 10 monclient: get_auth_request con 0x560bea08d0b0 auth_method 0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -107> 2024-04-22T20:08:52.519+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -106> 2024-04-22T20:08:52.519+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -105> 2024-04-22T20:08:52.519+0000 7fd180562640 10 monclient: get_auth_request con 0x560bea090910 auth_method 0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -104> 2024-04-22T20:08:52.519+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -103> 2024-04-22T20:08:52.519+0000 7fd182262800 20 searching for the correct realm 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -102> 2024-04-22T20:08:52.519+0000 7fd17fd61640 10 monclient: get_auth_request con 0x560bea095620 auth_method 0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -101> 2024-04-22T20:08:52.520+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd170082070 auth_method 0 2024-04-22T20:08:53.009 INFO:tasks.workunit.client.0.smithi136.stderr: -100> 2024-04-22T20:08:52.521+0000 7fd180562640 10 monclient: get_auth_request con 0x7fd178028070 auth_method 0 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -99> 2024-04-22T20:08:52.521+0000 7fd180562640 0 -- 172.21.15.136:0/1735850960 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fd178028070 msgr2=0x7fd178037960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -98> 2024-04-22T20:08:52.722+0000 7fd180562640 10 monclient: get_auth_request con 0x7fd178028070 auth_method 0 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -97> 2024-04-22T20:08:52.724+0000 7fd17fd61640 10 monclient: get_auth_request con 0x7fd178040b20 auth_method 0 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -96> 2024-04-22T20:08:52.726+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd17800a940 auth_method 0 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -95> 2024-04-22T20:08:52.726+0000 7fd17f560640 0 -- 172.21.15.136:0/1735850960 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fd17800a940 msgr2=0x7fd178051410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -94> 2024-04-22T20:08:52.927+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd17800a940 auth_method 0 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -93> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -92> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -91> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -90> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -89> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -88> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -87> 2024-04-22T20:08:52.932+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -86> 2024-04-22T20:08:52.932+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.010 INFO:tasks.workunit.client.0.smithi136.stderr: -85> 2024-04-22T20:08:52.932+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -84> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -83> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -82> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -81> 2024-04-22T20:08:52.933+0000 7fd182262800 20 zone default found 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -80> 2024-04-22T20:08:52.933+0000 7fd182262800 4 Realm: () 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -79> 2024-04-22T20:08:52.933+0000 7fd182262800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -78> 2024-04-22T20:08:52.933+0000 7fd182262800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -77> 2024-04-22T20:08:52.933+0000 7fd182262800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -76> 2024-04-22T20:08:52.933+0000 7fd182262800 20 zonegroup default 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -75> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -74> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -73> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -72> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -71> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.011 INFO:tasks.workunit.client.0.smithi136.stderr: -70> 2024-04-22T20:08:52.934+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -69> 2024-04-22T20:08:52.934+0000 7fd182262800 20 started sync module instance, tier type = 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -68> 2024-04-22T20:08:52.934+0000 7fd182262800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -67> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=0 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -66> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=1 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -65> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=2 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -64> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=3 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -63> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=4 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -62> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=5 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -61> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=6 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -60> 2024-04-22T20:08:52.943+0000 7fd182262800 20 add_watcher() i=7 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -59> 2024-04-22T20:08:52.943+0000 7fd182262800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -58> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding auth protocol: cephx 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -57> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding auth protocol: cephx 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -56> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding auth protocol: cephx 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -55> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.012 INFO:tasks.workunit.client.0.smithi136.stderr: -54> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -53> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -52> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -51> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -50> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -49> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -48> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -47> 2024-04-22T20:08:52.945+0000 7fd035fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -46> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -45> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -44> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -43> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -42> 2024-04-22T20:08:52.945+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -41> 2024-04-22T20:08:52.945+0000 7fd182262800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -40> 2024-04-22T20:08:52.945+0000 7fd182262800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -39> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cache list hook 0x560bea066500 2024-04-22T20:08:53.013 INFO:tasks.workunit.client.0.smithi136.stderr: -38> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cache inspect hook 0x560bea066500 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -37> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cache erase hook 0x560bea066500 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -36> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cache zap hook 0x560bea066500 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -35> 2024-04-22T20:08:52.945+0000 7fd182262800 5 note: GC not initialized 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -34> 2024-04-22T20:08:52.945+0000 7fd034fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -33> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command sync trace show hook 0x7fd1780043a0 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -32> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command sync trace history hook 0x7fd1780043a0 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -31> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command sync trace active hook 0x7fd1780043a0 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -30> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command sync trace active_short hook 0x7fd1780043a0 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -29> 2024-04-22T20:08:52.945+0000 7fd182262800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -28> 2024-04-22T20:08:52.945+0000 7fd182262800 20 Filter name: none 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -27> 2024-04-22T20:08:52.945+0000 7fcfae7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -26> 2024-04-22T20:08:52.945+0000 7fd182262800 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -25> 2024-04-22T20:08:52.945+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.014 INFO:tasks.workunit.client.0.smithi136.stderr: -24> 2024-04-22T20:08:52.946+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -23> 2024-04-22T20:08:52.946+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -22> 2024-04-22T20:08:52.946+0000 7fd182262800 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -21> 2024-04-22T20:08:52.946+0000 7fd182262800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 : miss 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -20> 2024-04-22T20:08:52.946+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -19> 2024-04-22T20:08:52.946+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -18> 2024-04-22T20:08:52.946+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -17> 2024-04-22T20:08:52.946+0000 7fd182262800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -16> 2024-04-22T20:08:52.946+0000 7fd182262800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -15> 2024-04-22T20:08:52.946+0000 7fd182262800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -14> 2024-04-22T20:08:52.946+0000 7fd182262800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -13> 2024-04-22T20:08:52.950+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -12> 2024-04-22T20:08:52.950+0000 7fd182262800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -11> 2024-04-22T20:08:52.950+0000 7fd182262800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -10> 2024-04-22T20:08:52.950+0000 7fd182262800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23, ofs0, ns] 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -9> 2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562454 cookie 94609171139344 notifier 6236 bl.length()=907 2024-04-22T20:08:53.015 INFO:tasks.workunit.client.0.smithi136.stderr: -8> 2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: -7> 2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: -6> 2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: -5> 2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_sources= new_sources= 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: -4> 2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential sources added=[] removed=[] 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: -3> 2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_dests= new_dests= 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: -2> 2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential dests added=[] removed=[] 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: -1> 2024-04-22T20:08:52.952+0000 7fd182262800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fd182262800 time 2024-04-22T20:08:52.953562+0000 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd185581f5c] 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd18558211a] 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560be8f25dc1] 2024-04-22T20:08:53.016 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2b4) [0x560be8f33e14] 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fd184429590] 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 0> 2024-04-22T20:08:52.953+0000 7fd182262800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fd182262800 thread_name:radosgw-admin 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd18443e6f0] 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd18448b94c] 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:08:53.017 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd185581fb6] 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd18558211a] 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560be8f25dc1] 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2b4) [0x560be8f33e14] 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fd184429590] 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr:--- logging levels --- 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 none 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 lockdep 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 context 2024-04-22T20:08:53.018 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 crush 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_balancer 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_locker 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log_expire 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_migrator 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 3/ 5 mds_quiesce 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 buffer 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 timer 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 filer 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 striper 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 objecter 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rados 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_mirror 2024-04-22T20:08:53.019 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_replay 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_pwl 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 journaler 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objectcacher 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 immutable_obj_cache 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 client 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 osd 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 optracker 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objclass 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 filestore 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 journal 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 ms 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mon 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/10 monc 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 paxos 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 tp 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 auth 2024-04-22T20:08:53.020 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 crypto 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 finisher 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 reserver 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 heartbeatmap 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 perfcounter 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 20/20 rgw 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_sync 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_datacache 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_access 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_dbstore 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_flight 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_lifecycle 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_notification 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 javaclient 2024-04-22T20:08:53.021 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 asok 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 throttle 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 refs 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 compressor 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluestore 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluefs 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 bdev 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 kstore 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 4/ 5 rocksdb 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 fuse 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 2/ 5 mgr 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mgrc 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 dpdk 2024-04-22T20:08:53.022 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 eventtrace 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 prioritycache 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 test 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephfs_mirror 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephsqlite 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 crimson_interrupt 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_onode 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_odata 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_omap 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_tm 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_t 2024-04-22T20:08:53.023 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cleaner 2024-04-22T20:08:53.025 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_epm 2024-04-22T20:08:53.025 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_lba 2024-04-22T20:08:53.025 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cache 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_journal 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_device 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_backref 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 alienstore 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mclock 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cyanstore 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 ceph_exporter 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 memstore 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 trace 2024-04-22T20:08:53.026 INFO:tasks.workunit.client.0.smithi136.stderr: -2/-2 (syslog threshold) 2024-04-22T20:08:53.028 INFO:tasks.workunit.client.0.smithi136.stderr: 99/99 (stderr threshold) 2024-04-22T20:08:53.028 INFO:tasks.workunit.client.0.smithi136.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-22T20:08:53.028 INFO:tasks.workunit.client.0.smithi136.stderr: 7fcfae7fc640 / http_manager 2024-04-22T20:08:53.029 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd034fe1640 / http_manager 2024-04-22T20:08:53.029 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd035fe3640 / rgw_dt_lg_renew 2024-04-22T20:08:53.029 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd05d7fa640 / ms_dispatch 2024-04-22T20:08:53.029 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd05effd640 / io_context_pool 2024-04-22T20:08:53.029 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd0657fa640 / ms_dispatch 2024-04-22T20:08:53.029 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd067fff640 / ms_dispatch 2024-04-22T20:08:53.029 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd06de0b640 / io_context_pool 2024-04-22T20:08:53.029 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd17f560640 / msgr-worker-2 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd17fd61640 / msgr-worker-1 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd180562640 / msgr-worker-0 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd180d63640 / io_context_pool 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd182262800 / radosgw-admin 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr: max_recent 500 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr: max_new 1000 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr: log_file /var/log/ceph/ceph-client.admin.95829.log 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr:--- end dump of recent events --- 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr:--- begin dump of recent events --- 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr: -499> 2024-04-22T20:08:52.500+0000 7fd06de0b640 10 monclient: dump: 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.030 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr: -498> 2024-04-22T20:08:52.500+0000 7fd182262800 5 monclient: authenticate success, global_id 6555 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr: -497> 2024-04-22T20:08:52.500+0000 7fd182262800 10 monclient: get_monmap_and_config success 2024-04-22T20:08:53.031 INFO:tasks.workunit.client.0.smithi136.stderr: -496> 2024-04-22T20:08:52.500+0000 7fd182262800 4 set_mon_vals no callback set 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr: -495> 2024-04-22T20:08:52.501+0000 7fd182262800 10 monclient: shutdown 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr: -494> 2024-04-22T20:08:52.501+0000 7fd182262800 5 asok(0x560be9d77d00) unregister_commands rotate-key 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr: -493> 2024-04-22T20:08:52.501+0000 7fd182262800 10 monclient: build_initial_monmap 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr: -492> 2024-04-22T20:08:52.501+0000 7fd182262800 1 build_initial for_mkfs: 0 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr: -491> 2024-04-22T20:08:52.501+0000 7fd182262800 10 monclient: monmap: 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:53.032 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -490> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding auth protocol: cephx 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -489> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding auth protocol: cephx 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -488> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding auth protocol: cephx 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -487> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -486> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -485> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -484> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -483> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -482> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -481> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -480> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:53.033 INFO:tasks.workunit.client.0.smithi136.stderr: -479> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -478> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -477> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: crc 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -476> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f9e780) adding con mode: secure 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -475> 2024-04-22T20:08:52.501+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -474> 2024-04-22T20:08:52.501+0000 7fd182262800 1 librados: starting msgr at 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -473> 2024-04-22T20:08:52.501+0000 7fd182262800 1 librados: starting objecter 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -472> 2024-04-22T20:08:52.501+0000 7fd182262800 5 asok(0x560be9d77d00) register_command objecter_requests hook 0x560be9fa0520 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -471> 2024-04-22T20:08:52.501+0000 7fd182262800 1 librados: setting wanted keys 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -470> 2024-04-22T20:08:52.501+0000 7fd182262800 1 librados: calling monclient init 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -469> 2024-04-22T20:08:52.501+0000 7fd182262800 10 monclient: init 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -468> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding auth protocol: cephx 2024-04-22T20:08:53.034 INFO:tasks.workunit.client.0.smithi136.stderr: -467> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding auth protocol: cephx 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -466> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding auth protocol: cephx 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -465> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -464> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -463> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -462> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -461> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -460> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -459> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -458> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -457> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -456> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -455> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: crc 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -454> 2024-04-22T20:08:52.501+0000 7fd182262800 5 AuthRegistry(0x560be9f98a20) adding con mode: secure 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -453> 2024-04-22T20:08:52.502+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.035 INFO:tasks.workunit.client.0.smithi136.stderr: -452> 2024-04-22T20:08:52.502+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -451> 2024-04-22T20:08:52.502+0000 7fd182262800 5 asok(0x560be9d77d00) register_command rotate-key hook 0x560be9f98b68 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -450> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -449> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient: _add_conns ranks=[0,1,2] 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -448> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): picked mon.noname-a con 0x560be9e43f40 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -447> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): picked mon.noname-b con 0x560be9e4ed30 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -446> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): picked mon.noname-c con 0x560be9ee0960 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -445> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -444> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -443> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -442> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -441> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient(hunting): authenticate will time out at 1921.854257s 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -440> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): get_auth_request con 0x560be9e43f40 auth_method 0 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -439> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -438> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.036 INFO:tasks.workunit.client.0.smithi136.stderr: -437> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -436> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): get_auth_request con 0x560be9ee0960 auth_method 0 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -435> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -434> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -433> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -432> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -431> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -430> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -429> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -428> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -427> 2024-04-22T20:08:52.502+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -426> 2024-04-22T20:08:52.502+0000 7fd17f560640 10 monclient(hunting): get_auth_request con 0x560be9e4ed30 auth_method 0 2024-04-22T20:08:53.037 INFO:tasks.workunit.client.0.smithi136.stderr: -425> 2024-04-22T20:08:52.502+0000 7fd17f560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -424> 2024-04-22T20:08:52.502+0000 7fd17f560640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -423> 2024-04-22T20:08:52.502+0000 7fd17f560640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -422> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient(hunting): handle_auth_done global_id 6558 payload 824 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -421> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient: _finish_hunting 0 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -420> 2024-04-22T20:08:52.502+0000 7fd17fd61640 1 monclient: found mon.noname-a 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -419> 2024-04-22T20:08:52.502+0000 7fd17fd61640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -418> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -417> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr: -416> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: dump: 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.038 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr: -415> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: _finish_auth 0 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr: -414> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: _check_auth_tickets 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr: -413> 2024-04-22T20:08:52.502+0000 7fd182262800 5 monclient: authenticate success, global_id 6558 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr: -412> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr: -411> 2024-04-22T20:08:52.502+0000 7fd182262800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr: -410> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:53.039 INFO:tasks.workunit.client.0.smithi136.stderr: -409> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr: -408> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr: -407> 2024-04-22T20:08:52.502+0000 7fd067fff640 10 monclient: dump: 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr: -406> 2024-04-22T20:08:52.502+0000 7fd06de0b640 4 set_mon_vals no callback set 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr: -405> 2024-04-22T20:08:52.503+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr: -404> 2024-04-22T20:08:52.503+0000 7fd182262800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:08:53.040 INFO:tasks.workunit.client.0.smithi136.stderr: -403> 2024-04-22T20:08:52.503+0000 7fd182262800 1 librados: init done 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -402> 2024-04-22T20:08:52.504+0000 7fd067fff640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -401> 2024-04-22T20:08:52.504+0000 7fd067fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -400> 2024-04-22T20:08:52.504+0000 7fd067fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -399> 2024-04-22T20:08:52.504+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd06806b470 auth_method 0 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -398> 2024-04-22T20:08:52.505+0000 7fd180562640 10 monclient: get_auth_request con 0x560be9fc3840 auth_method 0 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -397> 2024-04-22T20:08:52.505+0000 7fd17fd61640 10 monclient: get_auth_request con 0x560be9fe65e0 auth_method 0 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -396> 2024-04-22T20:08:52.506+0000 7fd17f560640 10 monclient: get_auth_request con 0x560be9fea940 auth_method 0 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -395> 2024-04-22T20:08:52.507+0000 7fd180562640 10 monclient: get_auth_request con 0x560be9fe9e90 auth_method 0 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -394> 2024-04-22T20:08:52.508+0000 7fd182262800 10 monclient: build_initial_monmap 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -393> 2024-04-22T20:08:52.508+0000 7fd182262800 1 build_initial for_mkfs: 0 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr: -392> 2024-04-22T20:08:52.508+0000 7fd182262800 10 monclient: monmap: 2024-04-22T20:08:53.041 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr: -391> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding auth protocol: cephx 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr: -390> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding auth protocol: cephx 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr: -389> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding auth protocol: cephx 2024-04-22T20:08:53.042 INFO:tasks.workunit.client.0.smithi136.stderr: -388> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -387> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -386> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -385> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -384> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -383> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -382> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -381> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -380> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -379> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:53.043 INFO:tasks.workunit.client.0.smithi136.stderr: -378> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: crc 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -377> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff2a80) adding con mode: secure 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -376> 2024-04-22T20:08:52.508+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -375> 2024-04-22T20:08:52.508+0000 7fd182262800 5 asok(0x560be9d77d00) register_command objecter_requests cmddesc objecter_requests hook 0x560be9ff8010 EEXIST 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -374> 2024-04-22T20:08:52.508+0000 7fd182262800 10 monclient: init 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -373> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding auth protocol: cephx 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -372> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding auth protocol: cephx 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -371> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding auth protocol: cephx 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -370> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -369> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -368> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -367> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -366> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:53.044 INFO:tasks.workunit.client.0.smithi136.stderr: -365> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -364> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -363> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -362> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -361> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -360> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: crc 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -359> 2024-04-22T20:08:52.508+0000 7fd182262800 5 AuthRegistry(0x560be9ff1958) adding con mode: secure 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -358> 2024-04-22T20:08:52.508+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -357> 2024-04-22T20:08:52.509+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -356> 2024-04-22T20:08:52.509+0000 7fd182262800 5 asok(0x560be9d77d00) register_command rotate-key cmddesc rotate-key hook 0x560be9ff1aa0 EEXIST 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -355> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -354> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _add_conns ranks=[2,1,0] 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -353> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): picked mon.noname-c con 0x560bea01cd70 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -352> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): picked mon.noname-b con 0x560bea01d150 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:53.045 INFO:tasks.workunit.client.0.smithi136.stderr: -351> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): picked mon.noname-a con 0x560bea018210 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -350> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -349> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -348> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -347> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -346> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient(hunting): authenticate will time out at 1921.861010s 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -345> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): get_auth_request con 0x560bea01cd70 auth_method 0 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -344> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -343> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -342> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -341> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): get_auth_request con 0x560bea018210 auth_method 0 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -340> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -339> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -338> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -337> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): get_auth_request con 0x560bea01d150 auth_method 0 2024-04-22T20:08:53.046 INFO:tasks.workunit.client.0.smithi136.stderr: -336> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -335> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -334> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -333> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -332> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -331> 2024-04-22T20:08:52.509+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -330> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -329> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -328> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -327> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -326> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -325> 2024-04-22T20:08:52.509+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -324> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient(hunting): handle_auth_done global_id 6230 payload 824 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -323> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient: _finish_hunting 0 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -322> 2024-04-22T20:08:52.509+0000 7fd17f560640 1 monclient: found mon.noname-c 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -321> 2024-04-22T20:08:52.509+0000 7fd17f560640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.047 INFO:tasks.workunit.client.0.smithi136.stderr: -320> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr: -319> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr: -318> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: dump: 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr: -317> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: _finish_auth 0 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr: -316> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: _check_auth_tickets 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr: -315> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:53.048 INFO:tasks.workunit.client.0.smithi136.stderr: -314> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr: -313> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr: -312> 2024-04-22T20:08:52.509+0000 7fd0657fa640 10 monclient: dump: 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr: -311> 2024-04-22T20:08:52.509+0000 7fd182262800 5 monclient: authenticate success, global_id 6230 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr: -310> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr: -309> 2024-04-22T20:08:52.509+0000 7fd180d63640 4 set_mon_vals no callback set 2024-04-22T20:08:53.049 INFO:tasks.workunit.client.0.smithi136.stderr: -308> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -307> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -306> 2024-04-22T20:08:52.509+0000 7fd182262800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -305> 2024-04-22T20:08:52.511+0000 7fd0657fa640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -304> 2024-04-22T20:08:52.511+0000 7fd0657fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -303> 2024-04-22T20:08:52.511+0000 7fd0657fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -302> 2024-04-22T20:08:52.511+0000 7fd180562640 10 monclient: get_auth_request con 0x7fd05806b1e0 auth_method 0 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -301> 2024-04-22T20:08:52.511+0000 7fd182262800 10 monclient: get_monmap_and_config 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -300> 2024-04-22T20:08:52.511+0000 7fd182262800 10 monclient: build_initial_monmap 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -299> 2024-04-22T20:08:52.511+0000 7fd182262800 1 build_initial for_mkfs: 0 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr: -298> 2024-04-22T20:08:52.511+0000 7fd182262800 10 monclient: monmap: 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:53.050 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -297> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -296> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -295> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -294> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -293> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -292> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -291> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -290> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.051 INFO:tasks.workunit.client.0.smithi136.stderr: -289> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -288> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -287> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -286> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -285> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -284> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -283> 2024-04-22T20:08:52.511+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -282> 2024-04-22T20:08:52.511+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -281> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient: init 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -280> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding auth protocol: cephx 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -279> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding auth protocol: cephx 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -278> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding auth protocol: cephx 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -277> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:53.052 INFO:tasks.workunit.client.0.smithi136.stderr: -276> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -275> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -274> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -273> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -272> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -271> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -270> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -269> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -268> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -267> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: crc 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -266> 2024-04-22T20:08:52.512+0000 7fd182262800 5 AuthRegistry(0x7fff0ce238c0) adding con mode: secure 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -265> 2024-04-22T20:08:52.512+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -264> 2024-04-22T20:08:52.512+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -263> 2024-04-22T20:08:52.512+0000 7fd182262800 5 asok(0x560be9d77d00) register_command rotate-key cmddesc rotate-key hook 0x7fff0ce23a08 EEXIST 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -262> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -261> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient: _add_conns ranks=[0,1,2] 2024-04-22T20:08:53.053 INFO:tasks.workunit.client.0.smithi136.stderr: -260> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): picked mon.noname-a con 0x560bea06f2c0 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -259> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): picked mon.noname-b con 0x560bea073f20 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -258> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): picked mon.noname-c con 0x560bea074c30 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -257> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -256> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -255> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -254> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -253> 2024-04-22T20:08:52.512+0000 7fd182262800 10 monclient(hunting): authenticate will time out at 1921.864473s 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -252> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): get_auth_request con 0x560bea074c30 auth_method 0 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -251> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -250> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -249> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -248> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): get_auth_request con 0x560bea06f2c0 auth_method 0 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -247> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -246> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.054 INFO:tasks.workunit.client.0.smithi136.stderr: -245> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -244> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -243> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -242> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -241> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -240> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -239> 2024-04-22T20:08:52.512+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -238> 2024-04-22T20:08:52.512+0000 7fd180562640 10 monclient(hunting): get_auth_request con 0x560bea073f20 auth_method 0 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -237> 2024-04-22T20:08:52.512+0000 7fd180562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -236> 2024-04-22T20:08:52.512+0000 7fd180562640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -235> 2024-04-22T20:08:52.512+0000 7fd180562640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -234> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient(hunting): handle_auth_done global_id 6233 payload 274 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -233> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient: _finish_hunting 0 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -232> 2024-04-22T20:08:52.512+0000 7fd17fd61640 1 monclient: found mon.noname-c 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -231> 2024-04-22T20:08:52.512+0000 7fd17fd61640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -230> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.055 INFO:tasks.workunit.client.0.smithi136.stderr: -229> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr: -228> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: dump: 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.056 INFO:tasks.workunit.client.0.smithi136.stderr: -227> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: _finish_auth 0 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr: -226> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: _check_auth_tickets 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr: -225> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr: -224> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr: -223> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr: -222> 2024-04-22T20:08:52.513+0000 7fd05effd640 10 monclient: dump: 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.057 INFO:tasks.workunit.client.0.smithi136.stderr: -221> 2024-04-22T20:08:52.513+0000 7fd182262800 5 monclient: authenticate success, global_id 6233 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr: -220> 2024-04-22T20:08:52.513+0000 7fd182262800 10 monclient: get_monmap_and_config success 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr: -219> 2024-04-22T20:08:52.513+0000 7fd182262800 4 set_mon_vals no callback set 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr: -218> 2024-04-22T20:08:52.513+0000 7fd182262800 10 monclient: shutdown 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr: -217> 2024-04-22T20:08:52.513+0000 7fd182262800 10 monclient: build_initial_monmap 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr: -216> 2024-04-22T20:08:52.513+0000 7fd182262800 1 build_initial for_mkfs: 0 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr: -215> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient: monmap: 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:08:53.058 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -214> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -213> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -212> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding auth protocol: cephx 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -211> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -210> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -209> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -208> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -207> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -206> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -205> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -204> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -203> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -202> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.059 INFO:tasks.workunit.client.0.smithi136.stderr: -201> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: crc 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -200> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea03a8c0) adding con mode: secure 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -199> 2024-04-22T20:08:52.514+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -198> 2024-04-22T20:08:52.514+0000 7fd182262800 1 librados: starting msgr at 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -197> 2024-04-22T20:08:52.514+0000 7fd182262800 1 librados: starting objecter 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -196> 2024-04-22T20:08:52.514+0000 7fd182262800 5 asok(0x560be9d77d00) register_command objecter_requests cmddesc objecter_requests hook 0x560bea06c550 EEXIST 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -195> 2024-04-22T20:08:52.514+0000 7fd182262800 1 librados: setting wanted keys 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -194> 2024-04-22T20:08:52.514+0000 7fd182262800 1 librados: calling monclient init 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -193> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient: init 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -192> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding auth protocol: cephx 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -191> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding auth protocol: cephx 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -190> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding auth protocol: cephx 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -189> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -188> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -187> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.060 INFO:tasks.workunit.client.0.smithi136.stderr: -186> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -185> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -184> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -183> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -182> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -181> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -180> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -179> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: crc 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -178> 2024-04-22T20:08:52.514+0000 7fd182262800 5 AuthRegistry(0x560bea039a90) adding con mode: secure 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -177> 2024-04-22T20:08:52.514+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -176> 2024-04-22T20:08:52.514+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -175> 2024-04-22T20:08:52.514+0000 7fd182262800 5 asok(0x560be9d77d00) register_command rotate-key cmddesc rotate-key hook 0x560bea039bd8 EEXIST 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -174> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient: _reopen_session rank -1 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -173> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient: _add_conns ranks=[1,0,2] 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -172> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): picked mon.noname-b con 0x560bea06f2c0 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:08:53.061 INFO:tasks.workunit.client.0.smithi136.stderr: -171> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): picked mon.noname-a con 0x560bea073f20 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -170> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): picked mon.noname-c con 0x560bea074c30 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -169> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -168> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -167> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): start opening mon connection 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -166> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): _renew_subs 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -165> 2024-04-22T20:08:52.514+0000 7fd182262800 10 monclient(hunting): authenticate will time out at 1921.866801s 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -164> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): get_auth_request con 0x560bea074c30 auth_method 0 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -163> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -162> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -161> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.062 INFO:tasks.workunit.client.0.smithi136.stderr: -160> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): get_auth_request con 0x560bea073f20 auth_method 0 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -159> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -158> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -157> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -156> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -155> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -154> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -153> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): get_auth_request con 0x560bea06f2c0 auth_method 0 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -152> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -151> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -150> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -149> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -148> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.063 INFO:tasks.workunit.client.0.smithi136.stderr: -147> 2024-04-22T20:08:52.515+0000 7fd180562640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -146> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -145> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -144> 2024-04-22T20:08:52.515+0000 7fd17f560640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -143> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient(hunting): handle_auth_done global_id 6236 payload 824 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -142> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient: _finish_hunting 0 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -141> 2024-04-22T20:08:52.515+0000 7fd17fd61640 1 monclient: found mon.noname-c 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -140> 2024-04-22T20:08:52.515+0000 7fd17fd61640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -139> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -138> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr: -137> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: dump: 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.064 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: -136> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: _finish_auth 0 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: -135> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: _check_auth_tickets 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: -134> 2024-04-22T20:08:52.515+0000 7fd182262800 5 monclient: authenticate success, global_id 6236 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: -133> 2024-04-22T20:08:52.515+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: -132> 2024-04-22T20:08:52.515+0000 7fd182262800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: -131> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: handle_config config(0 keys) 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: -130> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:08:53.065 INFO:tasks.workunit.client.0.smithi136.stderr: -129> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr: -128> 2024-04-22T20:08:52.515+0000 7fd05d7fa640 10 monclient: dump: 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr: -127> 2024-04-22T20:08:52.515+0000 7fd05effd640 4 set_mon_vals no callback set 2024-04-22T20:08:53.066 INFO:tasks.workunit.client.0.smithi136.stderr: -126> 2024-04-22T20:08:52.515+0000 7fd182262800 10 monclient: _renew_subs 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -125> 2024-04-22T20:08:52.515+0000 7fd182262800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -124> 2024-04-22T20:08:52.515+0000 7fd182262800 1 librados: init done 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -123> 2024-04-22T20:08:52.515+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cr dump hook 0x7fd170055f38 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -122> 2024-04-22T20:08:52.516+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -121> 2024-04-22T20:08:52.517+0000 7fd05d7fa640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -120> 2024-04-22T20:08:52.517+0000 7fd05d7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -119> 2024-04-22T20:08:52.517+0000 7fd05d7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -118> 2024-04-22T20:08:52.517+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd04806b2d0 auth_method 0 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -117> 2024-04-22T20:08:52.517+0000 7fd180562640 10 monclient: get_auth_request con 0x560bea088e80 auth_method 0 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -116> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -115> 2024-04-22T20:08:52.518+0000 7fd182262800 20 realm 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -114> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -113> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -112> 2024-04-22T20:08:52.518+0000 7fd182262800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:53.067 INFO:tasks.workunit.client.0.smithi136.stderr: -111> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -110> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -109> 2024-04-22T20:08:52.518+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -108> 2024-04-22T20:08:52.518+0000 7fd17f560640 10 monclient: get_auth_request con 0x560bea08d0b0 auth_method 0 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -107> 2024-04-22T20:08:52.519+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -106> 2024-04-22T20:08:52.519+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -105> 2024-04-22T20:08:52.519+0000 7fd180562640 10 monclient: get_auth_request con 0x560bea090910 auth_method 0 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -104> 2024-04-22T20:08:52.519+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -103> 2024-04-22T20:08:52.519+0000 7fd182262800 20 searching for the correct realm 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -102> 2024-04-22T20:08:52.519+0000 7fd17fd61640 10 monclient: get_auth_request con 0x560bea095620 auth_method 0 2024-04-22T20:08:53.068 INFO:tasks.workunit.client.0.smithi136.stderr: -101> 2024-04-22T20:08:52.520+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd170082070 auth_method 0 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -100> 2024-04-22T20:08:52.521+0000 7fd180562640 10 monclient: get_auth_request con 0x7fd178028070 auth_method 0 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -99> 2024-04-22T20:08:52.521+0000 7fd180562640 0 -- 172.21.15.136:0/1735850960 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fd178028070 msgr2=0x7fd178037960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -98> 2024-04-22T20:08:52.722+0000 7fd180562640 10 monclient: get_auth_request con 0x7fd178028070 auth_method 0 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -97> 2024-04-22T20:08:52.724+0000 7fd17fd61640 10 monclient: get_auth_request con 0x7fd178040b20 auth_method 0 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -96> 2024-04-22T20:08:52.726+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd17800a940 auth_method 0 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -95> 2024-04-22T20:08:52.726+0000 7fd17f560640 0 -- 172.21.15.136:0/1735850960 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fd17800a940 msgr2=0x7fd178051410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -94> 2024-04-22T20:08:52.927+0000 7fd17f560640 10 monclient: get_auth_request con 0x7fd17800a940 auth_method 0 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -93> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -92> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -91> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:53.069 INFO:tasks.workunit.client.0.smithi136.stderr: -90> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -89> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -88> 2024-04-22T20:08:52.932+0000 7fd182262800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -87> 2024-04-22T20:08:52.932+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -86> 2024-04-22T20:08:52.932+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -85> 2024-04-22T20:08:52.932+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -84> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -83> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -82> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -81> 2024-04-22T20:08:52.933+0000 7fd182262800 20 zone default found 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -80> 2024-04-22T20:08:52.933+0000 7fd182262800 4 Realm: () 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -79> 2024-04-22T20:08:52.933+0000 7fd182262800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -78> 2024-04-22T20:08:52.933+0000 7fd182262800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:53.070 INFO:tasks.workunit.client.0.smithi136.stderr: -77> 2024-04-22T20:08:52.933+0000 7fd182262800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -76> 2024-04-22T20:08:52.933+0000 7fd182262800 20 zonegroup default 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -75> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -74> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -73> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -72> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -71> 2024-04-22T20:08:52.933+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -70> 2024-04-22T20:08:52.934+0000 7fd182262800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -69> 2024-04-22T20:08:52.934+0000 7fd182262800 20 started sync module instance, tier type = 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -68> 2024-04-22T20:08:52.934+0000 7fd182262800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -67> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=0 2024-04-22T20:08:53.071 INFO:tasks.workunit.client.0.smithi136.stderr: -66> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=1 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -65> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=2 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -64> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=3 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -63> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=4 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -62> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=5 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -61> 2024-04-22T20:08:52.942+0000 7fd182262800 20 add_watcher() i=6 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -60> 2024-04-22T20:08:52.943+0000 7fd182262800 20 add_watcher() i=7 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -59> 2024-04-22T20:08:52.943+0000 7fd182262800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -58> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding auth protocol: cephx 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -57> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding auth protocol: cephx 2024-04-22T20:08:53.072 INFO:tasks.workunit.client.0.smithi136.stderr: -56> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding auth protocol: cephx 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -55> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -54> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -53> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -52> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -51> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -50> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -49> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -48> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -47> 2024-04-22T20:08:52.945+0000 7fd035fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -46> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -45> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -44> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: crc 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -43> 2024-04-22T20:08:52.945+0000 7fd182262800 5 AuthRegistry(0x7fff0ce23d50) adding con mode: secure 2024-04-22T20:08:53.073 INFO:tasks.workunit.client.0.smithi136.stderr: -42> 2024-04-22T20:08:52.945+0000 7fd182262800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -41> 2024-04-22T20:08:52.945+0000 7fd182262800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -40> 2024-04-22T20:08:52.945+0000 7fd182262800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -39> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cache list hook 0x560bea066500 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -38> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cache inspect hook 0x560bea066500 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -37> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cache erase hook 0x560bea066500 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -36> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command cache zap hook 0x560bea066500 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -35> 2024-04-22T20:08:52.945+0000 7fd182262800 5 note: GC not initialized 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -34> 2024-04-22T20:08:52.945+0000 7fd034fe1640 20 reqs_thread_entry: start 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -33> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command sync trace show hook 0x7fd1780043a0 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -32> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command sync trace history hook 0x7fd1780043a0 2024-04-22T20:08:53.074 INFO:tasks.workunit.client.0.smithi136.stderr: -31> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command sync trace active hook 0x7fd1780043a0 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -30> 2024-04-22T20:08:52.945+0000 7fd182262800 5 asok(0x560be9d77d00) register_command sync trace active_short hook 0x7fd1780043a0 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -29> 2024-04-22T20:08:52.945+0000 7fd182262800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -28> 2024-04-22T20:08:52.945+0000 7fd182262800 20 Filter name: none 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -27> 2024-04-22T20:08:52.945+0000 7fcfae7fc640 20 reqs_thread_entry: start 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -26> 2024-04-22T20:08:52.945+0000 7fd182262800 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -25> 2024-04-22T20:08:52.945+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -24> 2024-04-22T20:08:52.946+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -23> 2024-04-22T20:08:52.946+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -22> 2024-04-22T20:08:52.946+0000 7fd182262800 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -21> 2024-04-22T20:08:52.946+0000 7fd182262800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 : miss 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -20> 2024-04-22T20:08:52.946+0000 7fd182262800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -19> 2024-04-22T20:08:52.946+0000 7fd182262800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -18> 2024-04-22T20:08:52.946+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -17> 2024-04-22T20:08:52.946+0000 7fd182262800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:53.075 INFO:tasks.workunit.client.0.smithi136.stderr: -16> 2024-04-22T20:08:52.946+0000 7fd182262800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -15> 2024-04-22T20:08:52.946+0000 7fd182262800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -14> 2024-04-22T20:08:52.946+0000 7fd182262800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -13> 2024-04-22T20:08:52.950+0000 7fd182262800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -12> 2024-04-22T20:08:52.950+0000 7fd182262800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -11> 2024-04-22T20:08:52.950+0000 7fd182262800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -10> 2024-04-22T20:08:52.950+0000 7fd182262800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23, ofs0, ns] 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -9> 2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562454 cookie 94609171139344 notifier 6236 bl.length()=907 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -8> 2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -7> 2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -6> 2024-04-22T20:08:52.950+0000 7fd05effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -5> 2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_sources= new_sources= 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -4> 2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential sources added=[] removed=[] 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -3> 2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_dests= new_dests= 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -2> 2024-04-22T20:08:52.952+0000 7fd182262800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential dests added=[] removed=[] 2024-04-22T20:08:53.076 INFO:tasks.workunit.client.0.smithi136.stderr: -1> 2024-04-22T20:08:52.952+0000 7fd182262800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fd182262800 time 2024-04-22T20:08:52.953562+0000 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd185581f5c] 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd18558211a] 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560be8f25dc1] 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2b4) [0x560be8f33e14] 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fd184429590] 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 0> 2024-04-22T20:08:52.953+0000 7fd182262800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fd182262800 thread_name:radosgw-admin 2024-04-22T20:08:53.077 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd18443e6f0] 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd18448b94c] 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd185581fb6] 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd18558211a] 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x560be8f25dc1] 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2b4) [0x560be8f33e14] 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fd184429590] 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:08:53.078 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr:--- logging levels --- 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 none 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 lockdep 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 context 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 crush 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_balancer 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_locker 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log_expire 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_migrator 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 3/ 5 mds_quiesce 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 buffer 2024-04-22T20:08:53.079 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 timer 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 filer 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 striper 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 objecter 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rados 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_mirror 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_replay 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_pwl 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 journaler 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objectcacher 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 immutable_obj_cache 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 client 2024-04-22T20:08:53.080 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 osd 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 optracker 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objclass 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 filestore 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 journal 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 ms 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mon 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 0/10 monc 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 paxos 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 tp 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 auth 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 crypto 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 finisher 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 reserver 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 heartbeatmap 2024-04-22T20:08:53.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 perfcounter 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 20/20 rgw 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_sync 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_datacache 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_access 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_dbstore 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_flight 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_lifecycle 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_notification 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 javaclient 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 asok 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 throttle 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 refs 2024-04-22T20:08:53.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 compressor 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluestore 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluefs 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 bdev 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 kstore 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 4/ 5 rocksdb 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 fuse 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 2/ 5 mgr 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mgrc 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 dpdk 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 eventtrace 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 prioritycache 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 test 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephfs_mirror 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephsqlite 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 crimson_interrupt 2024-04-22T20:08:53.083 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_onode 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_odata 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_omap 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_tm 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_t 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cleaner 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_epm 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_lba 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cache 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_journal 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_device 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_backref 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 alienstore 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mclock 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cyanstore 2024-04-22T20:08:53.084 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 ceph_exporter 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 memstore 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 trace 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: -2/-2 (syslog threshold) 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 99/99 (stderr threshold) 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fcfae7fc640 / http_manager 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd034fe1640 / http_manager 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd035fe3640 / rgw_dt_lg_renew 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd05d7fa640 / ms_dispatch 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd05effd640 / io_context_pool 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd0657fa640 / ms_dispatch 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd067fff640 / ms_dispatch 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd06de0b640 / io_context_pool 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd17f560640 / msgr-worker-2 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd17fd61640 / msgr-worker-1 2024-04-22T20:08:53.085 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd180562640 / msgr-worker-0 2024-04-22T20:08:53.086 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd180d63640 / io_context_pool 2024-04-22T20:08:53.086 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd182262800 / radosgw-admin 2024-04-22T20:08:53.086 INFO:tasks.workunit.client.0.smithi136.stderr: max_recent 500 2024-04-22T20:08:53.086 INFO:tasks.workunit.client.0.smithi136.stderr: max_new 1000 2024-04-22T20:08:53.086 INFO:tasks.workunit.client.0.smithi136.stderr: log_file /var/lib/ceph/crash/2024-04-22T20:08:52.955023Z_ab043d67-0948-4a8b-9610-095e11ffc938/log 2024-04-22T20:08:53.086 INFO:tasks.workunit.client.0.smithi136.stderr:--- end dump of recent events --- 2024-04-22T20:08:53.086 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.026+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.027+0000 7f98df81f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.027+0000 7f98df81f800 20 realm 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.027+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.027+0000 7f98df81f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.027+0000 7f98df81f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:53.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.027+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.028+0000 7f98df81f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.028+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.029+0000 7f98df81f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:53.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.029+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.029+0000 7f98df81f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:53.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.029+0000 7f98df81f800 20 searching for the correct realm 2024-04-22T20:08:53.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.030+0000 7f98dd325640 0 -- 172.21.15.136:0/264886460 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x55cb4270cfd0 msgr2=0x55cb4270f470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:53.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.033+0000 7f98dd325640 0 -- 172.21.15.136:0/264886460 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x7f98d806a480 msgr2=0x7f98d8071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.242+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 20 zone default found 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 4 Realm: () 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 20 zonegroup default 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.243+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.244+0000 7f98df81f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.244+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.244+0000 7f98df81f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.244+0000 7f98df81f800 20 started sync module instance, tier type = 2024-04-22T20:08:53.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.244+0000 7f98df81f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:53.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.254+0000 7f98df81f800 20 add_watcher() i=0 2024-04-22T20:08:53.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.254+0000 7f98df81f800 20 add_watcher() i=1 2024-04-22T20:08:53.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.254+0000 7f98df81f800 20 add_watcher() i=2 2024-04-22T20:08:53.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.254+0000 7f98df81f800 20 add_watcher() i=3 2024-04-22T20:08:53.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.254+0000 7f98df81f800 20 add_watcher() i=4 2024-04-22T20:08:53.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.254+0000 7f98df81f800 20 add_watcher() i=5 2024-04-22T20:08:53.254 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.254+0000 7f98df81f800 20 add_watcher() i=6 2024-04-22T20:08:53.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.255+0000 7f98df81f800 20 add_watcher() i=7 2024-04-22T20:08:53.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.255+0000 7f98df81f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:53.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.256+0000 7f97937e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.257+0000 7f98df81f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.257+0000 7f98df81f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.257+0000 7f98df81f800 5 note: GC not initialized 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.257+0000 7f97927e4640 20 reqs_thread_entry: start 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.257+0000 7f98df81f800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.257+0000 7f98df81f800 20 Filter name: none 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.257+0000 7f9703fff640 20 reqs_thread_entry: start 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.257+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.258+0000 7f98df81f800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:53.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.258+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.258+0000 7f98df81f800 20 rados_obj.operate() r=0 bl.length=435 2024-04-22T20:08:53.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.258+0000 7f98df81f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:53.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.258+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.259+0000 7f98df81f800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:53.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.259+0000 7f98df81f800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.259+0000 7f98df81f800 20 rados_obj.operate() r=0 bl.length=435 2024-04-22T20:08:53.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.259+0000 7f98df81f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:08:53.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.260+0000 7f97937e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:53.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.263+0000 7f98df81f800 20 remove_watcher() i=0 2024-04-22T20:08:53.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.263+0000 7f98df81f800 2 removed watcher, disabling cache 2024-04-22T20:08:53.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.264+0000 7f98df81f800 20 remove_watcher() i=1 2024-04-22T20:08:53.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.265+0000 7f98df81f800 20 remove_watcher() i=2 2024-04-22T20:08:53.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.266+0000 7f98df81f800 20 remove_watcher() i=3 2024-04-22T20:08:53.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.267+0000 7f98df81f800 20 remove_watcher() i=4 2024-04-22T20:08:53.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.268+0000 7f98df81f800 20 remove_watcher() i=5 2024-04-22T20:08:53.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.269+0000 7f98df81f800 20 remove_watcher() i=6 2024-04-22T20:08:53.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.270+0000 7f98df81f800 20 remove_watcher() i=7 2024-04-22T20:08:53.279 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.279 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:08:53.279 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-block-writes id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:08:53.280 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:08:53.281 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:53.281 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:53.315 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:08:53.315 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:08:53.373 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:08:53.403 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:08:53.403 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:08:53.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.437+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.439+0000 7fcbcf416800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.439+0000 7fcbcf416800 20 realm 2024-04-22T20:08:53.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.439+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.439+0000 7fcbcf416800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.439+0000 7fcbcf416800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:08:53.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.439+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.439+0000 7fcbcf416800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.439+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.440+0000 7fcbcf416800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:53.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.440+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.440+0000 7fcbcf416800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:08:53.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.440+0000 7fcbcf416800 20 searching for the correct realm 2024-04-22T20:08:53.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.442+0000 7fcbcb285640 0 -- 172.21.15.136:0/1374769777 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fcbc405e040 msgr2=0x7fcbc407a440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:08:53.650 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.649+0000 7fcbc3fff640 0 -- 172.21.15.136:0/1374769777 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fcbc405e420 msgr2=0x7fcbc405e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:08:53.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.855+0000 7fcbcf416800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:08:53.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.855+0000 7fcbcf416800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:08:53.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.855+0000 7fcbcf416800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:08:53.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.855+0000 7fcbcf416800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:08:53.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.855+0000 7fcbcf416800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:08:53.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.855+0000 7fcbcf416800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:08:53.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.855+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.856+0000 7fcbcf416800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.856+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.856+0000 7fcbcf416800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:08:53.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.856+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:08:53.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 20 zone default found 2024-04-22T20:08:53.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 4 Realm: () 2024-04-22T20:08:53.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:08:53.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:08:53.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 20 zonegroup default 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.857+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.858+0000 7fcbcf416800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.858+0000 7fcbcf416800 20 started sync module instance, tier type = 2024-04-22T20:08:53.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.858+0000 7fcbcf416800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.867+0000 7fcbcf416800 20 add_watcher() i=0 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.867+0000 7fcbcf416800 20 add_watcher() i=1 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.867+0000 7fcbcf416800 20 add_watcher() i=2 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.867+0000 7fcbcf416800 20 add_watcher() i=3 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.867+0000 7fcbcf416800 20 add_watcher() i=4 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.867+0000 7fcbcf416800 20 add_watcher() i=5 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.867+0000 7fcbcf416800 20 add_watcher() i=6 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.868+0000 7fcbcf416800 20 add_watcher() i=7 2024-04-22T20:08:53.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.868+0000 7fcbcf416800 2 all 8 watchers are set, enabling cache 2024-04-22T20:08:53.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.870+0000 7fca817e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:53.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.870+0000 7fcbcf416800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:08:53.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.870+0000 7fcbcf416800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 5 note: GC not initialized 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fc9fbfff640 20 reqs_thread_entry: start 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 20 init_complete bucket index max shards: 1 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 20 Filter name: none 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fc9f9ffb640 20 reqs_thread_entry: start 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 : miss 2024-04-22T20:08:53.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.871+0000 7fcbcf416800 20 rados->read ofs=0 len=0 2024-04-22T20:08:53.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.872+0000 7fcbcf416800 20 rados_obj.operate() r=0 bl.length=435 2024-04-22T20:08:53.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.872+0000 7fcbcf416800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:08:53.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.872+0000 7fcbcf416800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:08:53.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.872+0000 7fcbcf416800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:08:53.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.872+0000 7fcbcf416800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:08:53.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.872+0000 7fcbcf416800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 2024-04-22T20:08:53.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.873+0000 7fcbcf416800 0 INFO: RGWReshardLock::lock found lock on abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to be held by another RGW process; skipping for now 2024-04-22T20:08:53.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.874+0000 7fca817e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:08:53.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.876+0000 7fcbcf416800 20 remove_watcher() i=0 2024-04-22T20:08:53.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.877+0000 7fcbcf416800 2 removed watcher, disabling cache 2024-04-22T20:08:53.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.878+0000 7fcbcf416800 20 remove_watcher() i=1 2024-04-22T20:08:53.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.879+0000 7fcbcf416800 20 remove_watcher() i=2 2024-04-22T20:08:53.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.880+0000 7fcbcf416800 20 remove_watcher() i=3 2024-04-22T20:08:53.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.881+0000 7fcbcf416800 20 remove_watcher() i=4 2024-04-22T20:08:53.882 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.882+0000 7fcbcf416800 20 remove_watcher() i=5 2024-04-22T20:08:53.883 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.883+0000 7fcbcf416800 20 remove_watcher() i=6 2024-04-22T20:08:53.884 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:08:53.884+0000 7fcbcf416800 20 remove_watcher() i=7 2024-04-22T20:08:53.894 INFO:tasks.workunit.client.0.smithi136.stderr:waiting 30 seconds for reshard lock to expire... 2024-04-22T20:09:23.908 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:09:23.943 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:23.943 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:23.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.984+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:23.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.985+0000 7f57c6f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:23.984 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.985+0000 7f57c6f2d800 20 realm 2024-04-22T20:09:23.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.985+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:23.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.985+0000 7f57c6f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:23.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.985+0000 7f57c6f2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:23.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.985+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:23.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.985+0000 7f57c6f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:23.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.985+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:23.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.986+0000 7f57c6f2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:23.985 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.986+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:23.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.987+0000 7f57c6f2d800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:23.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.987+0000 7f57c6f2d800 20 searching for the correct realm 2024-04-22T20:09:23.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.988+0000 7f57c2b7f640 0 -- 172.21.15.136:0/2526244484 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x56131d694210 msgr2=0x56131d6966d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:23.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:23.991+0000 7f57c2b7f640 0 -- 172.21.15.136:0/2526244484 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x56131d4d9870 msgr2=0x7f57bc015810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:24.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:24.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:24.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:24.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:24.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:24.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:24.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:24.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.202+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:24.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 20 zone default found 2024-04-22T20:09:24.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 4 Realm: () 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 20 zonegroup default 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.203+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.204+0000 7f57c6f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.204+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.204+0000 7f57c6f2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.204+0000 7f57c6f2d800 20 started sync module instance, tier type = 2024-04-22T20:09:24.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.204+0000 7f57c6f2d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:24.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.214+0000 7f57c6f2d800 20 add_watcher() i=0 2024-04-22T20:09:24.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.214+0000 7f57c6f2d800 20 add_watcher() i=1 2024-04-22T20:09:24.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.214+0000 7f57c6f2d800 20 add_watcher() i=2 2024-04-22T20:09:24.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.214+0000 7f57c6f2d800 20 add_watcher() i=3 2024-04-22T20:09:24.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.214+0000 7f57c6f2d800 20 add_watcher() i=4 2024-04-22T20:09:24.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.214+0000 7f57c6f2d800 20 add_watcher() i=5 2024-04-22T20:09:24.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.214+0000 7f57c6f2d800 20 add_watcher() i=6 2024-04-22T20:09:24.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.215+0000 7f57c6f2d800 20 add_watcher() i=7 2024-04-22T20:09:24.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.215+0000 7f57c6f2d800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:24.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.217+0000 7f56797e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:24.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f57c6f2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:24.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f57c6f2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:24.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f57c6f2d800 5 note: GC not initialized 2024-04-22T20:09:24.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f55f3fff640 20 reqs_thread_entry: start 2024-04-22T20:09:24.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f57c6f2d800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:24.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f57c6f2d800 20 Filter name: none 2024-04-22T20:09:24.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f55f1ffb640 20 reqs_thread_entry: start 2024-04-22T20:09:24.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f57c6f2d800 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-04-22T20:09:24.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.218+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 : miss 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 20 rados_obj.operate() r=0 bl.length=435 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:09:24.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:24.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.219+0000 7f57c6f2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 2024-04-22T20:09:24.220 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.221+0000 7f57c6f2d800 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-04-22T20:09:24.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.225+0000 7f57c6f2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:09:24.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.225+0000 7f57c6f2d800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:09:24.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.225+0000 7f57c6f2d800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:24.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.225+0000 7f57c6f2d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23, ofs0, ns] 2024-04-22T20:09:24.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.225+0000 7f56a67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562455 cookie 94640097829872 notifier 6621 bl.length()=907 2024-04-22T20:09:24.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.225+0000 7f56a67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:09:24.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.225+0000 7f56a67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:09:24.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.225+0000 7f56a67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:24.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.227+0000 7f57c6f2d800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_sources= new_sources= 2024-04-22T20:09:24.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.227+0000 7f57c6f2d800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential sources added=[] removed=[] 2024-04-22T20:09:24.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.227+0000 7f57c6f2d800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_dests= new_dests= 2024-04-22T20:09:24.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.227+0000 7f57c6f2d800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential dests added=[] removed=[] 2024-04-22T20:09:24.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.229+0000 7f57c6f2d800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23.2.0 2024-04-22T20:09:24.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.229+0000 7f57c6f2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23.2.0 2024-04-22T20:09:24.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.229+0000 7f57c6f2d800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23.2.1 2024-04-22T20:09:24.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.229+0000 7f57c6f2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23.2.1 2024-04-22T20:09:24.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.229+0000 7f57c6f2d800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23.0 2024-04-22T20:09:24.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.231+0000 7f57c6f2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 : hit (requested=0x17, cached=0x17) 2024-04-22T20:09:24.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.231+0000 7f57c6f2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 2024-04-22T20:09:24.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.233+0000 7f57c6f2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:09:24.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.233+0000 7f57c6f2d800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:09:24.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.233+0000 7f57c6f2d800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:24.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.233+0000 7f57c6f2d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23, ofs0, ns] 2024-04-22T20:09:24.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.233+0000 7f56a67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562456 cookie 94640097829872 notifier 6621 bl.length()=875 2024-04-22T20:09:24.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.233+0000 7f56a67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 info.flags=0x17 2024-04-22T20:09:24.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.233+0000 7f56a67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 to cache LRU end 2024-04-22T20:09:24.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.233+0000 7f56a67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:24.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.235+0000 7f57c6f2d800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_sources= new_sources= 2024-04-22T20:09:24.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.235+0000 7f57c6f2d800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential sources added=[] removed=[] 2024-04-22T20:09:24.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.235+0000 7f57c6f2d800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): orig_dests= new_dests= 2024-04-22T20:09:24.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.235+0000 7f57c6f2d800 20 handle_bi_update(): bucket=:abort-at-block-writes[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23]): potential dests added=[] removed=[] 2024-04-22T20:09:24.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.236+0000 7f57c6f2d800 1 execute INFO: reshard of bucket "abort-at-block-writes" completed successfully 2024-04-22T20:09:24.238 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.238+0000 7f56797e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:24.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.241+0000 7f57c6f2d800 20 remove_watcher() i=0 2024-04-22T20:09:24.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.241+0000 7f57c6f2d800 2 removed watcher, disabling cache 2024-04-22T20:09:24.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.242+0000 7f57c6f2d800 20 remove_watcher() i=1 2024-04-22T20:09:24.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.243+0000 7f57c6f2d800 20 remove_watcher() i=2 2024-04-22T20:09:24.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.244+0000 7f57c6f2d800 20 remove_watcher() i=3 2024-04-22T20:09:24.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.246+0000 7f57c6f2d800 20 remove_watcher() i=4 2024-04-22T20:09:24.246 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.247+0000 7f57c6f2d800 20 remove_watcher() i=5 2024-04-22T20:09:24.247 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.248+0000 7f57c6f2d800 20 remove_watcher() i=6 2024-04-22T20:09:24.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.249+0000 7f57c6f2d800 20 remove_watcher() i=7 2024-04-22T20:09:24.260 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-04-22T20:09:24.289 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:24.289 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:24.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.324+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.326+0000 7f3d31f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.326+0000 7f3d31f46800 20 realm 2024-04-22T20:09:24.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.326+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.326+0000 7f3d31f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.326+0000 7f3d31f46800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:24.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.326+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.326+0000 7f3d31f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.326+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.327+0000 7f3d31f46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:24.326 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.327+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.328+0000 7f3d31f46800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:24.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.328+0000 7f3d31f46800 20 searching for the correct realm 2024-04-22T20:09:24.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.329+0000 7f3d2fa45640 0 -- 172.21.15.136:0/3204219175 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5592f76ed4e0 msgr2=0x5592f76ef9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:24.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.332+0000 7f3d2fa45640 0 -- 172.21.15.136:0/3204219175 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x5592f74a5f90 msgr2=0x7f3d200b9010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:09:24.540 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:24.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.541+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:24.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 20 zone default found 2024-04-22T20:09:24.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 4 Realm: () 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 20 zonegroup default 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.542+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.543+0000 7f3d31f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.543+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.543+0000 7f3d31f46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.543+0000 7f3d31f46800 20 started sync module instance, tier type = 2024-04-22T20:09:24.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.543+0000 7f3d31f46800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:24.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.553+0000 7f3d31f46800 20 add_watcher() i=0 2024-04-22T20:09:24.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.553+0000 7f3d31f46800 20 add_watcher() i=1 2024-04-22T20:09:24.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.553+0000 7f3d31f46800 20 add_watcher() i=2 2024-04-22T20:09:24.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.553+0000 7f3d31f46800 20 add_watcher() i=3 2024-04-22T20:09:24.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.553+0000 7f3d31f46800 20 add_watcher() i=4 2024-04-22T20:09:24.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.553+0000 7f3d31f46800 20 add_watcher() i=5 2024-04-22T20:09:24.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.553+0000 7f3d31f46800 20 add_watcher() i=6 2024-04-22T20:09:24.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.554+0000 7f3d31f46800 20 add_watcher() i=7 2024-04-22T20:09:24.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.554+0000 7f3d31f46800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:24.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.556+0000 7f3be5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:24.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.556+0000 7f3d31f46800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:24.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.556+0000 7f3d31f46800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:24.556 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.556+0000 7f3d31f46800 5 note: GC not initialized 2024-04-22T20:09:24.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.557+0000 7f3be4fe1640 20 reqs_thread_entry: start 2024-04-22T20:09:24.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.557+0000 7f3d31f46800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:24.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.557+0000 7f3d31f46800 20 Filter name: none 2024-04-22T20:09:24.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.557+0000 7f3b5e7fc640 20 reqs_thread_entry: start 2024-04-22T20:09:24.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.557+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.557+0000 7f3d31f46800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:09:24.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.557+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.558+0000 7f3d31f46800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:09:24.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.558+0000 7f3d31f46800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:24.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.558+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.558+0000 7f3d31f46800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:09:24.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.558+0000 7f3d31f46800 20 rados->read ofs=0 len=0 2024-04-22T20:09:24.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.559+0000 7f3d31f46800 20 rados_obj.operate() r=0 bl.length=403 2024-04-22T20:09:24.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.559+0000 7f3d31f46800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:24.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.560+0000 7f3be5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:24.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.562+0000 7f3d31f46800 20 remove_watcher() i=0 2024-04-22T20:09:24.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.562+0000 7f3d31f46800 2 removed watcher, disabling cache 2024-04-22T20:09:24.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.564+0000 7f3d31f46800 20 remove_watcher() i=1 2024-04-22T20:09:24.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.565+0000 7f3d31f46800 20 remove_watcher() i=2 2024-04-22T20:09:24.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.566+0000 7f3d31f46800 20 remove_watcher() i=3 2024-04-22T20:09:24.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.567+0000 7f3d31f46800 20 remove_watcher() i=4 2024-04-22T20:09:24.567 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.568+0000 7f3d31f46800 20 remove_watcher() i=5 2024-04-22T20:09:24.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.569+0000 7f3d31f46800 20 remove_watcher() i=6 2024-04-22T20:09:24.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:24.570+0000 7f3d31f46800 20 remove_watcher() i=7 2024-04-22T20:09:24.578 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:24.578 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:09:24.578 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-block-writes id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-22T20:09:24.578 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:24.580 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:24.580 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:24.601 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:24.601 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:24.828 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:24.828 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:24.893 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with EIO injected at commit_target_layout 2024-04-22T20:09:24.893 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:24.894 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:24.894 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:24.935 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:24.935 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:24.992 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:24.992 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.028 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.028 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.064 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.064 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.101 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.102 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.140 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.140 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.174 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.174 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.208 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.209 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.243 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.243 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.277 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.278 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.312 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.312 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.348 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.348 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.382 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.382 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.416 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.417 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.452 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.453 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.487 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.487 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.522 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.522 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.556 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.557 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.591 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.591 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.625 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.625 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.660 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:25.660 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:25.694 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-22T20:09:25.722 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:25.722 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:25.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.755+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:25.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.757+0000 7f49cf331800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:25.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.757+0000 7f49cf331800 20 realm 2024-04-22T20:09:25.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.757+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:25.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.757+0000 7f49cf331800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:25.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.757+0000 7f49cf331800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:25.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.757+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:25.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.757+0000 7f49cf331800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:25.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.757+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:25.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.758+0000 7f49cf331800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:25.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.758+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:25.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.759+0000 7f49cf331800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:25.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.759+0000 7f49cf331800 20 searching for the correct realm 2024-04-22T20:09:25.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.761+0000 7f49cb830640 0 -- 172.21.15.136:0/3146384630 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f49c4035040 msgr2=0x7f49c4035420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:25.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:25.965+0000 7f49c282e640 0 -- 172.21.15.136:0/3146384630 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f49c4046920 msgr2=0x7f49c4016400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:26.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:26.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:26.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:26.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:26.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:26.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:26.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.171+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 zone default found 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 4 Realm: () 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 zonegroup default 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 started sync module instance, tier type = 2024-04-22T20:09:26.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.172+0000 7f49cf331800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:26.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.181+0000 7f49cf331800 20 add_watcher() i=0 2024-04-22T20:09:26.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.181+0000 7f49cf331800 20 add_watcher() i=1 2024-04-22T20:09:26.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.181+0000 7f49cf331800 20 add_watcher() i=2 2024-04-22T20:09:26.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.181+0000 7f49cf331800 20 add_watcher() i=3 2024-04-22T20:09:26.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.181+0000 7f49cf331800 20 add_watcher() i=4 2024-04-22T20:09:26.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.181+0000 7f49cf331800 20 add_watcher() i=5 2024-04-22T20:09:26.181 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.181+0000 7f49cf331800 20 add_watcher() i=6 2024-04-22T20:09:26.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.182+0000 7f49cf331800 20 add_watcher() i=7 2024-04-22T20:09:26.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.182+0000 7f49cf331800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:26.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.184+0000 7f4881fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:26.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.184+0000 7f49cf331800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:26.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.184+0000 7f49cf331800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:26.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.184+0000 7f49cf331800 5 note: GC not initialized 2024-04-22T20:09:26.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.184+0000 7f4880fe1640 20 reqs_thread_entry: start 2024-04-22T20:09:26.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.184+0000 7f49cf331800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:26.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.185+0000 7f49cf331800 20 Filter name: none 2024-04-22T20:09:26.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.185+0000 7f47fa7fc640 20 reqs_thread_entry: start 2024-04-22T20:09:26.185 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.185+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.186+0000 7f49cf331800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:26.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.186+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.186+0000 7f49cf331800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:26.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.187+0000 7f49cf331800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:26.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.187+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.187+0000 7f49cf331800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:26.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.187+0000 7f49cf331800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.187+0000 7f49cf331800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:26.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.187+0000 7f49cf331800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:26.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.189+0000 7f4881fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:26.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.191+0000 7f49cf331800 20 remove_watcher() i=0 2024-04-22T20:09:26.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.191+0000 7f49cf331800 2 removed watcher, disabling cache 2024-04-22T20:09:26.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.192+0000 7f49cf331800 20 remove_watcher() i=1 2024-04-22T20:09:26.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.193+0000 7f49cf331800 20 remove_watcher() i=2 2024-04-22T20:09:26.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.194+0000 7f49cf331800 20 remove_watcher() i=3 2024-04-22T20:09:26.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.195+0000 7f49cf331800 20 remove_watcher() i=4 2024-04-22T20:09:26.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.196+0000 7f49cf331800 20 remove_watcher() i=5 2024-04-22T20:09:26.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.197+0000 7f49cf331800 20 remove_watcher() i=6 2024-04-22T20:09:26.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.198+0000 7f49cf331800 20 remove_watcher() i=7 2024-04-22T20:09:26.207 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:26.207 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:09:26.207 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-commit-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:09:26.207 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:26.209 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout 2024-04-22T20:09:26.241 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:26.241 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:26.274 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.274+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.275+0000 7f4927729800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.275+0000 7f4927729800 20 realm 2024-04-22T20:09:26.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.275+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.276+0000 7f4927729800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.276+0000 7f4927729800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:26.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.276+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.276+0000 7f4927729800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.276 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.276+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.277+0000 7f4927729800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:26.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.277+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.277+0000 7f4927729800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:26.277 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.277+0000 7f4927729800 20 searching for the correct realm 2024-04-22T20:09:26.280 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.280+0000 7f4922b70640 0 -- 172.21.15.136:0/2107857635 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f491c017070 msgr2=0x7f491c06daa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:26.283 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.283+0000 7f4922b70640 0 -- 172.21.15.136:0/2107857635 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f4914024040 msgr2=0x7f49140625b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.890+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.891 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:26.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:26.892 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 zone default found 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 4 Realm: () 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 zonegroup default 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.891+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.892+0000 7f4927729800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.892+0000 7f4927729800 20 started sync module instance, tier type = 2024-04-22T20:09:26.893 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.892+0000 7f4927729800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:26.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 20 add_watcher() i=0 2024-04-22T20:09:26.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 20 add_watcher() i=1 2024-04-22T20:09:26.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 20 add_watcher() i=2 2024-04-22T20:09:26.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 20 add_watcher() i=3 2024-04-22T20:09:26.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 20 add_watcher() i=4 2024-04-22T20:09:26.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 20 add_watcher() i=5 2024-04-22T20:09:26.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 20 add_watcher() i=6 2024-04-22T20:09:26.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 20 add_watcher() i=7 2024-04-22T20:09:26.903 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.902+0000 7f4927729800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:26.904 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.904+0000 7f47d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:26.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f4927729800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:26.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f4927729800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:26.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f4927729800 5 note: GC not initialized 2024-04-22T20:09:26.905 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f47d8fe1640 20 reqs_thread_entry: start 2024-04-22T20:09:26.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f4927729800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:26.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f4927729800 20 Filter name: none 2024-04-22T20:09:26.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f47527fc640 20 reqs_thread_entry: start 2024-04-22T20:09:26.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f4927729800 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-04-22T20:09:26.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.905+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 : miss 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 20 rados->read ofs=0 len=0 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 info.flags=0x17 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 to cache LRU end 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:26.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.906+0000 7f4927729800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 2024-04-22T20:09:26.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.911+0000 7f4927729800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 info.flags=0x17 2024-04-22T20:09:26.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.911+0000 7f4927729800 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 to cache LRU end 2024-04-22T20:09:26.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.911+0000 7f4927729800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:26.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.911+0000 7f4927729800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24, ofs0, ns] 2024-04-22T20:09:26.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.911+0000 7f4806ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562461 cookie 94588402095264 notifier 6335 bl.length()=923 2024-04-22T20:09:26.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.911+0000 7f4806ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 info.flags=0x17 2024-04-22T20:09:26.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.911+0000 7f4806ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 to cache LRU end 2024-04-22T20:09:26.911 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.911+0000 7f4806ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:26.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.913+0000 7f4927729800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): orig_sources= new_sources= 2024-04-22T20:09:26.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.913+0000 7f4927729800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): potential sources added=[] removed=[] 2024-04-22T20:09:26.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.913+0000 7f4927729800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): orig_dests= new_dests= 2024-04-22T20:09:26.913 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.913+0000 7f4927729800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): potential dests added=[] removed=[] 2024-04-22T20:09:26.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.914+0000 7f4927729800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.1.0 2024-04-22T20:09:26.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.914+0000 7f4927729800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.1.0 2024-04-22T20:09:26.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.914+0000 7f4927729800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.1.1 2024-04-22T20:09:26.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.914+0000 7f4927729800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.1.1 2024-04-22T20:09:26.914 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.914+0000 7f4927729800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.0 2024-04-22T20:09:26.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.916+0000 7f4927729800 -1 Injecting error=-5 at location=commit_target_layout 2024-04-22T20:09:26.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.916+0000 7f4927729800 0 ERROR: commit_reshard failed to commit target index layout: (5) Input/output error 2024-04-22T20:09:26.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.919+0000 7f47d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:26.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.921+0000 7f4927729800 20 remove_watcher() i=0 2024-04-22T20:09:26.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.921+0000 7f4927729800 2 removed watcher, disabling cache 2024-04-22T20:09:26.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.922+0000 7f4927729800 20 remove_watcher() i=1 2024-04-22T20:09:26.923 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.923+0000 7f4927729800 20 remove_watcher() i=2 2024-04-22T20:09:26.924 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.924+0000 7f4927729800 20 remove_watcher() i=3 2024-04-22T20:09:26.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.925+0000 7f4927729800 20 remove_watcher() i=4 2024-04-22T20:09:26.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.926+0000 7f4927729800 20 remove_watcher() i=5 2024-04-22T20:09:26.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.927+0000 7f4927729800 20 remove_watcher() i=6 2024-04-22T20:09:26.928 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:26.928+0000 7f4927729800 20 remove_watcher() i=7 2024-04-22T20:09:26.939 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-22T20:09:26.966 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:26.966 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:27.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.001+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.003+0000 7f16a5136800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.003+0000 7f16a5136800 20 realm 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.003+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.003+0000 7f16a5136800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.003+0000 7f16a5136800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.003+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.003+0000 7f16a5136800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.003+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.004+0000 7f16a5136800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:27.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.004+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.004+0000 7f16a5136800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:27.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.004+0000 7f16a5136800 20 searching for the correct realm 2024-04-22T20:09:27.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.005+0000 7f16a0ecc640 0 -- 172.21.15.136:0/653521853 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5598ca2157f0 msgr2=0x5598ca217c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:27.610 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.610+0000 7f16a0ecc640 0 -- 172.21.15.136:0/653521853 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5598ca2157f0 msgr2=0x5598ca217c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:27.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.617+0000 7f16a5136800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:27.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.617+0000 7f16a5136800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:27.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.617+0000 7f16a5136800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:27.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.617+0000 7f16a5136800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:27.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.617+0000 7f16a5136800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:27.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.617+0000 7f16a5136800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:27.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:27.619 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:27.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:27.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 zone default found 2024-04-22T20:09:27.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 4 Realm: () 2024-04-22T20:09:27.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:27.620 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 zonegroup default 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.618+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.619+0000 7f16a5136800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.619+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.619+0000 7f16a5136800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.619+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.619+0000 7f16a5136800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.619+0000 7f16a5136800 20 started sync module instance, tier type = 2024-04-22T20:09:27.621 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.619+0000 7f16a5136800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:27.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.628+0000 7f16a5136800 20 add_watcher() i=0 2024-04-22T20:09:27.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.628+0000 7f16a5136800 20 add_watcher() i=1 2024-04-22T20:09:27.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.628+0000 7f16a5136800 20 add_watcher() i=2 2024-04-22T20:09:27.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.628+0000 7f16a5136800 20 add_watcher() i=3 2024-04-22T20:09:27.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.628+0000 7f16a5136800 20 add_watcher() i=4 2024-04-22T20:09:27.628 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.628+0000 7f16a5136800 20 add_watcher() i=5 2024-04-22T20:09:27.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.628+0000 7f16a5136800 20 add_watcher() i=6 2024-04-22T20:09:27.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.629+0000 7f16a5136800 20 add_watcher() i=7 2024-04-22T20:09:27.629 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.629+0000 7f16a5136800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:27.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.631+0000 7f1556fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:27.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.631+0000 7f16a5136800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:27.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.631+0000 7f16a5136800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:27.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.631+0000 7f16a5136800 5 note: GC not initialized 2024-04-22T20:09:27.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.631+0000 7f1555fe3640 20 reqs_thread_entry: start 2024-04-22T20:09:27.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.631+0000 7f16a5136800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:27.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.631+0000 7f16a5136800 20 Filter name: none 2024-04-22T20:09:27.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.632+0000 7f14c77fe640 20 reqs_thread_entry: start 2024-04-22T20:09:27.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.632+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.632+0000 7f16a5136800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:27.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.632+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.633+0000 7f16a5136800 20 rados_obj.operate() r=0 bl.length=443 2024-04-22T20:09:27.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.633+0000 7f16a5136800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:27.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.633+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.633+0000 7f16a5136800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:27.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.633+0000 7f16a5136800 20 rados->read ofs=0 len=0 2024-04-22T20:09:27.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.633+0000 7f16a5136800 20 rados_obj.operate() r=0 bl.length=443 2024-04-22T20:09:27.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.633+0000 7f16a5136800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:27.635 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.635+0000 7f1556fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:27.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.637+0000 7f16a5136800 20 remove_watcher() i=0 2024-04-22T20:09:27.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.637+0000 7f16a5136800 2 removed watcher, disabling cache 2024-04-22T20:09:27.638 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.638+0000 7f16a5136800 20 remove_watcher() i=1 2024-04-22T20:09:27.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.639+0000 7f16a5136800 20 remove_watcher() i=2 2024-04-22T20:09:27.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.640+0000 7f16a5136800 20 remove_watcher() i=3 2024-04-22T20:09:27.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.641+0000 7f16a5136800 20 remove_watcher() i=4 2024-04-22T20:09:27.642 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.642+0000 7f16a5136800 20 remove_watcher() i=5 2024-04-22T20:09:27.642 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.643+0000 7f16a5136800 20 remove_watcher() i=6 2024-04-22T20:09:27.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:27.644+0000 7f16a5136800 20 remove_watcher() i=7 2024-04-22T20:09:27.653 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:27.653 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:09:27.653 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-commit-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:09:27.653 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:27.655 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:27.655 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:27.687 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:27.687 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:27.744 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:09:27.777 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:27.777 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:28.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.012+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.013+0000 7f0c16f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.013+0000 7f0c16f2e800 20 realm 2024-04-22T20:09:28.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.013+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.013+0000 7f0c16f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.013+0000 7f0c16f2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:28.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.013+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.013+0000 7f0c16f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.013+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.014+0000 7f0c16f2e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:28.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.014+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.015+0000 7f0c16f2e800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:28.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.015+0000 7f0c16f2e800 20 searching for the correct realm 2024-04-22T20:09:28.017 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.017+0000 7f0c13468640 0 -- 172.21.15.136:0/3803059334 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f0c0c0c5c20 msgr2=0x7f0c0c09aa30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:28.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.021+0000 7f0c0a466640 0 -- 172.21.15.136:0/3803059334 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f0c0c0c53a0 msgr2=0x7f0c0c09a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:28.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:28.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:28.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:28.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:28.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:28.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:28.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.227+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:28.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 20 zone default found 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 4 Realm: () 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 20 zonegroup default 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.228+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.229+0000 7f0c16f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.229+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.229+0000 7f0c16f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.229+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.229+0000 7f0c16f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.229+0000 7f0c16f2e800 20 started sync module instance, tier type = 2024-04-22T20:09:28.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.229+0000 7f0c16f2e800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:28.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.239+0000 7f0c16f2e800 20 add_watcher() i=0 2024-04-22T20:09:28.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.239+0000 7f0c16f2e800 20 add_watcher() i=1 2024-04-22T20:09:28.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.239+0000 7f0c16f2e800 20 add_watcher() i=2 2024-04-22T20:09:28.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.239+0000 7f0c16f2e800 20 add_watcher() i=3 2024-04-22T20:09:28.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.239+0000 7f0c16f2e800 20 add_watcher() i=4 2024-04-22T20:09:28.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.239+0000 7f0c16f2e800 20 add_watcher() i=5 2024-04-22T20:09:28.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.239+0000 7f0c16f2e800 20 add_watcher() i=6 2024-04-22T20:09:28.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.240+0000 7f0c16f2e800 20 add_watcher() i=7 2024-04-22T20:09:28.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.240+0000 7f0c16f2e800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:28.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.241+0000 7f0ac97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:28.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0c16f2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:28.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0c16f2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:28.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0c16f2e800 5 note: GC not initialized 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0a43fff640 20 reqs_thread_entry: start 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0c16f2e800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0c16f2e800 20 Filter name: none 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0a41ffb640 20 reqs_thread_entry: start 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0c16f2e800 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.242+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 : miss 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 20 rados_obj.operate() r=0 bl.length=443 2024-04-22T20:09:28.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 info.flags=0x17 2024-04-22T20:09:28.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 to cache LRU end 2024-04-22T20:09:28.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:09:28.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:28.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.243+0000 7f0c16f2e800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 2024-04-22T20:09:28.245 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.245+0000 7f0c16f2e800 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-04-22T20:09:28.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.248+0000 7f0c16f2e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 info.flags=0x17 2024-04-22T20:09:28.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.248+0000 7f0c16f2e800 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 to cache LRU end 2024-04-22T20:09:28.248 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.248+0000 7f0c16f2e800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:28.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.248+0000 7f0c16f2e800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24, ofs0, ns] 2024-04-22T20:09:28.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.249+0000 7f0af67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562462 cookie 94873119274496 notifier 6708 bl.length()=923 2024-04-22T20:09:28.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.249+0000 7f0af67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 info.flags=0x17 2024-04-22T20:09:28.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.249+0000 7f0af67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 to cache LRU end 2024-04-22T20:09:28.249 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.249+0000 7f0af67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:28.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.251+0000 7f0c16f2e800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): orig_sources= new_sources= 2024-04-22T20:09:28.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.251+0000 7f0c16f2e800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): potential sources added=[] removed=[] 2024-04-22T20:09:28.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.251+0000 7f0c16f2e800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): orig_dests= new_dests= 2024-04-22T20:09:28.251 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.251+0000 7f0c16f2e800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): potential dests added=[] removed=[] 2024-04-22T20:09:28.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.252+0000 7f0c16f2e800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.2.0 2024-04-22T20:09:28.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.252+0000 7f0c16f2e800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.2.0 2024-04-22T20:09:28.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.252+0000 7f0c16f2e800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.2.1 2024-04-22T20:09:28.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.252+0000 7f0c16f2e800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.2.1 2024-04-22T20:09:28.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.252+0000 7f0c16f2e800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24.0 2024-04-22T20:09:28.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.254+0000 7f0c16f2e800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 : hit (requested=0x17, cached=0x17) 2024-04-22T20:09:28.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.254+0000 7f0c16f2e800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 2024-04-22T20:09:28.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.255+0000 7f0c16f2e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 info.flags=0x17 2024-04-22T20:09:28.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.255+0000 7f0c16f2e800 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 to cache LRU end 2024-04-22T20:09:28.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.255+0000 7f0c16f2e800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:28.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.255+0000 7f0c16f2e800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24, ofs0, ns] 2024-04-22T20:09:28.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.256+0000 7f0af5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562463 cookie 94873119274496 notifier 6708 bl.length()=891 2024-04-22T20:09:28.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.256+0000 7f0af5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 info.flags=0x17 2024-04-22T20:09:28.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.256+0000 7f0af5ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 to cache LRU end 2024-04-22T20:09:28.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.256+0000 7f0af5ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:28.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.257+0000 7f0c16f2e800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): orig_sources= new_sources= 2024-04-22T20:09:28.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.257+0000 7f0c16f2e800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): potential sources added=[] removed=[] 2024-04-22T20:09:28.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.257+0000 7f0c16f2e800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): orig_dests= new_dests= 2024-04-22T20:09:28.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.257+0000 7f0c16f2e800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24]): potential dests added=[] removed=[] 2024-04-22T20:09:28.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.258+0000 7f0c16f2e800 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-04-22T20:09:28.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.260+0000 7f0ac97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:28.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.262+0000 7f0c16f2e800 20 remove_watcher() i=0 2024-04-22T20:09:28.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.262+0000 7f0c16f2e800 2 removed watcher, disabling cache 2024-04-22T20:09:28.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.263+0000 7f0c16f2e800 20 remove_watcher() i=1 2024-04-22T20:09:28.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.264+0000 7f0c16f2e800 20 remove_watcher() i=2 2024-04-22T20:09:28.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.266+0000 7f0c16f2e800 20 remove_watcher() i=3 2024-04-22T20:09:28.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.267+0000 7f0c16f2e800 20 remove_watcher() i=4 2024-04-22T20:09:28.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.268+0000 7f0c16f2e800 20 remove_watcher() i=5 2024-04-22T20:09:28.269 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.269+0000 7f0c16f2e800 20 remove_watcher() i=6 2024-04-22T20:09:28.270 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.270+0000 7f0c16f2e800 20 remove_watcher() i=7 2024-04-22T20:09:28.281 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-22T20:09:28.308 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:28.308 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:28.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.341+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.343+0000 7fe9d2525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.343+0000 7fe9d2525800 20 realm 2024-04-22T20:09:28.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.343+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.343+0000 7fe9d2525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.343+0000 7fe9d2525800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:28.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.343+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.343+0000 7fe9d2525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.343+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.344+0000 7fe9d2525800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:28.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.344+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.345+0000 7fe9d2525800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:28.345 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.345+0000 7fe9d2525800 20 searching for the correct realm 2024-04-22T20:09:28.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.347+0000 7fe9ceab8640 0 -- 172.21.15.136:0/3693373151 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fe9c0046b10 msgr2=0x7fe9c003ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:28.553 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.553+0000 7fe9cdab6640 0 -- 172.21.15.136:0/3693373151 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fe9c0055450 msgr2=0x7fe9c00558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:28.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:28.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.760+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:28.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 20 zone default found 2024-04-22T20:09:28.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 4 Realm: () 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 20 zonegroup default 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.761+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.762+0000 7fe9d2525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.762+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.762+0000 7fe9d2525800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.762+0000 7fe9d2525800 20 started sync module instance, tier type = 2024-04-22T20:09:28.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.762+0000 7fe9d2525800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:28.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.771+0000 7fe9d2525800 20 add_watcher() i=0 2024-04-22T20:09:28.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.771+0000 7fe9d2525800 20 add_watcher() i=1 2024-04-22T20:09:28.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.771+0000 7fe9d2525800 20 add_watcher() i=2 2024-04-22T20:09:28.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.771+0000 7fe9d2525800 20 add_watcher() i=3 2024-04-22T20:09:28.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.771+0000 7fe9d2525800 20 add_watcher() i=4 2024-04-22T20:09:28.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.771+0000 7fe9d2525800 20 add_watcher() i=5 2024-04-22T20:09:28.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.771+0000 7fe9d2525800 20 add_watcher() i=6 2024-04-22T20:09:28.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.772+0000 7fe9d2525800 20 add_watcher() i=7 2024-04-22T20:09:28.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.772+0000 7fe9d2525800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:28.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.774+0000 7fe884fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:28.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.775+0000 7fe9d2525800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:28.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.775+0000 7fe9d2525800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:28.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.775+0000 7fe9d2525800 5 note: GC not initialized 2024-04-22T20:09:28.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.775+0000 7fe7ff7fe640 20 reqs_thread_entry: start 2024-04-22T20:09:28.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.775+0000 7fe9d2525800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:28.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.775+0000 7fe9d2525800 20 Filter name: none 2024-04-22T20:09:28.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.775+0000 7fe7fd7fa640 20 reqs_thread_entry: start 2024-04-22T20:09:28.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.775+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.776 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.776+0000 7fe9d2525800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:28.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.776+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.776+0000 7fe9d2525800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:28.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.776+0000 7fe9d2525800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:28.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.776+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.777+0000 7fe9d2525800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:28.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.777+0000 7fe9d2525800 20 rados->read ofs=0 len=0 2024-04-22T20:09:28.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.777+0000 7fe9d2525800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:28.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.777+0000 7fe9d2525800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:28.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.778+0000 7fe884fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:28.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.781+0000 7fe9d2525800 20 remove_watcher() i=0 2024-04-22T20:09:28.781 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.781+0000 7fe9d2525800 2 removed watcher, disabling cache 2024-04-22T20:09:28.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.782+0000 7fe9d2525800 20 remove_watcher() i=1 2024-04-22T20:09:28.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.783+0000 7fe9d2525800 20 remove_watcher() i=2 2024-04-22T20:09:28.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.784+0000 7fe9d2525800 20 remove_watcher() i=3 2024-04-22T20:09:28.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.785+0000 7fe9d2525800 20 remove_watcher() i=4 2024-04-22T20:09:28.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.786+0000 7fe9d2525800 20 remove_watcher() i=5 2024-04-22T20:09:28.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.787+0000 7fe9d2525800 20 remove_watcher() i=6 2024-04-22T20:09:28.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:28.788+0000 7fe9d2525800 20 remove_watcher() i=7 2024-04-22T20:09:28.798 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:28.798 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:09:28.798 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-commit-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-22T20:09:28.798 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:28.800 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:28.800 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:28.821 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:28.821 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.044 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.044 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.108 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with ECANCELED injected at commit_target_layout 2024-04-22T20:09:29.108 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:29.109 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.109 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.151 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.152 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.207 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.207 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.239 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.240 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.274 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.274 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.308 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.308 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.344 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.344 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.378 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.378 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.412 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.413 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.448 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.448 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.482 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.482 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.517 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.517 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.551 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.551 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.586 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.586 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.622 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.622 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.656 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.656 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.691 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.691 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.727 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.727 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.761 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.761 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.796 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.796 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.833 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.833 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.868 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:29.868 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:29.934 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-22T20:09:29.962 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:29.962 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:29.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.996+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:29.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.997+0000 7f4be2531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:29.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.997+0000 7f4be2531800 20 realm 2024-04-22T20:09:29.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.997+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:29.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.997+0000 7f4be2531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:29.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.997+0000 7f4be2531800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:29.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.997+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:29.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.998+0000 7f4be2531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:29.998 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.998+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:29.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.998+0000 7f4be2531800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:29.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.998+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:29.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.999+0000 7f4be2531800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:29.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:29.999+0000 7f4be2531800 20 searching for the correct realm 2024-04-22T20:09:30.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.000+0000 7f4bde22f640 0 -- 172.21.15.136:0/3877722648 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x563fd358e1a0 msgr2=0x563fd3590660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:09:30.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.003+0000 7f4bde22f640 0 -- 172.21.15.136:0/3877722648 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x563fd3346f70 msgr2=0x7f4bd808baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.211+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 20 zone default found 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 4 Realm: () 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 20 zonegroup default 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.212+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.213+0000 7f4be2531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.213+0000 7f4be2531800 20 started sync module instance, tier type = 2024-04-22T20:09:30.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.213+0000 7f4be2531800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:30.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.221+0000 7f4be2531800 20 add_watcher() i=0 2024-04-22T20:09:30.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.221+0000 7f4be2531800 20 add_watcher() i=1 2024-04-22T20:09:30.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.221+0000 7f4be2531800 20 add_watcher() i=2 2024-04-22T20:09:30.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.221+0000 7f4be2531800 20 add_watcher() i=3 2024-04-22T20:09:30.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.221+0000 7f4be2531800 20 add_watcher() i=4 2024-04-22T20:09:30.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.221+0000 7f4be2531800 20 add_watcher() i=5 2024-04-22T20:09:30.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.221+0000 7f4be2531800 20 add_watcher() i=6 2024-04-22T20:09:30.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.222+0000 7f4be2531800 20 add_watcher() i=7 2024-04-22T20:09:30.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.222+0000 7f4be2531800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:30.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4a94fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:30.224 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4be2531800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:30.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4be2531800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:30.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4be2531800 5 note: GC not initialized 2024-04-22T20:09:30.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4a0f7fe640 20 reqs_thread_entry: start 2024-04-22T20:09:30.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4be2531800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:30.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4be2531800 20 Filter name: none 2024-04-22T20:09:30.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4a0d7fa640 20 reqs_thread_entry: start 2024-04-22T20:09:30.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.224+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.225+0000 7f4be2531800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:30.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.225+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.226+0000 7f4be2531800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:30.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.226+0000 7f4be2531800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:30.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.226+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.226+0000 7f4be2531800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:30.226 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.226+0000 7f4be2531800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.226+0000 7f4be2531800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:30.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.226+0000 7f4be2531800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:30.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.227+0000 7f4a94fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:30.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.229+0000 7f4be2531800 20 remove_watcher() i=0 2024-04-22T20:09:30.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.229+0000 7f4be2531800 2 removed watcher, disabling cache 2024-04-22T20:09:30.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.230+0000 7f4be2531800 20 remove_watcher() i=1 2024-04-22T20:09:30.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.231+0000 7f4be2531800 20 remove_watcher() i=2 2024-04-22T20:09:30.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.232+0000 7f4be2531800 20 remove_watcher() i=3 2024-04-22T20:09:30.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.233+0000 7f4be2531800 20 remove_watcher() i=4 2024-04-22T20:09:30.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.234+0000 7f4be2531800 20 remove_watcher() i=5 2024-04-22T20:09:30.235 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.235+0000 7f4be2531800 20 remove_watcher() i=6 2024-04-22T20:09:30.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.236+0000 7f4be2531800 20 remove_watcher() i=7 2024-04-22T20:09:30.245 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:30.245 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:09:30.245 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-commit-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:09:30.245 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:30.247 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout --inject-error-code 125 2024-04-22T20:09:30.282 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:30.282 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:30.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.316+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.317+0000 7f01f1a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.317+0000 7f01f1a10800 20 realm 2024-04-22T20:09:30.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.317+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.317+0000 7f01f1a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.317+0000 7f01f1a10800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:30.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.317+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.317+0000 7f01f1a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.317+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.318+0000 7f01f1a10800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:30.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.318+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.319+0000 7f01f1a10800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:30.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.319+0000 7f01f1a10800 20 searching for the correct realm 2024-04-22T20:09:30.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.320+0000 7f01ed95d640 0 -- 172.21.15.136:0/3431690171 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x56345ecf1bc0 msgr2=0x56345ecf3fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:09:30.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.324+0000 7f01ec95b640 0 -- 172.21.15.136:0/3431690171 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f01e0018070 msgr2=0x7f01e0057930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:30.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:30.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:30.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:30.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:30.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:30.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:30.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:30.330 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:30.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 zone default found 2024-04-22T20:09:30.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 4 Realm: () 2024-04-22T20:09:30.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 zonegroup default 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.329+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.330+0000 7f01f1a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.330+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.330+0000 7f01f1a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.330+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.330+0000 7f01f1a10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.330+0000 7f01f1a10800 20 started sync module instance, tier type = 2024-04-22T20:09:30.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.330+0000 7f01f1a10800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:30.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 20 add_watcher() i=0 2024-04-22T20:09:30.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 20 add_watcher() i=1 2024-04-22T20:09:30.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 20 add_watcher() i=2 2024-04-22T20:09:30.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 20 add_watcher() i=3 2024-04-22T20:09:30.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 20 add_watcher() i=4 2024-04-22T20:09:30.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 20 add_watcher() i=5 2024-04-22T20:09:30.339 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 20 add_watcher() i=6 2024-04-22T20:09:30.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 20 add_watcher() i=7 2024-04-22T20:09:30.340 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.339+0000 7f01f1a10800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:30.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.341+0000 7f00a37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:30.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.341+0000 7f01f1a10800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:30.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.341+0000 7f01f1a10800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:30.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.341+0000 7f01f1a10800 5 note: GC not initialized 2024-04-22T20:09:30.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f00a27e4640 20 reqs_thread_entry: start 2024-04-22T20:09:30.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:30.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 20 Filter name: none 2024-04-22T20:09:30.342 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f0013fff640 20 reqs_thread_entry: start 2024-04-22T20:09:30.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-04-22T20:09:30.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.343 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 : miss 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.342+0000 7f01f1a10800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.343+0000 7f01f1a10800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.343+0000 7f01f1a10800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 info.flags=0x17 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.343+0000 7f01f1a10800 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 to cache LRU end 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.343+0000 7f01f1a10800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.343+0000 7f01f1a10800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:30.344 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.343+0000 7f01f1a10800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 2024-04-22T20:09:30.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.347+0000 7f01f1a10800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 info.flags=0x17 2024-04-22T20:09:30.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.347+0000 7f01f1a10800 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 to cache LRU end 2024-04-22T20:09:30.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.347+0000 7f01f1a10800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:30.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.347+0000 7f01f1a10800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25, ofs0, ns] 2024-04-22T20:09:30.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.347+0000 7f00bffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562468 cookie 94782928927440 notifier 6768 bl.length()=923 2024-04-22T20:09:30.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.347+0000 7f00bffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 info.flags=0x17 2024-04-22T20:09:30.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.347+0000 7f00bffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 to cache LRU end 2024-04-22T20:09:30.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.347+0000 7f00bffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:30.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.349+0000 7f01f1a10800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25]): orig_sources= new_sources= 2024-04-22T20:09:30.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.349+0000 7f01f1a10800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25]): potential sources added=[] removed=[] 2024-04-22T20:09:30.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.349+0000 7f01f1a10800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25]): orig_dests= new_dests= 2024-04-22T20:09:30.349 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.349+0000 7f01f1a10800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25]): potential dests added=[] removed=[] 2024-04-22T20:09:30.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.350+0000 7f01f1a10800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25.1.0 2024-04-22T20:09:30.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.350+0000 7f01f1a10800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25.1.0 2024-04-22T20:09:30.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.350+0000 7f01f1a10800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25.1.1 2024-04-22T20:09:30.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.350+0000 7f01f1a10800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25.1.1 2024-04-22T20:09:30.350 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.350+0000 7f01f1a10800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25.0 2024-04-22T20:09:30.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.352+0000 7f01f1a10800 -1 Injecting error=-125 at location=commit_target_layout 2024-04-22T20:09:30.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.352+0000 7f01f1a10800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 : hit (requested=0x17, cached=0x17) 2024-04-22T20:09:30.352 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.352+0000 7f01f1a10800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 2024-04-22T20:09:30.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.353+0000 7f01f1a10800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 info.flags=0x17 2024-04-22T20:09:30.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.353+0000 7f01f1a10800 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 to cache LRU end 2024-04-22T20:09:30.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.353+0000 7f01f1a10800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:30.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.353+0000 7f01f1a10800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25, ofs0, ns] 2024-04-22T20:09:30.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.353+0000 7f00d0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562469 cookie 94782928927440 notifier 6768 bl.length()=891 2024-04-22T20:09:30.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.353+0000 7f00d0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 info.flags=0x17 2024-04-22T20:09:30.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.353+0000 7f00d0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 to cache LRU end 2024-04-22T20:09:30.354 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.353+0000 7f00d0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:30.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.355+0000 7f01f1a10800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25]): orig_sources= new_sources= 2024-04-22T20:09:30.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.355+0000 7f01f1a10800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25]): potential sources added=[] removed=[] 2024-04-22T20:09:30.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.355+0000 7f01f1a10800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25]): orig_dests= new_dests= 2024-04-22T20:09:30.355 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.355+0000 7f01f1a10800 20 handle_bi_update(): bucket=:error-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25]): potential dests added=[] removed=[] 2024-04-22T20:09:30.356 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.356+0000 7f01f1a10800 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-04-22T20:09:30.358 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.358+0000 7f00a37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:30.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.360+0000 7f01f1a10800 20 remove_watcher() i=0 2024-04-22T20:09:30.360 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.360+0000 7f01f1a10800 2 removed watcher, disabling cache 2024-04-22T20:09:30.361 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.361+0000 7f01f1a10800 20 remove_watcher() i=1 2024-04-22T20:09:30.362 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.362+0000 7f01f1a10800 20 remove_watcher() i=2 2024-04-22T20:09:30.363 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.363+0000 7f01f1a10800 20 remove_watcher() i=3 2024-04-22T20:09:30.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.364+0000 7f01f1a10800 20 remove_watcher() i=4 2024-04-22T20:09:30.364 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.365+0000 7f01f1a10800 20 remove_watcher() i=5 2024-04-22T20:09:30.365 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.365+0000 7f01f1a10800 20 remove_watcher() i=6 2024-04-22T20:09:30.366 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.366+0000 7f01f1a10800 20 remove_watcher() i=7 2024-04-22T20:09:30.377 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-04-22T20:09:30.405 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:30.405 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:30.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.438+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.440+0000 7f5960c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.440+0000 7f5960c87800 20 realm 2024-04-22T20:09:30.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.440+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.440+0000 7f5960c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.440+0000 7f5960c87800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:30.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.440+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.440+0000 7f5960c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.440+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.441+0000 7f5960c87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:30.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.441+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.442+0000 7f5960c87800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:30.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.442+0000 7f5960c87800 20 searching for the correct realm 2024-04-22T20:09:30.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.444+0000 7f595ef8e640 0 -- 172.21.15.136:0/3883696009 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f5958083c20 msgr2=0x7f5958058980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:30.649 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.649+0000 7f595df8c640 0 -- 172.21.15.136:0/3883696009 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f5958058080 msgr2=0x7f5958014720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:30.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.854+0000 7f5960c87800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:30.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.854+0000 7f5960c87800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.854+0000 7f5960c87800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.854+0000 7f5960c87800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.854+0000 7f5960c87800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.854+0000 7f5960c87800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.854+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:30.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.857 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 20 zone default found 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 4 Realm: () 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.855+0000 7f5960c87800 20 zonegroup default 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.856+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.856+0000 7f5960c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.856+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.856+0000 7f5960c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.856+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.856+0000 7f5960c87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:30.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.856+0000 7f5960c87800 20 started sync module instance, tier type = 2024-04-22T20:09:30.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.856+0000 7f5960c87800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:30.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.866+0000 7f5960c87800 20 add_watcher() i=0 2024-04-22T20:09:30.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.866+0000 7f5960c87800 20 add_watcher() i=1 2024-04-22T20:09:30.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.866+0000 7f5960c87800 20 add_watcher() i=2 2024-04-22T20:09:30.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.866+0000 7f5960c87800 20 add_watcher() i=3 2024-04-22T20:09:30.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.866+0000 7f5960c87800 20 add_watcher() i=4 2024-04-22T20:09:30.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.866+0000 7f5960c87800 20 add_watcher() i=5 2024-04-22T20:09:30.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.866+0000 7f5960c87800 20 add_watcher() i=6 2024-04-22T20:09:30.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.867+0000 7f5960c87800 20 add_watcher() i=7 2024-04-22T20:09:30.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.867+0000 7f5960c87800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:30.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f5814fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:30.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f5960c87800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:30.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f5960c87800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:30.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f5960c87800 5 note: GC not initialized 2024-04-22T20:09:30.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f578f7fe640 20 reqs_thread_entry: start 2024-04-22T20:09:30.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f5960c87800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:30.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f5960c87800 20 Filter name: none 2024-04-22T20:09:30.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f578d7fa640 20 reqs_thread_entry: start 2024-04-22T20:09:30.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.869+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.870+0000 7f5960c87800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:30.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.870+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.870+0000 7f5960c87800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:30.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.870+0000 7f5960c87800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:30.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.870+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.871+0000 7f5960c87800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:30.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.871+0000 7f5960c87800 20 rados->read ofs=0 len=0 2024-04-22T20:09:30.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.871+0000 7f5960c87800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:30.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.871+0000 7f5960c87800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:30.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.872+0000 7f5814fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:30.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.874+0000 7f5960c87800 20 remove_watcher() i=0 2024-04-22T20:09:30.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.874+0000 7f5960c87800 2 removed watcher, disabling cache 2024-04-22T20:09:30.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.875+0000 7f5960c87800 20 remove_watcher() i=1 2024-04-22T20:09:30.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.877+0000 7f5960c87800 20 remove_watcher() i=2 2024-04-22T20:09:30.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.878+0000 7f5960c87800 20 remove_watcher() i=3 2024-04-22T20:09:30.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.878+0000 7f5960c87800 20 remove_watcher() i=4 2024-04-22T20:09:30.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.879+0000 7f5960c87800 20 remove_watcher() i=5 2024-04-22T20:09:30.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.880+0000 7f5960c87800 20 remove_watcher() i=6 2024-04-22T20:09:30.881 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:30.881+0000 7f5960c87800 20 remove_watcher() i=7 2024-04-22T20:09:30.891 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:30.891 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:09:30.891 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-commit-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25 num_objects: 20 size_kb: 1 num_shards: 2 2024-04-22T20:09:30.891 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:30.893 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:30.893 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:30.913 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:30.913 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.141 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.141 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.204 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with abort at commit_target_layout 2024-04-22T20:09:31.205 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:31.206 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.206 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.246 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.246 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.302 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.302 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.336 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.336 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.372 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.372 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.408 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.408 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.443 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.443 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.478 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.478 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.512 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.512 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.546 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.547 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.580 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.581 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.615 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.615 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.649 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.649 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.683 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.684 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.719 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.719 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.753 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.753 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.786 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.786 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.821 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.821 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.855 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.855 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.890 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.890 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.923 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.923 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.958 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:31.958 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:09:31.995 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-04-22T20:09:32.022 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:32.022 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:32.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.054+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.056+0000 7fe184e3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.056+0000 7fe184e3f800 20 realm 2024-04-22T20:09:32.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.056+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.056+0000 7fe184e3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.056+0000 7fe184e3f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:32.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.056+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.056+0000 7fe184e3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.056+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.057+0000 7fe184e3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:32.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.057+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.057+0000 7fe184e3f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:32.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.057+0000 7fe184e3f800 20 searching for the correct realm 2024-04-22T20:09:32.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.059+0000 7fe18313f640 0 -- 172.21.15.136:0/3304498085 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fe17c0801e0 msgr2=0x7fe17c057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:32.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.264+0000 7fe17bfff640 0 -- 172.21.15.136:0/3304498085 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fe17c0827c0 msgr2=0x7fe17c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:32.471 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 zone default found 2024-04-22T20:09:32.472 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 4 Realm: () 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 zonegroup default 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.470+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.471+0000 7fe184e3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.471+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.471+0000 7fe184e3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.471+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.471+0000 7fe184e3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.471+0000 7fe184e3f800 20 started sync module instance, tier type = 2024-04-22T20:09:32.473 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.471+0000 7fe184e3f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:32.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.480+0000 7fe184e3f800 20 add_watcher() i=0 2024-04-22T20:09:32.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.480+0000 7fe184e3f800 20 add_watcher() i=1 2024-04-22T20:09:32.480 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.480+0000 7fe184e3f800 20 add_watcher() i=2 2024-04-22T20:09:32.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.480+0000 7fe184e3f800 20 add_watcher() i=3 2024-04-22T20:09:32.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.480+0000 7fe184e3f800 20 add_watcher() i=4 2024-04-22T20:09:32.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.480+0000 7fe184e3f800 20 add_watcher() i=5 2024-04-22T20:09:32.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.480+0000 7fe184e3f800 20 add_watcher() i=6 2024-04-22T20:09:32.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.481+0000 7fe184e3f800 20 add_watcher() i=7 2024-04-22T20:09:32.481 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.481+0000 7fe184e3f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:32.483 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.482+0000 7fe0397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:32.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.483+0000 7fe184e3f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:32.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.483+0000 7fe184e3f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:32.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.483+0000 7fe184e3f800 5 note: GC not initialized 2024-04-22T20:09:32.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.483+0000 7fdfb3fff640 20 reqs_thread_entry: start 2024-04-22T20:09:32.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.483+0000 7fe184e3f800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:32.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.483+0000 7fe184e3f800 20 Filter name: none 2024-04-22T20:09:32.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.483+0000 7fdfb1ffb640 20 reqs_thread_entry: start 2024-04-22T20:09:32.484 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.483+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.484+0000 7fe184e3f800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:32.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.484+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.485+0000 7fe184e3f800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:32.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.485+0000 7fe184e3f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:32.485 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.485+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.485+0000 7fe184e3f800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:32.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.485+0000 7fe184e3f800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.485+0000 7fe184e3f800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:32.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.485+0000 7fe184e3f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:32.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.487+0000 7fe0397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:32.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.490+0000 7fe184e3f800 20 remove_watcher() i=0 2024-04-22T20:09:32.490 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.490+0000 7fe184e3f800 2 removed watcher, disabling cache 2024-04-22T20:09:32.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.490+0000 7fe184e3f800 20 remove_watcher() i=1 2024-04-22T20:09:32.491 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.492+0000 7fe184e3f800 20 remove_watcher() i=2 2024-04-22T20:09:32.492 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.492+0000 7fe184e3f800 20 remove_watcher() i=3 2024-04-22T20:09:32.493 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.493+0000 7fe184e3f800 20 remove_watcher() i=4 2024-04-22T20:09:32.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.494+0000 7fe184e3f800 20 remove_watcher() i=5 2024-04-22T20:09:32.495 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.495+0000 7fe184e3f800 20 remove_watcher() i=6 2024-04-22T20:09:32.497 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.497+0000 7fe184e3f800 20 remove_watcher() i=7 2024-04-22T20:09:32.506 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:32.506 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:09:32.506 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-commit-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:09:32.506 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:32.508 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at commit_target_layout 2024-04-22T20:09:32.540 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:32.540 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:32.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.573+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.574+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.574+0000 7fc4c8938800 20 realm 2024-04-22T20:09:32.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.574+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.575+0000 7fc4c8938800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:32.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:32.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:32.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.576+0000 7fc4c8938800 20 searching for the correct realm 2024-04-22T20:09:32.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.579+0000 7fc4c4e3b640 0 -- 172.21.15.136:0/2984211145 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fc4bc03eb60 msgr2=0x7fc4bc037410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:32.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.784+0000 7fc4c3e39640 0 -- 172.21.15.136:0/2984211145 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fc4bc0283e0 msgr2=0x7fc4bc028860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 zone default found 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 4 Realm: () 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 zonegroup default 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:32.993 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 started sync module instance, tier type = 2024-04-22T20:09:32.994 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:32.991+0000 7fc4c8938800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:33.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=0 2024-04-22T20:09:33.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=1 2024-04-22T20:09:33.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=2 2024-04-22T20:09:33.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=3 2024-04-22T20:09:33.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=4 2024-04-22T20:09:33.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=5 2024-04-22T20:09:33.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=6 2024-04-22T20:09:33.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.001+0000 7fc4c8938800 20 add_watcher() i=7 2024-04-22T20:09:33.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.001+0000 7fc4c8938800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc37afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc4c8938800 5 note: GC not initialized 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc379fdb640 20 reqs_thread_entry: start 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc4c8938800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc4c8938800 20 Filter name: none 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc2eb7fe640 20 reqs_thread_entry: start 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc4c8938800 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-04-22T20:09:33.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:33.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-04-22T20:09:33.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-04-22T20:09:33.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 : miss 2024-04-22T20:09:33.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:33.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:09:33.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.004+0000 7fc4c8938800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 2024-04-22T20:09:33.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.008+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.008+0000 7fc4c8938800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.008+0000 7fc4c8938800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.008 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.008+0000 7fc4c8938800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26, ofs0, ns] 2024-04-22T20:09:33.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562390 cookie 94306211823728 notifier 6825 bl.length()=923 2024-04-22T20:09:33.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_sources= new_sources= 2024-04-22T20:09:33.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential sources added=[] removed=[] 2024-04-22T20:09:33.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_dests= new_dests= 2024-04-22T20:09:33.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential dests added=[] removed=[] 2024-04-22T20:09:33.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.011+0000 7fc4c8938800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.0 2024-04-22T20:09:33.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.0 2024-04-22T20:09:33.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.011+0000 7fc4c8938800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.1 2024-04-22T20:09:33.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.1 2024-04-22T20:09:33.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.0 2024-04-22T20:09:33.014 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fc4c8938800 time 2024-04-22T20:09:33.015425+0000 2024-04-22T20:09:33.014 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:09:33.014 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:09:33.014 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc4c9f81f5c] 2024-04-22T20:09:33.014 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc4c9f8211a] 2024-04-22T20:09:33.014 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c55eb9cdc1] 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x5c7) [0x55c55ebab127] 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fc4c8e29590] 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.014+0000 7fc4c8938800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fc4c8938800 time 2024-04-22T20:09:33.015425+0000 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.015 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc4c9f81f5c] 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc4c9f8211a] 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c55eb9cdc1] 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x5c7) [0x55c55ebab127] 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fc4c8e29590] 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.016 INFO:tasks.workunit.client.0.smithi136.stderr:*** Caught signal (Aborted) ** 2024-04-22T20:09:33.017 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fc4c8938800 thread_name:radosgw-admin 2024-04-22T20:09:33.032 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:09:33.032 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fc4c8e3e6f0] 2024-04-22T20:09:33.032 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fc4c8e8b94c] 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc4c9f81fb6] 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc4c9f8211a] 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c55eb9cdc1] 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x5c7) [0x55c55ebab127] 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fc4c8e29590] 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.015+0000 7fc4c8938800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fc4c8938800 thread_name:radosgw-admin 2024-04-22T20:09:33.033 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fc4c8e3e6f0] 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fc4c8e8b94c] 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc4c9f81fb6] 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc4c9f8211a] 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c55eb9cdc1] 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x5c7) [0x55c55ebab127] 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fc4c8e29590] 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:09:33.034 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr:--- begin dump of recent events --- 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr: -499> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr: -498> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr: -497> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr: -496> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr: -495> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: dump: 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.035 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: -494> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6449 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: -493> 2024-04-22T20:09:32.558+0000 7fc4c8938800 10 monclient: get_monmap_and_config success 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: -492> 2024-04-22T20:09:32.558+0000 7fc4c8938800 4 set_mon_vals no callback set 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: -491> 2024-04-22T20:09:32.558+0000 7fc4c8938800 10 monclient: shutdown 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: -490> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 asok(0x55c5600f1d00) unregister_commands rotate-key 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: -489> 2024-04-22T20:09:32.558+0000 7fc4c8938800 10 monclient: build_initial_monmap 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: -488> 2024-04-22T20:09:32.558+0000 7fc4c8938800 1 build_initial for_mkfs: 0 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr: -487> 2024-04-22T20:09:32.558+0000 7fc4c8938800 10 monclient: monmap: 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:09:33.036 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -486> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding auth protocol: cephx 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -485> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding auth protocol: cephx 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -484> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding auth protocol: cephx 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -483> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -482> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -481> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -480> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -479> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.037 INFO:tasks.workunit.client.0.smithi136.stderr: -478> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -477> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -476> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -475> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -474> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -473> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -472> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -471> 2024-04-22T20:09:32.559+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -470> 2024-04-22T20:09:32.559+0000 7fc4c8938800 1 librados: starting msgr at 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -469> 2024-04-22T20:09:32.559+0000 7fc4c8938800 1 librados: starting objecter 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -468> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command objecter_requests hook 0x55c560314740 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -467> 2024-04-22T20:09:32.559+0000 7fc4c8938800 1 librados: setting wanted keys 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -466> 2024-04-22T20:09:32.559+0000 7fc4c8938800 1 librados: calling monclient init 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -465> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient: init 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -464> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding auth protocol: cephx 2024-04-22T20:09:33.038 INFO:tasks.workunit.client.0.smithi136.stderr: -463> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding auth protocol: cephx 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -462> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding auth protocol: cephx 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -461> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -460> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -459> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -458> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -457> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -456> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -455> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -454> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -453> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -452> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -451> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -450> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.039 INFO:tasks.workunit.client.0.smithi136.stderr: -449> 2024-04-22T20:09:32.559+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -448> 2024-04-22T20:09:32.559+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -447> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command rotate-key hook 0x55c560311ee8 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -446> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient: _reopen_session rank -1 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -445> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient: _add_conns ranks=[2,1,0] 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -444> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-c con 0x55c5601c9480 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -443> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-b con 0x55c560253160 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -442> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-a con 0x55c560259680 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -441> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -440> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -439> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -438> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): _renew_subs 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -437> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): authenticate will time out at 1961.911744s 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -436> 2024-04-22T20:09:32.559+0000 7fc4c463a640 10 monclient(hunting): get_auth_request con 0x55c5601c9480 auth_method 0 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -435> 2024-04-22T20:09:32.559+0000 7fc4c463a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.040 INFO:tasks.workunit.client.0.smithi136.stderr: -434> 2024-04-22T20:09:32.559+0000 7fc4c463a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -433> 2024-04-22T20:09:32.559+0000 7fc4c463a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -432> 2024-04-22T20:09:32.559+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request con 0x55c560259680 auth_method 0 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -431> 2024-04-22T20:09:32.559+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -430> 2024-04-22T20:09:32.559+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -429> 2024-04-22T20:09:32.559+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -428> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -427> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -426> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -425> 2024-04-22T20:09:32.560+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -424> 2024-04-22T20:09:32.560+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -423> 2024-04-22T20:09:32.560+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -422> 2024-04-22T20:09:32.560+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request con 0x55c560253160 auth_method 0 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -421> 2024-04-22T20:09:32.560+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.041 INFO:tasks.workunit.client.0.smithi136.stderr: -420> 2024-04-22T20:09:32.560+0000 7fc4c3e39640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr: -419> 2024-04-22T20:09:32.560+0000 7fc4c3e39640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr: -418> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient(hunting): handle_auth_done global_id 6452 payload 824 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr: -417> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient: _finish_hunting 0 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr: -416> 2024-04-22T20:09:32.560+0000 7fc4c463a640 1 monclient: found mon.noname-c 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr: -415> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr: -414> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr: -413> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr: -412> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: dump: 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.042 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr: -411> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: _finish_auth 0 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr: -410> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr: -409> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr: -408> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr: -407> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr: -406> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: dump: 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.043 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -405> 2024-04-22T20:09:32.560+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6452 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -404> 2024-04-22T20:09:32.560+0000 7fc3b260c640 4 set_mon_vals no callback set 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -403> 2024-04-22T20:09:32.560+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -402> 2024-04-22T20:09:32.560+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -401> 2024-04-22T20:09:32.560+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -400> 2024-04-22T20:09:32.560+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -399> 2024-04-22T20:09:32.560+0000 7fc4c8938800 1 librados: init done 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -398> 2024-04-22T20:09:32.562+0000 7fc3b0e09640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -397> 2024-04-22T20:09:32.562+0000 7fc3b0e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.044 INFO:tasks.workunit.client.0.smithi136.stderr: -396> 2024-04-22T20:09:32.562+0000 7fc3b0e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr: -395> 2024-04-22T20:09:32.562+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x7fc3ac06b2d0 auth_method 0 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr: -394> 2024-04-22T20:09:32.562+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x55c56033d910 auth_method 0 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr: -393> 2024-04-22T20:09:32.563+0000 7fc4c463a640 10 monclient: get_auth_request con 0x55c56035e0b0 auth_method 0 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr: -392> 2024-04-22T20:09:32.563+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x55c560362330 auth_method 0 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr: -391> 2024-04-22T20:09:32.564+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x55c560361930 auth_method 0 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr: -390> 2024-04-22T20:09:32.565+0000 7fc4c8938800 10 monclient: build_initial_monmap 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr: -389> 2024-04-22T20:09:32.565+0000 7fc4c8938800 1 build_initial for_mkfs: 0 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr: -388> 2024-04-22T20:09:32.565+0000 7fc4c8938800 10 monclient: monmap: 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.045 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -387> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding auth protocol: cephx 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -386> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding auth protocol: cephx 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -385> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding auth protocol: cephx 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -384> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -383> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -382> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -381> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -380> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -379> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.046 INFO:tasks.workunit.client.0.smithi136.stderr: -378> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -377> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -376> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -375> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -374> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -373> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -372> 2024-04-22T20:09:32.565+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -371> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command objecter_requests cmddesc objecter_requests hook 0x55c56036fa70 EEXIST 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -370> 2024-04-22T20:09:32.565+0000 7fc4c8938800 10 monclient: init 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -369> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding auth protocol: cephx 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -368> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding auth protocol: cephx 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -367> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding auth protocol: cephx 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -366> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -365> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -364> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.047 INFO:tasks.workunit.client.0.smithi136.stderr: -363> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -362> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -361> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -360> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -359> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -358> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -357> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -356> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -355> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -354> 2024-04-22T20:09:32.566+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -353> 2024-04-22T20:09:32.566+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -352> 2024-04-22T20:09:32.566+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command rotate-key cmddesc rotate-key hook 0x55c5603694c0 EEXIST 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -351> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient: _reopen_session rank -1 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -350> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -349> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-a con 0x55c5603947d0 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:09:33.048 INFO:tasks.workunit.client.0.smithi136.stderr: -348> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-c con 0x55c560394bb0 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -347> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-b con 0x55c56038fc70 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -346> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -345> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -344> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -343> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): _renew_subs 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -342> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): authenticate will time out at 1961.918336s 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -341> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request con 0x55c5603947d0 auth_method 0 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -340> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -339> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -338> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -337> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request con 0x55c560394bb0 auth_method 0 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -336> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -335> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -334> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.049 INFO:tasks.workunit.client.0.smithi136.stderr: -333> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -332> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -331> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -330> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -329> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -328> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -327> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_done global_id 6819 payload 824 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -326> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient: _finish_hunting 0 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -325> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 1 monclient: found mon.noname-a 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -324> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -323> 2024-04-22T20:09:32.566+0000 7fc4c463a640 10 monclient: get_auth_request con 0x55c56038fc70 auth_method 0 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -322> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -321> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr: -320> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: dump: 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.050 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr: -319> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: _finish_auth 0 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr: -318> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr: -317> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr: -316> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr: -315> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:09:33.051 INFO:tasks.workunit.client.0.smithi136.stderr: -314> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: dump: 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr: -313> 2024-04-22T20:09:32.567+0000 7fc4c563c640 4 set_mon_vals no callback set 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr: -312> 2024-04-22T20:09:32.567+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6819 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr: -311> 2024-04-22T20:09:32.567+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr: -310> 2024-04-22T20:09:32.567+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.052 INFO:tasks.workunit.client.0.smithi136.stderr: -309> 2024-04-22T20:09:32.567+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr: -308> 2024-04-22T20:09:32.567+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr: -307> 2024-04-22T20:09:32.568+0000 7fc3a9ffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr: -306> 2024-04-22T20:09:32.568+0000 7fc3a9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr: -305> 2024-04-22T20:09:32.568+0000 7fc3a9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr: -304> 2024-04-22T20:09:32.568+0000 7fc4c8938800 10 monclient: get_monmap_and_config 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr: -303> 2024-04-22T20:09:32.568+0000 7fc4c8938800 10 monclient: build_initial_monmap 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr: -302> 2024-04-22T20:09:32.568+0000 7fc4c8938800 1 build_initial for_mkfs: 0 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr: -301> 2024-04-22T20:09:32.568+0000 7fc4c8938800 10 monclient: monmap: 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:09:33.053 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -300> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -299> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -298> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -297> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -296> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -295> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -294> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -293> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.054 INFO:tasks.workunit.client.0.smithi136.stderr: -292> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -291> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -290> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -289> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -288> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -287> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -286> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -285> 2024-04-22T20:09:32.569+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x7fc39c06b1e0 auth_method 0 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -284> 2024-04-22T20:09:32.569+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -283> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient: init 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -282> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding auth protocol: cephx 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -281> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding auth protocol: cephx 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -280> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding auth protocol: cephx 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -279> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -278> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.055 INFO:tasks.workunit.client.0.smithi136.stderr: -277> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -276> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -275> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -274> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -273> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -272> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -271> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -270> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -269> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -268> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -267> 2024-04-22T20:09:32.569+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -266> 2024-04-22T20:09:32.569+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -265> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command rotate-key cmddesc rotate-key hook 0x7ffcf6b2dbc8 EEXIST 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -264> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient: _reopen_session rank -1 2024-04-22T20:09:33.056 INFO:tasks.workunit.client.0.smithi136.stderr: -263> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient: _add_conns ranks=[0,1,2] 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -262> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-a con 0x55c5603e6e10 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -261> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-b con 0x55c5603eba70 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -260> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-c con 0x55c5603ec780 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -259> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -258> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -257> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -256> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): _renew_subs 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -255> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): authenticate will time out at 1961.921719s 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -254> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request con 0x55c5603e6e10 auth_method 0 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -253> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -252> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -251> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -250> 2024-04-22T20:09:32.569+0000 7fc4c463a640 10 monclient(hunting): get_auth_request con 0x55c5603ec780 auth_method 0 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -249> 2024-04-22T20:09:32.569+0000 7fc4c463a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -248> 2024-04-22T20:09:32.569+0000 7fc4c463a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.057 INFO:tasks.workunit.client.0.smithi136.stderr: -247> 2024-04-22T20:09:32.569+0000 7fc4c463a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -246> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -245> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -244> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -243> 2024-04-22T20:09:32.570+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -242> 2024-04-22T20:09:32.570+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -241> 2024-04-22T20:09:32.570+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -240> 2024-04-22T20:09:32.570+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request con 0x55c5603eba70 auth_method 0 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -239> 2024-04-22T20:09:32.570+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -238> 2024-04-22T20:09:32.570+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -237> 2024-04-22T20:09:32.570+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -236> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_done global_id 6822 payload 274 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -235> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 10 monclient: _finish_hunting 0 2024-04-22T20:09:33.058 INFO:tasks.workunit.client.0.smithi136.stderr: -234> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 1 monclient: found mon.noname-a 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr: -233> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr: -232> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr: -231> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr: -230> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: dump: 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.059 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr: -229> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: _finish_auth 0 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr: -228> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr: -227> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr: -226> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr: -225> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr: -224> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: dump: 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.060 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr: -223> 2024-04-22T20:09:32.570+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6822 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr: -222> 2024-04-22T20:09:32.570+0000 7fc4c8938800 10 monclient: get_monmap_and_config success 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr: -221> 2024-04-22T20:09:32.570+0000 7fc4c8938800 4 set_mon_vals no callback set 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr: -220> 2024-04-22T20:09:32.570+0000 7fc4c8938800 10 monclient: shutdown 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr: -219> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: build_initial_monmap 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr: -218> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 build_initial for_mkfs: 0 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr: -217> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: monmap: 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.061 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -216> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -215> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -214> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -213> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -212> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -211> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -210> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -209> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -208> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -207> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.062 INFO:tasks.workunit.client.0.smithi136.stderr: -206> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -205> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -204> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -203> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -202> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -201> 2024-04-22T20:09:32.571+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -200> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 librados: starting msgr at 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -199> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 librados: starting objecter 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -198> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command objecter_requests cmddesc objecter_requests hook 0x55c5603aa2b0 EEXIST 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -197> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 librados: setting wanted keys 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -196> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 librados: calling monclient init 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -195> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: init 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -194> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding auth protocol: cephx 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -193> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding auth protocol: cephx 2024-04-22T20:09:33.063 INFO:tasks.workunit.client.0.smithi136.stderr: -192> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding auth protocol: cephx 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -191> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -190> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -189> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -188> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -187> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -186> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -185> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -184> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -183> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -182> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -181> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -180> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -179> 2024-04-22T20:09:32.571+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.064 INFO:tasks.workunit.client.0.smithi136.stderr: -178> 2024-04-22T20:09:32.571+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -177> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command rotate-key cmddesc rotate-key hook 0x55c5603b1638 EEXIST 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -176> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: _reopen_session rank -1 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -175> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: _add_conns ranks=[0,1,2] 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -174> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-a con 0x55c5603e6e10 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -173> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-b con 0x55c5603eba70 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -172> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-c con 0x55c5603ec780 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -171> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -170> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -169> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -168> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): _renew_subs 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -167> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): authenticate will time out at 1961.923809s 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -166> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request con 0x55c5603e6e10 auth_method 0 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -165> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.065 INFO:tasks.workunit.client.0.smithi136.stderr: -164> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -163> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -162> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): get_auth_request con 0x55c5603ec780 auth_method 0 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -161> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -160> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -159> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -158> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -157> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -156> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -155> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -154> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -153> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -152> 2024-04-22T20:09:32.572+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request con 0x55c5603eba70 auth_method 0 2024-04-22T20:09:33.066 INFO:tasks.workunit.client.0.smithi136.stderr: -151> 2024-04-22T20:09:32.572+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -150> 2024-04-22T20:09:32.572+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -149> 2024-04-22T20:09:32.572+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -148> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_done global_id 6825 payload 824 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -147> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient: _finish_hunting 0 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -146> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 1 monclient: found mon.noname-a 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -145> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -144> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -143> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr: -142> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: dump: 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.067 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr: -141> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: _finish_auth 0 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr: -140> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr: -139> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr: -138> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr: -137> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr: -136> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: dump: 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.068 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: -135> 2024-04-22T20:09:32.572+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6825 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: -134> 2024-04-22T20:09:32.572+0000 7fc39b7fe640 4 set_mon_vals no callback set 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: -133> 2024-04-22T20:09:32.572+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: -132> 2024-04-22T20:09:32.572+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: -131> 2024-04-22T20:09:32.572+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: -130> 2024-04-22T20:09:32.572+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: -129> 2024-04-22T20:09:32.572+0000 7fc4c8938800 1 librados: init done 2024-04-22T20:09:33.069 INFO:tasks.workunit.client.0.smithi136.stderr: -128> 2024-04-22T20:09:32.572+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cr dump hook 0x55c5603e6048 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -127> 2024-04-22T20:09:32.573+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -126> 2024-04-22T20:09:32.573+0000 7fc399ffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -125> 2024-04-22T20:09:32.573+0000 7fc399ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -124> 2024-04-22T20:09:32.573+0000 7fc399ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -123> 2024-04-22T20:09:32.574+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x7fc38c06b2d0 auth_method 0 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -122> 2024-04-22T20:09:32.574+0000 7fc4c463a640 10 monclient: get_auth_request con 0x55c560400e30 auth_method 0 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -121> 2024-04-22T20:09:32.574+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -120> 2024-04-22T20:09:32.574+0000 7fc4c8938800 20 realm 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -119> 2024-04-22T20:09:32.574+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -118> 2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -117> 2024-04-22T20:09:32.575+0000 7fc4c8938800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -116> 2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -115> 2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.070 INFO:tasks.workunit.client.0.smithi136.stderr: -114> 2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -113> 2024-04-22T20:09:32.575+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x55c5604050a0 auth_method 0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -112> 2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -111> 2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -110> 2024-04-22T20:09:32.576+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x55c560408900 auth_method 0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -109> 2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -108> 2024-04-22T20:09:32.576+0000 7fc4c8938800 20 searching for the correct realm 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -107> 2024-04-22T20:09:32.577+0000 7fc4c463a640 10 monclient: get_auth_request con 0x55c56040d6e0 auth_method 0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -106> 2024-04-22T20:09:32.577+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x7fc4ac09beb0 auth_method 0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -105> 2024-04-22T20:09:32.578+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x7fc4bc03eb60 auth_method 0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -104> 2024-04-22T20:09:32.579+0000 7fc4c4e3b640 0 -- 172.21.15.136:0/2984211145 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fc4bc03eb60 msgr2=0x7fc4bc037410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -103> 2024-04-22T20:09:32.780+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x7fc4bc03eb60 auth_method 0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -102> 2024-04-22T20:09:32.782+0000 7fc4c463a640 10 monclient: get_auth_request con 0x7fc4bc038bb0 auth_method 0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -101> 2024-04-22T20:09:32.784+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x7fc4bc0283e0 auth_method 0 2024-04-22T20:09:33.071 INFO:tasks.workunit.client.0.smithi136.stderr: -100> 2024-04-22T20:09:32.784+0000 7fc4c3e39640 0 -- 172.21.15.136:0/2984211145 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fc4bc0283e0 msgr2=0x7fc4bc028860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -99> 2024-04-22T20:09:32.985+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x7fc4bc0283e0 auth_method 0 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -98> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -97> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -96> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -95> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -94> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -93> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -92> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -91> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -90> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -89> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -88> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -87> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:33.072 INFO:tasks.workunit.client.0.smithi136.stderr: -86> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 zone default found 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -85> 2024-04-22T20:09:32.991+0000 7fc4c8938800 4 Realm: () 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -84> 2024-04-22T20:09:32.991+0000 7fc4c8938800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -83> 2024-04-22T20:09:32.991+0000 7fc4c8938800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -82> 2024-04-22T20:09:32.991+0000 7fc4c8938800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -81> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 zonegroup default 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -80> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -79> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -78> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -77> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -76> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -75> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -74> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 started sync module instance, tier type = 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -73> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:33.073 INFO:tasks.workunit.client.0.smithi136.stderr: -72> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=0 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -71> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=1 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -70> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=2 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -69> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=3 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -68> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=4 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -67> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=5 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -66> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=6 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -65> 2024-04-22T20:09:33.001+0000 7fc4c8938800 20 add_watcher() i=7 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -64> 2024-04-22T20:09:33.001+0000 7fc4c8938800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -63> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding auth protocol: cephx 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -62> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding auth protocol: cephx 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -61> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding auth protocol: cephx 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -60> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -59> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.074 INFO:tasks.workunit.client.0.smithi136.stderr: -58> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.075 INFO:tasks.workunit.client.0.smithi136.stderr: -57> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.075 INFO:tasks.workunit.client.0.smithi136.stderr: -56> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.075 INFO:tasks.workunit.client.0.smithi136.stderr: -55> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.077 INFO:tasks.workunit.client.0.smithi136.stderr: -54> 2024-04-22T20:09:33.003+0000 7fc37afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:33.077 INFO:tasks.workunit.client.0.smithi136.stderr: -53> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.077 INFO:tasks.workunit.client.0.smithi136.stderr: -52> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.077 INFO:tasks.workunit.client.0.smithi136.stderr: -51> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.077 INFO:tasks.workunit.client.0.smithi136.stderr: -50> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.077 INFO:tasks.workunit.client.0.smithi136.stderr: -49> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -48> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -47> 2024-04-22T20:09:33.003+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -46> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -45> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -44> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cache list hook 0x55c5603dd840 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -43> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cache inspect hook 0x55c5603dd840 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -42> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cache erase hook 0x55c5603dd840 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -41> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cache zap hook 0x55c5603dd840 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -40> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 note: GC not initialized 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -39> 2024-04-22T20:09:33.003+0000 7fc379fdb640 20 reqs_thread_entry: start 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -38> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command sync trace show hook 0x7fc4bc014040 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -37> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command sync trace history hook 0x7fc4bc014040 2024-04-22T20:09:33.078 INFO:tasks.workunit.client.0.smithi136.stderr: -36> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command sync trace active hook 0x7fc4bc014040 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -35> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command sync trace active_short hook 0x7fc4bc014040 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -34> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -33> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 Filter name: none 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -32> 2024-04-22T20:09:33.003+0000 7fc2eb7fe640 20 reqs_thread_entry: start 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -31> 2024-04-22T20:09:33.003+0000 7fc4c8938800 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -30> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -29> 2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -28> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -27> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -26> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 : miss 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -25> 2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -24> 2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -23> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.079 INFO:tasks.workunit.client.0.smithi136.stderr: -22> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -21> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -20> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -19> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -18> 2024-04-22T20:09:33.008+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -17> 2024-04-22T20:09:33.008+0000 7fc4c8938800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -16> 2024-04-22T20:09:33.008+0000 7fc4c8938800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -15> 2024-04-22T20:09:33.008+0000 7fc4c8938800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26, ofs0, ns] 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -14> 2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562390 cookie 94306211823728 notifier 6825 bl.length()=923 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -13> 2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -12> 2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -11> 2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -10> 2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_sources= new_sources= 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -9> 2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential sources added=[] removed=[] 2024-04-22T20:09:33.080 INFO:tasks.workunit.client.0.smithi136.stderr: -8> 2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_dests= new_dests= 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: -7> 2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential dests added=[] removed=[] 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: -6> 2024-04-22T20:09:33.011+0000 7fc4c8938800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.0 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: -5> 2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.0 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: -4> 2024-04-22T20:09:33.011+0000 7fc4c8938800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.1 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: -3> 2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.1 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: -2> 2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.0 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: -1> 2024-04-22T20:09:33.014+0000 7fc4c8938800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fc4c8938800 time 2024-04-22T20:09:33.015425+0000 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc4c9f81f5c] 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc4c9f8211a] 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c55eb9cdc1] 2024-04-22T20:09:33.081 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x5c7) [0x55c55ebab127] 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fc4c8e29590] 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 0> 2024-04-22T20:09:33.015+0000 7fc4c8938800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fc4c8938800 thread_name:radosgw-admin 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fc4c8e3e6f0] 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fc4c8e8b94c] 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:09:33.082 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc4c9f81fb6] 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc4c9f8211a] 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c55eb9cdc1] 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x5c7) [0x55c55ebab127] 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fc4c8e29590] 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr:--- logging levels --- 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 none 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 lockdep 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 context 2024-04-22T20:09:33.083 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 crush 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_balancer 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_locker 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log_expire 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_migrator 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 3/ 5 mds_quiesce 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 buffer 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 timer 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 filer 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 striper 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 objecter 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rados 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_mirror 2024-04-22T20:09:33.084 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_replay 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_pwl 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 journaler 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objectcacher 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 immutable_obj_cache 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 client 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 osd 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 optracker 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objclass 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 filestore 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 journal 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 ms 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mon 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 0/10 monc 2024-04-22T20:09:33.085 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 paxos 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 tp 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 auth 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 crypto 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 finisher 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 reserver 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 heartbeatmap 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 perfcounter 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 20/20 rgw 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_sync 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_datacache 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_access 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_dbstore 2024-04-22T20:09:33.086 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_flight 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_lifecycle 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_notification 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 javaclient 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 asok 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 throttle 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 refs 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 compressor 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluestore 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluefs 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 bdev 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 kstore 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 4/ 5 rocksdb 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 fuse 2024-04-22T20:09:33.087 INFO:tasks.workunit.client.0.smithi136.stderr: 2/ 5 mgr 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mgrc 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 dpdk 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 eventtrace 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 prioritycache 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 test 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephfs_mirror 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephsqlite 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 crimson_interrupt 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_onode 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_odata 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_omap 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_tm 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_t 2024-04-22T20:09:33.088 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cleaner 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_epm 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_lba 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cache 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_journal 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_device 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_backref 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 alienstore 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mclock 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cyanstore 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 ceph_exporter 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 memstore 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 trace 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: -2/-2 (syslog threshold) 2024-04-22T20:09:33.089 INFO:tasks.workunit.client.0.smithi136.stderr: 99/99 (stderr threshold) 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc2eb7fe640 / http_manager 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc379fdb640 / http_manager 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc37afdd640 / rgw_dt_lg_renew 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc399ffb640 / ms_dispatch 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc39b7fe640 / io_context_pool 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc3a9ffb640 / ms_dispatch 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc3b0e09640 / ms_dispatch 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc3b260c640 / io_context_pool 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c3e39640 / msgr-worker-2 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c463a640 / msgr-worker-1 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c4e3b640 / msgr-worker-0 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c563c640 / io_context_pool 2024-04-22T20:09:33.090 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c8938800 / radosgw-admin 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr: max_recent 500 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr: max_new 1000 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr: log_file /var/log/ceph/ceph-client.admin.104075.log 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr:--- end dump of recent events --- 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr:--- begin dump of recent events --- 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr: -499> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr: -498> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr: -497> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr: -496> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr: -495> 2024-04-22T20:09:32.558+0000 7fc3b260c640 10 monclient: dump: 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.091 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: -494> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6449 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: -493> 2024-04-22T20:09:32.558+0000 7fc4c8938800 10 monclient: get_monmap_and_config success 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: -492> 2024-04-22T20:09:32.558+0000 7fc4c8938800 4 set_mon_vals no callback set 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: -491> 2024-04-22T20:09:32.558+0000 7fc4c8938800 10 monclient: shutdown 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: -490> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 asok(0x55c5600f1d00) unregister_commands rotate-key 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: -489> 2024-04-22T20:09:32.558+0000 7fc4c8938800 10 monclient: build_initial_monmap 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: -488> 2024-04-22T20:09:32.558+0000 7fc4c8938800 1 build_initial for_mkfs: 0 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr: -487> 2024-04-22T20:09:32.558+0000 7fc4c8938800 10 monclient: monmap: 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:09:33.092 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr: -486> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding auth protocol: cephx 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr: -485> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding auth protocol: cephx 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr: -484> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding auth protocol: cephx 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr: -483> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr: -482> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr: -481> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.093 INFO:tasks.workunit.client.0.smithi136.stderr: -480> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -479> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -478> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -477> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -476> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -475> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -474> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -473> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: crc 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -472> 2024-04-22T20:09:32.558+0000 7fc4c8938800 5 AuthRegistry(0x55c560314c20) adding con mode: secure 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -471> 2024-04-22T20:09:32.559+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -470> 2024-04-22T20:09:32.559+0000 7fc4c8938800 1 librados: starting msgr at 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -469> 2024-04-22T20:09:32.559+0000 7fc4c8938800 1 librados: starting objecter 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -468> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command objecter_requests hook 0x55c560314740 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -467> 2024-04-22T20:09:32.559+0000 7fc4c8938800 1 librados: setting wanted keys 2024-04-22T20:09:33.094 INFO:tasks.workunit.client.0.smithi136.stderr: -466> 2024-04-22T20:09:32.559+0000 7fc4c8938800 1 librados: calling monclient init 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -465> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient: init 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -464> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding auth protocol: cephx 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -463> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding auth protocol: cephx 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -462> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding auth protocol: cephx 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -461> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -460> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -459> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -458> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -457> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -456> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -455> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -454> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -453> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -452> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.095 INFO:tasks.workunit.client.0.smithi136.stderr: -451> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: crc 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -450> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 AuthRegistry(0x55c560311da0) adding con mode: secure 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -449> 2024-04-22T20:09:32.559+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -448> 2024-04-22T20:09:32.559+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -447> 2024-04-22T20:09:32.559+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command rotate-key hook 0x55c560311ee8 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -446> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient: _reopen_session rank -1 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -445> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient: _add_conns ranks=[2,1,0] 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -444> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-c con 0x55c5601c9480 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -443> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-b con 0x55c560253160 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -442> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-a con 0x55c560259680 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -441> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -440> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -439> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -438> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): _renew_subs 2024-04-22T20:09:33.096 INFO:tasks.workunit.client.0.smithi136.stderr: -437> 2024-04-22T20:09:32.559+0000 7fc4c8938800 10 monclient(hunting): authenticate will time out at 1961.911744s 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -436> 2024-04-22T20:09:32.559+0000 7fc4c463a640 10 monclient(hunting): get_auth_request con 0x55c5601c9480 auth_method 0 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -435> 2024-04-22T20:09:32.559+0000 7fc4c463a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -434> 2024-04-22T20:09:32.559+0000 7fc4c463a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -433> 2024-04-22T20:09:32.559+0000 7fc4c463a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -432> 2024-04-22T20:09:32.559+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request con 0x55c560259680 auth_method 0 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -431> 2024-04-22T20:09:32.559+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -430> 2024-04-22T20:09:32.559+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -429> 2024-04-22T20:09:32.559+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -428> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -427> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -426> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -425> 2024-04-22T20:09:32.560+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -424> 2024-04-22T20:09:32.560+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -423> 2024-04-22T20:09:32.560+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.097 INFO:tasks.workunit.client.0.smithi136.stderr: -422> 2024-04-22T20:09:32.560+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request con 0x55c560253160 auth_method 0 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -421> 2024-04-22T20:09:32.560+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -420> 2024-04-22T20:09:32.560+0000 7fc4c3e39640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -419> 2024-04-22T20:09:32.560+0000 7fc4c3e39640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -418> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient(hunting): handle_auth_done global_id 6452 payload 824 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -417> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient: _finish_hunting 0 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -416> 2024-04-22T20:09:32.560+0000 7fc4c463a640 1 monclient: found mon.noname-c 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -415> 2024-04-22T20:09:32.560+0000 7fc4c463a640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -414> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -413> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr: -412> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: dump: 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.098 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr: -411> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: _finish_auth 0 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr: -410> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr: -409> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr: -408> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr: -407> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr: -406> 2024-04-22T20:09:32.560+0000 7fc3b0e09640 10 monclient: dump: 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.099 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: -405> 2024-04-22T20:09:32.560+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6452 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: -404> 2024-04-22T20:09:32.560+0000 7fc3b260c640 4 set_mon_vals no callback set 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: -403> 2024-04-22T20:09:32.560+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: -402> 2024-04-22T20:09:32.560+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: -401> 2024-04-22T20:09:32.560+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: -400> 2024-04-22T20:09:32.560+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: -399> 2024-04-22T20:09:32.560+0000 7fc4c8938800 1 librados: init done 2024-04-22T20:09:33.100 INFO:tasks.workunit.client.0.smithi136.stderr: -398> 2024-04-22T20:09:32.562+0000 7fc3b0e09640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -397> 2024-04-22T20:09:32.562+0000 7fc3b0e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -396> 2024-04-22T20:09:32.562+0000 7fc3b0e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -395> 2024-04-22T20:09:32.562+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x7fc3ac06b2d0 auth_method 0 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -394> 2024-04-22T20:09:32.562+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x55c56033d910 auth_method 0 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -393> 2024-04-22T20:09:32.563+0000 7fc4c463a640 10 monclient: get_auth_request con 0x55c56035e0b0 auth_method 0 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -392> 2024-04-22T20:09:32.563+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x55c560362330 auth_method 0 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -391> 2024-04-22T20:09:32.564+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x55c560361930 auth_method 0 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -390> 2024-04-22T20:09:32.565+0000 7fc4c8938800 10 monclient: build_initial_monmap 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -389> 2024-04-22T20:09:32.565+0000 7fc4c8938800 1 build_initial for_mkfs: 0 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr: -388> 2024-04-22T20:09:32.565+0000 7fc4c8938800 10 monclient: monmap: 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:09:33.101 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -387> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding auth protocol: cephx 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -386> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding auth protocol: cephx 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -385> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding auth protocol: cephx 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -384> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -383> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -382> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -381> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -380> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.102 INFO:tasks.workunit.client.0.smithi136.stderr: -379> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -378> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -377> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -376> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -375> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -374> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: crc 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -373> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c56036a590) adding con mode: secure 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -372> 2024-04-22T20:09:32.565+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -371> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command objecter_requests cmddesc objecter_requests hook 0x55c56036fa70 EEXIST 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -370> 2024-04-22T20:09:32.565+0000 7fc4c8938800 10 monclient: init 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -369> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding auth protocol: cephx 2024-04-22T20:09:33.103 INFO:tasks.workunit.client.0.smithi136.stderr: -368> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding auth protocol: cephx 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -367> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding auth protocol: cephx 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -366> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -365> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -364> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -363> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -362> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -361> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -360> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -359> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -358> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -357> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -356> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: crc 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -355> 2024-04-22T20:09:32.565+0000 7fc4c8938800 5 AuthRegistry(0x55c560369378) adding con mode: secure 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -354> 2024-04-22T20:09:32.566+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.104 INFO:tasks.workunit.client.0.smithi136.stderr: -353> 2024-04-22T20:09:32.566+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -352> 2024-04-22T20:09:32.566+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command rotate-key cmddesc rotate-key hook 0x55c5603694c0 EEXIST 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -351> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient: _reopen_session rank -1 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -350> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -349> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-a con 0x55c5603947d0 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -348> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-c con 0x55c560394bb0 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -347> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-b con 0x55c56038fc70 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -346> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -345> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -344> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -343> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): _renew_subs 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -342> 2024-04-22T20:09:32.566+0000 7fc4c8938800 10 monclient(hunting): authenticate will time out at 1961.918336s 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -341> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request con 0x55c5603947d0 auth_method 0 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -340> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.105 INFO:tasks.workunit.client.0.smithi136.stderr: -339> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -338> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -337> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request con 0x55c560394bb0 auth_method 0 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -336> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -335> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -334> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -333> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -332> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -331> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -330> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -329> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -328> 2024-04-22T20:09:32.566+0000 7fc4c4e3b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -327> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_done global_id 6819 payload 824 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -326> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient: _finish_hunting 0 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -325> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 1 monclient: found mon.noname-a 2024-04-22T20:09:33.106 INFO:tasks.workunit.client.0.smithi136.stderr: -324> 2024-04-22T20:09:32.566+0000 7fc4c3e39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr: -323> 2024-04-22T20:09:32.566+0000 7fc4c463a640 10 monclient: get_auth_request con 0x55c56038fc70 auth_method 0 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr: -322> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr: -321> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr: -320> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: dump: 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.107 INFO:tasks.workunit.client.0.smithi136.stderr: -319> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: _finish_auth 0 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr: -318> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr: -317> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr: -316> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr: -315> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr: -314> 2024-04-22T20:09:32.567+0000 7fc3a9ffb640 10 monclient: dump: 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.108 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -313> 2024-04-22T20:09:32.567+0000 7fc4c563c640 4 set_mon_vals no callback set 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -312> 2024-04-22T20:09:32.567+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6819 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -311> 2024-04-22T20:09:32.567+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -310> 2024-04-22T20:09:32.567+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -309> 2024-04-22T20:09:32.567+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -308> 2024-04-22T20:09:32.567+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -307> 2024-04-22T20:09:32.568+0000 7fc3a9ffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -306> 2024-04-22T20:09:32.568+0000 7fc3a9ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -305> 2024-04-22T20:09:32.568+0000 7fc3a9ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -304> 2024-04-22T20:09:32.568+0000 7fc4c8938800 10 monclient: get_monmap_and_config 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -303> 2024-04-22T20:09:32.568+0000 7fc4c8938800 10 monclient: build_initial_monmap 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -302> 2024-04-22T20:09:32.568+0000 7fc4c8938800 1 build_initial for_mkfs: 0 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr: -301> 2024-04-22T20:09:32.568+0000 7fc4c8938800 10 monclient: monmap: 2024-04-22T20:09:33.109 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr: -300> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr: -299> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr: -298> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr: -297> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.110 INFO:tasks.workunit.client.0.smithi136.stderr: -296> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -295> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -294> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -293> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -292> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -291> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -290> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -289> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -288> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -287> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -286> 2024-04-22T20:09:32.568+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -285> 2024-04-22T20:09:32.569+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x7fc39c06b1e0 auth_method 0 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -284> 2024-04-22T20:09:32.569+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -283> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient: init 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -282> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding auth protocol: cephx 2024-04-22T20:09:33.111 INFO:tasks.workunit.client.0.smithi136.stderr: -281> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding auth protocol: cephx 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -280> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding auth protocol: cephx 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -279> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -278> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -277> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -276> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -275> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -274> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -273> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -272> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -271> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -270> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -269> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: crc 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -268> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2da80) adding con mode: secure 2024-04-22T20:09:33.112 INFO:tasks.workunit.client.0.smithi136.stderr: -267> 2024-04-22T20:09:32.569+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -266> 2024-04-22T20:09:32.569+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -265> 2024-04-22T20:09:32.569+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command rotate-key cmddesc rotate-key hook 0x7ffcf6b2dbc8 EEXIST 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -264> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient: _reopen_session rank -1 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -263> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient: _add_conns ranks=[0,1,2] 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -262> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-a con 0x55c5603e6e10 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -261> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-b con 0x55c5603eba70 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -260> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-c con 0x55c5603ec780 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -259> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -258> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -257> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -256> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): _renew_subs 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -255> 2024-04-22T20:09:32.569+0000 7fc4c8938800 10 monclient(hunting): authenticate will time out at 1961.921719s 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -254> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request con 0x55c5603e6e10 auth_method 0 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -253> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.113 INFO:tasks.workunit.client.0.smithi136.stderr: -252> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -251> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -250> 2024-04-22T20:09:32.569+0000 7fc4c463a640 10 monclient(hunting): get_auth_request con 0x55c5603ec780 auth_method 0 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -249> 2024-04-22T20:09:32.569+0000 7fc4c463a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -248> 2024-04-22T20:09:32.569+0000 7fc4c463a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -247> 2024-04-22T20:09:32.569+0000 7fc4c463a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -246> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -245> 2024-04-22T20:09:32.569+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -244> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -243> 2024-04-22T20:09:32.570+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -242> 2024-04-22T20:09:32.570+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -241> 2024-04-22T20:09:32.570+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -240> 2024-04-22T20:09:32.570+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request con 0x55c5603eba70 auth_method 0 2024-04-22T20:09:33.114 INFO:tasks.workunit.client.0.smithi136.stderr: -239> 2024-04-22T20:09:32.570+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -238> 2024-04-22T20:09:32.570+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -237> 2024-04-22T20:09:32.570+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -236> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_done global_id 6822 payload 274 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -235> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 10 monclient: _finish_hunting 0 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -234> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 1 monclient: found mon.noname-a 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -233> 2024-04-22T20:09:32.570+0000 7fc4c3e39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -232> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -231> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr: -230> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: dump: 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.115 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr: -229> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: _finish_auth 0 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr: -228> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr: -227> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr: -226> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr: -225> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr: -224> 2024-04-22T20:09:32.570+0000 7fc39b7fe640 10 monclient: dump: 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.116 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr: -223> 2024-04-22T20:09:32.570+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6822 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr: -222> 2024-04-22T20:09:32.570+0000 7fc4c8938800 10 monclient: get_monmap_and_config success 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr: -221> 2024-04-22T20:09:32.570+0000 7fc4c8938800 4 set_mon_vals no callback set 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr: -220> 2024-04-22T20:09:32.570+0000 7fc4c8938800 10 monclient: shutdown 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr: -219> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: build_initial_monmap 2024-04-22T20:09:33.117 INFO:tasks.workunit.client.0.smithi136.stderr: -218> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 build_initial for_mkfs: 0 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr: -217> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: monmap: 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr: -216> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.118 INFO:tasks.workunit.client.0.smithi136.stderr: -215> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -214> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding auth protocol: cephx 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -213> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -212> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -211> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -210> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -209> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -208> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -207> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -206> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -205> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -204> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -203> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: crc 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -202> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b2320) adding con mode: secure 2024-04-22T20:09:33.119 INFO:tasks.workunit.client.0.smithi136.stderr: -201> 2024-04-22T20:09:32.571+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -200> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 librados: starting msgr at 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -199> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 librados: starting objecter 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -198> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command objecter_requests cmddesc objecter_requests hook 0x55c5603aa2b0 EEXIST 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -197> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 librados: setting wanted keys 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -196> 2024-04-22T20:09:32.571+0000 7fc4c8938800 1 librados: calling monclient init 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -195> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: init 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -194> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding auth protocol: cephx 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -193> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding auth protocol: cephx 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -192> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding auth protocol: cephx 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -191> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -190> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -189> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -188> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -187> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.120 INFO:tasks.workunit.client.0.smithi136.stderr: -186> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -185> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -184> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -183> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -182> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -181> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: crc 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -180> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 AuthRegistry(0x55c5603b14f0) adding con mode: secure 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -179> 2024-04-22T20:09:32.571+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -178> 2024-04-22T20:09:32.571+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -177> 2024-04-22T20:09:32.571+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command rotate-key cmddesc rotate-key hook 0x55c5603b1638 EEXIST 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -176> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: _reopen_session rank -1 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -175> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient: _add_conns ranks=[0,1,2] 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -174> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-a con 0x55c5603e6e10 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -173> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-b con 0x55c5603eba70 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -172> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): picked mon.noname-c con 0x55c5603ec780 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:09:33.121 INFO:tasks.workunit.client.0.smithi136.stderr: -171> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -170> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -169> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): start opening mon connection 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -168> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): _renew_subs 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -167> 2024-04-22T20:09:32.571+0000 7fc4c8938800 10 monclient(hunting): authenticate will time out at 1961.923809s 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -166> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request con 0x55c5603e6e10 auth_method 0 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -165> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -164> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -163> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -162> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): get_auth_request con 0x55c5603ec780 auth_method 0 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -161> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -160> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -159> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.122 INFO:tasks.workunit.client.0.smithi136.stderr: -158> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -157> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -156> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -155> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -154> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -153> 2024-04-22T20:09:32.572+0000 7fc4c463a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -152> 2024-04-22T20:09:32.572+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request con 0x55c5603eba70 auth_method 0 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -151> 2024-04-22T20:09:32.572+0000 7fc4c4e3b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -150> 2024-04-22T20:09:32.572+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -149> 2024-04-22T20:09:32.572+0000 7fc4c4e3b640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -148> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient(hunting): handle_auth_done global_id 6825 payload 824 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -147> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient: _finish_hunting 0 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -146> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 1 monclient: found mon.noname-a 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -145> 2024-04-22T20:09:32.572+0000 7fc4c3e39640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.123 INFO:tasks.workunit.client.0.smithi136.stderr: -144> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr: -143> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr: -142> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: dump: 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr: -141> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: _finish_auth 0 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr: -140> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: _check_auth_tickets 2024-04-22T20:09:33.124 INFO:tasks.workunit.client.0.smithi136.stderr: -139> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: handle_config config(0 keys) 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr: -138> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr: -137> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr: -136> 2024-04-22T20:09:32.572+0000 7fc399ffb640 10 monclient: dump: 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr: -135> 2024-04-22T20:09:32.572+0000 7fc4c8938800 5 monclient: authenticate success, global_id 6825 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr: -134> 2024-04-22T20:09:32.572+0000 7fc39b7fe640 4 set_mon_vals no callback set 2024-04-22T20:09:33.125 INFO:tasks.workunit.client.0.smithi136.stderr: -133> 2024-04-22T20:09:32.572+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -132> 2024-04-22T20:09:32.572+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -131> 2024-04-22T20:09:32.572+0000 7fc4c8938800 10 monclient: _renew_subs 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -130> 2024-04-22T20:09:32.572+0000 7fc4c8938800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -129> 2024-04-22T20:09:32.572+0000 7fc4c8938800 1 librados: init done 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -128> 2024-04-22T20:09:32.572+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cr dump hook 0x55c5603e6048 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -127> 2024-04-22T20:09:32.573+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -126> 2024-04-22T20:09:32.573+0000 7fc399ffb640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -125> 2024-04-22T20:09:32.573+0000 7fc399ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -124> 2024-04-22T20:09:32.573+0000 7fc399ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -123> 2024-04-22T20:09:32.574+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x7fc38c06b2d0 auth_method 0 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -122> 2024-04-22T20:09:32.574+0000 7fc4c463a640 10 monclient: get_auth_request con 0x55c560400e30 auth_method 0 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -121> 2024-04-22T20:09:32.574+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -120> 2024-04-22T20:09:32.574+0000 7fc4c8938800 20 realm 2024-04-22T20:09:33.126 INFO:tasks.workunit.client.0.smithi136.stderr: -119> 2024-04-22T20:09:32.574+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -118> 2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -117> 2024-04-22T20:09:32.575+0000 7fc4c8938800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -116> 2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -115> 2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -114> 2024-04-22T20:09:32.575+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -113> 2024-04-22T20:09:32.575+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x55c5604050a0 auth_method 0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -112> 2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -111> 2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -110> 2024-04-22T20:09:32.576+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x55c560408900 auth_method 0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -109> 2024-04-22T20:09:32.576+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -108> 2024-04-22T20:09:32.576+0000 7fc4c8938800 20 searching for the correct realm 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -107> 2024-04-22T20:09:32.577+0000 7fc4c463a640 10 monclient: get_auth_request con 0x55c56040d6e0 auth_method 0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -106> 2024-04-22T20:09:32.577+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x7fc4ac09beb0 auth_method 0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -105> 2024-04-22T20:09:32.578+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x7fc4bc03eb60 auth_method 0 2024-04-22T20:09:33.127 INFO:tasks.workunit.client.0.smithi136.stderr: -104> 2024-04-22T20:09:32.579+0000 7fc4c4e3b640 0 -- 172.21.15.136:0/2984211145 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fc4bc03eb60 msgr2=0x7fc4bc037410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -103> 2024-04-22T20:09:32.780+0000 7fc4c4e3b640 10 monclient: get_auth_request con 0x7fc4bc03eb60 auth_method 0 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -102> 2024-04-22T20:09:32.782+0000 7fc4c463a640 10 monclient: get_auth_request con 0x7fc4bc038bb0 auth_method 0 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -101> 2024-04-22T20:09:32.784+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x7fc4bc0283e0 auth_method 0 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -100> 2024-04-22T20:09:32.784+0000 7fc4c3e39640 0 -- 172.21.15.136:0/2984211145 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fc4bc0283e0 msgr2=0x7fc4bc028860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -99> 2024-04-22T20:09:32.985+0000 7fc4c3e39640 10 monclient: get_auth_request con 0x7fc4bc0283e0 auth_method 0 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -98> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -97> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -96> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -95> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -94> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -93> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -92> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -91> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.128 INFO:tasks.workunit.client.0.smithi136.stderr: -90> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -89> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -88> 2024-04-22T20:09:32.990+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -87> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -86> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 zone default found 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -85> 2024-04-22T20:09:32.991+0000 7fc4c8938800 4 Realm: () 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -84> 2024-04-22T20:09:32.991+0000 7fc4c8938800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -83> 2024-04-22T20:09:32.991+0000 7fc4c8938800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -82> 2024-04-22T20:09:32.991+0000 7fc4c8938800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -81> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 zonegroup default 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -80> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -79> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -78> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -77> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -76> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.129 INFO:tasks.workunit.client.0.smithi136.stderr: -75> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -74> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 started sync module instance, tier type = 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -73> 2024-04-22T20:09:32.991+0000 7fc4c8938800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -72> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=0 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -71> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=1 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -70> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=2 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -69> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=3 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -68> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=4 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -67> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=5 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -66> 2024-04-22T20:09:33.000+0000 7fc4c8938800 20 add_watcher() i=6 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -65> 2024-04-22T20:09:33.001+0000 7fc4c8938800 20 add_watcher() i=7 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -64> 2024-04-22T20:09:33.001+0000 7fc4c8938800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -63> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding auth protocol: cephx 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -62> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding auth protocol: cephx 2024-04-22T20:09:33.130 INFO:tasks.workunit.client.0.smithi136.stderr: -61> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding auth protocol: cephx 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -60> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -59> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -58> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -57> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -56> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -55> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -54> 2024-04-22T20:09:33.003+0000 7fc37afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -53> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -52> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -51> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -50> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -49> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: crc 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -48> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 AuthRegistry(0x7ffcf6b2df10) adding con mode: secure 2024-04-22T20:09:33.131 INFO:tasks.workunit.client.0.smithi136.stderr: -47> 2024-04-22T20:09:33.003+0000 7fc4c8938800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -46> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -45> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -44> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cache list hook 0x55c5603dd840 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -43> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cache inspect hook 0x55c5603dd840 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -42> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cache erase hook 0x55c5603dd840 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -41> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command cache zap hook 0x55c5603dd840 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -40> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 note: GC not initialized 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -39> 2024-04-22T20:09:33.003+0000 7fc379fdb640 20 reqs_thread_entry: start 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -38> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command sync trace show hook 0x7fc4bc014040 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -37> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command sync trace history hook 0x7fc4bc014040 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -36> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command sync trace active hook 0x7fc4bc014040 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -35> 2024-04-22T20:09:33.003+0000 7fc4c8938800 5 asok(0x55c5600f1d00) register_command sync trace active_short hook 0x7fc4bc014040 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -34> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:33.132 INFO:tasks.workunit.client.0.smithi136.stderr: -33> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 Filter name: none 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -32> 2024-04-22T20:09:33.003+0000 7fc2eb7fe640 20 reqs_thread_entry: start 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -31> 2024-04-22T20:09:33.003+0000 7fc4c8938800 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -30> 2024-04-22T20:09:33.003+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -29> 2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -28> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -27> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -26> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 : miss 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -25> 2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -24> 2024-04-22T20:09:33.004+0000 7fc4c8938800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -23> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -22> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -21> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -20> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.133 INFO:tasks.workunit.client.0.smithi136.stderr: -19> 2024-04-22T20:09:33.004+0000 7fc4c8938800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -18> 2024-04-22T20:09:33.008+0000 7fc4c8938800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -17> 2024-04-22T20:09:33.008+0000 7fc4c8938800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -16> 2024-04-22T20:09:33.008+0000 7fc4c8938800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -15> 2024-04-22T20:09:33.008+0000 7fc4c8938800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26, ofs0, ns] 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -14> 2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562390 cookie 94306211823728 notifier 6825 bl.length()=923 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -13> 2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -12> 2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -11> 2024-04-22T20:09:33.008+0000 7fc39b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -10> 2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_sources= new_sources= 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -9> 2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential sources added=[] removed=[] 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -8> 2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_dests= new_dests= 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -7> 2024-04-22T20:09:33.010+0000 7fc4c8938800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential dests added=[] removed=[] 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -6> 2024-04-22T20:09:33.011+0000 7fc4c8938800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.0 2024-04-22T20:09:33.134 INFO:tasks.workunit.client.0.smithi136.stderr: -5> 2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.0 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: -4> 2024-04-22T20:09:33.011+0000 7fc4c8938800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.1 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: -3> 2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.1 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: -2> 2024-04-22T20:09:33.011+0000 7fc4c8938800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.0 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: -1> 2024-04-22T20:09:33.014+0000 7fc4c8938800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fc4c8938800 time 2024-04-22T20:09:33.015425+0000 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fc4c9f81f5c] 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc4c9f8211a] 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c55eb9cdc1] 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x5c7) [0x55c55ebab127] 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fc4c8e29590] 2024-04-22T20:09:33.135 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 0> 2024-04-22T20:09:33.015+0000 7fc4c8938800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fc4c8938800 thread_name:radosgw-admin 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fc4c8e3e6f0] 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fc4c8e8b94c] 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fc4c9f81fb6] 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fc4c9f8211a] 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c55eb9cdc1] 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x5c7) [0x55c55ebab127] 2024-04-22T20:09:33.136 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fc4c8e29590] 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr:--- logging levels --- 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 none 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 lockdep 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 context 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 crush 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_balancer 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_locker 2024-04-22T20:09:33.137 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log_expire 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_migrator 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 3/ 5 mds_quiesce 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 buffer 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 timer 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 filer 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 striper 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 objecter 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rados 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_mirror 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_replay 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_pwl 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 journaler 2024-04-22T20:09:33.138 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objectcacher 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 immutable_obj_cache 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 client 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 osd 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 optracker 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objclass 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 filestore 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 journal 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 ms 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mon 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 0/10 monc 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 paxos 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 tp 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 auth 2024-04-22T20:09:33.139 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 crypto 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 finisher 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 reserver 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 heartbeatmap 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 perfcounter 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 20/20 rgw 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_sync 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_datacache 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_access 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_dbstore 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_flight 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_lifecycle 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_notification 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 javaclient 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 asok 2024-04-22T20:09:33.140 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 throttle 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 refs 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 compressor 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluestore 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluefs 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 bdev 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 kstore 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 4/ 5 rocksdb 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 fuse 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 2/ 5 mgr 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mgrc 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 dpdk 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 eventtrace 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 prioritycache 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 test 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephfs_mirror 2024-04-22T20:09:33.141 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephsqlite 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 crimson_interrupt 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_onode 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_odata 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_omap 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_tm 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_t 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cleaner 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_epm 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_lba 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cache 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_journal 2024-04-22T20:09:33.142 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_device 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_backref 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 alienstore 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mclock 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cyanstore 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 ceph_exporter 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 memstore 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 trace 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: -2/-2 (syslog threshold) 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 99/99 (stderr threshold) 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc2eb7fe640 / http_manager 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc379fdb640 / http_manager 2024-04-22T20:09:33.143 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc37afdd640 / rgw_dt_lg_renew 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc399ffb640 / ms_dispatch 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc39b7fe640 / io_context_pool 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc3a9ffb640 / ms_dispatch 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc3b0e09640 / ms_dispatch 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc3b260c640 / io_context_pool 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c3e39640 / msgr-worker-2 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c463a640 / msgr-worker-1 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c4e3b640 / msgr-worker-0 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c563c640 / io_context_pool 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: 7fc4c8938800 / radosgw-admin 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: max_recent 500 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: max_new 1000 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr: log_file /var/lib/ceph/crash/2024-04-22T20:09:33.016813Z_b87e4169-d96f-4489-86e0-c96378276406/log 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr:--- end dump of recent events --- 2024-04-22T20:09:33.144 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-04-22T20:09:33.146 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:09:33.146 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:09:33.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.085+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.087+0000 7feb0fe40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.087+0000 7feb0fe40800 20 realm 2024-04-22T20:09:33.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.087+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.087+0000 7feb0fe40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.087+0000 7feb0fe40800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.087+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.087+0000 7feb0fe40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.087+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.088+0000 7feb0fe40800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.088+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.089+0000 7feb0fe40800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.089+0000 7feb0fe40800 20 searching for the correct realm 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.091+0000 7feb0b2dd640 0 -- 172.21.15.136:0/1961518147 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7feafc013030 msgr2=0x7feafc045410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:33.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.095+0000 7feb0a2db640 0 -- 172.21.15.136:0/1961518147 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7feafc013410 msgr2=0x7feafc013890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.302 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.301+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.303 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 20 zone default found 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 4 Realm: () 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 20 zonegroup default 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.302+0000 7feb0fe40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.303+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.304 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.303+0000 7feb0fe40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.303+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.303+0000 7feb0fe40800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:09:33.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.303+0000 7feb0fe40800 20 started sync module instance, tier type = 2024-04-22T20:09:33.305 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.303+0000 7feb0fe40800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:09:33.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.313+0000 7feb0fe40800 20 add_watcher() i=0 2024-04-22T20:09:33.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.313+0000 7feb0fe40800 20 add_watcher() i=1 2024-04-22T20:09:33.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.313+0000 7feb0fe40800 20 add_watcher() i=2 2024-04-22T20:09:33.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.313+0000 7feb0fe40800 20 add_watcher() i=3 2024-04-22T20:09:33.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.313+0000 7feb0fe40800 20 add_watcher() i=4 2024-04-22T20:09:33.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.313+0000 7feb0fe40800 20 add_watcher() i=5 2024-04-22T20:09:33.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.313+0000 7feb0fe40800 20 add_watcher() i=6 2024-04-22T20:09:33.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.314+0000 7feb0fe40800 20 add_watcher() i=7 2024-04-22T20:09:33.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.314+0000 7feb0fe40800 2 all 8 watchers are set, enabling cache 2024-04-22T20:09:33.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7fe9c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:33.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7feb0fe40800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:09:33.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7feb0fe40800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:09:33.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7feb0fe40800 5 note: GC not initialized 2024-04-22T20:09:33.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7fe93bfff640 20 reqs_thread_entry: start 2024-04-22T20:09:33.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7feb0fe40800 20 init_complete bucket index max shards: 1 2024-04-22T20:09:33.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7feb0fe40800 20 Filter name: none 2024-04-22T20:09:33.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7fe939ffb640 20 reqs_thread_entry: start 2024-04-22T20:09:33.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.316+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.317+0000 7feb0fe40800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:33.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.317+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.317+0000 7feb0fe40800 20 rados_obj.operate() r=0 bl.length=443 2024-04-22T20:09:33.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.317+0000 7feb0fe40800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:33.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.317+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.318+0000 7feb0fe40800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:09:33.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.318+0000 7feb0fe40800 20 rados->read ofs=0 len=0 2024-04-22T20:09:33.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.318+0000 7feb0fe40800 20 rados_obj.operate() r=0 bl.length=443 2024-04-22T20:09:33.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.318+0000 7feb0fe40800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:09:33.320 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.320+0000 7fe9c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:09:33.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.322+0000 7feb0fe40800 20 remove_watcher() i=0 2024-04-22T20:09:33.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.322+0000 7feb0fe40800 2 removed watcher, disabling cache 2024-04-22T20:09:33.322 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.323+0000 7feb0fe40800 20 remove_watcher() i=1 2024-04-22T20:09:33.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.323+0000 7feb0fe40800 20 remove_watcher() i=2 2024-04-22T20:09:33.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.325+0000 7feb0fe40800 20 remove_watcher() i=3 2024-04-22T20:09:33.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.326+0000 7feb0fe40800 20 remove_watcher() i=4 2024-04-22T20:09:33.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.327+0000 7feb0fe40800 20 remove_watcher() i=5 2024-04-22T20:09:33.328 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.328+0000 7feb0fe40800 20 remove_watcher() i=6 2024-04-22T20:09:33.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:09:33.329+0000 7feb0fe40800 20 remove_watcher() i=7 2024-04-22T20:09:33.338 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.339 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:09:33.339 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-commit-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:09:33.339 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:09:33.340 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:09:33.340 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:03.474 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:03.475 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:03.533 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:10:03.564 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:03.564 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:03.600 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.600+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:03.601 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.602+0000 7f67e522f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:03.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.602+0000 7f67e522f800 20 realm 2024-04-22T20:10:03.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.602+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:03.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.602+0000 7f67e522f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:03.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.602+0000 7f67e522f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:03.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.602+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:03.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.602+0000 7f67e522f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:03.602 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.602+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:03.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.603+0000 7f67e522f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:03.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.603+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:03.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.604+0000 7f67e522f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:03.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.604+0000 7f67e522f800 20 searching for the correct realm 2024-04-22T20:10:03.605 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.606+0000 7f67e0d80640 0 -- 172.21.15.136:0/1767971948 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f67d8018040 msgr2=0x7f67d805da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:03.811 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:03.812+0000 7f67dfd7e640 0 -- 172.21.15.136:0/1767971948 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f67d805d190 msgr2=0x7f67d8088220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:04.019 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.019+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:04.020 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:04.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 20 zone default found 2024-04-22T20:10:04.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 4 Realm: () 2024-04-22T20:10:04.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:04.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:04.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:04.021 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 20 zonegroup default 2024-04-22T20:10:04.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.020+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.021+0000 7f67e522f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.021+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.021+0000 7f67e522f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.021+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.022+0000 7f67e522f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.022+0000 7f67e522f800 20 started sync module instance, tier type = 2024-04-22T20:10:04.022 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.022+0000 7f67e522f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.032+0000 7f67e522f800 20 add_watcher() i=0 2024-04-22T20:10:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.032+0000 7f67e522f800 20 add_watcher() i=1 2024-04-22T20:10:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.032+0000 7f67e522f800 20 add_watcher() i=2 2024-04-22T20:10:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.032+0000 7f67e522f800 20 add_watcher() i=3 2024-04-22T20:10:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.032+0000 7f67e522f800 20 add_watcher() i=4 2024-04-22T20:10:04.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.032+0000 7f67e522f800 20 add_watcher() i=5 2024-04-22T20:10:04.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.032+0000 7f67e522f800 20 add_watcher() i=6 2024-04-22T20:10:04.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.033+0000 7f67e522f800 20 add_watcher() i=7 2024-04-22T20:10:04.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.033+0000 7f67e522f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:04.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.035+0000 7f6696fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.035+0000 7f67e522f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.035+0000 7f67e522f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.035+0000 7f67e522f800 5 note: GC not initialized 2024-04-22T20:10:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f6695fdb640 20 reqs_thread_entry: start 2024-04-22T20:10:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 20 Filter name: none 2024-04-22T20:10:04.035 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f66077fe640 20 reqs_thread_entry: start 2024-04-22T20:10:04.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-04-22T20:10:04.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:10:04.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-04-22T20:10:04.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-04-22T20:10:04.036 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 : miss 2024-04-22T20:10:04.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.036+0000 7f67e522f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.037+0000 7f67e522f800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:10:04.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.037+0000 7f67e522f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:10:04.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.037+0000 7f67e522f800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:10:04.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.037+0000 7f67e522f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:04.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.037+0000 7f67e522f800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:04.037 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.037+0000 7f67e522f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 2024-04-22T20:10:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.041+0000 7f67e522f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:10:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.041+0000 7f67e522f800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:10:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.041+0000 7f67e522f800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.041+0000 7f67e522f800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26, ofs0, ns] 2024-04-22T20:10:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.042+0000 7f66b6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562392 cookie 94639173576400 notifier 6494 bl.length()=923 2024-04-22T20:10:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.042+0000 7f66b6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:10:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.042+0000 7f66b6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:10:04.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.042+0000 7f66b6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:04.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.043+0000 7f67e522f800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_sources= new_sources= 2024-04-22T20:10:04.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.043+0000 7f67e522f800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential sources added=[] removed=[] 2024-04-22T20:10:04.043 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.043+0000 7f67e522f800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_dests= new_dests= 2024-04-22T20:10:04.043 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.043+0000 7f67e522f800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential dests added=[] removed=[] 2024-04-22T20:10:04.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.045+0000 7f67e522f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.0 2024-04-22T20:10:04.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.045+0000 7f67e522f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.0 2024-04-22T20:10:04.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.045+0000 7f67e522f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.1 2024-04-22T20:10:04.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.045+0000 7f67e522f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.1.1 2024-04-22T20:10:04.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.045+0000 7f67e522f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26.0 2024-04-22T20:10:04.047 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.048+0000 7f67e522f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 : hit (requested=0x17, cached=0x17) 2024-04-22T20:10:04.047 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.048+0000 7f67e522f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 2024-04-22T20:10:04.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.049+0000 7f67e522f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:10:04.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.049+0000 7f67e522f800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:10:04.048 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.049+0000 7f67e522f800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:04.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.049+0000 7f67e522f800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26, ofs0, ns] 2024-04-22T20:10:04.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.049+0000 7f66b77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562393 cookie 94639173576400 notifier 6494 bl.length()=891 2024-04-22T20:10:04.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.049+0000 7f66b77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 info.flags=0x17 2024-04-22T20:10:04.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.049+0000 7f66b77fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 to cache LRU end 2024-04-22T20:10:04.049 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.049+0000 7f66b77fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:04.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.051+0000 7f67e522f800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_sources= new_sources= 2024-04-22T20:10:04.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.051+0000 7f67e522f800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential sources added=[] removed=[] 2024-04-22T20:10:04.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.051+0000 7f67e522f800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): orig_dests= new_dests= 2024-04-22T20:10:04.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.051+0000 7f67e522f800 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26]): potential dests added=[] removed=[] 2024-04-22T20:10:04.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.052+0000 7f67e522f800 1 execute INFO: reshard of bucket "abort-at-commit-target-layout" completed successfully 2024-04-22T20:10:04.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.054+0000 7f6696fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:04.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.056+0000 7f67e522f800 20 remove_watcher() i=0 2024-04-22T20:10:04.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.056+0000 7f67e522f800 2 removed watcher, disabling cache 2024-04-22T20:10:04.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.057+0000 7f67e522f800 20 remove_watcher() i=1 2024-04-22T20:10:04.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.059+0000 7f67e522f800 20 remove_watcher() i=2 2024-04-22T20:10:04.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.060+0000 7f67e522f800 20 remove_watcher() i=3 2024-04-22T20:10:04.060 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.061+0000 7f67e522f800 20 remove_watcher() i=4 2024-04-22T20:10:04.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.062+0000 7f67e522f800 20 remove_watcher() i=5 2024-04-22T20:10:04.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.063+0000 7f67e522f800 20 remove_watcher() i=6 2024-04-22T20:10:04.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.064+0000 7f67e522f800 20 remove_watcher() i=7 2024-04-22T20:10:04.075 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-04-22T20:10:04.103 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:04.103 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:04.138 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.139+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.141+0000 7f07e281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.141+0000 7f07e281f800 20 realm 2024-04-22T20:10:04.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.141+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.140 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.141+0000 7f07e281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.141+0000 7f07e281f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:04.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.141+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.141+0000 7f07e281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.141 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.141+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.142+0000 7f07e281f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:04.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.142+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.143+0000 7f07e281f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:04.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.143+0000 7f07e281f800 20 searching for the correct realm 2024-04-22T20:10:04.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.143+0000 7f07e0325640 0 -- 172.21.15.136:0/4087297609 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x564699b44710 msgr2=0x564699b46bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:04.347 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.348+0000 7f07e0325640 0 -- 172.21.15.136:0/4087297609 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x5646998fe220 msgr2=0x7f07d8038430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.557+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:04.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:04.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 20 zone default found 2024-04-22T20:10:04.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 4 Realm: () 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 20 zonegroup default 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.558+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.559+0000 7f07e281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.559+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.559+0000 7f07e281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.559+0000 7f07e281f800 20 started sync module instance, tier type = 2024-04-22T20:10:04.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.559+0000 7f07e281f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:04.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.569+0000 7f07e281f800 20 add_watcher() i=0 2024-04-22T20:10:04.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.569+0000 7f07e281f800 20 add_watcher() i=1 2024-04-22T20:10:04.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.569+0000 7f07e281f800 20 add_watcher() i=2 2024-04-22T20:10:04.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.569+0000 7f07e281f800 20 add_watcher() i=3 2024-04-22T20:10:04.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.569+0000 7f07e281f800 20 add_watcher() i=4 2024-04-22T20:10:04.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.569+0000 7f07e281f800 20 add_watcher() i=5 2024-04-22T20:10:04.568 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.569+0000 7f07e281f800 20 add_watcher() i=6 2024-04-22T20:10:04.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.570+0000 7f07e281f800 20 add_watcher() i=7 2024-04-22T20:10:04.569 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.570+0000 7f07e281f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:04.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.571+0000 7f0696fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:04.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.572+0000 7f07e281f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:04.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.572+0000 7f07e281f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:04.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.572+0000 7f07e281f800 5 note: GC not initialized 2024-04-22T20:10:04.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.572+0000 7f0695fdb640 20 reqs_thread_entry: start 2024-04-22T20:10:04.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.572+0000 7f07e281f800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:04.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.572+0000 7f07e281f800 20 Filter name: none 2024-04-22T20:10:04.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.572+0000 7f06077fe640 20 reqs_thread_entry: start 2024-04-22T20:10:04.572 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.572+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.573+0000 7f07e281f800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:10:04.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.573+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.573+0000 7f07e281f800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:10:04.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.573+0000 7f07e281f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:04.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.573+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.574+0000 7f07e281f800 20 rados_obj.operate() r=0 bl.length=197 2024-04-22T20:10:04.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.574+0000 7f07e281f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:04.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.574+0000 7f07e281f800 20 rados_obj.operate() r=0 bl.length=411 2024-04-22T20:10:04.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.574+0000 7f07e281f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:04.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.575+0000 7f0696fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:04.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.578+0000 7f07e281f800 20 remove_watcher() i=0 2024-04-22T20:10:04.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.578+0000 7f07e281f800 2 removed watcher, disabling cache 2024-04-22T20:10:04.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.578+0000 7f07e281f800 20 remove_watcher() i=1 2024-04-22T20:10:04.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.580+0000 7f07e281f800 20 remove_watcher() i=2 2024-04-22T20:10:04.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.581+0000 7f07e281f800 20 remove_watcher() i=3 2024-04-22T20:10:04.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.582+0000 7f07e281f800 20 remove_watcher() i=4 2024-04-22T20:10:04.582 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.583+0000 7f07e281f800 20 remove_watcher() i=5 2024-04-22T20:10:04.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.584+0000 7f07e281f800 20 remove_watcher() i=6 2024-04-22T20:10:04.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:04.585+0000 7f07e281f800 20 remove_watcher() i=7 2024-04-22T20:10:04.593 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:04.593 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:04.593 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-commit-target-layout id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-22T20:10:04.593 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:04.596 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:04.596 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:04.617 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:04.617 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:04.826 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:04.826 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:04.888 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with EIO injected at do_reshard 2024-04-22T20:10:04.889 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:04.890 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:04.890 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:04.931 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:04.931 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:04.988 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:04.988 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.022 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.022 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.057 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.057 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.094 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.094 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.129 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.129 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.167 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.167 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.202 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.202 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.236 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.236 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.272 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.272 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.307 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.307 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.342 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.342 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.378 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.378 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.413 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.413 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.448 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.449 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.483 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.484 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.518 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.518 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.553 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.553 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.588 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.588 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.623 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.623 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.659 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:05.660 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:05.697 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-04-22T20:10:05.724 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:05.724 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:05.757 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.758+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.760+0000 7fc283e16800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:05.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.760+0000 7fc283e16800 20 realm 2024-04-22T20:10:05.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.760+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.760+0000 7fc283e16800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:05.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.760+0000 7fc283e16800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:05.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.760+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.760+0000 7fc283e16800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:05.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.760+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.761+0000 7fc283e16800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:05.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.761+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.762+0000 7fc283e16800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:05.761 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.762+0000 7fc283e16800 20 searching for the correct realm 2024-04-22T20:10:05.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.764+0000 7fc27fc85640 0 -- 172.21.15.136:0/1240962418 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fc278024410 msgr2=0x7fc278024830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:05.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.769+0000 7fc27ec83640 0 -- 172.21.15.136:0/1240962418 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fc278039440 msgr2=0x7fc2780398a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.975+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.975 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:05.976 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 20 zone default found 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 4 Realm: () 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 20 zonegroup default 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.976+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.977+0000 7fc283e16800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.977+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.977 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.977+0000 7fc283e16800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:05.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.977+0000 7fc283e16800 20 started sync module instance, tier type = 2024-04-22T20:10:05.978 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.977+0000 7fc283e16800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:05.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.987+0000 7fc283e16800 20 add_watcher() i=0 2024-04-22T20:10:05.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.987+0000 7fc283e16800 20 add_watcher() i=1 2024-04-22T20:10:05.986 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.987+0000 7fc283e16800 20 add_watcher() i=2 2024-04-22T20:10:05.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.987+0000 7fc283e16800 20 add_watcher() i=3 2024-04-22T20:10:05.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.987+0000 7fc283e16800 20 add_watcher() i=4 2024-04-22T20:10:05.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.987+0000 7fc283e16800 20 add_watcher() i=5 2024-04-22T20:10:05.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.987+0000 7fc283e16800 20 add_watcher() i=6 2024-04-22T20:10:05.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.988+0000 7fc283e16800 20 add_watcher() i=7 2024-04-22T20:10:05.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.988+0000 7fc283e16800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:05.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.990+0000 7fc135fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:05.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.990+0000 7fc283e16800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:05.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.990+0000 7fc283e16800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:05.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.990+0000 7fc283e16800 5 note: GC not initialized 2024-04-22T20:10:05.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.990+0000 7fc134fe1640 20 reqs_thread_entry: start 2024-04-22T20:10:05.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.991+0000 7fc283e16800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:05.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.991+0000 7fc283e16800 20 Filter name: none 2024-04-22T20:10:05.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.991+0000 7fc0ae7fc640 20 reqs_thread_entry: start 2024-04-22T20:10:05.991 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.991+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.991+0000 7fc283e16800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.991+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.992+0000 7fc283e16800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.992+0000 7fc283e16800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.992+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.992+0000 7fc283e16800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.992+0000 7fc283e16800 20 rados->read ofs=0 len=0 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.992+0000 7fc283e16800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.992+0000 7fc283e16800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:05.992 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.993+0000 7fc135fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:05.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.996+0000 7fc283e16800 20 remove_watcher() i=0 2024-04-22T20:10:05.995 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.996+0000 7fc283e16800 2 removed watcher, disabling cache 2024-04-22T20:10:05.996 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.997+0000 7fc283e16800 20 remove_watcher() i=1 2024-04-22T20:10:05.997 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:05.998+0000 7fc283e16800 20 remove_watcher() i=2 2024-04-22T20:10:05.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.000+0000 7fc283e16800 20 remove_watcher() i=3 2024-04-22T20:10:06.000 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.001+0000 7fc283e16800 20 remove_watcher() i=4 2024-04-22T20:10:06.001 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.002+0000 7fc283e16800 20 remove_watcher() i=5 2024-04-22T20:10:06.002 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.003+0000 7fc283e16800 20 remove_watcher() i=6 2024-04-22T20:10:06.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.004+0000 7fc283e16800 20 remove_watcher() i=7 2024-04-22T20:10:06.012 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:06.012 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:06.012 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-do-reshard id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:10:06.013 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:06.014 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at do_reshard 2024-04-22T20:10:06.046 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:06.046 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:06.080 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.081+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.082+0000 7f0d9cb27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.082+0000 7f0d9cb27800 20 realm 2024-04-22T20:10:06.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.082+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.082+0000 7f0d9cb27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.082+0000 7f0d9cb27800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:06.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.082+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.083+0000 7f0d9cb27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.083+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.083+0000 7f0d9cb27800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:06.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.083+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.084+0000 7f0d9cb27800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:06.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.084+0000 7f0d9cb27800 20 searching for the correct realm 2024-04-22T20:10:06.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.087+0000 7f0d99061640 0 -- 172.21.15.136:0/2942437370 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f0d94033b90 msgr2=0x7f0d94011960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:06.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.090+0000 7f0d93fff640 0 -- 172.21.15.136:0/2942437370 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f0d9402a410 msgr2=0x7f0d9402a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.296+0000 7f0d9cb27800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.296+0000 7f0d9cb27800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.296+0000 7f0d9cb27800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.296+0000 7f0d9cb27800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.296+0000 7f0d9cb27800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.296+0000 7f0d9cb27800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.296+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.297+0000 7f0d9cb27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.296 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.297+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.297+0000 7f0d9cb27800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.297+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.297+0000 7f0d9cb27800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 zone default found 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 4 Realm: () 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 zonegroup default 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.298 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 started sync module instance, tier type = 2024-04-22T20:10:06.299 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.298+0000 7f0d9cb27800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:06.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.308+0000 7f0d9cb27800 20 add_watcher() i=0 2024-04-22T20:10:06.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.308+0000 7f0d9cb27800 20 add_watcher() i=1 2024-04-22T20:10:06.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.308+0000 7f0d9cb27800 20 add_watcher() i=2 2024-04-22T20:10:06.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.308+0000 7f0d9cb27800 20 add_watcher() i=3 2024-04-22T20:10:06.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.308+0000 7f0d9cb27800 20 add_watcher() i=4 2024-04-22T20:10:06.308 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.308+0000 7f0d9cb27800 20 add_watcher() i=5 2024-04-22T20:10:06.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.310+0000 7f0d9cb27800 20 add_watcher() i=6 2024-04-22T20:10:06.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.310+0000 7f0d9cb27800 20 add_watcher() i=7 2024-04-22T20:10:06.309 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.310+0000 7f0d9cb27800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:06.311 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.311+0000 7f0c4efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0d9cb27800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0d9cb27800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0d9cb27800 5 note: GC not initialized 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0c4dfe3640 20 reqs_thread_entry: start 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0d9cb27800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0d9cb27800 20 Filter name: none 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0bbf7fe640 20 reqs_thread_entry: start 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0d9cb27800 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.312+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:06.312 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-04-22T20:10:06.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-04-22T20:10:06.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 : miss 2024-04-22T20:10:06.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:06.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:06.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:06.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:06.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:06.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.313+0000 7f0d9cb27800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 2024-04-22T20:10:06.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.317+0000 7f0d9cb27800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:06.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.317+0000 7f0d9cb27800 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:06.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.317+0000 7f0d9cb27800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:06.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.317+0000 7f0d9cb27800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27, ofs0, ns] 2024-04-22T20:10:06.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.317+0000 7f0c6ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562474 cookie 94473904311440 notifier 6918 bl.length()=903 2024-04-22T20:10:06.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.317+0000 7f0c6ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:06.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.317+0000 7f0c6ffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:06.317 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.317+0000 7f0c6ffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:06.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.319+0000 7f0d9cb27800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): orig_sources= new_sources= 2024-04-22T20:10:06.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.319+0000 7f0d9cb27800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): potential sources added=[] removed=[] 2024-04-22T20:10:06.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.319+0000 7f0d9cb27800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): orig_dests= new_dests= 2024-04-22T20:10:06.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.319+0000 7f0d9cb27800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): potential dests added=[] removed=[] 2024-04-22T20:10:06.319 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.320+0000 7f0d9cb27800 -1 Injecting error=-5 at location=do_reshard 2024-04-22T20:10:06.321 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.322+0000 7f0d9cb27800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 : hit (requested=0x17, cached=0x17) 2024-04-22T20:10:06.321 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.322+0000 7f0d9cb27800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 2024-04-22T20:10:06.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.324+0000 7f0d9cb27800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:06.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.324+0000 7f0d9cb27800 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:06.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.324+0000 7f0d9cb27800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:06.323 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.324+0000 7f0d9cb27800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27, ofs0, ns] 2024-04-22T20:10:06.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.324+0000 7f0c6ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562475 cookie 94473904311440 notifier 6918 bl.length()=871 2024-04-22T20:10:06.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.324+0000 7f0c6ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:06.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.324+0000 7f0c6ffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:06.324 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.324+0000 7f0c6ffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:06.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.326+0000 7f0d9cb27800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): orig_sources= new_sources= 2024-04-22T20:10:06.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.326+0000 7f0d9cb27800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): potential sources added=[] removed=[] 2024-04-22T20:10:06.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.326+0000 7f0d9cb27800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): orig_dests= new_dests= 2024-04-22T20:10:06.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.326+0000 7f0d9cb27800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): potential dests added=[] removed=[] 2024-04-22T20:10:06.325 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.326+0000 7f0d9cb27800 1 execute INFO: reshard of bucket "error-at-do-reshard" canceled due to errors 2024-04-22T20:10:06.327 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.328+0000 7f0c4efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:06.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.330+0000 7f0d9cb27800 20 remove_watcher() i=0 2024-04-22T20:10:06.329 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.330+0000 7f0d9cb27800 2 removed watcher, disabling cache 2024-04-22T20:10:06.331 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.331+0000 7f0d9cb27800 20 remove_watcher() i=1 2024-04-22T20:10:06.332 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.333+0000 7f0d9cb27800 20 remove_watcher() i=2 2024-04-22T20:10:06.333 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.334+0000 7f0d9cb27800 20 remove_watcher() i=3 2024-04-22T20:10:06.334 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.335+0000 7f0d9cb27800 20 remove_watcher() i=4 2024-04-22T20:10:06.335 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.336+0000 7f0d9cb27800 20 remove_watcher() i=5 2024-04-22T20:10:06.336 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.337+0000 7f0d9cb27800 20 remove_watcher() i=6 2024-04-22T20:10:06.337 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.338+0000 7f0d9cb27800 20 remove_watcher() i=7 2024-04-22T20:10:06.348 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-04-22T20:10:06.376 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:06.376 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:06.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.410+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.411+0000 7fc29972b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.411+0000 7fc29972b800 20 realm 2024-04-22T20:10:06.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.411+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.412+0000 7fc29972b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.412+0000 7fc29972b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:06.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.412+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.412+0000 7fc29972b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.412+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.413+0000 7fc29972b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:06.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.413+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.413+0000 7fc29972b800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:06.413 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.413+0000 7fc29972b800 20 searching for the correct realm 2024-04-22T20:10:06.414 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.415+0000 7fc295cf0640 0 -- 172.21.15.136:0/2341138267 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55fe6aa6a6d0 msgr2=0x55fe6aa6cac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:06.418 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.419+0000 7fc295cf0640 0 -- 172.21.15.136:0/2341138267 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7fc16007ba30 msgr2=0x7fc16007de60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:06.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:06.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:06.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:06.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:06.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:06.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:06.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:06.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 zone default found 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 4 Realm: () 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 zonegroup default 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.424+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.425+0000 7fc29972b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.425+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.425+0000 7fc29972b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.425+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.426+0000 7fc29972b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.426+0000 7fc29972b800 20 started sync module instance, tier type = 2024-04-22T20:10:06.426 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.426+0000 7fc29972b800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:06.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.435+0000 7fc29972b800 20 add_watcher() i=0 2024-04-22T20:10:06.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.435+0000 7fc29972b800 20 add_watcher() i=1 2024-04-22T20:10:06.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.435+0000 7fc29972b800 20 add_watcher() i=2 2024-04-22T20:10:06.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.435+0000 7fc29972b800 20 add_watcher() i=3 2024-04-22T20:10:06.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.435+0000 7fc29972b800 20 add_watcher() i=4 2024-04-22T20:10:06.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.435+0000 7fc29972b800 20 add_watcher() i=5 2024-04-22T20:10:06.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.435+0000 7fc29972b800 20 add_watcher() i=6 2024-04-22T20:10:06.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.436+0000 7fc29972b800 20 add_watcher() i=7 2024-04-22T20:10:06.435 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.436+0000 7fc29972b800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:06.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.437+0000 7fc14b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:06.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.438+0000 7fc29972b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:06.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.438+0000 7fc29972b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:06.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.438+0000 7fc29972b800 5 note: GC not initialized 2024-04-22T20:10:06.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.438+0000 7fc14a7e4640 20 reqs_thread_entry: start 2024-04-22T20:10:06.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.438+0000 7fc29972b800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:06.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.438+0000 7fc29972b800 20 Filter name: none 2024-04-22T20:10:06.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.438+0000 7fc0bbfff640 20 reqs_thread_entry: start 2024-04-22T20:10:06.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.438+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.439+0000 7fc29972b800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.439+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.439+0000 7fc29972b800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.439+0000 7fc29972b800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:06.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.439+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.439+0000 7fc29972b800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.439+0000 7fc29972b800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.440+0000 7fc29972b800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.440+0000 7fc29972b800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:06.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.441+0000 7fc14b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:06.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.443+0000 7fc29972b800 20 remove_watcher() i=0 2024-04-22T20:10:06.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.443+0000 7fc29972b800 2 removed watcher, disabling cache 2024-04-22T20:10:06.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.444+0000 7fc29972b800 20 remove_watcher() i=1 2024-04-22T20:10:06.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.445+0000 7fc29972b800 20 remove_watcher() i=2 2024-04-22T20:10:06.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.446+0000 7fc29972b800 20 remove_watcher() i=3 2024-04-22T20:10:06.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.447+0000 7fc29972b800 20 remove_watcher() i=4 2024-04-22T20:10:06.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.448+0000 7fc29972b800 20 remove_watcher() i=5 2024-04-22T20:10:06.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.449+0000 7fc29972b800 20 remove_watcher() i=6 2024-04-22T20:10:06.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.450+0000 7fc29972b800 20 remove_watcher() i=7 2024-04-22T20:10:06.458 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:06.458 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:06.458 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-do-reshard id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:10:06.458 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:06.460 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:06.460 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:06.492 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:06.492 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:06.549 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:10:06.579 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:06.579 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:06.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.614+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.615+0000 7f1369415800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.615+0000 7f1369415800 20 realm 2024-04-22T20:10:06.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.615+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.615+0000 7f1369415800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.615+0000 7f1369415800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:06.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.615+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.615+0000 7f1369415800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:06.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.615+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.616+0000 7f1369415800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:06.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.616+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:06.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.617+0000 7f1369415800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:06.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.617+0000 7f1369415800 20 searching for the correct realm 2024-04-22T20:10:06.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.619+0000 7f13651dd640 0 -- 172.21.15.136:0/2913738584 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f136002d040 msgr2=0x7f1360030050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:06.824 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:06.824+0000 7f135ffff640 0 -- 172.21.15.136:0/2913738584 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f1360053400 msgr2=0x7f1360053880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:07.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:07.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:07.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:07.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:07.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:07.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:07.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.031 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:07.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.031+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:07.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 20 zone default found 2024-04-22T20:10:07.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 4 Realm: () 2024-04-22T20:10:07.032 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 20 zonegroup default 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.032+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.033+0000 7f1369415800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.033+0000 7f1369415800 20 started sync module instance, tier type = 2024-04-22T20:10:07.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.033+0000 7f1369415800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:07.041 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.042+0000 7f1369415800 20 add_watcher() i=0 2024-04-22T20:10:07.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.042+0000 7f1369415800 20 add_watcher() i=1 2024-04-22T20:10:07.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.042+0000 7f1369415800 20 add_watcher() i=2 2024-04-22T20:10:07.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.042+0000 7f1369415800 20 add_watcher() i=3 2024-04-22T20:10:07.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.042+0000 7f1369415800 20 add_watcher() i=4 2024-04-22T20:10:07.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.042+0000 7f1369415800 20 add_watcher() i=5 2024-04-22T20:10:07.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.042+0000 7f1369415800 20 add_watcher() i=6 2024-04-22T20:10:07.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.043+0000 7f1369415800 20 add_watcher() i=7 2024-04-22T20:10:07.042 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.043+0000 7f1369415800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:07.044 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f121afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:07.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f1369415800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:07.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f1369415800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:07.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f1369415800 5 note: GC not initialized 2024-04-22T20:10:07.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f1219fe3640 20 reqs_thread_entry: start 2024-04-22T20:10:07.045 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f1369415800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f1369415800 20 Filter name: none 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f118b7fe640 20 reqs_thread_entry: start 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f1369415800 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.045+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 : miss 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:07.046 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.046+0000 7f1369415800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 2024-04-22T20:10:07.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.050+0000 7f1369415800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:07.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.050+0000 7f1369415800 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:07.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.050+0000 7f1369415800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:07.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.050+0000 7f1369415800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27, ofs0, ns] 2024-04-22T20:10:07.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.051+0000 7f123bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562476 cookie 94819962673664 notifier 6954 bl.length()=903 2024-04-22T20:10:07.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.051+0000 7f123bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:07.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.051+0000 7f123bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:07.050 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.051+0000 7f123bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:07.051 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.052+0000 7f1369415800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): orig_sources= new_sources= 2024-04-22T20:10:07.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.052+0000 7f1369415800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): potential sources added=[] removed=[] 2024-04-22T20:10:07.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.052+0000 7f1369415800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): orig_dests= new_dests= 2024-04-22T20:10:07.052 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.052+0000 7f1369415800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): potential dests added=[] removed=[] 2024-04-22T20:10:07.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.053+0000 7f1369415800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27.1.0 2024-04-22T20:10:07.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.053+0000 7f1369415800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27.1.0 2024-04-22T20:10:07.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.053+0000 7f1369415800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27.1.1 2024-04-22T20:10:07.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.053+0000 7f1369415800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27.1.1 2024-04-22T20:10:07.053 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.053+0000 7f1369415800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27.0 2024-04-22T20:10:07.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.056+0000 7f1369415800 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 : hit (requested=0x17, cached=0x17) 2024-04-22T20:10:07.055 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.056+0000 7f1369415800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 2024-04-22T20:10:07.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.057+0000 7f1369415800 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:07.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.057+0000 7f1369415800 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:07.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.057+0000 7f1369415800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:07.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.057+0000 7f1369415800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27, ofs0, ns] 2024-04-22T20:10:07.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.058+0000 7f123b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562477 cookie 94819962673664 notifier 6954 bl.length()=871 2024-04-22T20:10:07.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.058+0000 7f123b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 info.flags=0x17 2024-04-22T20:10:07.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.058+0000 7f123b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 to cache LRU end 2024-04-22T20:10:07.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.058+0000 7f123b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:07.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.059+0000 7f1369415800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): orig_sources= new_sources= 2024-04-22T20:10:07.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.059+0000 7f1369415800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): potential sources added=[] removed=[] 2024-04-22T20:10:07.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.059+0000 7f1369415800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): orig_dests= new_dests= 2024-04-22T20:10:07.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.059+0000 7f1369415800 20 handle_bi_update(): bucket=:error-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27]): potential dests added=[] removed=[] 2024-04-22T20:10:07.059 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.060+0000 7f1369415800 1 execute INFO: reshard of bucket "error-at-do-reshard" completed successfully 2024-04-22T20:10:07.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.062+0000 7f121afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:07.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.064+0000 7f1369415800 20 remove_watcher() i=0 2024-04-22T20:10:07.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.064+0000 7f1369415800 2 removed watcher, disabling cache 2024-04-22T20:10:07.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.065+0000 7f1369415800 20 remove_watcher() i=1 2024-04-22T20:10:07.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.066+0000 7f1369415800 20 remove_watcher() i=2 2024-04-22T20:10:07.066 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.067+0000 7f1369415800 20 remove_watcher() i=3 2024-04-22T20:10:07.067 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.068+0000 7f1369415800 20 remove_watcher() i=4 2024-04-22T20:10:07.068 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.069+0000 7f1369415800 20 remove_watcher() i=5 2024-04-22T20:10:07.069 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.070+0000 7f1369415800 20 remove_watcher() i=6 2024-04-22T20:10:07.070 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.071+0000 7f1369415800 20 remove_watcher() i=7 2024-04-22T20:10:07.081 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-04-22T20:10:07.108 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:07.108 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:07.142 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.143+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.144+0000 7ff197c3d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.144+0000 7ff197c3d800 20 realm 2024-04-22T20:10:07.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.144+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.145+0000 7ff197c3d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.145+0000 7ff197c3d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:07.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.145+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.145+0000 7ff197c3d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.144 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.145+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.146+0000 7ff197c3d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:07.145 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.146+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.146+0000 7ff197c3d800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:07.146 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.146+0000 7ff197c3d800 20 searching for the correct realm 2024-04-22T20:10:07.147 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.148+0000 7ff195f3d640 0 -- 172.21.15.136:0/2592759679 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7ff19002abb0 msgr2=0x7ff1900293a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:07.353 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.354+0000 7ff18ffff640 0 -- 172.21.15.136:0/2592759679 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7ff190045a60 msgr2=0x7ff1900363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:07.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:07.559 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:07.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:07.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:07.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:07.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:07.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.560+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:07.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:07.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 20 zone default found 2024-04-22T20:10:07.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 4 Realm: () 2024-04-22T20:10:07.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 20 zonegroup default 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.561+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.562+0000 7ff197c3d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.562+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.562+0000 7ff197c3d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.562+0000 7ff197c3d800 20 started sync module instance, tier type = 2024-04-22T20:10:07.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.562+0000 7ff197c3d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:07.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.571+0000 7ff197c3d800 20 add_watcher() i=0 2024-04-22T20:10:07.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.571+0000 7ff197c3d800 20 add_watcher() i=1 2024-04-22T20:10:07.570 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.571+0000 7ff197c3d800 20 add_watcher() i=2 2024-04-22T20:10:07.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.571+0000 7ff197c3d800 20 add_watcher() i=3 2024-04-22T20:10:07.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.571+0000 7ff197c3d800 20 add_watcher() i=4 2024-04-22T20:10:07.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.571+0000 7ff197c3d800 20 add_watcher() i=5 2024-04-22T20:10:07.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.571+0000 7ff197c3d800 20 add_watcher() i=6 2024-04-22T20:10:07.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.572+0000 7ff197c3d800 20 add_watcher() i=7 2024-04-22T20:10:07.571 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.572+0000 7ff197c3d800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:07.573 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7ff04b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:07.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7ff197c3d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:07.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7ff197c3d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:07.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7ff197c3d800 5 note: GC not initialized 2024-04-22T20:10:07.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7ff04a7e4640 20 reqs_thread_entry: start 2024-04-22T20:10:07.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7ff197c3d800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:07.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7ff197c3d800 20 Filter name: none 2024-04-22T20:10:07.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7fefbbfff640 20 reqs_thread_entry: start 2024-04-22T20:10:07.574 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.574+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.575+0000 7ff197c3d800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:07.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.575+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.575+0000 7ff197c3d800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:07.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.575+0000 7ff197c3d800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:07.575 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.575+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.576+0000 7ff197c3d800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:07.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.576+0000 7ff197c3d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:07.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.576+0000 7ff197c3d800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:07.576 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.576+0000 7ff197c3d800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:07.577 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.578+0000 7ff04b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:07.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.580+0000 7ff197c3d800 20 remove_watcher() i=0 2024-04-22T20:10:07.579 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.580+0000 7ff197c3d800 2 removed watcher, disabling cache 2024-04-22T20:10:07.580 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.581+0000 7ff197c3d800 20 remove_watcher() i=1 2024-04-22T20:10:07.581 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.582+0000 7ff197c3d800 20 remove_watcher() i=2 2024-04-22T20:10:07.582 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.583+0000 7ff197c3d800 20 remove_watcher() i=3 2024-04-22T20:10:07.583 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.584+0000 7ff197c3d800 20 remove_watcher() i=4 2024-04-22T20:10:07.584 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.585+0000 7ff197c3d800 20 remove_watcher() i=5 2024-04-22T20:10:07.585 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.586+0000 7ff197c3d800 20 remove_watcher() i=6 2024-04-22T20:10:07.586 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:07.587+0000 7ff197c3d800 20 remove_watcher() i=7 2024-04-22T20:10:07.595 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:07.595 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:07.596 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: error-at-do-reshard id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-22T20:10:07.596 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:07.597 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:07.598 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:07.618 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:07.618 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:07.834 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:07.835 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:07.902 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: reshard bucket with abort at do_reshard 2024-04-22T20:10:07.902 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:07.904 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:07.904 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:07.946 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:07.947 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.003 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.003 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.036 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.037 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.072 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.072 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.108 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.108 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.142 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.142 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.177 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.177 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.212 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.212 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.246 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.247 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.282 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.282 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.317 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.317 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.354 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.354 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.388 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.388 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.422 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.422 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.457 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.457 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.490 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.491 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.526 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.527 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.562 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.562 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.596 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.596 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.631 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.631 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.666 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:08.666 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:08.700 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-04-22T20:10:08.730 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:08.730 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:08.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.763+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:08.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.765+0000 7f2bc0294800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:08.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.765+0000 7f2bc0294800 20 realm 2024-04-22T20:10:08.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.765+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:08.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.765+0000 7f2bc0294800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:08.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.765+0000 7f2bc0294800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:08.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.765+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:08.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.765+0000 7f2bc0294800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:08.765 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.765+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:08.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.766+0000 7f2bc0294800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:08.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.766+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:08.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.767+0000 7f2bc0294800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:08.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.767+0000 7f2bc0294800 20 searching for the correct realm 2024-04-22T20:10:08.768 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.769+0000 7f2bbe594640 0 -- 172.21.15.136:0/226531718 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f2bb8088070 msgr2=0x7f2bb8051a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:08.973 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:08.974+0000 7f2bb5592640 0 -- 172.21.15.136:0/226531718 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f2bb80096b0 msgr2=0x7f2bb805c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.180+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:09.180 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 20 zone default found 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 4 Realm: () 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 20 zonegroup default 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.181+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.182 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.182+0000 7f2bc0294800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.182+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.182+0000 7f2bc0294800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.182+0000 7f2bc0294800 20 started sync module instance, tier type = 2024-04-22T20:10:09.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.182+0000 7f2bc0294800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:09.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 20 add_watcher() i=0 2024-04-22T20:10:09.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 20 add_watcher() i=1 2024-04-22T20:10:09.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 20 add_watcher() i=2 2024-04-22T20:10:09.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 20 add_watcher() i=3 2024-04-22T20:10:09.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 20 add_watcher() i=4 2024-04-22T20:10:09.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 20 add_watcher() i=5 2024-04-22T20:10:09.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 20 add_watcher() i=6 2024-04-22T20:10:09.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 20 add_watcher() i=7 2024-04-22T20:10:09.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.191+0000 7f2bc0294800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:09.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.193+0000 7f2a6f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:09.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.193+0000 7f2bc0294800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:09.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.193+0000 7f2bc0294800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:09.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.194+0000 7f2bc0294800 5 note: GC not initialized 2024-04-22T20:10:09.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.194+0000 7f2a6e7fc640 20 reqs_thread_entry: start 2024-04-22T20:10:09.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.194+0000 7f2bc0294800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:09.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.194+0000 7f2bc0294800 20 Filter name: none 2024-04-22T20:10:09.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.194+0000 7f29dffff640 20 reqs_thread_entry: start 2024-04-22T20:10:09.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.194+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.195+0000 7f2bc0294800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:09.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.195+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.196+0000 7f2bc0294800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:09.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.196+0000 7f2bc0294800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:09.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.196+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.196+0000 7f2bc0294800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:09.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.196+0000 7f2bc0294800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.196+0000 7f2bc0294800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:09.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.196+0000 7f2bc0294800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:09.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.198+0000 7f2a6f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:09.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.200+0000 7f2bc0294800 20 remove_watcher() i=0 2024-04-22T20:10:09.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.200+0000 7f2bc0294800 2 removed watcher, disabling cache 2024-04-22T20:10:09.200 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.201+0000 7f2bc0294800 20 remove_watcher() i=1 2024-04-22T20:10:09.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.202+0000 7f2bc0294800 20 remove_watcher() i=2 2024-04-22T20:10:09.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.203+0000 7f2bc0294800 20 remove_watcher() i=3 2024-04-22T20:10:09.203 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.204+0000 7f2bc0294800 20 remove_watcher() i=4 2024-04-22T20:10:09.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.205+0000 7f2bc0294800 20 remove_watcher() i=5 2024-04-22T20:10:09.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.206+0000 7f2bc0294800 20 remove_watcher() i=6 2024-04-22T20:10:09.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.207+0000 7f2bc0294800 20 remove_watcher() i=7 2024-04-22T20:10:09.215 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.215 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:09.215 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-do-reshard id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:10:09.215 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.217 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2024-04-22T20:10:09.250 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:09.250 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:09.283 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.284+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.284 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.285+0000 7fd358d35800 20 realm 2024-04-22T20:10:09.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.285+0000 7fd358d35800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:09.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.286+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:09.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.286+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.287+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:09.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.287+0000 7fd358d35800 20 searching for the correct realm 2024-04-22T20:10:09.288 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.289+0000 7fd3551b6640 0 -- 172.21.15.136:0/700908430 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fd350018040 msgr2=0x7fd350024400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:09.494 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.494+0000 7fd34c1b4640 0 -- 172.21.15.136:0/700908430 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fd350024b00 msgr2=0x7fd350056910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:09.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:09.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:09.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:09.700 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:09.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:09.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:09.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:09.701 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 20 zone default found 2024-04-22T20:10:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 4 Realm: () 2024-04-22T20:10:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:09.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 20 zonegroup default 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.703+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.703+0000 7fd358d35800 20 started sync module instance, tier type = 2024-04-22T20:10:09.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.703+0000 7fd358d35800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:09.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=0 2024-04-22T20:10:09.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=1 2024-04-22T20:10:09.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=2 2024-04-22T20:10:09.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=3 2024-04-22T20:10:09.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=4 2024-04-22T20:10:09.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=5 2024-04-22T20:10:09.712 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=6 2024-04-22T20:10:09.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.714+0000 7fd358d35800 20 add_watcher() i=7 2024-04-22T20:10:09.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.714+0000 7fd358d35800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:09.714 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.715+0000 7fd20afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:09.715 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd358d35800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:09.715 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd358d35800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:09.715 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd358d35800 5 note: GC not initialized 2024-04-22T20:10:09.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd209fe3640 20 reqs_thread_entry: start 2024-04-22T20:10:09.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd358d35800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:09.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd358d35800 20 Filter name: none 2024-04-22T20:10:09.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd17b7fe640 20 reqs_thread_entry: start 2024-04-22T20:10:09.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd358d35800 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-04-22T20:10:09.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.716+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 : miss 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.717+0000 7fd358d35800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 2024-04-22T20:10:09.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.721+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.721+0000 7fd358d35800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.720 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.721+0000 7fd358d35800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.721+0000 7fd358d35800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28, ofs0, ns] 2024-04-22T20:10:09.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562387 cookie 93966328238640 notifier 6632 bl.length()=903 2024-04-22T20:10:09.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_sources= new_sources= 2024-04-22T20:10:09.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential sources added=[] removed=[] 2024-04-22T20:10:09.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_dests= new_dests= 2024-04-22T20:10:09.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential dests added=[] removed=[] 2024-04-22T20:10:09.724 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fd358d35800 time 2024-04-22T20:10:09.725584+0000 2024-04-22T20:10:09.724 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd35a381f5c] 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd35a38211a] 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55763bf0edc1] 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x324) [0x55763bf1ce84] 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fd359229590] 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr:*** Caught signal (Aborted) ** 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fd358d35800 thread_name:radosgw-admin 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.725+0000 7fd358d35800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fd358d35800 time 2024-04-22T20:10:09.725584+0000 2024-04-22T20:10:09.725 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd35a381f5c] 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd35a38211a] 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55763bf0edc1] 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x324) [0x55763bf1ce84] 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fd359229590] 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:10:09.726 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd35923e6f0] 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd35928b94c] 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd35a381fb6] 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd35a38211a] 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55763bf0edc1] 2024-04-22T20:10:09.750 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x324) [0x55763bf1ce84] 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fd359229590] 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.726+0000 7fd358d35800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fd358d35800 thread_name:radosgw-admin 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd35923e6f0] 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd35928b94c] 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd35a381fb6] 2024-04-22T20:10:09.751 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd35a38211a] 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55763bf0edc1] 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x324) [0x55763bf1ce84] 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fd359229590] 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr:--- begin dump of recent events --- 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: -499> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr: -498> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: dump: 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.752 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -497> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -496> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -495> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -494> 2024-04-22T20:10:09.268+0000 7fd358d35800 5 monclient: authenticate success, global_id 6996 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -493> 2024-04-22T20:10:09.268+0000 7fd358d35800 10 monclient: get_monmap_and_config success 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -492> 2024-04-22T20:10:09.268+0000 7fd358d35800 4 set_mon_vals no callback set 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -491> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -490> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr: -489> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: dump: 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.753 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr: -488> 2024-04-22T20:10:09.268+0000 7fd358d35800 10 monclient: shutdown 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr: -487> 2024-04-22T20:10:09.268+0000 7fd358d35800 5 asok(0x55763d6b5d00) unregister_commands rotate-key 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr: -486> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: build_initial_monmap 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr: -485> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 build_initial for_mkfs: 0 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr: -484> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: monmap: 2024-04-22T20:10:09.754 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr: -483> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding auth protocol: cephx 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr: -482> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding auth protocol: cephx 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr: -481> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding auth protocol: cephx 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr: -480> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr: -479> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.755 INFO:tasks.workunit.client.0.smithi136.stderr: -478> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -477> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -476> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -475> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -474> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -473> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -472> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -471> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -470> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -469> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -468> 2024-04-22T20:10:09.269+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -467> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 librados: starting msgr at 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -466> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 librados: starting objecter 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -465> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command objecter_requests hook 0x55763d8dc710 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -464> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 librados: setting wanted keys 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -463> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 librados: calling monclient init 2024-04-22T20:10:09.756 INFO:tasks.workunit.client.0.smithi136.stderr: -462> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: init 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -461> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding auth protocol: cephx 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -460> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding auth protocol: cephx 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -459> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding auth protocol: cephx 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -458> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -457> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -456> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -455> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -454> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -453> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -452> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -451> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -450> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.757 INFO:tasks.workunit.client.0.smithi136.stderr: -449> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -448> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -447> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -446> 2024-04-22T20:10:09.269+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -445> 2024-04-22T20:10:09.269+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -444> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command rotate-key hook 0x55763d8d6a48 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -443> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: _reopen_session rank -1 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -442> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -441> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-a con 0x55763d78d460 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -440> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-c con 0x55763d81d410 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -439> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-b con 0x55763d90bd00 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -438> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -437> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.758 INFO:tasks.workunit.client.0.smithi136.stderr: -436> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -435> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): _renew_subs 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -434> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): authenticate will time out at 1998.620860s 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -433> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request con 0x55763d81d410 auth_method 0 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -432> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -431> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -430> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -429> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): get_auth_request con 0x55763d78d460 auth_method 0 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -428> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -427> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -426> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -425> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -424> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -423> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -422> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -421> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.759 INFO:tasks.workunit.client.0.smithi136.stderr: -420> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -419> 2024-04-22T20:10:09.270+0000 7fd3551b6640 10 monclient(hunting): get_auth_request con 0x55763d90bd00 auth_method 0 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -418> 2024-04-22T20:10:09.270+0000 7fd3551b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -417> 2024-04-22T20:10:09.270+0000 7fd3551b6640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -416> 2024-04-22T20:10:09.270+0000 7fd3551b6640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -415> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): handle_auth_done global_id 6999 payload 824 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -414> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient: _finish_hunting 0 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -413> 2024-04-22T20:10:09.270+0000 7fd3549b5640 1 monclient: found mon.noname-a 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -412> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -411> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -410> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr: -409> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: dump: 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.760 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr: -408> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr: -407> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr: -406> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr: -405> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr: -404> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr: -403> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: dump: 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.761 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -402> 2024-04-22T20:10:09.270+0000 7fd358d35800 5 monclient: authenticate success, global_id 6999 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -401> 2024-04-22T20:10:09.270+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -400> 2024-04-22T20:10:09.270+0000 7fd242e0d640 4 set_mon_vals no callback set 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -399> 2024-04-22T20:10:09.270+0000 7fd358d35800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -398> 2024-04-22T20:10:09.270+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -397> 2024-04-22T20:10:09.270+0000 7fd358d35800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -396> 2024-04-22T20:10:09.270+0000 7fd358d35800 1 librados: init done 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -395> 2024-04-22T20:10:09.272+0000 7fd24160a640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:10:09.762 INFO:tasks.workunit.client.0.smithi136.stderr: -394> 2024-04-22T20:10:09.272+0000 7fd24160a640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -393> 2024-04-22T20:10:09.272+0000 7fd24160a640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -392> 2024-04-22T20:10:09.272+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x7fd23c06b2d0 auth_method 0 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -391> 2024-04-22T20:10:09.272+0000 7fd3551b6640 10 monclient: get_auth_request con 0x55763d904f60 auth_method 0 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -390> 2024-04-22T20:10:09.273+0000 7fd3549b5640 10 monclient: get_auth_request con 0x55763d922890 auth_method 0 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -389> 2024-04-22T20:10:09.274+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x55763d926a80 auth_method 0 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -388> 2024-04-22T20:10:09.275+0000 7fd3551b6640 10 monclient: get_auth_request con 0x55763d926140 auth_method 0 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -387> 2024-04-22T20:10:09.275+0000 7fd358d35800 10 monclient: build_initial_monmap 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -386> 2024-04-22T20:10:09.275+0000 7fd358d35800 1 build_initial for_mkfs: 0 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr: -385> 2024-04-22T20:10:09.275+0000 7fd358d35800 10 monclient: monmap: 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:10:09.763 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -384> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding auth protocol: cephx 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -383> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding auth protocol: cephx 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -382> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding auth protocol: cephx 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -381> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -380> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -379> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -378> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -377> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -376> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -375> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -374> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.764 INFO:tasks.workunit.client.0.smithi136.stderr: -373> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -372> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -371> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -370> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -369> 2024-04-22T20:10:09.276+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -368> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command objecter_requests cmddesc objecter_requests hook 0x55763d960850 EEXIST 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -367> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient: init 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -366> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding auth protocol: cephx 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -365> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding auth protocol: cephx 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -364> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding auth protocol: cephx 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -363> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -362> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -361> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -360> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.765 INFO:tasks.workunit.client.0.smithi136.stderr: -359> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -358> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -357> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -356> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -355> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -354> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -353> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -352> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -351> 2024-04-22T20:10:09.276+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -350> 2024-04-22T20:10:09.276+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -349> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command rotate-key cmddesc rotate-key hook 0x55763d92db40 EEXIST 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -348> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient: _reopen_session rank -1 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -347> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -346> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-a con 0x55763d958d90 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:10:09.766 INFO:tasks.workunit.client.0.smithi136.stderr: -345> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-c con 0x55763d959170 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -344> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-b con 0x55763d96ee20 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -343> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -342> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -341> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -340> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): _renew_subs 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -339> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): authenticate will time out at 1998.627663s 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -338> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request con 0x55763d958d90 auth_method 0 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -337> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -336> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -335> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -334> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): get_auth_request con 0x55763d959170 auth_method 0 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -333> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -332> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -331> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.767 INFO:tasks.workunit.client.0.smithi136.stderr: -330> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -329> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -328> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -327> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -326> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -325> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -324> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): get_auth_request con 0x55763d96ee20 auth_method 0 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -323> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -322> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -321> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -320> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -319> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -318> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -317> 2024-04-22T20:10:09.277+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_done global_id 7002 payload 824 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -316> 2024-04-22T20:10:09.277+0000 7fd34c1b4640 10 monclient: _finish_hunting 0 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -315> 2024-04-22T20:10:09.277+0000 7fd34c1b4640 1 monclient: found mon.noname-a 2024-04-22T20:10:09.768 INFO:tasks.workunit.client.0.smithi136.stderr: -314> 2024-04-22T20:10:09.277+0000 7fd34c1b4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr: -313> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr: -312> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr: -311> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: dump: 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr: -310> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.769 INFO:tasks.workunit.client.0.smithi136.stderr: -309> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr: -308> 2024-04-22T20:10:09.277+0000 7fd358d35800 5 monclient: authenticate success, global_id 7002 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr: -307> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr: -306> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr: -305> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr: -304> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: dump: 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.770 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -303> 2024-04-22T20:10:09.277+0000 7fd3559b7640 4 set_mon_vals no callback set 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -302> 2024-04-22T20:10:09.277+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -301> 2024-04-22T20:10:09.277+0000 7fd358d35800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -300> 2024-04-22T20:10:09.277+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -299> 2024-04-22T20:10:09.277+0000 7fd358d35800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -298> 2024-04-22T20:10:09.279+0000 7fd23a7fc640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -297> 2024-04-22T20:10:09.279+0000 7fd23a7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -296> 2024-04-22T20:10:09.279+0000 7fd23a7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -295> 2024-04-22T20:10:09.279+0000 7fd358d35800 10 monclient: get_monmap_and_config 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -294> 2024-04-22T20:10:09.279+0000 7fd358d35800 10 monclient: build_initial_monmap 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -293> 2024-04-22T20:10:09.279+0000 7fd358d35800 1 build_initial for_mkfs: 0 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr: -292> 2024-04-22T20:10:09.279+0000 7fd358d35800 10 monclient: monmap: 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:10:09.771 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: -291> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding auth protocol: cephx 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: -290> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding auth protocol: cephx 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: -289> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding auth protocol: cephx 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: -288> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: -287> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: -286> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: -285> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.772 INFO:tasks.workunit.client.0.smithi136.stderr: -284> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -283> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -282> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -281> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -280> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -279> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -278> 2024-04-22T20:10:09.279+0000 7fd3551b6640 10 monclient: get_auth_request con 0x7fd22c06b1e0 auth_method 0 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -277> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -276> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -275> 2024-04-22T20:10:09.279+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -274> 2024-04-22T20:10:09.279+0000 7fd358d35800 10 monclient: init 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -273> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding auth protocol: cephx 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -272> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding auth protocol: cephx 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -271> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding auth protocol: cephx 2024-04-22T20:10:09.773 INFO:tasks.workunit.client.0.smithi136.stderr: -270> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -269> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -268> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -267> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -266> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -265> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -264> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -263> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -262> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -261> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -260> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -259> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -258> 2024-04-22T20:10:09.280+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.774 INFO:tasks.workunit.client.0.smithi136.stderr: -257> 2024-04-22T20:10:09.280+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -256> 2024-04-22T20:10:09.280+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command rotate-key cmddesc rotate-key hook 0x7fff78f55608 EEXIST 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -255> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient: _reopen_session rank -1 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -254> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient: _add_conns ranks=[2,0,1] 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -253> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-c con 0x55763d9ab1a0 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -252> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-a con 0x55763d9ab580 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -251> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-b con 0x55763d9a5410 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -250> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -249> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -248> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -247> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): _renew_subs 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -246> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): authenticate will time out at 1998.631194s 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -245> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request con 0x55763d9ab1a0 auth_method 0 2024-04-22T20:10:09.775 INFO:tasks.workunit.client.0.smithi136.stderr: -244> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -243> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -242> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -241> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): get_auth_request con 0x55763d9ab580 auth_method 0 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -240> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -239> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -238> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -237> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -236> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -235> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -234> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -233> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -232> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -231> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_done global_id 6629 payload 274 2024-04-22T20:10:09.776 INFO:tasks.workunit.client.0.smithi136.stderr: -230> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient: _finish_hunting 0 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr: -229> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 1 monclient: found mon.noname-c 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr: -228> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr: -227> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr: -226> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr: -225> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: dump: 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.777 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr: -224> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr: -223> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr: -222> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr: -221> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr: -220> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr: -219> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: dump: 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.778 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr: -218> 2024-04-22T20:10:09.280+0000 7fd358d35800 5 monclient: authenticate success, global_id 6629 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr: -217> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient: get_monmap_and_config success 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr: -216> 2024-04-22T20:10:09.280+0000 7fd358d35800 4 set_mon_vals no callback set 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr: -215> 2024-04-22T20:10:09.281+0000 7fd358d35800 10 monclient: shutdown 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr: -214> 2024-04-22T20:10:09.281+0000 7fd358d35800 10 monclient: build_initial_monmap 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr: -213> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 build_initial for_mkfs: 0 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr: -212> 2024-04-22T20:10:09.281+0000 7fd358d35800 10 monclient: monmap: 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:10:09.779 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: -211> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding auth protocol: cephx 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: -210> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding auth protocol: cephx 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: -209> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding auth protocol: cephx 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: -208> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: -207> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: -206> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: -205> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.780 INFO:tasks.workunit.client.0.smithi136.stderr: -204> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -203> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -202> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -201> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -200> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -199> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -198> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -197> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -196> 2024-04-22T20:10:09.281+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -195> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 librados: starting msgr at 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -194> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 librados: starting objecter 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -193> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command objecter_requests cmddesc objecter_requests hook 0x55763d933d80 EEXIST 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -192> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 librados: setting wanted keys 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -191> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 librados: calling monclient init 2024-04-22T20:10:09.781 INFO:tasks.workunit.client.0.smithi136.stderr: -190> 2024-04-22T20:10:09.281+0000 7fd358d35800 10 monclient: init 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -189> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding auth protocol: cephx 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -188> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding auth protocol: cephx 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -187> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding auth protocol: cephx 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -186> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -185> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -184> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -183> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -182> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -181> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -180> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -179> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -178> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -177> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.782 INFO:tasks.workunit.client.0.smithi136.stderr: -176> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -175> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -174> 2024-04-22T20:10:09.282+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -173> 2024-04-22T20:10:09.282+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -172> 2024-04-22T20:10:09.282+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command rotate-key cmddesc rotate-key hook 0x55763d953ad8 EEXIST 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -171> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient: _reopen_session rank -1 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -170> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient: _add_conns ranks=[2,1,0] 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -169> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-c con 0x55763d9a5410 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -168> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-b con 0x55763d9ab1a0 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -167> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-a con 0x55763d9adb00 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -166> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -165> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -164> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -163> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): _renew_subs 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -162> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): authenticate will time out at 1998.633281s 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -161> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request con 0x55763d9a5410 auth_method 0 2024-04-22T20:10:09.783 INFO:tasks.workunit.client.0.smithi136.stderr: -160> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -159> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -158> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -157> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): get_auth_request con 0x55763d9adb00 auth_method 0 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -156> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -155> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -154> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -153> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -152> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -151> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -150> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -149> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -148> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -147> 2024-04-22T20:10:09.282+0000 7fd3551b6640 10 monclient(hunting): get_auth_request con 0x55763d9ab1a0 auth_method 0 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -146> 2024-04-22T20:10:09.282+0000 7fd3551b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -145> 2024-04-22T20:10:09.282+0000 7fd3551b6640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -144> 2024-04-22T20:10:09.282+0000 7fd3551b6640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.784 INFO:tasks.workunit.client.0.smithi136.stderr: -143> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_done global_id 6632 payload 824 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr: -142> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient: _finish_hunting 0 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr: -141> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 1 monclient: found mon.noname-c 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr: -140> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr: -139> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr: -138> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr: -137> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: dump: 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.785 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr: -136> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr: -135> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr: -134> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr: -133> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr: -132> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr: -131> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: dump: 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.786 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -130> 2024-04-22T20:10:09.282+0000 7fd358d35800 5 monclient: authenticate success, global_id 6632 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -129> 2024-04-22T20:10:09.282+0000 7fd22bfff640 4 set_mon_vals no callback set 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -128> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -127> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -126> 2024-04-22T20:10:09.283+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -125> 2024-04-22T20:10:09.283+0000 7fd358d35800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -124> 2024-04-22T20:10:09.283+0000 7fd358d35800 1 librados: init done 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -123> 2024-04-22T20:10:09.283+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cr dump hook 0x55763d9b07c8 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -122> 2024-04-22T20:10:09.284+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -121> 2024-04-22T20:10:09.284+0000 7fd22a7fc640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -120> 2024-04-22T20:10:09.284+0000 7fd22a7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -119> 2024-04-22T20:10:09.284+0000 7fd22a7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.787 INFO:tasks.workunit.client.0.smithi136.stderr: -118> 2024-04-22T20:10:09.284+0000 7fd3551b6640 10 monclient: get_auth_request con 0x7fd22006b2d0 auth_method 0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -117> 2024-04-22T20:10:09.285+0000 7fd3549b5640 10 monclient: get_auth_request con 0x55763d9c5140 auth_method 0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -116> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -115> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 realm 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -114> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -113> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -112> 2024-04-22T20:10:09.285+0000 7fd358d35800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -111> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -110> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -109> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -108> 2024-04-22T20:10:09.286+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x55763d9c94c0 auth_method 0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -107> 2024-04-22T20:10:09.286+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -106> 2024-04-22T20:10:09.286+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.788 INFO:tasks.workunit.client.0.smithi136.stderr: -105> 2024-04-22T20:10:09.286+0000 7fd3551b6640 10 monclient: get_auth_request con 0x55763d9ccbc0 auth_method 0 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -104> 2024-04-22T20:10:09.287+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -103> 2024-04-22T20:10:09.287+0000 7fd358d35800 20 searching for the correct realm 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -102> 2024-04-22T20:10:09.287+0000 7fd3549b5640 10 monclient: get_auth_request con 0x55763d9d19a0 auth_method 0 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -101> 2024-04-22T20:10:09.288+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x7fd344028070 auth_method 0 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -100> 2024-04-22T20:10:09.289+0000 7fd3551b6640 10 monclient: get_auth_request con 0x7fd350018040 auth_method 0 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -99> 2024-04-22T20:10:09.289+0000 7fd3551b6640 0 -- 172.21.15.136:0/700908430 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fd350018040 msgr2=0x7fd350024400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -98> 2024-04-22T20:10:09.490+0000 7fd3551b6640 10 monclient: get_auth_request con 0x7fd350018040 auth_method 0 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -97> 2024-04-22T20:10:09.492+0000 7fd3549b5640 10 monclient: get_auth_request con 0x7fd350007c20 auth_method 0 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -96> 2024-04-22T20:10:09.494+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x7fd350024b00 auth_method 0 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -95> 2024-04-22T20:10:09.494+0000 7fd34c1b4640 0 -- 172.21.15.136:0/700908430 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fd350024b00 msgr2=0x7fd350056910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -94> 2024-04-22T20:10:09.695+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x7fd350024b00 auth_method 0 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -93> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -92> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -91> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -90> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:09.789 INFO:tasks.workunit.client.0.smithi136.stderr: -89> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -88> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -87> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -86> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -85> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -84> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -83> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -82> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -81> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 zone default found 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -80> 2024-04-22T20:10:09.702+0000 7fd358d35800 4 Realm: () 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -79> 2024-04-22T20:10:09.702+0000 7fd358d35800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -78> 2024-04-22T20:10:09.702+0000 7fd358d35800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -77> 2024-04-22T20:10:09.702+0000 7fd358d35800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -76> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 zonegroup default 2024-04-22T20:10:09.790 INFO:tasks.workunit.client.0.smithi136.stderr: -75> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -74> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -73> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -72> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -71> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -70> 2024-04-22T20:10:09.703+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -69> 2024-04-22T20:10:09.703+0000 7fd358d35800 20 started sync module instance, tier type = 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -68> 2024-04-22T20:10:09.703+0000 7fd358d35800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -67> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=0 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -66> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=1 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -65> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=2 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -64> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=3 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -63> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=4 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -62> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=5 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -61> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=6 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -60> 2024-04-22T20:10:09.714+0000 7fd358d35800 20 add_watcher() i=7 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -59> 2024-04-22T20:10:09.714+0000 7fd358d35800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:09.791 INFO:tasks.workunit.client.0.smithi136.stderr: -58> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding auth protocol: cephx 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -57> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding auth protocol: cephx 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -56> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding auth protocol: cephx 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -55> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -54> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -53> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -52> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -51> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -50> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -49> 2024-04-22T20:10:09.715+0000 7fd20afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -48> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -47> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -46> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -45> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -44> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.792 INFO:tasks.workunit.client.0.smithi136.stderr: -43> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -42> 2024-04-22T20:10:09.716+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -41> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -40> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -39> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cache list hook 0x55763d9adf20 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -38> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cache inspect hook 0x55763d9adf20 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -37> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cache erase hook 0x55763d9adf20 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -36> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cache zap hook 0x55763d9adf20 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -35> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 note: GC not initialized 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -34> 2024-04-22T20:10:09.716+0000 7fd209fe3640 20 reqs_thread_entry: start 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -33> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command sync trace show hook 0x7fd33c004020 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -32> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command sync trace history hook 0x7fd33c004020 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -31> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command sync trace active hook 0x7fd33c004020 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -30> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command sync trace active_short hook 0x7fd33c004020 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -29> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -28> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 Filter name: none 2024-04-22T20:10:09.793 INFO:tasks.workunit.client.0.smithi136.stderr: -27> 2024-04-22T20:10:09.716+0000 7fd17b7fe640 20 reqs_thread_entry: start 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -26> 2024-04-22T20:10:09.716+0000 7fd358d35800 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -25> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -24> 2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -23> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -22> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -21> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 : miss 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -20> 2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -19> 2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -18> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -17> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -16> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -15> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -14> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -13> 2024-04-22T20:10:09.721+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.794 INFO:tasks.workunit.client.0.smithi136.stderr: -12> 2024-04-22T20:10:09.721+0000 7fd358d35800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -11> 2024-04-22T20:10:09.721+0000 7fd358d35800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -10> 2024-04-22T20:10:09.721+0000 7fd358d35800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28, ofs0, ns] 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -9> 2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562387 cookie 93966328238640 notifier 6632 bl.length()=903 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -8> 2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -7> 2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -6> 2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -5> 2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_sources= new_sources= 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -4> 2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential sources added=[] removed=[] 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -3> 2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_dests= new_dests= 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -2> 2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential dests added=[] removed=[] 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: -1> 2024-04-22T20:10:09.725+0000 7fd358d35800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fd358d35800 time 2024-04-22T20:10:09.725584+0000 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd35a381f5c] 2024-04-22T20:10:09.795 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd35a38211a] 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55763bf0edc1] 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x324) [0x55763bf1ce84] 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fd359229590] 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 0> 2024-04-22T20:10:09.726+0000 7fd358d35800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fd358d35800 thread_name:radosgw-admin 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd35923e6f0] 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd35928b94c] 2024-04-22T20:10:09.796 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd35a381fb6] 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd35a38211a] 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55763bf0edc1] 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x324) [0x55763bf1ce84] 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fd359229590] 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr:--- logging levels --- 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 none 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 lockdep 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 context 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 crush 2024-04-22T20:10:09.797 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_balancer 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_locker 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log_expire 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_migrator 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 3/ 5 mds_quiesce 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 buffer 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 timer 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 filer 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 striper 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 objecter 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rados 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_mirror 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_replay 2024-04-22T20:10:09.798 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_pwl 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 journaler 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objectcacher 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 immutable_obj_cache 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 client 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 osd 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 optracker 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objclass 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 filestore 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 journal 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 ms 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mon 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/10 monc 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 paxos 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 tp 2024-04-22T20:10:09.799 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 auth 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 crypto 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 finisher 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 reserver 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 heartbeatmap 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 perfcounter 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 20/20 rgw 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_sync 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_datacache 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_access 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_dbstore 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_flight 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_lifecycle 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_notification 2024-04-22T20:10:09.800 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 javaclient 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 asok 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 throttle 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 refs 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 compressor 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluestore 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluefs 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 bdev 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 kstore 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 4/ 5 rocksdb 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 fuse 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 2/ 5 mgr 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mgrc 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 dpdk 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 eventtrace 2024-04-22T20:10:09.801 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 prioritycache 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 test 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephfs_mirror 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephsqlite 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 crimson_interrupt 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_onode 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_odata 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_omap 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_tm 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_t 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cleaner 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_epm 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_lba 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-22T20:10:09.802 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cache 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_journal 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_device 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_backref 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 alienstore 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mclock 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cyanstore 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 ceph_exporter 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 memstore 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 trace 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: -2/-2 (syslog threshold) 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 99/99 (stderr threshold) 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd17b7fe640 / http_manager 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd209fe3640 / http_manager 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd20afe5640 / rgw_dt_lg_renew 2024-04-22T20:10:09.803 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd22a7fc640 / ms_dispatch 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd22bfff640 / io_context_pool 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd23a7fc640 / ms_dispatch 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd24160a640 / ms_dispatch 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd242e0d640 / io_context_pool 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd34c1b4640 / msgr-worker-2 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd3549b5640 / msgr-worker-1 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd3551b6640 / msgr-worker-0 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd3559b7640 / io_context_pool 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd358d35800 / radosgw-admin 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: max_recent 500 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: max_new 1000 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: log_file /var/log/ceph/ceph-client.admin.109966.log 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr:--- end dump of recent events --- 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr:--- begin dump of recent events --- 2024-04-22T20:10:09.804 INFO:tasks.workunit.client.0.smithi136.stderr: -499> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr: -498> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: dump: 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr: -497> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr: -496> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.805 INFO:tasks.workunit.client.0.smithi136.stderr: -495> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr: -494> 2024-04-22T20:10:09.268+0000 7fd358d35800 5 monclient: authenticate success, global_id 6996 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr: -493> 2024-04-22T20:10:09.268+0000 7fd358d35800 10 monclient: get_monmap_and_config success 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr: -492> 2024-04-22T20:10:09.268+0000 7fd358d35800 4 set_mon_vals no callback set 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr: -491> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr: -490> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr: -489> 2024-04-22T20:10:09.268+0000 7fd242e0d640 10 monclient: dump: 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.806 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr: -488> 2024-04-22T20:10:09.268+0000 7fd358d35800 10 monclient: shutdown 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr: -487> 2024-04-22T20:10:09.268+0000 7fd358d35800 5 asok(0x55763d6b5d00) unregister_commands rotate-key 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr: -486> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: build_initial_monmap 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr: -485> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 build_initial for_mkfs: 0 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr: -484> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: monmap: 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.807 INFO:tasks.workunit.client.0.smithi136.stderr: -483> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding auth protocol: cephx 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -482> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding auth protocol: cephx 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -481> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding auth protocol: cephx 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -480> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -479> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -478> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -477> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -476> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -475> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -474> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -473> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -472> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -471> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -470> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: crc 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -469> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d8760) adding con mode: secure 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -468> 2024-04-22T20:10:09.269+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.808 INFO:tasks.workunit.client.0.smithi136.stderr: -467> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 librados: starting msgr at 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -466> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 librados: starting objecter 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -465> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command objecter_requests hook 0x55763d8dc710 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -464> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 librados: setting wanted keys 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -463> 2024-04-22T20:10:09.269+0000 7fd358d35800 1 librados: calling monclient init 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -462> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: init 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -461> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding auth protocol: cephx 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -460> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding auth protocol: cephx 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -459> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding auth protocol: cephx 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -458> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -457> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -456> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -455> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -454> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -453> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.809 INFO:tasks.workunit.client.0.smithi136.stderr: -452> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -451> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -450> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -449> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -448> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: crc 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -447> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 AuthRegistry(0x55763d8d6900) adding con mode: secure 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -446> 2024-04-22T20:10:09.269+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -445> 2024-04-22T20:10:09.269+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -444> 2024-04-22T20:10:09.269+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command rotate-key hook 0x55763d8d6a48 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -443> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: _reopen_session rank -1 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -442> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -441> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-a con 0x55763d78d460 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -440> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-c con 0x55763d81d410 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -439> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-b con 0x55763d90bd00 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:10:09.810 INFO:tasks.workunit.client.0.smithi136.stderr: -438> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -437> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -436> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -435> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): _renew_subs 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -434> 2024-04-22T20:10:09.269+0000 7fd358d35800 10 monclient(hunting): authenticate will time out at 1998.620860s 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -433> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request con 0x55763d81d410 auth_method 0 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -432> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -431> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -430> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -429> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): get_auth_request con 0x55763d78d460 auth_method 0 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -428> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -427> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -426> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -425> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -424> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -423> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.811 INFO:tasks.workunit.client.0.smithi136.stderr: -422> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -421> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -420> 2024-04-22T20:10:09.270+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -419> 2024-04-22T20:10:09.270+0000 7fd3551b6640 10 monclient(hunting): get_auth_request con 0x55763d90bd00 auth_method 0 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -418> 2024-04-22T20:10:09.270+0000 7fd3551b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -417> 2024-04-22T20:10:09.270+0000 7fd3551b6640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -416> 2024-04-22T20:10:09.270+0000 7fd3551b6640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -415> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient(hunting): handle_auth_done global_id 6999 payload 824 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -414> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient: _finish_hunting 0 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -413> 2024-04-22T20:10:09.270+0000 7fd3549b5640 1 monclient: found mon.noname-a 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -412> 2024-04-22T20:10:09.270+0000 7fd3549b5640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -411> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -410> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr: -409> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: dump: 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.812 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr: -408> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr: -407> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr: -406> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr: -405> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr: -404> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr: -403> 2024-04-22T20:10:09.270+0000 7fd24160a640 10 monclient: dump: 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.813 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -402> 2024-04-22T20:10:09.270+0000 7fd358d35800 5 monclient: authenticate success, global_id 6999 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -401> 2024-04-22T20:10:09.270+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -400> 2024-04-22T20:10:09.270+0000 7fd242e0d640 4 set_mon_vals no callback set 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -399> 2024-04-22T20:10:09.270+0000 7fd358d35800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -398> 2024-04-22T20:10:09.270+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -397> 2024-04-22T20:10:09.270+0000 7fd358d35800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -396> 2024-04-22T20:10:09.270+0000 7fd358d35800 1 librados: init done 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -395> 2024-04-22T20:10:09.272+0000 7fd24160a640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:10:09.814 INFO:tasks.workunit.client.0.smithi136.stderr: -394> 2024-04-22T20:10:09.272+0000 7fd24160a640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -393> 2024-04-22T20:10:09.272+0000 7fd24160a640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -392> 2024-04-22T20:10:09.272+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x7fd23c06b2d0 auth_method 0 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -391> 2024-04-22T20:10:09.272+0000 7fd3551b6640 10 monclient: get_auth_request con 0x55763d904f60 auth_method 0 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -390> 2024-04-22T20:10:09.273+0000 7fd3549b5640 10 monclient: get_auth_request con 0x55763d922890 auth_method 0 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -389> 2024-04-22T20:10:09.274+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x55763d926a80 auth_method 0 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -388> 2024-04-22T20:10:09.275+0000 7fd3551b6640 10 monclient: get_auth_request con 0x55763d926140 auth_method 0 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -387> 2024-04-22T20:10:09.275+0000 7fd358d35800 10 monclient: build_initial_monmap 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -386> 2024-04-22T20:10:09.275+0000 7fd358d35800 1 build_initial for_mkfs: 0 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr: -385> 2024-04-22T20:10:09.275+0000 7fd358d35800 10 monclient: monmap: 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.815 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -384> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding auth protocol: cephx 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -383> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding auth protocol: cephx 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -382> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding auth protocol: cephx 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -381> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -380> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -379> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -378> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -377> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -376> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -375> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -374> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.816 INFO:tasks.workunit.client.0.smithi136.stderr: -373> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -372> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -371> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: crc 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -370> 2024-04-22T20:10:09.275+0000 7fd358d35800 5 AuthRegistry(0x55763d92eb80) adding con mode: secure 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -369> 2024-04-22T20:10:09.276+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -368> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command objecter_requests cmddesc objecter_requests hook 0x55763d960850 EEXIST 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -367> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient: init 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -366> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding auth protocol: cephx 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -365> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding auth protocol: cephx 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -364> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding auth protocol: cephx 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -363> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -362> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -361> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -360> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -359> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.817 INFO:tasks.workunit.client.0.smithi136.stderr: -358> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -357> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -356> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -355> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -354> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -353> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: crc 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -352> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 AuthRegistry(0x55763d92d9f8) adding con mode: secure 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -351> 2024-04-22T20:10:09.276+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -350> 2024-04-22T20:10:09.276+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -349> 2024-04-22T20:10:09.276+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command rotate-key cmddesc rotate-key hook 0x55763d92db40 EEXIST 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -348> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient: _reopen_session rank -1 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -347> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient: _add_conns ranks=[0,2,1] 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -346> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-a con 0x55763d958d90 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -345> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-c con 0x55763d959170 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -344> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-b con 0x55763d96ee20 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -343> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.818 INFO:tasks.workunit.client.0.smithi136.stderr: -342> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -341> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -340> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): _renew_subs 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -339> 2024-04-22T20:10:09.276+0000 7fd358d35800 10 monclient(hunting): authenticate will time out at 1998.627663s 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -338> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request con 0x55763d958d90 auth_method 0 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -337> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -336> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -335> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -334> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): get_auth_request con 0x55763d959170 auth_method 0 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -333> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -332> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -331> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -330> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -329> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -328> 2024-04-22T20:10:09.276+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -327> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.819 INFO:tasks.workunit.client.0.smithi136.stderr: -326> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -325> 2024-04-22T20:10:09.276+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -324> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): get_auth_request con 0x55763d96ee20 auth_method 0 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -323> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -322> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -321> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -320> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -319> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -318> 2024-04-22T20:10:09.277+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -317> 2024-04-22T20:10:09.277+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_done global_id 7002 payload 824 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -316> 2024-04-22T20:10:09.277+0000 7fd34c1b4640 10 monclient: _finish_hunting 0 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -315> 2024-04-22T20:10:09.277+0000 7fd34c1b4640 1 monclient: found mon.noname-a 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -314> 2024-04-22T20:10:09.277+0000 7fd34c1b4640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -313> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -312> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr: -311> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: dump: 2024-04-22T20:10:09.820 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr: -310> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr: -309> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr: -308> 2024-04-22T20:10:09.277+0000 7fd358d35800 5 monclient: authenticate success, global_id 7002 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr: -307> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr: -306> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr: -305> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr: -304> 2024-04-22T20:10:09.277+0000 7fd23a7fc640 10 monclient: dump: 2024-04-22T20:10:09.821 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr: -303> 2024-04-22T20:10:09.277+0000 7fd3559b7640 4 set_mon_vals no callback set 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr: -302> 2024-04-22T20:10:09.277+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr: -301> 2024-04-22T20:10:09.277+0000 7fd358d35800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr: -300> 2024-04-22T20:10:09.277+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.822 INFO:tasks.workunit.client.0.smithi136.stderr: -299> 2024-04-22T20:10:09.277+0000 7fd358d35800 10 monclient: _send_mon_message to mon.a at v2:172.21.15.136:3300/0 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr: -298> 2024-04-22T20:10:09.279+0000 7fd23a7fc640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr: -297> 2024-04-22T20:10:09.279+0000 7fd23a7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr: -296> 2024-04-22T20:10:09.279+0000 7fd23a7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr: -295> 2024-04-22T20:10:09.279+0000 7fd358d35800 10 monclient: get_monmap_and_config 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr: -294> 2024-04-22T20:10:09.279+0000 7fd358d35800 10 monclient: build_initial_monmap 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr: -293> 2024-04-22T20:10:09.279+0000 7fd358d35800 1 build_initial for_mkfs: 0 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr: -292> 2024-04-22T20:10:09.279+0000 7fd358d35800 10 monclient: monmap: 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:10:09.823 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -291> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding auth protocol: cephx 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -290> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding auth protocol: cephx 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -289> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding auth protocol: cephx 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -288> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -287> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -286> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -285> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -284> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -283> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -282> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -281> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -280> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.824 INFO:tasks.workunit.client.0.smithi136.stderr: -279> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -278> 2024-04-22T20:10:09.279+0000 7fd3551b6640 10 monclient: get_auth_request con 0x7fd22c06b1e0 auth_method 0 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -277> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: crc 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -276> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x55763d9767e0) adding con mode: secure 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -275> 2024-04-22T20:10:09.279+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -274> 2024-04-22T20:10:09.279+0000 7fd358d35800 10 monclient: init 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -273> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding auth protocol: cephx 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -272> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding auth protocol: cephx 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -271> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding auth protocol: cephx 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -270> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -269> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -268> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -267> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -266> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -265> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -264> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.825 INFO:tasks.workunit.client.0.smithi136.stderr: -263> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -262> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -261> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -260> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: crc 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -259> 2024-04-22T20:10:09.279+0000 7fd358d35800 5 AuthRegistry(0x7fff78f554c0) adding con mode: secure 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -258> 2024-04-22T20:10:09.280+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -257> 2024-04-22T20:10:09.280+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -256> 2024-04-22T20:10:09.280+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command rotate-key cmddesc rotate-key hook 0x7fff78f55608 EEXIST 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -255> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient: _reopen_session rank -1 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -254> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient: _add_conns ranks=[2,0,1] 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -253> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-c con 0x55763d9ab1a0 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -252> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-a con 0x55763d9ab580 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -251> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-b con 0x55763d9a5410 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:10:09.826 INFO:tasks.workunit.client.0.smithi136.stderr: -250> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -249> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -248> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -247> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): _renew_subs 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -246> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient(hunting): authenticate will time out at 1998.631194s 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -245> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request con 0x55763d9ab1a0 auth_method 0 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -244> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -243> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -242> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -241> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): get_auth_request con 0x55763d9ab580 auth_method 0 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -240> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -239> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -238> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -237> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.827 INFO:tasks.workunit.client.0.smithi136.stderr: -236> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -235> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -234> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -233> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -232> 2024-04-22T20:10:09.280+0000 7fd3551b6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -231> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_done global_id 6629 payload 274 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -230> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient: _finish_hunting 0 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -229> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 1 monclient: found mon.noname-c 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -228> 2024-04-22T20:10:09.280+0000 7fd34c1b4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -227> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -226> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr: -225> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: dump: 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.828 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr: -224> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr: -223> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr: -222> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr: -221> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr: -220> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr: -219> 2024-04-22T20:10:09.280+0000 7fd22bfff640 10 monclient: dump: 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.829 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr: -218> 2024-04-22T20:10:09.280+0000 7fd358d35800 5 monclient: authenticate success, global_id 6629 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr: -217> 2024-04-22T20:10:09.280+0000 7fd358d35800 10 monclient: get_monmap_and_config success 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr: -216> 2024-04-22T20:10:09.280+0000 7fd358d35800 4 set_mon_vals no callback set 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr: -215> 2024-04-22T20:10:09.281+0000 7fd358d35800 10 monclient: shutdown 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr: -214> 2024-04-22T20:10:09.281+0000 7fd358d35800 10 monclient: build_initial_monmap 2024-04-22T20:10:09.830 INFO:tasks.workunit.client.0.smithi136.stderr: -213> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 build_initial for_mkfs: 0 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr: -212> 2024-04-22T20:10:09.281+0000 7fd358d35800 10 monclient: monmap: 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 0 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 0.000000 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:created 0.000000 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 0 (unknown) 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.noname-a 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.noname-b 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.noname-c 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr: -211> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding auth protocol: cephx 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr: -210> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding auth protocol: cephx 2024-04-22T20:10:09.831 INFO:tasks.workunit.client.0.smithi136.stderr: -209> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding auth protocol: cephx 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -208> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -207> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -206> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -205> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -204> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -203> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -202> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -201> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -200> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -199> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -198> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: crc 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -197> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d976790) adding con mode: secure 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -196> 2024-04-22T20:10:09.281+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.832 INFO:tasks.workunit.client.0.smithi136.stderr: -195> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 librados: starting msgr at 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -194> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 librados: starting objecter 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -193> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command objecter_requests cmddesc objecter_requests hook 0x55763d933d80 EEXIST 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -192> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 librados: setting wanted keys 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -191> 2024-04-22T20:10:09.281+0000 7fd358d35800 1 librados: calling monclient init 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -190> 2024-04-22T20:10:09.281+0000 7fd358d35800 10 monclient: init 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -189> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding auth protocol: cephx 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -188> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding auth protocol: cephx 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -187> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding auth protocol: cephx 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -186> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -185> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -184> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -183> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -182> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.833 INFO:tasks.workunit.client.0.smithi136.stderr: -181> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -180> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -179> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -178> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -177> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -176> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: crc 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -175> 2024-04-22T20:10:09.281+0000 7fd358d35800 5 AuthRegistry(0x55763d953990) adding con mode: secure 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -174> 2024-04-22T20:10:09.282+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -173> 2024-04-22T20:10:09.282+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -172> 2024-04-22T20:10:09.282+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command rotate-key cmddesc rotate-key hook 0x55763d953ad8 EEXIST 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -171> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient: _reopen_session rank -1 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -170> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient: _add_conns ranks=[2,1,0] 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -169> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-c con 0x55763d9a5410 addr [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -168> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-b con 0x55763d9ab1a0 addr [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] 2024-04-22T20:10:09.834 INFO:tasks.workunit.client.0.smithi136.stderr: -167> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): picked mon.noname-a con 0x55763d9adb00 addr [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -166> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -165> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -164> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): start opening mon connection 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -163> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): _renew_subs 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -162> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient(hunting): authenticate will time out at 1998.633281s 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -161> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request con 0x55763d9a5410 auth_method 0 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -160> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -159> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -158> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -157> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): get_auth_request con 0x55763d9adb00 auth_method 0 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -156> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -155> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -154> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -153> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.835 INFO:tasks.workunit.client.0.smithi136.stderr: -152> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -151> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -150> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -149> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -148> 2024-04-22T20:10:09.282+0000 7fd3549b5640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -147> 2024-04-22T20:10:09.282+0000 7fd3551b6640 10 monclient(hunting): get_auth_request con 0x55763d9ab1a0 auth_method 0 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -146> 2024-04-22T20:10:09.282+0000 7fd3551b6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -145> 2024-04-22T20:10:09.282+0000 7fd3551b6640 10 monclient(hunting): _init_auth method 2 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -144> 2024-04-22T20:10:09.282+0000 7fd3551b6640 10 monclient(hunting): _init_auth creating new auth 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -143> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient(hunting): handle_auth_done global_id 6632 payload 824 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -142> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient: _finish_hunting 0 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -141> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 1 monclient: found mon.noname-c 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -140> 2024-04-22T20:10:09.282+0000 7fd34c1b4640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.136:3301/0 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -139> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.836 INFO:tasks.workunit.client.0.smithi136.stderr: -138> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr: -137> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: dump: 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr: -136> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: _finish_auth 0 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr: -135> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: _check_auth_tickets 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr: -134> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: handle_config config(0 keys) 2024-04-22T20:10:09.837 INFO:tasks.workunit.client.0.smithi136.stderr: -133> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr: -132> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr: -131> 2024-04-22T20:10:09.282+0000 7fd22a7fc640 10 monclient: dump: 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:epoch 1 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:fsid b2f95cfa-7b26-4a63-b13e-7105b011d9c4 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:last_changed 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:created 2024-04-22T19:50:19.969705+0000 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:min_mon_release 19 (squid) 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:election_strategy: 1 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:0: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.a 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.838 INFO:tasks.workunit.client.0.smithi136.stderr: -130> 2024-04-22T20:10:09.282+0000 7fd358d35800 5 monclient: authenticate success, global_id 6632 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -129> 2024-04-22T20:10:09.282+0000 7fd22bfff640 4 set_mon_vals no callback set 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -128> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -127> 2024-04-22T20:10:09.282+0000 7fd358d35800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -126> 2024-04-22T20:10:09.283+0000 7fd358d35800 10 monclient: _renew_subs 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -125> 2024-04-22T20:10:09.283+0000 7fd358d35800 10 monclient: _send_mon_message to mon.c at v2:172.21.15.136:3301/0 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -124> 2024-04-22T20:10:09.283+0000 7fd358d35800 1 librados: init done 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -123> 2024-04-22T20:10:09.283+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cr dump hook 0x55763d9b07c8 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -122> 2024-04-22T20:10:09.284+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -121> 2024-04-22T20:10:09.284+0000 7fd22a7fc640 4 mgrc handle_mgr_map Got map version 5 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -120> 2024-04-22T20:10:09.284+0000 7fd22a7fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -119> 2024-04-22T20:10:09.284+0000 7fd22a7fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.164:6832/3034844420,v1:172.21.15.164:6833/3034844420] 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -118> 2024-04-22T20:10:09.284+0000 7fd3551b6640 10 monclient: get_auth_request con 0x7fd22006b2d0 auth_method 0 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -117> 2024-04-22T20:10:09.285+0000 7fd3549b5640 10 monclient: get_auth_request con 0x55763d9c5140 auth_method 0 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -116> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.839 INFO:tasks.workunit.client.0.smithi136.stderr: -115> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 realm 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -114> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -113> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -112> 2024-04-22T20:10:09.285+0000 7fd358d35800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -111> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -110> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -109> 2024-04-22T20:10:09.285+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -108> 2024-04-22T20:10:09.286+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x55763d9c94c0 auth_method 0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -107> 2024-04-22T20:10:09.286+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -106> 2024-04-22T20:10:09.286+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -105> 2024-04-22T20:10:09.286+0000 7fd3551b6640 10 monclient: get_auth_request con 0x55763d9ccbc0 auth_method 0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -104> 2024-04-22T20:10:09.287+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -103> 2024-04-22T20:10:09.287+0000 7fd358d35800 20 searching for the correct realm 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -102> 2024-04-22T20:10:09.287+0000 7fd3549b5640 10 monclient: get_auth_request con 0x55763d9d19a0 auth_method 0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -101> 2024-04-22T20:10:09.288+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x7fd344028070 auth_method 0 2024-04-22T20:10:09.840 INFO:tasks.workunit.client.0.smithi136.stderr: -100> 2024-04-22T20:10:09.289+0000 7fd3551b6640 10 monclient: get_auth_request con 0x7fd350018040 auth_method 0 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -99> 2024-04-22T20:10:09.289+0000 7fd3551b6640 0 -- 172.21.15.136:0/700908430 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fd350018040 msgr2=0x7fd350024400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -98> 2024-04-22T20:10:09.490+0000 7fd3551b6640 10 monclient: get_auth_request con 0x7fd350018040 auth_method 0 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -97> 2024-04-22T20:10:09.492+0000 7fd3549b5640 10 monclient: get_auth_request con 0x7fd350007c20 auth_method 0 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -96> 2024-04-22T20:10:09.494+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x7fd350024b00 auth_method 0 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -95> 2024-04-22T20:10:09.494+0000 7fd34c1b4640 0 -- 172.21.15.136:0/700908430 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fd350024b00 msgr2=0x7fd350056910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -94> 2024-04-22T20:10:09.695+0000 7fd34c1b4640 10 monclient: get_auth_request con 0x7fd350024b00 auth_method 0 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -93> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -92> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -91> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -90> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -89> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:09.841 INFO:tasks.workunit.client.0.smithi136.stderr: -88> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -87> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -86> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -85> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -84> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -83> 2024-04-22T20:10:09.701+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -82> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -81> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 zone default found 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -80> 2024-04-22T20:10:09.702+0000 7fd358d35800 4 Realm: () 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -79> 2024-04-22T20:10:09.702+0000 7fd358d35800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -78> 2024-04-22T20:10:09.702+0000 7fd358d35800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -77> 2024-04-22T20:10:09.702+0000 7fd358d35800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -76> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 zonegroup default 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -75> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.842 INFO:tasks.workunit.client.0.smithi136.stderr: -74> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -73> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -72> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -71> 2024-04-22T20:10:09.702+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -70> 2024-04-22T20:10:09.703+0000 7fd358d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -69> 2024-04-22T20:10:09.703+0000 7fd358d35800 20 started sync module instance, tier type = 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -68> 2024-04-22T20:10:09.703+0000 7fd358d35800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -67> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=0 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -66> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=1 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -65> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=2 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -64> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=3 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -63> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=4 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -62> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=5 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -61> 2024-04-22T20:10:09.713+0000 7fd358d35800 20 add_watcher() i=6 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -60> 2024-04-22T20:10:09.714+0000 7fd358d35800 20 add_watcher() i=7 2024-04-22T20:10:09.843 INFO:tasks.workunit.client.0.smithi136.stderr: -59> 2024-04-22T20:10:09.714+0000 7fd358d35800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -58> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding auth protocol: cephx 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -57> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding auth protocol: cephx 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -56> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding auth protocol: cephx 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -55> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -54> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -53> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -52> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -51> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -50> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -49> 2024-04-22T20:10:09.715+0000 7fd20afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -48> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -47> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -46> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -45> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -44> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: crc 2024-04-22T20:10:09.844 INFO:tasks.workunit.client.0.smithi136.stderr: -43> 2024-04-22T20:10:09.715+0000 7fd358d35800 5 AuthRegistry(0x7fff78f55950) adding con mode: secure 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -42> 2024-04-22T20:10:09.716+0000 7fd358d35800 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -41> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -40> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -39> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cache list hook 0x55763d9adf20 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -38> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cache inspect hook 0x55763d9adf20 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -37> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cache erase hook 0x55763d9adf20 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -36> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command cache zap hook 0x55763d9adf20 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -35> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 note: GC not initialized 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -34> 2024-04-22T20:10:09.716+0000 7fd209fe3640 20 reqs_thread_entry: start 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -33> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command sync trace show hook 0x7fd33c004020 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -32> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command sync trace history hook 0x7fd33c004020 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -31> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command sync trace active hook 0x7fd33c004020 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -30> 2024-04-22T20:10:09.716+0000 7fd358d35800 5 asok(0x55763d6b5d00) register_command sync trace active_short hook 0x7fd33c004020 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -29> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:09.845 INFO:tasks.workunit.client.0.smithi136.stderr: -28> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 Filter name: none 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -27> 2024-04-22T20:10:09.716+0000 7fd17b7fe640 20 reqs_thread_entry: start 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -26> 2024-04-22T20:10:09.716+0000 7fd358d35800 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -25> 2024-04-22T20:10:09.716+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -24> 2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -23> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -22> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -21> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 : miss 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -20> 2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -19> 2024-04-22T20:10:09.717+0000 7fd358d35800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -18> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -17> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -16> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -15> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.846 INFO:tasks.workunit.client.0.smithi136.stderr: -14> 2024-04-22T20:10:09.717+0000 7fd358d35800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -13> 2024-04-22T20:10:09.721+0000 7fd358d35800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -12> 2024-04-22T20:10:09.721+0000 7fd358d35800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -11> 2024-04-22T20:10:09.721+0000 7fd358d35800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -10> 2024-04-22T20:10:09.721+0000 7fd358d35800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28, ofs0, ns] 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -9> 2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562387 cookie 93966328238640 notifier 6632 bl.length()=903 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -8> 2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -7> 2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -6> 2024-04-22T20:10:09.722+0000 7fd22bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -5> 2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_sources= new_sources= 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -4> 2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential sources added=[] removed=[] 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -3> 2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_dests= new_dests= 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -2> 2024-04-22T20:10:09.723+0000 7fd358d35800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential dests added=[] removed=[] 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: -1> 2024-04-22T20:10:09.725+0000 7fd358d35800 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fd358d35800 time 2024-04-22T20:10:09.725584+0000 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3152-g9ed8263e/rpm/el9/BUILD/ceph-19.0.0-3152-g9ed8263e/src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.847 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fd35a381f5c] 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd35a38211a] 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55763bf0edc1] 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x324) [0x55763bf1ce84] 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 5: main() 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /lib64/libc.so.6(+0x29590) [0x7fd359229590] 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 7: __libc_start_main() 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 8: _start() 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 0> 2024-04-22T20:10:09.726+0000 7fd358d35800 -1 *** Caught signal (Aborted) ** 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: in thread 7fd358d35800 thread_name:radosgw-admin 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: ceph version 19.0.0-3152-g9ed8263e (9ed8263edff7e87907bf24b054e5c7144c194dbe) squid (dev) 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fd35923e6f0] 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 2: /lib64/libc.so.6(+0x8b94c) [0x7fd35928b94c] 2024-04-22T20:10:09.848 INFO:tasks.workunit.client.0.smithi136.stderr: 3: raise() 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 4: abort() 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fd35a381fb6] 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18211a) [0x7fd35a38211a] 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55763bf0edc1] 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x324) [0x55763bf1ce84] 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 9: main() 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 10: /lib64/libc.so.6(+0x29590) [0x7fd359229590] 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 11: __libc_start_main() 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 12: _start() 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr:--- logging levels --- 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 none 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 lockdep 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 context 2024-04-22T20:10:09.849 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 crush 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_balancer 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_locker 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_log_expire 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mds_migrator 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 3/ 5 mds_quiesce 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 buffer 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 timer 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 filer 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 striper 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 1 objecter 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rados 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd 2024-04-22T20:10:09.850 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_mirror 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_replay 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 rbd_pwl 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 journaler 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objectcacher 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 immutable_obj_cache 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 client 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 osd 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 optracker 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 objclass 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 filestore 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 journal 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 ms 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mon 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/10 monc 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 paxos 2024-04-22T20:10:09.851 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 tp 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 auth 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 crypto 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 finisher 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 reserver 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 heartbeatmap 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 perfcounter 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 20/20 rgw 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_sync 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_datacache 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_access 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_dbstore 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_flight 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_lifecycle 2024-04-22T20:10:09.852 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 rgw_notification 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 javaclient 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 asok 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 1 throttle 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 0 refs 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 compressor 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluestore 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 bluefs 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 3 bdev 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 kstore 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 4/ 5 rocksdb 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 fuse 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 2/ 5 mgr 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mgrc 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 dpdk 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 eventtrace 2024-04-22T20:10:09.853 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 prioritycache 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 test 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephfs_mirror 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cephsqlite 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 crimson_interrupt 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_onode 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_odata 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_omap 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_tm 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_t 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cleaner 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_epm 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_lba 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_fixedkv_tree 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_cache 2024-04-22T20:10:09.854 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_journal 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_device 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 seastore_backref 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 alienstore 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 mclock 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 0/ 5 cyanstore 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 ceph_exporter 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 memstore 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 1/ 5 trace 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: -2/-2 (syslog threshold) 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 99/99 (stderr threshold) 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr:--- pthread ID / name mapping for recent threads --- 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd17b7fe640 / http_manager 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd209fe3640 / http_manager 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd20afe5640 / rgw_dt_lg_renew 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd22a7fc640 / ms_dispatch 2024-04-22T20:10:09.855 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd22bfff640 / io_context_pool 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd23a7fc640 / ms_dispatch 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd24160a640 / ms_dispatch 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd242e0d640 / io_context_pool 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd34c1b4640 / msgr-worker-2 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd3549b5640 / msgr-worker-1 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd3551b6640 / msgr-worker-0 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd3559b7640 / io_context_pool 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: 7fd358d35800 / radosgw-admin 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: max_recent 500 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: max_new 1000 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr: log_file /var/lib/ceph/crash/2024-04-22T20:10:09.727044Z_bfcbcfcd-3a2b-46be-bf16-8639f66ae33a/log 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr:--- end dump of recent events --- 2024-04-22T20:10:09.856 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-04-22T20:10:09.857 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.799+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.800+0000 7fe882e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.800+0000 7fe882e11800 20 realm 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.800+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.801+0000 7fe882e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.801+0000 7fe882e11800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.801+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.801+0000 7fe882e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.801+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.802+0000 7fe882e11800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.802+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:09.858 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.802+0000 7fe882e11800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:09.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.802+0000 7fe882e11800 20 searching for the correct realm 2024-04-22T20:10:09.859 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:09.804+0000 7fe87ec15640 0 -- 172.21.15.136:0/1883776835 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fe878099250 msgr2=0x7fe878001250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:10.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.010+0000 7fe877fff640 0 -- 172.21.15.136:0/1883776835 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fe878044b90 msgr2=0x7fe878008030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.216+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:10.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 20 zone default found 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 4 Realm: () 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 20 zonegroup default 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.217+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.218+0000 7fe882e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.218+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.218+0000 7fe882e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:10.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.218+0000 7fe882e11800 20 started sync module instance, tier type = 2024-04-22T20:10:10.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.218+0000 7fe882e11800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.228+0000 7fe882e11800 20 add_watcher() i=0 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.228+0000 7fe882e11800 20 add_watcher() i=1 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.228+0000 7fe882e11800 20 add_watcher() i=2 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.228+0000 7fe882e11800 20 add_watcher() i=3 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.228+0000 7fe882e11800 20 add_watcher() i=4 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.228+0000 7fe882e11800 20 add_watcher() i=5 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.228+0000 7fe882e11800 20 add_watcher() i=6 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.229+0000 7fe882e11800 20 add_watcher() i=7 2024-04-22T20:10:10.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.229+0000 7fe882e11800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:10.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.230+0000 7fe734fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:10.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.231+0000 7fe882e11800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:10.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.231+0000 7fe882e11800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:10.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.231+0000 7fe882e11800 5 note: GC not initialized 2024-04-22T20:10:10.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.231+0000 7fe6af7fe640 20 reqs_thread_entry: start 2024-04-22T20:10:10.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.231+0000 7fe882e11800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:10.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.231+0000 7fe882e11800 20 Filter name: none 2024-04-22T20:10:10.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.231+0000 7fe6ad7fa640 20 reqs_thread_entry: start 2024-04-22T20:10:10.231 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.231+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.232+0000 7fe882e11800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:10.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.232+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.232+0000 7fe882e11800 20 rados_obj.operate() r=0 bl.length=433 2024-04-22T20:10:10.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.232+0000 7fe882e11800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:10.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.232+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.233+0000 7fe882e11800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:10.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.233+0000 7fe882e11800 20 rados->read ofs=0 len=0 2024-04-22T20:10:10.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.234+0000 7fe882e11800 20 rados_obj.operate() r=0 bl.length=433 2024-04-22T20:10:10.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.234+0000 7fe882e11800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:10.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.235+0000 7fe734fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:10.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.237+0000 7fe882e11800 20 remove_watcher() i=0 2024-04-22T20:10:10.236 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.237+0000 7fe882e11800 2 removed watcher, disabling cache 2024-04-22T20:10:10.237 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.238+0000 7fe882e11800 20 remove_watcher() i=1 2024-04-22T20:10:10.239 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.240+0000 7fe882e11800 20 remove_watcher() i=2 2024-04-22T20:10:10.240 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.241+0000 7fe882e11800 20 remove_watcher() i=3 2024-04-22T20:10:10.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.242+0000 7fe882e11800 20 remove_watcher() i=4 2024-04-22T20:10:10.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.243+0000 7fe882e11800 20 remove_watcher() i=5 2024-04-22T20:10:10.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.244+0000 7fe882e11800 20 remove_watcher() i=6 2024-04-22T20:10:10.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:10.245+0000 7fe882e11800 20 remove_watcher() i=7 2024-04-22T20:10:10.253 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:10.253 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:10.253 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-do-reshard id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 num_objects: 20 size_kb: 1 num_shards: 1 2024-04-22T20:10:10.253 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:10.255 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:10.255 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:40.356 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:40.357 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:40.414 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-04-22T20:10:40.449 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:40.449 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:40.486 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.487+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.488+0000 7f221d40f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:40.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.488+0000 7f221d40f800 20 realm 2024-04-22T20:10:40.487 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.488+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.488+0000 7f221d40f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:40.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.488+0000 7f221d40f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:40.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.488+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.488+0000 7f221d40f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:40.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.488+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.488 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.489+0000 7f221d40f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:40.489 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.489+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.489 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.490+0000 7f221d40f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:40.489 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.490+0000 7f221d40f800 20 searching for the correct realm 2024-04-22T20:10:40.489 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.490+0000 7f2218a14640 0 -- 172.21.15.136:0/1405013697 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x562090c0c030 msgr2=0x562090c0e4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:40.695 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.696+0000 7f2218a14640 0 -- 172.21.15.136:0/1405013697 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x7f2214086710 msgr2=0x7f22140022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:40.906 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.905+0000 7f221d40f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.905+0000 7f221d40f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.905+0000 7f221d40f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.905+0000 7f221d40f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.905+0000 7f221d40f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.905+0000 7f221d40f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.905+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 20 zone default found 2024-04-22T20:10:40.907 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 4 Realm: () 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 20 zonegroup default 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.906+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.907+0000 7f221d40f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.907+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.907+0000 7f221d40f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.907+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.907+0000 7f221d40f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.907+0000 7f221d40f800 20 started sync module instance, tier type = 2024-04-22T20:10:40.908 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.907+0000 7f221d40f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:40.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.917+0000 7f221d40f800 20 add_watcher() i=0 2024-04-22T20:10:40.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.917+0000 7f221d40f800 20 add_watcher() i=1 2024-04-22T20:10:40.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.917+0000 7f221d40f800 20 add_watcher() i=2 2024-04-22T20:10:40.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.917+0000 7f221d40f800 20 add_watcher() i=3 2024-04-22T20:10:40.916 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.917+0000 7f221d40f800 20 add_watcher() i=4 2024-04-22T20:10:40.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.917+0000 7f221d40f800 20 add_watcher() i=5 2024-04-22T20:10:40.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.917+0000 7f221d40f800 20 add_watcher() i=6 2024-04-22T20:10:40.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.918+0000 7f221d40f800 20 add_watcher() i=7 2024-04-22T20:10:40.917 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.918+0000 7f221d40f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:40.919 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.920+0000 7f20cefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.920+0000 7f221d40f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.920+0000 7f221d40f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.920+0000 7f221d40f800 5 note: GC not initialized 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.920+0000 7f20cdfe3640 20 reqs_thread_entry: start 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.920+0000 7f221d40f800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.920+0000 7f221d40f800 20 Filter name: none 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.921+0000 7f203f7fe640 20 reqs_thread_entry: start 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.921+0000 7f221d40f800 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-04-22T20:10:40.920 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.921+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.921+0000 7f221d40f800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:40.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.921+0000 7f221d40f800 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-04-22T20:10:40.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.921+0000 7f221d40f800 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-04-22T20:10:40.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.921+0000 7f221d40f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 : miss 2024-04-22T20:10:40.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.921+0000 7f221d40f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:40.921 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.922+0000 7f221d40f800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:40.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.922+0000 7f221d40f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:40.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.922+0000 7f221d40f800 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:40.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.922+0000 7f221d40f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:40.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.922+0000 7f221d40f800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:40.922 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.922+0000 7f221d40f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 2024-04-22T20:10:40.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.926+0000 7f221d40f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:40.925 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.926+0000 7f221d40f800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:40.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.926+0000 7f221d40f800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:40.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.926+0000 7f221d40f800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28, ofs0, ns] 2024-04-22T20:10:40.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.926+0000 7f20effff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562389 cookie 94697867522576 notifier 6662 bl.length()=903 2024-04-22T20:10:40.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.926+0000 7f20effff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:40.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.926+0000 7f20effff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:40.926 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.926+0000 7f20effff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:40.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.928+0000 7f221d40f800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_sources= new_sources= 2024-04-22T20:10:40.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.928+0000 7f221d40f800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential sources added=[] removed=[] 2024-04-22T20:10:40.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.928+0000 7f221d40f800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_dests= new_dests= 2024-04-22T20:10:40.927 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.928+0000 7f221d40f800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential dests added=[] removed=[] 2024-04-22T20:10:40.929 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.930+0000 7f221d40f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28.1.0 2024-04-22T20:10:40.929 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.930+0000 7f221d40f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28.1.0 2024-04-22T20:10:40.929 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.930+0000 7f221d40f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28.1.1 2024-04-22T20:10:40.929 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.930+0000 7f221d40f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28.1.1 2024-04-22T20:10:40.929 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.930+0000 7f221d40f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28.0 2024-04-22T20:10:40.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.932+0000 7f221d40f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 : hit (requested=0x17, cached=0x17) 2024-04-22T20:10:40.932 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.932+0000 7f221d40f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 2024-04-22T20:10:40.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.934+0000 7f221d40f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:40.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.934+0000 7f221d40f800 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:40.933 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.934+0000 7f221d40f800 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:40.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.934+0000 7f221d40f800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28, ofs0, ns] 2024-04-22T20:10:40.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.934+0000 7f20ef7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562390 cookie 94697867522576 notifier 6662 bl.length()=871 2024-04-22T20:10:40.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.934+0000 7f20ef7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 info.flags=0x17 2024-04-22T20:10:40.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.934+0000 7f20ef7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 to cache LRU end 2024-04-22T20:10:40.934 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.934+0000 7f20ef7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-04-22T20:10:40.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.936+0000 7f221d40f800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_sources= new_sources= 2024-04-22T20:10:40.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.936+0000 7f221d40f800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential sources added=[] removed=[] 2024-04-22T20:10:40.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.936+0000 7f221d40f800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): orig_dests= new_dests= 2024-04-22T20:10:40.935 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.936+0000 7f221d40f800 20 handle_bi_update(): bucket=:abort-at-do-reshard[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28]): potential dests added=[] removed=[] 2024-04-22T20:10:40.936 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.937+0000 7f221d40f800 1 execute INFO: reshard of bucket "abort-at-do-reshard" completed successfully 2024-04-22T20:10:40.938 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.939+0000 7f20cefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:40.941 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.941+0000 7f221d40f800 20 remove_watcher() i=0 2024-04-22T20:10:40.941 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.941+0000 7f221d40f800 2 removed watcher, disabling cache 2024-04-22T20:10:40.942 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.942+0000 7f221d40f800 20 remove_watcher() i=1 2024-04-22T20:10:40.943 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.944+0000 7f221d40f800 20 remove_watcher() i=2 2024-04-22T20:10:40.944 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.945+0000 7f221d40f800 20 remove_watcher() i=3 2024-04-22T20:10:40.945 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.946+0000 7f221d40f800 20 remove_watcher() i=4 2024-04-22T20:10:40.946 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.947+0000 7f221d40f800 20 remove_watcher() i=5 2024-04-22T20:10:40.947 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.948+0000 7f221d40f800 20 remove_watcher() i=6 2024-04-22T20:10:40.948 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:40.949+0000 7f221d40f800 20 remove_watcher() i=7 2024-04-22T20:10:40.962 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-04-22T20:10:40.990 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:40.991 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:41.024 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.025+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.026+0000 7fe0c1319800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.026+0000 7fe0c1319800 20 realm 2024-04-22T20:10:41.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.026+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.026+0000 7fe0c1319800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.026+0000 7fe0c1319800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:41.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.026+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.027+0000 7fe0c1319800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.027+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.027 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.027+0000 7fe0c1319800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:41.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.027+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.028+0000 7fe0c1319800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:41.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.028+0000 7fe0c1319800 20 searching for the correct realm 2024-04-22T20:10:41.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.031+0000 7fe0bf620640 0 -- 172.21.15.136:0/3284636864 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fe0b8035780 msgr2=0x7fe0b801c6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:41.034 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.035+0000 7fe0be61e640 0 -- 172.21.15.136:0/3284636864 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fe0b802b990 msgr2=0x7fe0b8029430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:41.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.241+0000 7fe0c1319800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:41.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.241+0000 7fe0c1319800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:41.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.241+0000 7fe0c1319800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:41.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.241+0000 7fe0c1319800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:41.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.241+0000 7fe0c1319800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:41.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.241+0000 7fe0c1319800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:41.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.241+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:41.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:41.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 20 zone default found 2024-04-22T20:10:41.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 4 Realm: () 2024-04-22T20:10:41.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:41.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:41.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:41.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 20 zonegroup default 2024-04-22T20:10:41.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.242+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.243+0000 7fe0c1319800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.243+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.243+0000 7fe0c1319800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.243+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.243+0000 7fe0c1319800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.243+0000 7fe0c1319800 20 started sync module instance, tier type = 2024-04-22T20:10:41.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.243+0000 7fe0c1319800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:41.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.253+0000 7fe0c1319800 20 add_watcher() i=0 2024-04-22T20:10:41.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.253+0000 7fe0c1319800 20 add_watcher() i=1 2024-04-22T20:10:41.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.253+0000 7fe0c1319800 20 add_watcher() i=2 2024-04-22T20:10:41.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.253+0000 7fe0c1319800 20 add_watcher() i=3 2024-04-22T20:10:41.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.253+0000 7fe0c1319800 20 add_watcher() i=4 2024-04-22T20:10:41.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.253+0000 7fe0c1319800 20 add_watcher() i=5 2024-04-22T20:10:41.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.253+0000 7fe0c1319800 20 add_watcher() i=6 2024-04-22T20:10:41.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.254+0000 7fe0c1319800 20 add_watcher() i=7 2024-04-22T20:10:41.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.254+0000 7fe0c1319800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:41.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fdf757e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fe0c1319800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fe0c1319800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fe0c1319800 5 note: GC not initialized 2024-04-22T20:10:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fdeeffff640 20 reqs_thread_entry: start 2024-04-22T20:10:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fe0c1319800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fe0c1319800 20 Filter name: none 2024-04-22T20:10:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fdeedffb640 20 reqs_thread_entry: start 2024-04-22T20:10:41.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.256+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.257+0000 7fe0c1319800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:41.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.257+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.258+0000 7fe0c1319800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:41.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.258+0000 7fe0c1319800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:41.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.258+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.258+0000 7fe0c1319800 20 rados_obj.operate() r=0 bl.length=187 2024-04-22T20:10:41.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.258+0000 7fe0c1319800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.259+0000 7fe0c1319800 20 rados_obj.operate() r=0 bl.length=401 2024-04-22T20:10:41.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.259+0000 7fe0c1319800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:41.259 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.260+0000 7fdf757e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:41.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.262+0000 7fe0c1319800 20 remove_watcher() i=0 2024-04-22T20:10:41.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.262+0000 7fe0c1319800 2 removed watcher, disabling cache 2024-04-22T20:10:41.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.263+0000 7fe0c1319800 20 remove_watcher() i=1 2024-04-22T20:10:41.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.264+0000 7fe0c1319800 20 remove_watcher() i=2 2024-04-22T20:10:41.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.265+0000 7fe0c1319800 20 remove_watcher() i=3 2024-04-22T20:10:41.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.266+0000 7fe0c1319800 20 remove_watcher() i=4 2024-04-22T20:10:41.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.267+0000 7fe0c1319800 20 remove_watcher() i=5 2024-04-22T20:10:41.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.268+0000 7fe0c1319800 20 remove_watcher() i=6 2024-04-22T20:10:41.268 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.269+0000 7fe0c1319800 20 remove_watcher() i=7 2024-04-22T20:10:41.278 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:41.278 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:41.278 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: abort-at-do-reshard id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28 num_objects: 19 size_kb: 1 num_shards: 2 2024-04-22T20:10:41.278 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:41.280 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:41.280 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:41.300 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:41.300 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:41.510 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:41.510 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:41.573 INFO:tasks.workunit.client.0.smithi136.stderr: test: reshard versioned bucket 2024-04-22T20:10:41.575 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket myver 2024-04-22T20:10:41.604 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:41.604 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:41.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.638+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.640+0000 7f2ab1b2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.640+0000 7f2ab1b2d800 20 realm 2024-04-22T20:10:41.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.640+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.639 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.640+0000 7f2ab1b2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.640+0000 7f2ab1b2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:41.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.640+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.640+0000 7f2ab1b2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.640+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.841 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.842+0000 7f2ab1b2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:41.841 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.842+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.843+0000 7f2ab1b2d800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:41.842 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.843+0000 7f2ab1b2d800 20 searching for the correct realm 2024-04-22T20:10:41.844 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.845+0000 7f2aadf79640 0 -- 172.21.15.136:0/3178880840 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x5584ec21d250 msgr2=0x5584ec21f640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:41.847 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.848+0000 7f2aadf79640 0 -- 172.21.15.136:0/3178880840 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f297807ba00 msgr2=0x7f297807de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.854+0000 7f2ab1b2d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.854+0000 7f2ab1b2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.854+0000 7f2ab1b2d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.854+0000 7f2ab1b2d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.854+0000 7f2ab1b2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.854+0000 7f2ab1b2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.854+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:41.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.855 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 20 zone default found 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 4 Realm: () 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 20 zonegroup default 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.855+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.856+0000 7f2ab1b2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.856+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.856+0000 7f2ab1b2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.856+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.856+0000 7f2ab1b2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.856+0000 7f2ab1b2d800 20 started sync module instance, tier type = 2024-04-22T20:10:41.856 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.856+0000 7f2ab1b2d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:41.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.865+0000 7f2ab1b2d800 20 add_watcher() i=0 2024-04-22T20:10:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.865+0000 7f2ab1b2d800 20 add_watcher() i=1 2024-04-22T20:10:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.865+0000 7f2ab1b2d800 20 add_watcher() i=2 2024-04-22T20:10:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.865+0000 7f2ab1b2d800 20 add_watcher() i=3 2024-04-22T20:10:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.865+0000 7f2ab1b2d800 20 add_watcher() i=4 2024-04-22T20:10:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.865+0000 7f2ab1b2d800 20 add_watcher() i=5 2024-04-22T20:10:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.865+0000 7f2ab1b2d800 20 add_watcher() i=6 2024-04-22T20:10:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.866+0000 7f2ab1b2d800 20 add_watcher() i=7 2024-04-22T20:10:41.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.866+0000 7f2ab1b2d800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:41.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.868+0000 7f29637e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:41.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.868+0000 7f2ab1b2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:41.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.868+0000 7f2ab1b2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:41.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.868+0000 7f2ab1b2d800 5 note: GC not initialized 2024-04-22T20:10:41.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.869+0000 7f29627e4640 20 reqs_thread_entry: start 2024-04-22T20:10:41.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.869+0000 7f2ab1b2d800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:41.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.869+0000 7f2ab1b2d800 20 Filter name: none 2024-04-22T20:10:41.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.869+0000 7f28d3fff640 20 reqs_thread_entry: start 2024-04-22T20:10:41.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.869+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.869+0000 7f2ab1b2d800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.869+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.870+0000 7f2ab1b2d800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.870+0000 7f2ab1b2d800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.870+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.870+0000 7f2ab1b2d800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.870+0000 7f2ab1b2d800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.870+0000 7f2ab1b2d800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:41.870 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.870+0000 7f2ab1b2d800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:41.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.872+0000 7f29637e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:41.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.874+0000 7f2ab1b2d800 20 remove_watcher() i=0 2024-04-22T20:10:41.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.874+0000 7f2ab1b2d800 2 removed watcher, disabling cache 2024-04-22T20:10:41.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.875+0000 7f2ab1b2d800 20 remove_watcher() i=1 2024-04-22T20:10:41.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.876+0000 7f2ab1b2d800 20 remove_watcher() i=2 2024-04-22T20:10:41.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.877+0000 7f2ab1b2d800 20 remove_watcher() i=3 2024-04-22T20:10:41.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.878+0000 7f2ab1b2d800 20 remove_watcher() i=4 2024-04-22T20:10:41.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.879+0000 7f2ab1b2d800 20 remove_watcher() i=5 2024-04-22T20:10:41.879 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.880+0000 7f2ab1b2d800 20 remove_watcher() i=6 2024-04-22T20:10:41.880 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.881+0000 7f2ab1b2d800 20 remove_watcher() i=7 2024-04-22T20:10:41.890 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:41.890 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:41.890 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: myver id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 num_objects: 0 size_kb: 0 num_shards: 1 2024-04-22T20:10:41.890 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:41.891 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 2 2024-04-22T20:10:41.921 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:41.921 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:41.956 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.957+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.959+0000 7f58caf33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.959+0000 7f58caf33800 20 realm 2024-04-22T20:10:41.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.959+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.959+0000 7f58caf33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.959+0000 7f58caf33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:41.958 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.959+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.959+0000 7f58caf33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:41.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.959+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.960+0000 7f58caf33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:41.959 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.960+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:41.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.960+0000 7f58caf33800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:41.960 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.960+0000 7f58caf33800 20 searching for the correct realm 2024-04-22T20:10:41.962 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.963+0000 7f58bffff640 0 -- 172.21.15.136:0/3095428348 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f58b805faf0 msgr2=0x7f58b805e750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:41.966 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:41.967+0000 7f58bffff640 0 -- 172.21.15.136:0/3095428348 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f58c00619b0 msgr2=0x7f58c0059a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:42.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:42.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:42.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:42.172 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:42.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:42.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:42.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:42.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.173+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:42.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 20 zone default found 2024-04-22T20:10:42.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 4 Realm: () 2024-04-22T20:10:42.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:42.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 20 zonegroup default 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.174+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.175+0000 7f58caf33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.175+0000 7f58caf33800 20 started sync module instance, tier type = 2024-04-22T20:10:42.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.175+0000 7f58caf33800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.184+0000 7f58caf33800 20 add_watcher() i=0 2024-04-22T20:10:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.184+0000 7f58caf33800 20 add_watcher() i=1 2024-04-22T20:10:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.184+0000 7f58caf33800 20 add_watcher() i=2 2024-04-22T20:10:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.184+0000 7f58caf33800 20 add_watcher() i=3 2024-04-22T20:10:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.184+0000 7f58caf33800 20 add_watcher() i=4 2024-04-22T20:10:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.184+0000 7f58caf33800 20 add_watcher() i=5 2024-04-22T20:10:42.183 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.184+0000 7f58caf33800 20 add_watcher() i=6 2024-04-22T20:10:42.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.185+0000 7f58caf33800 20 add_watcher() i=7 2024-04-22T20:10:42.184 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.185+0000 7f58caf33800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:42.186 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f577d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f58caf33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f58caf33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f58caf33800 5 note: GC not initialized 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f56f7fff640 20 reqs_thread_entry: start 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f58caf33800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f58caf33800 20 Filter name: none 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f56f5ffb640 20 reqs_thread_entry: start 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f58caf33800 10 cache get: name=default.rgw.meta+root+myver : miss 2024-04-22T20:10:42.187 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.187+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:42.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-04-22T20:10:42.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 10 adding default.rgw.meta+root+myver to cache LRU end 2024-04-22T20:10:42.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 : miss 2024-04-22T20:10:42.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:42.188 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:42.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 10 adding default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:42.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:42.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:42.189 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.188+0000 7f58caf33800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 2024-04-22T20:10:42.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.192+0000 7f58caf33800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:42.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.192+0000 7f58caf33800 10 moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:42.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.192+0000 7f58caf33800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:42.191 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.192+0000 7f58caf33800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18, ofs0, ns] 2024-04-22T20:10:42.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.193+0000 7f57aa7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562480 cookie 94676227271456 notifier 7104 bl.length()=813 2024-04-22T20:10:42.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.193+0000 7f57aa7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:42.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.193+0000 7f57aa7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:42.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.193+0000 7f57aa7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:42.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.195+0000 7f58caf33800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_sources= new_sources= 2024-04-22T20:10:42.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.195+0000 7f58caf33800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential sources added=[] removed=[] 2024-04-22T20:10:42.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.195+0000 7f58caf33800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_dests= new_dests= 2024-04-22T20:10:42.194 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.195+0000 7f58caf33800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential dests added=[] removed=[] 2024-04-22T20:10:42.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.196+0000 7f58caf33800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.1.0 2024-04-22T20:10:42.195 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.196+0000 7f58caf33800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.1.0 2024-04-22T20:10:42.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.196+0000 7f58caf33800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.1.1 2024-04-22T20:10:42.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.196+0000 7f58caf33800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.1.1 2024-04-22T20:10:42.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.196+0000 7f58caf33800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.0 2024-04-22T20:10:42.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.197+0000 7f58caf33800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 : hit (requested=0x17, cached=0x17) 2024-04-22T20:10:42.196 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.197+0000 7f58caf33800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 2024-04-22T20:10:42.197 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.198+0000 7f58caf33800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:42.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.198+0000 7f58caf33800 10 moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:42.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.198+0000 7f58caf33800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:42.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.198+0000 7f58caf33800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18, ofs0, ns] 2024-04-22T20:10:42.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.198+0000 7f57a9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562481 cookie 94676227271456 notifier 7104 bl.length()=781 2024-04-22T20:10:42.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.198+0000 7f57a9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:42.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.198+0000 7f57a9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:42.198 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.198+0000 7f57a9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:42.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.200+0000 7f58caf33800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_sources= new_sources= 2024-04-22T20:10:42.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.200+0000 7f58caf33800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential sources added=[] removed=[] 2024-04-22T20:10:42.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.200+0000 7f58caf33800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_dests= new_dests= 2024-04-22T20:10:42.199 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.200+0000 7f58caf33800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential dests added=[] removed=[] 2024-04-22T20:10:42.200 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.201+0000 7f58caf33800 1 execute INFO: reshard of bucket "myver" completed successfully 2024-04-22T20:10:42.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.203+0000 7f577d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:42.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.206+0000 7f58caf33800 20 remove_watcher() i=0 2024-04-22T20:10:42.205 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.206+0000 7f58caf33800 2 removed watcher, disabling cache 2024-04-22T20:10:42.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.207+0000 7f58caf33800 20 remove_watcher() i=1 2024-04-22T20:10:42.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.208+0000 7f58caf33800 20 remove_watcher() i=2 2024-04-22T20:10:42.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.209+0000 7f58caf33800 20 remove_watcher() i=3 2024-04-22T20:10:42.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.210+0000 7f58caf33800 20 remove_watcher() i=4 2024-04-22T20:10:42.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.211+0000 7f58caf33800 20 remove_watcher() i=5 2024-04-22T20:10:42.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.212+0000 7f58caf33800 20 remove_watcher() i=6 2024-04-22T20:10:42.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.213+0000 7f58caf33800 20 remove_watcher() i=7 2024-04-22T20:10:42.222 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket myver 2024-04-22T20:10:42.250 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:42.250 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:42.283 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.284+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.286+0000 7f8455213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.286+0000 7f8455213800 20 realm 2024-04-22T20:10:42.285 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.286+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.286+0000 7f8455213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.286+0000 7f8455213800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:42.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.286+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.286+0000 7f8455213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.286+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.286 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.287+0000 7f8455213800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:42.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.287+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.288+0000 7f8455213800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:42.287 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.288+0000 7f8455213800 20 searching for the correct realm 2024-04-22T20:10:42.289 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.290+0000 7f8451022640 0 -- 172.21.15.136:0/4020912669 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f844c09a030 msgr2=0x7f844c0919f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:42.495 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.495+0000 7f844bfff640 0 -- 172.21.15.136:0/4020912669 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f844c05ca00 msgr2=0x7f844c052290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.702+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.702 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:42.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.703 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 20 zone default found 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 4 Realm: () 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 20 zonegroup default 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.703+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.704+0000 7f8455213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.704+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.704+0000 7f8455213800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.704+0000 7f8455213800 20 started sync module instance, tier type = 2024-04-22T20:10:42.704 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.704+0000 7f8455213800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.713+0000 7f8455213800 20 add_watcher() i=0 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.713+0000 7f8455213800 20 add_watcher() i=1 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.713+0000 7f8455213800 20 add_watcher() i=2 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.713+0000 7f8455213800 20 add_watcher() i=3 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.713+0000 7f8455213800 20 add_watcher() i=4 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.713+0000 7f8455213800 20 add_watcher() i=5 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.714+0000 7f8455213800 20 add_watcher() i=6 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.714+0000 7f8455213800 20 add_watcher() i=7 2024-04-22T20:10:42.713 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.714+0000 7f8455213800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:42.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.716+0000 7f8306fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:42.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.716+0000 7f8455213800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:42.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.716+0000 7f8455213800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:42.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.717+0000 7f8455213800 5 note: GC not initialized 2024-04-22T20:10:42.716 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.717+0000 7f8305fe3640 20 reqs_thread_entry: start 2024-04-22T20:10:42.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.717+0000 7f8455213800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:42.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.717+0000 7f8455213800 20 Filter name: none 2024-04-22T20:10:42.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.717+0000 7f82777fe640 20 reqs_thread_entry: start 2024-04-22T20:10:42.717 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.717+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.717+0000 7f8455213800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.717+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.718+0000 7f8455213800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.718+0000 7f8455213800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.718+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.718+0000 7f8455213800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.718+0000 7f8455213800 20 rados->read ofs=0 len=0 2024-04-22T20:10:42.718 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.719+0000 7f8455213800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:42.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.719+0000 7f8455213800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:42.719 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.720+0000 7f8306fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:42.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.722+0000 7f8455213800 20 remove_watcher() i=0 2024-04-22T20:10:42.721 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.722+0000 7f8455213800 2 removed watcher, disabling cache 2024-04-22T20:10:42.722 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.723+0000 7f8455213800 20 remove_watcher() i=1 2024-04-22T20:10:42.723 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.724+0000 7f8455213800 20 remove_watcher() i=2 2024-04-22T20:10:42.724 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.725+0000 7f8455213800 20 remove_watcher() i=3 2024-04-22T20:10:42.725 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.726+0000 7f8455213800 20 remove_watcher() i=4 2024-04-22T20:10:42.726 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.727+0000 7f8455213800 20 remove_watcher() i=5 2024-04-22T20:10:42.727 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.728+0000 7f8455213800 20 remove_watcher() i=6 2024-04-22T20:10:42.728 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:42.729+0000 7f8455213800 20 remove_watcher() i=7 2024-04-22T20:10:42.738 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:42.738 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:42.738 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: myver id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 num_objects: 0 size_kb: 0 num_shards: 2 2024-04-22T20:10:42.738 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:42.740 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:42.740 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:42.763 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:42.763 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:42.817 INFO:tasks.workunit.client.0.smithi136.stderr: test: reshard removes olh entries with empty name 2024-04-22T20:10:42.819 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:42.819 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:42.889 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:42.889 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:42.994 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-04-22T20:10:43.021 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:43.021 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:43.054 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.055+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.057+0000 7f338611f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.056 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.057+0000 7f338611f800 20 realm 2024-04-22T20:10:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.057+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.057+0000 7f338611f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.057+0000 7f338611f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.057+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.057+0000 7f338611f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.057+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.058+0000 7f338611f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:43.057 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.058+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.059+0000 7f338611f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:43.058 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.059+0000 7f338611f800 20 searching for the correct realm 2024-04-22T20:10:43.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.062+0000 7f33825ef640 0 -- 172.21.15.136:0/4277500529 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f337c05f940 msgr2=0x7f337c08c410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:43.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.065+0000 7f33825ef640 0 -- 172.21.15.136:0/4277500529 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x5645492009e0 msgr2=0x564549202dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.071+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.071 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:43.072 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 zone default found 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 4 Realm: () 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 zonegroup default 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.072+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.073+0000 7f338611f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.073+0000 7f338611f800 20 started sync module instance, tier type = 2024-04-22T20:10:43.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.073+0000 7f338611f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:43.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.082+0000 7f338611f800 20 add_watcher() i=0 2024-04-22T20:10:43.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.082+0000 7f338611f800 20 add_watcher() i=1 2024-04-22T20:10:43.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.082+0000 7f338611f800 20 add_watcher() i=2 2024-04-22T20:10:43.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.082+0000 7f338611f800 20 add_watcher() i=3 2024-04-22T20:10:43.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.082+0000 7f338611f800 20 add_watcher() i=4 2024-04-22T20:10:43.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.082+0000 7f338611f800 20 add_watcher() i=5 2024-04-22T20:10:43.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.082+0000 7f338611f800 20 add_watcher() i=6 2024-04-22T20:10:43.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.083+0000 7f338611f800 20 add_watcher() i=7 2024-04-22T20:10:43.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.083+0000 7f338611f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.085+0000 7f32337fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.085+0000 7f338611f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.085+0000 7f338611f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.085+0000 7f338611f800 5 note: GC not initialized 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.085+0000 7f32327fc640 20 reqs_thread_entry: start 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.085+0000 7f338611f800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.085+0000 7f338611f800 20 Filter name: none 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.086+0000 7f31a3fff640 20 reqs_thread_entry: start 2024-04-22T20:10:43.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.086+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.086+0000 7f338611f800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:10:43.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.086+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.086+0000 7f338611f800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:10:43.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.087+0000 7f338611f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:43.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.087+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.087+0000 7f338611f800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:10:43.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.087+0000 7f338611f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.087+0000 7f338611f800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:10:43.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.087+0000 7f338611f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:43.087 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.088+0000 7f32337fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:43.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.091+0000 7f338611f800 20 remove_watcher() i=0 2024-04-22T20:10:43.090 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.091+0000 7f338611f800 2 removed watcher, disabling cache 2024-04-22T20:10:43.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.092+0000 7f338611f800 20 remove_watcher() i=1 2024-04-22T20:10:43.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.093+0000 7f338611f800 20 remove_watcher() i=2 2024-04-22T20:10:43.093 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.094+0000 7f338611f800 20 remove_watcher() i=3 2024-04-22T20:10:43.094 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.095+0000 7f338611f800 20 remove_watcher() i=4 2024-04-22T20:10:43.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.096+0000 7f338611f800 20 remove_watcher() i=5 2024-04-22T20:10:43.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.096+0000 7f338611f800 20 remove_watcher() i=6 2024-04-22T20:10:43.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.097+0000 7f338611f800 20 remove_watcher() i=7 2024-04-22T20:10:43.106 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:43.106 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:43.106 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: a-bucket id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 num_objects: 0 size_kb: 0 num_shards: 3 2024-04-22T20:10:43.106 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:43.108 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket layout --bucket a-bucket 2024-04-22T20:10:43.135 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:43.135 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:43.168 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.169+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.170+0000 7ff202e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.170+0000 7ff202e62800 20 realm 2024-04-22T20:10:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.170+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.170+0000 7ff202e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.170+0000 7ff202e62800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.170+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.171+0000 7ff202e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.170 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.171+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.171+0000 7ff202e62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:43.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.171+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.172+0000 7ff202e62800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:43.171 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.172+0000 7ff202e62800 20 searching for the correct realm 2024-04-22T20:10:43.173 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.174+0000 7ff201162640 0 -- 172.21.15.136:0/2489842718 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7ff1fc001530 msgr2=0x7ff1fc001990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:43.178 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.179+0000 7ff1fbfff640 0 -- 172.21.15.136:0/2489842718 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7ff1fc018930 msgr2=0x7ff1fc032400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:43.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:43.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:43.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:43.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:43.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:43.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:43.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.386 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:43.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.386+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:43.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 20 zone default found 2024-04-22T20:10:43.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 4 Realm: () 2024-04-22T20:10:43.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:43.387 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 20 zonegroup default 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.387+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.388+0000 7ff202e62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.388+0000 7ff202e62800 20 started sync module instance, tier type = 2024-04-22T20:10:43.388 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.388+0000 7ff202e62800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:43.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.397+0000 7ff202e62800 20 add_watcher() i=0 2024-04-22T20:10:43.396 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.397+0000 7ff202e62800 20 add_watcher() i=1 2024-04-22T20:10:43.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.397+0000 7ff202e62800 20 add_watcher() i=2 2024-04-22T20:10:43.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.397+0000 7ff202e62800 20 add_watcher() i=3 2024-04-22T20:10:43.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.397+0000 7ff202e62800 20 add_watcher() i=4 2024-04-22T20:10:43.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.397+0000 7ff202e62800 20 add_watcher() i=5 2024-04-22T20:10:43.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.397+0000 7ff202e62800 20 add_watcher() i=6 2024-04-22T20:10:43.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.398+0000 7ff202e62800 20 add_watcher() i=7 2024-04-22T20:10:43.397 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.398+0000 7ff202e62800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:43.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.400+0000 7ff0b6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:43.399 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.400+0000 7ff202e62800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:43.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.400+0000 7ff202e62800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:43.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.400+0000 7ff202e62800 5 note: GC not initialized 2024-04-22T20:10:43.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.401+0000 7ff0b5fe3640 20 reqs_thread_entry: start 2024-04-22T20:10:43.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.401+0000 7ff202e62800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:43.400 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.401+0000 7ff202e62800 20 Filter name: none 2024-04-22T20:10:43.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.401+0000 7ff0277fe640 20 reqs_thread_entry: start 2024-04-22T20:10:43.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.401+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.402+0000 7ff202e62800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:10:43.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.402+0000 7ff202e62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.402+0000 7ff202e62800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:10:43.401 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.402+0000 7ff202e62800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:43.402 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.403+0000 7ff0b6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:43.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.405+0000 7ff202e62800 20 remove_watcher() i=0 2024-04-22T20:10:43.405 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.405+0000 7ff202e62800 2 removed watcher, disabling cache 2024-04-22T20:10:43.406 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.407+0000 7ff202e62800 20 remove_watcher() i=1 2024-04-22T20:10:43.407 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.408+0000 7ff202e62800 20 remove_watcher() i=2 2024-04-22T20:10:43.408 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.409+0000 7ff202e62800 20 remove_watcher() i=3 2024-04-22T20:10:43.409 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.410+0000 7ff202e62800 20 remove_watcher() i=4 2024-04-22T20:10:43.410 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.411+0000 7ff202e62800 20 remove_watcher() i=5 2024-04-22T20:10:43.411 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.412+0000 7ff202e62800 20 remove_watcher() i=6 2024-04-22T20:10:43.412 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.413+0000 7ff202e62800 20 remove_watcher() i=7 2024-04-22T20:10:43.423 INFO:tasks.workunit.client.0.smithi136.stderr:rados -p default.rgw.buckets.index setomapval .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.0 --omap-key-file /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_key < /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_val 2024-04-22T20:10:43.448 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:43.448 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:43.464 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket a-bucket 2024-04-22T20:10:43.491 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:43.491 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:43.526 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.527+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.528+0000 7fce39d1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.528+0000 7fce39d1f800 20 realm 2024-04-22T20:10:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.528+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.529+0000 7fce39d1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.529+0000 7fce39d1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.529+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.529+0000 7fce39d1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.528 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.529+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.530+0000 7fce39d1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:43.529 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.530+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.530+0000 7fce39d1f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:43.530 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.530+0000 7fce39d1f800 20 searching for the correct realm 2024-04-22T20:10:43.531 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.532+0000 7fce361e8640 0 -- 172.21.15.136:0/1281701990 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55c23648cf50 msgr2=0x55c23648f340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:43.535 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.536+0000 7fce361e8640 0 -- 172.21.15.136:0/1281701990 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7fcd0007b930 msgr2=0x7fcd0007dd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:43.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:43.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:43.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:43.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:43.541 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:43.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:43.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:43.542 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.542+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 zone default found 2024-04-22T20:10:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 4 Realm: () 2024-04-22T20:10:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 zonegroup default 2024-04-22T20:10:43.543 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 started sync module instance, tier type = 2024-04-22T20:10:43.544 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.543+0000 7fce39d1f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:43.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.552+0000 7fce39d1f800 20 add_watcher() i=0 2024-04-22T20:10:43.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.552+0000 7fce39d1f800 20 add_watcher() i=1 2024-04-22T20:10:43.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.552+0000 7fce39d1f800 20 add_watcher() i=2 2024-04-22T20:10:43.551 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.552+0000 7fce39d1f800 20 add_watcher() i=3 2024-04-22T20:10:43.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.552+0000 7fce39d1f800 20 add_watcher() i=4 2024-04-22T20:10:43.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.552+0000 7fce39d1f800 20 add_watcher() i=5 2024-04-22T20:10:43.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.552+0000 7fce39d1f800 20 add_watcher() i=6 2024-04-22T20:10:43.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.553+0000 7fce39d1f800 20 add_watcher() i=7 2024-04-22T20:10:43.552 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.553+0000 7fce39d1f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.555+0000 7fcce77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.555+0000 7fce39d1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.555+0000 7fce39d1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.555+0000 7fce39d1f800 5 note: GC not initialized 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.555+0000 7fcce67fc640 20 reqs_thread_entry: start 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.555+0000 7fce39d1f800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.555+0000 7fce39d1f800 20 Filter name: none 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.555+0000 7fcc57fff640 20 reqs_thread_entry: start 2024-04-22T20:10:43.555 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 20 INFO: main: starting shard=0 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.0 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.556+0000 7fce39d1f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.0 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.557+0000 7fce39d1f800 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.557+0000 7fce39d1f800 20 INFO: main: starting shard=1 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.557+0000 7fce39d1f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.1 2024-04-22T20:10:43.557 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.557+0000 7fce39d1f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.1 2024-04-22T20:10:43.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.557+0000 7fce39d1f800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:43.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.557+0000 7fce39d1f800 20 INFO: main: starting shard=2 2024-04-22T20:10:43.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.557+0000 7fce39d1f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.2 2024-04-22T20:10:43.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.557+0000 7fce39d1f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.2 2024-04-22T20:10:43.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.558+0000 7fce39d1f800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:43.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.558+0000 7fce39d1f800 20 INFO: main: done 2024-04-22T20:10:43.558 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.559+0000 7fcce77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:43.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.561+0000 7fce39d1f800 20 remove_watcher() i=0 2024-04-22T20:10:43.560 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.561+0000 7fce39d1f800 2 removed watcher, disabling cache 2024-04-22T20:10:43.561 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.562+0000 7fce39d1f800 20 remove_watcher() i=1 2024-04-22T20:10:43.562 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.563+0000 7fce39d1f800 20 remove_watcher() i=2 2024-04-22T20:10:43.563 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.564+0000 7fce39d1f800 20 remove_watcher() i=3 2024-04-22T20:10:43.564 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.565+0000 7fce39d1f800 20 remove_watcher() i=4 2024-04-22T20:10:43.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.566+0000 7fce39d1f800 20 remove_watcher() i=5 2024-04-22T20:10:43.565 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.566+0000 7fce39d1f800 20 remove_watcher() i=6 2024-04-22T20:10:43.566 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.567+0000 7fce39d1f800 20 remove_watcher() i=7 2024-04-22T20:10:43.577 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 1 --yes-i-really-mean-it 2024-04-22T20:10:43.608 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:43.608 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:43.641 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.642+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.644+0000 7f4f8e125800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.644+0000 7f4f8e125800 20 realm 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.644+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.644+0000 7f4f8e125800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.644+0000 7f4f8e125800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.644+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.644+0000 7f4f8e125800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.644+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.645+0000 7f4f8e125800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:43.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.645+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:43.645 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.646+0000 7f4f8e125800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:43.645 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.646+0000 7f4f8e125800 20 searching for the correct realm 2024-04-22T20:10:43.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.648+0000 7f4f8a6b8640 0 -- 172.21.15.136:0/2530165667 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f4f84025070 msgr2=0x7f4f84025960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:43.854 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:43.854+0000 7f4f896b6640 0 -- 172.21.15.136:0/2530165667 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f4f840243b0 msgr2=0x7f4f84024830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:44.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:44.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:44.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:44.061 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:44.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:44.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:44.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.062 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:44.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.062+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.063 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 20 zone default found 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 4 Realm: () 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 20 zonegroup default 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.064 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.063+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.064+0000 7f4f8e125800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.064+0000 7f4f8e125800 20 started sync module instance, tier type = 2024-04-22T20:10:44.065 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.064+0000 7f4f8e125800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.073+0000 7f4f8e125800 20 add_watcher() i=0 2024-04-22T20:10:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.073+0000 7f4f8e125800 20 add_watcher() i=1 2024-04-22T20:10:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.073+0000 7f4f8e125800 20 add_watcher() i=2 2024-04-22T20:10:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.073+0000 7f4f8e125800 20 add_watcher() i=3 2024-04-22T20:10:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.073+0000 7f4f8e125800 20 add_watcher() i=4 2024-04-22T20:10:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.073+0000 7f4f8e125800 20 add_watcher() i=5 2024-04-22T20:10:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.073+0000 7f4f8e125800 20 add_watcher() i=6 2024-04-22T20:10:44.073 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.074+0000 7f4f8e125800 20 add_watcher() i=7 2024-04-22T20:10:44.074 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.074+0000 7f4f8e125800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:44.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.076+0000 7f4e3b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:44.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4f8e125800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:44.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4f8e125800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:44.076 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4f8e125800 5 note: GC not initialized 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4e3a7fc640 20 reqs_thread_entry: start 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4f8e125800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4f8e125800 20 Filter name: none 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4dabfff640 20 reqs_thread_entry: start 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4f8e125800 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.077+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 : miss 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.078 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:10:44.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:10:44.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:10:44.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:44.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:44.079 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.078+0000 7f4f8e125800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 2024-04-22T20:10:44.081 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.082+0000 7f4f8e125800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:10:44.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.082+0000 7f4f8e125800 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:10:44.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.082+0000 7f4f8e125800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:44.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.082+0000 7f4f8e125800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17, ofs0, ns] 2024-04-22T20:10:44.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.082+0000 7f4e6cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562395 cookie 94358966172736 notifier 6806 bl.length()=819 2024-04-22T20:10:44.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.082+0000 7f4e6cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:10:44.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.082+0000 7f4e6cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:10:44.082 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.082+0000 7f4e6cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:44.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.084+0000 7f4f8e125800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_sources= new_sources= 2024-04-22T20:10:44.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.084+0000 7f4f8e125800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential sources added=[] removed=[] 2024-04-22T20:10:44.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.084+0000 7f4f8e125800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_dests= new_dests= 2024-04-22T20:10:44.083 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.084+0000 7f4f8e125800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential dests added=[] removed=[] 2024-04-22T20:10:44.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.085+0000 7f4f8e125800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.3.0 2024-04-22T20:10:44.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.085+0000 7f4f8e125800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.3.0 2024-04-22T20:10:44.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.085+0000 7f4f8e125800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.0 2024-04-22T20:10:44.085 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.085+0000 7f4f8e125800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.0 2024-04-22T20:10:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.086+0000 7f4f8e125800 10 Dropping entry with empty name, idx=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2024-04-22T20:10:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.086+0000 7f4f8e125800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.1 2024-04-22T20:10:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.086+0000 7f4f8e125800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.1 2024-04-22T20:10:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.086+0000 7f4f8e125800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.2 2024-04-22T20:10:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.086+0000 7f4f8e125800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.2.2 2024-04-22T20:10:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.087+0000 7f4f8e125800 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 : hit (requested=0x17, cached=0x17) 2024-04-22T20:10:44.086 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.087+0000 7f4f8e125800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 2024-04-22T20:10:44.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.088+0000 7f4f8e125800 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:10:44.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.088+0000 7f4f8e125800 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:10:44.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.088+0000 7f4f8e125800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:44.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.088+0000 7f4f8e125800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17, ofs0, ns] 2024-04-22T20:10:44.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.089+0000 7f4e6cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562396 cookie 94358966172736 notifier 6806 bl.length()=787 2024-04-22T20:10:44.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.089+0000 7f4e6cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 info.flags=0x17 2024-04-22T20:10:44.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.089+0000 7f4e6cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17 to cache LRU end 2024-04-22T20:10:44.088 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.089+0000 7f4e6cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:44.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.090+0000 7f4f8e125800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_sources= new_sources= 2024-04-22T20:10:44.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.090+0000 7f4f8e125800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential sources added=[] removed=[] 2024-04-22T20:10:44.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.090+0000 7f4f8e125800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): orig_dests= new_dests= 2024-04-22T20:10:44.089 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.090+0000 7f4f8e125800 20 handle_bi_update(): bucket=:a-bucket[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17]): potential dests added=[] removed=[] 2024-04-22T20:10:44.091 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.092+0000 7f4f8e125800 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-04-22T20:10:44.092 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.093+0000 7f4e3b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:44.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.096+0000 7f4f8e125800 20 remove_watcher() i=0 2024-04-22T20:10:44.095 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.096+0000 7f4f8e125800 2 removed watcher, disabling cache 2024-04-22T20:10:44.096 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.097+0000 7f4f8e125800 20 remove_watcher() i=1 2024-04-22T20:10:44.097 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.098+0000 7f4f8e125800 20 remove_watcher() i=2 2024-04-22T20:10:44.098 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.099+0000 7f4f8e125800 20 remove_watcher() i=3 2024-04-22T20:10:44.099 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.100+0000 7f4f8e125800 20 remove_watcher() i=4 2024-04-22T20:10:44.100 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.101+0000 7f4f8e125800 20 remove_watcher() i=5 2024-04-22T20:10:44.101 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.102+0000 7f4f8e125800 20 remove_watcher() i=6 2024-04-22T20:10:44.102 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.103+0000 7f4f8e125800 20 remove_watcher() i=7 2024-04-22T20:10:44.113 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket a-bucket 2024-04-22T20:10:44.140 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:44.140 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:44.174 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.174+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.175 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.176+0000 7fd7c1d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.176+0000 7fd7c1d35800 20 realm 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.176+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.176+0000 7fd7c1d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.176+0000 7fd7c1d35800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.176+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.176+0000 7fd7c1d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.176+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.177+0000 7fd7c1d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:44.176 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.177+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.178+0000 7fd7c1d35800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:44.177 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.178+0000 7fd7c1d35800 20 searching for the correct realm 2024-04-22T20:10:44.179 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.180+0000 7fd7be1b6640 0 -- 172.21.15.136:0/1710906483 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fd7b8060260 msgr2=0x7fd7b805eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:44.385 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.386+0000 7fd7bd1b4640 0 -- 172.21.15.136:0/1710906483 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fd7b80901c0 msgr2=0x7fd7b807f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:44.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:44.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:44.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:44.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:44.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:44.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:44.592 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.592+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:44.593 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:44.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 20 zone default found 2024-04-22T20:10:44.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 4 Realm: () 2024-04-22T20:10:44.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:44.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:44.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:44.594 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 20 zonegroup default 2024-04-22T20:10:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.593+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.594+0000 7fd7c1d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.594+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.594+0000 7fd7c1d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.594+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.594+0000 7fd7c1d35800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.594+0000 7fd7c1d35800 20 started sync module instance, tier type = 2024-04-22T20:10:44.595 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.594+0000 7fd7c1d35800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:44.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.604+0000 7fd7c1d35800 20 add_watcher() i=0 2024-04-22T20:10:44.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.604+0000 7fd7c1d35800 20 add_watcher() i=1 2024-04-22T20:10:44.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.604+0000 7fd7c1d35800 20 add_watcher() i=2 2024-04-22T20:10:44.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.604+0000 7fd7c1d35800 20 add_watcher() i=3 2024-04-22T20:10:44.603 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.604+0000 7fd7c1d35800 20 add_watcher() i=4 2024-04-22T20:10:44.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.604+0000 7fd7c1d35800 20 add_watcher() i=5 2024-04-22T20:10:44.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.604+0000 7fd7c1d35800 20 add_watcher() i=6 2024-04-22T20:10:44.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.605+0000 7fd7c1d35800 20 add_watcher() i=7 2024-04-22T20:10:44.604 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.605+0000 7fd7c1d35800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:44.606 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd66f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd7c1d35800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd7c1d35800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd7c1d35800 5 note: GC not initialized 2024-04-22T20:10:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd66e7fc640 20 reqs_thread_entry: start 2024-04-22T20:10:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd7c1d35800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd7c1d35800 20 Filter name: none 2024-04-22T20:10:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd5dffff640 20 reqs_thread_entry: start 2024-04-22T20:10:44.607 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.607+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.608+0000 7fd7c1d35800 20 rados_obj.operate() r=0 bl.length=176 2024-04-22T20:10:44.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.608+0000 7fd7c1d35800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.609+0000 7fd7c1d35800 20 rados_obj.operate() r=0 bl.length=390 2024-04-22T20:10:44.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.609+0000 7fd7c1d35800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:44.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.609+0000 7fd7c1d35800 20 INFO: main: max_entries=1000, index=gen=3, max_shards=1 2024-04-22T20:10:44.608 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.609+0000 7fd7c1d35800 20 INFO: main: starting shard=0 2024-04-22T20:10:44.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.609+0000 7fd7c1d35800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.3.0 2024-04-22T20:10:44.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.609+0000 7fd7c1d35800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17.3.0 2024-04-22T20:10:44.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.609+0000 7fd7c1d35800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:44.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.609+0000 7fd7c1d35800 20 INFO: main: done 2024-04-22T20:10:44.609 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.610+0000 7fd66f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:44.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.612+0000 7fd7c1d35800 20 remove_watcher() i=0 2024-04-22T20:10:44.611 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.612+0000 7fd7c1d35800 2 removed watcher, disabling cache 2024-04-22T20:10:44.612 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.613+0000 7fd7c1d35800 20 remove_watcher() i=1 2024-04-22T20:10:44.613 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.614+0000 7fd7c1d35800 20 remove_watcher() i=2 2024-04-22T20:10:44.614 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.615+0000 7fd7c1d35800 20 remove_watcher() i=3 2024-04-22T20:10:44.615 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.616+0000 7fd7c1d35800 20 remove_watcher() i=4 2024-04-22T20:10:44.616 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.617+0000 7fd7c1d35800 20 remove_watcher() i=5 2024-04-22T20:10:44.617 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.618+0000 7fd7c1d35800 20 remove_watcher() i=6 2024-04-22T20:10:44.618 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.619+0000 7fd7c1d35800 20 remove_watcher() i=7 2024-04-22T20:10:44.628 INFO:tasks.workunit.client.0.smithi136.stderr: test: PUT succeeds during reshard 2024-04-22T20:10:44.629 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket myver 2024-04-22T20:10:44.657 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:44.657 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:44.691 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.692+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.693+0000 7fa976333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.693+0000 7fa976333800 20 realm 2024-04-22T20:10:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.693+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.694+0000 7fa976333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.694+0000 7fa976333800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.694+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.694+0000 7fa976333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:44.693 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.694+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.695+0000 7fa976333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:44.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.695+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:44.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.695+0000 7fa976333800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:44.694 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.695+0000 7fa976333800 20 searching for the correct realm 2024-04-22T20:10:44.696 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.697+0000 7fa9727d2640 0 -- 172.21.15.136:0/886669183 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fa96c040c00 msgr2=0x7fa96c036a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:44.902 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:44.903+0000 7fa9697d0640 0 -- 172.21.15.136:0/886669183 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fa96c04f900 msgr2=0x7fa96c00e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:45.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.109+0000 7fa976333800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:45.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.109+0000 7fa976333800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:45.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.109+0000 7fa976333800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:45.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.109+0000 7fa976333800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:45.110 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.109+0000 7fa976333800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.109+0000 7fa976333800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.109+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 20 zone default found 2024-04-22T20:10:45.111 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 4 Realm: () 2024-04-22T20:10:45.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:45.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:45.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:45.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 20 zonegroup default 2024-04-22T20:10:45.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.110+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.111+0000 7fa976333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.112 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.111+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.111+0000 7fa976333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.111+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.111+0000 7fa976333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.111+0000 7fa976333800 20 started sync module instance, tier type = 2024-04-22T20:10:45.113 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.111+0000 7fa976333800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:45.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.121+0000 7fa976333800 20 add_watcher() i=0 2024-04-22T20:10:45.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.121+0000 7fa976333800 20 add_watcher() i=1 2024-04-22T20:10:45.120 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.121+0000 7fa976333800 20 add_watcher() i=2 2024-04-22T20:10:45.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.122+0000 7fa976333800 20 add_watcher() i=3 2024-04-22T20:10:45.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.122+0000 7fa976333800 20 add_watcher() i=4 2024-04-22T20:10:45.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.122+0000 7fa976333800 20 add_watcher() i=5 2024-04-22T20:10:45.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.122+0000 7fa976333800 20 add_watcher() i=6 2024-04-22T20:10:45.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.122+0000 7fa976333800 20 add_watcher() i=7 2024-04-22T20:10:45.122 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.122+0000 7fa976333800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:45.123 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.124+0000 7fa8237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.125+0000 7fa976333800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.125+0000 7fa976333800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.125+0000 7fa976333800 5 note: GC not initialized 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.125+0000 7fa8227fc640 20 reqs_thread_entry: start 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.125+0000 7fa976333800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.125+0000 7fa976333800 20 Filter name: none 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.125+0000 7fa793fff640 20 reqs_thread_entry: start 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.125+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.126+0000 7fa976333800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:45.125 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.126+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.126+0000 7fa976333800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:45.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.126+0000 7fa976333800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:45.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.126+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.126 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.126+0000 7fa976333800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:45.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.126+0000 7fa976333800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.127+0000 7fa976333800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:45.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.127+0000 7fa976333800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:45.127 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.128+0000 7fa8237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:45.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.130+0000 7fa976333800 20 remove_watcher() i=0 2024-04-22T20:10:45.129 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.130+0000 7fa976333800 2 removed watcher, disabling cache 2024-04-22T20:10:45.130 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.131+0000 7fa976333800 20 remove_watcher() i=1 2024-04-22T20:10:45.131 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.132+0000 7fa976333800 20 remove_watcher() i=2 2024-04-22T20:10:45.132 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.133+0000 7fa976333800 20 remove_watcher() i=3 2024-04-22T20:10:45.133 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.135+0000 7fa976333800 20 remove_watcher() i=4 2024-04-22T20:10:45.134 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.135+0000 7fa976333800 20 remove_watcher() i=5 2024-04-22T20:10:45.135 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.136+0000 7fa976333800 20 remove_watcher() i=6 2024-04-22T20:10:45.136 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.137+0000 7fa976333800 20 remove_watcher() i=7 2024-04-22T20:10:45.146 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:45.146 INFO:tasks.workunit.client.0.smithi136.stderr:BUCKET_STATS: 2024-04-22T20:10:45.146 INFO:tasks.workunit.client.0.smithi136.stderr:bucket: myver id: 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 num_objects: 0 size_kb: 0 num_shards: 2 2024-04-22T20:10:45.146 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:45.147 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin --inject-delay-at=do_reshard --inject-delay-ms=5000 bucket reshard --bucket myver --num-shards 3 2024-04-22T20:10:45.179 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:45.180 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:45.213 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.214+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.215+0000 7f96e3411800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.215+0000 7f96e3411800 20 realm 2024-04-22T20:10:45.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.215+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.216+0000 7f96e3411800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.216+0000 7f96e3411800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:45.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.216+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.216+0000 7f96e3411800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.216+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.217+0000 7f96e3411800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:45.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.217+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.217+0000 7f96e3411800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:45.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.217+0000 7f96e3411800 20 searching for the correct realm 2024-04-22T20:10:45.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.219+0000 7f96df26f640 0 -- 172.21.15.136:0/1718427795 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f96d8097040 msgr2=0x7f96d80179c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:45.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.425+0000 7f96d626d640 0 -- 172.21.15.136:0/1718427795 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f96d8084c40 msgr2=0x7f96d8083930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:45.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.631+0000 7f96e3411800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:45.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.631+0000 7f96e3411800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:45.631 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.631+0000 7f96e3411800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.631+0000 7f96e3411800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.631+0000 7f96e3411800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.631+0000 7f96e3411800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.631+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:45.632 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:45.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 20 zone default found 2024-04-22T20:10:45.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 4 Realm: () 2024-04-22T20:10:45.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:45.633 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.632+0000 7f96e3411800 20 zonegroup default 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.633+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.633+0000 7f96e3411800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.633+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.633+0000 7f96e3411800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.633+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.633+0000 7f96e3411800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.633+0000 7f96e3411800 20 started sync module instance, tier type = 2024-04-22T20:10:45.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.633+0000 7f96e3411800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:45.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.644+0000 7f96e3411800 20 add_watcher() i=0 2024-04-22T20:10:45.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.644+0000 7f96e3411800 20 add_watcher() i=1 2024-04-22T20:10:45.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.644+0000 7f96e3411800 20 add_watcher() i=2 2024-04-22T20:10:45.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.644+0000 7f96e3411800 20 add_watcher() i=3 2024-04-22T20:10:45.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.644+0000 7f96e3411800 20 add_watcher() i=4 2024-04-22T20:10:45.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.644+0000 7f96e3411800 20 add_watcher() i=5 2024-04-22T20:10:45.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.644+0000 7f96e3411800 20 add_watcher() i=6 2024-04-22T20:10:45.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.645+0000 7f96e3411800 20 add_watcher() i=7 2024-04-22T20:10:45.644 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.645+0000 7f96e3411800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:45.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.646+0000 7f95957e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:45.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f96e3411800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:45.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f96e3411800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:45.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f96e3411800 5 note: GC not initialized 2024-04-22T20:10:45.647 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f950ffff640 20 reqs_thread_entry: start 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f96e3411800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f96e3411800 20 Filter name: none 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f950dffb640 20 reqs_thread_entry: start 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f96e3411800 10 cache get: name=default.rgw.meta+root+myver : miss 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.647+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 10 adding default.rgw.meta+root+myver to cache LRU end 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 : miss 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 20 rados->read ofs=0 len=0 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 10 adding default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:45.648 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.648+0000 7f96e3411800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 2024-04-22T20:10:45.651 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.652+0000 7f96e3411800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:45.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.652+0000 7f96e3411800 10 moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:45.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.652+0000 7f96e3411800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:45.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.652+0000 7f96e3411800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18, ofs0, ns] 2024-04-22T20:10:45.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.653+0000 7f95c1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562482 cookie 94093645374896 notifier 7257 bl.length()=813 2024-04-22T20:10:45.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.653+0000 7f95c1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:45.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.653+0000 7f95c1ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:45.652 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.653+0000 7f95c1ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:45.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.655+0000 7f96e3411800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_sources= new_sources= 2024-04-22T20:10:45.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.655+0000 7f96e3411800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential sources added=[] removed=[] 2024-04-22T20:10:45.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.655+0000 7f96e3411800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_dests= new_dests= 2024-04-22T20:10:45.654 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.655+0000 7f96e3411800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential dests added=[] removed=[] 2024-04-22T20:10:45.655 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:45.656+0000 7f96e3411800 -1 Injecting delay=5.000000000s at location=do_reshard 2024-04-22T20:10:46.151 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:46.152 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:50.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.656+0000 7f96e3411800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:50.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.656+0000 7f96e3411800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:50.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.656+0000 7f96e3411800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:50.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.656+0000 7f96e3411800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:50.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.656+0000 7f96e3411800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:50.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.656+0000 7f96e3411800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:50.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.656+0000 7f96e3411800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.1.0 2024-04-22T20:10:50.656 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.656+0000 7f96e3411800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.1.0 2024-04-22T20:10:50.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.657+0000 7f96e3411800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.1.1 2024-04-22T20:10:50.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.657+0000 7f96e3411800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.1.1 2024-04-22T20:10:50.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.658+0000 7f96e3411800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 : hit (requested=0x17, cached=0x17) 2024-04-22T20:10:50.657 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.658+0000 7f96e3411800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 2024-04-22T20:10:50.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.661+0000 7f96e3411800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:50.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.661+0000 7f96e3411800 10 moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:50.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.661+0000 7f96e3411800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:50.660 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.661+0000 7f96e3411800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18, ofs0, ns] 2024-04-22T20:10:50.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.661+0000 7f95c27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562483 cookie 94093645374896 notifier 7257 bl.length()=781 2024-04-22T20:10:50.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.661+0000 7f95c27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:50.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.661+0000 7f95c27fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:50.661 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.661+0000 7f95c27fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:50.662 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.663+0000 7f96e3411800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_sources= new_sources= 2024-04-22T20:10:50.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.663+0000 7f96e3411800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential sources added=[] removed=[] 2024-04-22T20:10:50.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.663+0000 7f96e3411800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_dests= new_dests= 2024-04-22T20:10:50.663 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.663+0000 7f96e3411800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential dests added=[] removed=[] 2024-04-22T20:10:50.664 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.665+0000 7f96e3411800 1 execute INFO: reshard of bucket "myver" completed successfully 2024-04-22T20:10:50.666 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.667+0000 7f95957e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:50.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.671+0000 7f96e3411800 20 remove_watcher() i=0 2024-04-22T20:10:50.670 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.671+0000 7f96e3411800 2 removed watcher, disabling cache 2024-04-22T20:10:50.671 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.672+0000 7f96e3411800 20 remove_watcher() i=1 2024-04-22T20:10:50.672 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.673+0000 7f96e3411800 20 remove_watcher() i=2 2024-04-22T20:10:50.673 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.674+0000 7f96e3411800 20 remove_watcher() i=3 2024-04-22T20:10:50.675 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.676+0000 7f96e3411800 20 remove_watcher() i=4 2024-04-22T20:10:50.676 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.677+0000 7f96e3411800 20 remove_watcher() i=5 2024-04-22T20:10:50.677 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.678+0000 7f96e3411800 20 remove_watcher() i=6 2024-04-22T20:10:50.678 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:50.679+0000 7f96e3411800 20 remove_watcher() i=7 2024-04-22T20:10:51.232 INFO:tasks.workunit.client.0.smithi136.stderr:put object successful 2024-04-22T20:10:51.232 INFO:tasks.workunit.client.0.smithi136.stderr:TEST: check that bucket stats are correct after reshard with unlinked entries 2024-04-22T20:10:51.232 INFO:tasks.workunit.client.0.smithi136.stderr: 2024-04-22T20:10:51.234 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:51.234 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:51.301 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:51.301 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:51.367 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:51.367 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:51.419 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:51.419 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:51.446 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:51.446 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:51.497 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:51.497 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:51.521 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:51.521 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:51.571 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:51.571 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:51.596 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-04-22T20:10:51.989 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-04-22T20:10:53.410 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:53.410 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:53.499 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T20:10:53.501 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket myver --object x 2024-04-22T20:10:53.532 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:53.533 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:53.762 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.763+0000 7f35c887d640 0 --2- 172.21.15.136:0/4240841281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aafe596e70 0x55aafe5ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-04-22T20:10:53.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.771+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.772+0000 7f35cd210800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:53.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.772+0000 7f35cd210800 20 realm 2024-04-22T20:10:53.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.772+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.773+0000 7f35cd210800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:53.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.773+0000 7f35cd210800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:53.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.773+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.773+0000 7f35cd210800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:53.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.773+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.774+0000 7f35cd210800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:53.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.774+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.774+0000 7f35cd210800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:53.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.774+0000 7f35cd210800 20 searching for the correct realm 2024-04-22T20:10:53.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.775+0000 7f35c3fff640 0 -- 172.21.15.136:0/3774183264 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f35bc05b070 msgr2=0x7f35bc05c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:53.980 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.981+0000 7f35c907e640 0 -- 172.21.15.136:0/3774183264 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55aafe608e90 msgr2=0x55aafe60b280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:53.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:53.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:53.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:53.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:53.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:53.987 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:53.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:53.988 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.988+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:53.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:53.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 20 zone default found 2024-04-22T20:10:53.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 4 Realm: () 2024-04-22T20:10:53.989 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 20 zonegroup default 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.989+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.990+0000 7f35cd210800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.990+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.990+0000 7f35cd210800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.990+0000 7f35cd210800 20 started sync module instance, tier type = 2024-04-22T20:10:53.990 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:53.990+0000 7f35cd210800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 20 add_watcher() i=0 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 20 add_watcher() i=1 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 20 add_watcher() i=2 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 20 add_watcher() i=3 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 20 add_watcher() i=4 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 20 add_watcher() i=5 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 20 add_watcher() i=6 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 20 add_watcher() i=7 2024-04-22T20:10:53.999 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.000+0000 7f35cd210800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.003+0000 7f347efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.003+0000 7f35cd210800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.003+0000 7f35cd210800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.003+0000 7f35cd210800 5 note: GC not initialized 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.003+0000 7f347dfe3640 20 reqs_thread_entry: start 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.003+0000 7f35cd210800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.003+0000 7f35cd210800 20 Filter name: none 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.004+0000 7f33ef7fe640 20 reqs_thread_entry: start 2024-04-22T20:10:54.003 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.004+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.004+0000 7f35cd210800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:54.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.004+0000 7f35cd210800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:54.004 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:54.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-04-22T20:10:54.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 20 INFO: main: starting shard=0 2024-04-22T20:10:54.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:54.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:54.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-22T20:10:54.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 20 INFO: main: starting shard=1 2024-04-22T20:10:54.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:54.005 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.005+0000 7f35cd210800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:54.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.006+0000 7f35cd210800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:54.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.006+0000 7f35cd210800 20 INFO: main: starting shard=2 2024-04-22T20:10:54.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.006+0000 7f35cd210800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:54.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.006+0000 7f35cd210800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:54.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.007+0000 7f35cd210800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:54.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.007+0000 7f35cd210800 20 INFO: main: done 2024-04-22T20:10:54.006 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.007+0000 7f347efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:54.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.010+0000 7f35cd210800 20 remove_watcher() i=0 2024-04-22T20:10:54.009 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.010+0000 7f35cd210800 2 removed watcher, disabling cache 2024-04-22T20:10:54.010 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.011+0000 7f35cd210800 20 remove_watcher() i=1 2024-04-22T20:10:54.011 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.012+0000 7f35cd210800 20 remove_watcher() i=2 2024-04-22T20:10:54.012 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.013+0000 7f35cd210800 20 remove_watcher() i=3 2024-04-22T20:10:54.013 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.014+0000 7f35cd210800 20 remove_watcher() i=4 2024-04-22T20:10:54.014 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.015+0000 7f35cd210800 20 remove_watcher() i=5 2024-04-22T20:10:54.015 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.016+0000 7f35cd210800 20 remove_watcher() i=6 2024-04-22T20:10:54.016 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.017+0000 7f35cd210800 20 remove_watcher() i=7 2024-04-22T20:10:54.031 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:54.031 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:54.059 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:54.059 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:54.142 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T20:10:54.144 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket myver --object y 2024-04-22T20:10:54.172 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:54.172 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:54.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.207+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.209+0000 7f71f811f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.209+0000 7f71f811f800 20 realm 2024-04-22T20:10:54.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.209+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.209+0000 7f71f811f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.209+0000 7f71f811f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:54.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.209+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.209+0000 7f71f811f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.208 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.209+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.210+0000 7f71f811f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:54.209 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.210+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.210+0000 7f71f811f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:54.210 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.210+0000 7f71f811f800 20 searching for the correct realm 2024-04-22T20:10:54.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.213+0000 7f71f465a640 0 -- 172.21.15.136:0/1481175558 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f71ec082030 msgr2=0x7f71ec06b8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:54.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.217+0000 7f71ebfff640 0 -- 172.21.15.136:0/1481175558 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f71ec0c1b20 msgr2=0x7f71ec0605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:54.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:54.422 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:54.423 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.423+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:54.424 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 20 zone default found 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 4 Realm: () 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 20 zonegroup default 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.424+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.425+0000 7f71f811f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.425+0000 7f71f811f800 20 started sync module instance, tier type = 2024-04-22T20:10:54.425 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.425+0000 7f71f811f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:54.433 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.434+0000 7f71f811f800 20 add_watcher() i=0 2024-04-22T20:10:54.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.434+0000 7f71f811f800 20 add_watcher() i=1 2024-04-22T20:10:54.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.434+0000 7f71f811f800 20 add_watcher() i=2 2024-04-22T20:10:54.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.434+0000 7f71f811f800 20 add_watcher() i=3 2024-04-22T20:10:54.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.434+0000 7f71f811f800 20 add_watcher() i=4 2024-04-22T20:10:54.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.434+0000 7f71f811f800 20 add_watcher() i=5 2024-04-22T20:10:54.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.434+0000 7f71f811f800 20 add_watcher() i=6 2024-04-22T20:10:54.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.435+0000 7f71f811f800 20 add_watcher() i=7 2024-04-22T20:10:54.434 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.435+0000 7f71f811f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:54.436 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.437+0000 7f70a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:54.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.437+0000 7f71f811f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:54.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.437+0000 7f71f811f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:54.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.437+0000 7f71f811f800 5 note: GC not initialized 2024-04-22T20:10:54.437 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.437+0000 7f70a8fe1640 20 reqs_thread_entry: start 2024-04-22T20:10:54.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:54.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 Filter name: none 2024-04-22T20:10:54.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f70227fc640 20 reqs_thread_entry: start 2024-04-22T20:10:54.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:54.438 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:54.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:54.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-04-22T20:10:54.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 INFO: main: starting shard=0 2024-04-22T20:10:54.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:54.439 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.438+0000 7f71f811f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.439+0000 7f71f811f800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.439+0000 7f71f811f800 20 INFO: main: starting shard=1 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.439+0000 7f71f811f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.439+0000 7f71f811f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.440+0000 7f71f811f800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.440+0000 7f71f811f800 20 INFO: main: starting shard=2 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.440+0000 7f71f811f800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.440+0000 7f71f811f800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.440+0000 7f71f811f800 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-22T20:10:54.440 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.440+0000 7f71f811f800 20 INFO: main: done 2024-04-22T20:10:54.441 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.441+0000 7f70a9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:54.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.443+0000 7f71f811f800 20 remove_watcher() i=0 2024-04-22T20:10:54.442 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.443+0000 7f71f811f800 2 removed watcher, disabling cache 2024-04-22T20:10:54.443 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.444+0000 7f71f811f800 20 remove_watcher() i=1 2024-04-22T20:10:54.444 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.445+0000 7f71f811f800 20 remove_watcher() i=2 2024-04-22T20:10:54.445 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.446+0000 7f71f811f800 20 remove_watcher() i=3 2024-04-22T20:10:54.446 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.447+0000 7f71f811f800 20 remove_watcher() i=4 2024-04-22T20:10:54.447 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.448+0000 7f71f811f800 20 remove_watcher() i=5 2024-04-22T20:10:54.448 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.449+0000 7f71f811f800 20 remove_watcher() i=6 2024-04-22T20:10:54.449 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.450+0000 7f71f811f800 20 remove_watcher() i=7 2024-04-22T20:10:54.460 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:54.460 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:54.483 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:54.483 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:54.569 INFO:tasks.workunit.client.0.smithi136.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-04-22T20:10:54.571 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bi list --bucket myver --object z 2024-04-22T20:10:54.600 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:54.601 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:54.634 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.634+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.636+0000 7f224a810800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.636+0000 7f224a810800 20 realm 2024-04-22T20:10:54.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.636+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.636+0000 7f224a810800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.636+0000 7f224a810800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:54.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.636+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.637+0000 7f224a810800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.636 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.637+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.637+0000 7f224a810800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:54.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.637+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.638+0000 7f224a810800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:54.637 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.638+0000 7f224a810800 20 searching for the correct realm 2024-04-22T20:10:54.640 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.641+0000 7f22465bf640 0 -- 172.21.15.136:0/2813010323 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f224004f060 msgr2=0x7f2240008290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:54.643 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.644+0000 7f223d5bd640 0 -- 172.21.15.136:0/2813010323 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f2240017070 msgr2=0x7f22400088d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.850 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.850+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 20 zone default found 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 4 Realm: () 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 20 zonegroup default 2024-04-22T20:10:54.852 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.851+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.852+0000 7f224a810800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.852+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.852+0000 7f224a810800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.852+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.852+0000 7f224a810800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:54.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.852+0000 7f224a810800 20 started sync module instance, tier type = 2024-04-22T20:10:54.853 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.852+0000 7f224a810800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:54.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.862+0000 7f224a810800 20 add_watcher() i=0 2024-04-22T20:10:54.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.862+0000 7f224a810800 20 add_watcher() i=1 2024-04-22T20:10:54.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.862+0000 7f224a810800 20 add_watcher() i=2 2024-04-22T20:10:54.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.862+0000 7f224a810800 20 add_watcher() i=3 2024-04-22T20:10:54.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.862+0000 7f224a810800 20 add_watcher() i=4 2024-04-22T20:10:54.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.862+0000 7f224a810800 20 add_watcher() i=5 2024-04-22T20:10:54.861 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.862+0000 7f224a810800 20 add_watcher() i=6 2024-04-22T20:10:54.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.863+0000 7f224a810800 20 add_watcher() i=7 2024-04-22T20:10:54.862 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.863+0000 7f224a810800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:54.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.865+0000 7f20f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:54.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.865+0000 7f224a810800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:54.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.865+0000 7f224a810800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:54.864 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.865+0000 7f224a810800 5 note: GC not initialized 2024-04-22T20:10:54.865 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.865+0000 7f20f67fc640 20 reqs_thread_entry: start 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.865+0000 7f224a810800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.865+0000 7f224a810800 20 Filter name: none 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f2067fff640 20 reqs_thread_entry: start 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 20 rados->read ofs=0 len=0 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 20 INFO: main: starting shard=0 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:54.866 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.866+0000 7f224a810800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:54.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.867+0000 7f224a810800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:54.867 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.867+0000 7f224a810800 20 INFO: main: starting shard=1 2024-04-22T20:10:54.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.867+0000 7f224a810800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:54.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.867+0000 7f224a810800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:54.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.867+0000 7f224a810800 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-04-22T20:10:54.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.868+0000 7f224a810800 20 INFO: main: starting shard=2 2024-04-22T20:10:54.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.868+0000 7f224a810800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:54.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.868+0000 7f224a810800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:54.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.868+0000 7f224a810800 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-04-22T20:10:54.868 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.868+0000 7f224a810800 20 INFO: main: done 2024-04-22T20:10:54.869 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.869+0000 7f20f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:54.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.872+0000 7f224a810800 20 remove_watcher() i=0 2024-04-22T20:10:54.871 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.872+0000 7f224a810800 2 removed watcher, disabling cache 2024-04-22T20:10:54.872 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.873+0000 7f224a810800 20 remove_watcher() i=1 2024-04-22T20:10:54.873 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.874+0000 7f224a810800 20 remove_watcher() i=2 2024-04-22T20:10:54.874 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.875+0000 7f224a810800 20 remove_watcher() i=3 2024-04-22T20:10:54.875 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.876+0000 7f224a810800 20 remove_watcher() i=4 2024-04-22T20:10:54.876 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.877+0000 7f224a810800 20 remove_watcher() i=5 2024-04-22T20:10:54.877 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.878+0000 7f224a810800 20 remove_watcher() i=6 2024-04-22T20:10:54.878 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:54.879+0000 7f224a810800 20 remove_watcher() i=7 2024-04-22T20:10:54.890 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:54.890 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:54.913 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-04-22T20:10:55.312 INFO:tasks.workunit.client.0.smithi136.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-04-22T20:10:55.705 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 17 --yes-i-really-mean-it 2024-04-22T20:10:55.738 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:55.738 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:55.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.772+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:55.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.773+0000 7f0e68133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.773+0000 7f0e68133800 20 realm 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.773+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.773+0000 7f0e68133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.773+0000 7f0e68133800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.773+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.773+0000 7f0e68133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.773+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.774+0000 7f0e68133800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:55.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.774+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:55.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.775+0000 7f0e68133800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:55.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.775+0000 7f0e68133800 20 searching for the correct realm 2024-04-22T20:10:55.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.777+0000 7f0e645d2640 0 -- 172.21.15.136:0/3034105214 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f0e5c02d070 msgr2=0x7f0e5c050910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:55.983 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:55.983+0000 7f0e5bfff640 0 -- 172.21.15.136:0/3034105214 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f0e5c043920 msgr2=0x7f0e5c02c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.189+0000 7f0e68133800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.189+0000 7f0e68133800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.189+0000 7f0e68133800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.189+0000 7f0e68133800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.189+0000 7f0e68133800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.189+0000 7f0e68133800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.189+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.190 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 20 zone default found 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 4 Realm: () 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 20 zonegroup default 2024-04-22T20:10:56.192 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.190+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.191+0000 7f0e68133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.191+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.191+0000 7f0e68133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.191+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.192+0000 7f0e68133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.192+0000 7f0e68133800 20 started sync module instance, tier type = 2024-04-22T20:10:56.193 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.192+0000 7f0e68133800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:56.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.201+0000 7f0e68133800 20 add_watcher() i=0 2024-04-22T20:10:56.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.201+0000 7f0e68133800 20 add_watcher() i=1 2024-04-22T20:10:56.201 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.201+0000 7f0e68133800 20 add_watcher() i=2 2024-04-22T20:10:56.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.201+0000 7f0e68133800 20 add_watcher() i=3 2024-04-22T20:10:56.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.201+0000 7f0e68133800 20 add_watcher() i=4 2024-04-22T20:10:56.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.201+0000 7f0e68133800 20 add_watcher() i=5 2024-04-22T20:10:56.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.201+0000 7f0e68133800 20 add_watcher() i=6 2024-04-22T20:10:56.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.202+0000 7f0e68133800 20 add_watcher() i=7 2024-04-22T20:10:56.202 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.202+0000 7f0e68133800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:56.204 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.204+0000 7f0d19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.204+0000 7f0e68133800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.204+0000 7f0e68133800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.204+0000 7f0e68133800 5 note: GC not initialized 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.204+0000 7f0d18fe1640 20 reqs_thread_entry: start 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.204+0000 7f0e68133800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 20 Filter name: none 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0c927fc640 20 reqs_thread_entry: start 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 cache get: name=default.rgw.meta+root+myver : miss 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 adding default.rgw.meta+root+myver to cache LRU end 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 : miss 2024-04-22T20:10:56.206 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:56.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:56.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 adding default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:56.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:56.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:56.207 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.205+0000 7f0e68133800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 2024-04-22T20:10:56.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.212+0000 7f0e68133800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:56.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.212+0000 7f0e68133800 10 moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:56.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.212+0000 7f0e68133800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:56.212 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.212+0000 7f0e68133800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18, ofs0, ns] 2024-04-22T20:10:56.213 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.212+0000 7f0d42ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562484 cookie 94585438429824 notifier 7347 bl.length()=813 2024-04-22T20:10:56.213 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.212+0000 7f0d42ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:56.213 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.212+0000 7f0d42ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:56.213 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.212+0000 7f0d42ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:56.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.214+0000 7f0e68133800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_sources= new_sources= 2024-04-22T20:10:56.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.214+0000 7f0e68133800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential sources added=[] removed=[] 2024-04-22T20:10:56.214 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.214+0000 7f0e68133800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_dests= new_dests= 2024-04-22T20:10:56.215 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.214+0000 7f0e68133800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential dests added=[] removed=[] 2024-04-22T20:10:56.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.0 2024-04-22T20:10:56.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.0 2024-04-22T20:10:56.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.1 2024-04-22T20:10:56.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.1 2024-04-22T20:10:56.216 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.2 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.2 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.3 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.3 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.4 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.4 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.5 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.5 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.6 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.6 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.7 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.7 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.8 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.8 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.217 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.10 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.10 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.11 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.11 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.12 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.12 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.13 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.13 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.14 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.14 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.15 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.15 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.16 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.16 2024-04-22T20:10:56.218 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:56.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.0 2024-04-22T20:10:56.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:56.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.216+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.1 2024-04-22T20:10:56.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.217+0000 7f0e68133800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:56.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.217+0000 7f0e68133800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.2.2 2024-04-22T20:10:56.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.219+0000 7f0e68133800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 : hit (requested=0x17, cached=0x17) 2024-04-22T20:10:56.219 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.219+0000 7f0e68133800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 2024-04-22T20:10:56.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.220+0000 7f0e68133800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:56.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.220+0000 7f0e68133800 10 moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:56.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.220+0000 7f0e68133800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:56.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.220+0000 7f0e68133800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18, ofs0, ns] 2024-04-22T20:10:56.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.221+0000 7f0d427fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562485 cookie 94585438429824 notifier 7347 bl.length()=781 2024-04-22T20:10:56.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.221+0000 7f0d427fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:56.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.221+0000 7f0d427fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:56.221 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.221+0000 7f0d427fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:56.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.222+0000 7f0e68133800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_sources= new_sources= 2024-04-22T20:10:56.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.222+0000 7f0e68133800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential sources added=[] removed=[] 2024-04-22T20:10:56.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.222+0000 7f0e68133800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): orig_dests= new_dests= 2024-04-22T20:10:56.222 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.222+0000 7f0e68133800 20 handle_bi_update(): bucket=:myver[19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18]): potential dests added=[] removed=[] 2024-04-22T20:10:56.223 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.223+0000 7f0e68133800 1 execute INFO: reshard of bucket "myver" completed successfully 2024-04-22T20:10:56.225 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.225+0000 7f0d19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:56.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.227+0000 7f0e68133800 20 remove_watcher() i=0 2024-04-22T20:10:56.227 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.227+0000 7f0e68133800 2 removed watcher, disabling cache 2024-04-22T20:10:56.228 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.228+0000 7f0e68133800 20 remove_watcher() i=1 2024-04-22T20:10:56.229 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.229+0000 7f0e68133800 20 remove_watcher() i=2 2024-04-22T20:10:56.230 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.230+0000 7f0e68133800 20 remove_watcher() i=3 2024-04-22T20:10:56.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.231+0000 7f0e68133800 20 remove_watcher() i=4 2024-04-22T20:10:56.232 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.232+0000 7f0e68133800 20 remove_watcher() i=5 2024-04-22T20:10:56.233 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.233+0000 7f0e68133800 20 remove_watcher() i=6 2024-04-22T20:10:56.234 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.234+0000 7f0e68133800 20 remove_watcher() i=7 2024-04-22T20:10:56.245 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket check unlinked --bucket myver --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-04-22T20:10:56.279 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:56.279 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:56.313 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.313+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.314 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.314+0000 7f5dd9a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.314+0000 7f5dd9a62800 20 realm 2024-04-22T20:10:56.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.314+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.314+0000 7f5dd9a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.314+0000 7f5dd9a62800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:56.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.314+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.314+0000 7f5dd9a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.315 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.314+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.315+0000 7f5dd9a62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:56.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.315+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.316+0000 7f5dd9a62800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:56.316 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.316+0000 7f5dd9a62800 20 searching for the correct realm 2024-04-22T20:10:56.318 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.318+0000 7f5dd7d62640 0 -- 172.21.15.136:0/3444086354 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f5dd0020070 msgr2=0x7f5dd002d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:56.525 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.524+0000 7f5dd6d60640 0 -- 172.21.15.136:0/3444086354 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f5dd0055400 msgr2=0x7f5dd0055840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.730+0000 7f5dd9a62800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.730+0000 7f5dd9a62800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.730+0000 7f5dd9a62800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.730+0000 7f5dd9a62800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.730+0000 7f5dd9a62800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.730+0000 7f5dd9a62800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.730+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.731 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.732 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 20 zone default found 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 4 Realm: () 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 20 zonegroup default 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.731+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.732+0000 7f5dd9a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.732+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.732+0000 7f5dd9a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.732+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.732+0000 7f5dd9a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.732+0000 7f5dd9a62800 20 started sync module instance, tier type = 2024-04-22T20:10:56.733 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.732+0000 7f5dd9a62800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:56.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.742+0000 7f5dd9a62800 20 add_watcher() i=0 2024-04-22T20:10:56.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.742+0000 7f5dd9a62800 20 add_watcher() i=1 2024-04-22T20:10:56.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.742+0000 7f5dd9a62800 20 add_watcher() i=2 2024-04-22T20:10:56.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.742+0000 7f5dd9a62800 20 add_watcher() i=3 2024-04-22T20:10:56.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.742+0000 7f5dd9a62800 20 add_watcher() i=4 2024-04-22T20:10:56.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.742+0000 7f5dd9a62800 20 add_watcher() i=5 2024-04-22T20:10:56.742 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.742+0000 7f5dd9a62800 20 add_watcher() i=6 2024-04-22T20:10:56.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.743+0000 7f5dd9a62800 20 add_watcher() i=7 2024-04-22T20:10:56.743 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.743+0000 7f5dd9a62800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:56.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.744+0000 7f5c8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:56.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5dd9a62800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:56.745 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5dd9a62800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:56.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5dd9a62800 5 note: GC not initialized 2024-04-22T20:10:56.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5c8cfe1640 20 reqs_thread_entry: start 2024-04-22T20:10:56.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5dd9a62800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:56.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5dd9a62800 20 Filter name: none 2024-04-22T20:10:56.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5c067fc640 20 reqs_thread_entry: start 2024-04-22T20:10:56.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5dd9a62800 10 cache get: name=default.rgw.meta+root+myver : miss 2024-04-22T20:10:56.746 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.745+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:56.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-04-22T20:10:56.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 adding default.rgw.meta+root+myver to cache LRU end 2024-04-22T20:10:56.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 : miss 2024-04-22T20:10:56.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 20 rados->read ofs=0 len=0 2024-04-22T20:10:56.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:56.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 info.flags=0x17 2024-04-22T20:10:56.747 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 adding default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 to cache LRU end 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.0 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.746+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.0 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.747+0000 7f5dd9a62800 1 NOTICE: finished shard 0 (0 entries removed) 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.747+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.1 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.747+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.1 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.747+0000 7f5dd9a62800 1 NOTICE: finished shard 1 (0 entries removed) 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.747+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.2 2024-04-22T20:10:56.748 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.747+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.2 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.748+0000 7f5dd9a62800 1 NOTICE: finished shard 2 (0 entries removed) 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.748+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.3 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.748+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.3 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.748+0000 7f5dd9a62800 1 NOTICE: finished shard 3 (0 entries removed) 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.748+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.4 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.748+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.4 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.749+0000 7f5dd9a62800 1 NOTICE: finished shard 4 (0 entries removed) 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.749+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.5 2024-04-22T20:10:56.749 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.749+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.5 2024-04-22T20:10:56.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.750+0000 7f5dd9a62800 1 NOTICE: finished shard 5 (0 entries removed) 2024-04-22T20:10:56.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.750+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.6 2024-04-22T20:10:56.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.750+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.6 2024-04-22T20:10:56.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.750+0000 7f5dd9a62800 1 NOTICE: finished shard 6 (0 entries removed) 2024-04-22T20:10:56.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.750+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.7 2024-04-22T20:10:56.750 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.750+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.7 2024-04-22T20:10:56.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.751+0000 7f5dd9a62800 1 NOTICE: finished shard 7 (0 entries removed) 2024-04-22T20:10:56.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.751+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.8 2024-04-22T20:10:56.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.751+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.8 2024-04-22T20:10:56.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.751+0000 7f5dd9a62800 1 NOTICE: finished shard 8 (0 entries removed) 2024-04-22T20:10:56.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.751+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.751 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.751+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.752+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.10 2024-04-22T20:10:56.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.752+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.10 2024-04-22T20:10:56.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.752+0000 7f5dd9a62800 1 NOTICE: finished shard 10 (0 entries removed) 2024-04-22T20:10:56.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.752+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.11 2024-04-22T20:10:56.752 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.752+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.11 2024-04-22T20:10:56.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.753+0000 7f5dd9a62800 1 NOTICE: finished shard 11 (0 entries removed) 2024-04-22T20:10:56.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.753+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.12 2024-04-22T20:10:56.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.753+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.12 2024-04-22T20:10:56.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.753+0000 7f5dd9a62800 1 NOTICE: finished shard 12 (0 entries removed) 2024-04-22T20:10:56.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.753+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.13 2024-04-22T20:10:56.753 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.753+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.13 2024-04-22T20:10:56.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 1 NOTICE: finished shard 13 (0 entries removed) 2024-04-22T20:10:56.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.14 2024-04-22T20:10:56.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.14 2024-04-22T20:10:56.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 1 NOTICE: finished shard 14 (0 entries removed) 2024-04-22T20:10:56.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.15 2024-04-22T20:10:56.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.15 2024-04-22T20:10:56.754 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 1 NOTICE: finished shard 15 (0 entries removed) 2024-04-22T20:10:56.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 10 bucket_obj is .dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.16 2024-04-22T20:10:56.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.754+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.16 2024-04-22T20:10:56.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.755+0000 7f5dd9a62800 1 NOTICE: finished shard 16 (0 entries removed) 2024-04-22T20:10:56.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.755+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.755 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.756+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:x state=0x559012e6d208 s->prefetch_data=0 2024-04-22T20:10:56.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.756+0000 7f5dd9a62800 20 get_obj_state: setting s->obj_tag to j0c02vyo2a5w1xqdnun8fh97t14e51ua 2024-04-22T20:10:56.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.756+0000 7f5dd9a62800 20 get_obj_state_impl: setting s->olh_tag to t3nf2g61ctdsn9019wqt3umkf4k99cw2 2024-04-22T20:10:56.756 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.756+0000 7f5dd9a62800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=t3nf2g61ctdsn9019wqt3umkf4k99cw2 2024-04-22T20:10:56.758 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.757+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.759+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.759 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.759+0000 7f5dd9a62800 20 olh_log_entry: epoch=2 op=2 key=x[] 2024-04-22T20:10:56.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.759+0000 7f5dd9a62800 20 olh_log_entry: epoch=2 op=3 key=x[WstFTN46Ab0i67f7H3D5OyGfb9y.P-v] 2024-04-22T20:10:56.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.759+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:_:WstFTN46Ab0i67f7H3D5OyGfb9y.P-v_x state=0x559012eb0a48 s->prefetch_data=0 2024-04-22T20:10:56.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.760+0000 7f5dd9a62800 10 manifest: total_size = 22 2024-04-22T20:10:56.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.760+0000 7f5dd9a62800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11061729397265966124 2024-04-22T20:10:56.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.760+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:_:WstFTN46Ab0i67f7H3D5OyGfb9y.P-v_x state=0x559012eb0a48 s->prefetch_data=0 2024-04-22T20:10:56.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.760+0000 7f5dd9a62800 20 prepare_atomic_modification: state is not atomic. state=0x559012eb0a48 2024-04-22T20:10:56.760 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.760+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.763+0000 7f5dd9a62800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:10:56.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.763+0000 7f5dd9a62800 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-04-22T20:10:56.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.763+0000 7f5dd9a62800 0 deleting objects inline since gc isn't initialized 2024-04-22T20:10:56.763 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.763+0000 7f5dd9a62800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18__shadow:WstFTN46Ab0i67f7H3D5OyGfb9y.P-v_.yX-CZg_5imjw3Wf79ZwxFPTLJWbIY1b_0 2024-04-22T20:10:56.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.764+0000 7f5cbaffd640 20 handle_completion(): completion ok for obj=x[WstFTN46Ab0i67f7H3D5OyGfb9y.P-v] 2024-04-22T20:10:56.764 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.764+0000 7f5dd9a62800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T20:10:56.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.766+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:x state=0x559012e6d208 s->prefetch_data=0 2024-04-22T20:10:56.766 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.766+0000 7f5dd9a62800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=t3nf2g61ctdsn9019wqt3umkf4k99cw2 2024-04-22T20:10:56.769 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.769+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.770+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.770 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.770+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.770+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:y state=0x559012e6b2f8 s->prefetch_data=0 2024-04-22T20:10:56.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.771+0000 7f5dd9a62800 20 get_obj_state: setting s->obj_tag to mvcb1qtzzs0450sol2whue3lt33845qn 2024-04-22T20:10:56.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.771+0000 7f5dd9a62800 20 get_obj_state_impl: setting s->olh_tag to 1zzn51jokl2jj1kdw4f1rb1gm0oncko1 2024-04-22T20:10:56.771 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.771+0000 7f5dd9a62800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=1zzn51jokl2jj1kdw4f1rb1gm0oncko1 2024-04-22T20:10:56.772 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.772+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.773 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 20 olh_log_entry: epoch=2 op=2 key=y[] 2024-04-22T20:10:56.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 20 olh_log_entry: epoch=2 op=3 key=y[xptPQv4OGGpglR0OhVPSADONjCPM-WW] 2024-04-22T20:10:56.774 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:_:xptPQv4OGGpglR0OhVPSADONjCPM-WW_y state=0x559012eb80e8 s->prefetch_data=0 2024-04-22T20:10:56.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 10 manifest: total_size = 9 2024-04-22T20:10:56.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14640602103348015379 2024-04-22T20:10:56.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:_:xptPQv4OGGpglR0OhVPSADONjCPM-WW_y state=0x559012eb80e8 s->prefetch_data=0 2024-04-22T20:10:56.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 20 prepare_atomic_modification: state is not atomic. state=0x559012eb80e8 2024-04-22T20:10:56.775 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.774+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.777+0000 7f5dd9a62800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:10:56.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.777+0000 7f5dd9a62800 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-04-22T20:10:56.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.777+0000 7f5dd9a62800 0 deleting objects inline since gc isn't initialized 2024-04-22T20:10:56.777 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.777+0000 7f5dd9a62800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18__shadow:xptPQv4OGGpglR0OhVPSADONjCPM-WW_.026BZ6T4wgz1DD1Hs9NzKA9aOKKrlmR_0 2024-04-22T20:10:56.778 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.778+0000 7f5cba7fc640 20 handle_completion(): completion ok for obj=y[xptPQv4OGGpglR0OhVPSADONjCPM-WW] 2024-04-22T20:10:56.779 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.778+0000 7f5dd9a62800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T20:10:56.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.780+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:y state=0x559012e6b2f8 s->prefetch_data=0 2024-04-22T20:10:56.780 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.780+0000 7f5dd9a62800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=1zzn51jokl2jj1kdw4f1rb1gm0oncko1 2024-04-22T20:10:56.782 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.782+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.783 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.783+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.784+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.784 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.784+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:z state=0x559012e6b2f8 s->prefetch_data=0 2024-04-22T20:10:56.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.785+0000 7f5dd9a62800 20 get_obj_state: setting s->obj_tag to xaz1hb3kfxbwe662h80joay37vxyxuko 2024-04-22T20:10:56.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.785+0000 7f5dd9a62800 20 get_obj_state_impl: setting s->olh_tag to oq28zusdns4zfq5vew4t82sw58bch53j 2024-04-22T20:10:56.785 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.785+0000 7f5dd9a62800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=oq28zusdns4zfq5vew4t82sw58bch53j 2024-04-22T20:10:56.786 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.786+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.787 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.787+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.787+0000 7f5dd9a62800 20 olh_log_entry: epoch=2 op=2 key=z[] 2024-04-22T20:10:56.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.787+0000 7f5dd9a62800 20 olh_log_entry: epoch=2 op=3 key=z[DtHInKSHcqwxo6Hx4NHDS-K5wI7C0dD] 2024-04-22T20:10:56.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.787+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:_:DtHInKSHcqwxo6Hx4NHDS-K5wI7C0dD_z state=0x559012eb0a48 s->prefetch_data=0 2024-04-22T20:10:56.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.788+0000 7f5dd9a62800 10 manifest: total_size = 22 2024-04-22T20:10:56.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.788+0000 7f5dd9a62800 20 get_obj_state: setting s->obj_tag to 19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8510580854808000381 2024-04-22T20:10:56.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.788+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:_:DtHInKSHcqwxo6Hx4NHDS-K5wI7C0dD_z state=0x559012eb0a48 s->prefetch_data=0 2024-04-22T20:10:56.788 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.788+0000 7f5dd9a62800 20 prepare_atomic_modification: state is not atomic. state=0x559012eb0a48 2024-04-22T20:10:56.789 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.788+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.791+0000 7f5dd9a62800 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-04-22T20:10:56.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.791+0000 7f5dd9a62800 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-04-22T20:10:56.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.791+0000 7f5dd9a62800 0 deleting objects inline since gc isn't initialized 2024-04-22T20:10:56.791 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.791+0000 7f5dd9a62800 5 delete_objs_inline: removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18__shadow:DtHInKSHcqwxo6Hx4NHDS-K5wI7C0dD_.Trk5M8imXibJmUc5ssmK8H14Q25kXM0_0 2024-04-22T20:10:56.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.792+0000 7f5dd9a62800 5 delete_objs_inline: refcount put returned error -2 2024-04-22T20:10:56.792 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.792+0000 7f5cbaffd640 20 handle_completion(): completion ok for obj=z[DtHInKSHcqwxo6Hx4NHDS-K5wI7C0dD] 2024-04-22T20:10:56.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.793+0000 7f5dd9a62800 20 get_obj_state: octx=0x559012ba1570 obj=myver:z state=0x559012e6b2f8 s->prefetch_data=0 2024-04-22T20:10:56.793 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.793+0000 7f5dd9a62800 20 bucket_index_guard_olh_op(): olh_state.olh_tag=oq28zusdns4zfq5vew4t82sw58bch53j 2024-04-22T20:10:56.796 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.796+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.797+0000 7f5dd9a62800 20 bucket index object: :.dir.19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18.3.9 2024-04-22T20:10:56.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.797+0000 7f5dd9a62800 1 NOTICE: finished shard 9 (3 entries removed) 2024-04-22T20:10:56.797 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.797+0000 7f5dd9a62800 1 NOTICE: finished all shards (3 entries removed) 2024-04-22T20:10:56.798 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.798+0000 7f5c8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:56.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.800+0000 7f5dd9a62800 20 remove_watcher() i=0 2024-04-22T20:10:56.800 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.800+0000 7f5dd9a62800 2 removed watcher, disabling cache 2024-04-22T20:10:56.801 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.801+0000 7f5dd9a62800 20 remove_watcher() i=1 2024-04-22T20:10:56.802 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.802+0000 7f5dd9a62800 20 remove_watcher() i=2 2024-04-22T20:10:56.803 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.803+0000 7f5dd9a62800 20 remove_watcher() i=3 2024-04-22T20:10:56.804 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.804+0000 7f5dd9a62800 20 remove_watcher() i=4 2024-04-22T20:10:56.805 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.805+0000 7f5dd9a62800 20 remove_watcher() i=5 2024-04-22T20:10:56.806 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.806+0000 7f5dd9a62800 20 remove_watcher() i=6 2024-04-22T20:10:56.807 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:56.807+0000 7f5dd9a62800 20 remove_watcher() i=7 2024-04-22T20:10:56.818 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:56.818 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:56.860 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:56.860 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:56.965 INFO:tasks.workunit.client.0.smithi136.stderr:radosgw-admin bucket stats --bucket myver 2024-04-22T20:10:56.992 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:10:56.992 INFO:tasks.workunit.client.0.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:10:57.026 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.026+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.027+0000 7f8754533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:57.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.027+0000 7f8754533800 20 realm 2024-04-22T20:10:57.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.027+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.028+0000 7f8754533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:57.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.028+0000 7f8754533800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:10:57.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.028+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.028+0000 7f8754533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:57.028 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.028+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.029+0000 7f8754533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:57.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.029+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.029+0000 7f8754533800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:10:57.029 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.029+0000 7f8754533800 20 searching for the correct realm 2024-04-22T20:10:57.030 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.030+0000 7f87501d1640 0 -- 172.21.15.136:0/1719375743 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5567c4cec3d0 msgr2=0x5567c4cee890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:10:57.033 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.033+0000 7f87501d1640 0 -- 172.21.15.136:0/1719375743 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x5567c4aa9230 msgr2=0x7f874807d420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:10:57.241 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:10:57.242 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.241+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:10:57.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 20 zone default found 2024-04-22T20:10:57.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 4 Realm: () 2024-04-22T20:10:57.243 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 20 zonegroup default 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.242+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.243+0000 7f8754533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.243+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.243+0000 7f8754533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.243+0000 7f8754533800 20 started sync module instance, tier type = 2024-04-22T20:10:57.244 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.243+0000 7f8754533800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:10:57.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.252+0000 7f8754533800 20 add_watcher() i=0 2024-04-22T20:10:57.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.252+0000 7f8754533800 20 add_watcher() i=1 2024-04-22T20:10:57.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.252+0000 7f8754533800 20 add_watcher() i=2 2024-04-22T20:10:57.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.252+0000 7f8754533800 20 add_watcher() i=3 2024-04-22T20:10:57.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.252+0000 7f8754533800 20 add_watcher() i=4 2024-04-22T20:10:57.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.252+0000 7f8754533800 20 add_watcher() i=5 2024-04-22T20:10:57.252 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.252+0000 7f8754533800 20 add_watcher() i=6 2024-04-22T20:10:57.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.253+0000 7f8754533800 20 add_watcher() i=7 2024-04-22T20:10:57.253 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.253+0000 7f8754533800 2 all 8 watchers are set, enabling cache 2024-04-22T20:10:57.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.254+0000 7f8606fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:57.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.254+0000 7f8754533800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:10:57.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.254+0000 7f8754533800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:10:57.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.255+0000 7f8754533800 5 note: GC not initialized 2024-04-22T20:10:57.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.255+0000 7f8605fdb640 20 reqs_thread_entry: start 2024-04-22T20:10:57.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.255+0000 7f8754533800 20 init_complete bucket index max shards: 1 2024-04-22T20:10:57.255 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.255+0000 7f8754533800 20 Filter name: none 2024-04-22T20:10:57.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.255+0000 7f85777fe640 20 reqs_thread_entry: start 2024-04-22T20:10:57.256 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.255+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.255+0000 7f8754533800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.255+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.256+0000 7f8754533800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.256+0000 7f8754533800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.256+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.256+0000 7f8754533800 20 rados_obj.operate() r=0 bl.length=173 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.256+0000 7f8754533800 20 rados->read ofs=0 len=0 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.256+0000 7f8754533800 20 rados_obj.operate() r=0 bl.length=387 2024-04-22T20:10:57.257 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.256+0000 7f8754533800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-22T20:10:57.258 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.258+0000 7f8606fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:10:57.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.260+0000 7f8754533800 20 remove_watcher() i=0 2024-04-22T20:10:57.260 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.260+0000 7f8754533800 2 removed watcher, disabling cache 2024-04-22T20:10:57.261 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.261+0000 7f8754533800 20 remove_watcher() i=1 2024-04-22T20:10:57.262 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.262+0000 7f8754533800 20 remove_watcher() i=2 2024-04-22T20:10:57.263 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.263+0000 7f8754533800 20 remove_watcher() i=3 2024-04-22T20:10:57.264 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.264+0000 7f8754533800 20 remove_watcher() i=4 2024-04-22T20:10:57.265 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.265+0000 7f8754533800 20 remove_watcher() i=5 2024-04-22T20:10:57.266 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.266+0000 7f8754533800 20 remove_watcher() i=6 2024-04-22T20:10:57.267 INFO:tasks.workunit.client.0.smithi136.stderr:2024-04-22T20:10:57.267+0000 7f8754533800 20 remove_watcher() i=7 2024-04-22T20:10:57.277 INFO:tasks.workunit.client.0.smithi136.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-04-22T20:10:57.277 INFO:tasks.workunit.client.0.smithi136.stderr:Deleting bucket a-bucket 2024-04-22T20:10:57.279 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:57.279 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:57.302 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:57.303 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:57.353 INFO:tasks.workunit.client.0.smithi136.stderr:Deleting bucket myver 2024-04-22T20:10:57.354 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:57.354 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:57.427 INFO:tasks.workunit.client.0.smithi136.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib64/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:10:57.428 INFO:tasks.workunit.client.0.smithi136.stderr: warnings.warn( 2024-04-22T20:10:57.588 INFO:tasks.workunit.client.0.smithi136.stderr:Completed resharding tests 2024-04-22T20:10:57.589 INFO:tasks.workunit.client.0.smithi136.stdout:Could not connect to the endpoint URL: "http://localhost:80/" 2024-04-22T20:10:57.589 INFO:tasks.workunit.client.0.smithi136.stdout:Could not connect to the endpoint URL: "http://localhost:8000/" 2024-04-22T20:10:57.589 INFO:tasks.workunit.client.0.smithi136.stdout:connected to https://localhost:443 2024-04-22T20:10:57.662 INFO:tasks.workunit.client.0.smithi136.stderr:+ deactivate 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -n /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin ']' 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ PATH=/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ export PATH 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ unset _OLD_VIRTUAL_PATH 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -n '' ']' 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ hash -r 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' -n '' ']' 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ unset VIRTUAL_ENV 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ '[' '!' '' = nondestructive ']' 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ unset -f deactivate 2024-04-22T20:10:57.663 INFO:tasks.workunit.client.0.smithi136.stderr:+ echo OK. 2024-04-22T20:10:57.664 INFO:tasks.workunit.client.0.smithi136.stdout:OK. 2024-04-22T20:10:57.665 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-22T20:10:57.665 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-22T20:10:57.741 INFO:tasks.workunit:Stopping ['rgw/run-reshard.sh'] on client.0... 2024-04-22T20:10:57.741 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-22T20:10:58.129 DEBUG:teuthology.parallel:result is None 2024-04-22T20:10:58.129 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:10:58.164 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:10:58.164 DEBUG:teuthology.orchestra.run.smithi136:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-04-22T20:10:58.231 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-04-22T20:10:58.231 INFO:teuthology.run_tasks:Running task s3tests-java... 2024-04-22T20:10:58.248 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2024-04-22T20:10:58.249 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2024-04-22T20:10:58.249 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2024-04-22T20:10:58.249 DEBUG:teuthology.orchestra.run.smithi136:> git clone -b ceph-master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2024-04-22T20:10:58.299 INFO:teuthology.orchestra.run.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2024-04-22T20:11:13.843 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2024-04-22T20:11:13.844 DEBUG:teuthology.orchestra.run.smithi136:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2024-04-22T20:11:15.299 INFO:teuthology.orchestra.run.smithi136.stderr:/home/ubuntu/cephtest/s3-tests-java/bootstrap.sh: missing required RPM packages. Installing via sudo. 2024-04-22T20:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout:Last metadata expiration check: 0:22:50 ago on Mon 22 Apr 2024 07:48:25 PM UTC. 2024-04-22T20:11:15.940 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-22T20:11:15.941 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T20:11:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-22T20:11:15.941 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T20:11:15.941 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-22T20:11:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: java-1.8.0-openjdk x86_64 1:1.8.0.362.b09-4.el9 appstream 426 k 2024-04-22T20:11:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: java-1.8.0-openjdk-devel x86_64 1:1.8.0.362.b09-4.el9 appstream 9.3 M 2024-04-22T20:11:15.941 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-22T20:11:15.941 INFO:teuthology.orchestra.run.smithi136.stdout: java-1.8.0-openjdk-headless x86_64 1:1.8.0.362.b09-4.el9 appstream 32 M 2024-04-22T20:11:15.942 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-04-22T20:11:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: gtk2 x86_64 2.24.33-8.el9 appstream 3.5 M 2024-04-22T20:11:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: libcanberra-gtk2 x86_64 0.30-27.el9 appstream 26 k 2024-04-22T20:11:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:15.942 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-22T20:11:15.942 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-04-22T20:11:15.942 INFO:teuthology.orchestra.run.smithi136.stdout:Install 5 Packages 2024-04-22T20:11:15.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:15.943 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 46 M 2024-04-22T20:11:15.943 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 166 M 2024-04-22T20:11:15.943 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-22T20:11:16.429 INFO:teuthology.orchestra.run.smithi136.stdout:(1/5): java-1.8.0-openjdk-1.8.0.362.b09-4.el9.x 1.2 MB/s | 426 kB 00:00 2024-04-22T20:11:16.487 INFO:teuthology.orchestra.run.smithi136.stdout:(2/5): gtk2-2.24.33-8.el9.x86_64.rpm 8.4 MB/s | 3.5 MB 00:00 2024-04-22T20:11:16.529 INFO:teuthology.orchestra.run.smithi136.stdout:(3/5): libcanberra-gtk2-0.30-27.el9.x86_64.rpm 622 kB/s | 26 kB 00:00 2024-04-22T20:11:16.704 INFO:teuthology.orchestra.run.smithi136.stdout:(4/5): java-1.8.0-openjdk-devel-1.8.0.362.b09-4 15 MB/s | 9.3 MB 00:00 2024-04-22T20:11:18.271 INFO:teuthology.orchestra.run.smithi136.stdout:(5/5): java-1.8.0-openjdk-headless-1.8.0.362.b0 18 MB/s | 32 MB 00:01 2024-04-22T20:11:18.272 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-22T20:11:18.272 INFO:teuthology.orchestra.run.smithi136.stdout:Total 20 MB/s | 46 MB 00:02 2024-04-22T20:11:18.697 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-22T20:11:18.716 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-22T20:11:18.716 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-22T20:11:19.189 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-22T20:11:19.189 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-22T20:11:19.709 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-1.8.0-openjdk-headless-1:1.8.0.362.b09-4.el9.x8 1/1 2024-04-22T20:11:19.849 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-22T20:11:20.032 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcanberra-gtk2-0.30-27.el9.x86_64 1/5 2024-04-22T20:11:20.647 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gtk2-2.24.33-8.el9.x86_64 2/5 2024-04-22T20:11:20.723 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : java-1.8.0-openjdk-headless-1:1.8.0.362.b09-4.el9.x8 3/5 2024-04-22T20:11:20.946 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-1.8.0-openjdk-headless-1:1.8.0.362.b09-4.el9.x8 3/5 2024-04-22T20:11:20.974 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : java-1.8.0-openjdk-1:1.8.0.362.b09-4.el9.x86_64 4/5 2024-04-22T20:11:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-1.8.0-openjdk-1:1.8.0.362.b09-4.el9.x86_64 4/5 2024-04-22T20:11:21.245 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : java-1.8.0-openjdk-devel-1:1.8.0.362.b09-4.el9.x86_6 5/5 2024-04-22T20:11:21.252 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-1.8.0-openjdk-devel-1:1.8.0.362.b09-4.el9.x86_6 5/5 2024-04-22T20:11:21.261 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-1.8.0-openjdk-headless-1:1.8.0.362.b09-4.el9.x8 5/5 2024-04-22T20:11:21.305 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-1.8.0-openjdk-1:1.8.0.362.b09-4.el9.x86_64 5/5 2024-04-22T20:11:24.250 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: java-1.8.0-openjdk-devel-1:1.8.0.362.b09-4.el9.x86_6 5/5 2024-04-22T20:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gtk2-2.24.33-8.el9.x86_64 1/5 2024-04-22T20:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : java-1.8.0-openjdk-1:1.8.0.362.b09-4.el9.x86_64 2/5 2024-04-22T20:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : java-1.8.0-openjdk-devel-1:1.8.0.362.b09-4.el9.x86_6 3/5 2024-04-22T20:11:24.251 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : java-1.8.0-openjdk-headless-1:1.8.0.362.b09-4.el9.x8 4/5 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcanberra-gtk2-0.30-27.el9.x86_64 5/5 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: gtk2-2.24.33-8.el9.x86_64 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: java-1.8.0-openjdk-1:1.8.0.362.b09-4.el9.x86_64 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: java-1.8.0-openjdk-devel-1:1.8.0.362.b09-4.el9.x86_64 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: java-1.8.0-openjdk-headless-1:1.8.0.362.b09-4.el9.x86_64 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: libcanberra-gtk2-0.30-27.el9.x86_64 2024-04-22T20:11:24.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:24.670 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-22T20:11:25.137 INFO:teuthology.orchestra.run.smithi136.stdout: Gradle will be installed under /opt/gradle 2024-04-22T20:11:25.234 INFO:teuthology.orchestra.run.smithi136.stderr:--2024-04-22 20:11:25-- https://services.gradle.org/distributions/gradle-6.0.1-bin.zip 2024-04-22T20:11:25.339 INFO:teuthology.orchestra.run.smithi136.stderr:Resolving services.gradle.org (services.gradle.org)... 104.16.73.101, 104.16.72.101, 2606:4700::6810:4865, ... 2024-04-22T20:11:25.356 INFO:teuthology.orchestra.run.smithi136.stderr:Connecting to services.gradle.org (services.gradle.org)|104.16.73.101|:443... connected. 2024-04-22T20:11:25.517 INFO:teuthology.orchestra.run.smithi136.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2024-04-22T20:11:25.517 INFO:teuthology.orchestra.run.smithi136.stderr:Location: https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip [following] 2024-04-22T20:11:25.517 INFO:teuthology.orchestra.run.smithi136.stderr:--2024-04-22 20:11:25-- https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip 2024-04-22T20:11:25.517 INFO:teuthology.orchestra.run.smithi136.stderr:Resolving github.com (github.com)... 140.82.113.3 2024-04-22T20:11:25.550 INFO:teuthology.orchestra.run.smithi136.stderr:Connecting to github.com (github.com)|140.82.113.3|:443... connected. 2024-04-22T20:11:25.724 INFO:teuthology.orchestra.run.smithi136.stderr:HTTP request sent, awaiting response... 302 Found 2024-04-22T20:11:25.724 INFO:teuthology.orchestra.run.smithi136.stderr:Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240422%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240422T201125Z&X-Amz-Expires=300&X-Amz-Signature=041647a72f48f2588a9ad8abf8a6066f0487a05dcb762c0100d06f63ff17e018&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream [following] 2024-04-22T20:11:25.724 INFO:teuthology.orchestra.run.smithi136.stderr:--2024-04-22 20:11:25-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240422%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240422T201125Z&X-Amz-Expires=300&X-Amz-Signature=041647a72f48f2588a9ad8abf8a6066f0487a05dcb762c0100d06f63ff17e018&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream 2024-04-22T20:11:25.724 INFO:teuthology.orchestra.run.smithi136.stderr:Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.111.133, 185.199.110.133, ... 2024-04-22T20:11:25.765 INFO:teuthology.orchestra.run.smithi136.stderr:Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected. 2024-04-22T20:11:25.904 INFO:teuthology.orchestra.run.smithi136.stderr:HTTP request sent, awaiting response... 200 OK 2024-04-22T20:11:25.904 INFO:teuthology.orchestra.run.smithi136.stderr:Length: 95129672 (91M) [application/octet-stream] 2024-04-22T20:11:25.904 INFO:teuthology.orchestra.run.smithi136.stderr:Saving to: ‘gradle-6.0.1-bin.zip’ 2024-04-22T20:11:25.905 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-22T20:11:25.932 INFO:teuthology.orchestra.run.smithi136.stderr: 0K .......... .......... .......... .......... .......... 0% 2.01M 45s 2024-04-22T20:11:25.941 INFO:teuthology.orchestra.run.smithi136.stderr: 50K .......... .......... .......... .......... .......... 0% 3.71M 35s 2024-04-22T20:11:25.952 INFO:teuthology.orchestra.run.smithi136.stderr: 100K .......... .......... .......... .......... .......... 0% 4.55M 30s 2024-04-22T20:11:25.956 INFO:teuthology.orchestra.run.smithi136.stderr: 150K .......... .......... .......... .......... .......... 0% 13.0M 24s 2024-04-22T20:11:25.957 INFO:teuthology.orchestra.run.smithi136.stderr: 200K .......... .......... .......... .......... .......... 0% 56.4M 20s 2024-04-22T20:11:25.963 INFO:teuthology.orchestra.run.smithi136.stderr: 250K .......... .......... .......... .......... .......... 0% 7.65M 18s 2024-04-22T20:11:25.968 INFO:teuthology.orchestra.run.smithi136.stderr: 300K .......... .......... .......... .......... .......... 0% 9.95M 17s 2024-04-22T20:11:25.970 INFO:teuthology.orchestra.run.smithi136.stderr: 350K .......... .......... .......... .......... .......... 0% 27.8M 15s 2024-04-22T20:11:25.971 INFO:teuthology.orchestra.run.smithi136.stderr: 400K .......... .......... .......... .......... .......... 0% 34.9M 14s 2024-04-22T20:11:25.974 INFO:teuthology.orchestra.run.smithi136.stderr: 450K .......... .......... .......... .......... .......... 0% 24.0M 13s 2024-04-22T20:11:25.974 INFO:teuthology.orchestra.run.smithi136.stderr: 500K .......... .......... .......... .......... .......... 0% 45.8M 12s 2024-04-22T20:11:25.975 INFO:teuthology.orchestra.run.smithi136.stderr: 550K .......... .......... .......... .......... .......... 0% 49.5M 11s 2024-04-22T20:11:25.980 INFO:teuthology.orchestra.run.smithi136.stderr: 600K .......... .......... .......... .......... .......... 0% 10.7M 11s 2024-04-22T20:11:25.982 INFO:teuthology.orchestra.run.smithi136.stderr: 650K .......... .......... .......... .......... .......... 0% 23.6M 10s 2024-04-22T20:11:25.984 INFO:teuthology.orchestra.run.smithi136.stderr: 700K .......... .......... .......... .......... .......... 0% 36.5M 10s 2024-04-22T20:11:25.989 INFO:teuthology.orchestra.run.smithi136.stderr: 750K .......... .......... .......... .......... .......... 0% 31.9M 9s 2024-04-22T20:11:25.989 INFO:teuthology.orchestra.run.smithi136.stderr: 800K .......... .......... .......... .......... .......... 0% 56.3M 9s 2024-04-22T20:11:25.989 INFO:teuthology.orchestra.run.smithi136.stderr: 850K .......... .......... .......... .......... .......... 0% 124M 8s 2024-04-22T20:11:25.989 INFO:teuthology.orchestra.run.smithi136.stderr: 900K .......... .......... .......... .......... .......... 1% 43.4M 8s 2024-04-22T20:11:25.989 INFO:teuthology.orchestra.run.smithi136.stderr: 950K .......... .......... .......... .......... .......... 1% 115M 8s 2024-04-22T20:11:25.989 INFO:teuthology.orchestra.run.smithi136.stderr: 1000K .......... .......... .......... .......... .......... 1% 61.6M 7s 2024-04-22T20:11:25.990 INFO:teuthology.orchestra.run.smithi136.stderr: 1050K .......... .......... .......... .......... .......... 1% 119M 7s 2024-04-22T20:11:25.992 INFO:teuthology.orchestra.run.smithi136.stderr: 1100K .......... .......... .......... .......... .......... 1% 39.2M 7s 2024-04-22T20:11:25.992 INFO:teuthology.orchestra.run.smithi136.stderr: 1150K .......... .......... .......... .......... .......... 1% 127M 7s 2024-04-22T20:11:25.996 INFO:teuthology.orchestra.run.smithi136.stderr: 1200K .......... .......... .......... .......... .......... 1% 12.2M 7s 2024-04-22T20:11:25.996 INFO:teuthology.orchestra.run.smithi136.stderr: 1250K .......... .......... .......... .......... .......... 1% 113M 6s 2024-04-22T20:11:25.996 INFO:teuthology.orchestra.run.smithi136.stderr: 1300K .......... .......... .......... .......... .......... 1% 117M 6s 2024-04-22T20:11:27.407 INFO:teuthology.orchestra.run.smithi136.stderr: 1350K .......... .......... .......... .......... .......... 1% 23.8M 6s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1400K .......... .......... .......... .......... .......... 1% 102M 6s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1450K .......... .......... .......... .......... .......... 1% 97.4M 6s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1500K .......... .......... .......... .......... .......... 1% 81.7M 6s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1550K .......... .......... .......... .......... .......... 1% 120M 5s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1600K .......... .......... .......... .......... .......... 1% 60.5M 5s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1650K .......... .......... .......... .......... .......... 1% 115M 5s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1700K .......... .......... .......... .......... .......... 1% 102M 5s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1750K .......... .......... .......... .......... .......... 1% 109M 5s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1800K .......... .......... .......... .......... .......... 1% 129M 5s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1850K .......... .......... .......... .......... .......... 2% 45.8M 5s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1900K .......... .......... .......... .......... .......... 2% 88.5M 5s 2024-04-22T20:11:27.408 INFO:teuthology.orchestra.run.smithi136.stderr: 1950K .......... .......... .......... .......... .......... 2% 111M 5s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2000K .......... .......... .......... .......... .......... 2% 120M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2050K .......... .......... .......... .......... .......... 2% 109M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2100K .......... .......... .......... .......... .......... 2% 110M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2150K .......... .......... .......... .......... .......... 2% 113M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2200K .......... .......... .......... .......... .......... 2% 117M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2250K .......... .......... .......... .......... .......... 2% 114M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2300K .......... .......... .......... .......... .......... 2% 88.9M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2350K .......... .......... .......... .......... .......... 2% 108M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2400K .......... .......... .......... .......... .......... 2% 102M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2450K .......... .......... .......... .......... .......... 2% 86.0M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2500K .......... .......... .......... .......... .......... 2% 31.5M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2550K .......... .......... .......... .......... .......... 2% 281M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2600K .......... .......... .......... .......... .......... 2% 306M 4s 2024-04-22T20:11:27.409 INFO:teuthology.orchestra.run.smithi136.stderr: 2650K .......... .......... .......... .......... .......... 2% 112M 4s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 2700K .......... .......... .......... .......... .......... 2% 92.6M 4s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 2750K .......... .......... .......... .......... .......... 3% 26.9M 4s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 2800K .......... .......... .......... .......... .......... 3% 114M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 2850K .......... .......... .......... .......... .......... 3% 132M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 2900K .......... .......... .......... .......... .......... 3% 116M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 2950K .......... .......... .......... .......... .......... 3% 117M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3000K .......... .......... .......... .......... .......... 3% 111M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3050K .......... .......... .......... .......... .......... 3% 119M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3100K .......... .......... .......... .......... .......... 3% 87.9M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3150K .......... .......... .......... .......... .......... 3% 116M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3200K .......... .......... .......... .......... .......... 3% 115M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3250K .......... .......... .......... .......... .......... 3% 117M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3300K .......... .......... .......... .......... .......... 3% 115M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3350K .......... .......... .......... .......... .......... 3% 107M 3s 2024-04-22T20:11:27.410 INFO:teuthology.orchestra.run.smithi136.stderr: 3400K .......... .......... .......... .......... .......... 3% 120M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3450K .......... .......... .......... .......... .......... 3% 110M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3500K .......... .......... .......... .......... .......... 3% 75.7M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3550K .......... .......... .......... .......... .......... 3% 142M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3600K .......... .......... .......... .......... .......... 3% 121M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3650K .......... .......... .......... .......... .......... 3% 116M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3700K .......... .......... .......... .......... .......... 4% 103M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3750K .......... .......... .......... .......... .......... 4% 105M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3800K .......... .......... .......... .......... .......... 4% 118M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3850K .......... .......... .......... .......... .......... 4% 112M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3900K .......... .......... .......... .......... .......... 4% 94.4M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 3950K .......... .......... .......... .......... .......... 4% 120M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 4000K .......... .......... .......... .......... .......... 4% 107M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 4050K .......... .......... .......... .......... .......... 4% 104M 3s 2024-04-22T20:11:27.411 INFO:teuthology.orchestra.run.smithi136.stderr: 4100K .......... .......... .......... .......... .......... 4% 104M 3s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4150K .......... .......... .......... .......... .......... 4% 135M 3s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4200K .......... .......... .......... .......... .......... 4% 95.2M 3s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4250K .......... .......... .......... .......... .......... 4% 126M 3s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4300K .......... .......... .......... .......... .......... 4% 97.7M 3s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4350K .......... .......... .......... .......... .......... 4% 122M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4400K .......... .......... .......... .......... .......... 4% 120M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4450K .......... .......... .......... .......... .......... 4% 90.9M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4500K .......... .......... .......... .......... .......... 4% 112M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4550K .......... .......... .......... .......... .......... 4% 87.0M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4600K .......... .......... .......... .......... .......... 5% 174M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4650K .......... .......... .......... .......... .......... 5% 116M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4700K .......... .......... .......... .......... .......... 5% 86.9M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4750K .......... .......... .......... .......... .......... 5% 119M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4800K .......... .......... .......... .......... .......... 5% 112M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4850K .......... .......... .......... .......... .......... 5% 119M 2s 2024-04-22T20:11:27.412 INFO:teuthology.orchestra.run.smithi136.stderr: 4900K .......... .......... .......... .......... .......... 5% 112M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 4950K .......... .......... .......... .......... .......... 5% 112M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5000K .......... .......... .......... .......... .......... 5% 117M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5050K .......... .......... .......... .......... .......... 5% 111M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5100K .......... .......... .......... .......... .......... 5% 70.6M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5150K .......... .......... .......... .......... .......... 5% 58.1M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5200K .......... .......... .......... .......... .......... 5% 36.6M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5250K .......... .......... .......... .......... .......... 5% 138M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5300K .......... .......... .......... .......... .......... 5% 57.4M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5350K .......... .......... .......... .......... .......... 5% 51.6M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5400K .......... .......... .......... .......... .......... 5% 114M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5450K .......... .......... .......... .......... .......... 5% 112M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5500K .......... .......... .......... .......... .......... 5% 113M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5550K .......... .......... .......... .......... .......... 6% 25.6M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5600K .......... .......... .......... .......... .......... 6% 84.7M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5650K .......... .......... .......... .......... .......... 6% 115M 2s 2024-04-22T20:11:27.413 INFO:teuthology.orchestra.run.smithi136.stderr: 5700K .......... .......... .......... .......... .......... 6% 35.2M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 5750K .......... .......... .......... .......... .......... 6% 92.9M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 5800K .......... .......... .......... .......... .......... 6% 87.2M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 5850K .......... .......... .......... .......... .......... 6% 112M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 5900K .......... .......... .......... .......... .......... 6% 77.0M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 5950K .......... .......... .......... .......... .......... 6% 172M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6000K .......... .......... .......... .......... .......... 6% 127M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6050K .......... .......... .......... .......... .......... 6% 87.2M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6100K .......... .......... .......... .......... .......... 6% 107M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6150K .......... .......... .......... .......... .......... 6% 71.6M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6200K .......... .......... .......... .......... .......... 6% 118M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6250K .......... .......... .......... .......... .......... 6% 116M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6300K .......... .......... .......... .......... .......... 6% 112M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6350K .......... .......... .......... .......... .......... 6% 116M 2s 2024-04-22T20:11:27.414 INFO:teuthology.orchestra.run.smithi136.stderr: 6400K .......... .......... .......... .......... .......... 6% 115M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6450K .......... .......... .......... .......... .......... 6% 72.5M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6500K .......... .......... .......... .......... .......... 7% 71.2M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6550K .......... .......... .......... .......... .......... 7% 133M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6600K .......... .......... .......... .......... .......... 7% 150M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6650K .......... .......... .......... .......... .......... 7% 122M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6700K .......... .......... .......... .......... .......... 7% 88.9M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6750K .......... .......... .......... .......... .......... 7% 111M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6800K .......... .......... .......... .......... .......... 7% 72.5M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6850K .......... .......... .......... .......... .......... 7% 47.1M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6900K .......... .......... .......... .......... .......... 7% 113M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 6950K .......... .......... .......... .......... .......... 7% 103M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 7000K .......... .......... .......... .......... .......... 7% 128M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 7050K .......... .......... .......... .......... .......... 7% 118M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 7100K .......... .......... .......... .......... .......... 7% 57.5M 2s 2024-04-22T20:11:27.415 INFO:teuthology.orchestra.run.smithi136.stderr: 7150K .......... .......... .......... .......... .......... 7% 104M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7200K .......... .......... .......... .......... .......... 7% 94.5M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7250K .......... .......... .......... .......... .......... 7% 160M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7300K .......... .......... .......... .......... .......... 7% 121M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7350K .......... .......... .......... .......... .......... 7% 123M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7400K .......... .......... .......... .......... .......... 8% 117M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7450K .......... .......... .......... .......... .......... 8% 116M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7500K .......... .......... .......... .......... .......... 8% 77.5M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7550K .......... .......... .......... .......... .......... 8% 75.2M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7600K .......... .......... .......... .......... .......... 8% 116M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7650K .......... .......... .......... .......... .......... 8% 116M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7700K .......... .......... .......... .......... .......... 8% 113M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7750K .......... .......... .......... .......... .......... 8% 69.2M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7800K .......... .......... .......... .......... .......... 8% 114M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7850K .......... .......... .......... .......... .......... 8% 116M 2s 2024-04-22T20:11:27.416 INFO:teuthology.orchestra.run.smithi136.stderr: 7900K .......... .......... .......... .......... .......... 8% 86.2M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 7950K .......... .......... .......... .......... .......... 8% 123M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8000K .......... .......... .......... .......... .......... 8% 65.2M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8050K .......... .......... .......... .......... .......... 8% 115M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8100K .......... .......... .......... .......... .......... 8% 115M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8150K .......... .......... .......... .......... .......... 8% 69.4M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8200K .......... .......... .......... .......... .......... 8% 76.8M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8250K .......... .......... .......... .......... .......... 8% 196M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8300K .......... .......... .......... .......... .......... 8% 91.6M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8350K .......... .......... .......... .......... .......... 9% 96.0M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8400K .......... .......... .......... .......... .......... 9% 98.7M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8450K .......... .......... .......... .......... .......... 9% 143M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8500K .......... .......... .......... .......... .......... 9% 86.5M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8550K .......... .......... .......... .......... .......... 9% 180M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8600K .......... .......... .......... .......... .......... 9% 109M 2s 2024-04-22T20:11:27.417 INFO:teuthology.orchestra.run.smithi136.stderr: 8650K .......... .......... .......... .......... .......... 9% 113M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 8700K .......... .......... .......... .......... .......... 9% 89.3M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 8750K .......... .......... .......... .......... .......... 9% 100M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 8800K .......... .......... .......... .......... .......... 9% 113M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 8850K .......... .......... .......... .......... .......... 9% 133M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 8900K .......... .......... .......... .......... .......... 9% 116M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 8950K .......... .......... .......... .......... .......... 9% 81.1M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 9000K .......... .......... .......... .......... .......... 9% 167M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 9050K .......... .......... .......... .......... .......... 9% 113M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 9100K .......... .......... .......... .......... .......... 9% 77.3M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 9150K .......... .......... .......... .......... .......... 9% 102M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 9200K .......... .......... .......... .......... .......... 9% 95.8M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 9250K .......... .......... .......... .......... .......... 10% 86.5M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 9300K .......... .......... .......... .......... .......... 10% 203M 2s 2024-04-22T20:11:27.418 INFO:teuthology.orchestra.run.smithi136.stderr: 9350K .......... .......... .......... .......... .......... 10% 115M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9400K .......... .......... .......... .......... .......... 10% 102M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9450K .......... .......... .......... .......... .......... 10% 107M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9500K .......... .......... .......... .......... .......... 10% 88.5M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9550K .......... .......... .......... .......... .......... 10% 116M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9600K .......... .......... .......... .......... .......... 10% 84.3M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9650K .......... .......... .......... .......... .......... 10% 64.0M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9700K .......... .......... .......... .......... .......... 10% 170M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9750K .......... .......... .......... .......... .......... 10% 152M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9800K .......... .......... .......... .......... .......... 10% 122M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9850K .......... .......... .......... .......... .......... 10% 113M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9900K .......... .......... .......... .......... .......... 10% 89.5M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 9950K .......... .......... .......... .......... .......... 10% 115M 2s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 10000K .......... .......... .......... .......... .......... 10% 114M 1s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 10050K .......... .......... .......... .......... .......... 10% 103M 1s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 10100K .......... .......... .......... .......... .......... 10% 117M 1s 2024-04-22T20:11:27.419 INFO:teuthology.orchestra.run.smithi136.stderr: 10150K .......... .......... .......... .......... .......... 10% 120M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10200K .......... .......... .......... .......... .......... 11% 103M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10250K .......... .......... .......... .......... .......... 11% 108M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10300K .......... .......... .......... .......... .......... 11% 75.5M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10350K .......... .......... .......... .......... .......... 11% 134M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10400K .......... .......... .......... .......... .......... 11% 123M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10450K .......... .......... .......... .......... .......... 11% 106M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10500K .......... .......... .......... .......... .......... 11% 104M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10550K .......... .......... .......... .......... .......... 11% 104M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10600K .......... .......... .......... .......... .......... 11% 145M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10650K .......... .......... .......... .......... .......... 11% 110M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10700K .......... .......... .......... .......... .......... 11% 78.3M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10750K .......... .......... .......... .......... .......... 11% 127M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10800K .......... .......... .......... .......... .......... 11% 106M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10850K .......... .......... .......... .......... .......... 11% 118M 1s 2024-04-22T20:11:27.420 INFO:teuthology.orchestra.run.smithi136.stderr: 10900K .......... .......... .......... .......... .......... 11% 105M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 10950K .......... .......... .......... .......... .......... 11% 113M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11000K .......... .......... .......... .......... .......... 11% 106M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11050K .......... .......... .......... .......... .......... 11% 78.3M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11100K .......... .......... .......... .......... .......... 12% 152M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11150K .......... .......... .......... .......... .......... 12% 142M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11200K .......... .......... .......... .......... .......... 12% 84.3M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11250K .......... .......... .......... .......... .......... 12% 117M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11300K .......... .......... .......... .......... .......... 12% 115M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11350K .......... .......... .......... .......... .......... 12% 118M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11400K .......... .......... .......... .......... .......... 12% 85.4M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11450K .......... .......... .......... .......... .......... 12% 138M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11500K .......... .......... .......... .......... .......... 12% 87.4M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11550K .......... .......... .......... .......... .......... 12% 93.7M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11600K .......... .......... .......... .......... .......... 12% 112M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11650K .......... .......... .......... .......... .......... 12% 107M 1s 2024-04-22T20:11:27.421 INFO:teuthology.orchestra.run.smithi136.stderr: 11700K .......... .......... .......... .......... .......... 12% 114M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 11750K .......... .......... .......... .......... .......... 12% 112M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 11800K .......... .......... .......... .......... .......... 12% 101M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 11850K .......... .......... .......... .......... .......... 12% 108M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 11900K .......... .......... .......... .......... .......... 12% 110M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 11950K .......... .......... .......... .......... .......... 12% 93.2M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12000K .......... .......... .......... .......... .......... 12% 145M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12050K .......... .......... .......... .......... .......... 13% 118M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12100K .......... .......... .......... .......... .......... 13% 106M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12150K .......... .......... .......... .......... .......... 13% 123M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12200K .......... .......... .......... .......... .......... 13% 110M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12250K .......... .......... .......... .......... .......... 13% 108M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12300K .......... .......... .......... .......... .......... 13% 86.1M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12350K .......... .......... .......... .......... .......... 13% 113M 1s 2024-04-22T20:11:27.422 INFO:teuthology.orchestra.run.smithi136.stderr: 12400K .......... .......... .......... .......... .......... 13% 117M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12450K .......... .......... .......... .......... .......... 13% 110M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12500K .......... .......... .......... .......... .......... 13% 121M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12550K .......... .......... .......... .......... .......... 13% 115M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12600K .......... .......... .......... .......... .......... 13% 109M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12650K .......... .......... .......... .......... .......... 13% 73.1M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12700K .......... .......... .......... .......... .......... 13% 145M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12750K .......... .......... .......... .......... .......... 13% 114M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12800K .......... .......... .......... .......... .......... 13% 83.9M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12850K .......... .......... .......... .......... .......... 13% 150M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12900K .......... .......... .......... .......... .......... 13% 128M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 12950K .......... .......... .......... .......... .......... 13% 112M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 13000K .......... .......... .......... .......... .......... 14% 114M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 13050K .......... .......... .......... .......... .......... 14% 118M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 13100K .......... .......... .......... .......... .......... 14% 87.9M 1s 2024-04-22T20:11:27.423 INFO:teuthology.orchestra.run.smithi136.stderr: 13150K .......... .......... .......... .......... .......... 14% 114M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13200K .......... .......... .......... .......... .......... 14% 112M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13250K .......... .......... .......... .......... .......... 14% 98.4M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13300K .......... .......... .......... .......... .......... 14% 138M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13350K .......... .......... .......... .......... .......... 14% 115M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13400K .......... .......... .......... .......... .......... 14% 114M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13450K .......... .......... .......... .......... .......... 14% 107M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13500K .......... .......... .......... .......... .......... 14% 60.2M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13550K .......... .......... .......... .......... .......... 14% 79.5M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13600K .......... .......... .......... .......... .......... 14% 109M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13650K .......... .......... .......... .......... .......... 14% 123M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13700K .......... .......... .......... .......... .......... 14% 114M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13750K .......... .......... .......... .......... .......... 14% 104M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13800K .......... .......... .......... .......... .......... 14% 120M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13850K .......... .......... .......... .......... .......... 14% 114M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13900K .......... .......... .......... .......... .......... 15% 90.4M 1s 2024-04-22T20:11:27.424 INFO:teuthology.orchestra.run.smithi136.stderr: 13950K .......... .......... .......... .......... .......... 15% 110M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14000K .......... .......... .......... .......... .......... 15% 116M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14050K .......... .......... .......... .......... .......... 15% 116M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14100K .......... .......... .......... .......... .......... 15% 110M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14150K .......... .......... .......... .......... .......... 15% 75.4M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14200K .......... .......... .......... .......... .......... 15% 237M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14250K .......... .......... .......... .......... .......... 15% 119M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14300K .......... .......... .......... .......... .......... 15% 87.7M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14350K .......... .......... .......... .......... .......... 15% 86.5M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14400K .......... .......... .......... .......... .......... 15% 148M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14450K .......... .......... .......... .......... .......... 15% 129M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14500K .......... .......... .......... .......... .......... 15% 112M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14550K .......... .......... .......... .......... .......... 15% 111M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14600K .......... .......... .......... .......... .......... 15% 117M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14650K .......... .......... .......... .......... .......... 15% 112M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14700K .......... .......... .......... .......... .......... 15% 85.6M 1s 2024-04-22T20:11:27.425 INFO:teuthology.orchestra.run.smithi136.stderr: 14750K .......... .......... .......... .......... .......... 15% 117M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 14800K .......... .......... .......... .......... .......... 15% 112M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 14850K .......... .......... .......... .......... .......... 16% 111M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 14900K .......... .......... .......... .......... .......... 16% 117M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 14950K .......... .......... .......... .......... .......... 16% 117M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15000K .......... .......... .......... .......... .......... 16% 80.0M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15050K .......... .......... .......... .......... .......... 16% 89.2M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15100K .......... .......... .......... .......... .......... 16% 129M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15150K .......... .......... .......... .......... .......... 16% 151M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15200K .......... .......... .......... .......... .......... 16% 103M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15250K .......... .......... .......... .......... .......... 16% 106M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15300K .......... .......... .......... .......... .......... 16% 120M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15350K .......... .......... .......... .......... .......... 16% 116M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15400K .......... .......... .......... .......... .......... 16% 92.2M 1s 2024-04-22T20:11:27.426 INFO:teuthology.orchestra.run.smithi136.stderr: 15450K .......... .......... .......... .......... .......... 16% 108M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15500K .......... .......... .......... .......... .......... 16% 115M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15550K .......... .......... .......... .......... .......... 16% 111M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15600K .......... .......... .......... .......... .......... 16% 101M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15650K .......... .......... .......... .......... .......... 16% 91.7M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15700K .......... .......... .......... .......... .......... 16% 117M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15750K .......... .......... .......... .......... .......... 17% 152M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15800K .......... .......... .......... .......... .......... 17% 98.4M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15850K .......... .......... .......... .......... .......... 17% 120M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15900K .......... .......... .......... .......... .......... 17% 91.6M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 15950K .......... .......... .......... .......... .......... 17% 121M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 16000K .......... .......... .......... .......... .......... 17% 104M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 16050K .......... .......... .......... .......... .......... 17% 126M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 16100K .......... .......... .......... .......... .......... 17% 109M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 16150K .......... .......... .......... .......... .......... 17% 121M 1s 2024-04-22T20:11:27.427 INFO:teuthology.orchestra.run.smithi136.stderr: 16200K .......... .......... .......... .......... .......... 17% 120M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16250K .......... .......... .......... .......... .......... 17% 106M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16300K .......... .......... .......... .......... .......... 17% 90.8M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16350K .......... .......... .......... .......... .......... 17% 115M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16400K .......... .......... .......... .......... .......... 17% 112M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16450K .......... .......... .......... .......... .......... 17% 122M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16500K .......... .......... .......... .......... .......... 17% 104M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16550K .......... .......... .......... .......... .......... 17% 110M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16600K .......... .......... .......... .......... .......... 17% 118M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16650K .......... .......... .......... .......... .......... 17% 116M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16700K .......... .......... .......... .......... .......... 18% 76.4M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16750K .......... .......... .......... .......... .......... 18% 113M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16800K .......... .......... .......... .......... .......... 18% 117M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16850K .......... .......... .......... .......... .......... 18% 111M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16900K .......... .......... .......... .......... .......... 18% 112M 1s 2024-04-22T20:11:27.428 INFO:teuthology.orchestra.run.smithi136.stderr: 16950K .......... .......... .......... .......... .......... 18% 123M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17000K .......... .......... .......... .......... .......... 18% 114M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17050K .......... .......... .......... .......... .......... 18% 117M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17100K .......... .......... .......... .......... .......... 18% 87.2M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17150K .......... .......... .......... .......... .......... 18% 111M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17200K .......... .......... .......... .......... .......... 18% 117M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17250K .......... .......... .......... .......... .......... 18% 70.7M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17300K .......... .......... .......... .......... .......... 18% 135M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17350K .......... .......... .......... .......... .......... 18% 119M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17400K .......... .......... .......... .......... .......... 18% 117M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17450K .......... .......... .......... .......... .......... 18% 108M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17500K .......... .......... .......... .......... .......... 18% 87.1M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17550K .......... .......... .......... .......... .......... 18% 103M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17600K .......... .......... .......... .......... .......... 18% 134M 1s 2024-04-22T20:11:27.429 INFO:teuthology.orchestra.run.smithi136.stderr: 17650K .......... .......... .......... .......... .......... 19% 118M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 17700K .......... .......... .......... .......... .......... 19% 105M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 17750K .......... .......... .......... .......... .......... 19% 118M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 17800K .......... .......... .......... .......... .......... 19% 121M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 17850K .......... .......... .......... .......... .......... 19% 113M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 17900K .......... .......... .......... .......... .......... 19% 90.1M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 17950K .......... .......... .......... .......... .......... 19% 107M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18000K .......... .......... .......... .......... .......... 19% 75.1M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18050K .......... .......... .......... .......... .......... 19% 261M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18100K .......... .......... .......... .......... .......... 19% 111M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18150K .......... .......... .......... .......... .......... 19% 92.4M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18200K .......... .......... .......... .......... .......... 19% 147M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18250K .......... .......... .......... .......... .......... 19% 111M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18300K .......... .......... .......... .......... .......... 19% 62.9M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18350K .......... .......... .......... .......... .......... 19% 119M 1s 2024-04-22T20:11:27.430 INFO:teuthology.orchestra.run.smithi136.stderr: 18400K .......... .......... .......... .......... .......... 19% 115M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18450K .......... .......... .......... .......... .......... 19% 110M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18500K .......... .......... .......... .......... .......... 19% 105M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18550K .......... .......... .......... .......... .......... 20% 117M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18600K .......... .......... .......... .......... .......... 20% 114M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18650K .......... .......... .......... .......... .......... 20% 111M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18700K .......... .......... .......... .......... .......... 20% 92.6M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18750K .......... .......... .......... .......... .......... 20% 91.7M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18800K .......... .......... .......... .......... .......... 20% 97.3M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18850K .......... .......... .......... .......... .......... 20% 173M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18900K .......... .......... .......... .......... .......... 20% 109M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 18950K .......... .......... .......... .......... .......... 20% 114M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 19000K .......... .......... .......... .......... .......... 20% 113M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 19050K .......... .......... .......... .......... .......... 20% 112M 1s 2024-04-22T20:11:27.431 INFO:teuthology.orchestra.run.smithi136.stderr: 19100K .......... .......... .......... .......... .......... 20% 89.1M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19150K .......... .......... .......... .......... .......... 20% 117M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19200K .......... .......... .......... .......... .......... 20% 114M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19250K .......... .......... .......... .......... .......... 20% 114M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19300K .......... .......... .......... .......... .......... 20% 117M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19350K .......... .......... .......... .......... .......... 20% 113M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19400K .......... .......... .......... .......... .......... 20% 116M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19450K .......... .......... .......... .......... .......... 20% 98.1M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19500K .......... .......... .......... .......... .......... 21% 88.8M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19550K .......... .......... .......... .......... .......... 21% 120M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19600K .......... .......... .......... .......... .......... 21% 114M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19650K .......... .......... .......... .......... .......... 21% 116M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19700K .......... .......... .......... .......... .......... 21% 109M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19750K .......... .......... .......... .......... .......... 21% 84.6M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19800K .......... .......... .......... .......... .......... 21% 162M 1s 2024-04-22T20:11:27.432 INFO:teuthology.orchestra.run.smithi136.stderr: 19850K .......... .......... .......... .......... .......... 21% 98.9M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 19900K .......... .......... .......... .......... .......... 21% 96.7M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 19950K .......... .......... .......... .......... .......... 21% 116M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20000K .......... .......... .......... .......... .......... 21% 119M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20050K .......... .......... .......... .......... .......... 21% 114M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20100K .......... .......... .......... .......... .......... 21% 112M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20150K .......... .......... .......... .......... .......... 21% 67.1M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20200K .......... .......... .......... .......... .......... 21% 114M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20250K .......... .......... .......... .......... .......... 21% 117M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20300K .......... .......... .......... .......... .......... 21% 74.0M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20350K .......... .......... .......... .......... .......... 21% 155M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20400K .......... .......... .......... .......... .......... 22% 116M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20450K .......... .......... .......... .......... .......... 22% 116M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20500K .......... .......... .......... .......... .......... 22% 107M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20550K .......... .......... .......... .......... .......... 22% 111M 1s 2024-04-22T20:11:27.433 INFO:teuthology.orchestra.run.smithi136.stderr: 20600K .......... .......... .......... .......... .......... 22% 116M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 20650K .......... .......... .......... .......... .......... 22% 117M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 20700K .......... .......... .......... .......... .......... 22% 86.7M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 20750K .......... .......... .......... .......... .......... 22% 116M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 20800K .......... .......... .......... .......... .......... 22% 114M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 20850K .......... .......... .......... .......... .......... 22% 116M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 20900K .......... .......... .......... .......... .......... 22% 113M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 20950K .......... .......... .......... .......... .......... 22% 89.6M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 21000K .......... .......... .......... .......... .......... 22% 118M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 21050K .......... .......... .......... .......... .......... 22% 76.3M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 21100K .......... .......... .......... .......... .......... 22% 131M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 21150K .......... .......... .......... .......... .......... 22% 119M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 21200K .......... .......... .......... .......... .......... 22% 100M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 21250K .......... .......... .......... .......... .......... 22% 119M 1s 2024-04-22T20:11:27.434 INFO:teuthology.orchestra.run.smithi136.stderr: 21300K .......... .......... .......... .......... .......... 22% 111M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21350K .......... .......... .......... .......... .......... 23% 117M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21400K .......... .......... .......... .......... .......... 23% 118M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21450K .......... .......... .......... .......... .......... 23% 105M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21500K .......... .......... .......... .......... .......... 23% 89.2M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21550K .......... .......... .......... .......... .......... 23% 115M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21600K .......... .......... .......... .......... .......... 23% 117M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21650K .......... .......... .......... .......... .......... 23% 96.9M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21700K .......... .......... .......... .......... .......... 23% 130M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21750K .......... .......... .......... .......... .......... 23% 50.4M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21800K .......... .......... .......... .......... .......... 23% 245M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21850K .......... .......... .......... .......... .......... 23% 343M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21900K .......... .......... .......... .......... .......... 23% 88.8M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 21950K .......... .......... .......... .......... .......... 23% 118M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 22000K .......... .......... .......... .......... .......... 23% 101M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 22050K .......... .......... .......... .......... .......... 23% 113M 1s 2024-04-22T20:11:27.435 INFO:teuthology.orchestra.run.smithi136.stderr: 22100K .......... .......... .......... .......... .......... 23% 115M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22150K .......... .......... .......... .......... .......... 23% 116M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22200K .......... .......... .......... .......... .......... 23% 112M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22250K .......... .......... .......... .......... .......... 24% 119M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22300K .......... .......... .......... .......... .......... 24% 80.8M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22350K .......... .......... .......... .......... .......... 24% 119M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22400K .......... .......... .......... .......... .......... 24% 117M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22450K .......... .......... .......... .......... .......... 24% 116M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22500K .......... .......... .......... .......... .......... 24% 105M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22550K .......... .......... .......... .......... .......... 24% 92.9M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22600K .......... .......... .......... .......... .......... 24% 161M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22650K .......... .......... .......... .......... .......... 24% 115M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22700K .......... .......... .......... .......... .......... 24% 87.0M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22750K .......... .......... .......... .......... .......... 24% 94.3M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22800K .......... .......... .......... .......... .......... 24% 122M 1s 2024-04-22T20:11:27.436 INFO:teuthology.orchestra.run.smithi136.stderr: 22850K .......... .......... .......... .......... .......... 24% 120M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 22900K .......... .......... .......... .......... .......... 24% 105M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 22950K .......... .......... .......... .......... .......... 24% 120M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23000K .......... .......... .......... .......... .......... 24% 116M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23050K .......... .......... .......... .......... .......... 24% 115M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23100K .......... .......... .......... .......... .......... 24% 86.9M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23150K .......... .......... .......... .......... .......... 24% 103M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23200K .......... .......... .......... .......... .......... 25% 116M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23250K .......... .......... .......... .......... .......... 25% 108M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23300K .......... .......... .......... .......... .......... 25% 95.0M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23350K .......... .......... .......... .......... .......... 25% 151M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23400K .......... .......... .......... .......... .......... 25% 114M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23450K .......... .......... .......... .......... .......... 25% 118M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23500K .......... .......... .......... .......... .......... 25% 88.1M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23550K .......... .......... .......... .......... .......... 25% 112M 1s 2024-04-22T20:11:27.437 INFO:teuthology.orchestra.run.smithi136.stderr: 23600K .......... .......... .......... .......... .......... 25% 111M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 23650K .......... .......... .......... .......... .......... 25% 118M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 23700K .......... .......... .......... .......... .......... 25% 111M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 23750K .......... .......... .......... .......... .......... 25% 112M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 23800K .......... .......... .......... .......... .......... 25% 114M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 23850K .......... .......... .......... .......... .......... 25% 67.5M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 23900K .......... .......... .......... .......... .......... 25% 188M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 23950K .......... .......... .......... .......... .......... 25% 96.5M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 24000K .......... .......... .......... .......... .......... 25% 143M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 24050K .......... .......... .......... .......... .......... 25% 109M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 24100K .......... .......... .......... .......... .......... 25% 98.1M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 24150K .......... .......... .......... .......... .......... 26% 128M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 24200K .......... .......... .......... .......... .......... 26% 84.7M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 24250K .......... .......... .......... .......... .......... 26% 113M 1s 2024-04-22T20:11:27.438 INFO:teuthology.orchestra.run.smithi136.stderr: 24300K .......... .......... .......... .......... .......... 26% 87.4M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24350K .......... .......... .......... .......... .......... 26% 115M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24400K .......... .......... .......... .......... .......... 26% 114M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24450K .......... .......... .......... .......... .......... 26% 83.4M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24500K .......... .......... .......... .......... .......... 26% 146M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24550K .......... .......... .......... .......... .......... 26% 141M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24600K .......... .......... .......... .......... .......... 26% 110M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24650K .......... .......... .......... .......... .......... 26% 123M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24700K .......... .......... .......... .......... .......... 26% 87.2M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24750K .......... .......... .......... .......... .......... 26% 94.6M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24800K .......... .......... .......... .......... .......... 26% 159M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24850K .......... .......... .......... .......... .......... 26% 111M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24900K .......... .......... .......... .......... .......... 26% 101M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 24950K .......... .......... .......... .......... .......... 26% 97.8M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 25000K .......... .......... .......... .......... .......... 26% 119M 1s 2024-04-22T20:11:27.439 INFO:teuthology.orchestra.run.smithi136.stderr: 25050K .......... .......... .......... .......... .......... 27% 112M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25100K .......... .......... .......... .......... .......... 27% 73.7M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25150K .......... .......... .......... .......... .......... 27% 164M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25200K .......... .......... .......... .......... .......... 27% 115M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25250K .......... .......... .......... .......... .......... 27% 115M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25300K .......... .......... .......... .......... .......... 27% 110M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25350K .......... .......... .......... .......... .......... 27% 116M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25400K .......... .......... .......... .......... .......... 27% 113M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25450K .......... .......... .......... .......... .......... 27% 117M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25500K .......... .......... .......... .......... .......... 27% 85.5M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25550K .......... .......... .......... .......... .......... 27% 117M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25600K .......... .......... .......... .......... .......... 27% 103M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25650K .......... .......... .......... .......... .......... 27% 110M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25700K .......... .......... .......... .......... .......... 27% 114M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25750K .......... .......... .......... .......... .......... 27% 111M 1s 2024-04-22T20:11:27.440 INFO:teuthology.orchestra.run.smithi136.stderr: 25800K .......... .......... .......... .......... .......... 27% 98.4M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 25850K .......... .......... .......... .......... .......... 27% 138M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 25900K .......... .......... .......... .......... .......... 27% 61.6M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 25950K .......... .......... .......... .......... .......... 27% 116M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26000K .......... .......... .......... .......... .......... 28% 114M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26050K .......... .......... .......... .......... .......... 28% 122M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26100K .......... .......... .......... .......... .......... 28% 108M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26150K .......... .......... .......... .......... .......... 28% 112M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26200K .......... .......... .......... .......... .......... 28% 114M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26250K .......... .......... .......... .......... .......... 28% 117M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26300K .......... .......... .......... .......... .......... 28% 87.0M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26350K .......... .......... .......... .......... .......... 28% 116M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26400K .......... .......... .......... .......... .......... 28% 109M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26450K .......... .......... .......... .......... .......... 28% 103M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26500K .......... .......... .......... .......... .......... 28% 114M 1s 2024-04-22T20:11:27.441 INFO:teuthology.orchestra.run.smithi136.stderr: 26550K .......... .......... .......... .......... .......... 28% 98.2M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 26600K .......... .......... .......... .......... .......... 28% 139M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 26650K .......... .......... .......... .......... .......... 28% 123M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 26700K .......... .......... .......... .......... .......... 28% 74.4M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 26750K .......... .......... .......... .......... .......... 28% 118M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 26800K .......... .......... .......... .......... .......... 28% 73.3M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 26850K .......... .......... .......... .......... .......... 28% 195M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 26900K .......... .......... .......... .......... .......... 29% 109M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 26950K .......... .......... .......... .......... .......... 29% 116M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 27000K .......... .......... .......... .......... .......... 29% 112M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 27050K .......... .......... .......... .......... .......... 29% 121M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 27100K .......... .......... .......... .......... .......... 29% 87.2M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 27150K .......... .......... .......... .......... .......... 29% 117M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 27200K .......... .......... .......... .......... .......... 29% 117M 1s 2024-04-22T20:11:27.442 INFO:teuthology.orchestra.run.smithi136.stderr: 27250K .......... .......... .......... .......... .......... 29% 114M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27300K .......... .......... .......... .......... .......... 29% 61.1M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27350K .......... .......... .......... .......... .......... 29% 111M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27400K .......... .......... .......... .......... .......... 29% 121M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27450K .......... .......... .......... .......... .......... 29% 105M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27500K .......... .......... .......... .......... .......... 29% 90.2M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27550K .......... .......... .......... .......... .......... 29% 83.0M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27600K .......... .......... .......... .......... .......... 29% 172M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27650K .......... .......... .......... .......... .......... 29% 122M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27700K .......... .......... .......... .......... .......... 29% 111M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27750K .......... .......... .......... .......... .......... 29% 96.0M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27800K .......... .......... .......... .......... .......... 29% 148M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27850K .......... .......... .......... .......... .......... 30% 112M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27900K .......... .......... .......... .......... .......... 30% 88.3M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 27950K .......... .......... .......... .......... .......... 30% 116M 1s 2024-04-22T20:11:27.443 INFO:teuthology.orchestra.run.smithi136.stderr: 28000K .......... .......... .......... .......... .......... 30% 115M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28050K .......... .......... .......... .......... .......... 30% 111M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28100K .......... .......... .......... .......... .......... 30% 110M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28150K .......... .......... .......... .......... .......... 30% 80.3M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28200K .......... .......... .......... .......... .......... 30% 194M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28250K .......... .......... .......... .......... .......... 30% 117M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28300K .......... .......... .......... .......... .......... 30% 83.8M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28350K .......... .......... .......... .......... .......... 30% 115M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28400K .......... .......... .......... .......... .......... 30% 120M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28450K .......... .......... .......... .......... .......... 30% 113M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28500K .......... .......... .......... .......... .......... 30% 108M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28550K .......... .......... .......... .......... .......... 30% 117M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28600K .......... .......... .......... .......... .......... 30% 117M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28650K .......... .......... .......... .......... .......... 30% 115M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28700K .......... .......... .......... .......... .......... 30% 88.9M 1s 2024-04-22T20:11:27.444 INFO:teuthology.orchestra.run.smithi136.stderr: 28750K .......... .......... .......... .......... .......... 31% 114M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 28800K .......... .......... .......... .......... .......... 31% 113M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 28850K .......... .......... .......... .......... .......... 31% 108M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 28900K .......... .......... .......... .......... .......... 31% 118M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 28950K .......... .......... .......... .......... .......... 31% 92.8M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29000K .......... .......... .......... .......... .......... 31% 123M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29050K .......... .......... .......... .......... .......... 31% 144M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29100K .......... .......... .......... .......... .......... 31% 72.6M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29150K .......... .......... .......... .......... .......... 31% 135M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29200K .......... .......... .......... .......... .......... 31% 113M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29250K .......... .......... .......... .......... .......... 31% 120M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29300K .......... .......... .......... .......... .......... 31% 111M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29350K .......... .......... .......... .......... .......... 31% 115M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29400K .......... .......... .......... .......... .......... 31% 113M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29450K .......... .......... .......... .......... .......... 31% 119M 1s 2024-04-22T20:11:27.445 INFO:teuthology.orchestra.run.smithi136.stderr: 29500K .......... .......... .......... .......... .......... 31% 86.6M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29550K .......... .......... .......... .......... .......... 31% 102M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29600K .......... .......... .......... .......... .......... 31% 139M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29650K .......... .......... .......... .......... .......... 31% 75.0M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29700K .......... .......... .......... .......... .......... 32% 223M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29750K .......... .......... .......... .......... .......... 32% 114M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29800K .......... .......... .......... .......... .......... 32% 64.7M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29850K .......... .......... .......... .......... .......... 32% 100M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29900K .......... .......... .......... .......... .......... 32% 98.4M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 29950K .......... .......... .......... .......... .......... 32% 116M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 30000K .......... .......... .......... .......... .......... 32% 106M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 30050K .......... .......... .......... .......... .......... 32% 123M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 30100K .......... .......... .......... .......... .......... 32% 108M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 30150K .......... .......... .......... .......... .......... 32% 117M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 30200K .......... .......... .......... .......... .......... 32% 115M 1s 2024-04-22T20:11:27.446 INFO:teuthology.orchestra.run.smithi136.stderr: 30250K .......... .......... .......... .......... .......... 32% 115M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30300K .......... .......... .......... .......... .......... 32% 87.9M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30350K .......... .......... .......... .......... .......... 32% 111M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30400K .......... .......... .......... .......... .......... 32% 120M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30450K .......... .......... .......... .......... .......... 32% 108M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30500K .......... .......... .......... .......... .......... 32% 116M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30550K .......... .......... .......... .......... .......... 32% 112M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30600K .......... .......... .......... .......... .......... 32% 114M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30650K .......... .......... .......... .......... .......... 33% 47.2M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30700K .......... .......... .......... .......... .......... 33% 96.5M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30750K .......... .......... .......... .......... .......... 33% 118M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30800K .......... .......... .......... .......... .......... 33% 113M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30850K .......... .......... .......... .......... .......... 33% 120M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30900K .......... .......... .......... .......... .......... 33% 108M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 30950K .......... .......... .......... .......... .......... 33% 120M 1s 2024-04-22T20:11:27.447 INFO:teuthology.orchestra.run.smithi136.stderr: 31000K .......... .......... .......... .......... .......... 33% 104M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31050K .......... .......... .......... .......... .......... 33% 119M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31100K .......... .......... .......... .......... .......... 33% 88.9M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31150K .......... .......... .......... .......... .......... 33% 112M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31200K .......... .......... .......... .......... .......... 33% 118M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31250K .......... .......... .......... .......... .......... 33% 112M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31300K .......... .......... .......... .......... .......... 33% 110M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31350K .......... .......... .......... .......... .......... 33% 105M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31400K .......... .......... .......... .......... .......... 33% 116M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31450K .......... .......... .......... .......... .......... 33% 115M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31500K .......... .......... .......... .......... .......... 33% 58.5M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31550K .......... .......... .......... .......... .......... 34% 260M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31600K .......... .......... .......... .......... .......... 34% 116M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31650K .......... .......... .......... .......... .......... 34% 121M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31700K .......... .......... .......... .......... .......... 34% 92.0M 1s 2024-04-22T20:11:27.448 INFO:teuthology.orchestra.run.smithi136.stderr: 31750K .......... .......... .......... .......... .......... 34% 136M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 31800K .......... .......... .......... .......... .......... 34% 111M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 31850K .......... .......... .......... .......... .......... 34% 118M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 31900K .......... .......... .......... .......... .......... 34% 90.4M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 31950K .......... .......... .......... .......... .......... 34% 118M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32000K .......... .......... .......... .......... .......... 34% 112M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32050K .......... .......... .......... .......... .......... 34% 111M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32100K .......... .......... .......... .......... .......... 34% 115M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32150K .......... .......... .......... .......... .......... 34% 110M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32200K .......... .......... .......... .......... .......... 34% 101M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32250K .......... .......... .......... .......... .......... 34% 106M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32300K .......... .......... .......... .......... .......... 34% 108M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32350K .......... .......... .......... .......... .......... 34% 116M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32400K .......... .......... .......... .......... .......... 34% 109M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32450K .......... .......... .......... .......... .......... 34% 120M 1s 2024-04-22T20:11:27.449 INFO:teuthology.orchestra.run.smithi136.stderr: 32500K .......... .......... .......... .......... .......... 35% 107M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32550K .......... .......... .......... .......... .......... 35% 118M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32600K .......... .......... .......... .......... .......... 35% 107M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32650K .......... .......... .......... .......... .......... 35% 118M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32700K .......... .......... .......... .......... .......... 35% 90.0M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32750K .......... .......... .......... .......... .......... 35% 56.1M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32800K .......... .......... .......... .......... .......... 35% 98.1M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32850K .......... .......... .......... .......... .......... 35% 77.6M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32900K .......... .......... .......... .......... .......... 35% 112M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 32950K .......... .......... .......... .......... .......... 35% 112M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 33000K .......... .......... .......... .......... .......... 35% 83.7M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 33050K .......... .......... .......... .......... .......... 35% 114M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 33100K .......... .......... .......... .......... .......... 35% 91.1M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 33150K .......... .......... .......... .......... .......... 35% 113M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 33200K .......... .......... .......... .......... .......... 35% 90.0M 1s 2024-04-22T20:11:27.450 INFO:teuthology.orchestra.run.smithi136.stderr: 33250K .......... .......... .......... .......... .......... 35% 148M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33300K .......... .......... .......... .......... .......... 35% 114M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33350K .......... .......... .......... .......... .......... 35% 112M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33400K .......... .......... .......... .......... .......... 36% 115M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33450K .......... .......... .......... .......... .......... 36% 116M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33500K .......... .......... .......... .......... .......... 36% 87.6M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33550K .......... .......... .......... .......... .......... 36% 119M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33600K .......... .......... .......... .......... .......... 36% 115M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33650K .......... .......... .......... .......... .......... 36% 106M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33700K .......... .......... .......... .......... .......... 36% 108M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33750K .......... .......... .......... .......... .......... 36% 114M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33800K .......... .......... .......... .......... .......... 36% 121M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33850K .......... .......... .......... .......... .......... 36% 108M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33900K .......... .......... .......... .......... .......... 36% 86.7M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 33950K .......... .......... .......... .......... .......... 36% 121M 1s 2024-04-22T20:11:27.451 INFO:teuthology.orchestra.run.smithi136.stderr: 34000K .......... .......... .......... .......... .......... 36% 116M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34050K .......... .......... .......... .......... .......... 36% 87.4M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34100K .......... .......... .......... .......... .......... 36% 101M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34150K .......... .......... .......... .......... .......... 36% 87.6M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34200K .......... .......... .......... .......... .......... 36% 186M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34250K .......... .......... .......... .......... .......... 36% 102M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34300K .......... .......... .......... .......... .......... 36% 100M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34350K .......... .......... .......... .......... .......... 37% 119M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34400K .......... .......... .......... .......... .......... 37% 111M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34450K .......... .......... .......... .......... .......... 37% 112M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34500K .......... .......... .......... .......... .......... 37% 115M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34550K .......... .......... .......... .......... .......... 37% 111M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34600K .......... .......... .......... .......... .......... 37% 117M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34650K .......... .......... .......... .......... .......... 37% 119M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34700K .......... .......... .......... .......... .......... 37% 89.2M 1s 2024-04-22T20:11:27.452 INFO:teuthology.orchestra.run.smithi136.stderr: 34750K .......... .......... .......... .......... .......... 37% 103M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 34800K .......... .......... .......... .......... .......... 37% 123M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 34850K .......... .......... .......... .......... .......... 37% 118M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 34900K .......... .......... .......... .......... .......... 37% 109M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 34950K .......... .......... .......... .......... .......... 37% 115M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35000K .......... .......... .......... .......... .......... 37% 116M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35050K .......... .......... .......... .......... .......... 37% 117M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35100K .......... .......... .......... .......... .......... 37% 90.0M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35150K .......... .......... .......... .......... .......... 37% 114M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35200K .......... .......... .......... .......... .......... 37% 117M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35250K .......... .......... .......... .......... .......... 37% 115M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35300K .......... .......... .......... .......... .......... 38% 109M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35350K .......... .......... .......... .......... .......... 38% 120M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35400K .......... .......... .......... .......... .......... 38% 77.7M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35450K .......... .......... .......... .......... .......... 38% 162M 1s 2024-04-22T20:11:27.453 INFO:teuthology.orchestra.run.smithi136.stderr: 35500K .......... .......... .......... .......... .......... 38% 82.3M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35550K .......... .......... .......... .......... .......... 38% 127M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35600K .......... .......... .......... .......... .......... 38% 120M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35650K .......... .......... .......... .......... .......... 38% 119M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35700K .......... .......... .......... .......... .......... 38% 109M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35750K .......... .......... .......... .......... .......... 38% 112M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35800K .......... .......... .......... .......... .......... 38% 118M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35850K .......... .......... .......... .......... .......... 38% 119M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35900K .......... .......... .......... .......... .......... 38% 90.1M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 35950K .......... .......... .......... .......... .......... 38% 116M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 36000K .......... .......... .......... .......... .......... 38% 79.6M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 36050K .......... .......... .......... .......... .......... 38% 179M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 36100K .......... .......... .......... .......... .......... 38% 111M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 36150K .......... .......... .......... .......... .......... 38% 116M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 36200K .......... .......... .......... .......... .......... 39% 113M 1s 2024-04-22T20:11:27.454 INFO:teuthology.orchestra.run.smithi136.stderr: 36250K .......... .......... .......... .......... .......... 39% 110M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36300K .......... .......... .......... .......... .......... 39% 92.3M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36350K .......... .......... .......... .......... .......... 39% 118M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36400K .......... .......... .......... .......... .......... 39% 110M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36450K .......... .......... .......... .......... .......... 39% 112M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36500K .......... .......... .......... .......... .......... 39% 113M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36550K .......... .......... .......... .......... .......... 39% 118M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36600K .......... .......... .......... .......... .......... 39% 111M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36650K .......... .......... .......... .......... .......... 39% 117M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36700K .......... .......... .......... .......... .......... 39% 85.6M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36750K .......... .......... .......... .......... .......... 39% 115M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36800K .......... .......... .......... .......... .......... 39% 87.4M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36850K .......... .......... .......... .......... .......... 39% 176M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36900K .......... .......... .......... .......... .......... 39% 109M 1s 2024-04-22T20:11:27.455 INFO:teuthology.orchestra.run.smithi136.stderr: 36950K .......... .......... .......... .......... .......... 39% 102M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37000K .......... .......... .......... .......... .......... 39% 130M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37050K .......... .......... .......... .......... .......... 39% 110M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37100K .......... .......... .......... .......... .......... 39% 90.2M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37150K .......... .......... .......... .......... .......... 40% 89.5M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37200K .......... .......... .......... .......... .......... 40% 86.5M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37250K .......... .......... .......... .......... .......... 40% 177M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37300K .......... .......... .......... .......... .......... 40% 103M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37350K .......... .......... .......... .......... .......... 40% 116M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37400K .......... .......... .......... .......... .......... 40% 108M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37450K .......... .......... .......... .......... .......... 40% 119M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37500K .......... .......... .......... .......... .......... 40% 92.2M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37550K .......... .......... .......... .......... .......... 40% 107M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37600K .......... .......... .......... .......... .......... 40% 74.6M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37650K .......... .......... .......... .......... .......... 40% 147M 1s 2024-04-22T20:11:27.456 INFO:teuthology.orchestra.run.smithi136.stderr: 37700K .......... .......... .......... .......... .......... 40% 105M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 37750K .......... .......... .......... .......... .......... 40% 121M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 37800K .......... .......... .......... .......... .......... 40% 120M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 37850K .......... .......... .......... .......... .......... 40% 100M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 37900K .......... .......... .......... .......... .......... 40% 34.0M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 37950K .......... .......... .......... .......... .......... 40% 95.6M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38000K .......... .......... .......... .......... .......... 40% 96.3M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38050K .......... .......... .......... .......... .......... 41% 115M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38100K .......... .......... .......... .......... .......... 41% 113M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38150K .......... .......... .......... .......... .......... 41% 114M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38200K .......... .......... .......... .......... .......... 41% 85.4M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38250K .......... .......... .......... .......... .......... 41% 186M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38300K .......... .......... .......... .......... .......... 41% 86.5M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38350K .......... .......... .......... .......... .......... 41% 110M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38400K .......... .......... .......... .......... .......... 41% 113M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38450K .......... .......... .......... .......... .......... 41% 115M 1s 2024-04-22T20:11:27.457 INFO:teuthology.orchestra.run.smithi136.stderr: 38500K .......... .......... .......... .......... .......... 41% 96.9M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38550K .......... .......... .......... .......... .......... 41% 134M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38600K .......... .......... .......... .......... .......... 41% 123M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38650K .......... .......... .......... .......... .......... 41% 112M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38700K .......... .......... .......... .......... .......... 41% 91.3M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38750K .......... .......... .......... .......... .......... 41% 119M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38800K .......... .......... .......... .......... .......... 41% 114M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38850K .......... .......... .......... .......... .......... 41% 107M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38900K .......... .......... .......... .......... .......... 41% 113M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 38950K .......... .......... .......... .......... .......... 41% 112M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 39000K .......... .......... .......... .......... .......... 42% 117M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 39050K .......... .......... .......... .......... .......... 42% 118M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 39100K .......... .......... .......... .......... .......... 42% 78.2M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 39150K .......... .......... .......... .......... .......... 42% 136M 1s 2024-04-22T20:11:27.458 INFO:teuthology.orchestra.run.smithi136.stderr: 39200K .......... .......... .......... .......... .......... 42% 110M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39250K .......... .......... .......... .......... .......... 42% 97.4M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39300K .......... .......... .......... .......... .......... 42% 141M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39350K .......... .......... .......... .......... .......... 42% 117M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39400K .......... .......... .......... .......... .......... 42% 89.6M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39450K .......... .......... .......... .......... .......... 42% 156M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39500K .......... .......... .......... .......... .......... 42% 80.1M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39550K .......... .......... .......... .......... .......... 42% 116M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39600K .......... .......... .......... .......... .......... 42% 119M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39650K .......... .......... .......... .......... .......... 42% 116M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39700K .......... .......... .......... .......... .......... 42% 105M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39750K .......... .......... .......... .......... .......... 42% 96.5M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39800K .......... .......... .......... .......... .......... 42% 148M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39850K .......... .......... .......... .......... .......... 42% 120M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39900K .......... .......... .......... .......... .......... 43% 86.8M 1s 2024-04-22T20:11:27.459 INFO:teuthology.orchestra.run.smithi136.stderr: 39950K .......... .......... .......... .......... .......... 43% 112M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40000K .......... .......... .......... .......... .......... 43% 119M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40050K .......... .......... .......... .......... .......... 43% 114M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40100K .......... .......... .......... .......... .......... 43% 108M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40150K .......... .......... .......... .......... .......... 43% 118M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40200K .......... .......... .......... .......... .......... 43% 112M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40250K .......... .......... .......... .......... .......... 43% 99.1M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40300K .......... .......... .......... .......... .......... 43% 106M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40350K .......... .......... .......... .......... .......... 43% 117M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40400K .......... .......... .......... .......... .......... 43% 116M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40450K .......... .......... .......... .......... .......... 43% 104M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40500K .......... .......... .......... .......... .......... 43% 112M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40550K .......... .......... .......... .......... .......... 43% 115M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40600K .......... .......... .......... .......... .......... 43% 112M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40650K .......... .......... .......... .......... .......... 43% 90.0M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40700K .......... .......... .......... .......... .......... 43% 120M 1s 2024-04-22T20:11:27.460 INFO:teuthology.orchestra.run.smithi136.stderr: 40750K .......... .......... .......... .......... .......... 43% 111M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 40800K .......... .......... .......... .......... .......... 43% 120M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 40850K .......... .......... .......... .......... .......... 44% 108M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 40900K .......... .......... .......... .......... .......... 44% 113M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 40950K .......... .......... .......... .......... .......... 44% 92.9M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41000K .......... .......... .......... .......... .......... 44% 147M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41050K .......... .......... .......... .......... .......... 44% 113M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41100K .......... .......... .......... .......... .......... 44% 80.8M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41150K .......... .......... .......... .......... .......... 44% 130M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41200K .......... .......... .......... .......... .......... 44% 108M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41250K .......... .......... .......... .......... .......... 44% 118M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41300K .......... .......... .......... .......... .......... 44% 112M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41350K .......... .......... .......... .......... .......... 44% 113M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41400K .......... .......... .......... .......... .......... 44% 87.7M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41450K .......... .......... .......... .......... .......... 44% 175M 1s 2024-04-22T20:11:27.461 INFO:teuthology.orchestra.run.smithi136.stderr: 41500K .......... .......... .......... .......... .......... 44% 87.7M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41550K .......... .......... .......... .......... .......... 44% 112M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41600K .......... .......... .......... .......... .......... 44% 110M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41650K .......... .......... .......... .......... .......... 44% 115M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41700K .......... .......... .......... .......... .......... 44% 115M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41750K .......... .......... .......... .......... .......... 44% 106M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41800K .......... .......... .......... .......... .......... 45% 111M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41850K .......... .......... .......... .......... .......... 45% 114M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41900K .......... .......... .......... .......... .......... 45% 87.0M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 41950K .......... .......... .......... .......... .......... 45% 115M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 42000K .......... .......... .......... .......... .......... 45% 112M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 42050K .......... .......... .......... .......... .......... 45% 118M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 42100K .......... .......... .......... .......... .......... 45% 95.0M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 42150K .......... .......... .......... .......... .......... 45% 132M 1s 2024-04-22T20:11:27.462 INFO:teuthology.orchestra.run.smithi136.stderr: 42200K .......... .......... .......... .......... .......... 45% 120M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42250K .......... .......... .......... .......... .......... 45% 113M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42300K .......... .......... .......... .......... .......... 45% 81.5M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42350K .......... .......... .......... .......... .......... 45% 124M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42400K .......... .......... .......... .......... .......... 45% 115M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42450K .......... .......... .......... .......... .......... 45% 98.0M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42500K .......... .......... .......... .......... .......... 45% 100M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42550K .......... .......... .......... .......... .......... 45% 117M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42600K .......... .......... .......... .......... .......... 45% 116M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42650K .......... .......... .......... .......... .......... 45% 116M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42700K .......... .......... .......... .......... .......... 46% 75.2M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42750K .......... .......... .......... .......... .......... 46% 151M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42800K .......... .......... .......... .......... .......... 46% 112M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42850K .......... .......... .......... .......... .......... 46% 118M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42900K .......... .......... .......... .......... .......... 46% 107M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 42950K .......... .......... .......... .......... .......... 46% 115M 1s 2024-04-22T20:11:27.463 INFO:teuthology.orchestra.run.smithi136.stderr: 43000K .......... .......... .......... .......... .......... 46% 117M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43050K .......... .......... .......... .......... .......... 46% 116M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43100K .......... .......... .......... .......... .......... 46% 92.1M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43150K .......... .......... .......... .......... .......... 46% 109M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43200K .......... .......... .......... .......... .......... 46% 66.0M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43250K .......... .......... .......... .......... .......... 46% 136M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43300K .......... .......... .......... .......... .......... 46% 113M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43350K .......... .......... .......... .......... .......... 46% 104M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43400K .......... .......... .......... .......... .......... 46% 43.7M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43450K .......... .......... .......... .......... .......... 46% 103M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43500K .......... .......... .......... .......... .......... 46% 75.4M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43550K .......... .......... .......... .......... .......... 46% 112M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43600K .......... .......... .......... .......... .......... 46% 108M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43650K .......... .......... .......... .......... .......... 47% 115M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43700K .......... .......... .......... .......... .......... 47% 113M 1s 2024-04-22T20:11:27.464 INFO:teuthology.orchestra.run.smithi136.stderr: 43750K .......... .......... .......... .......... .......... 47% 116M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 43800K .......... .......... .......... .......... .......... 47% 114M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 43850K .......... .......... .......... .......... .......... 47% 112M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 43900K .......... .......... .......... .......... .......... 47% 68.0M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 43950K .......... .......... .......... .......... .......... 47% 175M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44000K .......... .......... .......... .......... .......... 47% 119M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44050K .......... .......... .......... .......... .......... 47% 110M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44100K .......... .......... .......... .......... .......... 47% 91.4M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44150K .......... .......... .......... .......... .......... 47% 115M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44200K .......... .......... .......... .......... .......... 47% 113M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44250K .......... .......... .......... .......... .......... 47% 107M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44300K .......... .......... .......... .......... .......... 47% 78.2M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44350K .......... .......... .......... .......... .......... 47% 109M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44400K .......... .......... .......... .......... .......... 47% 150M 1s 2024-04-22T20:11:27.465 INFO:teuthology.orchestra.run.smithi136.stderr: 44450K .......... .......... .......... .......... .......... 47% 118M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44500K .......... .......... .......... .......... .......... 47% 114M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44550K .......... .......... .......... .......... .......... 48% 121M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44600K .......... .......... .......... .......... .......... 48% 114M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44650K .......... .......... .......... .......... .......... 48% 115M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44700K .......... .......... .......... .......... .......... 48% 88.3M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44750K .......... .......... .......... .......... .......... 48% 121M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44800K .......... .......... .......... .......... .......... 48% 120M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44850K .......... .......... .......... .......... .......... 48% 96.8M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44900K .......... .......... .......... .......... .......... 48% 110M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 44950K .......... .......... .......... .......... .......... 48% 120M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 45000K .......... .......... .......... .......... .......... 48% 86.3M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 45050K .......... .......... .......... .......... .......... 48% 175M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 45100K .......... .......... .......... .......... .......... 48% 89.8M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 45150K .......... .......... .......... .......... .......... 48% 123M 1s 2024-04-22T20:11:27.466 INFO:teuthology.orchestra.run.smithi136.stderr: 45200K .......... .......... .......... .......... .......... 48% 111M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45250K .......... .......... .......... .......... .......... 48% 117M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45300K .......... .......... .......... .......... .......... 48% 108M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45350K .......... .......... .......... .......... .......... 48% 117M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45400K .......... .......... .......... .......... .......... 48% 116M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45450K .......... .......... .......... .......... .......... 48% 78.1M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45500K .......... .......... .......... .......... .......... 49% 146M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45550K .......... .......... .......... .......... .......... 49% 113M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45600K .......... .......... .......... .......... .......... 49% 117M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45650K .......... .......... .......... .......... .......... 49% 115M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45700K .......... .......... .......... .......... .......... 49% 104M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45750K .......... .......... .......... .......... .......... 49% 118M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45800K .......... .......... .......... .......... .......... 49% 116M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45850K .......... .......... .......... .......... .......... 49% 117M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45900K .......... .......... .......... .......... .......... 49% 90.0M 1s 2024-04-22T20:11:27.467 INFO:teuthology.orchestra.run.smithi136.stderr: 45950K .......... .......... .......... .......... .......... 49% 112M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46000K .......... .......... .......... .......... .......... 49% 88.0M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46050K .......... .......... .......... .......... .......... 49% 115M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46100K .......... .......... .......... .......... .......... 49% 114M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46150K .......... .......... .......... .......... .......... 49% 92.2M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46200K .......... .......... .......... .......... .......... 49% 154M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46250K .......... .......... .......... .......... .......... 49% 106M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46300K .......... .......... .......... .......... .......... 49% 89.5M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46350K .......... .......... .......... .......... .......... 49% 115M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46400K .......... .......... .......... .......... .......... 49% 101M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46450K .......... .......... .......... .......... .......... 50% 132M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46500K .......... .......... .......... .......... .......... 50% 112M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46550K .......... .......... .......... .......... .......... 50% 109M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46600K .......... .......... .......... .......... .......... 50% 114M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46650K .......... .......... .......... .......... .......... 50% 64.4M 1s 2024-04-22T20:11:27.468 INFO:teuthology.orchestra.run.smithi136.stderr: 46700K .......... .......... .......... .......... .......... 50% 112M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 46750K .......... .......... .......... .......... .......... 50% 159M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 46800K .......... .......... .......... .......... .......... 50% 120M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 46850K .......... .......... .......... .......... .......... 50% 122M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 46900K .......... .......... .......... .......... .......... 50% 98.4M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 46950K .......... .......... .......... .......... .......... 50% 121M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47000K .......... .......... .......... .......... .......... 50% 111M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47050K .......... .......... .......... .......... .......... 50% 112M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47100K .......... .......... .......... .......... .......... 50% 96.4M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47150K .......... .......... .......... .......... .......... 50% 111M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47200K .......... .......... .......... .......... .......... 50% 117M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47250K .......... .......... .......... .......... .......... 50% 107M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47300K .......... .......... .......... .......... .......... 50% 103M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47350K .......... .......... .......... .......... .......... 51% 121M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47400K .......... .......... .......... .......... .......... 51% 125M 1s 2024-04-22T20:11:27.469 INFO:teuthology.orchestra.run.smithi136.stderr: 47450K .......... .......... .......... .......... .......... 51% 109M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47500K .......... .......... .......... .......... .......... 51% 58.0M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47550K .......... .......... .......... .......... .......... 51% 121M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47600K .......... .......... .......... .......... .......... 51% 202M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47650K .......... .......... .......... .......... .......... 51% 122M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47700K .......... .......... .......... .......... .......... 51% 104M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47750K .......... .......... .......... .......... .......... 51% 114M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47800K .......... .......... .......... .......... .......... 51% 122M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47850K .......... .......... .......... .......... .......... 51% 104M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47900K .......... .......... .......... .......... .......... 51% 99.8M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 47950K .......... .......... .......... .......... .......... 51% 112M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 48000K .......... .......... .......... .......... .......... 51% 114M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 48050K .......... .......... .......... .......... .......... 51% 118M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 48100K .......... .......... .......... .......... .......... 51% 82.8M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 48150K .......... .......... .......... .......... .......... 51% 161M 0s 2024-04-22T20:11:27.470 INFO:teuthology.orchestra.run.smithi136.stderr: 48200K .......... .......... .......... .......... .......... 51% 122M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48250K .......... .......... .......... .......... .......... 51% 101M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48300K .......... .......... .......... .......... .......... 52% 89.3M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48350K .......... .......... .......... .......... .......... 52% 117M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48400K .......... .......... .......... .......... .......... 52% 120M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48450K .......... .......... .......... .......... .......... 52% 114M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48500K .......... .......... .......... .......... .......... 52% 109M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48550K .......... .......... .......... .......... .......... 52% 113M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48600K .......... .......... .......... .......... .......... 52% 113M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48650K .......... .......... .......... .......... .......... 52% 113M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48700K .......... .......... .......... .......... .......... 52% 92.6M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48750K .......... .......... .......... .......... .......... 52% 116M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48800K .......... .......... .......... .......... .......... 52% 88.1M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48850K .......... .......... .......... .......... .......... 52% 114M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48900K .......... .......... .......... .......... .......... 52% 112M 0s 2024-04-22T20:11:27.471 INFO:teuthology.orchestra.run.smithi136.stderr: 48950K .......... .......... .......... .......... .......... 52% 94.3M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49000K .......... .......... .......... .......... .......... 52% 111M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49050K .......... .......... .......... .......... .......... 52% 76.2M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49100K .......... .......... .......... .......... .......... 52% 79.5M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49150K .......... .......... .......... .......... .......... 52% 141M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49200K .......... .......... .......... .......... .......... 53% 95.4M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49250K .......... .......... .......... .......... .......... 53% 140M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49300K .......... .......... .......... .......... .......... 53% 64.6M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49350K .......... .......... .......... .......... .......... 53% 383M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49400K .......... .......... .......... .......... .......... 53% 114M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49450K .......... .......... .......... .......... .......... 53% 123M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49500K .......... .......... .......... .......... .......... 53% 89.6M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49550K .......... .......... .......... .......... .......... 53% 90.7M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49600K .......... .......... .......... .......... .......... 53% 120M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49650K .......... .......... .......... .......... .......... 53% 77.7M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49700K .......... .......... .......... .......... .......... 53% 38.5M 0s 2024-04-22T20:11:27.472 INFO:teuthology.orchestra.run.smithi136.stderr: 49750K .......... .......... .......... .......... .......... 53% 107M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 49800K .......... .......... .......... .......... .......... 53% 112M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 49850K .......... .......... .......... .......... .......... 53% 110M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 49900K .......... .......... .......... .......... .......... 53% 34.8M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 49950K .......... .......... .......... .......... .......... 53% 131M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50000K .......... .......... .......... .......... .......... 53% 112M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50050K .......... .......... .......... .......... .......... 53% 115M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50100K .......... .......... .......... .......... .......... 53% 110M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50150K .......... .......... .......... .......... .......... 54% 104M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50200K .......... .......... .......... .......... .......... 54% 131M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50250K .......... .......... .......... .......... .......... 54% 89.1M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50300K .......... .......... .......... .......... .......... 54% 110M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50350K .......... .......... .......... .......... .......... 54% 105M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50400K .......... .......... .......... .......... .......... 54% 126M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50450K .......... .......... .......... .......... .......... 54% 106M 0s 2024-04-22T20:11:27.473 INFO:teuthology.orchestra.run.smithi136.stderr: 50500K .......... .......... .......... .......... .......... 54% 117M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50550K .......... .......... .......... .......... .......... 54% 120M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50600K .......... .......... .......... .......... .......... 54% 113M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50650K .......... .......... .......... .......... .......... 54% 110M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50700K .......... .......... .......... .......... .......... 54% 87.6M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50750K .......... .......... .......... .......... .......... 54% 106M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50800K .......... .......... .......... .......... .......... 54% 103M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50850K .......... .......... .......... .......... .......... 54% 142M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50900K .......... .......... .......... .......... .......... 54% 106M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 50950K .......... .......... .......... .......... .......... 54% 116M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 51000K .......... .......... .......... .......... .......... 54% 117M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 51050K .......... .......... .......... .......... .......... 55% 114M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 51100K .......... .......... .......... .......... .......... 55% 88.3M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 51150K .......... .......... .......... .......... .......... 55% 116M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 51200K .......... .......... .......... .......... .......... 55% 98.2M 0s 2024-04-22T20:11:27.474 INFO:teuthology.orchestra.run.smithi136.stderr: 51250K .......... .......... .......... .......... .......... 55% 90.5M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51300K .......... .......... .......... .......... .......... 55% 155M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51350K .......... .......... .......... .......... .......... 55% 83.0M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51400K .......... .......... .......... .......... .......... 55% 179M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51450K .......... .......... .......... .......... .......... 55% 117M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51500K .......... .......... .......... .......... .......... 55% 86.5M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51550K .......... .......... .......... .......... .......... 55% 116M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51600K .......... .......... .......... .......... .......... 55% 114M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51650K .......... .......... .......... .......... .......... 55% 118M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51700K .......... .......... .......... .......... .......... 55% 100M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51750K .......... .......... .......... .......... .......... 55% 89.9M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51800K .......... .......... .......... .......... .......... 55% 167M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51850K .......... .......... .......... .......... .......... 55% 119M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51900K .......... .......... .......... .......... .......... 55% 70.6M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 51950K .......... .......... .......... .......... .......... 55% 168M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 52000K .......... .......... .......... .......... .......... 56% 101M 0s 2024-04-22T20:11:27.475 INFO:teuthology.orchestra.run.smithi136.stderr: 52050K .......... .......... .......... .......... .......... 56% 134M 0s 2024-04-22T20:11:27.476 INFO:teuthology.orchestra.run.smithi136.stderr: 52100K .......... .......... .......... .......... .......... 56% 114M 0s 2024-04-22T20:11:27.476 INFO:teuthology.orchestra.run.smithi136.stderr: 52150K .......... .......... .......... .......... .......... 56% 108M 0s 2024-04-22T20:11:27.476 INFO:teuthology.orchestra.run.smithi136.stderr: 52200K .......... .......... .......... .......... .......... 56% 94.6M 0s 2024-04-22T20:11:27.476 INFO:teuthology.orchestra.run.smithi136.stderr: 52250K .......... .......... .......... .......... .......... 56% 137M 0s 2024-04-22T20:11:27.476 INFO:teuthology.orchestra.run.smithi136.stderr: 52300K .......... .......... .......... .......... .......... 56% 94.7M 0s 2024-04-22T20:11:27.476 INFO:teuthology.orchestra.run.smithi136.stderr: 52350K .......... .......... .......... .......... .......... 56% 117M 0s 2024-04-22T20:11:27.476 INFO:teuthology.orchestra.run.smithi136.stderr: 52400K .......... .......... .......... .......... .......... 56% 103M 0s 2024-04-22T20:11:27.477 INFO:teuthology.orchestra.run.smithi136.stderr: 52450K .......... .......... .......... .......... .......... 56% 93.1M 0s 2024-04-22T20:11:27.477 INFO:teuthology.orchestra.run.smithi136.stderr: 52500K .......... .......... .......... .......... .......... 56% 124M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52550K .......... .......... .......... .......... .......... 56% 51.4M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52600K .......... .......... .......... .......... .......... 56% 112M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52650K .......... .......... .......... .......... .......... 56% 112M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52700K .......... .......... .......... .......... .......... 56% 91.0M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52750K .......... .......... .......... .......... .......... 56% 108M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52800K .......... .......... .......... .......... .......... 56% 88.7M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52850K .......... .......... .......... .......... .......... 56% 167M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52900K .......... .......... .......... .......... .......... 56% 111M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 52950K .......... .......... .......... .......... .......... 57% 106M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 53000K .......... .......... .......... .......... .......... 57% 125M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 53050K .......... .......... .......... .......... .......... 57% 108M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 53100K .......... .......... .......... .......... .......... 57% 85.0M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 53150K .......... .......... .......... .......... .......... 57% 132M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 53200K .......... .......... .......... .......... .......... 57% 113M 0s 2024-04-22T20:11:27.478 INFO:teuthology.orchestra.run.smithi136.stderr: 53250K .......... .......... .......... .......... .......... 57% 116M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53300K .......... .......... .......... .......... .......... 57% 108M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53350K .......... .......... .......... .......... .......... 57% 105M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53400K .......... .......... .......... .......... .......... 57% 111M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53450K .......... .......... .......... .......... .......... 57% 123M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53500K .......... .......... .......... .......... .......... 57% 86.7M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53550K .......... .......... .......... .......... .......... 57% 97.6M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53600K .......... .......... .......... .......... .......... 57% 104M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53650K .......... .......... .......... .......... .......... 57% 108M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53700K .......... .......... .......... .......... .......... 57% 131M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53750K .......... .......... .......... .......... .......... 57% 108M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53800K .......... .......... .......... .......... .......... 57% 117M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53850K .......... .......... .......... .......... .......... 58% 109M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53900K .......... .......... .......... .......... .......... 58% 88.2M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 53950K .......... .......... .......... .......... .......... 58% 122M 0s 2024-04-22T20:11:27.479 INFO:teuthology.orchestra.run.smithi136.stderr: 54000K .......... .......... .......... .......... .......... 58% 85.7M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54050K .......... .......... .......... .......... .......... 58% 165M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54100K .......... .......... .......... .......... .......... 58% 107M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54150K .......... .......... .......... .......... .......... 58% 110M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54200K .......... .......... .......... .......... .......... 58% 109M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54250K .......... .......... .......... .......... .......... 58% 113M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54300K .......... .......... .......... .......... .......... 58% 79.5M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54350K .......... .......... .......... .......... .......... 58% 82.3M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54400K .......... .......... .......... .......... .......... 58% 79.0M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54450K .......... .......... .......... .......... .......... 58% 206M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54500K .......... .......... .......... .......... .......... 58% 117M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54550K .......... .......... .......... .......... .......... 58% 115M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54600K .......... .......... .......... .......... .......... 58% 117M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54650K .......... .......... .......... .......... .......... 58% 85.4M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54700K .......... .......... .......... .......... .......... 58% 71.1M 0s 2024-04-22T20:11:27.480 INFO:teuthology.orchestra.run.smithi136.stderr: 54750K .......... .......... .......... .......... .......... 58% 128M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 54800K .......... .......... .......... .......... .......... 59% 118M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 54850K .......... .......... .......... .......... .......... 59% 115M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 54900K .......... .......... .......... .......... .......... 59% 101M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 54950K .......... .......... .......... .......... .......... 59% 123M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55000K .......... .......... .......... .......... .......... 59% 108M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55050K .......... .......... .......... .......... .......... 59% 107M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55100K .......... .......... .......... .......... .......... 59% 85.5M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55150K .......... .......... .......... .......... .......... 59% 110M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55200K .......... .......... .......... .......... .......... 59% 115M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55250K .......... .......... .......... .......... .......... 59% 97.8M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55300K .......... .......... .......... .......... .......... 59% 61.9M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55350K .......... .......... .......... .......... .......... 59% 92.6M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55400K .......... .......... .......... .......... .......... 59% 104M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55450K .......... .......... .......... .......... .......... 59% 114M 0s 2024-04-22T20:11:27.481 INFO:teuthology.orchestra.run.smithi136.stderr: 55500K .......... .......... .......... .......... .......... 59% 81.7M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55550K .......... .......... .......... .......... .......... 59% 28.2M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55600K .......... .......... .......... .......... .......... 59% 91.2M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55650K .......... .......... .......... .......... .......... 59% 116M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55700K .......... .......... .......... .......... .......... 60% 109M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55750K .......... .......... .......... .......... .......... 60% 122M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55800K .......... .......... .......... .......... .......... 60% 111M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55850K .......... .......... .......... .......... .......... 60% 113M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55900K .......... .......... .......... .......... .......... 60% 83.7M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 55950K .......... .......... .......... .......... .......... 60% 115M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 56000K .......... .......... .......... .......... .......... 60% 17.0M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 56050K .......... .......... .......... .......... .......... 60% 218M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 56100K .......... .......... .......... .......... .......... 60% 348M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 56150K .......... .......... .......... .......... .......... 60% 389M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 56200K .......... .......... .......... .......... .......... 60% 381M 0s 2024-04-22T20:11:27.482 INFO:teuthology.orchestra.run.smithi136.stderr: 56250K .......... .......... .......... .......... .......... 60% 433M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56300K .......... .......... .......... .......... .......... 60% 329M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56350K .......... .......... .......... .......... .......... 60% 188M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56400K .......... .......... .......... .......... .......... 60% 399M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56450K .......... .......... .......... .......... .......... 60% 105M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56500K .......... .......... .......... .......... .......... 60% 91.4M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56550K .......... .......... .......... .......... .......... 60% 46.8M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56600K .......... .......... .......... .......... .......... 60% 181M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56650K .......... .......... .......... .......... .......... 61% 168M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56700K .......... .......... .......... .......... .......... 61% 190M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56750K .......... .......... .......... .......... .......... 61% 194M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56800K .......... .......... .......... .......... .......... 61% 113M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56850K .......... .......... .......... .......... .......... 61% 118M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56900K .......... .......... .......... .......... .......... 61% 106M 0s 2024-04-22T20:11:27.483 INFO:teuthology.orchestra.run.smithi136.stderr: 56950K .......... .......... .......... .......... .......... 61% 120M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57000K .......... .......... .......... .......... .......... 61% 111M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57050K .......... .......... .......... .......... .......... 61% 104M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57100K .......... .......... .......... .......... .......... 61% 92.6M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57150K .......... .......... .......... .......... .......... 61% 118M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57200K .......... .......... .......... .......... .......... 61% 120M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57250K .......... .......... .......... .......... .......... 61% 112M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57300K .......... .......... .......... .......... .......... 61% 108M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57350K .......... .......... .......... .......... .......... 61% 118M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57400K .......... .......... .......... .......... .......... 61% 81.5M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57450K .......... .......... .......... .......... .......... 61% 202M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57500K .......... .......... .......... .......... .......... 61% 71.7M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57550K .......... .......... .......... .......... .......... 62% 134M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57600K .......... .......... .......... .......... .......... 62% 135M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57650K .......... .......... .......... .......... .......... 62% 114M 0s 2024-04-22T20:11:27.484 INFO:teuthology.orchestra.run.smithi136.stderr: 57700K .......... .......... .......... .......... .......... 62% 108M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 57750K .......... .......... .......... .......... .......... 62% 115M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 57800K .......... .......... .......... .......... .......... 62% 114M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 57850K .......... .......... .......... .......... .......... 62% 103M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 57900K .......... .......... .......... .......... .......... 62% 84.1M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 57950K .......... .......... .......... .......... .......... 62% 85.8M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58000K .......... .......... .......... .......... .......... 62% 188M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58050K .......... .......... .......... .......... .......... 62% 118M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58100K .......... .......... .......... .......... .......... 62% 107M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58150K .......... .......... .......... .......... .......... 62% 118M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58200K .......... .......... .......... .......... .......... 62% 115M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58250K .......... .......... .......... .......... .......... 62% 85.9M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58300K .......... .......... .......... .......... .......... 62% 116M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58350K .......... .......... .......... .......... .......... 62% 88.1M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58400K .......... .......... .......... .......... .......... 62% 132M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58450K .......... .......... .......... .......... .......... 62% 141M 0s 2024-04-22T20:11:27.485 INFO:teuthology.orchestra.run.smithi136.stderr: 58500K .......... .......... .......... .......... .......... 63% 114M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58550K .......... .......... .......... .......... .......... 63% 115M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58600K .......... .......... .......... .......... .......... 63% 112M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58650K .......... .......... .......... .......... .......... 63% 114M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58700K .......... .......... .......... .......... .......... 63% 62.3M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58750K .......... .......... .......... .......... .......... 63% 283M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58800K .......... .......... .......... .......... .......... 63% 100M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58850K .......... .......... .......... .......... .......... 63% 132M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58900K .......... .......... .......... .......... .......... 63% 112M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 58950K .......... .......... .......... .......... .......... 63% 117M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 59000K .......... .......... .......... .......... .......... 63% 112M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 59050K .......... .......... .......... .......... .......... 63% 114M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 59100K .......... .......... .......... .......... .......... 63% 89.9M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 59150K .......... .......... .......... .......... .......... 63% 108M 0s 2024-04-22T20:11:27.486 INFO:teuthology.orchestra.run.smithi136.stderr: 59200K .......... .......... .......... .......... .......... 63% 113M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59250K .......... .......... .......... .......... .......... 63% 124M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59300K .......... .......... .......... .......... .......... 63% 109M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59350K .......... .......... .......... .......... .......... 63% 122M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59400K .......... .......... .......... .......... .......... 63% 112M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59450K .......... .......... .......... .......... .......... 64% 91.3M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59500K .......... .......... .......... .......... .......... 64% 76.8M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59550K .......... .......... .......... .......... .......... 64% 139M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59600K .......... .......... .......... .......... .......... 64% 114M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59650K .......... .......... .......... .......... .......... 64% 121M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59700K .......... .......... .......... .......... .......... 64% 107M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59750K .......... .......... .......... .......... .......... 64% 110M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59800K .......... .......... .......... .......... .......... 64% 114M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59850K .......... .......... .......... .......... .......... 64% 98.6M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59900K .......... .......... .......... .......... .......... 64% 87.3M 0s 2024-04-22T20:11:27.487 INFO:teuthology.orchestra.run.smithi136.stderr: 59950K .......... .......... .......... .......... .......... 64% 116M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60000K .......... .......... .......... .......... .......... 64% 107M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60050K .......... .......... .......... .......... .......... 64% 135M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60100K .......... .......... .......... .......... .......... 64% 113M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60150K .......... .......... .......... .......... .......... 64% 115M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60200K .......... .......... .......... .......... .......... 64% 112M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60250K .......... .......... .......... .......... .......... 64% 112M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60300K .......... .......... .......... .......... .......... 64% 69.6M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60350K .......... .......... .......... .......... .......... 65% 192M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60400K .......... .......... .......... .......... .......... 65% 94.3M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60450K .......... .......... .......... .......... .......... 65% 147M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60500K .......... .......... .......... .......... .......... 65% 111M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60550K .......... .......... .......... .......... .......... 65% 109M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60600K .......... .......... .......... .......... .......... 65% 117M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60650K .......... .......... .......... .......... .......... 65% 111M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60700K .......... .......... .......... .......... .......... 65% 68.4M 0s 2024-04-22T20:11:27.488 INFO:teuthology.orchestra.run.smithi136.stderr: 60750K .......... .......... .......... .......... .......... 65% 178M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 60800K .......... .......... .......... .......... .......... 65% 112M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 60850K .......... .......... .......... .......... .......... 65% 115M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 60900K .......... .......... .......... .......... .......... 65% 111M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 60950K .......... .......... .......... .......... .......... 65% 105M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61000K .......... .......... .......... .......... .......... 65% 116M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61050K .......... .......... .......... .......... .......... 65% 114M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61100K .......... .......... .......... .......... .......... 65% 89.3M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61150K .......... .......... .......... .......... .......... 65% 69.8M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61200K .......... .......... .......... .......... .......... 65% 226M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61250K .......... .......... .......... .......... .......... 65% 114M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61300K .......... .......... .......... .......... .......... 66% 107M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61350K .......... .......... .......... .......... .......... 66% 116M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61400K .......... .......... .......... .......... .......... 66% 95.4M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61450K .......... .......... .......... .......... .......... 66% 99.9M 0s 2024-04-22T20:11:27.489 INFO:teuthology.orchestra.run.smithi136.stderr: 61500K .......... .......... .......... .......... .......... 66% 101M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61550K .......... .......... .......... .......... .......... 66% 114M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61600K .......... .......... .......... .......... .......... 66% 119M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61650K .......... .......... .......... .......... .......... 66% 88.8M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61700K .......... .......... .......... .......... .......... 66% 127M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61750K .......... .......... .......... .......... .......... 66% 131M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61800K .......... .......... .......... .......... .......... 66% 115M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61850K .......... .......... .......... .......... .......... 66% 114M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61900K .......... .......... .......... .......... .......... 66% 91.2M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 61950K .......... .......... .......... .......... .......... 66% 109M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 62000K .......... .......... .......... .......... .......... 66% 124M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 62050K .......... .......... .......... .......... .......... 66% 114M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 62100K .......... .......... .......... .......... .......... 66% 103M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 62150K .......... .......... .......... .......... .......... 66% 130M 0s 2024-04-22T20:11:27.490 INFO:teuthology.orchestra.run.smithi136.stderr: 62200K .......... .......... .......... .......... .......... 67% 114M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62250K .......... .......... .......... .......... .......... 67% 114M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62300K .......... .......... .......... .......... .......... 67% 86.1M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62350K .......... .......... .......... .......... .......... 67% 114M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62400K .......... .......... .......... .......... .......... 67% 121M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62450K .......... .......... .......... .......... .......... 67% 113M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62500K .......... .......... .......... .......... .......... 67% 112M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62550K .......... .......... .......... .......... .......... 67% 106M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62600K .......... .......... .......... .......... .......... 67% 124M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62650K .......... .......... .......... .......... .......... 67% 90.1M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62700K .......... .......... .......... .......... .......... 67% 113M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62750K .......... .......... .......... .......... .......... 67% 116M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62800K .......... .......... .......... .......... .......... 67% 104M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62850K .......... .......... .......... .......... .......... 67% 88.5M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62900K .......... .......... .......... .......... .......... 67% 109M 0s 2024-04-22T20:11:27.491 INFO:teuthology.orchestra.run.smithi136.stderr: 62950K .......... .......... .......... .......... .......... 67% 125M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63000K .......... .......... .......... .......... .......... 67% 112M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63050K .......... .......... .......... .......... .......... 67% 95.6M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63100K .......... .......... .......... .......... .......... 67% 81.8M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63150K .......... .......... .......... .......... .......... 68% 103M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63200K .......... .......... .......... .......... .......... 68% 96.8M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63250K .......... .......... .......... .......... .......... 68% 190M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63300K .......... .......... .......... .......... .......... 68% 91.7M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63350K .......... .......... .......... .......... .......... 68% 122M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63400K .......... .......... .......... .......... .......... 68% 119M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63450K .......... .......... .......... .......... .......... 68% 89.8M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63500K .......... .......... .......... .......... .......... 68% 120M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63550K .......... .......... .......... .......... .......... 68% 109M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63600K .......... .......... .......... .......... .......... 68% 120M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63650K .......... .......... .......... .......... .......... 68% 107M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63700K .......... .......... .......... .......... .......... 68% 121M 0s 2024-04-22T20:11:27.492 INFO:teuthology.orchestra.run.smithi136.stderr: 63750K .......... .......... .......... .......... .......... 68% 117M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 63800K .......... .......... .......... .......... .......... 68% 106M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 63850K .......... .......... .......... .......... .......... 68% 119M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 63900K .......... .......... .......... .......... .......... 68% 88.9M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 63950K .......... .......... .......... .......... .......... 68% 103M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64000K .......... .......... .......... .......... .......... 68% 115M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64050K .......... .......... .......... .......... .......... 68% 110M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64100K .......... .......... .......... .......... .......... 69% 110M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64150K .......... .......... .......... .......... .......... 69% 87.3M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64200K .......... .......... .......... .......... .......... 69% 168M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64250K .......... .......... .......... .......... .......... 69% 105M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64300K .......... .......... .......... .......... .......... 69% 68.0M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64350K .......... .......... .......... .......... .......... 69% 194M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64400K .......... .......... .......... .......... .......... 69% 112M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64450K .......... .......... .......... .......... .......... 69% 88.3M 0s 2024-04-22T20:11:27.493 INFO:teuthology.orchestra.run.smithi136.stderr: 64500K .......... .......... .......... .......... .......... 69% 156M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64550K .......... .......... .......... .......... .......... 69% 119M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64600K .......... .......... .......... .......... .......... 69% 112M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64650K .......... .......... .......... .......... .......... 69% 91.6M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64700K .......... .......... .......... .......... .......... 69% 88.0M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64750K .......... .......... .......... .......... .......... 69% 110M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64800K .......... .......... .......... .......... .......... 69% 122M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64850K .......... .......... .......... .......... .......... 69% 106M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64900K .......... .......... .......... .......... .......... 69% 116M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 64950K .......... .......... .......... .......... .......... 69% 106M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 65000K .......... .......... .......... .......... .......... 70% 119M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 65050K .......... .......... .......... .......... .......... 70% 118M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 65100K .......... .......... .......... .......... .......... 70% 88.4M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 65150K .......... .......... .......... .......... .......... 70% 121M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 65200K .......... .......... .......... .......... .......... 70% 112M 0s 2024-04-22T20:11:27.494 INFO:teuthology.orchestra.run.smithi136.stderr: 65250K .......... .......... .......... .......... .......... 70% 107M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65300K .......... .......... .......... .......... .......... 70% 113M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65350K .......... .......... .......... .......... .......... 70% 116M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65400K .......... .......... .......... .......... .......... 70% 107M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65450K .......... .......... .......... .......... .......... 70% 105M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65500K .......... .......... .......... .......... .......... 70% 94.5M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65550K .......... .......... .......... .......... .......... 70% 120M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65600K .......... .......... .......... .......... .......... 70% 111M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65650K .......... .......... .......... .......... .......... 70% 115M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65700K .......... .......... .......... .......... .......... 70% 114M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65750K .......... .......... .......... .......... .......... 70% 115M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65800K .......... .......... .......... .......... .......... 70% 117M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65850K .......... .......... .......... .......... .......... 70% 112M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65900K .......... .......... .......... .......... .......... 70% 90.0M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 65950K .......... .......... .......... .......... .......... 71% 112M 0s 2024-04-22T20:11:27.495 INFO:teuthology.orchestra.run.smithi136.stderr: 66000K .......... .......... .......... .......... .......... 71% 123M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66050K .......... .......... .......... .......... .......... 71% 88.3M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66100K .......... .......... .......... .......... .......... 71% 131M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66150K .......... .......... .......... .......... .......... 71% 72.7M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66200K .......... .......... .......... .......... .......... 71% 86.2M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66250K .......... .......... .......... .......... .......... 71% 175M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66300K .......... .......... .......... .......... .......... 71% 87.4M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66350K .......... .......... .......... .......... .......... 71% 117M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66400K .......... .......... .......... .......... .......... 71% 111M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66450K .......... .......... .......... .......... .......... 71% 110M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66500K .......... .......... .......... .......... .......... 71% 122M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66550K .......... .......... .......... .......... .......... 71% 106M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66600K .......... .......... .......... .......... .......... 71% 110M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66650K .......... .......... .......... .......... .......... 71% 127M 0s 2024-04-22T20:11:27.496 INFO:teuthology.orchestra.run.smithi136.stderr: 66700K .......... .......... .......... .......... .......... 71% 82.9M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 66750K .......... .......... .......... .......... .......... 71% 129M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 66800K .......... .......... .......... .......... .......... 71% 117M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 66850K .......... .......... .......... .......... .......... 72% 113M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 66900K .......... .......... .......... .......... .......... 72% 92.3M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 66950K .......... .......... .......... .......... .......... 72% 127M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67000K .......... .......... .......... .......... .......... 72% 125M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67050K .......... .......... .......... .......... .......... 72% 94.2M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67100K .......... .......... .......... .......... .......... 72% 98.0M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67150K .......... .......... .......... .......... .......... 72% 127M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67200K .......... .......... .......... .......... .......... 72% 94.5M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67250K .......... .......... .......... .......... .......... 72% 139M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67300K .......... .......... .......... .......... .......... 72% 103M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67350K .......... .......... .......... .......... .......... 72% 108M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67400K .......... .......... .......... .......... .......... 72% 113M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67450K .......... .......... .......... .......... .......... 72% 116M 0s 2024-04-22T20:11:27.497 INFO:teuthology.orchestra.run.smithi136.stderr: 67500K .......... .......... .......... .......... .......... 72% 91.3M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67550K .......... .......... .......... .......... .......... 72% 121M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67600K .......... .......... .......... .......... .......... 72% 114M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67650K .......... .......... .......... .......... .......... 72% 114M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67700K .......... .......... .......... .......... .......... 72% 99.3M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67750K .......... .......... .......... .......... .......... 72% 118M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67800K .......... .......... .......... .......... .......... 73% 107M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67850K .......... .......... .......... .......... .......... 73% 116M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67900K .......... .......... .......... .......... .......... 73% 91.2M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 67950K .......... .......... .......... .......... .......... 73% 109M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 68000K .......... .......... .......... .......... .......... 73% 121M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 68050K .......... .......... .......... .......... .......... 73% 113M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 68100K .......... .......... .......... .......... .......... 73% 109M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 68150K .......... .......... .......... .......... .......... 73% 120M 0s 2024-04-22T20:11:27.498 INFO:teuthology.orchestra.run.smithi136.stderr: 68200K .......... .......... .......... .......... .......... 73% 109M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68250K .......... .......... .......... .......... .......... 73% 115M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68300K .......... .......... .......... .......... .......... 73% 72.9M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68350K .......... .......... .......... .......... .......... 73% 155M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68400K .......... .......... .......... .......... .......... 73% 101M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68450K .......... .......... .......... .......... .......... 73% 126M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68500K .......... .......... .......... .......... .......... 73% 102M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68550K .......... .......... .......... .......... .......... 73% 115M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68600K .......... .......... .......... .......... .......... 73% 112M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68650K .......... .......... .......... .......... .......... 73% 87.7M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68700K .......... .......... .......... .......... .......... 74% 96.9M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68750K .......... .......... .......... .......... .......... 74% 104M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68800K .......... .......... .......... .......... .......... 74% 132M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68850K .......... .......... .......... .......... .......... 74% 113M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68900K .......... .......... .......... .......... .......... 74% 110M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 68950K .......... .......... .......... .......... .......... 74% 114M 0s 2024-04-22T20:11:27.499 INFO:teuthology.orchestra.run.smithi136.stderr: 69000K .......... .......... .......... .......... .......... 74% 104M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69050K .......... .......... .......... .......... .......... 74% 130M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69100K .......... .......... .......... .......... .......... 74% 84.7M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69150K .......... .......... .......... .......... .......... 74% 123M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69200K .......... .......... .......... .......... .......... 74% 121M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69250K .......... .......... .......... .......... .......... 74% 100M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69300K .......... .......... .......... .......... .......... 74% 78.0M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69350K .......... .......... .......... .......... .......... 74% 133M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69400K .......... .......... .......... .......... .......... 74% 112M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69450K .......... .......... .......... .......... .......... 74% 120M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69500K .......... .......... .......... .......... .......... 74% 85.3M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69550K .......... .......... .......... .......... .......... 74% 117M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69600K .......... .......... .......... .......... .......... 74% 121M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69650K .......... .......... .......... .......... .......... 75% 104M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69700K .......... .......... .......... .......... .......... 75% 126M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69750K .......... .......... .......... .......... .......... 75% 76.6M 0s 2024-04-22T20:11:27.500 INFO:teuthology.orchestra.run.smithi136.stderr: 69800K .......... .......... .......... .......... .......... 75% 94.8M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 69850K .......... .......... .......... .......... .......... 75% 91.7M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 69900K .......... .......... .......... .......... .......... 75% 75.3M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 69950K .......... .......... .......... .......... .......... 75% 91.5M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70000K .......... .......... .......... .......... .......... 75% 161M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70050K .......... .......... .......... .......... .......... 75% 111M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70100K .......... .......... .......... .......... .......... 75% 105M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70150K .......... .......... .......... .......... .......... 75% 121M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70200K .......... .......... .......... .......... .......... 75% 112M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70250K .......... .......... .......... .......... .......... 75% 111M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70300K .......... .......... .......... .......... .......... 75% 90.0M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70350K .......... .......... .......... .......... .......... 75% 114M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70400K .......... .......... .......... .......... .......... 75% 114M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70450K .......... .......... .......... .......... .......... 75% 113M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70500K .......... .......... .......... .......... .......... 75% 120M 0s 2024-04-22T20:11:27.501 INFO:teuthology.orchestra.run.smithi136.stderr: 70550K .......... .......... .......... .......... .......... 75% 104M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 70600K .......... .......... .......... .......... .......... 76% 117M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 70650K .......... .......... .......... .......... .......... 76% 103M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 70700K .......... .......... .......... .......... .......... 76% 104M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 70750K .......... .......... .......... .......... .......... 76% 113M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 70800K .......... .......... .......... .......... .......... 76% 113M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 70850K .......... .......... .......... .......... .......... 76% 120M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 70900K .......... .......... .......... .......... .......... 76% 105M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 70950K .......... .......... .......... .......... .......... 76% 115M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 71000K .......... .......... .......... .......... .......... 76% 113M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 71050K .......... .......... .......... .......... .......... 76% 85.3M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 71100K .......... .......... .......... .......... .......... 76% 121M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 71150K .......... .......... .......... .......... .......... 76% 100M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 71200K .......... .......... .......... .......... .......... 76% 94.8M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 71250K .......... .......... .......... .......... .......... 76% 147M 0s 2024-04-22T20:11:27.502 INFO:teuthology.orchestra.run.smithi136.stderr: 71300K .......... .......... .......... .......... .......... 76% 116M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71350K .......... .......... .......... .......... .......... 76% 119M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71400K .......... .......... .......... .......... .......... 76% 101M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71450K .......... .......... .......... .......... .......... 76% 117M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71500K .......... .......... .......... .......... .......... 77% 90.5M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71550K .......... .......... .......... .......... .......... 77% 96.4M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71600K .......... .......... .......... .......... .......... 77% 115M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71650K .......... .......... .......... .......... .......... 77% 116M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71700K .......... .......... .......... .......... .......... 77% 113M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71750K .......... .......... .......... .......... .......... 77% 113M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71800K .......... .......... .......... .......... .......... 77% 117M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71850K .......... .......... .......... .......... .......... 77% 115M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71900K .......... .......... .......... .......... .......... 77% 89.2M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 71950K .......... .......... .......... .......... .......... 77% 106M 0s 2024-04-22T20:11:27.503 INFO:teuthology.orchestra.run.smithi136.stderr: 72000K .......... .......... .......... .......... .......... 77% 121M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72050K .......... .......... .......... .......... .......... 77% 120M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72100K .......... .......... .......... .......... .......... 77% 107M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72150K .......... .......... .......... .......... .......... 77% 117M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72200K .......... .......... .......... .......... .......... 77% 108M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72250K .......... .......... .......... .......... .......... 77% 122M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72300K .......... .......... .......... .......... .......... 77% 90.6M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72350K .......... .......... .......... .......... .......... 77% 110M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72400K .......... .......... .......... .......... .......... 77% 112M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72450K .......... .......... .......... .......... .......... 78% 118M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72500K .......... .......... .......... .......... .......... 78% 87.0M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72550K .......... .......... .......... .......... .......... 78% 168M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72600K .......... .......... .......... .......... .......... 78% 111M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72650K .......... .......... .......... .......... .......... 78% 80.9M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72700K .......... .......... .......... .......... .......... 78% 86.6M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72750K .......... .......... .......... .......... .......... 78% 100M 0s 2024-04-22T20:11:27.504 INFO:teuthology.orchestra.run.smithi136.stderr: 72800K .......... .......... .......... .......... .......... 78% 90.7M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 72850K .......... .......... .......... .......... .......... 78% 58.5M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 72900K .......... .......... .......... .......... .......... 78% 137M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 72950K .......... .......... .......... .......... .......... 78% 111M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73000K .......... .......... .......... .......... .......... 78% 113M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73050K .......... .......... .......... .......... .......... 78% 115M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73100K .......... .......... .......... .......... .......... 78% 88.4M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73150K .......... .......... .......... .......... .......... 78% 117M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73200K .......... .......... .......... .......... .......... 78% 97.5M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73250K .......... .......... .......... .......... .......... 78% 120M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73300K .......... .......... .......... .......... .......... 78% 121M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73350K .......... .......... .......... .......... .......... 79% 110M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73400K .......... .......... .......... .......... .......... 79% 110M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73450K .......... .......... .......... .......... .......... 79% 106M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73500K .......... .......... .......... .......... .......... 79% 98.5M 0s 2024-04-22T20:11:27.505 INFO:teuthology.orchestra.run.smithi136.stderr: 73550K .......... .......... .......... .......... .......... 79% 106M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 73600K .......... .......... .......... .......... .......... 79% 111M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 73650K .......... .......... .......... .......... .......... 79% 117M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 73700K .......... .......... .......... .......... .......... 79% 117M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 73750K .......... .......... .......... .......... .......... 79% 94.0M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 73800K .......... .......... .......... .......... .......... 79% 116M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 73850K .......... .......... .......... .......... .......... 79% 111M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 73900K .......... .......... .......... .......... .......... 79% 84.9M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 73950K .......... .......... .......... .......... .......... 79% 116M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 74000K .......... .......... .......... .......... .......... 79% 111M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 74050K .......... .......... .......... .......... .......... 79% 113M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 74100K .......... .......... .......... .......... .......... 79% 110M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 74150K .......... .......... .......... .......... .......... 79% 113M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 74200K .......... .......... .......... .......... .......... 79% 117M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 74250K .......... .......... .......... .......... .......... 79% 114M 0s 2024-04-22T20:11:27.506 INFO:teuthology.orchestra.run.smithi136.stderr: 74300K .......... .......... .......... .......... .......... 80% 88.2M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74350K .......... .......... .......... .......... .......... 80% 64.2M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74400K .......... .......... .......... .......... .......... 80% 370M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74450K .......... .......... .......... .......... .......... 80% 124M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74500K .......... .......... .......... .......... .......... 80% 105M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74550K .......... .......... .......... .......... .......... 80% 114M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74600K .......... .......... .......... .......... .......... 80% 106M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74650K .......... .......... .......... .......... .......... 80% 110M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74700K .......... .......... .......... .......... .......... 80% 95.3M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74750K .......... .......... .......... .......... .......... 80% 121M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74800K .......... .......... .......... .......... .......... 80% 105M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74850K .......... .......... .......... .......... .......... 80% 92.2M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74900K .......... .......... .......... .......... .......... 80% 122M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 74950K .......... .......... .......... .......... .......... 80% 115M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 75000K .......... .......... .......... .......... .......... 80% 104M 0s 2024-04-22T20:11:27.507 INFO:teuthology.orchestra.run.smithi136.stderr: 75050K .......... .......... .......... .......... .......... 80% 107M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75100K .......... .......... .......... .......... .......... 80% 121M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75150K .......... .......... .......... .......... .......... 80% 114M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75200K .......... .......... .......... .......... .......... 81% 116M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75250K .......... .......... .......... .......... .......... 81% 108M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75300K .......... .......... .......... .......... .......... 81% 89.8M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75350K .......... .......... .......... .......... .......... 81% 110M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75400K .......... .......... .......... .......... .......... 81% 123M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75450K .......... .......... .......... .......... .......... 81% 113M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75500K .......... .......... .......... .......... .......... 81% 80.1M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75550K .......... .......... .......... .......... .......... 81% 129M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75600K .......... .......... .......... .......... .......... 81% 115M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75650K .......... .......... .......... .......... .......... 81% 104M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75700K .......... .......... .......... .......... .......... 81% 95.4M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75750K .......... .......... .......... .......... .......... 81% 122M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75800K .......... .......... .......... .......... .......... 81% 118M 0s 2024-04-22T20:11:27.508 INFO:teuthology.orchestra.run.smithi136.stderr: 75850K .......... .......... .......... .......... .......... 81% 61.4M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 75900K .......... .......... .......... .......... .......... 81% 117M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 75950K .......... .......... .......... .......... .......... 81% 109M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76000K .......... .......... .......... .......... .......... 81% 96.0M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76050K .......... .......... .......... .......... .......... 81% 125M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76100K .......... .......... .......... .......... .......... 81% 114M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76150K .......... .......... .......... .......... .......... 82% 110M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76200K .......... .......... .......... .......... .......... 82% 114M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76250K .......... .......... .......... .......... .......... 82% 123M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76300K .......... .......... .......... .......... .......... 82% 73.4M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76350K .......... .......... .......... .......... .......... 82% 151M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76400K .......... .......... .......... .......... .......... 82% 83.9M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76450K .......... .......... .......... .......... .......... 82% 127M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76500K .......... .......... .......... .......... .......... 82% 133M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76550K .......... .......... .......... .......... .......... 82% 114M 0s 2024-04-22T20:11:27.509 INFO:teuthology.orchestra.run.smithi136.stderr: 76600K .......... .......... .......... .......... .......... 82% 109M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 76650K .......... .......... .......... .......... .......... 82% 115M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 76700K .......... .......... .......... .......... .......... 82% 41.0M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 76750K .......... .......... .......... .......... .......... 82% 110M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 76800K .......... .......... .......... .......... .......... 82% 127M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 76850K .......... .......... .......... .......... .......... 82% 115M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 76900K .......... .......... .......... .......... .......... 82% 111M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 76950K .......... .......... .......... .......... .......... 82% 112M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 77000K .......... .......... .......... .......... .......... 82% 103M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 77050K .......... .......... .......... .......... .......... 82% 118M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 77100K .......... .......... .......... .......... .......... 83% 99.7M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 77150K .......... .......... .......... .......... .......... 83% 112M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 77200K .......... .......... .......... .......... .......... 83% 114M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 77250K .......... .......... .......... .......... .......... 83% 108M 0s 2024-04-22T20:11:27.510 INFO:teuthology.orchestra.run.smithi136.stderr: 77300K .......... .......... .......... .......... .......... 83% 112M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77350K .......... .......... .......... .......... .......... 83% 106M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77400K .......... .......... .......... .......... .......... 83% 99.1M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77450K .......... .......... .......... .......... .......... 83% 134M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77500K .......... .......... .......... .......... .......... 83% 93.7M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77550K .......... .......... .......... .......... .......... 83% 116M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77600K .......... .......... .......... .......... .......... 83% 119M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77650K .......... .......... .......... .......... .......... 83% 109M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77700K .......... .......... .......... .......... .......... 83% 87.3M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77750K .......... .......... .......... .......... .......... 83% 117M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77800K .......... .......... .......... .......... .......... 83% 123M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77850K .......... .......... .......... .......... .......... 83% 118M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77900K .......... .......... .......... .......... .......... 83% 84.8M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 77950K .......... .......... .......... .......... .......... 83% 92.4M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 78000K .......... .......... .......... .......... .......... 84% 156M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 78050K .......... .......... .......... .......... .......... 84% 122M 0s 2024-04-22T20:11:27.511 INFO:teuthology.orchestra.run.smithi136.stderr: 78100K .......... .......... .......... .......... .......... 84% 106M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78150K .......... .......... .......... .......... .......... 84% 121M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78200K .......... .......... .......... .......... .......... 84% 115M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78250K .......... .......... .......... .......... .......... 84% 108M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78300K .......... .......... .......... .......... .......... 84% 70.8M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78350K .......... .......... .......... .......... .......... 84% 83.3M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78400K .......... .......... .......... .......... .......... 84% 90.2M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78450K .......... .......... .......... .......... .......... 84% 77.7M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78500K .......... .......... .......... .......... .......... 84% 198M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78550K .......... .......... .......... .......... .......... 84% 115M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78600K .......... .......... .......... .......... .......... 84% 114M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78650K .......... .......... .......... .......... .......... 84% 124M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78700K .......... .......... .......... .......... .......... 84% 86.3M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78750K .......... .......... .......... .......... .......... 84% 117M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78800K .......... .......... .......... .......... .......... 84% 111M 0s 2024-04-22T20:11:27.512 INFO:teuthology.orchestra.run.smithi136.stderr: 78850K .......... .......... .......... .......... .......... 84% 90.0M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 78900K .......... .......... .......... .......... .......... 84% 151M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 78950K .......... .......... .......... .......... .......... 85% 114M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79000K .......... .......... .......... .......... .......... 85% 115M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79050K .......... .......... .......... .......... .......... 85% 119M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79100K .......... .......... .......... .......... .......... 85% 85.9M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79150K .......... .......... .......... .......... .......... 85% 116M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79200K .......... .......... .......... .......... .......... 85% 117M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79250K .......... .......... .......... .......... .......... 85% 106M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79300K .......... .......... .......... .......... .......... 85% 94.1M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79350K .......... .......... .......... .......... .......... 85% 105M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79400K .......... .......... .......... .......... .......... 85% 136M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79450K .......... .......... .......... .......... .......... 85% 132M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79500K .......... .......... .......... .......... .......... 85% 80.1M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79550K .......... .......... .......... .......... .......... 85% 133M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79600K .......... .......... .......... .......... .......... 85% 98.5M 0s 2024-04-22T20:11:27.513 INFO:teuthology.orchestra.run.smithi136.stderr: 79650K .......... .......... .......... .......... .......... 85% 121M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 79700K .......... .......... .......... .......... .......... 85% 103M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 79750K .......... .......... .......... .......... .......... 85% 140M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 79800K .......... .......... .......... .......... .......... 85% 101M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 79850K .......... .......... .......... .......... .......... 86% 117M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 79900K .......... .......... .......... .......... .......... 86% 93.7M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 79950K .......... .......... .......... .......... .......... 86% 103M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 80000K .......... .......... .......... .......... .......... 86% 130M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 80050K .......... .......... .......... .......... .......... 86% 113M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 80100K .......... .......... .......... .......... .......... 86% 110M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 80150K .......... .......... .......... .......... .......... 86% 111M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 80200K .......... .......... .......... .......... .......... 86% 120M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 80250K .......... .......... .......... .......... .......... 86% 118M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 80300K .......... .......... .......... .......... .......... 86% 86.8M 0s 2024-04-22T20:11:27.514 INFO:teuthology.orchestra.run.smithi136.stderr: 80350K .......... .......... .......... .......... .......... 86% 114M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80400K .......... .......... .......... .......... .......... 86% 123M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80450K .......... .......... .......... .......... .......... 86% 89.9M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80500K .......... .......... .......... .......... .......... 86% 146M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80550K .......... .......... .......... .......... .......... 86% 122M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80600K .......... .......... .......... .......... .......... 86% 64.0M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80650K .......... .......... .......... .......... .......... 86% 134M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80700K .......... .......... .......... .......... .......... 86% 125M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80750K .......... .......... .......... .......... .......... 86% 116M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80800K .......... .......... .......... .......... .......... 87% 121M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80850K .......... .......... .......... .......... .......... 87% 99.1M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80900K .......... .......... .......... .......... .......... 87% 132M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 80950K .......... .......... .......... .......... .......... 87% 126M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 81000K .......... .......... .......... .......... .......... 87% 77.8M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 81050K .......... .......... .......... .......... .......... 87% 94.2M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 81100K .......... .......... .......... .......... .......... 87% 94.7M 0s 2024-04-22T20:11:27.515 INFO:teuthology.orchestra.run.smithi136.stderr: 81150K .......... .......... .......... .......... .......... 87% 125M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81200K .......... .......... .......... .......... .......... 87% 122M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81250K .......... .......... .......... .......... .......... 87% 124M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81300K .......... .......... .......... .......... .......... 87% 95.6M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81350K .......... .......... .......... .......... .......... 87% 127M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81400K .......... .......... .......... .......... .......... 87% 88.0M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81450K .......... .......... .......... .......... .......... 87% 110M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81500K .......... .......... .......... .......... .......... 87% 74.4M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81550K .......... .......... .......... .......... .......... 87% 111M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81600K .......... .......... .......... .......... .......... 87% 112M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81650K .......... .......... .......... .......... .......... 87% 95.5M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81700K .......... .......... .......... .......... .......... 87% 61.8M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81750K .......... .......... .......... .......... .......... 88% 106M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81800K .......... .......... .......... .......... .......... 88% 92.3M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81850K .......... .......... .......... .......... .......... 88% 121M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81900K .......... .......... .......... .......... .......... 88% 107M 0s 2024-04-22T20:11:27.516 INFO:teuthology.orchestra.run.smithi136.stderr: 81950K .......... .......... .......... .......... .......... 88% 122M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82000K .......... .......... .......... .......... .......... 88% 105M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82050K .......... .......... .......... .......... .......... 88% 116M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82100K .......... .......... .......... .......... .......... 88% 120M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82150K .......... .......... .......... .......... .......... 88% 114M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82200K .......... .......... .......... .......... .......... 88% 92.2M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82250K .......... .......... .......... .......... .......... 88% 93.3M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82300K .......... .......... .......... .......... .......... 88% 116M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82350K .......... .......... .......... .......... .......... 88% 131M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82400K .......... .......... .......... .......... .......... 88% 85.7M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82450K .......... .......... .......... .......... .......... 88% 136M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82500K .......... .......... .......... .......... .......... 88% 135M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82550K .......... .......... .......... .......... .......... 88% 114M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82600K .......... .......... .......... .......... .......... 88% 83.1M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82650K .......... .......... .......... .......... .......... 89% 131M 0s 2024-04-22T20:11:27.517 INFO:teuthology.orchestra.run.smithi136.stderr: 82700K .......... .......... .......... .......... .......... 89% 106M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 82750K .......... .......... .......... .......... .......... 89% 115M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 82800K .......... .......... .......... .......... .......... 89% 113M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 82850K .......... .......... .......... .......... .......... 89% 88.1M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 82900K .......... .......... .......... .......... .......... 89% 150M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 82950K .......... .......... .......... .......... .......... 89% 113M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83000K .......... .......... .......... .......... .......... 89% 79.9M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83050K .......... .......... .......... .......... .......... 89% 150M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83100K .......... .......... .......... .......... .......... 89% 111M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83150K .......... .......... .......... .......... .......... 89% 118M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83200K .......... .......... .......... .......... .......... 89% 106M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83250K .......... .......... .......... .......... .......... 89% 119M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83300K .......... .......... .......... .......... .......... 89% 111M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83350K .......... .......... .......... .......... .......... 89% 116M 0s 2024-04-22T20:11:27.518 INFO:teuthology.orchestra.run.smithi136.stderr: 83400K .......... .......... .......... .......... .......... 89% 76.9M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83450K .......... .......... .......... .......... .......... 89% 119M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83500K .......... .......... .......... .......... .......... 89% 118M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83550K .......... .......... .......... .......... .......... 89% 111M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83600K .......... .......... .......... .......... .......... 90% 115M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83650K .......... .......... .......... .......... .......... 90% 115M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83700K .......... .......... .......... .......... .......... 90% 82.7M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83750K .......... .......... .......... .......... .......... 90% 134M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83800K .......... .......... .......... .......... .......... 90% 113M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83850K .......... .......... .......... .......... .......... 90% 113M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83900K .......... .......... .......... .......... .......... 90% 116M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 83950K .......... .......... .......... .......... .......... 90% 113M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 84000K .......... .......... .......... .......... .......... 90% 79.5M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 84050K .......... .......... .......... .......... .......... 90% 178M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 84100K .......... .......... .......... .......... .......... 90% 115M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 84150K .......... .......... .......... .......... .......... 90% 116M 0s 2024-04-22T20:11:27.519 INFO:teuthology.orchestra.run.smithi136.stderr: 84200K .......... .......... .......... .......... .......... 90% 93.1M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84250K .......... .......... .......... .......... .......... 90% 109M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84300K .......... .......... .......... .......... .......... 90% 89.2M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84350K .......... .......... .......... .......... .......... 90% 141M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84400K .......... .......... .......... .......... .......... 90% 115M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84450K .......... .......... .......... .......... .......... 90% 92.7M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84500K .......... .......... .......... .......... .......... 91% 89.4M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84550K .......... .......... .......... .......... .......... 91% 111M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84600K .......... .......... .......... .......... .......... 91% 65.7M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84650K .......... .......... .......... .......... .......... 91% 145M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84700K .......... .......... .......... .......... .......... 91% 119M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84750K .......... .......... .......... .......... .......... 91% 120M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84800K .......... .......... .......... .......... .......... 91% 68.0M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84850K .......... .......... .......... .......... .......... 91% 110M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84900K .......... .......... .......... .......... .......... 91% 117M 0s 2024-04-22T20:11:27.520 INFO:teuthology.orchestra.run.smithi136.stderr: 84950K .......... .......... .......... .......... .......... 91% 113M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85000K .......... .......... .......... .......... .......... 91% 87.1M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85050K .......... .......... .......... .......... .......... 91% 120M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85100K .......... .......... .......... .......... .......... 91% 66.5M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85150K .......... .......... .......... .......... .......... 91% 77.3M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85200K .......... .......... .......... .......... .......... 91% 77.5M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85250K .......... .......... .......... .......... .......... 91% 112M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85300K .......... .......... .......... .......... .......... 91% 105M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85350K .......... .......... .......... .......... .......... 91% 119M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85400K .......... .......... .......... .......... .......... 91% 90.0M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85450K .......... .......... .......... .......... .......... 92% 73.9M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85500K .......... .......... .......... .......... .......... 92% 229M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85550K .......... .......... .......... .......... .......... 92% 107M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85600K .......... .......... .......... .......... .......... 92% 107M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85650K .......... .......... .......... .......... .......... 92% 123M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85700K .......... .......... .......... .......... .......... 92% 106M 0s 2024-04-22T20:11:27.521 INFO:teuthology.orchestra.run.smithi136.stderr: 85750K .......... .......... .......... .......... .......... 92% 99.5M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 85800K .......... .......... .......... .......... .......... 92% 97.6M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 85850K .......... .......... .......... .......... .......... 92% 118M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 85900K .......... .......... .......... .......... .......... 92% 109M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 85950K .......... .......... .......... .......... .......... 92% 115M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86000K .......... .......... .......... .......... .......... 92% 110M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86050K .......... .......... .......... .......... .......... 92% 119M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86100K .......... .......... .......... .......... .......... 92% 111M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86150K .......... .......... .......... .......... .......... 92% 109M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86200K .......... .......... .......... .......... .......... 92% 91.6M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86250K .......... .......... .......... .......... .......... 92% 115M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86300K .......... .......... .......... .......... .......... 92% 116M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86350K .......... .......... .......... .......... .......... 93% 107M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86400K .......... .......... .......... .......... .......... 93% 115M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86450K .......... .......... .......... .......... .......... 93% 92.8M 0s 2024-04-22T20:11:27.522 INFO:teuthology.orchestra.run.smithi136.stderr: 86500K .......... .......... .......... .......... .......... 93% 130M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86550K .......... .......... .......... .......... .......... 93% 109M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86600K .......... .......... .......... .......... .......... 93% 82.0M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86650K .......... .......... .......... .......... .......... 93% 117M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86700K .......... .......... .......... .......... .......... 93% 101M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86750K .......... .......... .......... .......... .......... 93% 143M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86800K .......... .......... .......... .......... .......... 93% 110M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86850K .......... .......... .......... .......... .......... 93% 117M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86900K .......... .......... .......... .......... .......... 93% 107M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 86950K .......... .......... .......... .......... .......... 93% 106M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 87000K .......... .......... .......... .......... .......... 93% 84.6M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 87050K .......... .......... .......... .......... .......... 93% 134M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 87100K .......... .......... .......... .......... .......... 93% 118M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 87150K .......... .......... .......... .......... .......... 93% 114M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 87200K .......... .......... .......... .......... .......... 93% 103M 0s 2024-04-22T20:11:27.523 INFO:teuthology.orchestra.run.smithi136.stderr: 87250K .......... .......... .......... .......... .......... 93% 127M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87300K .......... .......... .......... .......... .......... 94% 94.3M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87350K .......... .......... .......... .......... .......... 94% 126M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87400K .......... .......... .......... .......... .......... 94% 75.6M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87450K .......... .......... .......... .......... .......... 94% 132M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87500K .......... .......... .......... .......... .......... 94% 126M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87550K .......... .......... .......... .......... .......... 94% 117M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87600K .......... .......... .......... .......... .......... 94% 91.6M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87650K .......... .......... .......... .......... .......... 94% 136M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87700K .......... .......... .......... .......... .......... 94% 129M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87750K .......... .......... .......... .......... .......... 94% 83.4M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87800K .......... .......... .......... .......... .......... 94% 84.7M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87850K .......... .......... .......... .......... .......... 94% 67.8M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87900K .......... .......... .......... .......... .......... 94% 80.9M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 87950K .......... .......... .......... .......... .......... 94% 163M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 88000K .......... .......... .......... .......... .......... 94% 93.1M 0s 2024-04-22T20:11:27.524 INFO:teuthology.orchestra.run.smithi136.stderr: 88050K .......... .......... .......... .......... .......... 94% 123M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88100K .......... .......... .......... .......... .......... 94% 134M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88150K .......... .......... .......... .......... .......... 94% 119M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88200K .......... .......... .......... .......... .......... 94% 86.9M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88250K .......... .......... .......... .......... .......... 95% 86.6M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88300K .......... .......... .......... .......... .......... 95% 151M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88350K .......... .......... .......... .......... .......... 95% 114M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88400K .......... .......... .......... .......... .......... 95% 111M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88450K .......... .......... .......... .......... .......... 95% 114M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88500K .......... .......... .......... .......... .......... 95% 108M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88550K .......... .......... .......... .......... .......... 95% 40.1M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88600K .......... .......... .......... .......... .......... 95% 51.9M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88650K .......... .......... .......... .......... .......... 95% 123M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88700K .......... .......... .......... .......... .......... 95% 60.3M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88750K .......... .......... .......... .......... .......... 95% 138M 0s 2024-04-22T20:11:27.525 INFO:teuthology.orchestra.run.smithi136.stderr: 88800K .......... .......... .......... .......... .......... 95% 112M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 88850K .......... .......... .......... .......... .......... 95% 114M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 88900K .......... .......... .......... .......... .......... 95% 114M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 88950K .......... .......... .......... .......... .......... 95% 112M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89000K .......... .......... .......... .......... .......... 95% 86.0M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89050K .......... .......... .......... .......... .......... 95% 120M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89100K .......... .......... .......... .......... .......... 95% 114M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89150K .......... .......... .......... .......... .......... 96% 109M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89200K .......... .......... .......... .......... .......... 96% 110M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89250K .......... .......... .......... .......... .......... 96% 101M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89300K .......... .......... .......... .......... .......... 96% 115M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89350K .......... .......... .......... .......... .......... 96% 106M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89400K .......... .......... .......... .......... .......... 96% 106M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89450K .......... .......... .......... .......... .......... 96% 72.1M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89500K .......... .......... .......... .......... .......... 96% 171M 0s 2024-04-22T20:11:27.526 INFO:teuthology.orchestra.run.smithi136.stderr: 89550K .......... .......... .......... .......... .......... 96% 111M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 89600K .......... .......... .......... .......... .......... 96% 107M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 89650K .......... .......... .......... .......... .......... 96% 160M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 89700K .......... .......... .......... .......... .......... 96% 112M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 89750K .......... .......... .......... .......... .......... 96% 119M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 89800K .......... .......... .......... .......... .......... 96% 87.8M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 89850K .......... .......... .......... .......... .......... 96% 110M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 89900K .......... .......... .......... .......... .......... 96% 115M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 89950K .......... .......... .......... .......... .......... 96% 104M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 90000K .......... .......... .......... .......... .......... 96% 114M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 90050K .......... .......... .......... .......... .......... 96% 114M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 90100K .......... .......... .......... .......... .......... 97% 115M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 90150K .......... .......... .......... .......... .......... 97% 118M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 90200K .......... .......... .......... .......... .......... 97% 111M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 90250K .......... .......... .......... .......... .......... 97% 111M 0s 2024-04-22T20:11:27.527 INFO:teuthology.orchestra.run.smithi136.stderr: 90300K .......... .......... .......... .......... .......... 97% 94.7M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90350K .......... .......... .......... .......... .......... 97% 81.4M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90400K .......... .......... .......... .......... .......... 97% 180M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90450K .......... .......... .......... .......... .......... 97% 110M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90500K .......... .......... .......... .......... .......... 97% 117M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90550K .......... .......... .......... .......... .......... 97% 106M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90600K .......... .......... .......... .......... .......... 97% 109M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90650K .......... .......... .......... .......... .......... 97% 124M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90700K .......... .......... .......... .......... .......... 97% 91.4M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90750K .......... .......... .......... .......... .......... 97% 118M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90800K .......... .......... .......... .......... .......... 97% 106M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90850K .......... .......... .......... .......... .......... 97% 121M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90900K .......... .......... .......... .......... .......... 97% 109M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 90950K .......... .......... .......... .......... .......... 97% 113M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 91000K .......... .......... .......... .......... .......... 98% 109M 0s 2024-04-22T20:11:27.528 INFO:teuthology.orchestra.run.smithi136.stderr: 91050K .......... .......... .......... .......... .......... 98% 121M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91100K .......... .......... .......... .......... .......... 98% 65.9M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91150K .......... .......... .......... .......... .......... 98% 93.5M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91200K .......... .......... .......... .......... .......... 98% 85.9M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91250K .......... .......... .......... .......... .......... 98% 163M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91300K .......... .......... .......... .......... .......... 98% 46.4M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91350K .......... .......... .......... .......... .......... 98% 121M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91400K .......... .......... .......... .......... .......... 98% 113M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91450K .......... .......... .......... .......... .......... 98% 115M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91500K .......... .......... .......... .......... .......... 98% 76.6M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91550K .......... .......... .......... .......... .......... 98% 109M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91600K .......... .......... .......... .......... .......... 98% 137M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91650K .......... .......... .......... .......... .......... 98% 96.6M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91700K .......... .......... .......... .......... .......... 98% 113M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91750K .......... .......... .......... .......... .......... 98% 138M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91800K .......... .......... .......... .......... .......... 98% 92.8M 0s 2024-04-22T20:11:27.529 INFO:teuthology.orchestra.run.smithi136.stderr: 91850K .......... .......... .......... .......... .......... 98% 129M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 91900K .......... .......... .......... .......... .......... 98% 95.6M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 91950K .......... .......... .......... .......... .......... 99% 25.5M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92000K .......... .......... .......... .......... .......... 99% 116M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92050K .......... .......... .......... .......... .......... 99% 80.3M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92100K .......... .......... .......... .......... .......... 99% 59.9M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92150K .......... .......... .......... .......... .......... 99% 115M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92200K .......... .......... .......... .......... .......... 99% 79.0M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92250K .......... .......... .......... .......... .......... 99% 57.5M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92300K .......... .......... .......... .......... .......... 99% 85.2M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92350K .......... .......... .......... .......... .......... 99% 119M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92400K .......... .......... .......... .......... .......... 99% 84.5M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92450K .......... .......... .......... .......... .......... 99% 125M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92500K .......... .......... .......... .......... .......... 99% 128M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92550K .......... .......... .......... .......... .......... 99% 118M 0s 2024-04-22T20:11:27.530 INFO:teuthology.orchestra.run.smithi136.stderr: 92600K .......... .......... .......... .......... .......... 99% 115M 0s 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr: 92650K .......... .......... .......... .......... .......... 99% 118M 0s 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr: 92700K .......... .......... .......... .......... .......... 99% 78.1M 0s 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr: 92750K .......... .......... .......... .......... .......... 99% 131M 0s 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr: 92800K .......... .......... .......... .......... .......... 99% 107M 0s 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr: 92850K .......... .......... .......... .......... .......... 99% 148M 0s 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr: 92900K 100% 134G=0.9s 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22 20:11:26 (96.0 MB/s) - ‘gradle-6.0.1-bin.zip’ saved [95129672/95129672] 2024-04-22T20:11:27.531 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-22T20:11:27.562 INFO:teuthology.orchestra.run.smithi136.stdout:Archive: gradle-6.0.1-bin.zip 2024-04-22T20:11:27.562 INFO:teuthology.orchestra.run.smithi136.stdout: creating: /opt/gradle/gradle-6.0.1/ 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/getting-started.html 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/LICENSE 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/NOTICE 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: creating: /opt/gradle/gradle-6.0.1/init.d/ 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/init.d/readme.txt 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: creating: /opt/gradle/gradle-6.0.1/bin/ 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle.bat 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: creating: /opt/gradle/gradle-6.0.1/lib/ 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-api-metadata-6.0.1.jar 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-launcher-6.0.1.jar 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-runtime-api-info-6.0.1.jar 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-6.0.1.jar 2024-04-22T20:11:27.563 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-tooling-api-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-wrapper-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-installation-beacon-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-bootstrap-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-file-collections-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-groovy-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-execution-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-core-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-packaging-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-api-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-persistent-cache-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-worker-processes-6.0.1.jar 2024-04-22T20:11:27.564 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-logging-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-jvm-services-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-process-services-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-resources-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-messaging-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-native-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-groovy-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-option-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-cli-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-snapshots-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-files-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-tooling-models-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-hashing-6.0.1.jar 2024-04-22T20:11:27.565 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-pineapple-6.0.1.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-docs-6.0.1.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/groovy-all-1.3-2.5.8.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jul-to-slf4j-1.7.28.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/log4j-over-slf4j-1.7.28.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jcl-over-slf4j-1.7.28.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/slf4j-api-1.7.28.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/guava-27.1-android.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-io-2.6.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-lang-2.6.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-commons-7.1.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-analysis-7.1.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-tree-7.1.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-7.1.jar 2024-04-22T20:11:27.566 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-1.10.7.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk8-1.3.50.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/javax.inject-1.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-common-1.3.50.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-1.3.50.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-host-embeddable-1.3.50.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-embeddable-1.3.50.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-sam-with-receiver-compiler-plugin-1.3.50.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlinx-metadata-jvm-0.1.0.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-compress-1.19.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/fastutil-8.3.0-min.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jansi-1.18.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jsr305-3.0.2.jar 2024-04-22T20:11:27.567 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kryo-2.24.0.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-0.18.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/objenesis-2.6.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/trove4j-1.0.20181211.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/xml-apis-1.4.01.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-reflect-1.3.50.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk7-1.3.50.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-1.3.50.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/annotations-13.0.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-launcher-1.10.7.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/minlog-1.2.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-script-runtime-1.3.50.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-daemon-embeddable-1.3.50.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/failureaccess-1.0.1.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-common-1.3.50.jar 2024-04-22T20:11:27.568 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libcpp-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libstdcpp-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libcpp-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libstdcpp-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-osx-amd64-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-min-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-min-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-ncurses5-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses5-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses6-0.18.jar 2024-04-22T20:11:27.569 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses5-0.18.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses6-0.18.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: creating: /opt/gradle/gradle-6.0.1/lib/plugins/ 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-antlr-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-code-quality-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-signing-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-native-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-native-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-native-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-native-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-provider-plugins-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-development-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-maven-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-play-6.0.1.jar 2024-04-22T20:11:27.570 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-scala-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ear-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-play-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-javascript-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ivy-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-jacoco-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-init-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-tooling-builders-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugins-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-test-kit-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-groovy-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-scala-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-junit-platform-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-api-builders-6.0.1.jar 2024-04-22T20:11:27.571 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-jvm-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-java-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-jvm-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-jvm-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-native-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-diagnostics-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-publish-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-base-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-base-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-use-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-instant-execution-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-version-control-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-dependency-management-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-composite-builds-6.0.1.jar 2024-04-22T20:11:27.572 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-reporting-6.0.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-s3-6.0.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-gcs-6.0.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-cache-http-6.0.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-http-6.0.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-sftp-6.0.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-profile-6.0.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-workers-6.0.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/org.eclipse.jgit-5.5.0.201909110433-r.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-3.0.0.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-shared-3.0.0.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-services-storage-v1-rev136-1.25.0.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-client-1.25.0.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/asm-util-7.1.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ivy-2.3.0.jar 2024-04-22T20:11:27.573 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-compat-3.0.4.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-core-3.0.4.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-common-0.8-20100325.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-groovy-0.8-20100325.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-file-3.0.0.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-container-default-1.7.1.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-connector-wagon-1.13.1.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpg-jdk15on-1.63.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/rhino-1.7.10.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gson-2.8.5.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/simple-4.1.21.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jatl-0.2.3.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-plugin-api-3.0.4.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-oauth-client-1.25.0.jar 2024-04-22T20:11:27.574 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-jackson2-1.25.0.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-1.25.0.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/testng-6.3.1.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/snakeyaml-1.17.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/dd-plist-1.21.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-4.12.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bsh-2.0b6.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-launcher-1.3.1.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-s3-1.11.633.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-kms-1.11.633.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-core-1.11.633.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpclient-4.5.10.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcifs-1.3.17.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/nekohtml-1.9.22.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xercesImpl-2.12.0.jar 2024-04-22T20:11:27.575 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jsch-0.1.55.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jaxb-impl-2.3.2.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jmespath-java-1.11.633.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-databind-2.9.10.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-core-2.9.10.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-annotations-2.9.10.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/joda-time-2.10.4.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpkix-jdk15on-1.61.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcprov-jdk15on-1.63.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/commons-codec-1.13.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-aether-provider-3.0.4.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-impl-1.13.1.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-util-1.13.1.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-spi-1.13.1.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-api-1.13.1.jar 2024-04-22T20:11:27.576 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/hamcrest-core-1.3.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpcore-4.4.12.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcommander-1.72.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jzlib-1.1.3.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-artifact-3.0.4.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-builder-3.0.4.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-3.0.4.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-repository-metadata-3.0.4.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-builder-3.0.4.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-3.0.4.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-sec-dispatcher-1.3.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-cipher-1.7.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-classworlds-2.5.1.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-component-annotations-1.5.5.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-interpolation-1.14.jar 2024-04-22T20:11:27.577 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-provider-api-3.0.0.jar 2024-04-22T20:11:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-utils-3.1.0.jar 2024-04-22T20:11:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xbean-reflect-3.7.jar 2024-04-22T20:11:27.578 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ion-java-1.0.2.jar 2024-04-22T20:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-engine-1.3.1.jar 2024-04-22T20:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-commons-1.3.1.jar 2024-04-22T20:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/apiguardian-api-1.0.0.jar 2024-04-22T20:11:27.579 INFO:teuthology.orchestra.run.smithi136.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/opentest4j-1.1.1.jar 2024-04-22T20:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout:export PATH=/opt/gradle/gradle-6.0.1/bin:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout:Welcome to Gradle 6.0.1! 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout:Here are the highlights of this release: 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: - Substantial improvements in dependency management, including 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: - Publishing Gradle Module Metadata in addition to pom.xml 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: - Advanced control of transitive versions 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: - Support for optional features and dependencies 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: - Rules to tweak published metadata 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: - Support for Java 13 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: - Faster incremental Java and Groovy compilation 2024-04-22T20:11:27.930 INFO:teuthology.orchestra.run.smithi136.stdout: - New Zinc compiler for Scala 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: - VS2019 support 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: - Support for Gradle Enterprise plugin 3.0 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout:For more details see https://docs.gradle.org/6.0.1/release-notes.html 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout:------------------------------------------------------------ 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout:Gradle 6.0.1 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout:------------------------------------------------------------ 2024-04-22T20:11:27.931 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:27.932 INFO:teuthology.orchestra.run.smithi136.stdout:Build time: 2019-11-18 20:25:01 UTC 2024-04-22T20:11:27.933 INFO:teuthology.orchestra.run.smithi136.stdout:Revision: fad121066a68c4701acd362daf4287a7c309a0f5 2024-04-22T20:11:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:27.933 INFO:teuthology.orchestra.run.smithi136.stdout:Kotlin: 1.3.50 2024-04-22T20:11:27.933 INFO:teuthology.orchestra.run.smithi136.stdout:Groovy: 2.5.8 2024-04-22T20:11:27.933 INFO:teuthology.orchestra.run.smithi136.stdout:Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 2024-04-22T20:11:27.933 INFO:teuthology.orchestra.run.smithi136.stdout:JVM: 11.0.18 (Red Hat, Inc. 11.0.18+10-LTS) 2024-04-22T20:11:27.933 INFO:teuthology.orchestra.run.smithi136.stdout:OS: Linux 5.14.0-437.el9.x86_64 amd64 2024-04-22T20:11:27.933 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:27.936 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2024-04-22T20:11:27.936 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-22T20:11:27.936 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T20:11:27.936 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-22T20:11:27.937 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T20:11:27.937 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2024-04-22T20:11:28.533 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 'access_secret': None, 'region': 'mexico', 'endpoint': None, 'port': None, 'display_name': None, 'email': None, 'is_secure': None, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data'}} 2024-04-22T20:11:28.533 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2024-04-22T20:11:28.534 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'CMFMMIFZSZAWERQOBMJF', 'access_secret': 'cdjoB+87zCVtthpGODbbwEPyXGYitPwRQD/oT7CfPofGhOOSwiQ3KA==', 'region': 'mexico', 'endpoint': 'smithi136.front.sepia.ceph.com:443', 'port': 443, 'display_name': 'Ms. tester.client.0', 'email': 'tester.client.0_test@test.test', 'is_secure': True, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data', 'user_id': 'tester.client.0', 'host': 'smithi136.front.sepia.ceph.com'} 2024-04-22T20:11:28.534 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2024-04-22T20:11:28.534 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'CMFMMIFZSZAWERQOBMJF', '--secret', 'cdjoB+87zCVtthpGODbbwEPyXGYitPwRQD/oT7CfPofGhOOSwiQ3KA==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2024-04-22T20:11:28.534 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key CMFMMIFZSZAWERQOBMJF --secret cdjoB+87zCVtthpGODbbwEPyXGYitPwRQD/oT7CfPofGhOOSwiQ3KA== --email tester.client.0_test@test.test --cluster ceph 2024-04-22T20:11:28.596 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:11:28.596 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:11:28.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.629+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:28.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.630+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:28.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.630+0000 7ffb9fb2d800 20 realm 2024-04-22T20:11:28.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.630+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:28.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.631+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:28.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.631+0000 7ffb9fb2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:11:28.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.631+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:28.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.631+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:28.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.631+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:28.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.632+0000 7ffb9fb2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:11:28.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.632+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:28.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.632+0000 7ffb9fb2d800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:11:28.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.632+0000 7ffb9fb2d800 20 searching for the correct realm 2024-04-22T20:11:28.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.635+0000 7ffb9bf80640 0 -- 172.21.15.136:0/3462178353 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7ffb94031070 msgr2=0x7ffb94061b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:11:28.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:28.841+0000 7ffb9af7e640 0 -- 172.21.15.136:0/3462178353 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7ffb940620d0 msgr2=0x7ffb94062530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:11:29.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:11:29.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:11:29.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:11:29.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:11:29.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:11:29.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:11:29.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:11:29.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.048+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:11:29.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 zone default found 2024-04-22T20:11:29.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 4 Realm: () 2024-04-22T20:11:29.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:11:29.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 zonegroup default 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 started sync module instance, tier type = 2024-04-22T20:11:29.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.049+0000 7ffb9fb2d800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:11:29.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.059+0000 7ffb9fb2d800 20 add_watcher() i=0 2024-04-22T20:11:29.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.059+0000 7ffb9fb2d800 20 add_watcher() i=1 2024-04-22T20:11:29.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.059+0000 7ffb9fb2d800 20 add_watcher() i=2 2024-04-22T20:11:29.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.059+0000 7ffb9fb2d800 20 add_watcher() i=3 2024-04-22T20:11:29.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.059+0000 7ffb9fb2d800 20 add_watcher() i=4 2024-04-22T20:11:29.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.059+0000 7ffb9fb2d800 20 add_watcher() i=5 2024-04-22T20:11:29.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.059+0000 7ffb9fb2d800 20 add_watcher() i=6 2024-04-22T20:11:29.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.060+0000 7ffb9fb2d800 20 add_watcher() i=7 2024-04-22T20:11:29.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.060+0000 7ffb9fb2d800 2 all 8 watchers are set, enabling cache 2024-04-22T20:11:29.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.062+0000 7ffa51fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:11:29.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.062+0000 7ffb9fb2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:11:29.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.062+0000 7ffb9fb2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:11:29.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.062+0000 7ffb9fb2d800 5 note: GC not initialized 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.062+0000 7ffa50fe1640 20 reqs_thread_entry: start 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 20 init_complete bucket index max shards: 1 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 20 Filter name: none 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ff9ca7fc640 20 reqs_thread_entry: start 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 10 cache get: name=default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF : miss 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.063+0000 7ffb9fb2d800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.064+0000 7ffb9fb2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.064+0000 7ffb9fb2d800 10 cache put: name=default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF info.flags=0x0 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.064+0000 7ffb9fb2d800 10 adding default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF to cache LRU end 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.064+0000 7ffb9fb2d800 10 cache get: name=default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF : hit (negative entry) 2024-04-22T20:11:29.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.064+0000 7ffb9fb2d800 10 cache get: name=default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF : hit (negative entry) 2024-04-22T20:11:29.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.065+0000 7ffb9fb2d800 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-04-22T20:11:29.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.065+0000 7ffb9fb2d800 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-04-22T20:11:29.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.065+0000 7ffb9fb2d800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-04-22T20:11:29.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.065+0000 7ffa7effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562399 cookie 94643337540896 notifier 7407 bl.length()=670 2024-04-22T20:11:29.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.065+0000 7ffa7effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-04-22T20:11:29.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.065+0000 7ffa7effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-04-22T20:11:29.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.070+0000 7ffb9fb2d800 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-04-22T20:11:29.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.070+0000 7ffb9fb2d800 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-04-22T20:11:29.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.070+0000 7ffb9fb2d800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-04-22T20:11:29.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.071+0000 7ffa7e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562393 cookie 94643337521552 notifier 7407 bl.length()=200 2024-04-22T20:11:29.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.071+0000 7ffa7e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-04-22T20:11:29.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.071+0000 7ffa7e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-04-22T20:11:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.073+0000 7ffb9fb2d800 10 cache put: name=default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF info.flags=0x7 2024-04-22T20:11:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.073+0000 7ffb9fb2d800 10 moving default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF to cache LRU end 2024-04-22T20:11:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.073+0000 7ffb9fb2d800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:CMFMMIFZSZAWERQOBMJF, ofs0, ns] 2024-04-22T20:11:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.073+0000 7ffa7e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562394 cookie 94643337521552 notifier 7407 bl.length()=189 2024-04-22T20:11:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.073+0000 7ffa7e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF info.flags=0x7 2024-04-22T20:11:29.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.073+0000 7ffa7e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+CMFMMIFZSZAWERQOBMJF to cache LRU end 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "tester.client.0", 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Ms. tester.client.0", 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "tester.client.0_test@test.test", 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:11:29.074 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "tester.client.0", 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "CMFMMIFZSZAWERQOBMJF", 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "cdjoB+87zCVtthpGODbbwEPyXGYitPwRQD/oT7CfPofGhOOSwiQ3KA==", 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:11:29.064604Z" 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:11:29.075 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "rgw", 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:11:29.076 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "", 2024-04-22T20:11:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:11:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:11:29.064591Z", 2024-04-22T20:11:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:11:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:11:29.077 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:11:29.077 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:29.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.075+0000 7ffa51fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:11:29.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.078+0000 7ffb9fb2d800 20 remove_watcher() i=0 2024-04-22T20:11:29.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.078+0000 7ffb9fb2d800 2 removed watcher, disabling cache 2024-04-22T20:11:29.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.079+0000 7ffb9fb2d800 20 remove_watcher() i=1 2024-04-22T20:11:29.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.080+0000 7ffb9fb2d800 20 remove_watcher() i=2 2024-04-22T20:11:29.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.081+0000 7ffb9fb2d800 20 remove_watcher() i=3 2024-04-22T20:11:29.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.082+0000 7ffb9fb2d800 20 remove_watcher() i=4 2024-04-22T20:11:29.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.083+0000 7ffb9fb2d800 20 remove_watcher() i=5 2024-04-22T20:11:29.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.084+0000 7ffb9fb2d800 20 remove_watcher() i=6 2024-04-22T20:11:29.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.085+0000 7ffb9fb2d800 20 remove_watcher() i=7 2024-04-22T20:11:29.097 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T20:11:29.097 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2024-04-22T20:11:29.127 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2024-04-22T20:11:29.127 DEBUG:teuthology.orchestra.run.smithi136:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2024-04-22T20:11:29.193 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:11:29.282 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:11:29.282 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:11:29.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.318+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.319+0000 7fe99da87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.319+0000 7fe99da87800 20 realm 2024-04-22T20:11:29.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.319+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.319+0000 7fe99da87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.319+0000 7fe99da87800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:11:29.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.319+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.319+0000 7fe99da87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.319+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.320+0000 7fe99da87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:11:29.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.320+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.321+0000 7fe99da87800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:11:29.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.321+0000 7fe99da87800 20 searching for the correct realm 2024-04-22T20:11:29.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.321+0000 7fe99b586640 0 -- 172.21.15.136:0/2373980078 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5606af8b17b0 msgr2=0x5606af8b3c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:11:29.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.325+0000 7fe99b586640 0 -- 172.21.15.136:0/2373980078 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x5606af66e130 msgr2=0x7fe99409ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.534+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:11:29.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:11:29.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 20 zone default found 2024-04-22T20:11:29.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 4 Realm: () 2024-04-22T20:11:29.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:11:29.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:11:29.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:11:29.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 20 zonegroup default 2024-04-22T20:11:29.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.535+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.536+0000 7fe99da87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.536+0000 7fe99da87800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.536+0000 7fe99da87800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.536+0000 7fe99da87800 20 started sync module instance, tier type = 2024-04-22T20:11:29.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.536+0000 7fe99da87800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:11:29.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 20 add_watcher() i=0 2024-04-22T20:11:29.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 20 add_watcher() i=1 2024-04-22T20:11:29.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 20 add_watcher() i=2 2024-04-22T20:11:29.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 20 add_watcher() i=3 2024-04-22T20:11:29.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 20 add_watcher() i=4 2024-04-22T20:11:29.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 20 add_watcher() i=5 2024-04-22T20:11:29.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 20 add_watcher() i=6 2024-04-22T20:11:29.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 20 add_watcher() i=7 2024-04-22T20:11:29.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.546+0000 7fe99da87800 2 all 8 watchers are set, enabling cache 2024-04-22T20:11:29.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.548+0000 7fe851fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:11:29.548 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.549+0000 7fe99da87800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:11:29.548 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.549+0000 7fe99da87800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:11:29.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.585+0000 7fe850fe1640 20 reqs_thread_entry: start 2024-04-22T20:11:29.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.585+0000 7fe99da87800 20 init_complete bucket index max shards: 1 2024-04-22T20:11:29.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.585+0000 7fe99da87800 20 Filter name: none 2024-04-22T20:11:29.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.585+0000 7fe7ca7fc640 20 reqs_thread_entry: start 2024-04-22T20:11:29.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.585+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:11:29.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.587+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.587+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.587+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.587+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5739523068998276773', time=2024-04-22T22:06:19.869737+0000, chain.objs.size()=1 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8__shadow_.oMYKmlV3J3w8kgf6x2jb6iW79FqnSfq_0 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9107094611199885695', time=2024-04-22T22:08:11.954637+0000, chain.objs.size()=1 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.vR-Zgb0Fuy5QuSu0drZcEC6TwpBr4mC_0 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2964716211257460922', time=2024-04-22T22:08:47.175085+0000, chain.objs.size()=1 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.t9Ag-pKsAqBHCpLU1VOnfuUq4ygMPpy_0 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7964595161980244258', time=2024-04-22T22:08:47.233221+0000, chain.objs.size()=1 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.lpvNm4crrcmVj70KhRNkcXi9vqnNAXw_0 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1170281151476708835', time=2024-04-22T22:09:24.723316+0000, chain.objs.size()=1 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.4BXoHPtjMD7aNiImE1__t1GjPntBE6T_0 2024-04-22T20:11:29.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3473323058810628944', time=2024-04-22T22:09:24.800572+0000, chain.objs.size()=1 2024-04-22T20:11:29.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.IEpNQirvQAqdmdCIOVRMR6na_aS9egM_0 2024-04-22T20:11:29.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7169849948793204183', time=2024-04-22T22:10:07.722005+0000, chain.objs.size()=1 2024-04-22T20:11:29.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.bSmta_HEMr8LWNoKbIKk5RcNAFVP1YK_0 2024-04-22T20:11:29.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11601370424559973483', time=2024-04-22T22:10:07.815639+0000, chain.objs.size()=1 2024-04-22T20:11:29.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.588+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.1GthqyPUsKTHfUr_8k8yfI7Swj8Dfe-_0 2024-04-22T20:11:29.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.591+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.595+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:11:29.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-04-22T20:11:29.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4464368107970261222', time=2024-04-22T22:06:19.695289+0000, chain.objs.size()=5 2024-04-22T20:11:29.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1 2024-04-22T20:11:29.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.2 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.2_1 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.3 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_WqZ4Z-g4nmIIq5kxPHNKvdcggYXBkRo', time=2024-04-22T22:06:19.706706+0000, chain.objs.size()=3 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.9ESM-r6fMpvpG23M5kYZLhyRx602i2o_0 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.9ESM-r6fMpvpG23M5kYZLhyRx602i2o_1 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.9ESM-r6fMpvpG23M5kYZLhyRx602i2o_2 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4166639611523310562', time=2024-04-22T22:08:47.176591+0000, chain.objs.size()=1 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.pgh6iI4E4dh6iNZeBXKgsaPEZjECiSm_0 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3123998119471614822', time=2024-04-22T22:08:47.177348+0000, chain.objs.size()=1 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.cSXTqTzyr8QB66aAA7DOgcyQ7UUGzTE_0 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8808699260027216339', time=2024-04-22T22:08:50.990306+0000, chain.objs.size()=1 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.7-QI5ypnBgeDGeNtv679GgqzmNfjV1O_0 2024-04-22T20:11:29.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3070393823785874596', time=2024-04-22T22:09:24.765102+0000, chain.objs.size()=1 2024-04-22T20:11:29.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.597+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.C2pm6dmKUhqYmfgk9tOdph8MaMAh18w_0 2024-04-22T20:11:29.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.599+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4771134534021770480', time=2024-04-22T22:09:24.815775+0000, chain.objs.size()=1 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.599+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.m3G5Fl--zYk4GBxxJoj0W0I6QMSFbTe_0 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.599+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.648563066327011481', time=2024-04-22T22:09:28.976833+0000, chain.objs.size()=1 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.599+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.eHRi1OU6j002lY_5IlUe7EOaEPyvATc_0 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.600+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.265859956945058627', time=2024-04-22T22:10:04.777988+0000, chain.objs.size()=1 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.600+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.HiUPqYxhEMfxUR_KuZ1YgNTHKVmPNIM_0 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.600+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18105363524532089755', time=2024-04-22T22:10:07.785404+0000, chain.objs.size()=1 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.600+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.YzurhKtu0fsrGkKWXHDPM9va-4NiMun_0 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.600+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5320890541855066309', time=2024-04-22T22:10:07.786021+0000, chain.objs.size()=1 2024-04-22T20:11:29.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.601+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.JJMVf5YwOGmX2sxNF4Ithqi8u61Pj2j_0 2024-04-22T20:11:29.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.604+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.606+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:11:29.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.607+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-22T20:11:29.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5138276915250340076', time=2024-04-22T22:06:19.666635+0000, chain.objs.size()=1 2024-04-22T20:11:29.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_0 2024-04-22T20:11:29.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_yIwytN-8WsngtTzPsXRyGdEzEw3Vugx', time=2024-04-22T22:06:19.722662+0000, chain.objs.size()=3 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.HPk3nza-Vr8B2GmMK5-QJguftEJzdxi_0 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.HPk3nza-Vr8B2GmMK5-QJguftEJzdxi_1 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.HPk3nza-Vr8B2GmMK5-QJguftEJzdxi_2 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14237087433929750989', time=2024-04-22T22:08:12.049052+0000, chain.objs.size()=1 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.ENzDkcVNa2kSO6p9kBRLbIR0lvwHPOZ_0 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5787976949922237389', time=2024-04-22T22:09:28.935668+0000, chain.objs.size()=1 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.V2EAYB7C3uHTU1uozJQzVbKetUXJ_Dp_0 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14877371042042248455', time=2024-04-22T22:09:29.021183+0000, chain.objs.size()=1 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.n8dr9ocf22drpsBfZWu47Rjit81lGCB_0 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14539811948445878759', time=2024-04-22T22:09:29.026957+0000, chain.objs.size()=1 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.Rsgu8-Z1P-v1kVA1txLbGusfLqV5lFG_0 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16079138426400170839', time=2024-04-22T22:09:31.056546+0000, chain.objs.size()=1 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.C3JLpmgWbYXTPiSAyJGB-HJk3ih4XxH_0 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17191672566984494141', time=2024-04-22T22:10:07.829950+0000, chain.objs.size()=1 2024-04-22T20:11:29.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.608+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.nAKoJ6R_tce7t97h2fNKnu-WUnlDCGy_0 2024-04-22T20:11:29.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.612+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.615+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:11:29.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.616+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2545840975760552851', time=2024-04-22T22:08:14.316663+0000, chain.objs.size()=1 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.02qxEySBd5EccXizQTz6b3w0L8RYm8l_0 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16317528547546481699', time=2024-04-22T22:09:24.779452+0000, chain.objs.size()=1 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.zzcTzkqGHlj7WREtGKdGGse2XjpqzwS_0 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7682778945953351121', time=2024-04-22T22:09:29.036900+0000, chain.objs.size()=1 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.YX_U4ihEjMQ0aiTrUHyQAQozdP32xJm_0 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13226572769849431403', time=2024-04-22T22:09:31.052195+0000, chain.objs.size()=1 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.S9GqSuGtQVwE6fj_evQRZUJhAQUsmbd_0 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8555130120343074771', time=2024-04-22T22:10:41.471491+0000, chain.objs.size()=1 2024-04-22T20:11:29.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.617+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.eRq9U-kI-Nr3unmmBOYmQy8TlMITzQs_0 2024-04-22T20:11:29.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.619+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.622+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:11:29.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.623+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:11:29.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_M8lXXSD9nblSW6RCjyDJC6PsBf6YIR-', time=2024-04-22T22:06:19.670709+0000, chain.objs.size()=1 2024-04-22T20:11:29.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.hi4eBSriCAUiCvTkbFVudNOAff5R-Yx_0 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16469392166702829532', time=2024-04-22T22:06:19.755546+0000, chain.objs.size()=5 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.1 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__shadow_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.1_1 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.2 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__shadow_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.2_1 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-FROZEN.2~JoE-GoiNmK5llfN2GNC21_tM4pFCevk.3 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6428598243891000905', time=2024-04-22T22:08:11.976497+0000, chain.objs.size()=1 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.O9w3bVjxmLX7ykusKGGyDOGBZrz0fFw_0 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14585887693810554086', time=2024-04-22T22:08:14.278362+0000, chain.objs.size()=1 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.Oxd5DeNMmCwjT0uYD2XErlswMyBC_O1_0 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3926305149909237219', time=2024-04-22T22:08:50.991480+0000, chain.objs.size()=1 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.rbAcQOvvBJUWuVhHNU9VUVeFrddTreX_0 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2093668483742774182', time=2024-04-22T22:08:53.310934+0000, chain.objs.size()=1 2024-04-22T20:11:29.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.r36lFemSzmCQSF1hReRQ_JiGHv5r1Lw_0 2024-04-22T20:11:29.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11684226068732859787', time=2024-04-22T22:10:07.721524+0000, chain.objs.size()=1 2024-04-22T20:11:29.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.wIMd8mkZtKEzYQV_L0aOK3kmrM38NO3_0 2024-04-22T20:11:29.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2848578774714482879', time=2024-04-22T22:10:07.787249+0000, chain.objs.size()=1 2024-04-22T20:11:29.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.624+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.9R5fL0Enldfvn6699NDRLwxQrtyuihE_0 2024-04-22T20:11:29.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.626+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9835706313119996856', time=2024-04-22T22:10:07.787867+0000, chain.objs.size()=1 2024-04-22T20:11:29.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.626+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.hXVAQVcubXE1dB4tTSa3qC2dafJVb4n_0 2024-04-22T20:11:29.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.629+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.631+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:11:29.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.633+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.633+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-04-22T20:11:29.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4783100475266623634', time=2024-04-22T22:08:11.988825+0000, chain.objs.size()=1 2024-04-22T20:11:29.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.bw0A_mjpLMQO3BgC1Ga9T4I7g3c-BlA_0 2024-04-22T20:11:29.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4280318782822583721', time=2024-04-22T22:08:11.989376+0000, chain.objs.size()=1 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.jsFBQoFWO4cW0WdjmJwblYsvBbr-z4H_0 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.655921693527215496', time=2024-04-22T22:08:11.994418+0000, chain.objs.size()=1 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_._Ocsi1CdYS5yoBTw1zpuk74GvEKDoJn_0 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5241239492348368209', time=2024-04-22T22:08:47.164150+0000, chain.objs.size()=1 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.qGo60tfFtc8Y5LrV7xDRdwGGzpbcnTZ_0 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10354002639767680028', time=2024-04-22T22:09:24.762719+0000, chain.objs.size()=1 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.W-QiKDKtZH9B3RMMFBV0QXTUxvGCF4G_0 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603609879579199162', time=2024-04-22T22:09:28.975737+0000, chain.objs.size()=1 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.vLqNvOpQzX0OKCWWOGH-A7wfuXiS5Zc_0 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16896257489023193968', time=2024-04-22T22:09:28.976804+0000, chain.objs.size()=1 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.tX7aDfTZyuvjyZerWXbIJpAVxX9AwzS_0 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13291324143382240935', time=2024-04-22T22:09:31.131335+0000, chain.objs.size()=1 2024-04-22T20:11:29.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.T_ximyc_4DExKjXK2C-j3qALtOde8-E_0 2024-04-22T20:11:29.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10874511426609700445', time=2024-04-22T22:10:04.820449+0000, chain.objs.size()=1 2024-04-22T20:11:29.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.AFaZT6hBRzrR_dn_thlhF000zkaEnXA_0 2024-04-22T20:11:29.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17014595344725469285', time=2024-04-22T22:10:41.459960+0000, chain.objs.size()=1 2024-04-22T20:11:29.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.eO5ygD2OSMpgrJ5A8YyOeJPESCaK73u_0 2024-04-22T20:11:29.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6079653781502262505', time=2024-04-22T22:10:41.506113+0000, chain.objs.size()=1 2024-04-22T20:11:29.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.634+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.k8mTypDodHo_c9nlR4Wl_ED247IqkeV_0 2024-04-22T20:11:29.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.639+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.641+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:11:29.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.642+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.642+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1_1f0c5ea57d715e24c2f266da123be979', time=2024-04-22T22:06:19.699072+0000, chain.objs.size()=5 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.1_1 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.2 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.2_1 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-FROZEN.2~96Y3CSM5NWpsMUp9_XbXfwuIhWFCWLe.3 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_tbzq519WG5reY-jETQte14-lG29QV04', time=2024-04-22T22:06:19.714754+0000, chain.objs.size()=3 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.PkJhzRsAMqhbTakknTyEjIdaznR3ieb_0 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.PkJhzRsAMqhbTakknTyEjIdaznR3ieb_1 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.PkJhzRsAMqhbTakknTyEjIdaznR3ieb_2 2024-04-22T20:11:29.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11523405661129086315', time=2024-04-22T22:08:12.000370+0000, chain.objs.size()=1 2024-04-22T20:11:29.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.V4-NEKCvTOaWaktAI1n95GWnDdHA_6Z_0 2024-04-22T20:11:29.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1284187938496512723', time=2024-04-22T22:08:47.208474+0000, chain.objs.size()=1 2024-04-22T20:11:29.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.rzXPPUNCuzX8nSymceSgkGV4zXNcdB8_0 2024-04-22T20:11:29.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6098889823536162461', time=2024-04-22T22:10:04.756465+0000, chain.objs.size()=1 2024-04-22T20:11:29.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.yspCWT3l-2cauoJd-ufgSAcVBPB840H_0 2024-04-22T20:11:29.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8465089848046705130', time=2024-04-22T22:10:41.408790+0000, chain.objs.size()=1 2024-04-22T20:11:29.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.643+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.c5l5Wx4_kuig3X2DFAUVxpC5vXC8H86_0 2024-04-22T20:11:29.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.646+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.649+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:11:29.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.650+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.650+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.650+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=15, truncated=0, next_marker='' 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.650+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5167284803758613191', time=2024-04-22T22:06:19.545157+0000, chain.objs.size()=1 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.650+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.SYRR4aq-6lZ-bpcY8Yd2vOy7RkWbzex_0 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13745958173434414860', time=2024-04-22T22:06:19.599188+0000, chain.objs.size()=7 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__multipart_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.1 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__shadow_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.1_1 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__multipart_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.2 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__shadow_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.2_1 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__multipart_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.3 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__shadow_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.3_1 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10__multipart_foo.2~EilzLcDdILkngmE_1U3bT2Qtn03xxEF.4 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10290313714166553581', time=2024-04-22T22:06:19.710364+0000, chain.objs.size()=5 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1 2024-04-22T20:11:29.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.2 2024-04-22T20:11:29.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.651+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.2_1 2024-04-22T20:11:29.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.652+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.3 2024-04-22T20:11:29.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.653+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11941620205226592606', time=2024-04-22T22:08:12.001137+0000, chain.objs.size()=1 2024-04-22T20:11:29.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.653+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.ga7pJm2L9_EiuMsmqRBBfcseYsnAW7F_0 2024-04-22T20:11:29.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.653+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15430070186361562932', time=2024-04-22T22:08:12.001873+0000, chain.objs.size()=1 2024-04-22T20:11:29.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.653+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.Zeh0F5wgMR5e_Ud3XPFVKHi6lOsU8O-_0 2024-04-22T20:11:29.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.654+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2337520201056489358', time=2024-04-22T22:08:47.169030+0000, chain.objs.size()=1 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.654+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.huS_uYdPyCXZDclBLwZsvA8nXI02DZc_0 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.654+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12597049118267325939', time=2024-04-22T22:08:47.247556+0000, chain.objs.size()=1 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.654+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.njNRxP3zWVSWf0s5vhvGrgGOlMB17M5_0 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8009025047965136378', time=2024-04-22T22:08:49.728837+0000, chain.objs.size()=1 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.yrjX5cxC1jbJ6A-HSw8m7wqKj9SkzGQ_0 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9861556819759557778', time=2024-04-22T22:08:51.017881+0000, chain.objs.size()=1 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.RIoGXUe0Rx3ZyIG6cMyGZkTD8yIpQso_0 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14983156001675823046', time=2024-04-22T22:10:04.731647+0000, chain.objs.size()=1 2024-04-22T20:11:29.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.SNHJuJF82i7n9WbPkihaApaQ0NHqM1w_0 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16958906725902438224', time=2024-04-22T22:10:04.783053+0000, chain.objs.size()=1 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.gK_gyjRwJIZgE4P9WxBwoaAxkowkfbx_0 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2601193484992650364', time=2024-04-22T22:10:41.462517+0000, chain.objs.size()=1 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.DKxHSo9n9B6DDuOR6TT6b8f5ZHhNINU_0 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9091953369342523710', time=2024-04-22T22:10:41.463194+0000, chain.objs.size()=1 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.655+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.Wz2d0dmimdqTy7regRZDnHbmQIGoi9E_0 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.656+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.852647793307241834', time=2024-04-22T22:10:41.463813+0000, chain.objs.size()=1 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.656+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.TiA-Iyj61VZXAuJh2IExM6C5HmVIPR2_0 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.656+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1083599638609047362', time=2024-04-22T22:10:56.934602+0000, chain.objs.size()=1 2024-04-22T20:11:29.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.656+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18__shadow:enje3Wv6Kw-XY2D7MI6Wn-XZo4zkW76_.SwuLDrpU6tCbUSIaJvxnfDaVOLTLEvy_0 2024-04-22T20:11:29.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.660+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.662+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:11:29.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7929408699918258305', time=2024-04-22T22:08:14.251510+0000, chain.objs.size()=1 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.tVMfMJZsVqfOTqHN5TOxoWj7x_OfaiE_0 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13222017839783315487', time=2024-04-22T22:08:47.267597+0000, chain.objs.size()=1 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.VN1600zfX4j0-JiTYz92HWohs1Kqi-S_0 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15785583648623888641', time=2024-04-22T22:09:24.731909+0000, chain.objs.size()=1 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.qP9Fzn3UPKAp60c9AW5_KTlKP5dq5_s_0 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8456133291529328415', time=2024-04-22T22:09:28.998608+0000, chain.objs.size()=1 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.J6jvyg_S0T73oDl82_nSjwWPVPHlkKE_0 2024-04-22T20:11:29.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15718760594288955085', time=2024-04-22T22:09:29.035976+0000, chain.objs.size()=1 2024-04-22T20:11:29.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.bSxc7C5cIAZCCL2S6CkQ9mDlWk4fAc7_0 2024-04-22T20:11:29.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10365563798652130406', time=2024-04-22T22:09:31.057203+0000, chain.objs.size()=1 2024-04-22T20:11:29.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.N1wUNFAFjTJP6xJwUMR-dEGal-veBBJ_0 2024-04-22T20:11:29.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5645003905331831040', time=2024-04-22T22:10:07.786095+0000, chain.objs.size()=1 2024-04-22T20:11:29.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.664+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.j_Y7EsaZ27C3lRfw6QB38uP1zLC0RBR_0 2024-04-22T20:11:29.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.667+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.669+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:11:29.669 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.670+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-22T20:11:29.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3379786376601497935', time=2024-04-22T22:06:19.532499+0000, chain.objs.size()=1 2024-04-22T20:11:29.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.f5JvrGljUf27eqP98Og8vVifu89mhse_0 2024-04-22T20:11:29.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3546291817057341720', time=2024-04-22T22:06:19.618493+0000, chain.objs.size()=4 2024-04-22T20:11:29.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9__multipart_foo.2~JKgiZvhRhypZ9CtQWmdlGR8okhMlwqF.1 2024-04-22T20:11:29.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9__multipart_foo.2~JKgiZvhRhypZ9CtQWmdlGR8okhMlwqF.2 2024-04-22T20:11:29.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9__multipart_foo.2~JKgiZvhRhypZ9CtQWmdlGR8okhMlwqF.3 2024-04-22T20:11:29.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9__multipart_foo.2~JKgiZvhRhypZ9CtQWmdlGR8okhMlwqF.4 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1_bbab19ceef56354f88e20599ba4eb0c4', time=2024-04-22T22:06:19.658803+0000, chain.objs.size()=2 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_0 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6906545616172387648', time=2024-04-22T22:06:19.779858+0000, chain.objs.size()=3 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.SnH2lIpXBu0QqES7HyPPx_3RaWcu64h_0 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.SnH2lIpXBu0QqES7HyPPx_3RaWcu64h_1 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.SnH2lIpXBu0QqES7HyPPx_3RaWcu64h_2 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10185704584046095922', time=2024-04-22T22:08:12.000404+0000, chain.objs.size()=1 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.2ElWlWPq3p2bC200BoeRY_CghXCDAOd_0 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16541433971411457611', time=2024-04-22T22:10:41.446222+0000, chain.objs.size()=1 2024-04-22T20:11:29.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.671+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.IGNBl5kBu1S9lGq7poTNn-eijMWeHke_0 2024-04-22T20:11:29.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.676+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.678+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:11:29.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.679+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-22T20:11:29.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9061042827645264942', time=2024-04-22T22:06:19.527712+0000, chain.objs.size()=1 2024-04-22T20:11:29.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.LnhnG9lxQ8Y_3PRp9qYtH0w_7V8TIsO_0 2024-04-22T20:11:29.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8358918825814176564', time=2024-04-22T22:08:14.229959+0000, chain.objs.size()=1 2024-04-22T20:11:29.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.7wK88mi_ZrrExsyLxwOyYCMWLbfNJFH_0 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11969127686708281463', time=2024-04-22T22:08:47.175112+0000, chain.objs.size()=1 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.KEBBjH4d88koepexHpFavdOJOg6xIwA_0 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12400975212227408860', time=2024-04-22T22:09:24.723904+0000, chain.objs.size()=1 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.jkt8jM6Nk6YYIMcgj0RbucbbFCkjPmg_0 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11418224232875372809', time=2024-04-22T22:10:04.746875+0000, chain.objs.size()=1 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.4F4J0VVh7Ol2Mr0Ja_NE5ioHfQeHArj_0 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4231400466270620089', time=2024-04-22T22:10:04.780268+0000, chain.objs.size()=1 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.oany665Xpe0iKXYPZS8o5D3iapm_VEp_0 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14416382528084254650', time=2024-04-22T22:10:07.813943+0000, chain.objs.size()=1 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.4oJlZhWtbG3QXgiRnxM2PLsQsFE25mm_0 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14393686094015207848', time=2024-04-22T22:10:41.465902+0000, chain.objs.size()=1 2024-04-22T20:11:29.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.680+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.2yjH6A2yQu1Vt5b-mySPUH3p3JiTe7F_0 2024-04-22T20:11:29.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.683+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.685 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.686+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:11:29.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-22T20:11:29.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7868511335926160054', time=2024-04-22T22:06:19.510524+0000, chain.objs.size()=1 2024-04-22T20:11:29.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.2e1ZY4wuTILZpJrEndGeJvP1BW4ampx_0 2024-04-22T20:11:29.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.941870224416043255', time=2024-04-22T22:06:19.540750+0000, chain.objs.size()=1 2024-04-22T20:11:29.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.cDInpDvqWPWqzxV6dzfd-HYizr-LM4D_0 2024-04-22T20:11:29.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13250830488287904107', time=2024-04-22T22:06:19.865974+0000, chain.objs.size()=1 2024-04-22T20:11:29.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8__shadow_.ZKUWNsr4y1YGwrEfcLoxr8Er7wQXTuT_0 2024-04-22T20:11:29.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2133779243801308404', time=2024-04-22T22:08:14.324677+0000, chain.objs.size()=1 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.I1SGvvnWBWVzDmsNkZpHbsj9vSXg9F9_0 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4714885585085508141', time=2024-04-22T22:08:51.001715+0000, chain.objs.size()=1 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.IDQv3wupBPNSnpijU4FXonEaI0NEIl0_0 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4870153452025159588', time=2024-04-22T22:09:28.985508+0000, chain.objs.size()=1 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.PM-TU9Pmu2AuH7xEjyzyMlMeK7mDQg0_0 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4154917319237353429', time=2024-04-22T22:10:04.755834+0000, chain.objs.size()=1 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.XLmn3uCx-cWD7EU05gtDGB25vzcelt-_0 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10039586054739470194', time=2024-04-22T22:10:07.748738+0000, chain.objs.size()=1 2024-04-22T20:11:29.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.688+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.kigTjIAGteINUQYVqNW9ERCumRiONpG_0 2024-04-22T20:11:29.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.692+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.694+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:11:29.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.695+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.696+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.696+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.696+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_1_99992276c73b41acc0150aa4ff2209a0', time=2024-04-22T22:06:19.678271+0000, chain.objs.size()=1 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.sKtamzJl0OI2L5iPwmTB1KI8XbV5WZ8_0 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7813906502599313086', time=2024-04-22T22:08:11.982513+0000, chain.objs.size()=1 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.AFXR_HaQAw71uc1EfoYpggCsKF0U1Cu_0 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8135742529893055442', time=2024-04-22T22:08:11.983770+0000, chain.objs.size()=1 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.3CwE97W434oUp3q4fNdBnOSZGaSboIb_0 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15978201876392136403', time=2024-04-22T22:08:14.257373+0000, chain.objs.size()=1 2024-04-22T20:11:29.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.64ra32qC2JwADYDqXVU4yiQ-c3jkejL_0 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13797210986188975174', time=2024-04-22T22:09:29.000246+0000, chain.objs.size()=1 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.YOhUeussYGA2gI4YgUiqEvZPT-HLrk8_0 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8806446877799457824', time=2024-04-22T22:09:31.082971+0000, chain.objs.size()=1 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.tZiuXSRVjLOXtpxovJf6zLRxOqytX-Z_0 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6399315050119620484', time=2024-04-22T22:10:04.755028+0000, chain.objs.size()=1 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.FnHveoABfmcP5BtkLNzhtsMp7OmRd5-_0 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9799631339925535359', time=2024-04-22T22:10:04.816444+0000, chain.objs.size()=1 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.2rMoieRbpx2F3oE0uJl6i3V4wFpNeQo_0 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1493538804742917144', time=2024-04-22T22:10:07.774280+0000, chain.objs.size()=1 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.SANMiT3DWoRwHmfW6xHWDdXgubWo-mC_0 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17063822461105105664', time=2024-04-22T22:10:42.977398+0000, chain.objs.size()=1 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17__shadow_.7qe5wFiSJTwRa0yGFJMxWrEBf91j-uq_0 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7311536683562188562', time=2024-04-22T22:10:56.941635+0000, chain.objs.size()=1 2024-04-22T20:11:29.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.697+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18__shadow:Xo.biuK3RWmJyaB7TAyvgzJ1NR4I2zY_.WuVFzreqAFeJPbHwDHbrp0sG9HJNs0P_0 2024-04-22T20:11:29.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.700+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.703+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:11:29.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5296583134551112091', time=2024-04-22T22:06:19.759599+0000, chain.objs.size()=5 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.1 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__shadow_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.1_1 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.2 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__shadow_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.2_1 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-LUKEWARM.2~RV6VSXYx5tLpAd61-BFhlbjFEFo6CwC.3 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16270238826618142801', time=2024-04-22T22:06:19.828029+0000, chain.objs.size()=1 2024-04-22T20:11:29.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.705+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7__shadow_.8UnRz0Vygc5QVIYmXBDPp8kg_Zwfx4w_0 2024-04-22T20:11:29.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.708+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.710+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:11:29.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.711+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:11:29.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14254305980892923405', time=2024-04-22T22:06:19.635445+0000, chain.objs.size()=2 2024-04-22T20:11:29.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_0 2024-04-22T20:11:29.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1 2024-04-22T20:11:29.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_Icc7jkR0y1f3HcPlZDGMma3uWpIUDGl', time=2024-04-22T22:06:19.702837+0000, chain.objs.size()=3 2024-04-22T20:11:29.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_._GMSN2cCpZMxK-PNVzix3jlIdiMNCIN_0 2024-04-22T20:11:29.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_._GMSN2cCpZMxK-PNVzix3jlIdiMNCIN_1 2024-04-22T20:11:29.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_._GMSN2cCpZMxK-PNVzix3jlIdiMNCIN_2 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.305818996096588399', time=2024-04-22T22:08:10.701673+0000, chain.objs.size()=1 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.KraMXKRuPD06xjox0K_E9IHwlJFfADH_0 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2736262070045196429', time=2024-04-22T22:08:47.185235+0000, chain.objs.size()=1 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.cZkIMO16nL6G3Zdi2NwsH2rEMQgNS-W_0 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13475884159170178619', time=2024-04-22T22:08:51.065844+0000, chain.objs.size()=1 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.vWUfenTYSvoUVDhpAXufAWtT0CRxuUY_0 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1796776453754820611', time=2024-04-22T22:09:24.744409+0000, chain.objs.size()=1 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_._cZH1rLJTegA9eENklw2P4voTM6ZYBH_0 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12063680512444327971', time=2024-04-22T22:09:31.052163+0000, chain.objs.size()=1 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.XHgZxiGNMeTSpa-FNx1LxGzwBW4NzlO_0 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12268240169887258345', time=2024-04-22T22:10:04.813556+0000, chain.objs.size()=1 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.gbk0Y1sjSkeZQnQamQ7XNHH5je8lI1N_0 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5018882665420158681', time=2024-04-22T22:10:06.489078+0000, chain.objs.size()=1 2024-04-22T20:11:29.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.712+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.vdJZ7mCUUDLJfi1HGweT8wgh7h4Mqoc_0 2024-04-22T20:11:29.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.717+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.719+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:11:29.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.720+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.720+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-22T20:11:29.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_Rpx47fHzqjW40yD-UQjGSmuqb_Vkc7l', time=2024-04-22T22:06:19.654891+0000, chain.objs.size()=2 2024-04-22T20:11:29.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.QfMAtLs_3laUIorPAVnrMFfVe0UNULP_0 2024-04-22T20:11:29.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.QfMAtLs_3laUIorPAVnrMFfVe0UNULP_1 2024-04-22T20:11:29.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4590915619532309436', time=2024-04-22T22:06:19.833611+0000, chain.objs.size()=1 2024-04-22T20:11:29.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7__shadow_.FgVJ4iwDdbLemdkJk9E5hvmRuYtj1P8_0 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8515233154692155770', time=2024-04-22T22:06:19.858259+0000, chain.objs.size()=1 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8__shadow_.8iXGndIFhlyXbcS_Qxi03egpRmPWLsi_0 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2623494781125828813', time=2024-04-22T22:08:12.015485+0000, chain.objs.size()=1 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.Cj-wWrSwiuQTFQO3WO_gBtBF1mr9GtN_0 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2558117196460874063', time=2024-04-22T22:08:14.250521+0000, chain.objs.size()=1 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.kN6XYHtkgNeaHTidGTBwAN5ACTm9Huv_0 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15314693252744058009', time=2024-04-22T22:10:07.748702+0000, chain.objs.size()=1 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.JLjniC6nhAUn3R0--VNaye8PCn0HoD0_0 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2296593831063357583', time=2024-04-22T22:10:42.976132+0000, chain.objs.size()=1 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17__shadow_.ar3biyn9_KlVVighOqL9ei4oIKL4DL9_0 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9431889090952877496', time=2024-04-22T22:10:42.976863+0000, chain.objs.size()=1 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.721+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17__shadow_.2mJASoLJwu-7GawCHFpR5eu5u0oTczk_0 2024-04-22T20:11:29.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.724+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.726+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:11:29.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.727+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_eXtGZCCfrb-zKwh_gM2lUzoSpbvaUS0', time=2024-04-22T22:06:19.647461+0000, chain.objs.size()=2 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.X9ye873tkmd1aAm4_K9pS1FBSzTsRTz_0 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.X9ye873tkmd1aAm4_K9pS1FBSzTsRTz_1 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1745339574441415955', time=2024-04-22T22:08:14.256946+0000, chain.objs.size()=1 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.sv_lM_eFt-giQnkccr4qWw6twSYZKjK_0 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.305641580346646998', time=2024-04-22T22:08:47.206595+0000, chain.objs.size()=1 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.B_kc8yyVH5UISMa8o0CeouUMJMJn8-E_0 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9497809656074903737', time=2024-04-22T22:09:24.801636+0000, chain.objs.size()=1 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.aLqiub9IysEib28Plg4fRfsFBfClP8V_0 2024-04-22T20:11:29.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2110975316597086687', time=2024-04-22T22:09:31.027816+0000, chain.objs.size()=1 2024-04-22T20:11:29.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.lt64AHJCAWvyoTVjhTMwiIxoFF-76Mo_0 2024-04-22T20:11:29.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9432413607201054093', time=2024-04-22T22:10:41.382251+0000, chain.objs.size()=1 2024-04-22T20:11:29.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.CeksvB7Y4X2nesBSV9VOy-2bMQ9Whdh_0 2024-04-22T20:11:29.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6059352740306158106', time=2024-04-22T22:10:41.470693+0000, chain.objs.size()=1 2024-04-22T20:11:29.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.728+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.YSYHftwq1UsQyKn40pTz0sIAC5PztqE_0 2024-04-22T20:11:29.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.731+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.733+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:11:29.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.735+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.735+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:11:29.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15625124244436283087', time=2024-04-22T22:06:19.536678+0000, chain.objs.size()=1 2024-04-22T20:11:29.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.rivaAMaP8BOaOkle9zHSr1qEnqPQsbg_0 2024-04-22T20:11:29.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1020013301252061178', time=2024-04-22T22:06:19.787805+0000, chain.objs.size()=1 2024-04-22T20:11:29.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.RbjQZkc1FXL1vx7YqvANq2tS2jJWbSz_0 2024-04-22T20:11:29.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10898389801978136611', time=2024-04-22T22:08:14.257375+0000, chain.objs.size()=1 2024-04-22T20:11:29.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.X_0bB-8BpngM4GZevSQLNX-tofp7TRU_0 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8530265163938975910', time=2024-04-22T22:08:14.279416+0000, chain.objs.size()=1 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.kxXYjPjjhVgOe7oxTZm9IEOGlegPrEe_0 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3592776360608266933', time=2024-04-22T22:08:14.322331+0000, chain.objs.size()=1 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.pdSlWc-1SlB9dIG3qvNSwYAQIW-9WW__0 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17080800271119979521', time=2024-04-22T22:08:51.040760+0000, chain.objs.size()=1 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.UN5zoLGTFZPjHWTWh0oPr36JbqPOCFn_0 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5674204179815720225', time=2024-04-22T22:09:28.953903+0000, chain.objs.size()=1 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.nikgnPOIpLl8bIyedo1fhxs7UrOTfBm_0 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10978942316340059557', time=2024-04-22T22:10:41.497345+0000, chain.objs.size()=1 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.viLlg_5GZUTu3R46LrhHh9mY_tZXHuw_0 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14837889896904060650', time=2024-04-22T22:10:42.985666+0000, chain.objs.size()=1 2024-04-22T20:11:29.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.736+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17__shadow_._x5ytDB_lBaYLBQMU6ZqlRnOl6T1sVY_0 2024-04-22T20:11:29.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.738+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.741+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:11:29.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_7Tq7Ujhj4ovtHzIpHFfHuRqC9V-vz4Z', time=2024-04-22T22:06:19.674361+0000, chain.objs.size()=1 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.gHghNPhtFNO_WaXqx_dgFfRtHUJc2N7_0 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4379106960492299791', time=2024-04-22T22:09:24.823282+0000, chain.objs.size()=1 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.lXFxnih90WjATy9Mu1PPsdJKvXry5Ml_0 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4268720040532705390', time=2024-04-22T22:09:31.051082+0000, chain.objs.size()=1 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.JLACHNMd8XwBbCHiluHOW1HX1vflHRL_0 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10210348783846110936', time=2024-04-22T22:09:31.132198+0000, chain.objs.size()=1 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.743+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.4QW6e_ipRQiEJPel3m0O_A95ieESIr3_0 2024-04-22T20:11:29.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14886436539110105407', time=2024-04-22T22:09:31.132764+0000, chain.objs.size()=1 2024-04-22T20:11:29.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.5voYK-6dsVE1TeS2F5EUwHyVT-9IiOa_0 2024-04-22T20:11:29.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13132137433863185331', time=2024-04-22T22:10:04.753783+0000, chain.objs.size()=1 2024-04-22T20:11:29.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.AHNXzon9JCS0q4lry-_zenGTJ4jw9KX_0 2024-04-22T20:11:29.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7563858520623153258', time=2024-04-22T22:10:04.755612+0000, chain.objs.size()=1 2024-04-22T20:11:29.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.glnF3Xj547_nPz6X5qYX2VjcqF5_syN_0 2024-04-22T20:11:29.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1428866598814307457', time=2024-04-22T22:10:04.780228+0000, chain.objs.size()=1 2024-04-22T20:11:29.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.tP0uAfJU8BIlAt-aJa7wluJrr7VAU3k_0 2024-04-22T20:11:29.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2525276625238888410', time=2024-04-22T22:10:07.810515+0000, chain.objs.size()=1 2024-04-22T20:11:29.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.Aa1EPLWdlsrNudP_UWgh5zexyrzRId1_0 2024-04-22T20:11:29.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5055751442829617969', time=2024-04-22T22:10:42.962161+0000, chain.objs.size()=1 2024-04-22T20:11:29.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.744+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17__shadow_.RTYChyjeYUf_V3bdqjJbhYphOy-SjX4_0 2024-04-22T20:11:29.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.747+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.749+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:11:29.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.750+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1941564247898136352', time=2024-04-22T22:06:19.523436+0000, chain.objs.size()=1 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.NdmkIMb4RQ0j7NZZbGdF8pEJCwgh99A_0 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17022221606317145684', time=2024-04-22T22:08:14.264903+0000, chain.objs.size()=1 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.7yJwnm0Sse8dIGrx8yZ-dJpFVAfDNjc_0 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3917649647185382474', time=2024-04-22T22:09:28.972012+0000, chain.objs.size()=1 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.PiQqLwApJEbNsxC6kQZ0MMSOVXq31-F_0 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13463466332289952975', time=2024-04-22T22:10:42.972469+0000, chain.objs.size()=1 2024-04-22T20:11:29.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.751+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17__shadow_.DbtEi0t5uQJeJo334959BMuciJsA3fG_0 2024-04-22T20:11:29.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.753+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.755+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:11:29.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.756+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:11:29.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2269165753410918401', time=2024-04-22T22:06:19.651085+0000, chain.objs.size()=2 2024-04-22T20:11:29.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_0 2024-04-22T20:11:29.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.l_yy6WMPuOhcXGoK6uuUPf5217cBnFe_1 2024-04-22T20:11:29.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7695979420783704415', time=2024-04-22T22:08:11.983849+0000, chain.objs.size()=1 2024-04-22T20:11:29.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.cRBawSfkLQMGtsAnGk7N9HXR31xwprr_0 2024-04-22T20:11:29.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3339233334529633845', time=2024-04-22T22:08:47.206687+0000, chain.objs.size()=1 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.cx6T_d5LEronb8qkHYkf2w5G3zrV6az_0 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13208445976224603567', time=2024-04-22T22:08:47.240055+0000, chain.objs.size()=1 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.5n51zNMV25_Der71hbSyL4OFEj-Ec95_0 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14923260702197653620', time=2024-04-22T22:08:51.006671+0000, chain.objs.size()=1 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.rYW05sMjywL8y9STTav9TuzIPd4pYrK_0 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5474754828268575168', time=2024-04-22T22:08:51.070028+0000, chain.objs.size()=1 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.-ICmdxizboOiii-h8ThhboYs_7KFlnD_0 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.380538504087710261', time=2024-04-22T22:09:24.782119+0000, chain.objs.size()=1 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.7ehGbzZEiw5lHED5gVaXP3TK6WilVk6_0 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9272429498551242376', time=2024-04-22T22:09:29.000275+0000, chain.objs.size()=1 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.EH-gPInQGaC8tAgjtmeTLzZkohW9yZR_0 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5598537707877275806', time=2024-04-22T22:09:31.049768+0000, chain.objs.size()=1 2024-04-22T20:11:29.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.757+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.rczbHEg0Fhr26tGtZJKtKMLId7QIUX6_0 2024-04-22T20:11:29.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.761+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.763+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:11:29.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.765+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.765+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:11:29.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15723389894443242038', time=2024-04-22T22:06:19.514889+0000, chain.objs.size()=1 2024-04-22T20:11:29.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.m7uCOIrMFkCmPl-gNeqU_IroBn2oDiK_0 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17415478986783585847', time=2024-04-22T22:06:19.763901+0000, chain.objs.size()=3 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-STANDARD.2~dHoKXv965g23tcG9lyjbpj1RNE3-FrA.1 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-STANDARD.2~dHoKXv965g23tcG9lyjbpj1RNE3-FrA.2 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14__multipart_foo-STANDARD.2~dHoKXv965g23tcG9lyjbpj1RNE3-FrA.3 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5510026797356098222', time=2024-04-22T22:06:19.783953+0000, chain.objs.size()=3 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.GnhA_1A4gaaM1Xk0gMoLKTZOKcOQXBb_0 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.GnhA_1A4gaaM1Xk0gMoLKTZOKcOQXBb_1 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13__shadow_.GnhA_1A4gaaM1Xk0gMoLKTZOKcOQXBb_2 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4826913521811364234', time=2024-04-22T22:06:19.841697+0000, chain.objs.size()=1 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7__shadow_.Cn2J41Okq5qm_n1UonStT2ILRs_pJjx_0 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9282974316338752385', time=2024-04-22T22:08:50.990322+0000, chain.objs.size()=1 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.uHxGqD_rAldEOfh9hfKLFOwYjC_-8x__0 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7505623138062732377', time=2024-04-22T22:09:24.765226+0000, chain.objs.size()=1 2024-04-22T20:11:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.PauJCAM2FzwgSq2eJkn0nvtssK3kw2R_0 2024-04-22T20:11:29.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16493943145655697075', time=2024-04-22T22:09:28.929834+0000, chain.objs.size()=1 2024-04-22T20:11:29.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.EWW6YNoPqEnX7f8LMLHsf4Gi6rsP3aU_0 2024-04-22T20:11:29.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3299592913577847936', time=2024-04-22T22:10:07.774517+0000, chain.objs.size()=1 2024-04-22T20:11:29.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.766+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.D3wKwul6Pj3iqVPYBelQ7zxXGETALHy_0 2024-04-22T20:11:29.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.768+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5937081116896347088', time=2024-04-22T22:10:41.470893+0000, chain.objs.size()=1 2024-04-22T20:11:29.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.768+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.07hvVhpurIKObv6jE96qFfT5jVfoRdR_0 2024-04-22T20:11:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.770+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.772+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:11:29.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.773+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-04-22T20:11:29.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1781355800613501990', time=2024-04-22T22:06:19.578286+0000, chain.objs.size()=4 2024-04-22T20:11:29.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12__multipart_foo.2~-LTG0TDmGoM7WCJ_xnQAU3_VmkBqwCg.1 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12__multipart_foo.2~-LTG0TDmGoM7WCJ_xnQAU3_VmkBqwCg.2 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12__multipart_foo.2~-LTG0TDmGoM7WCJ_xnQAU3_VmkBqwCg.3 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12__multipart_foo.2~-LTG0TDmGoM7WCJ_xnQAU3_VmkBqwCg.4 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1_ff389ce5235714e61b9ab067ddc64c64', time=2024-04-22T22:06:19.639292+0000, chain.objs.size()=2 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_0 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.wojp01y4ywXEy-ohggfcO7y2PVDs9S8_1 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_ZK4INnDQKVXSiHlgtMHLmJnKJsQBNZN', time=2024-04-22T22:06:19.643364+0000, chain.objs.size()=2 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.A_dg0zF3R_4Vkdi09XtyLIYFFoOpt4t_0 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.A_dg0zF3R_4Vkdi09XtyLIYFFoOpt4t_1 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2_8aac982f249899867ff058f048594b83', time=2024-04-22T22:06:19.738788+0000, chain.objs.size()=3 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.1 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.3 2024-04-22T20:11:29.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7251962201324226871', time=2024-04-22T22:08:14.251203+0000, chain.objs.size()=1 2024-04-22T20:11:29.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.774+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.Dv4_82sioftCvHqiCBZT9twJ35sCSop_0 2024-04-22T20:11:29.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.776+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3580941500643564446', time=2024-04-22T22:09:28.953435+0000, chain.objs.size()=1 2024-04-22T20:11:29.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.776+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.zHxAkWrt582m6Hm45lcKIZgyej_6g5C_0 2024-04-22T20:11:29.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.777+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14777349400242274645', time=2024-04-22T22:09:28.954234+0000, chain.objs.size()=1 2024-04-22T20:11:29.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.777+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.QzZwt6-f24s2zPBDe7sfeBNxHp0VjLR_0 2024-04-22T20:11:29.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.777+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10880944764584654032', time=2024-04-22T22:09:31.081664+0000, chain.objs.size()=1 2024-04-22T20:11:29.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.777+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.q0Xdm2lBGp5CZ3SqGwK8v8zU2bOeFLT_0 2024-04-22T20:11:29.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.777+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14802631379759375188', time=2024-04-22T22:09:31.099281+0000, chain.objs.size()=1 2024-04-22T20:11:29.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.777+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.F8lXThplm49lUTkaPsRcvp0KacZeiNu_0 2024-04-22T20:11:29.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.777+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7221196484910034776', time=2024-04-22T22:10:07.749426+0000, chain.objs.size()=1 2024-04-22T20:11:29.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.777+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.AYPatNNr0DB1APb5Qdk3JYGb0RKuLEm_0 2024-04-22T20:11:29.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.780+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.782+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:11:29.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.783+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:11:29.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.345409528465350665', time=2024-04-22T22:08:14.252364+0000, chain.objs.size()=1 2024-04-22T20:11:29.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.yK7BoIdscbwpTTsDAaeRSGqVSHZC9K1_0 2024-04-22T20:11:29.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16441379960337138926', time=2024-04-22T22:08:14.316488+0000, chain.objs.size()=1 2024-04-22T20:11:29.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.vMMXc2aLeryUE6iIaON62MMoTO3C4qH_0 2024-04-22T20:11:29.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18014648329350816303', time=2024-04-22T22:08:47.210674+0000, chain.objs.size()=1 2024-04-22T20:11:29.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.TCE2mMdmFakuvucoaaGdW8h4Rraw8al_0 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9479968724825393349', time=2024-04-22T22:08:50.984345+0000, chain.objs.size()=1 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.sSORX4mqxILdWem9nFDt1fwot5g2nqd_0 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1477374235969624375', time=2024-04-22T22:08:50.984997+0000, chain.objs.size()=1 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.RYSw1NPg5Jib8fvVU0DjNbkKECNKPc0_0 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.570842267537603731', time=2024-04-22T22:08:51.028856+0000, chain.objs.size()=1 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.2MyxBWlYyGP0X7mJiqurW9bSkG41E3t_0 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11213656537338230050', time=2024-04-22T22:10:04.736279+0000, chain.objs.size()=1 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.S4mVvfLSWuEpTEd6toTG4pWB4Bv1yKb_0 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10488152910676447009', time=2024-04-22T22:10:41.449676+0000, chain.objs.size()=1 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.5m7aiRxgtfOzR7c70H00PlZGXZTTsRv_0 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.850591529410853091', time=2024-04-22T22:10:42.962436+0000, chain.objs.size()=1 2024-04-22T20:11:29.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.784+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17__shadow_.2BAv9I2JmPjCv2tbreNMfMXj990dfTr_0 2024-04-22T20:11:29.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.787+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.789+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:11:29.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8284680768097130031', time=2024-04-22T22:08:50.999728+0000, chain.objs.size()=1 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.z8_Pp1sD_5NfiFZiyW_ncUneCDJOS_U_0 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7445956933932922603', time=2024-04-22T22:09:29.000981+0000, chain.objs.size()=1 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.w51cQySg1fYkhHvNdwwZI3Vie3Hvaj7_0 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4888120325586253420', time=2024-04-22T22:10:03.469720+0000, chain.objs.size()=1 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.ek8Bw_9ZHllc7pesr6XNedfLmvzbNrQ_0 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18189743610556647099', time=2024-04-22T22:10:40.352284+0000, chain.objs.size()=1 2024-04-22T20:11:29.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.791+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.wz8cjEnGW1RKsRo1gel3Yup0dQTTkao_0 2024-04-22T20:11:29.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.793+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.795+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:11:29.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.796+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10697927379977224298', time=2024-04-22T22:06:19.862291+0000, chain.objs.size()=1 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8__shadow_.NJjgqNtQld1kigVXQ6wsdN1918t0-6S_0 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2375027469005684118', time=2024-04-22T22:08:51.064301+0000, chain.objs.size()=1 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.IoE5zwLKKf38LaLRIm5cSabgnp7BUIl_0 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9970455332925099285', time=2024-04-22T22:09:24.765140+0000, chain.objs.size()=1 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.zDIfGpNnHjAn9fjr21vvVJ72rAc1qJp_0 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14070522050293762615', time=2024-04-22T22:09:24.765852+0000, chain.objs.size()=1 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.gNL2TE6mw173McUs5n5wLh_7Um_MmPz_0 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2979281789848357209', time=2024-04-22T22:10:56.940920+0000, chain.objs.size()=1 2024-04-22T20:11:29.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.797+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18__shadow:NbHgeeoopnmJXhIqnk9tfe6kJqVJ3fg_.nRLU-l0YejYm67I1ExAKwLwQLOtNira_0 2024-04-22T20:11:29.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.799+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.801+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:11:29.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.803+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.803+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.803+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-22T20:11:29.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.803+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6960362847265545089', time=2024-04-22T22:06:19.561072+0000, chain.objs.size()=4 2024-04-22T20:11:29.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.803+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11__multipart_foo.2~lRRu1mPsTn6E7sRY658QyGFcN5djD_9.1 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.803+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11__multipart_foo.2~lRRu1mPsTn6E7sRY658QyGFcN5djD_9.2 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11__multipart_foo.2~lRRu1mPsTn6E7sRY658QyGFcN5djD_9.3 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11__multipart_foo.2~lRRu1mPsTn6E7sRY658QyGFcN5djD_9.4 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_6ZPjoRbO_uWpCX0a5tGcOaw3v4F84sp', time=2024-04-22T22:06:19.662908+0000, chain.objs.size()=2 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.lHF_y9q7KsXveMvFfDmCcXxYx783WkW_0 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15__shadow_.lHF_y9q7KsXveMvFfDmCcXxYx783WkW_1 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17296839473840161267', time=2024-04-22T22:08:11.993198+0000, chain.objs.size()=1 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.TRfcycyTpZ_f-ds-R2ZV7sd6DUd62A9_0 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5408459408352459159', time=2024-04-22T22:09:24.782445+0000, chain.objs.size()=1 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.ayTmzmSq1yVIO2Fuprxkki_QMfE1_-u_0 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10927516470883982415', time=2024-04-22T22:09:28.976808+0000, chain.objs.size()=1 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.yCzYAVs4jH6uZZpLGPUOd9gR2TwHvSG_0 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9722158779303491691', time=2024-04-22T22:10:04.748686+0000, chain.objs.size()=1 2024-04-22T20:11:29.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.Nrhl8G-HApjeaElolsCWxlb4F1pC4vj_0 2024-04-22T20:11:29.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4806686842113445866', time=2024-04-22T22:10:07.742702+0000, chain.objs.size()=1 2024-04-22T20:11:29.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.JPLRyHmWCoin5kMPVYv_Xq0sPBijPLW_0 2024-04-22T20:11:29.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16206768370533117756', time=2024-04-22T22:10:41.419297+0000, chain.objs.size()=1 2024-04-22T20:11:29.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.804+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.LtZ_LjQ73_Iuu7SqrC78tbS3-xgLdH5_0 2024-04-22T20:11:29.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.808+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.810+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:11:29.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.812+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.812+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-04-22T20:11:29.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14434792078798744896', time=2024-04-22T22:08:12.038918+0000, chain.objs.size()=1 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.SndXPIw4ZBrh5wp5JRKwki3rEhnbT6d_0 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16630266459387969680', time=2024-04-22T22:08:14.250519+0000, chain.objs.size()=1 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.y98w9heHwUGyxrk5VFbAP5P4KuchGcc_0 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11631683149376076599', time=2024-04-22T22:08:14.278362+0000, chain.objs.size()=1 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.p-4pkjwNzM9FpGfLML0WknDb7lBUANS_0 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8330898658664306687', time=2024-04-22T22:08:15.741630+0000, chain.objs.size()=1 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.AVgBgQ7mnjUXMb_Q5c73ow-skFjn8eu_0 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9341625766374439751', time=2024-04-22T22:08:47.263961+0000, chain.objs.size()=1 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.V7n9G2GPOXsllUoXmOIqaYPnyXzZvbt_0 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2282792296626181413', time=2024-04-22T22:08:51.025612+0000, chain.objs.size()=1 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.Y9P7jnUy-XU46eXEwzpZG9x2Xeik7v4_0 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.497879393442132175', time=2024-04-22T22:08:51.026342+0000, chain.objs.size()=1 2024-04-22T20:11:29.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.wXo4saOKrI0fQtk2JTTWif0_z7iiW-2_0 2024-04-22T20:11:29.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17681660289253946841', time=2024-04-22T22:10:41.468179+0000, chain.objs.size()=1 2024-04-22T20:11:29.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.FnyBwOkkweqX2UO7AXuFPlINH9ooDGj_0 2024-04-22T20:11:29.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4053348484965909497', time=2024-04-22T22:10:41.468982+0000, chain.objs.size()=1 2024-04-22T20:11:29.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.9Ouj8N1ZvQnGw8MCigyb1foqNrCA0lu_0 2024-04-22T20:11:29.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5706713344472781206', time=2024-04-22T22:10:41.490788+0000, chain.objs.size()=1 2024-04-22T20:11:29.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.813+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.t3f7zQ3ojwFtmrAL0dHDISEjWNDi_8b_0 2024-04-22T20:11:29.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.816+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.819+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:11:29.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.820+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.820+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.820+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.820+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.489055305490215571', time=2024-04-22T22:08:11.991875+0000, chain.objs.size()=1 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.sBrbkkBm78zGSDyoy0QiSrO24CH6CyF_0 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13578687823382168399', time=2024-04-22T22:08:11.999418+0000, chain.objs.size()=1 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.qOa7pGjxJPy4tAOM7qFZeADMhaFqkO0_0 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14988525807231364783', time=2024-04-22T22:08:14.278967+0000, chain.objs.size()=1 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.6jwiMrBFlPQLrYQfxY_4UjUlAA8SQcu_0 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7888447205407700456', time=2024-04-22T22:08:47.162820+0000, chain.objs.size()=1 2024-04-22T20:11:29.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.S1i6Wrg2kyczi3UUg590JUIizq4fkDJ_0 2024-04-22T20:11:29.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18130257042692230322', time=2024-04-22T22:08:50.985642+0000, chain.objs.size()=1 2024-04-22T20:11:29.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.z0_k4m2bSnX9l5Yuzvch8NBaREX7pgB_0 2024-04-22T20:11:29.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9578906437246652715', time=2024-04-22T22:09:31.028321+0000, chain.objs.size()=1 2024-04-22T20:11:29.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.H72_AHh0iweSfWNOuTAHpq5ik7oWW_T_0 2024-04-22T20:11:29.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17172255310272385039', time=2024-04-22T22:10:04.778774+0000, chain.objs.size()=1 2024-04-22T20:11:29.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.G-9qB9HBKt4sPkGjsdyoVbRGArBwOkD_0 2024-04-22T20:11:29.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7564928974436508321', time=2024-04-22T22:10:41.421144+0000, chain.objs.size()=1 2024-04-22T20:11:29.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.821+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.28__shadow_.UDT-0T6bPOHnwjnmGZ6-AS8lJeVUIKs_0 2024-04-22T20:11:29.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.824+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.826+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:11:29.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.827+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:11:29.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11282243673100401722', time=2024-04-22T22:08:14.278408+0000, chain.objs.size()=1 2024-04-22T20:11:29.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.Mu16hOCYuPMRGP7vW8UIRPQtWRu-QK6_0 2024-04-22T20:11:29.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11586593557694490626', time=2024-04-22T22:08:47.210611+0000, chain.objs.size()=1 2024-04-22T20:11:29.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.-Vc_fEhWVhM5uLO0lgJ8tatnKVUvzj7_0 2024-04-22T20:11:29.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5648841999746685138', time=2024-04-22T22:08:51.027181+0000, chain.objs.size()=1 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.SJb6NDU7G5pCYPwGSWjHWFKmxZistem_0 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9643965938439476399', time=2024-04-22T22:09:24.761128+0000, chain.objs.size()=1 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.9ArXnOQPZRyCgz3-SlA0AD7eEY7krBb_0 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9900662199563463743', time=2024-04-22T22:09:31.058960+0000, chain.objs.size()=1 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.jOrGW7zHgsqFr6vdBSoMG4PLwTpidB-_0 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.964019844739709961', time=2024-04-22T22:09:31.102983+0000, chain.objs.size()=1 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.fjvT88enKZtnyYHx7_dv39PAtwkkgT7_0 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1398276990322264080', time=2024-04-22T22:10:04.746648+0000, chain.objs.size()=1 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.8491KQ9vimHklVP-_92W2Pl0WSGWaCI_0 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3675785089350962148', time=2024-04-22T22:10:04.749317+0000, chain.objs.size()=1 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.26__shadow_.vxEDYmrcVGPYiHLFArABhULimYzkBAv_0 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.621978524705060664', time=2024-04-22T22:10:51.350064+0000, chain.objs.size()=1 2024-04-22T20:11:29.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.828+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18__shadow:fg67xW-.Nu9Q5MIXCd3AWIJIdivL9aI_.zxCLyrXQ0Uowdmz2uHO4ojH975jdqVY_0 2024-04-22T20:11:29.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.832+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.834+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:11:29.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-04-22T20:11:29.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1_0e2e20e50dc96ed1fcb7c15ef4ac7841', time=2024-04-22T22:06:19.718891+0000, chain.objs.size()=5 2024-04-22T20:11:29.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1 2024-04-22T20:11:29.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.1_1 2024-04-22T20:11:29.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.2 2024-04-22T20:11:29.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.2_1 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-LUKEWARM.2~fqSn61C653658GZ2RpaZ3trEttYCkJ3.3 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_VFhY0mFB_acH2IZTC-APyWDQTsvMV15', time=2024-04-22T22:06:19.730367+0000, chain.objs.size()=1 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.XAjAhvOYfhKID58FNR8ZFXo8VB07YAo_0 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='_VnrFze2BWDJaxSsxARBaGgwL4i3zZkA', time=2024-04-22T22:06:19.734420+0000, chain.objs.size()=1 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__shadow_.K3vdaO7MbBEkMt_gxtcunVtumba-y_H_0 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1657659686465603911', time=2024-04-22T22:06:19.837634+0000, chain.objs.size()=1 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7__shadow_.RKqyAG_gaeu_MaoqxB_xK62RwcNOr49_0 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16320780840345231486', time=2024-04-22T22:08:47.256689+0000, chain.objs.size()=1 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.21__shadow_.rlhGKhZGQ8HKdXZIiSCkghGecbAYuLD_0 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.974759853138394046', time=2024-04-22T22:09:24.812120+0000, chain.objs.size()=1 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.836+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.KDN3KDqLTVQwrP7OgBDhxvROgv1hK-e_0 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.837+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18008202413465692689', time=2024-04-22T22:09:27.684028+0000, chain.objs.size()=1 2024-04-22T20:11:29.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.837+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.24__shadow_.beWX1cQzqFHgVmZ4ORTxsvPiDI4GUI8_0 2024-04-22T20:11:29.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.837+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17982391141659408077', time=2024-04-22T22:09:31.069459+0000, chain.objs.size()=1 2024-04-22T20:11:29.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.837+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.h17DUmtzY4DjAjKIU0cribfP0Q3vnP5_0 2024-04-22T20:11:29.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.838+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3624243029760370458', time=2024-04-22T22:09:31.131037+0000, chain.objs.size()=1 2024-04-22T20:11:29.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.838+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.vHdFNxWpCkgqsLWTLdIdMRH0cu9q_r0_0 2024-04-22T20:11:29.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.838+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8889241039034374738', time=2024-04-22T22:10:07.822092+0000, chain.objs.size()=1 2024-04-22T20:11:29.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.838+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.BRUBiOK-r7r--Ou9XeN0kByY1ms8xV__0 2024-04-22T20:11:29.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.839+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8637249416675583299', time=2024-04-22T22:10:42.972453+0000, chain.objs.size()=1 2024-04-22T20:11:29.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.839+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17__shadow_.LJxpW7gA1h4etw23H4S6e2gTDk60nb__0 2024-04-22T20:11:29.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.841+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.843+0000 7fe99da87800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:11:29.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.844+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:29.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5288031347509135434', time=2024-04-22T22:06:19.519185+0000, chain.objs.size()=1 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4__shadow_.QcMBS1inwWBH_6hy7pxjBEBwZ-dPZE4_0 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15524819901381658645', time=2024-04-22T22:06:19.726727+0000, chain.objs.size()=3 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.1 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.2 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16__multipart_foo-STANDARD.2~hc53aX6m52ssUm4glyv8kgG88eMQV2N.3 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13535250855706792583', time=2024-04-22T22:08:12.051518+0000, chain.objs.size()=1 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.19__shadow_.oobhepc3NDQ3-00ck-mINZ_jc3Ncx6o_0 2024-04-22T20:11:29.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18163399101502846257', time=2024-04-22T22:08:14.234813+0000, chain.objs.size()=1 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.20__shadow_.II24Ze-6_5nZjcvbsVqa0GTITsDLHip_0 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13978245922933103941', time=2024-04-22T22:08:50.999729+0000, chain.objs.size()=1 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.22__shadow_.mvorSlLVqJV3LyTFa8FqKmo-vPIUCFT_0 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10012065351097492216', time=2024-04-22T22:09:24.801619+0000, chain.objs.size()=1 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.23__shadow_.PnAQb5bkYHm1mqKiPWoPz6GEKnF8Tmf_0 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17005413401593048861', time=2024-04-22T22:09:31.048369+0000, chain.objs.size()=1 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.aVqkSKWol4MA-4-dAjlKqVouyHJzM47_0 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1332331046039901091', time=2024-04-22T22:09:31.052533+0000, chain.objs.size()=1 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.25__shadow_.fDjkO_0yj0hQTuTkYqi7WvaTNDRRlE6_0 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4337011461187464444', time=2024-04-22T22:10:07.748342+0000, chain.objs.size()=1 2024-04-22T20:11:29.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.845+0000 7fe99da87800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.27__shadow_.XrdWaG47xa5qKP1AZNBQqHiMUuMVwMZ_0 2024-04-22T20:11:29.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.848+0000 7fe99da87800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:11:29.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.855+0000 7fe851fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:11:29.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.857+0000 7fe99da87800 20 remove_watcher() i=0 2024-04-22T20:11:29.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.857+0000 7fe99da87800 2 removed watcher, disabling cache 2024-04-22T20:11:29.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.858+0000 7fe99da87800 20 remove_watcher() i=1 2024-04-22T20:11:29.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.859+0000 7fe99da87800 20 remove_watcher() i=2 2024-04-22T20:11:29.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.860+0000 7fe99da87800 20 remove_watcher() i=3 2024-04-22T20:11:29.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.861+0000 7fe99da87800 20 remove_watcher() i=4 2024-04-22T20:11:29.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.862+0000 7fe99da87800 20 remove_watcher() i=5 2024-04-22T20:11:29.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.863+0000 7fe99da87800 20 remove_watcher() i=6 2024-04-22T20:11:29.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.864+0000 7fe99da87800 20 remove_watcher() i=7 2024-04-22T20:11:29.873 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:11:29.921 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:11:29.921 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:11:29.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.956+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.957+0000 7f51a5e15800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.957+0000 7f51a5e15800 20 realm 2024-04-22T20:11:29.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.957+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.957+0000 7f51a5e15800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.957+0000 7f51a5e15800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:11:29.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.957+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.958+0000 7f51a5e15800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.958+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.958+0000 7f51a5e15800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:11:29.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.958+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.959+0000 7f51a5e15800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:11:29.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.959+0000 7f51a5e15800 20 searching for the correct realm 2024-04-22T20:11:29.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.961+0000 7f51a1bdd640 0 -- 172.21.15.136:0/338250701 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x562c0188f1b0 msgr2=0x562c018915e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:11:29.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.964+0000 7f51a1bdd640 0 -- 172.21.15.136:0/338250701 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f506c07ba00 msgr2=0x7f506c07de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:11:29.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:11:29.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:11:29.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:11:29.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:11:29.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:11:29.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:11:29.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:11:29.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.970+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:11:29.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 zone default found 2024-04-22T20:11:29.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 4 Realm: () 2024-04-22T20:11:29.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:11:29.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 zonegroup default 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 rados->read ofs=0 len=0 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 started sync module instance, tier type = 2024-04-22T20:11:29.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.971+0000 7f51a5e15800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:11:29.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.980+0000 7f51a5e15800 20 add_watcher() i=0 2024-04-22T20:11:29.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.980+0000 7f51a5e15800 20 add_watcher() i=1 2024-04-22T20:11:29.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.980+0000 7f51a5e15800 20 add_watcher() i=2 2024-04-22T20:11:29.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.980+0000 7f51a5e15800 20 add_watcher() i=3 2024-04-22T20:11:29.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.980+0000 7f51a5e15800 20 add_watcher() i=4 2024-04-22T20:11:29.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.980+0000 7f51a5e15800 20 add_watcher() i=5 2024-04-22T20:11:29.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.980+0000 7f51a5e15800 20 add_watcher() i=6 2024-04-22T20:11:29.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.981+0000 7f51a5e15800 20 add_watcher() i=7 2024-04-22T20:11:29.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.981+0000 7f51a5e15800 2 all 8 watchers are set, enabling cache 2024-04-22T20:11:29.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.983+0000 7f50577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:11:29.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.983+0000 7f51a5e15800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:11:29.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:29.983+0000 7f51a5e15800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:11:30.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.015+0000 7f50567e4640 20 reqs_thread_entry: start 2024-04-22T20:11:30.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.015+0000 7f51a5e15800 20 init_complete bucket index max shards: 1 2024-04-22T20:11:30.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.015+0000 7f51a5e15800 20 Filter name: none 2024-04-22T20:11:30.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.015+0000 7f4fc7fff640 20 reqs_thread_entry: start 2024-04-22T20:11:30.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.015+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:11:30.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.016+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.017+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.017+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.018+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:11:30.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.019+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.020+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.020+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.021+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:11:30.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.023+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.023+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.023+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.024+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:11:30.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.026+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.026+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.027+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.027+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:11:30.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.029+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.029+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.029+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.030+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:11:30.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.032+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.032+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.033+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.034+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:11:30.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.035+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.035+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.036+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.037+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:11:30.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.038+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.038+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.039+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.039+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:11:30.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.041+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.041+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.041+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.042+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:11:30.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.043+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.044+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.044+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.045+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:11:30.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.046+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.047+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.047+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.048+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:11:30.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.049+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.050+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.050+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.051+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:11:30.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.052+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.053+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.053+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.054+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:11:30.054 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.055+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.056+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.056+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.057+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:11:30.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.058+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.059+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.059+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.060+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:11:30.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.061+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.061+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.062+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.062+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:11:30.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.064+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.064+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.064+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.065+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:11:30.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.067+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.067+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.067+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.068+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:11:30.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.070+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.070+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.070+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.071+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:11:30.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.072+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.073+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.073+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.074+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:11:30.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.075+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.076+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.076+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.077+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:11:30.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.078+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.079+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.079+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.080+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:11:30.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.081+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.082+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.082+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.083+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:11:30.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.084+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.084+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.085+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.086+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:11:30.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.087+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.088+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.088+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.089+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:11:30.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.090+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.091+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.091+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.092+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:11:30.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.093+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.094+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.094+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.095+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:11:30.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.096+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.097+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.097+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.098+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:11:30.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.099+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.100+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.100+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.101+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:11:30.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.102+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.103+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.103+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.104+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:11:30.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.105+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.106+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.106+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.107+0000 7f51a5e15800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:11:30.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.108+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.109+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:11:30.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.109+0000 7f51a5e15800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:11:30.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.110+0000 7f50577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:11:30.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.112+0000 7f51a5e15800 20 remove_watcher() i=0 2024-04-22T20:11:30.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.112+0000 7f51a5e15800 2 removed watcher, disabling cache 2024-04-22T20:11:30.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.113+0000 7f51a5e15800 20 remove_watcher() i=1 2024-04-22T20:11:30.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.114+0000 7f51a5e15800 20 remove_watcher() i=2 2024-04-22T20:11:30.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.115+0000 7f51a5e15800 20 remove_watcher() i=3 2024-04-22T20:11:30.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.116+0000 7f51a5e15800 20 remove_watcher() i=4 2024-04-22T20:11:30.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.117+0000 7f51a5e15800 20 remove_watcher() i=5 2024-04-22T20:11:30.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.118+0000 7f51a5e15800 20 remove_watcher() i=6 2024-04-22T20:11:30.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:11:30.119+0000 7f51a5e15800 20 remove_watcher() i=7 2024-04-22T20:11:30.131 DEBUG:teuthology.orchestra.run.smithi136:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2024-04-22T20:11:30.607 INFO:teuthology.orchestra.run.smithi136.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2024-04-22T20:11:34.207 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :clean UP-TO-DATE 2024-04-22T20:11:58.306 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :compileJava 2024-04-22T20:11:58.306 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :processResources 2024-04-22T20:11:58.306 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :classes 2024-04-22T20:11:59.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:59.005 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :compileTestJava 2024-04-22T20:11:59.006 INFO:teuthology.orchestra.run.smithi136.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-04-22T20:11:59.006 INFO:teuthology.orchestra.run.smithi136.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-04-22T20:11:59.007 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:11:59.007 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :processTestResources NO-SOURCE 2024-04-22T20:11:59.007 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :testClasses 2024-04-22T20:12:04.906 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:04.906 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :test 2024-04-22T20:12:04.906 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:04.906 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2024-04-22T20:12:10.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:10.006 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2024-04-22T20:12:10.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:10.305 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2024-04-22T20:12:10.505 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:10.505 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2024-04-22T20:12:10.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:10.805 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2024-04-22T20:12:19.405 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:19.405 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2024-04-22T20:12:19.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:19.805 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2024-04-22T20:12:19.806 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:19.806 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2024-04-22T20:12:19.906 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:19.906 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2024-04-22T20:12:20.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:20.005 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2024-04-22T20:12:20.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:20.305 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2024-04-22T20:12:34.306 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:34.306 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2024-04-22T20:12:34.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:34.905 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2024-04-22T20:12:35.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:35.105 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STANDARD_OUT 2024-04-22T20:12:35.106 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22 20:12:35,048 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-04-22T20:12:59.406 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:12:59.406 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2024-04-22T20:13:00.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:00.006 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2024-04-22T20:13:00.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:00.006 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2024-04-22T20:13:00.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:00.305 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2024-04-22T20:13:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:00.405 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2024-04-22T20:13:00.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:00.906 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2024-04-22T20:13:06.405 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:06.406 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2024-04-22T20:13:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:06.705 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2024-04-22T20:13:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:06.805 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2024-04-22T20:13:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:06.905 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2024-04-22T20:13:14.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:14.705 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2024-04-22T20:13:16.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:16.006 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests STANDARD_OUT 2024-04-22T20:13:16.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22 20:13:15,991 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-04-22T20:13:16.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22 20:13:15,994 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-04-22T20:13:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:16.305 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2024-04-22T20:13:35.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:35.006 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2024-04-22T20:13:35.505 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:35.505 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2024-04-22T20:13:38.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:38.006 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2024-04-22T20:13:38.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:13:38.306 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2024-04-22T20:15:03.806 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:03.806 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2024-04-22T20:15:04.205 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:04.205 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2024-04-22T20:15:12.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:12.106 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2024-04-22T20:15:12.405 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:12.405 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2024-04-22T20:15:17.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:17.306 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2024-04-22T20:15:17.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:17.705 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2024-04-22T20:15:19.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:19.605 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2024-04-22T20:15:21.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:21.806 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2024-04-22T20:15:22.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:22.005 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2024-04-22T20:15:22.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:22.705 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2024-04-22T20:15:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:23.705 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2024-04-22T20:15:24.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:24.005 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2024-04-22T20:15:24.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:24.105 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2024-04-22T20:15:24.505 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:24.506 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2024-04-22T20:15:24.506 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:24.506 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2024-04-22T20:15:24.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:24.905 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2024-04-22T20:15:25.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:25.005 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2024-04-22T20:15:25.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:25.305 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2024-04-22T20:15:25.405 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:25.405 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2024-04-22T20:15:25.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:25.705 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2024-04-22T20:15:25.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:25.805 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2024-04-22T20:15:26.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:26.105 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2024-04-22T20:15:38.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:38.005 INFO:teuthology.orchestra.run.smithi136.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout:---------------------------------------------------------------------- 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout:---------------------------------------------------------------------- 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:38.487 INFO:teuthology.orchestra.run.smithi136.stdout:BUILD SUCCESSFUL in 4m 8s 2024-04-22T20:15:38.488 INFO:teuthology.orchestra.run.smithi136.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2024-04-22T20:15:38.803 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:15:38.852 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:15:38.853 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:15:38.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.887+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:38.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.888+0000 7f6764336800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:38.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.888+0000 7f6764336800 20 realm 2024-04-22T20:15:38.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.888+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:38.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.889+0000 7f6764336800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:38.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.889+0000 7f6764336800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:15:38.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.889+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:38.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.889+0000 7f6764336800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:38.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.889+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:38.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.890+0000 7f6764336800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:38.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.890+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:38.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.890+0000 7f6764336800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:15:38.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.890+0000 7f6764336800 20 searching for the correct realm 2024-04-22T20:15:38.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.891+0000 7f67600cc640 0 -- 172.21.15.136:0/3866766394 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x56229b6d5440 msgr2=0x56229b6d7900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:38.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:38.895+0000 7f67600cc640 0 -- 172.21.15.136:0/3866766394 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x56229b524760 msgr2=0x7f675808e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.104+0000 7f6764336800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.104+0000 7f6764336800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.104+0000 7f6764336800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.104+0000 7f6764336800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.104+0000 7f6764336800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.104+0000 7f6764336800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.104+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 20 zone default found 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 4 Realm: () 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 20 zonegroup default 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.105+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.106+0000 7f6764336800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.106+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.106+0000 7f6764336800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.106+0000 7f6764336800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.106+0000 7f6764336800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.106+0000 7f6764336800 20 started sync module instance, tier type = 2024-04-22T20:15:39.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.106+0000 7f6764336800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:15:39.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.116+0000 7f6764336800 20 add_watcher() i=0 2024-04-22T20:15:39.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.116+0000 7f6764336800 20 add_watcher() i=1 2024-04-22T20:15:39.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.116+0000 7f6764336800 20 add_watcher() i=2 2024-04-22T20:15:39.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.116+0000 7f6764336800 20 add_watcher() i=3 2024-04-22T20:15:39.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.116+0000 7f6764336800 20 add_watcher() i=4 2024-04-22T20:15:39.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.116+0000 7f6764336800 20 add_watcher() i=5 2024-04-22T20:15:39.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.116+0000 7f6764336800 20 add_watcher() i=6 2024-04-22T20:15:39.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.117+0000 7f6764336800 20 add_watcher() i=7 2024-04-22T20:15:39.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.117+0000 7f6764336800 2 all 8 watchers are set, enabling cache 2024-04-22T20:15:39.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.119+0000 7f6616fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:39.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.119+0000 7f6764336800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:15:39.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.119+0000 7f6764336800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:15:39.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.155+0000 7f6615fdb640 20 reqs_thread_entry: start 2024-04-22T20:15:39.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.155+0000 7f6764336800 20 init_complete bucket index max shards: 1 2024-04-22T20:15:39.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.155+0000 7f6764336800 20 Filter name: none 2024-04-22T20:15:39.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.155+0000 7f65877fe640 20 reqs_thread_entry: start 2024-04-22T20:15:39.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.156+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:15:39.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.157+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.157+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7835363498191825096', time=2024-04-22T22:15:17.426428+0000, chain.objs.size()=9 2024-04-22T20:15:39.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__multipart_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.1 2024-04-22T20:15:39.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__shadow_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.1_1 2024-04-22T20:15:39.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__multipart_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.2 2024-04-22T20:15:39.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__shadow_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.2_1 2024-04-22T20:15:39.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__multipart_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.3 2024-04-22T20:15:39.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__shadow_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.3_1 2024-04-22T20:15:39.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__multipart_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.4 2024-04-22T20:15:39.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__shadow_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.4_1 2024-04-22T20:15:39.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.158+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.46__multipart_key1.2~RbWOQOkSAyp-iKi0quTFxkXHNOM6AS2.5 2024-04-22T20:15:39.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.162+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.164+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:15:39.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.165+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4958145193574288545', time=2024-04-22T22:12:56.503205+0000, chain.objs.size()=7 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~ixI9CSjJtviFcAdrcVUineQozGiBS66.1 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~ixI9CSjJtviFcAdrcVUineQozGiBS66.1_1 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~ixI9CSjJtviFcAdrcVUineQozGiBS66.1_2 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~ixI9CSjJtviFcAdrcVUineQozGiBS66.2 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~ixI9CSjJtviFcAdrcVUineQozGiBS66.2_1 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~ixI9CSjJtviFcAdrcVUineQozGiBS66.2_2 2024-04-22T20:15:39.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.166+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~ixI9CSjJtviFcAdrcVUineQozGiBS66.3 2024-04-22T20:15:39.170 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.170+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.172+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:15:39.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.174+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.174+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.174+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.174+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.319405037453307831', time=2024-04-22T22:15:03.713099+0000, chain.objs.size()=16 2024-04-22T20:15:39.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.174+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.1 2024-04-22T20:15:39.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.1_1 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.1_2 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.2 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.2_1 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.2_2 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.3 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.3_1 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.3_2 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.4 2024-04-22T20:15:39.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.175+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.4_1 2024-04-22T20:15:39.177 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.176+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.4_2 2024-04-22T20:15:39.178 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.178+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.5 2024-04-22T20:15:39.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.180+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.5_1 2024-04-22T20:15:39.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.180+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.5_2 2024-04-22T20:15:39.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.180+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~-j25M276gKcnHxCUpeqUIryGU-UeuKa.6 2024-04-22T20:15:39.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.184+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.186+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:15:39.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.188+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.188+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.189+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.189+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10061644369786376699', time=2024-04-22T22:15:22.171679+0000, chain.objs.size()=1 2024-04-22T20:15:39.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.189+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.48__shadow_.7ewB0QvNjyiSh4yVaUzssDCTzvfdvg8_0 2024-04-22T20:15:39.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.190+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.192+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:15:39.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.193+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J', time=2024-04-22T22:15:20.890653+0000, chain.objs.size()=20 2024-04-22T20:15:39.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.1 2024-04-22T20:15:39.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.2 2024-04-22T20:15:39.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.3 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.4 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.5 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.6 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.7 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.8 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.9 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.10 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.11 2024-04-22T20:15:39.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.194+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.12 2024-04-22T20:15:39.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.197+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.13 2024-04-22T20:15:39.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.197+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.14 2024-04-22T20:15:39.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.197+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.15 2024-04-22T20:15:39.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.198+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.16 2024-04-22T20:15:39.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.198+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.17 2024-04-22T20:15:39.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.198+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.18 2024-04-22T20:15:39.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.198+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.19 2024-04-22T20:15:39.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.198+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.47__multipart_key1.2~d3HYGmyryxEd3Dnyk-5-mbbBqKIrU1J.20 2024-04-22T20:15:39.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.203+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.207+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:15:39.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.208+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.209+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.209+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:15:39.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.209+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4817010929066877464', time=2024-04-22T22:13:35.263590+0000, chain.objs.size()=1 2024-04-22T20:15:39.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.209+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_.vtv85rQr43VejyM0V8ANlgVENRUFj9P_0 2024-04-22T20:15:39.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.209+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9684836499190835610', time=2024-04-22T22:15:25.490997+0000, chain.objs.size()=1 2024-04-22T20:15:39.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.209+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.53__shadow_.RpEIMwkik3FIaOLDPbc6o4ITC-zS2BL_0 2024-04-22T20:15:39.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.211+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.213+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:15:39.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.214+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:15:39.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.1_1_7463b9b8f13cddd5550ccbd2acd64c2c', time=2024-04-22T22:12:34.389923+0000, chain.objs.size()=9 2024-04-22T20:15:39.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.1 2024-04-22T20:15:39.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.1_1 2024-04-22T20:15:39.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.2 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.2_1 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.3 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.3_1 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.4 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.4_1 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.5 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.525912475659726394', time=2024-04-22T22:14:00.481373+0000, chain.objs.size()=21 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.1 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.1_1 2024-04-22T20:15:39.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.215+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.2 2024-04-22T20:15:39.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.218+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.2_1 2024-04-22T20:15:39.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.218+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.3 2024-04-22T20:15:39.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.219+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.3_1 2024-04-22T20:15:39.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.219+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.4 2024-04-22T20:15:39.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.219+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.4_1 2024-04-22T20:15:39.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.219+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.5 2024-04-22T20:15:39.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.219+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.5_1 2024-04-22T20:15:39.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.219+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.6 2024-04-22T20:15:39.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.219+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.6_1 2024-04-22T20:15:39.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.219+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.7 2024-04-22T20:15:39.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.220+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.7_1 2024-04-22T20:15:39.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.223+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.8 2024-04-22T20:15:39.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.224+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.8_1 2024-04-22T20:15:39.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.224+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.9 2024-04-22T20:15:39.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.224+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.9_1 2024-04-22T20:15:39.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.225+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.10 2024-04-22T20:15:39.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.225+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.10_1 2024-04-22T20:15:39.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.225+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~yeER_pIBRVhL0ZGmcIKIUC_Bj4eOARU.11 2024-04-22T20:15:39.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.229+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.231+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:15:39.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.232+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.233+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.233+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.234+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:15:39.236 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.236 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11885761240225111420', time=2024-04-22T22:13:06.477163+0000, chain.objs.size()=9 2024-04-22T20:15:39.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__multipart_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.1 2024-04-22T20:15:39.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__shadow_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.1_1 2024-04-22T20:15:39.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__multipart_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.2 2024-04-22T20:15:39.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__shadow_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.2_1 2024-04-22T20:15:39.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__multipart_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.3 2024-04-22T20:15:39.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__shadow_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.3_1 2024-04-22T20:15:39.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__multipart_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.4 2024-04-22T20:15:39.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__shadow_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.4_1 2024-04-22T20:15:39.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.236+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.40__multipart_key1.2~FG12V-cUN7PkzL-LeEo3Fac6Bq5jijK.5 2024-04-22T20:15:39.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.240+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.243+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:15:39.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.244+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.244+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.245+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.246+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:15:39.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.247+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.248+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.248+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.249+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:15:39.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.250+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1454330256881496', time=2024-04-22T22:12:59.710697+0000, chain.objs.size()=6 2024-04-22T20:15:39.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.35__shadow_.ZRXFaC_wYGAYUHafSmT0A7BEint3lL-_0 2024-04-22T20:15:39.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.35__shadow_.ZRXFaC_wYGAYUHafSmT0A7BEint3lL-_1 2024-04-22T20:15:39.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.35__shadow_.ZRXFaC_wYGAYUHafSmT0A7BEint3lL-_2 2024-04-22T20:15:39.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.35__shadow_.ZRXFaC_wYGAYUHafSmT0A7BEint3lL-_3 2024-04-22T20:15:39.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.35__shadow_.ZRXFaC_wYGAYUHafSmT0A7BEint3lL-_4 2024-04-22T20:15:39.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.251+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.35__shadow_.ZRXFaC_wYGAYUHafSmT0A7BEint3lL-_5 2024-04-22T20:15:39.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.254+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.257+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:15:39.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.258+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10049166150898411789', time=2024-04-22T22:12:34.620897+0000, chain.objs.size()=9 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.1 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.1_1 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.2 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.2_1 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.3 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.3_1 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.4 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__shadow_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.4_1 2024-04-22T20:15:39.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.259+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.33__multipart_key1.2~0NlsqTUU-iF-YkxBfq0ytiQ5Zycmb4b.5 2024-04-22T20:15:39.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.261+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.263+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:15:39.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.265+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.265+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.265+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.266+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:15:39.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.268+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.268+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.268+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.268+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2854772284133846332', time=2024-04-22T22:13:35.298715+0000, chain.objs.size()=1 2024-04-22T20:15:39.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.268+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_.AjjBCZXqSumyT_MlJJxuCYq0goLUvpR_0 2024-04-22T20:15:39.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.269+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.272+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:15:39.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.273+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565476765572953000', time=2024-04-22T22:12:19.549626+0000, chain.objs.size()=9 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__multipart_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.1 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__shadow_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.1_1 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__multipart_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.2 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__shadow_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.2_1 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__multipart_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.3 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__shadow_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.3_1 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__multipart_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.4 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__shadow_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.4_1 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31__multipart_key1.2~6Id3Eo0Ikk1L4Nr5rKPU2YtI6l9G2NU.5 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.674307076271607680', time=2024-04-22T22:15:12.138080+0000, chain.objs.size()=9 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__multipart_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.1 2024-04-22T20:15:39.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__shadow_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.1_1 2024-04-22T20:15:39.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.274+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__multipart_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.2 2024-04-22T20:15:39.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.276+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__shadow_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.2_1 2024-04-22T20:15:39.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.277+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__multipart_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.3 2024-04-22T20:15:39.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.279+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__shadow_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.3_1 2024-04-22T20:15:39.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.282+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__multipart_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.4 2024-04-22T20:15:39.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.282+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__shadow_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.4_1 2024-04-22T20:15:39.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.282+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__multipart_key1.2~iZjQocPzVRW5H75NqIEfbF6DNoHtkEw.5 2024-04-22T20:15:39.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.286+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.288+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:15:39.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.289+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.290 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3084348331122836121', time=2024-04-22T22:13:35.154491+0000, chain.objs.size()=9 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.1 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.1_1 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.2 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.2_1 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.3 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.3_1 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.4 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.4_1 2024-04-22T20:15:39.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.290+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file.mpg.2~d0apNxSJ4ohzE14i2u-xS7C2uGsXAog.5 2024-04-22T20:15:39.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.297+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.299 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.299+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:15:39.302 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.302+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.303+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.303+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.303+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7544779478052388032', time=2024-04-22T22:15:24.643799+0000, chain.objs.size()=1 2024-04-22T20:15:39.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.303+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.51__shadow_.49SI6OxG2eoS1KoIWEFDnxkwv4vWpCt_0 2024-04-22T20:15:39.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.304+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.310 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.308+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:15:39.310 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.309+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.309+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.309+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.310+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:15:39.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.311+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:15:39.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11536578620036135320', time=2024-04-22T22:12:50.369923+0000, chain.objs.size()=9 2024-04-22T20:15:39.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.1 2024-04-22T20:15:39.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.1_1 2024-04-22T20:15:39.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.2 2024-04-22T20:15:39.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.2_1 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.3 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.3_1 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.4 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.4_1 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~7Gc1nyiMyT5i6ca7Q_guY60fUpuKUj4.5 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7924260396149440800', time=2024-04-22T22:14:22.255252+0000, chain.objs.size()=19 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.1 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.1_1 2024-04-22T20:15:39.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.312+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.2 2024-04-22T20:15:39.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.315+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.2_1 2024-04-22T20:15:39.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.317+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.3 2024-04-22T20:15:39.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.317+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.3_1 2024-04-22T20:15:39.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.317+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.4 2024-04-22T20:15:39.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.317+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.4_1 2024-04-22T20:15:39.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.317+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.5 2024-04-22T20:15:39.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.317+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.5_1 2024-04-22T20:15:39.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.317+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.6 2024-04-22T20:15:39.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.319+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.6_1 2024-04-22T20:15:39.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.319+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.7 2024-04-22T20:15:39.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.319+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.7_1 2024-04-22T20:15:39.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.319+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.8 2024-04-22T20:15:39.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.319+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.8_1 2024-04-22T20:15:39.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.320+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.9 2024-04-22T20:15:39.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.320+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.9_1 2024-04-22T20:15:39.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.322+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~sRYkJ_Dq5_aqxb4HVbXzZcDt0ng5zce.10 2024-04-22T20:15:39.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.326+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.328 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.328+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:15:39.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.329+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.330+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.330+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.331+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:15:39.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.332+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.332+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15585096865377837496', time=2024-04-22T22:14:11.722709+0000, chain.objs.size()=21 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.1 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.1_1 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.2 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.2_1 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.3 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.3_1 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.4 2024-04-22T20:15:39.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.4_1 2024-04-22T20:15:39.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.5 2024-04-22T20:15:39.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.5_1 2024-04-22T20:15:39.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.6 2024-04-22T20:15:39.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.333+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.6_1 2024-04-22T20:15:39.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.335+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.7 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.7_1 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.8 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.8_1 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.9 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.9_1 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.10 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.10_1 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~XoK-e__3DBSTn8HTSiTIP_gbF0PxRvW.11 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9970619189357180444', time=2024-04-22T22:14:33.253695+0000, chain.objs.size()=16 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.1 2024-04-22T20:15:39.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.337+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.1_1 2024-04-22T20:15:39.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.340+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.1_2 2024-04-22T20:15:39.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.341+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.2 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.341+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.2_1 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.342+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.2_2 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.342+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.3 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.342+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.3_1 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.342+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.3_2 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.342+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.4 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.342+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.4_1 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.342+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.4_2 2024-04-22T20:15:39.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.342+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.5 2024-04-22T20:15:39.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.346+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.5_1 2024-04-22T20:15:39.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.346+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.5_2 2024-04-22T20:15:39.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.347+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~rqIKB2V0-V2f4tZlQo_1lHbquflNKwY.6 2024-04-22T20:15:39.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.352+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.354+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:15:39.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.355+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6938671125578502467', time=2024-04-22T22:13:15.989800+0000, chain.objs.size()=9 2024-04-22T20:15:39.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__multipart_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.1 2024-04-22T20:15:39.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__shadow_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.1_1 2024-04-22T20:15:39.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__multipart_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.2 2024-04-22T20:15:39.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__shadow_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.2_1 2024-04-22T20:15:39.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__multipart_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.3 2024-04-22T20:15:39.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__shadow_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.3_1 2024-04-22T20:15:39.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__multipart_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.4 2024-04-22T20:15:39.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__shadow_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.4_1 2024-04-22T20:15:39.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.356+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.41__multipart_key1.2~xQUnzYCACl1euXaucwo2zYMQcqlEfhb.5 2024-04-22T20:15:39.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.360+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.362+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:15:39.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.363+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.364+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.364+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:15:39.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.364+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15417333949589170978', time=2024-04-22T22:13:35.188186+0000, chain.objs.size()=1 2024-04-22T20:15:39.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.364+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_.bvQYev8wJMmXVD-6vpmq4_BfpIB5Id2_0 2024-04-22T20:15:39.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.364+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5791754371006792878', time=2024-04-22T22:15:25.068285+0000, chain.objs.size()=1 2024-04-22T20:15:39.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.364+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.52__shadow_.WlClXmk1wiuNxsbBdLNo44KKZnJhRRd_0 2024-04-22T20:15:39.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.366+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.368+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:15:39.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.369+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.369+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:15:39.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6301136451191041147', time=2024-04-22T22:12:59.479809+0000, chain.objs.size()=7 2024-04-22T20:15:39.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~b0alXqKHBGFElAYH_8OiZPjW-2f06Up.1 2024-04-22T20:15:39.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~b0alXqKHBGFElAYH_8OiZPjW-2f06Up.1_1 2024-04-22T20:15:39.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~b0alXqKHBGFElAYH_8OiZPjW-2f06Up.1_2 2024-04-22T20:15:39.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~b0alXqKHBGFElAYH_8OiZPjW-2f06Up.2 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~b0alXqKHBGFElAYH_8OiZPjW-2f06Up.2_1 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~b0alXqKHBGFElAYH_8OiZPjW-2f06Up.2_2 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~b0alXqKHBGFElAYH_8OiZPjW-2f06Up.3 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14612352862324807186', time=2024-04-22T22:15:03.931881+0000, chain.objs.size()=16 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.1 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.1_1 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.1_2 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.2 2024-04-22T20:15:39.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.370+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.2_1 2024-04-22T20:15:39.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.374+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.2_2 2024-04-22T20:15:39.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.374+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.3 2024-04-22T20:15:39.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.374+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.3_1 2024-04-22T20:15:39.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.374+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.3_2 2024-04-22T20:15:39.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.374+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.4 2024-04-22T20:15:39.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.374+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.4_1 2024-04-22T20:15:39.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.374+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.4_2 2024-04-22T20:15:39.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.376+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.5 2024-04-22T20:15:39.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.376+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.5_1 2024-04-22T20:15:39.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.376+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__shadow_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.5_2 2024-04-22T20:15:39.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.376+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44__multipart_key1.2~NCkdU821BjYVACch2pRMNRa6RrdnZVp.6 2024-04-22T20:15:39.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.381+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.383+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:15:39.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.384+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.384+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18384034240810277148', time=2024-04-22T22:12:46.532812+0000, chain.objs.size()=9 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.1 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.1_1 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.2 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.2_1 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.3 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.3_1 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.4 2024-04-22T20:15:39.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.4_1 2024-04-22T20:15:39.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.385+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~g3euL7pl5ErDuOtqdHDDztKghxIMunv.5 2024-04-22T20:15:39.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.389+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.391 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.391+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:15:39.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.392+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.393+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.393+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.393+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11040372004891964707', time=2024-04-22T22:15:09.072447+0000, chain.objs.size()=1 2024-04-22T20:15:39.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.393+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.45__shadow_.ykYvUt_BMMjK7lr-0yECq8-a090u6mB_0 2024-04-22T20:15:39.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.394+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.396+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:15:39.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.397+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:15:39.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16399959132659671195', time=2024-04-22T22:12:59.264830+0000, chain.objs.size()=7 2024-04-22T20:15:39.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~SfozkLiUeKwT4H6j3wUUjV8DEWuU9im.1 2024-04-22T20:15:39.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~SfozkLiUeKwT4H6j3wUUjV8DEWuU9im.1_1 2024-04-22T20:15:39.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~SfozkLiUeKwT4H6j3wUUjV8DEWuU9im.1_2 2024-04-22T20:15:39.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~SfozkLiUeKwT4H6j3wUUjV8DEWuU9im.2 2024-04-22T20:15:39.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~SfozkLiUeKwT4H6j3wUUjV8DEWuU9im.2_1 2024-04-22T20:15:39.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~SfozkLiUeKwT4H6j3wUUjV8DEWuU9im.2_2 2024-04-22T20:15:39.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~SfozkLiUeKwT4H6j3wUUjV8DEWuU9im.3 2024-04-22T20:15:39.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3488213784002629408', time=2024-04-22T22:15:23.805160+0000, chain.objs.size()=1 2024-04-22T20:15:39.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.49__shadow_.rO6beFS5rvtfE6GNYTXt_eSjJRoKb7__0 2024-04-22T20:15:39.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3389843582936116152', time=2024-04-22T22:15:38.163722+0000, chain.objs.size()=21 2024-04-22T20:15:39.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.1 2024-04-22T20:15:39.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.1_1 2024-04-22T20:15:39.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.2 2024-04-22T20:15:39.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.398+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.2_1 2024-04-22T20:15:39.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.401+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.3 2024-04-22T20:15:39.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.401+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.3_1 2024-04-22T20:15:39.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.402+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.4 2024-04-22T20:15:39.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.403+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.4_1 2024-04-22T20:15:39.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.403+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.5 2024-04-22T20:15:39.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.403+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.5_1 2024-04-22T20:15:39.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.403+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.6 2024-04-22T20:15:39.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.403+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.6_1 2024-04-22T20:15:39.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.403+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.7 2024-04-22T20:15:39.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.403+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.7_1 2024-04-22T20:15:39.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.403+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.8 2024-04-22T20:15:39.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.405+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.8_1 2024-04-22T20:15:39.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.408+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.9 2024-04-22T20:15:39.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.408+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.9_1 2024-04-22T20:15:39.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.408+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.10 2024-04-22T20:15:39.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.408+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__shadow_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.10_1 2024-04-22T20:15:39.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.408+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.55__multipart_key1.2~9UpYT73H9euDBd9vgbZh-L4f3JfxnHQ.11 2024-04-22T20:15:39.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.412+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.414+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:15:39.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.415+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07', time=2024-04-22T22:12:09.958941+0000, chain.objs.size()=9 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__multipart_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.1 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__shadow_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.1_1 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__multipart_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.2 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__shadow_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.2_1 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__multipart_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.3 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__shadow_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.3_1 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__multipart_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.4 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__shadow_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.4_1 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.29__multipart_key1.2~Euc_lJHEfUg6fEZewAAT8BxcTIuo-07.5 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14880062263840110679', time=2024-04-22T22:12:53.157497+0000, chain.objs.size()=9 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.1 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.1_1 2024-04-22T20:15:39.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.416+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.2 2024-04-22T20:15:39.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.420+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.2_1 2024-04-22T20:15:39.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.420+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.3 2024-04-22T20:15:39.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.420+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.3_1 2024-04-22T20:15:39.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.420+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.4 2024-04-22T20:15:39.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.420+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__shadow_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.4_1 2024-04-22T20:15:39.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.420+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36__multipart_key1.2~9uShpvzSeCT8AIFAf90plE1s0nNDAJO.5 2024-04-22T20:15:39.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.421+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4168692136621393826', time=2024-04-22T22:15:24.227911+0000, chain.objs.size()=1 2024-04-22T20:15:39.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.421+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.50__shadow_.WnxmxTqqzgeXXACA_unaHZ6BuKrATYu_0 2024-04-22T20:15:39.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.424+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.427+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:15:39.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.428+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.428+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16651938333875825676', time=2024-04-22T22:13:35.225984+0000, chain.objs.size()=9 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.1 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.1_1 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.2 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.2_1 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.3 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.3_1 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.4 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__shadow_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.4_1 2024-04-22T20:15:39.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.429+0000 7f6764336800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.42__multipart_file2.mpg.2~Zp6wgXn843LYL1hifjr6a4ky6pvOdEV.5 2024-04-22T20:15:39.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.433+0000 7f6764336800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:39.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.435+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:15:39.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.437+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.437+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.437+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.438+0000 7f6764336800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:15:39.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.439+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.440+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:39.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.440+0000 7f6764336800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:39.442 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.442+0000 7f6616fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:39.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.444+0000 7f6764336800 20 remove_watcher() i=0 2024-04-22T20:15:39.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.444+0000 7f6764336800 2 removed watcher, disabling cache 2024-04-22T20:15:39.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.445+0000 7f6764336800 20 remove_watcher() i=1 2024-04-22T20:15:39.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.446+0000 7f6764336800 20 remove_watcher() i=2 2024-04-22T20:15:39.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.447+0000 7f6764336800 20 remove_watcher() i=3 2024-04-22T20:15:39.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.447+0000 7f6764336800 20 remove_watcher() i=4 2024-04-22T20:15:39.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.448+0000 7f6764336800 20 remove_watcher() i=5 2024-04-22T20:15:39.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.449+0000 7f6764336800 20 remove_watcher() i=6 2024-04-22T20:15:39.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.450+0000 7f6764336800 20 remove_watcher() i=7 2024-04-22T20:15:39.461 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:15:39.509 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:15:39.510 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:15:39.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.543+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.545+0000 7f29ff325800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.545+0000 7f29ff325800 20 realm 2024-04-22T20:15:39.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.545+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.545+0000 7f29ff325800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.545+0000 7f29ff325800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:15:39.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.545+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.545+0000 7f29ff325800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.545+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.546+0000 7f29ff325800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:39.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.546+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.546+0000 7f29ff325800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:15:39.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.546+0000 7f29ff325800 20 searching for the correct realm 2024-04-22T20:15:39.548 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.548+0000 7f29fb8b8640 0 -- 172.21.15.136:0/3688277650 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f29f4018040 msgr2=0x7f29f4063170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:39.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.754+0000 7f29f28b6640 0 -- 172.21.15.136:0/3688277650 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f29f4078b60 msgr2=0x7f29f40771d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:39.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:15:39.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:15:39.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:15:39.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:15:39.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:15:39.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:15:39.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:39.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.960+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:15:39.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 20 zone default found 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 4 Realm: () 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 20 zonegroup default 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.961+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.962+0000 7f29ff325800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.962+0000 7f29ff325800 20 rados->read ofs=0 len=0 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.962+0000 7f29ff325800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.962+0000 7f29ff325800 20 started sync module instance, tier type = 2024-04-22T20:15:39.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.962+0000 7f29ff325800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:15:39.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.971+0000 7f29ff325800 20 add_watcher() i=0 2024-04-22T20:15:39.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.971+0000 7f29ff325800 20 add_watcher() i=1 2024-04-22T20:15:39.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.971+0000 7f29ff325800 20 add_watcher() i=2 2024-04-22T20:15:39.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.971+0000 7f29ff325800 20 add_watcher() i=3 2024-04-22T20:15:39.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.971+0000 7f29ff325800 20 add_watcher() i=4 2024-04-22T20:15:39.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.971+0000 7f29ff325800 20 add_watcher() i=5 2024-04-22T20:15:39.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.971+0000 7f29ff325800 20 add_watcher() i=6 2024-04-22T20:15:39.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.972+0000 7f29ff325800 20 add_watcher() i=7 2024-04-22T20:15:39.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.972+0000 7f29ff325800 2 all 8 watchers are set, enabling cache 2024-04-22T20:15:39.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.973+0000 7f28b1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:39.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.974+0000 7f29ff325800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:15:39.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:39.974+0000 7f29ff325800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:15:40.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.009+0000 7f28b0fe1640 20 reqs_thread_entry: start 2024-04-22T20:15:40.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.010+0000 7f29ff325800 20 init_complete bucket index max shards: 1 2024-04-22T20:15:40.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.010+0000 7f29ff325800 20 Filter name: none 2024-04-22T20:15:40.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.010+0000 7f282a7fc640 20 reqs_thread_entry: start 2024-04-22T20:15:40.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.010+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:15:40.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.011+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.011+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.012+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.013 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.013+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:15:40.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.014+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.014+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.015+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.015+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:15:40.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.017+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.017+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.017+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.018+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:15:40.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.019+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.020+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.020+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.021+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:15:40.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.022+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.023+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.023+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.024+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:15:40.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.026+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.026+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.026+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.027+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:15:40.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.028+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.029+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.029+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.030+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:15:40.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.031+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.032+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.032+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.033+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:15:40.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.035+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.035+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.035+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.036+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:15:40.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.038+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.038+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.039+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.040+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:15:40.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.041+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.042+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.042+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.043+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:15:40.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.044+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.045+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.045+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.046+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:15:40.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.047+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.048+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.048+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.049+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:15:40.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.050+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.051+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.051+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.052+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:15:40.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.053+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.053 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.054+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.054 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.054+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.055+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:15:40.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.056+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.056+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.057+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.058+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:15:40.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.059+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.059+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.060+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.061+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:15:40.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.062+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.062+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.063+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.064+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:15:40.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.065+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.066+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.066+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.067+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:15:40.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.069+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.069+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.069+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.070+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:15:40.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.072+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.073+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.073+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.074+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:15:40.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.075+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.075+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.076+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.077+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:15:40.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.078+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.078+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.079+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.080+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:15:40.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.081+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.082+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.082+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.083+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:15:40.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.084+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.085+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.085+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.086+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:15:40.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.087+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.088+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.088+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.089+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:15:40.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.090+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.091+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.091+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.092+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:15:40.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.094+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.094+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.095+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.095+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:15:40.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.097+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.097+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.098+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.099+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:15:40.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.100+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.101+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.101+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.102+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:15:40.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.103+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.103+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.104+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.104+0000 7f29ff325800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:15:40.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.106+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.107+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.107+0000 7f29ff325800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.109+0000 7f28b1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:40.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.111+0000 7f29ff325800 20 remove_watcher() i=0 2024-04-22T20:15:40.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.111+0000 7f29ff325800 2 removed watcher, disabling cache 2024-04-22T20:15:40.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.112+0000 7f29ff325800 20 remove_watcher() i=1 2024-04-22T20:15:40.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.113+0000 7f29ff325800 20 remove_watcher() i=2 2024-04-22T20:15:40.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.114+0000 7f29ff325800 20 remove_watcher() i=3 2024-04-22T20:15:40.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.115+0000 7f29ff325800 20 remove_watcher() i=4 2024-04-22T20:15:40.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.116+0000 7f29ff325800 20 remove_watcher() i=5 2024-04-22T20:15:40.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.117+0000 7f29ff325800 20 remove_watcher() i=6 2024-04-22T20:15:40.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.118+0000 7f29ff325800 20 remove_watcher() i=7 2024-04-22T20:15:40.129 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:15:40.177 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:15:40.178 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:15:40.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.211+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.212+0000 7f5f4fc38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:40.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.212+0000 7f5f4fc38800 20 realm 2024-04-22T20:15:40.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.212+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.213+0000 7f5f4fc38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:40.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.213+0000 7f5f4fc38800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:15:40.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.213+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.213+0000 7f5f4fc38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:40.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.213+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.213+0000 7f5f4fc38800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:40.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.213+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.214+0000 7f5f4fc38800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:15:40.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.214+0000 7f5f4fc38800 20 searching for the correct realm 2024-04-22T20:15:40.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.216+0000 7f5f4df38640 0 -- 172.21.15.136:0/3359088386 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f5f4801f070 msgr2=0x7f5f4805e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:40.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.421+0000 7f5f4cf36640 0 -- 172.21.15.136:0/3359088386 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f5f4805e7e0 msgr2=0x7f5f4808f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:40.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.626+0000 7f5f4fc38800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:15:40.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.626+0000 7f5f4fc38800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:15:40.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.626+0000 7f5f4fc38800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:15:40.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.626+0000 7f5f4fc38800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:15:40.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.626+0000 7f5f4fc38800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:15:40.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.626+0000 7f5f4fc38800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:15:40.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.626+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.627+0000 7f5f4fc38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:40.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.627+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.627+0000 7f5f4fc38800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:40.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.627+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:15:40.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 20 zone default found 2024-04-22T20:15:40.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 4 Realm: () 2024-04-22T20:15:40.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 20 zonegroup default 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.628+0000 7f5f4fc38800 20 rados->read ofs=0 len=0 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.629+0000 7f5f4fc38800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.629+0000 7f5f4fc38800 20 started sync module instance, tier type = 2024-04-22T20:15:40.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.629+0000 7f5f4fc38800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:15:40.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 20 add_watcher() i=0 2024-04-22T20:15:40.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 20 add_watcher() i=1 2024-04-22T20:15:40.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 20 add_watcher() i=2 2024-04-22T20:15:40.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 20 add_watcher() i=3 2024-04-22T20:15:40.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 20 add_watcher() i=4 2024-04-22T20:15:40.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 20 add_watcher() i=5 2024-04-22T20:15:40.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 20 add_watcher() i=6 2024-04-22T20:15:40.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 20 add_watcher() i=7 2024-04-22T20:15:40.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.638+0000 7f5f4fc38800 2 all 8 watchers are set, enabling cache 2024-04-22T20:15:40.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.640+0000 7f5e037e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:40.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.641+0000 7f5f4fc38800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:15:40.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.641+0000 7f5f4fc38800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:15:40.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.676+0000 7f5e027e4640 20 reqs_thread_entry: start 2024-04-22T20:15:40.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.676+0000 7f5f4fc38800 20 init_complete bucket index max shards: 1 2024-04-22T20:15:40.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.676+0000 7f5f4fc38800 20 Filter name: none 2024-04-22T20:15:40.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.676+0000 7f5d73fff640 20 reqs_thread_entry: start 2024-04-22T20:15:40.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.676+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:15:40.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.677+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.678+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.678+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.679+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:15:40.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.680+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.681+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.681+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.682+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:15:40.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.684+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.684+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.684+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.685 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.685+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:15:40.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.687+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.687+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.688+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.689+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:15:40.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.690+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.690+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.691+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.692+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:15:40.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.693+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.694+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.694+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.695+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:15:40.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.696+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.696+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.697+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.698+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:15:40.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.699+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.700+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.700+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.701+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:15:40.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.703+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.703+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.703+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.704+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:15:40.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.705+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.706+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.706+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.707+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:15:40.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.708+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.709+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.709+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.710+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:15:40.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.711+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.712+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.712+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.713+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:15:40.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.714+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.715+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.715+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.716+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:15:40.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.717+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.717+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.718+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.719+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:15:40.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.720+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.720+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.720+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.721+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:15:40.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.723+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.723+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.723+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.724+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:15:40.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.726+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.726+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.726+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.728+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:15:40.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.729+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.729+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.730+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.731+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:15:40.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.732+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.733+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.733+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.734+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:15:40.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.735+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.736+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.736+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.737+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:15:40.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.738+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.739+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.739+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.740+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:15:40.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.741+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.742+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.742+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.743+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:15:40.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.744+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.745+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.745+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.746+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:15:40.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.747+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.748+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.748+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.749+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:15:40.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.750+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.751+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.751+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.753+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:15:40.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.754+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.754+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.755+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.756+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:15:40.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.757+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.757+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.758+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.759+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:15:40.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.760+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.760+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.760+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.761+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:15:40.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.762+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.763+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.763+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.764+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:15:40.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.766+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.766+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.766+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.767+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:15:40.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.769+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.769+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.770+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.771+0000 7f5f4fc38800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:15:40.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.772+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.772+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:40.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.773+0000 7f5f4fc38800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:40.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.774+0000 7f5e037e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:40.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.776+0000 7f5f4fc38800 20 remove_watcher() i=0 2024-04-22T20:15:40.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.776+0000 7f5f4fc38800 2 removed watcher, disabling cache 2024-04-22T20:15:40.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.777+0000 7f5f4fc38800 20 remove_watcher() i=1 2024-04-22T20:15:40.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.778+0000 7f5f4fc38800 20 remove_watcher() i=2 2024-04-22T20:15:40.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.779+0000 7f5f4fc38800 20 remove_watcher() i=3 2024-04-22T20:15:40.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.780+0000 7f5f4fc38800 20 remove_watcher() i=4 2024-04-22T20:15:40.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.781+0000 7f5f4fc38800 20 remove_watcher() i=5 2024-04-22T20:15:40.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.782+0000 7f5f4fc38800 20 remove_watcher() i=6 2024-04-22T20:15:40.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:40.783+0000 7f5f4fc38800 20 remove_watcher() i=7 2024-04-22T20:15:40.793 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:15:40.842 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:15:40.843 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:15:41.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.076+0000 7f72d86a7640 0 --2- 172.21.15.136:0/1575080248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5578c0519bd0 0x5578c0548150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-04-22T20:15:41.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.078+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.079+0000 7f72db3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:41.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.079+0000 7f72db3a2800 20 realm 2024-04-22T20:15:41.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.079+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.079+0000 7f72db3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:41.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.079+0000 7f72db3a2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:15:41.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.079+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.079+0000 7f72db3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:41.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.079+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.080+0000 7f72db3a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:41.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.080+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.081+0000 7f72db3a2800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:15:41.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.081+0000 7f72db3a2800 20 searching for the correct realm 2024-04-22T20:15:41.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.081+0000 7f72d8ea8640 0 -- 172.21.15.136:0/1575080248 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5578c056b600 msgr2=0x5578c056dac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:15:41.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.084+0000 7f72d8ea8640 0 -- 172.21.15.136:0/1575080248 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x7f72d0073040 msgr2=0x7f72d00ba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.292+0000 7f72db3a2800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.292+0000 7f72db3a2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.292+0000 7f72db3a2800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.292+0000 7f72db3a2800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.292+0000 7f72db3a2800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.292+0000 7f72db3a2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.292+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 zone default found 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 4 Realm: () 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 zonegroup default 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.293+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.294+0000 7f72db3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:41.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.294+0000 7f72db3a2800 20 rados->read ofs=0 len=0 2024-04-22T20:15:41.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.294+0000 7f72db3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:41.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.294+0000 7f72db3a2800 20 started sync module instance, tier type = 2024-04-22T20:15:41.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.294+0000 7f72db3a2800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:15:41.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.303+0000 7f72db3a2800 20 add_watcher() i=0 2024-04-22T20:15:41.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.303+0000 7f72db3a2800 20 add_watcher() i=1 2024-04-22T20:15:41.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.303+0000 7f72db3a2800 20 add_watcher() i=2 2024-04-22T20:15:41.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.303+0000 7f72db3a2800 20 add_watcher() i=3 2024-04-22T20:15:41.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.303+0000 7f72db3a2800 20 add_watcher() i=4 2024-04-22T20:15:41.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.303+0000 7f72db3a2800 20 add_watcher() i=5 2024-04-22T20:15:41.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.303+0000 7f72db3a2800 20 add_watcher() i=6 2024-04-22T20:15:41.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.304+0000 7f72db3a2800 20 add_watcher() i=7 2024-04-22T20:15:41.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.304+0000 7f72db3a2800 2 all 8 watchers are set, enabling cache 2024-04-22T20:15:41.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.305+0000 7f718f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:41.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.306+0000 7f72db3a2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:15:41.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.306+0000 7f72db3a2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:15:41.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.341+0000 7f718e7e4640 20 reqs_thread_entry: start 2024-04-22T20:15:41.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.341+0000 7f72db3a2800 20 init_complete bucket index max shards: 1 2024-04-22T20:15:41.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.341+0000 7f72db3a2800 20 Filter name: none 2024-04-22T20:15:41.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.341+0000 7f71877fe640 20 reqs_thread_entry: start 2024-04-22T20:15:41.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.341+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:15:41.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.342+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.343+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.343+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.344 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.344+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:15:41.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.345+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.346+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.346+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.347+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:15:41.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.348+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.348+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.349+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.350+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:15:41.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.351+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.352+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.352+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.354+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:15:41.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.355+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.356+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.356+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.357+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:15:41.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.358+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.359+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.359+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.360+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:15:41.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.361+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.362+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.362+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.363+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:15:41.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.364+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.365+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.365+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.366+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:15:41.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.367+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.367+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.368+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.369+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:15:41.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.370+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.370+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.370+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.371+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:15:41.373 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.373+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.374+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.374+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.375+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:15:41.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.376+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.377+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.377+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.378+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:15:41.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.379+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.379+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.380+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.381+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:15:41.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.382+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.382+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.383+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.384+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:15:41.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.385+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.385+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.386+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.387+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:15:41.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.388+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.389+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.389+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.391 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.391+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:15:41.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.392+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.393+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.393+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.394+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:15:41.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.395+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.396+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.396+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.397+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:15:41.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.398+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.399+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.399+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.400+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:15:41.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.401+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.402+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.402+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.404+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:15:41.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.405+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.405+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.406 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.406+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.406 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.406+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:15:41.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.408+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.408+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.408+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.409 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.409+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:15:41.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.410+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.411+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.411+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.412+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:15:41.413 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.413+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.414+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.414+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.415+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:15:41.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.416+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.416+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.417+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.417+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:15:41.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.419+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.419+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.419+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.421+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:15:41.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.422+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.422+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.422+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.423+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:15:41.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.425+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.425+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.426+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.427+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:15:41.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.428+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.428+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.428+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.429+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:15:41.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.431+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.431+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.431+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.432+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:15:41.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.434+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.434+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.434+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.435+0000 7f72db3a2800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:15:41.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.436+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.437+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:41.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.437+0000 7f72db3a2800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:41.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.439+0000 7f718f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:41.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.441+0000 7f72db3a2800 20 remove_watcher() i=0 2024-04-22T20:15:41.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.441+0000 7f72db3a2800 2 removed watcher, disabling cache 2024-04-22T20:15:41.442 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.442+0000 7f72db3a2800 20 remove_watcher() i=1 2024-04-22T20:15:41.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.443+0000 7f72db3a2800 20 remove_watcher() i=2 2024-04-22T20:15:41.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.444+0000 7f72db3a2800 20 remove_watcher() i=3 2024-04-22T20:15:41.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.445+0000 7f72db3a2800 20 remove_watcher() i=4 2024-04-22T20:15:41.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.446+0000 7f72db3a2800 20 remove_watcher() i=5 2024-04-22T20:15:41.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.447+0000 7f72db3a2800 20 remove_watcher() i=6 2024-04-22T20:15:41.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:41.448+0000 7f72db3a2800 20 remove_watcher() i=7 2024-04-22T20:15:41.459 DEBUG:teuthology.orchestra.run.smithi136:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2024-04-22T20:15:41.940 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :clean 2024-04-22T20:15:42.736 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :compileJava 2024-04-22T20:15:42.736 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :processResources 2024-04-22T20:15:42.736 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :classes 2024-04-22T20:15:43.235 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:43.236 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :compileTestJava 2024-04-22T20:15:43.237 INFO:teuthology.orchestra.run.smithi136.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-04-22T20:15:43.237 INFO:teuthology.orchestra.run.smithi136.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-04-22T20:15:43.238 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:43.238 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :processTestResources NO-SOURCE 2024-04-22T20:15:43.238 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :testClasses 2024-04-22T20:15:45.137 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.137 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :test 2024-04-22T20:15:45.137 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.137 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests STANDARD_OUT 2024-04-22T20:15:45.137 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22 20:15:44,484 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-04-22T20:15:45.138 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.138 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2024-04-22T20:15:45.138 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.138 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2024-04-22T20:15:45.138 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.138 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2024-04-22T20:15:45.236 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.236 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2024-04-22T20:15:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.536 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2024-04-22T20:15:45.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.536 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2024-04-22T20:15:45.835 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.835 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2024-04-22T20:15:45.935 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:45.935 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2024-04-22T20:15:46.035 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:46.035 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2024-04-22T20:15:46.135 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:46.136 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2024-04-22T20:15:46.435 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:46.436 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2024-04-22T20:15:46.535 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:46.535 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2024-04-22T20:15:46.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:46.635 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2024-04-22T20:15:46.735 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:46.735 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2024-04-22T20:15:47.235 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:47.235 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2024-04-22T20:15:47.335 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:47.335 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2024-04-22T20:15:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:47.636 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2024-04-22T20:15:47.636 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:47.636 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2024-04-22T20:15:47.736 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------- 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------- 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout:BUILD SUCCESSFUL in 6s 2024-04-22T20:15:47.737 INFO:teuthology.orchestra.run.smithi136.stdout:5 actionable tasks: 5 executed 2024-04-22T20:15:47.758 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:15:47.807 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:15:47.807 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:15:47.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.841+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:47.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.842+0000 7f9232016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:47.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.842+0000 7f9232016800 20 realm 2024-04-22T20:15:47.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.842+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:47.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.843+0000 7f9232016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:47.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.843+0000 7f9232016800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:15:47.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.843+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:47.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.843+0000 7f9232016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:47.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.843+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:47.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.844+0000 7f9232016800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:47.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.844+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:47.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.845+0000 7f9232016800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:15:47.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.845+0000 7f9232016800 20 searching for the correct realm 2024-04-22T20:15:47.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.847+0000 7f922de85640 0 -- 172.21.15.136:0/3975846882 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f9228021760 msgr2=0x7f9228003050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:47.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:47.851+0000 7f922ce83640 0 -- 172.21.15.136:0/3975846882 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f9228055180 msgr2=0x7f9228083980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.058+0000 7f9232016800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.058+0000 7f9232016800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.058+0000 7f9232016800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.058+0000 7f9232016800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.058+0000 7f9232016800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.058+0000 7f9232016800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.058+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 20 zone default found 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 4 Realm: () 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:15:48.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 20 zonegroup default 2024-04-22T20:15:48.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.059+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.060+0000 7f9232016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.060+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.060+0000 7f9232016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.060+0000 7f9232016800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.060+0000 7f9232016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.060+0000 7f9232016800 20 started sync module instance, tier type = 2024-04-22T20:15:48.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.060+0000 7f9232016800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:15:48.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 20 add_watcher() i=0 2024-04-22T20:15:48.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 20 add_watcher() i=1 2024-04-22T20:15:48.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 20 add_watcher() i=2 2024-04-22T20:15:48.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 20 add_watcher() i=3 2024-04-22T20:15:48.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 20 add_watcher() i=4 2024-04-22T20:15:48.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 20 add_watcher() i=5 2024-04-22T20:15:48.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 20 add_watcher() i=6 2024-04-22T20:15:48.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 20 add_watcher() i=7 2024-04-22T20:15:48.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.072+0000 7f9232016800 2 all 8 watchers are set, enabling cache 2024-04-22T20:15:48.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.074+0000 7f90e37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:48.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.075+0000 7f9232016800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:15:48.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.075+0000 7f9232016800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:15:48.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.114+0000 7f90e27e4640 20 reqs_thread_entry: start 2024-04-22T20:15:48.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.114+0000 7f9232016800 20 init_complete bucket index max shards: 1 2024-04-22T20:15:48.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.114+0000 7f9232016800 20 Filter name: none 2024-04-22T20:15:48.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.115+0000 7f9053fff640 20 reqs_thread_entry: start 2024-04-22T20:15:48.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.115+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:15:48.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.116+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.117+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.117+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.118+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:15:48.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.119+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.120+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.120+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.121+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:15:48.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.122+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.123+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.123+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.124+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:15:48.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.125+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.126 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.126+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.126 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.126+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.127+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:15:48.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.128+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.129+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.129+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.130+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:15:48.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.131+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.132+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.132+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.133+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:15:48.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.135+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.136+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.136+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.137+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:15:48.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.138+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.139+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.139+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.140+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:15:48.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.141+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.141+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.142+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.143+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:15:48.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.144+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.144+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.145+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.146+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:15:48.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.147+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.148+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.148+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.149+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:15:48.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.150+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.151+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.151+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.152+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:15:48.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.153+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.153+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.154+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.155+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:15:48.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.156+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.156+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.157+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.157+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:15:48.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.159+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.159+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.159+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:48.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.159+0000 7f9232016800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14837729137167649523', time=2024-04-22T22:15:46.236559+0000, chain.objs.size()=1 2024-04-22T20:15:48.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.159+0000 7f9232016800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.60__shadow_.tG8CSTnmeQM4U-hSn3GvPpec-sFA-Xw_0 2024-04-22T20:15:48.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.161+0000 7f9232016800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:48.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.163+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:15:48.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.164+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.165+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.165+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.166+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:15:48.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.167+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.167+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.168+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.169 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.169+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:15:48.170 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.170+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.171+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.171+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.172+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:15:48.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.173+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.174+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.174+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:15:48.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.174+0000 7f9232016800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3953648420621270172', time=2024-04-22T22:15:47.015488+0000, chain.objs.size()=1 2024-04-22T20:15:48.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.174+0000 7f9232016800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.61__shadow_.jNMaC28F4vKUZMduFzMdXDCuC1wsSEL_0 2024-04-22T20:15:48.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.175+0000 7f9232016800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:15:48.177 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.177+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:15:48.179 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.179+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.179 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.179+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.180+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.182+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:15:48.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.182+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.183+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.183+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.184+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:15:48.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.186+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.186+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.186+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.188+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:15:48.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.189+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.189+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.190+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.191+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:15:48.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.192+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.193+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.193+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.194+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:15:48.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.196+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.196+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.196+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.197+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:15:48.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.199+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.199+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.200+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.201+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:15:48.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.202+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.203+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.203+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.205+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:15:48.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.206+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.206+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.207+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.208+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:15:48.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.209+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.209+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.209+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.210+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:15:48.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.212+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.212+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.212+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.213+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:15:48.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.214+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.215+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.215+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.216+0000 7f9232016800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:15:48.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.218+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.218+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.219+0000 7f9232016800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.221+0000 7f90e37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:48.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.222+0000 7f9232016800 20 remove_watcher() i=0 2024-04-22T20:15:48.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.222+0000 7f9232016800 2 removed watcher, disabling cache 2024-04-22T20:15:48.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.223+0000 7f9232016800 20 remove_watcher() i=1 2024-04-22T20:15:48.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.224+0000 7f9232016800 20 remove_watcher() i=2 2024-04-22T20:15:48.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.225+0000 7f9232016800 20 remove_watcher() i=3 2024-04-22T20:15:48.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.226+0000 7f9232016800 20 remove_watcher() i=4 2024-04-22T20:15:48.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.228+0000 7f9232016800 20 remove_watcher() i=5 2024-04-22T20:15:48.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.230+0000 7f9232016800 20 remove_watcher() i=6 2024-04-22T20:15:48.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.231+0000 7f9232016800 20 remove_watcher() i=7 2024-04-22T20:15:48.241 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:15:48.290 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:15:48.290 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:15:48.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.324+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.325+0000 7fba92813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.325+0000 7fba92813800 20 realm 2024-04-22T20:15:48.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.325+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.326+0000 7fba92813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.326+0000 7fba92813800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:15:48.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.326+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.326+0000 7fba92813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.326+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.326+0000 7fba92813800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:48.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.326+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.327+0000 7fba92813800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:15:48.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.327+0000 7fba92813800 20 searching for the correct realm 2024-04-22T20:15:48.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.329+0000 7fba8e622640 0 -- 172.21.15.136:0/3836629374 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fba88024070 msgr2=0x7fba88032940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:15:48.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.534+0000 7fba8d620640 0 -- 172.21.15.136:0/3836629374 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fba88096120 msgr2=0x7fba88074930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:48.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:15:48.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:15:48.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:15:48.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:15:48.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:15:48.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.740+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 20 zone default found 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 4 Realm: () 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 20 zonegroup default 2024-04-22T20:15:48.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.741+0000 7fba92813800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.742+0000 7fba92813800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.742+0000 7fba92813800 20 started sync module instance, tier type = 2024-04-22T20:15:48.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.742+0000 7fba92813800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:15:48.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.751+0000 7fba92813800 20 add_watcher() i=0 2024-04-22T20:15:48.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.751+0000 7fba92813800 20 add_watcher() i=1 2024-04-22T20:15:48.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.751+0000 7fba92813800 20 add_watcher() i=2 2024-04-22T20:15:48.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.751+0000 7fba92813800 20 add_watcher() i=3 2024-04-22T20:15:48.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.751+0000 7fba92813800 20 add_watcher() i=4 2024-04-22T20:15:48.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.751+0000 7fba92813800 20 add_watcher() i=5 2024-04-22T20:15:48.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.751+0000 7fba92813800 20 add_watcher() i=6 2024-04-22T20:15:48.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.752+0000 7fba92813800 20 add_watcher() i=7 2024-04-22T20:15:48.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.752+0000 7fba92813800 2 all 8 watchers are set, enabling cache 2024-04-22T20:15:48.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.754+0000 7fb93f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:48.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.754+0000 7fba92813800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:15:48.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.754+0000 7fba92813800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:15:48.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.788+0000 7fb93e7fc640 20 reqs_thread_entry: start 2024-04-22T20:15:48.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.788+0000 7fba92813800 20 init_complete bucket index max shards: 1 2024-04-22T20:15:48.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.788+0000 7fba92813800 20 Filter name: none 2024-04-22T20:15:48.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.789+0000 7fb8affff640 20 reqs_thread_entry: start 2024-04-22T20:15:48.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.789+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:15:48.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.790+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.790+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.791+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.792+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:15:48.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.793+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.794+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.794+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.795+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:15:48.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.796+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.797+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.797+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.798+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:15:48.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.799+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.799+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.800+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.801+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:15:48.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.802+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.802+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.803+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.804+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:15:48.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.806+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.806+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.806+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.807+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:15:48.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.809+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.809+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.809+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.810+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:15:48.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.812+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.812+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.813+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.814+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:15:48.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.815+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.816+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.816+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.817+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:15:48.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.818+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.819+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.819+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.820+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:15:48.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.821+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.822+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.822+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.823+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:15:48.824 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.824+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.824+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.825+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.826+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:15:48.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.827+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.828+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.828+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.829+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:15:48.830 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.830+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.831+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.831+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.832+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:15:48.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.833+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.834+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.834+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.835+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:15:48.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.836+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.837+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.837+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.838+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:15:48.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.839+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.840+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.840+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.842+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:15:48.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.843+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.843+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.844+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.844+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:15:48.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.846+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.846+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.846+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.848+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:15:48.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.849+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.849+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.850+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.850+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:15:48.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.852+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.853+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.853+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.854+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:15:48.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.855+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.856+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.856+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.857+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:15:48.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.858+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.859+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.859+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.860+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:15:48.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.861+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.862+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.862+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.863+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:15:48.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.864+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.865+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.865+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.866+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:15:48.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.868+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.868+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.868+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.869+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:15:48.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.871+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.871+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.871+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.872+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:15:48.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.873+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.874+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.874+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.875+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:15:48.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.876+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.877+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.877+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.878+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:15:48.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.881+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.882+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.882+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.883+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:15:48.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.884+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.885+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.885+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.886+0000 7fba92813800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:15:48.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.887+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.888+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:48.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.888+0000 7fba92813800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:48.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.890+0000 7fb93f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:48.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.892+0000 7fba92813800 20 remove_watcher() i=0 2024-04-22T20:15:48.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.892+0000 7fba92813800 2 removed watcher, disabling cache 2024-04-22T20:15:48.893 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.893+0000 7fba92813800 20 remove_watcher() i=1 2024-04-22T20:15:48.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.894+0000 7fba92813800 20 remove_watcher() i=2 2024-04-22T20:15:48.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.895+0000 7fba92813800 20 remove_watcher() i=3 2024-04-22T20:15:48.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.896+0000 7fba92813800 20 remove_watcher() i=4 2024-04-22T20:15:48.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.897+0000 7fba92813800 20 remove_watcher() i=5 2024-04-22T20:15:48.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.898+0000 7fba92813800 20 remove_watcher() i=6 2024-04-22T20:15:48.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.900+0000 7fba92813800 20 remove_watcher() i=7 2024-04-22T20:15:48.911 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:15:48.960 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:15:48.961 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:15:48.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.994+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.996+0000 7f6fbe03b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.996+0000 7f6fbe03b800 20 realm 2024-04-22T20:15:48.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.996+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.996+0000 7f6fbe03b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.996+0000 7f6fbe03b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:15:48.997 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.996+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.997 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.996+0000 7f6fbe03b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:48.997 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.996+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.997 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.997+0000 7f6fbe03b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:48.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.997+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:48.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.997+0000 7f6fbe03b800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:15:48.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.997+0000 7f6fbe03b800 20 searching for the correct realm 2024-04-22T20:15:48.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:48.999+0000 7f6fb3339640 0 -- 172.21.15.136:0/3360488477 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f6fac075070 msgr2=0x7f6fac01a9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:49.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.002+0000 7f6fbc33b640 0 -- 172.21.15.136:0/3360488477 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x5587567311f0 msgr2=0x558756733600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:49.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:15:49.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:15:49.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:15:49.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:15:49.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:15:49.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:15:49.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:49.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.209+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:15:49.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 zone default found 2024-04-22T20:15:49.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 4 Realm: () 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 zonegroup default 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.210+0000 7f6fbe03b800 20 started sync module instance, tier type = 2024-04-22T20:15:49.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.211+0000 7f6fbe03b800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:15:49.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.219+0000 7f6fbe03b800 20 add_watcher() i=0 2024-04-22T20:15:49.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.219+0000 7f6fbe03b800 20 add_watcher() i=1 2024-04-22T20:15:49.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.219+0000 7f6fbe03b800 20 add_watcher() i=2 2024-04-22T20:15:49.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.219+0000 7f6fbe03b800 20 add_watcher() i=3 2024-04-22T20:15:49.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.219+0000 7f6fbe03b800 20 add_watcher() i=4 2024-04-22T20:15:49.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.219+0000 7f6fbe03b800 20 add_watcher() i=5 2024-04-22T20:15:49.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.219+0000 7f6fbe03b800 20 add_watcher() i=6 2024-04-22T20:15:49.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.220+0000 7f6fbe03b800 20 add_watcher() i=7 2024-04-22T20:15:49.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.220+0000 7f6fbe03b800 2 all 8 watchers are set, enabling cache 2024-04-22T20:15:49.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.222+0000 7f6e71fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:49.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.222+0000 7f6fbe03b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:15:49.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.222+0000 7f6fbe03b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:15:49.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.256+0000 7f6e70fe1640 20 reqs_thread_entry: start 2024-04-22T20:15:49.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.256+0000 7f6fbe03b800 20 init_complete bucket index max shards: 1 2024-04-22T20:15:49.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.256+0000 7f6fbe03b800 20 Filter name: none 2024-04-22T20:15:49.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.256+0000 7f6dea7fc640 20 reqs_thread_entry: start 2024-04-22T20:15:49.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.256+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:15:49.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.258+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.258+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.258+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.259+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:15:49.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.260+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.261+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.261+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.263+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:15:49.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.264+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.264+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.265+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.265+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:15:49.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.267+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.267+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.267+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.268+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:15:49.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.269+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.270+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.270+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.271+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:15:49.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.272+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.273+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.273+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.274+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:15:49.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.276+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.276+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.277+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.278+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:15:49.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.279+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.279+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.280+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.281+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:15:49.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.282+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.283+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.283+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.284+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:15:49.285 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.285+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.286+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.286+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.287+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:15:49.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.288+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.289+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.289+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.290 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.290+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:15:49.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.291+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.292+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.292+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.293+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:15:49.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.294+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.295+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.295+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.296+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:15:49.298 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.298+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.298 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.298+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.298 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.298+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.299 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.299+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:15:49.301 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.301+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.301 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.301+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.302 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.302+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.302 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.303+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:15:49.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.304+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.304+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.304+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.305+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:15:49.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.306+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.307+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.307+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.308+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:15:49.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.309+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.310 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.310+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.310 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.310+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.311+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:15:49.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.312+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.313+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.313+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.314+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:15:49.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.315+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.316+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.316+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.317+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:15:49.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.318+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.318+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.319+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.320+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:15:49.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.321+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.321+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.322+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.324+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:15:49.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.324+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.325+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.325+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.326+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:15:49.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.327+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.328 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.328+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.329+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.330+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:15:49.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.331+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.331+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.332+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.333+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:15:49.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.334+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.334+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.334+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.335+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:15:49.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.337+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.337+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.337+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.338+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:15:49.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.339+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.340+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.340+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.341+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:15:49.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.342+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.343+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.343+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.344 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.344+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:15:49.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.345+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.346+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.346+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.347+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:15:49.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.349+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.349+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.349+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.351+0000 7f6fbe03b800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:15:49.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.351+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.352+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.352+0000 7f6fbe03b800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.354+0000 7f6e71fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.357+0000 7f6fbe03b800 20 remove_watcher() i=0 2024-04-22T20:15:49.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.357+0000 7f6fbe03b800 2 removed watcher, disabling cache 2024-04-22T20:15:49.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.358+0000 7f6fbe03b800 20 remove_watcher() i=1 2024-04-22T20:15:49.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.359+0000 7f6fbe03b800 20 remove_watcher() i=2 2024-04-22T20:15:49.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.360+0000 7f6fbe03b800 20 remove_watcher() i=3 2024-04-22T20:15:49.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.362+0000 7f6fbe03b800 20 remove_watcher() i=4 2024-04-22T20:15:49.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.363+0000 7f6fbe03b800 20 remove_watcher() i=5 2024-04-22T20:15:49.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.363+0000 7f6fbe03b800 20 remove_watcher() i=6 2024-04-22T20:15:49.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.364+0000 7f6fbe03b800 20 remove_watcher() i=7 2024-04-22T20:15:49.375 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:15:49.423 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:15:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:15:49.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.457+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.459+0000 7f2a74575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.459+0000 7f2a74575800 20 realm 2024-04-22T20:15:49.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.459+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.459+0000 7f2a74575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.459+0000 7f2a74575800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:15:49.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.459+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.459+0000 7f2a74575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.459+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.460+0000 7f2a74575800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:49.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.460+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.460+0000 7f2a74575800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:15:49.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.460+0000 7f2a74575800 20 searching for the correct realm 2024-04-22T20:15:49.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.462+0000 7f2a72875640 0 -- 172.21.15.136:0/650699013 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f2a6c0d2050 msgr2=0x7f2a6c0c19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:15:49.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.667+0000 7f2a72875640 0 -- 172.21.15.136:0/650699013 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x55e456769260 msgr2=0x55e45676b670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:15:49.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:15:49.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:15:49.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:15:49.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:15:49.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:15:49.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:15:49.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:15:49.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:15:49.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 zone default found 2024-04-22T20:15:49.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 4 Realm: () 2024-04-22T20:15:49.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:15:49.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:15:49.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:15:49.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 zonegroup default 2024-04-22T20:15:49.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.673+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.674+0000 7f2a74575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.674+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.674+0000 7f2a74575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.674+0000 7f2a74575800 20 rados->read ofs=0 len=0 2024-04-22T20:15:49.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.674+0000 7f2a74575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:15:49.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.674+0000 7f2a74575800 20 started sync module instance, tier type = 2024-04-22T20:15:49.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.674+0000 7f2a74575800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:15:49.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 20 add_watcher() i=0 2024-04-22T20:15:49.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 20 add_watcher() i=1 2024-04-22T20:15:49.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 20 add_watcher() i=2 2024-04-22T20:15:49.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 20 add_watcher() i=3 2024-04-22T20:15:49.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 20 add_watcher() i=4 2024-04-22T20:15:49.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 20 add_watcher() i=5 2024-04-22T20:15:49.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 20 add_watcher() i=6 2024-04-22T20:15:49.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 20 add_watcher() i=7 2024-04-22T20:15:49.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.683+0000 7f2a74575800 2 all 8 watchers are set, enabling cache 2024-04-22T20:15:49.685 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.685+0000 7f2928fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:49.686 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.686+0000 7f2a74575800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:15:49.686 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.686+0000 7f2a74575800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:15:49.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.720+0000 7f28a37fe640 20 reqs_thread_entry: start 2024-04-22T20:15:49.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.720+0000 7f2a74575800 20 init_complete bucket index max shards: 1 2024-04-22T20:15:49.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.721+0000 7f2a74575800 20 Filter name: none 2024-04-22T20:15:49.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.721+0000 7f28a17fa640 20 reqs_thread_entry: start 2024-04-22T20:15:49.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.721+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:15:49.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.722+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.722+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.723+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.723+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:15:49.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.725+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.725+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.725+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.726+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:15:49.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.728+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.728+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.728+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.729+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:15:49.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.732+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.733+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.733+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.733+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:15:49.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.735+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.735+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.735+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.736+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:15:49.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.737+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.737+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.739+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.739+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:15:49.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.740+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.740+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.740+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.741+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:15:49.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.742+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.743+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.743+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.744+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:15:49.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.746+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.747+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.747+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.748+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:15:49.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.748+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.749+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.749+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.751+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:15:49.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.752+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.753+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.753+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.754+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:15:49.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.755+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.756+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.756+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.757+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:15:49.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.758+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.758+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.758+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.759+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:15:49.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.760+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.761+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.761+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.761+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:15:49.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.763+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.763+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.763+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.765+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:15:49.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.766+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.766+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.766+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.767+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:15:49.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.769+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.769+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.769+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.770+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:15:49.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.771+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.772+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.772+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.773+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:15:49.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.774+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.775+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.775+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.776+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:15:49.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.778+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.778+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.779+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.780+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:15:49.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.781+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.781+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.782+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.783+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:15:49.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.784+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.784+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.785+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.785+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:15:49.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.787+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.787+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.787+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.788+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:15:49.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.790+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.790+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.790+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.791+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:15:49.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.792+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.793+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.793+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.794+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:15:49.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.795+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.795+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.796+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.797+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:15:49.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.798+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.798+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.798+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.799+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:15:49.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.801+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.801+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.801+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.802+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:15:49.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.804+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.804+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.804+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.805+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:15:49.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.806+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.807+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.807+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.808+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:15:49.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.809+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.809+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.810+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.810+0000 7f2a74575800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:15:49.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.812+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.812+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:15:49.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.812+0000 7f2a74575800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:15:49.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.814+0000 7f2928fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:15:49.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.816+0000 7f2a74575800 20 remove_watcher() i=0 2024-04-22T20:15:49.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.816+0000 7f2a74575800 2 removed watcher, disabling cache 2024-04-22T20:15:49.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.817+0000 7f2a74575800 20 remove_watcher() i=1 2024-04-22T20:15:49.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.818+0000 7f2a74575800 20 remove_watcher() i=2 2024-04-22T20:15:49.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.819+0000 7f2a74575800 20 remove_watcher() i=3 2024-04-22T20:15:49.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.820+0000 7f2a74575800 20 remove_watcher() i=4 2024-04-22T20:15:49.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.820+0000 7f2a74575800 20 remove_watcher() i=5 2024-04-22T20:15:49.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.821+0000 7f2a74575800 20 remove_watcher() i=6 2024-04-22T20:15:49.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:15:49.822+0000 7f2a74575800 20 remove_watcher() i=7 2024-04-22T20:15:49.832 DEBUG:teuthology.orchestra.run.smithi136:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2024-04-22T20:15:50.314 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :clean 2024-04-22T20:15:51.110 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :compileJava 2024-04-22T20:15:51.111 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :processResources 2024-04-22T20:15:51.111 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :classes 2024-04-22T20:15:51.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:51.610 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :compileTestJava 2024-04-22T20:15:51.611 INFO:teuthology.orchestra.run.smithi136.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-04-22T20:15:51.611 INFO:teuthology.orchestra.run.smithi136.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-04-22T20:15:51.612 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:51.612 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :processTestResources NO-SOURCE 2024-04-22T20:15:51.612 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :testClasses 2024-04-22T20:15:53.511 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:53.512 INFO:teuthology.orchestra.run.smithi136.stdout:> Task :test 2024-04-22T20:15:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:53.512 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests STANDARD_OUT 2024-04-22T20:15:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22 20:15:52,807 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-04-22T20:15:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:53.512 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2024-04-22T20:15:53.512 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:53.512 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2024-04-22T20:15:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:53.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2024-04-22T20:15:53.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:53.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2024-04-22T20:15:54.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:54.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2024-04-22T20:15:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:54.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2024-04-22T20:15:54.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:15:54.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2024-04-22T20:16:08.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:08.011 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2024-04-22T20:16:08.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:08.314 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2024-04-22T20:16:08.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:08.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2024-04-22T20:16:08.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:08.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2024-04-22T20:16:08.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:08.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2024-04-22T20:16:09.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:09.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2024-04-22T20:16:09.111 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:09.111 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2024-04-22T20:16:09.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:09.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2024-04-22T20:16:09.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:09.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2024-04-22T20:16:09.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:09.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2024-04-22T20:16:14.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:14.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2024-04-22T20:16:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:15.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2024-04-22T20:16:15.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:15.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2024-04-22T20:16:15.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:15.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2024-04-22T20:16:15.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:15.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2024-04-22T20:16:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:15.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2024-04-22T20:16:15.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:15.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2024-04-22T20:16:16.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:16.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2024-04-22T20:16:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:16.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2024-04-22T20:16:16.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:16.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2024-04-22T20:16:16.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:16.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2024-04-22T20:16:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:16.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2024-04-22T20:16:17.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:17.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2024-04-22T20:16:17.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:17.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2024-04-22T20:16:17.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:17.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2024-04-22T20:16:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:17.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2024-04-22T20:16:23.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:23.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2024-04-22T20:16:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:23.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2024-04-22T20:16:55.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:55.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2024-04-22T20:16:55.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:55.511 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2024-04-22T20:16:55.511 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:55.511 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2024-04-22T20:16:55.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:55.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2024-04-22T20:16:55.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:55.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2024-04-22T20:16:56.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:16:56.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2024-04-22T20:17:01.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:01.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2024-04-22T20:17:02.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:02.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2024-04-22T20:17:02.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:02.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2024-04-22T20:17:02.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:02.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2024-04-22T20:17:17.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:17.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2024-04-22T20:17:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:19.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests STANDARD_OUT 2024-04-22T20:17:19.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22 20:17:19,418 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-04-22T20:17:20.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:20.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2024-04-22T20:17:30.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:30.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2024-04-22T20:17:30.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:30.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2024-04-22T20:17:35.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:35.311 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2024-04-22T20:17:35.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:17:35.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2024-04-22T20:18:13.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:13.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2024-04-22T20:18:14.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:14.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2024-04-22T20:18:30.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:30.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2024-04-22T20:18:31.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:31.011 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2024-04-22T20:18:35.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:35.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2024-04-22T20:18:36.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:36.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2024-04-22T20:18:37.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:37.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2024-04-22T20:18:40.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:40.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2024-04-22T20:18:40.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:40.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2024-04-22T20:18:40.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:40.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2024-04-22T20:18:41.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:41.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2024-04-22T20:18:42.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:42.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2024-04-22T20:18:42.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:42.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2024-04-22T20:18:42.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:42.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2024-04-22T20:18:42.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:42.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2024-04-22T20:18:43.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:43.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2024-04-22T20:18:43.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:43.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2024-04-22T20:18:43.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:43.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2024-04-22T20:18:43.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:43.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2024-04-22T20:18:43.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:43.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2024-04-22T20:18:43.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:43.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2024-04-22T20:18:44.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:44.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2024-04-22T20:18:44.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:44.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2024-04-22T20:18:44.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:44.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2024-04-22T20:18:45.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:45.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2024-04-22T20:18:45.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:45.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2024-04-22T20:18:45.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:45.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2024-04-22T20:18:46.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:46.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2024-04-22T20:18:46.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:46.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2024-04-22T20:18:46.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:46.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2024-04-22T20:18:46.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:46.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2024-04-22T20:18:47.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:47.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2024-04-22T20:18:47.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:47.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2024-04-22T20:18:47.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:47.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2024-04-22T20:18:47.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:47.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2024-04-22T20:18:48.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:48.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2024-04-22T20:18:48.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:48.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2024-04-22T20:18:48.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:48.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2024-04-22T20:18:49.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:49.111 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2024-04-22T20:18:49.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:49.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2024-04-22T20:18:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:49.711 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2024-04-22T20:18:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:50.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2024-04-22T20:18:50.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:50.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2024-04-22T20:18:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:50.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2024-04-22T20:18:50.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:50.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2024-04-22T20:18:51.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:51.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2024-04-22T20:18:51.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:51.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2024-04-22T20:18:51.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:51.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2024-04-22T20:18:51.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:51.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2024-04-22T20:18:52.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:52.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2024-04-22T20:18:52.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:52.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2024-04-22T20:18:52.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:52.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2024-04-22T20:18:53.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:53.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2024-04-22T20:18:53.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:53.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2024-04-22T20:18:53.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:53.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2024-04-22T20:18:54.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:54.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2024-04-22T20:18:54.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:54.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2024-04-22T20:18:54.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:54.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2024-04-22T20:18:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:54.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2024-04-22T20:18:55.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:55.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2024-04-22T20:18:55.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:55.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2024-04-22T20:18:55.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:55.710 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2024-04-22T20:18:55.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:55.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2024-04-22T20:18:56.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:56.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2024-04-22T20:18:56.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:56.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2024-04-22T20:18:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:56.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2024-04-22T20:18:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:56.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2024-04-22T20:18:57.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:57.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2024-04-22T20:18:57.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:57.231 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2024-04-22T20:18:57.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:57.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2024-04-22T20:18:57.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:57.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2024-04-22T20:18:57.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:57.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2024-04-22T20:18:57.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:57.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2024-04-22T20:18:58.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:58.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2024-04-22T20:18:58.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:58.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2024-04-22T20:18:58.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:58.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2024-04-22T20:18:58.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:58.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2024-04-22T20:18:59.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:59.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2024-04-22T20:18:59.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:59.110 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2024-04-22T20:18:59.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:59.410 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2024-04-22T20:18:59.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:59.510 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2024-04-22T20:18:59.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:59.810 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2024-04-22T20:18:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:18:59.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2024-04-22T20:19:00.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:00.210 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2024-04-22T20:19:00.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:00.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2024-04-22T20:19:00.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:00.610 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2024-04-22T20:19:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:06.010 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2024-04-22T20:19:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:06.310 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testUploadFileList STARTED 2024-04-22T20:19:11.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:11.910 INFO:teuthology.orchestra.run.smithi136.stdout:suite > Object tests > ObjectTest.testUploadFileList PASSED 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout:---------------------------------------------------------------------- 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout:| Results: SUCCESS (70 tests, 70 successes, 0 failures, 0 skipped) | 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout:---------------------------------------------------------------------- 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-04-22T20:19:12.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:12.288 INFO:teuthology.orchestra.run.smithi136.stdout:BUILD SUCCESSFUL in 3m 22s 2024-04-22T20:19:12.288 INFO:teuthology.orchestra.run.smithi136.stdout:5 actionable tasks: 5 executed 2024-04-22T20:19:12.301 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:19:12.351 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:12.351 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:12.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.385+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.386+0000 7fc146133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:12.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.386+0000 7fc146133800 20 realm 2024-04-22T20:19:12.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.386+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.386+0000 7fc146133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:12.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.386+0000 7fc146133800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:12.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.386+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.386+0000 7fc146133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:12.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.386+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.387+0000 7fc146133800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:12.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.387+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.388+0000 7fc146133800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:12.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.388+0000 7fc146133800 20 searching for the correct realm 2024-04-22T20:19:12.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.390+0000 7fc1425d2640 0 -- 172.21.15.136:0/2572885445 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fc13c050070 msgr2=0x7fc13c09f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:12.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.596+0000 7fc1415d0640 0 -- 172.21.15.136:0/2572885445 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fc13c0999d0 msgr2=0x7fc13c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:19:12.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.802+0000 7fc146133800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:12.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.802+0000 7fc146133800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:12.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.802+0000 7fc146133800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:12.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.802+0000 7fc146133800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:12.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.802+0000 7fc146133800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:12.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.802+0000 7fc146133800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:12.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.802+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:12.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 20 zone default found 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 4 Realm: () 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.803+0000 7fc146133800 20 zonegroup default 2024-04-22T20:19:12.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.804+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.804+0000 7fc146133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:12.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.804+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.804+0000 7fc146133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:12.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.804+0000 7fc146133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:12.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.804+0000 7fc146133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:12.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.804+0000 7fc146133800 20 started sync module instance, tier type = 2024-04-22T20:19:12.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.804+0000 7fc146133800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:12.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.814+0000 7fc146133800 20 add_watcher() i=0 2024-04-22T20:19:12.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.814+0000 7fc146133800 20 add_watcher() i=1 2024-04-22T20:19:12.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.814+0000 7fc146133800 20 add_watcher() i=2 2024-04-22T20:19:12.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.814+0000 7fc146133800 20 add_watcher() i=3 2024-04-22T20:19:12.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.814+0000 7fc146133800 20 add_watcher() i=4 2024-04-22T20:19:12.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.814+0000 7fc146133800 20 add_watcher() i=5 2024-04-22T20:19:12.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.814+0000 7fc146133800 20 add_watcher() i=6 2024-04-22T20:19:12.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.815+0000 7fc146133800 20 add_watcher() i=7 2024-04-22T20:19:12.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.815+0000 7fc146133800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:12.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.817+0000 7fbff37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:12.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.817+0000 7fc146133800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:12.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.817+0000 7fc146133800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:12.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.855+0000 7fbff27fc640 20 reqs_thread_entry: start 2024-04-22T20:19:12.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.855+0000 7fc146133800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:12.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.855+0000 7fc146133800 20 Filter name: none 2024-04-22T20:19:12.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.855+0000 7fbf63fff640 20 reqs_thread_entry: start 2024-04-22T20:19:12.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.855+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:19:12.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.857+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.857+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.858+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:12.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.858+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16089074251714167784', time=2024-04-22T22:18:46.335089+0000, chain.objs.size()=1 2024-04-22T20:19:12.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.858+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.104__shadow_.lmWGSlmVN_YLHWckqGi572XhGVCglOh_0 2024-04-22T20:19:12.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.858+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5321564528429789418', time=2024-04-22T22:18:49.243899+0000, chain.objs.size()=1 2024-04-22T20:19:12.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.858+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.109__shadow_._LdoG3iP7O232jOZrLZYg_XufNYCpZg_0 2024-04-22T20:19:12.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.858+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8051735264979675681', time=2024-04-22T22:18:54.962061+0000, chain.objs.size()=1 2024-04-22T20:19:12.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.858+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.119__shadow_.dD0vNmXEFAn7R84cvjD93CC3tz62gsn_0 2024-04-22T20:19:12.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.859+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.862+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:19:12.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.863+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:12.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb', time=2024-04-22T22:16:07.950917+0000, chain.objs.size()=9 2024-04-22T20:19:12.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__multipart_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.1 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__shadow_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.1_1 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__multipart_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.2 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__shadow_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.2_1 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__multipart_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.3 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__shadow_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.3_1 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__multipart_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.4 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__shadow_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.4_1 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.68__multipart_key1.2~JFxRdPWRUnSJeObjrHZ0psJMxki2QSb.5 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17614813925949096423', time=2024-04-22T22:18:49.824591+0000, chain.objs.size()=1 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.110__shadow_.z1FOntNM184NrBy1M25M4hXqtxreWnp_0 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1854313857865440661', time=2024-04-22T22:18:52.679920+0000, chain.objs.size()=1 2024-04-22T20:19:12.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.864+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.115__shadow_.HVX2QWtgp_ROPYIzf_el9IqQrHNmnKz_0 2024-04-22T20:19:12.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.869+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.872+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:19:12.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.873+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.874+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.874+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:19:12.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.874+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3004033072610994227', time=2024-04-22T22:18:52.653846+0000, chain.objs.size()=1 2024-04-22T20:19:12.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.874+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.115__shadow_.jEh7c6DETF3aajeBtDbijiwtgpg_fRE_0 2024-04-22T20:19:12.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.875+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.877+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:19:12.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.879+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.879+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-22T20:19:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1207527925473420630', time=2024-04-22T22:16:17.384249+0000, chain.objs.size()=1 2024-04-22T20:19:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.79__shadow_.uKwmYc9u47ci400PqifCvCmISklVDxw_0 2024-04-22T20:19:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6335542288248309370', time=2024-04-22T22:18:48.637871+0000, chain.objs.size()=1 2024-04-22T20:19:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.108__shadow_.euvFoXjAi-uzTGBFTzl45zOEFfx0JJg_0 2024-04-22T20:19:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12657683129983855088', time=2024-04-22T22:18:49.217313+0000, chain.objs.size()=1 2024-04-22T20:19:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.109__shadow_.DJyGL54vzGduJi5S-vxeXwGmzUdg5Aw_0 2024-04-22T20:19:12.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10312565205469209248', time=2024-04-22T22:18:53.772222+0000, chain.objs.size()=1 2024-04-22T20:19:12.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.117__shadow_.WNrQjJ9IGKg5wnyGBdYbMbi6N_bTyih_0 2024-04-22T20:19:12.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14560665620966090431', time=2024-04-22T22:18:53.798722+0000, chain.objs.size()=1 2024-04-22T20:19:12.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.117__shadow_.Mq8k8R8Viih3KzdsQJaiTMcHvNaF3dL_0 2024-04-22T20:19:12.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2799005240741755358', time=2024-04-22T22:18:55.468754+0000, chain.objs.size()=1 2024-04-22T20:19:12.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.880+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.120__shadow_.k3nQ_OAuPynEuWoxxcKQH7Dy43pEmku_0 2024-04-22T20:19:12.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.884+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.886+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:19:12.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.888+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-22T20:19:12.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15756781965256613039', time=2024-04-22T22:16:54.888124+0000, chain.objs.size()=7 2024-04-22T20:19:12.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~9m0Ord66q1W8XUr5kvhtrWuaeevePWl.1 2024-04-22T20:19:12.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~9m0Ord66q1W8XUr5kvhtrWuaeevePWl.1_1 2024-04-22T20:19:12.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~9m0Ord66q1W8XUr5kvhtrWuaeevePWl.1_2 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~9m0Ord66q1W8XUr5kvhtrWuaeevePWl.2 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~9m0Ord66q1W8XUr5kvhtrWuaeevePWl.2_1 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~9m0Ord66q1W8XUr5kvhtrWuaeevePWl.2_2 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~9m0Ord66q1W8XUr5kvhtrWuaeevePWl.3 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10258110121643967682', time=2024-04-22T22:18:41.302094+0000, chain.objs.size()=1 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.95__shadow_.Yk9ibM37nagB4zlFXPXkqX2nK9FUe2k_0 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1821208905715168919', time=2024-04-22T22:18:44.550549+0000, chain.objs.size()=1 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.101__shadow_.NR3fNu4lCrD220TPzn7zmfQFkXiAx8X_0 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16406065429938121059', time=2024-04-22T22:18:45.195380+0000, chain.objs.size()=1 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.102__shadow_.OpTGVodtAAOCxEgyJJPCaRkEq3J0HIS_0 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11167615325928339416', time=2024-04-22T22:18:50.361435+0000, chain.objs.size()=1 2024-04-22T20:19:12.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.889+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.111__shadow_.bzVMyt6hLsSfdRd-i82ZsuNKL7efnsT_0 2024-04-22T20:19:12.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.894+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.896+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:19:12.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.898+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.898+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-22T20:19:12.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11970251063522817064', time=2024-04-22T22:16:52.374083+0000, chain.objs.size()=7 2024-04-22T20:19:12.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~yGawxnNb8WIiV5gq6fk9fV5yjZ0YHdE.1 2024-04-22T20:19:12.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~yGawxnNb8WIiV5gq6fk9fV5yjZ0YHdE.1_1 2024-04-22T20:19:12.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~yGawxnNb8WIiV5gq6fk9fV5yjZ0YHdE.1_2 2024-04-22T20:19:12.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~yGawxnNb8WIiV5gq6fk9fV5yjZ0YHdE.2 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~yGawxnNb8WIiV5gq6fk9fV5yjZ0YHdE.2_1 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~yGawxnNb8WIiV5gq6fk9fV5yjZ0YHdE.2_2 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~yGawxnNb8WIiV5gq6fk9fV5yjZ0YHdE.3 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14716172032256094730', time=2024-04-22T22:18:49.851034+0000, chain.objs.size()=1 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.110__shadow_.NwjXfZcU8cHlj0ZKMojgvTTY6_pKEoO_0 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5088153039188988180', time=2024-04-22T22:18:50.968963+0000, chain.objs.size()=1 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.899+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.112__shadow_._KhcZEUrwsvtSmPNAD8ZhOuHYvq0Jst_0 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.900+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7527444444222126731', time=2024-04-22T22:18:53.215379+0000, chain.objs.size()=1 2024-04-22T20:19:12.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.900+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.116__shadow_.TYYRSbo9g4fZ8MhjKNZxVA4l0a9hK1X_0 2024-04-22T20:19:12.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.903+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.906+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:19:12.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.907+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.907+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-22T20:19:12.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5141392480990231453', time=2024-04-22T22:16:14.750213+0000, chain.objs.size()=3 2024-04-22T20:19:12.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.72__shadow_.Zs5X8jX73-UFXXIxpzQl4kvfVjg9cgq_0 2024-04-22T20:19:12.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.72__shadow_.Zs5X8jX73-UFXXIxpzQl4kvfVjg9cgq_1 2024-04-22T20:19:12.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.72__shadow_.Zs5X8jX73-UFXXIxpzQl4kvfVjg9cgq_2 2024-04-22T20:19:12.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2912236623836553963', time=2024-04-22T22:18:13.640249+0000, chain.objs.size()=7 2024-04-22T20:19:12.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~p1lzRespIc84rKFVEBZiW0mp8K8-WnH.1 2024-04-22T20:19:12.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~p1lzRespIc84rKFVEBZiW0mp8K8-WnH.1_1 2024-04-22T20:19:12.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~p1lzRespIc84rKFVEBZiW0mp8K8-WnH.1_2 2024-04-22T20:19:12.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~p1lzRespIc84rKFVEBZiW0mp8K8-WnH.2 2024-04-22T20:19:12.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~p1lzRespIc84rKFVEBZiW0mp8K8-WnH.2_1 2024-04-22T20:19:12.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~p1lzRespIc84rKFVEBZiW0mp8K8-WnH.2_2 2024-04-22T20:19:12.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~p1lzRespIc84rKFVEBZiW0mp8K8-WnH.3 2024-04-22T20:19:12.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10421461517446786430', time=2024-04-22T22:18:51.473152+0000, chain.objs.size()=1 2024-04-22T20:19:12.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.113__shadow_.q9xruBi-A_IGZx-hVbYEnOIj1meRA8x_0 2024-04-22T20:19:12.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13289250686597225474', time=2024-04-22T22:18:52.089005+0000, chain.objs.size()=1 2024-04-22T20:19:12.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.908+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.114__shadow_.oMX3WCwX4cYVWxHnGpvvW-oMhNzBV2Y_0 2024-04-22T20:19:12.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.911+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2597331297770214059', time=2024-04-22T22:18:54.405104+0000, chain.objs.size()=1 2024-04-22T20:19:12.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.911+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.118__shadow_.xVEyQAOb65Dr22zExWdo7hB1O4zFb33_0 2024-04-22T20:19:12.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.915+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.917+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:19:12.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.918+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-22T20:19:12.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2518530684212959813', time=2024-04-22T22:16:23.269639+0000, chain.objs.size()=9 2024-04-22T20:19:12.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.1 2024-04-22T20:19:12.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.1_1 2024-04-22T20:19:12.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.2 2024-04-22T20:19:12.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.2_1 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.3 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.3_1 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.4 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.4_1 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.5 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17529333467134856014', time=2024-04-22T22:16:55.080893+0000, chain.objs.size()=6 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.82__shadow_.E7eUjQxyRRHEtEvP4QA_W8un3hRDgir_0 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.82__shadow_.E7eUjQxyRRHEtEvP4QA_W8un3hRDgir_1 2024-04-22T20:19:12.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.919+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.82__shadow_.E7eUjQxyRRHEtEvP4QA_W8un3hRDgir_2 2024-04-22T20:19:12.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.922+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.82__shadow_.E7eUjQxyRRHEtEvP4QA_W8un3hRDgir_3 2024-04-22T20:19:12.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.922+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.82__shadow_.E7eUjQxyRRHEtEvP4QA_W8un3hRDgir_4 2024-04-22T20:19:12.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.922+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.82__shadow_.E7eUjQxyRRHEtEvP4QA_W8un3hRDgir_5 2024-04-22T20:19:12.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.923+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17999180944452123143', time=2024-04-22T22:18:44.603212+0000, chain.objs.size()=1 2024-04-22T20:19:12.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.923+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.101__shadow_.gUESy96oMWgyktJ_W6EV09KDqNvaV55_0 2024-04-22T20:19:12.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.923+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2363639991367887374', time=2024-04-22T22:18:47.446835+0000, chain.objs.size()=1 2024-04-22T20:19:12.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.923+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.106__shadow_.nhPBAnf1dIyJtr8iHzKJXPk3Bi5mTEC_0 2024-04-22T20:19:12.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.923+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12342639537100831973', time=2024-04-22T22:18:50.415805+0000, chain.objs.size()=1 2024-04-22T20:19:12.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.923+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.111__shadow_.lv3HYrPMfr7pYpkEQwNtvpZy2x-cTZq_0 2024-04-22T20:19:12.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.927+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.929+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:19:12.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.931+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.931+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.932 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.931+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:19:12.932 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.932+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12520173204138674232', time=2024-04-22T22:18:55.442359+0000, chain.objs.size()=1 2024-04-22T20:19:12.932 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.932+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.120__shadow_.16qoFWXQi_9f0KV_jXTHPgm5PldI0lU_0 2024-04-22T20:19:12.933 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.933+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.935+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:19:12.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.936+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-22T20:19:12.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10447959650092993080', time=2024-04-22T22:17:30.305078+0000, chain.objs.size()=5 2024-04-22T20:19:12.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file2.mpg.2~jGwFkC-0e-rbGgKqekmz5tfnbWvK1_w.1 2024-04-22T20:19:12.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file2.mpg.2~jGwFkC-0e-rbGgKqekmz5tfnbWvK1_w.2 2024-04-22T20:19:12.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file2.mpg.2~jGwFkC-0e-rbGgKqekmz5tfnbWvK1_w.3 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file2.mpg.2~jGwFkC-0e-rbGgKqekmz5tfnbWvK1_w.4 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file2.mpg.2~jGwFkC-0e-rbGgKqekmz5tfnbWvK1_w.5 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10352649897019744503', time=2024-04-22T22:17:30.358864+0000, chain.objs.size()=1 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__shadow_.26OnYkU4iSmfW4LGWk2XqjTz1UruvWs_0 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10950145849251384292', time=2024-04-22T22:18:08.859854+0000, chain.objs.size()=7 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~sbtlG5XklJdT9A0tnruJyqqrAAr_aP-.1 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~sbtlG5XklJdT9A0tnruJyqqrAAr_aP-.1_1 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~sbtlG5XklJdT9A0tnruJyqqrAAr_aP-.1_2 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~sbtlG5XklJdT9A0tnruJyqqrAAr_aP-.2 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~sbtlG5XklJdT9A0tnruJyqqrAAr_aP-.2_1 2024-04-22T20:19:12.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.937+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~sbtlG5XklJdT9A0tnruJyqqrAAr_aP-.2_2 2024-04-22T20:19:12.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~sbtlG5XklJdT9A0tnruJyqqrAAr_aP-.3 2024-04-22T20:19:12.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17623249034996427072', time=2024-04-22T22:18:30.773762+0000, chain.objs.size()=9 2024-04-22T20:19:12.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__multipart_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.1 2024-04-22T20:19:12.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__shadow_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.1_1 2024-04-22T20:19:12.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__multipart_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.2 2024-04-22T20:19:12.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__shadow_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.2_1 2024-04-22T20:19:12.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__multipart_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.3 2024-04-22T20:19:12.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__shadow_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.3_1 2024-04-22T20:19:12.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.942+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__multipart_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.4 2024-04-22T20:19:12.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.944+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__shadow_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.4_1 2024-04-22T20:19:12.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.944+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__multipart_key1.2~Ki_p_tmvX4hq2dBhHVPhj9ObYI6QcNk.5 2024-04-22T20:19:12.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.944+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9700951035498574385', time=2024-04-22T22:18:46.308969+0000, chain.objs.size()=1 2024-04-22T20:19:12.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.944+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.104__shadow_.gBlnKo2w1579hs7mWK9qFFlqElqQqCM_0 2024-04-22T20:19:12.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.945+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.392720013575569620', time=2024-04-22T22:18:48.083719+0000, chain.objs.size()=1 2024-04-22T20:19:12.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.945+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.107__shadow_.iXVMf8GvCtJRuBKfYFgO7KLwnYQoogD_0 2024-04-22T20:19:12.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.945+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14690831464631618635', time=2024-04-22T22:18:57.650732+0000, chain.objs.size()=1 2024-04-22T20:19:12.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.945+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.125__shadow_.nv4zXI8zXEWkMf5RifoUKHZvTfpnkjN_0 2024-04-22T20:19:12.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.949+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.952+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:19:12.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.953+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.953+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.954+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:12.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.954+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9514905617269717554', time=2024-04-22T22:18:46.891391+0000, chain.objs.size()=1 2024-04-22T20:19:12.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.954+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.105__shadow_.gRYS-5YtwQ0flMYxif9AX8jVjMiwiZx_0 2024-04-22T20:19:12.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.954+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5199865274020732390', time=2024-04-22T22:18:52.061664+0000, chain.objs.size()=1 2024-04-22T20:19:12.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.954+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.114__shadow_.S0YgxPHGxwTDj7D8u7YcIA_gf2ATs3k_0 2024-04-22T20:19:12.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.954+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1221915276559221088', time=2024-04-22T22:18:55.496071+0000, chain.objs.size()=1 2024-04-22T20:19:12.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.954+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.120__shadow_.m5_otwpmZnLGzfm8HmXPasSkT0L__33_0 2024-04-22T20:19:12.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.956+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.958+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:19:12.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.959+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.959+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.960+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:12.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.960+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15929692312928913144', time=2024-04-22T22:15:53.897785+0000, chain.objs.size()=1 2024-04-22T20:19:12.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.960+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.66__shadow_.vlVeMGj93124VGSmg--T07NU3fkGcMe_0 2024-04-22T20:19:12.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.960+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16362379584140229361', time=2024-04-22T22:18:44.577594+0000, chain.objs.size()=1 2024-04-22T20:19:12.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.960+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.101__shadow_.ebIF2d-OiQTZsmWCl0L7Vm3dIy2lmac_0 2024-04-22T20:19:12.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.960+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9224714922575766511', time=2024-04-22T22:18:52.008651+0000, chain.objs.size()=1 2024-04-22T20:19:12.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.960+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.114__shadow_.8UDwlVee227A_H-pZ7iEc1ISNQAzvAi_0 2024-04-22T20:19:12.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.962+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.964+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:19:12.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.965+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.202305576542330342', time=2024-04-22T22:16:15.188485+0000, chain.objs.size()=1 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.73__shadow_.YpPoZ61l1--8cMrSxJIe4HuDiI6cw9P_0 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16580595836919762029', time=2024-04-22T22:18:53.188967+0000, chain.objs.size()=1 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.116__shadow_.11yPy23uItnFWjV7X_HiLdxw5DSjjyi_0 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5096368748852538084', time=2024-04-22T22:18:53.851841+0000, chain.objs.size()=1 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.117__shadow_.xdsnenJuRGxsln367rS25xJldWM5y6y_0 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2914731207607919983', time=2024-04-22T22:18:59.175844+0000, chain.objs.size()=1 2024-04-22T20:19:12.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.966+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.128__shadow_.e7UT4UnSxuba_pC9Ezrf9Tru2nP0Lwu_0 2024-04-22T20:19:12.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.968+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.970+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:19:12.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.971+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.971+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.972+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-22T20:19:12.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.972+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.607766375296234564', time=2024-04-22T22:18:53.825038+0000, chain.objs.size()=1 2024-04-22T20:19:12.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.972+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.117__shadow_.HgvydIN8xhdzcTcyd-S2VNIKTTHYCZ3_0 2024-04-22T20:19:12.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.973+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.975+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:19:12.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.976+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-22T20:19:12.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14158422084983529854', time=2024-04-22T22:16:38.342695+0000, chain.objs.size()=9 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.1 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.1_1 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.2 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.2_1 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.3 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.3_1 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.4 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.4_1 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~jR9AuJXFSTdzX2NUp4lC8fA5x7UQjhL.5 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10423968206678206011', time=2024-04-22T22:18:03.732386+0000, chain.objs.size()=9 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.1 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.1_1 2024-04-22T20:19:12.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.977+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.2 2024-04-22T20:19:12.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.980+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.2_1 2024-04-22T20:19:12.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.980+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.3 2024-04-22T20:19:12.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.980+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.3_1 2024-04-22T20:19:12.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.4 2024-04-22T20:19:12.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.4_1 2024-04-22T20:19:12.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~H72QXBmhTtfOKsBKEFOj4OjzHUe3Mvu.5 2024-04-22T20:19:12.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15336724528612743985', time=2024-04-22T22:18:50.916121+0000, chain.objs.size()=1 2024-04-22T20:19:12.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.112__shadow_.ETWBC-PY2xwMKzVDai_ZMQVO8iU6ZWy_0 2024-04-22T20:19:12.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17321055109859756983', time=2024-04-22T22:18:53.267934+0000, chain.objs.size()=1 2024-04-22T20:19:12.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.116__shadow_.gbmDOVzfG818t2hBlxRXjX2MKYsDVfd_0 2024-04-22T20:19:12.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17116480306096114794', time=2024-04-22T22:18:54.935668+0000, chain.objs.size()=1 2024-04-22T20:19:12.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.119__shadow_.8GRRSzR7oGLSVv0dQrcNufZx6twrNow_0 2024-04-22T20:19:12.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.524987615934583372', time=2024-04-22T22:18:58.445085+0000, chain.objs.size()=1 2024-04-22T20:19:12.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.982+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.127__shadow_.tqV1EEPtfqaqMdY-FAJD3zz00L5xAId_0 2024-04-22T20:19:12.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.987+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.989+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:19:12.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.990+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.990+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-22T20:19:12.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5802678289196788236', time=2024-04-22T22:18:45.169883+0000, chain.objs.size()=1 2024-04-22T20:19:12.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.102__shadow_.a7068eZxLV76quJFblcmGCO9sfB1s_y_0 2024-04-22T20:19:12.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3088804989420705498', time=2024-04-22T22:18:45.721749+0000, chain.objs.size()=1 2024-04-22T20:19:12.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.103__shadow_.D6Q_dgY4eJUGuHDUr6Y09VkkD8R-Asx_0 2024-04-22T20:19:12.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4479280108449249557', time=2024-04-22T22:18:48.031207+0000, chain.objs.size()=1 2024-04-22T20:19:12.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.107__shadow_.AkjNZN4UTsG-d41nDo2V48XOu-ptJyW_0 2024-04-22T20:19:12.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6450391486514475455', time=2024-04-22T22:18:56.551062+0000, chain.objs.size()=1 2024-04-22T20:19:12.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.991+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.122__shadow_.SDCRTvEsGOobkJ2WDphzTnHlTz1uA5g_0 2024-04-22T20:19:12.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.992+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:12.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.995+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:19:12.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.996+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:12.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.996+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:12.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-22T20:19:12.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3956129208849726120', time=2024-04-22T22:16:08.920212+0000, chain.objs.size()=1 2024-04-22T20:19:12.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.70__shadow_.wtmuv8MKiIcb690lYVzteAwter_FhIx_0 2024-04-22T20:19:12.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4072478769044966950', time=2024-04-22T22:18:35.863022+0000, chain.objs.size()=9 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__multipart_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.1 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__shadow_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.1_1 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__multipart_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.2 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__shadow_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.2_1 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__multipart_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.3 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__shadow_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.3_1 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__multipart_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.4 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__shadow_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.4_1 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.93__multipart_key1.2~QMobLRBWL-QFY5t2MeZxj1sf3OZnAjq.5 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12681643542017096307', time=2024-04-22T22:18:44.523460+0000, chain.objs.size()=1 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.101__shadow_.h2SpucF29Efg65tzvbotktFJKCj6A9O_0 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18149529484858791822', time=2024-04-22T22:18:48.110567+0000, chain.objs.size()=1 2024-04-22T20:19:13.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.997+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.107__shadow_.rcDeIcDFj1UR9Kue84hKhd8kCAxTAxh_0 2024-04-22T20:19:13.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.999+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.663738232249029507', time=2024-04-22T22:18:51.500180+0000, chain.objs.size()=1 2024-04-22T20:19:13.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.999+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.113__shadow_.sTrT7sgGEEV0_b5ecrbs-knAiQTrqrq_0 2024-04-22T20:19:13.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.999+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10600010039511272388', time=2024-04-22T22:18:52.034812+0000, chain.objs.size()=1 2024-04-22T20:19:13.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:12.999+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.114__shadow_.csTjuJhVBci_Iq08-LvHL94532vWjWf_0 2024-04-22T20:19:13.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.003+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.006 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.005+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:19:13.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.007+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.007+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.008 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:19:13.008 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15387889407666255308', time=2024-04-22T22:16:41.090629+0000, chain.objs.size()=9 2024-04-22T20:19:13.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.1 2024-04-22T20:19:13.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.1_1 2024-04-22T20:19:13.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.2 2024-04-22T20:19:13.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.2_1 2024-04-22T20:19:13.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.3 2024-04-22T20:19:13.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.3_1 2024-04-22T20:19:13.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.4 2024-04-22T20:19:13.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.4_1 2024-04-22T20:19:13.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~PeReo24vl8U9WI3LAn664sOETYLNt-U.5 2024-04-22T20:19:13.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5338457679681805109', time=2024-04-22T22:18:58.039289+0000, chain.objs.size()=1 2024-04-22T20:19:13.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.008+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.126__shadow_.CWARVirzyGoNsDJaea-ZySZYGV3G6UH_0 2024-04-22T20:19:13.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.012+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.015+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:19:13.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.016+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.016+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.017+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:13.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.017+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11193039673987988400', time=2024-04-22T22:18:45.694425+0000, chain.objs.size()=1 2024-04-22T20:19:13.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.017+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.103__shadow_.bDeIhEEBPtEnf1lmyF6LCvyTO7STRCO_0 2024-04-22T20:19:13.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.017+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3446487545253220570', time=2024-04-22T22:18:49.271421+0000, chain.objs.size()=1 2024-04-22T20:19:13.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.017+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.109__shadow_.RbAXJwrMDtG1XuBEzymD_2YLAv4m1yK_0 2024-04-22T20:19:13.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.017+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8512063506300839235', time=2024-04-22T22:18:56.523455+0000, chain.objs.size()=1 2024-04-22T20:19:13.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.017+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.122__shadow_.oBGankCJty4pjM4CBOSDabQrWkLgr4F_0 2024-04-22T20:19:13.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.018+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.020+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:19:13.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.021+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.022+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.022+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:13.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.022+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17776686222224825579', time=2024-04-22T22:18:45.748992+0000, chain.objs.size()=1 2024-04-22T20:19:13.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.022+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.103__shadow_.gcwTxxkZ9cvCSjX075lo5mKwP0GzY9Z_0 2024-04-22T20:19:13.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.022+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1772267353208287021', time=2024-04-22T22:18:54.349291+0000, chain.objs.size()=1 2024-04-22T20:19:13.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.022+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.118__shadow_.gD_f30UbPpkF5W34Mcy-UvQJy-_zDA5_0 2024-04-22T20:19:13.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.022+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4148861752362104784', time=2024-04-22T22:18:55.970310+0000, chain.objs.size()=1 2024-04-22T20:19:13.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.022+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.121__shadow_.60YgHetKBYkrWsPVQ9l2jlOUOm9lsVn_0 2024-04-22T20:19:13.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.024+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.026+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:19:13.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.027+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.027+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-22T20:19:13.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1558737143987527731', time=2024-04-22T22:18:47.526163+0000, chain.objs.size()=1 2024-04-22T20:19:13.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.106__shadow_.AbjFOQFXxx1VtZ_YsGStmwbx13ZtGaS_0 2024-04-22T20:19:13.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15639293168955991203', time=2024-04-22T22:18:48.611717+0000, chain.objs.size()=1 2024-04-22T20:19:13.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.108__shadow_.1Tp8B23FuDWXRK-ziyX4VN5fxk5kMFc_0 2024-04-22T20:19:13.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12386854359505357955', time=2024-04-22T22:18:49.798305+0000, chain.objs.size()=1 2024-04-22T20:19:13.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.110__shadow_.AS1NRTmufnO6O4kecHOmUbEk3_N0rzD_0 2024-04-22T20:19:13.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17298903893943253698', time=2024-04-22T22:18:59.590144+0000, chain.objs.size()=1 2024-04-22T20:19:13.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.028+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.129__shadow_.cP-_LWoPyPpwsjIWrBtA_6Uy9mnsadW_0 2024-04-22T20:19:13.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.030+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.032+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:19:13.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.033+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14916942605539509372', time=2024-04-22T22:17:30.249493+0000, chain.objs.size()=9 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.1 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__shadow_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.1_1 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.2 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__shadow_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.2_1 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.3 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__shadow_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.3_1 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.4 2024-04-22T20:19:13.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__shadow_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.4_1 2024-04-22T20:19:13.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__multipart_file.mpg.2~uuV-gfzf6UA5LbwMgjOkfpSi_JdcCLO.5 2024-04-22T20:19:13.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4341299678139802313', time=2024-04-22T22:18:46.361612+0000, chain.objs.size()=1 2024-04-22T20:19:13.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.104__shadow_.gQHeehLVO9OPJT0I5ggmuZitgd8-Bdn_0 2024-04-22T20:19:13.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6948318657546669168', time=2024-04-22T22:18:46.864679+0000, chain.objs.size()=1 2024-04-22T20:19:13.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.105__shadow_.-zLCk9HAeYHLVnLP0Hgiqir6zJpTSZ-_0 2024-04-22T20:19:13.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.799430848720744189', time=2024-04-22T22:18:53.240893+0000, chain.objs.size()=1 2024-04-22T20:19:13.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.034+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.116__shadow_.4iBKo6RQhoLhq6hoa81lLTQNZrVJu2X_0 2024-04-22T20:19:13.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4374467595003372821', time=2024-04-22T22:18:54.908995+0000, chain.objs.size()=1 2024-04-22T20:19:13.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.119__shadow_.7O1gGUJnffZUS41B3vKfOLD85QKdBKt_0 2024-04-22T20:19:13.039 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18181127160128530159', time=2024-04-22T22:19:06.059744+0000, chain.objs.size()=9 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__multipart_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.1 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__shadow_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.1_1 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__multipart_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.2 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__shadow_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.2_1 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__multipart_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.3 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__shadow_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.3_1 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__multipart_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.4 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__shadow_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.4_1 2024-04-22T20:19:13.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.038+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.132__multipart_key1.2~SEZI7o0BIFCh4t83iRNv1yZnpdrol5s.5 2024-04-22T20:19:13.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.044+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.046+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:19:13.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.048+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.048+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.048+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.048+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16978552255470233479', time=2024-04-22T22:16:55.273228+0000, chain.objs.size()=7 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.048+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~1pYqEViemV31kPteHWRoujEbvoFp3TE.1 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.048+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~1pYqEViemV31kPteHWRoujEbvoFp3TE.1_1 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.048+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~1pYqEViemV31kPteHWRoujEbvoFp3TE.1_2 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.048+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~1pYqEViemV31kPteHWRoujEbvoFp3TE.2 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~1pYqEViemV31kPteHWRoujEbvoFp3TE.2_1 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~1pYqEViemV31kPteHWRoujEbvoFp3TE.2_2 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~1pYqEViemV31kPteHWRoujEbvoFp3TE.3 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh', time=2024-04-22T22:18:39.196136+0000, chain.objs.size()=1 2024-04-22T20:19:13.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.13 2024-04-22T20:19:13.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh', time=2024-04-22T22:18:39.254065+0000, chain.objs.size()=20 2024-04-22T20:19:13.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.1 2024-04-22T20:19:13.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.2 2024-04-22T20:19:13.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.3 2024-04-22T20:19:13.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.049+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.4 2024-04-22T20:19:13.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.051+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.5 2024-04-22T20:19:13.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.051+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.6 2024-04-22T20:19:13.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.051+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.7 2024-04-22T20:19:13.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.051+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.8 2024-04-22T20:19:13.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.052+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.9 2024-04-22T20:19:13.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.054+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.10 2024-04-22T20:19:13.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.054+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.11 2024-04-22T20:19:13.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.054+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.12 2024-04-22T20:19:13.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.054+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.pzAkh52EaXtNrs7EHeSDIA4ZkjZxwb4C.13 2024-04-22T20:19:13.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.054+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.14 2024-04-22T20:19:13.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.054+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.15 2024-04-22T20:19:13.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.055+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.16 2024-04-22T20:19:13.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.057+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.17 2024-04-22T20:19:13.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.057+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.18 2024-04-22T20:19:13.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.057+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.19 2024-04-22T20:19:13.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.057+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.94__multipart_key1.2~EkNJzf2Tv9ShrGvtrDUu1-djGBZVxBh.20 2024-04-22T20:19:13.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.059+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18254255469068915829', time=2024-04-22T22:18:49.770947+0000, chain.objs.size()=1 2024-04-22T20:19:13.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.059+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.110__shadow_.fMi9oqhgxztOu3hNbTHkar-K3NDWGfY_0 2024-04-22T20:19:13.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.059+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17405539920891405325', time=2024-04-22T22:18:54.431681+0000, chain.objs.size()=1 2024-04-22T20:19:13.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.059+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.118__shadow_.xB17GLcKOK0-cKfAw-YUJGgNLVEX6qK_0 2024-04-22T20:19:13.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.061+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.064+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:19:13.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.065+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.065+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7385071344685808151', time=2024-04-22T22:17:01.759851+0000, chain.objs.size()=9 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__multipart_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.1 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__shadow_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.1_1 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__multipart_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.2 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__shadow_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.2_1 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__multipart_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.3 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__shadow_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.3_1 2024-04-22T20:19:13.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__multipart_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.4 2024-04-22T20:19:13.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__shadow_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.4_1 2024-04-22T20:19:13.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.87__multipart_key1.2~qHaLjr7haf3qQCa1iqf0i5ytTKsnlDQ.5 2024-04-22T20:19:13.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9913432828555610492', time=2024-04-22T22:18:27.407186+0000, chain.objs.size()=1 2024-04-22T20:19:13.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.92__shadow_.5ymFqxHUElVdNeTfJhNM5ziah8vRfVx_0 2024-04-22T20:19:13.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5882341371148524495', time=2024-04-22T22:18:48.058037+0000, chain.objs.size()=1 2024-04-22T20:19:13.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.066+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.107__shadow_.7f04NWvifjEYVtRN3k_UkDEkCrABzeW_0 2024-04-22T20:19:13.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.071+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.073+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:19:13.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-04-22T20:19:13.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2940160587261130346', time=2024-04-22T22:17:30.277065+0000, chain.objs.size()=1 2024-04-22T20:19:13.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__shadow_.bcpKG--wYqPyyksjiC6mwdvyr-7lBHF_0 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10589839030685148342', time=2024-04-22T22:17:53.736285+0000, chain.objs.size()=9 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.1 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.1_1 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.2 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.2_1 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.3 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.3_1 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.4 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.4_1 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~D3qDymHrhHTBeOmOMNyp-Z3uebYnYkj.5 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.075+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2719295968202775522', time=2024-04-22T22:18:45.142497+0000, chain.objs.size()=1 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.076+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.102__shadow_.pIfSEIM5m7JByqFk3VHPqWx0k89CsOh_0 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.076+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13150795718817661108', time=2024-04-22T22:18:46.945099+0000, chain.objs.size()=1 2024-04-22T20:19:13.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.076+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.105__shadow_.TKlIVdZmywLU2tzb2GokmHsxSOrOo0z_0 2024-04-22T20:19:13.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.079+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10936235539621245960', time=2024-04-22T22:18:50.942368+0000, chain.objs.size()=1 2024-04-22T20:19:13.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.079+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.112__shadow_.Co4LwyFpSqBOoT2FsuB_noGswHkUaWI_0 2024-04-22T20:19:13.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.079+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10378865586111371687', time=2024-04-22T22:18:58.358038+0000, chain.objs.size()=1 2024-04-22T20:19:13.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.079+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.127__shadow_.oAe1B4lKtCXs-8JYQgX3co3Aoqt-g3J_0 2024-04-22T20:19:13.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.081+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.083+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:19:13.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.085+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.085+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.085+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:19:13.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.085+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7018038301295302597', time=2024-04-22T22:18:47.472442+0000, chain.objs.size()=1 2024-04-22T20:19:13.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.085+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.106__shadow_.SfWu-pmeTsoOZzjlver6YW2I6SBe86K_0 2024-04-22T20:19:13.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.085+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3244879105340496677', time=2024-04-22T22:18:56.022791+0000, chain.objs.size()=1 2024-04-22T20:19:13.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.085+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.121__shadow_.uVNr2j-BwD7ogCxb3ayjp6kamkHt0eg_0 2024-04-22T20:19:13.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.086+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.089+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:19:13.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:13.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15186899108578957453', time=2024-04-22T22:18:48.665116+0000, chain.objs.size()=1 2024-04-22T20:19:13.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.108__shadow_.4O3h64l7z_r71_WZEK6M7U3V0W0t9xx_0 2024-04-22T20:19:13.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17995620970673453428', time=2024-04-22T22:18:56.497196+0000, chain.objs.size()=1 2024-04-22T20:19:13.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.122__shadow_.zi3aTkihE49PAA0pbZGb59ubRkEG1b9_0 2024-04-22T20:19:13.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13126659750769870786', time=2024-04-22T22:19:12.072455+0000, chain.objs.size()=1 2024-04-22T20:19:13.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.090+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__shadow_.3VDtUiicGZjgqpmwfew3HIOpgkPargJ_0 2024-04-22T20:19:13.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.092+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.094+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:19:13.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.095+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9424728486916961870', time=2024-04-22T22:16:15.621351+0000, chain.objs.size()=1 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.74__shadow_.-isX8e9wNfz8Df0L4pzR14cpn8qal0a_0 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18154544997937098072', time=2024-04-22T22:17:58.960953+0000, chain.objs.size()=9 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.1 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.1_1 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.2 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.2_1 2024-04-22T20:19:13.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.3 2024-04-22T20:19:13.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.3_1 2024-04-22T20:19:13.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.4 2024-04-22T20:19:13.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.4_1 2024-04-22T20:19:13.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~W54HP5-5M43oMi7FlGGx5ICdyHFankr.5 2024-04-22T20:19:13.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2580691188644092617', time=2024-04-22T22:18:45.775190+0000, chain.objs.size()=1 2024-04-22T20:19:13.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.103__shadow_.Q9mjkfRQSjMvRfjt6itHAMw0PaP5jBY_0 2024-04-22T20:19:13.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14984056070655736636', time=2024-04-22T22:18:46.918107+0000, chain.objs.size()=1 2024-04-22T20:19:13.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.096+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.105__shadow_.bhK7xL23QGjerw8bxoCrHoykde1tlyo_0 2024-04-22T20:19:13.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.099+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3583719185104665356', time=2024-04-22T22:18:47.499597+0000, chain.objs.size()=1 2024-04-22T20:19:13.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.099+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.106__shadow_.xoJpMYLnktEvL3CmbiRcdCxn_l4CkGC_0 2024-04-22T20:19:13.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.101+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.104+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:19:13.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.105+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.105+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-04-22T20:19:13.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12707685257802008249', time=2024-04-22T22:17:30.332497+0000, chain.objs.size()=1 2024-04-22T20:19:13.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89__shadow_.wwCjpRuLYBbhg4glchhS0lOY9k3HA6W_0 2024-04-22T20:19:13.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17976093510680580919', time=2024-04-22T22:18:45.112371+0000, chain.objs.size()=1 2024-04-22T20:19:13.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.102__shadow_.OW_xrMf70E20vLzv3loJn4TUsZJSMJq_0 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16303223303993018049', time=2024-04-22T22:18:49.187834+0000, chain.objs.size()=1 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.109__shadow_.87go2xwMpZCPX3htMpVsoJ4uhrOrCoj_0 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.189058275464599383', time=2024-04-22T22:18:50.388698+0000, chain.objs.size()=1 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.111__shadow_.w4HzHK2f_SYIi7ZKqSfXKXHgnRhJ5qR_0 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4434997124380362161', time=2024-04-22T22:18:50.441957+0000, chain.objs.size()=1 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.111__shadow_.mOdaKnXQJBpc1fakf7T5EPDs6YGNjBK_0 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8723427390079493079', time=2024-04-22T22:18:51.439658+0000, chain.objs.size()=1 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.113__shadow_.SkT7uMcKk2rZM2x85lPF3K2Bq1LToUP_0 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16853223721000387067', time=2024-04-22T22:18:54.377198+0000, chain.objs.size()=1 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.118__shadow_.iyc30e-rZhwbtzo8tsn4eLk_rTSgXiM_0 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11040745435893270942', time=2024-04-22T22:19:12.046062+0000, chain.objs.size()=9 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__multipart_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.1 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__shadow_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.1_1 2024-04-22T20:19:13.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__multipart_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.2 2024-04-22T20:19:13.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__shadow_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.2_1 2024-04-22T20:19:13.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.106+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__multipart_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.3 2024-04-22T20:19:13.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.109+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__shadow_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.3_1 2024-04-22T20:19:13.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.109+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__multipart_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.4 2024-04-22T20:19:13.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.109+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__shadow_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.4_1 2024-04-22T20:19:13.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.111+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133__multipart_key1/data/file.mpg.2~0758Ir4aGRDGBOi6VxFNtueX20pw1j_.5 2024-04-22T20:19:13.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.113+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.116+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:19:13.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.117+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.117+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.576004996066964037', time=2024-04-22T22:17:19.501217+0000, chain.objs.size()=9 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__multipart_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.1 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__shadow_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.1_1 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__multipart_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.2 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__shadow_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.2_1 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__multipart_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.3 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__shadow_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.3_1 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__multipart_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.4 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__shadow_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.4_1 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.88__multipart_key1.2~5tBOYVY9kt15gtpFk8LuWPQ5sRze6z0.5 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8283453220330620306', time=2024-04-22T22:18:13.835696+0000, chain.objs.size()=7 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~wnylE6AMVO4lJiBDxb5Ac6LXwDEK3xg.1 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~wnylE6AMVO4lJiBDxb5Ac6LXwDEK3xg.1_1 2024-04-22T20:19:13.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.118+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~wnylE6AMVO4lJiBDxb5Ac6LXwDEK3xg.1_2 2024-04-22T20:19:13.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.121+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~wnylE6AMVO4lJiBDxb5Ac6LXwDEK3xg.2 2024-04-22T20:19:13.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.122+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~wnylE6AMVO4lJiBDxb5Ac6LXwDEK3xg.2_1 2024-04-22T20:19:13.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.122+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__shadow_key1.2~wnylE6AMVO4lJiBDxb5Ac6LXwDEK3xg.2_2 2024-04-22T20:19:13.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.122+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.91__multipart_key1.2~wnylE6AMVO4lJiBDxb5Ac6LXwDEK3xg.3 2024-04-22T20:19:13.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.122+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10556677485455421885', time=2024-04-22T22:18:52.600926+0000, chain.objs.size()=1 2024-04-22T20:19:13.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.122+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.115__shadow_.ijQM2lhPcE52KdHJgu1CX1PQiylRUWf_0 2024-04-22T20:19:13.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.123+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1341712683091962338', time=2024-04-22T22:18:55.995862+0000, chain.objs.size()=1 2024-04-22T20:19:13.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.123+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.121__shadow_.LzY46jlmivWFWEe8zFFpvUfpik5KNqY_0 2024-04-22T20:19:13.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.127+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.129+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:19:13.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.130+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9774696286959133319', time=2024-04-22T22:16:35.157077+0000, chain.objs.size()=9 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.1 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.1_1 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.2 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.2_1 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.3 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.3_1 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.4 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__shadow_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.4_1 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.83__multipart_key1.2~49aLJsVI0GVsoJDy9su55HJgAtfZ7I4.5 2024-04-22T20:19:13.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15434615847815723700', time=2024-04-22T22:18:52.627280+0000, chain.objs.size()=1 2024-04-22T20:19:13.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.131+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.115__shadow_.Ik8iRyhD6JicGEkoj9UIcCDJ6sl3ZSy_0 2024-04-22T20:19:13.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.135+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.137+0000 7fc146133800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:19:13.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.138+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-04-22T20:19:13.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.1_1_c96f15f49a15b32ce91ddf6f093e4d3a', time=2024-04-22T22:16:23.463290+0000, chain.objs.size()=9 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.1 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.1_1 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.2 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.2_1 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.3 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.3_1 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.4 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__shadow_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.4_1 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80__multipart_key1.2~RpFgaCWtlkMOwIOhO3dNdWZozB3Z_DA.5 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15126023144428873347', time=2024-04-22T22:18:46.281236+0000, chain.objs.size()=1 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.104__shadow_.aXdwnPcFEjk7WoJrhQZJqfQP_SBYJaw_0 2024-04-22T20:19:13.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9273726124964840558', time=2024-04-22T22:18:48.691165+0000, chain.objs.size()=1 2024-04-22T20:19:13.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.139+0000 7fc146133800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.108__shadow_.ejk7YXAv6ZGJ1sk331KhIPXJoa2cYAj_0 2024-04-22T20:19:13.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.142+0000 7fc146133800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:19:13.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.145+0000 7fbff37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:13.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.147+0000 7fc146133800 20 remove_watcher() i=0 2024-04-22T20:19:13.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.147+0000 7fc146133800 2 removed watcher, disabling cache 2024-04-22T20:19:13.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.148+0000 7fc146133800 20 remove_watcher() i=1 2024-04-22T20:19:13.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.149+0000 7fc146133800 20 remove_watcher() i=2 2024-04-22T20:19:13.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.150+0000 7fc146133800 20 remove_watcher() i=3 2024-04-22T20:19:13.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.151+0000 7fc146133800 20 remove_watcher() i=4 2024-04-22T20:19:13.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.151+0000 7fc146133800 20 remove_watcher() i=5 2024-04-22T20:19:13.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.152+0000 7fc146133800 20 remove_watcher() i=6 2024-04-22T20:19:13.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.153+0000 7fc146133800 20 remove_watcher() i=7 2024-04-22T20:19:13.164 DEBUG:teuthology.orchestra.run.smithi136:> radosgw-admin gc process --include-all 2024-04-22T20:19:13.212 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:13.212 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:13.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.246+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.247+0000 7f3e28323800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.247+0000 7f3e28323800 20 realm 2024-04-22T20:19:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.247+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.247+0000 7f3e28323800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.247+0000 7f3e28323800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.247+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.247+0000 7f3e28323800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.248+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.248+0000 7f3e28323800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:13.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.248+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.249+0000 7f3e28323800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:13.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.249+0000 7f3e28323800 20 searching for the correct realm 2024-04-22T20:19:13.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.249+0000 7f3e240b7640 0 -- 172.21.15.136:0/2135995198 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x55d3976660f0 msgr2=0x55d3976685b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:13.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.253+0000 7f3e240b7640 0 -- 172.21.15.136:0/2135995198 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x7f3e1c0be480 msgr2=0x7f3e1c0907f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:13.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.461+0000 7f3e28323800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:13.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.461+0000 7f3e28323800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:13.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.461+0000 7f3e28323800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:13.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.461+0000 7f3e28323800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:13.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.461+0000 7f3e28323800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:13.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.461+0000 7f3e28323800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:13.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.461+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.462+0000 7f3e28323800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:13.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.462+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.462+0000 7f3e28323800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:13.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.462+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 zone default found 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 4 Realm: () 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 zonegroup default 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 rados->read ofs=0 len=0 2024-04-22T20:19:13.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:13.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 started sync module instance, tier type = 2024-04-22T20:19:13.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.463+0000 7f3e28323800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:13.472 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.472+0000 7f3e28323800 20 add_watcher() i=0 2024-04-22T20:19:13.472 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.472+0000 7f3e28323800 20 add_watcher() i=1 2024-04-22T20:19:13.472 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.472+0000 7f3e28323800 20 add_watcher() i=2 2024-04-22T20:19:13.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.472+0000 7f3e28323800 20 add_watcher() i=3 2024-04-22T20:19:13.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.472+0000 7f3e28323800 20 add_watcher() i=4 2024-04-22T20:19:13.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.472+0000 7f3e28323800 20 add_watcher() i=5 2024-04-22T20:19:13.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.472+0000 7f3e28323800 20 add_watcher() i=6 2024-04-22T20:19:13.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.473+0000 7f3e28323800 20 add_watcher() i=7 2024-04-22T20:19:13.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.473+0000 7f3e28323800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:13.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.474+0000 7f3cdafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:13.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.475+0000 7f3e28323800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:13.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.475+0000 7f3e28323800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:13.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.510+0000 7f3cd9fdb640 20 reqs_thread_entry: start 2024-04-22T20:19:13.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.510+0000 7f3e28323800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:13.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.510+0000 7f3e28323800 20 Filter name: none 2024-04-22T20:19:13.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.510+0000 7f3c4b7fe640 20 reqs_thread_entry: start 2024-04-22T20:19:13.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.510+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:19:13.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.511+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.512+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.512+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.513+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:19:13.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.514+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.515+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.515+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.516+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:19:13.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.517+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.518+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.518+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.519+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:19:13.520 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.520+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.521+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.521+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.522+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:19:13.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.523+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.524+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.524+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.525 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.525+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:19:13.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.526+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.527+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.527+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.528+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:19:13.529 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.529+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.529 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.529+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.530 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.530+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.531+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:19:13.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.532+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.532+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.533+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.534+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:19:13.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.535+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.535+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.536+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.537+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:19:13.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.538+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.539+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.539+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.540+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:19:13.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.541+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.541+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.542+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.542+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:19:13.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.544+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.545+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.545+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.546+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:19:13.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.547+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.548 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.547+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.548 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.548+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.549 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.549+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:19:13.550 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.550+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.551 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.551+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.551 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.551+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.552 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.552+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:19:13.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.553+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.553+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.554 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.554+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.555 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.555+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:19:13.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.556+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.556+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.557 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.557+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.558+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:19:13.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.559+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.559+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.560+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.560+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:19:13.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.562+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.562+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.562+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.563+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:19:13.565 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.565+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.565 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.565+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.565+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.566+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:19:13.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.568+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.568+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.568+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.569+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:19:13.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.570+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.571+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.571+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.572 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.572+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:19:13.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.573+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.574+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.574+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.575+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:19:13.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.577+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.577+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.577+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.578+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:19:13.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.580+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.580+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.580+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.581+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:19:13.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.582+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.583+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.583+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.584+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:19:13.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.585+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.585+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.586+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.586+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:19:13.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.588+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.588+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.589+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.589+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:19:13.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.591+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.591+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.591+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.592+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:19:13.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.594+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.594+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.594+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.595+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:19:13.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.596+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.597+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.597+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.598+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:19:13.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.599+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.599+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.600+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.601+0000 7f3e28323800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:19:13.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.602+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.602+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:19:13.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.603+0000 7f3e28323800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:19:13.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.604+0000 7f3cdafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:13.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.606+0000 7f3e28323800 20 remove_watcher() i=0 2024-04-22T20:19:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.606+0000 7f3e28323800 2 removed watcher, disabling cache 2024-04-22T20:19:13.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.607+0000 7f3e28323800 20 remove_watcher() i=1 2024-04-22T20:19:13.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.608+0000 7f3e28323800 20 remove_watcher() i=2 2024-04-22T20:19:13.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.609+0000 7f3e28323800 20 remove_watcher() i=3 2024-04-22T20:19:13.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.610+0000 7f3e28323800 20 remove_watcher() i=4 2024-04-22T20:19:13.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.612+0000 7f3e28323800 20 remove_watcher() i=5 2024-04-22T20:19:13.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.612+0000 7f3e28323800 20 remove_watcher() i=6 2024-04-22T20:19:13.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:13.613+0000 7f3e28323800 20 remove_watcher() i=7 2024-04-22T20:19:13.624 INFO:teuthology.run_tasks:Running task s3tests... 2024-04-22T20:19:13.641 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 tenant': 'RGW33333333333333333'}, 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}} 2024-04-22T20:19:13.642 INFO:tasks.s3tests:Downloading s3-tests... 2024-04-22T20:19:13.642 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2024-04-22T20:19:13.642 DEBUG:teuthology.orchestra.run.smithi136:> git clone -b ceph-master https://git.ceph.com/s3-tests.git /home/ubuntu/cephtest/s3-tests-client.0 2024-04-22T20:19:13.670 INFO:teuthology.orchestra.run.smithi136.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-client.0'... 2024-04-22T20:19:14.388 INFO:tasks.s3tests:Creating rgw users... 2024-04-22T20:19:14.389 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2024-04-22T20:19:14.389 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name Mr.foo.client.0 --email foo+test@test.test --access-key HBOZBNCKXACENFYQIJKD --secret ngwUYZcia0cV5Q8cMvWhNySkDhrEk/XGejiNxicsYnpiPS7DYhQI0w== --cluster ceph 2024-04-22T20:19:14.491 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:14.491 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:14.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.521+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.523+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.523+0000 7faf84133800 20 realm 2024-04-22T20:19:14.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.523+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.523+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.523+0000 7faf84133800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:14.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.523+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.523+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.523+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.524+0000 7faf84133800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:14.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.524+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.525 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.525+0000 7faf84133800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:14.525 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.525+0000 7faf84133800 20 searching for the correct realm 2024-04-22T20:19:14.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.527+0000 7faf806f3640 0 -- 172.21.15.136:0/2049043387 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7faf78054d50 msgr2=0x7faf78057ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:14.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.733+0000 7faf7f6f1640 0 -- 172.21.15.136:0/2049043387 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7faf7800f810 msgr2=0x7faf78071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.940+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:14.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 20 zone default found 2024-04-22T20:19:14.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 4 Realm: () 2024-04-22T20:19:14.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:14.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:14.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:14.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 20 zonegroup default 2024-04-22T20:19:14.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.941+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.942+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.942+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.942+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.942+0000 7faf84133800 20 started sync module instance, tier type = 2024-04-22T20:19:14.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.942+0000 7faf84133800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:14.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.952+0000 7faf84133800 20 add_watcher() i=0 2024-04-22T20:19:14.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.952+0000 7faf84133800 20 add_watcher() i=1 2024-04-22T20:19:14.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.952+0000 7faf84133800 20 add_watcher() i=2 2024-04-22T20:19:14.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.952+0000 7faf84133800 20 add_watcher() i=3 2024-04-22T20:19:14.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.952+0000 7faf84133800 20 add_watcher() i=4 2024-04-22T20:19:14.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.952+0000 7faf84133800 20 add_watcher() i=5 2024-04-22T20:19:14.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.952+0000 7faf84133800 20 add_watcher() i=6 2024-04-22T20:19:14.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.953+0000 7faf84133800 20 add_watcher() i=7 2024-04-22T20:19:14.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.953+0000 7faf84133800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:14.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.954+0000 7fae35fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.954+0000 7faf84133800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.954+0000 7faf84133800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.954+0000 7faf84133800 5 note: GC not initialized 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7fae34fe1640 20 reqs_thread_entry: start 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 20 Filter name: none 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7fadae7fc640 20 reqs_thread_entry: start 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-04-22T20:19:14.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2024-04-22T20:19:14.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-22T20:19:14.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 10 cache get: name=default.rgw.meta+users.email+foo+test@test.test : miss 2024-04-22T20:19:14.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.955+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x0 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 10 adding default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 10 cache get: name=default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD : miss 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 20 rados->read ofs=0 len=0 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 10 cache put: name=default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD info.flags=0x0 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 10 adding default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD to cache LRU end 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 10 cache get: name=default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD : hit (negative entry) 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.956+0000 7faf84133800 10 cache get: name=default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD : hit (negative entry) 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.957+0000 7faf84133800 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.957+0000 7faf84133800 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.957+0000 7faf84133800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-04-22T20:19:14.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.957+0000 7fae5effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562621 cookie 94764433653632 notifier 7638 bl.length()=645 2024-04-22T20:19:14.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.957+0000 7fae5effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-04-22T20:19:14.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.957+0000 7fae5effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-22T20:19:14.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.961+0000 7faf84133800 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-04-22T20:19:14.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.961+0000 7faf84133800 10 moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-04-22T20:19:14.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.961+0000 7faf84133800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:foo+test@test.test, ofs0, ns] 2024-04-22T20:19:14.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.961+0000 7fae5effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562622 cookie 94764433666544 notifier 7638 bl.length()=185 2024-04-22T20:19:14.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.961+0000 7fae5effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-04-22T20:19:14.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.961+0000 7fae5effd640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-04-22T20:19:14.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.964+0000 7faf84133800 10 cache put: name=default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD info.flags=0x7 2024-04-22T20:19:14.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.964+0000 7faf84133800 10 moving default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD to cache LRU end 2024-04-22T20:19:14.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.964+0000 7faf84133800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:HBOZBNCKXACENFYQIJKD, ofs0, ns] 2024-04-22T20:19:14.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.965+0000 7fae5e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1108101562506 cookie 94764433671584 notifier 7638 bl.length()=186 2024-04-22T20:19:14.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.965+0000 7fae5e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD info.flags=0x7 2024-04-22T20:19:14.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.965+0000 7fae5e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+HBOZBNCKXACENFYQIJKD to cache LRU end 2024-04-22T20:19:14.966 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "foo.client.0", 2024-04-22T20:19:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr.foo.client.0", 2024-04-22T20:19:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "foo+test@test.test", 2024-04-22T20:19:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:19:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "foo.client.0", 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "HBOZBNCKXACENFYQIJKD", 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "ngwUYZcia0cV5Q8cMvWhNySkDhrEk/XGejiNxicsYnpiPS7DYhQI0w==", 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:14.957738Z" 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:19:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "rgw", 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "", 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:14.957723Z", 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:19:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:19:14.970 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:14.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.967+0000 7fae35fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:14.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.970+0000 7faf84133800 20 remove_watcher() i=0 2024-04-22T20:19:14.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.970+0000 7faf84133800 2 removed watcher, disabling cache 2024-04-22T20:19:14.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.971+0000 7faf84133800 20 remove_watcher() i=1 2024-04-22T20:19:14.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.972+0000 7faf84133800 20 remove_watcher() i=2 2024-04-22T20:19:14.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.973+0000 7faf84133800 20 remove_watcher() i=3 2024-04-22T20:19:14.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.974+0000 7faf84133800 20 remove_watcher() i=4 2024-04-22T20:19:14.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.975+0000 7faf84133800 20 remove_watcher() i=5 2024-04-22T20:19:14.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.976+0000 7faf84133800 20 remove_watcher() i=6 2024-04-22T20:19:14.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:14.977+0000 7faf84133800 20 remove_watcher() i=7 2024-04-22T20:19:14.988 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 7404601046 --totp-seed V64TSR7VMX7KZ45V455KWE5N3APFBPZHP27MCAJXPATJLPTZRXZB7YBGDNAWGBUX --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-22T20:19:15.053 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:15.053 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:15.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.084+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.085+0000 7f2cd7b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:15.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.085+0000 7f2cd7b2e800 20 realm 2024-04-22T20:19:15.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.085+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.085+0000 7f2cd7b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:15.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.085+0000 7f2cd7b2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:15.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.085+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.085+0000 7f2cd7b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:15.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.085+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.086+0000 7f2cd7b2e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:15.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.086+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.087+0000 7f2cd7b2e800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:15.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.087+0000 7f2cd7b2e800 20 searching for the correct realm 2024-04-22T20:19:15.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.088+0000 7f2cd3066640 0 -- 172.21.15.136:0/533570262 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f2cc4061bb0 msgr2=0x7f2cc4068040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:15.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.092+0000 7f2cd3066640 0 -- 172.21.15.136:0/533570262 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7f2b9c07ba00 msgr2=0x7f2b9c07de30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:15.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:15.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:15.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.099+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:15.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 20 zone default found 2024-04-22T20:19:15.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 4 Realm: () 2024-04-22T20:19:15.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 20 zonegroup default 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.100+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.101+0000 7f2cd7b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.101+0000 7f2cd7b2e800 20 started sync module instance, tier type = 2024-04-22T20:19:15.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.101+0000 7f2cd7b2e800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:15.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.110+0000 7f2cd7b2e800 20 add_watcher() i=0 2024-04-22T20:19:15.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.110+0000 7f2cd7b2e800 20 add_watcher() i=1 2024-04-22T20:19:15.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.110+0000 7f2cd7b2e800 20 add_watcher() i=2 2024-04-22T20:19:15.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.110+0000 7f2cd7b2e800 20 add_watcher() i=3 2024-04-22T20:19:15.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.110+0000 7f2cd7b2e800 20 add_watcher() i=4 2024-04-22T20:19:15.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.110+0000 7f2cd7b2e800 20 add_watcher() i=5 2024-04-22T20:19:15.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.110+0000 7f2cd7b2e800 20 add_watcher() i=6 2024-04-22T20:19:15.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.111+0000 7f2cd7b2e800 20 add_watcher() i=7 2024-04-22T20:19:15.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.111+0000 7f2cd7b2e800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:15.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.112+0000 7f2b89fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:15.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.112+0000 7f2cd7b2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:15.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.112+0000 7f2cd7b2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:15.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.112+0000 7f2cd7b2e800 5 note: GC not initialized 2024-04-22T20:19:15.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2b88fe1640 20 reqs_thread_entry: start 2024-04-22T20:19:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2cd7b2e800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2cd7b2e800 20 Filter name: none 2024-04-22T20:19:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2b027fc640 20 reqs_thread_entry: start 2024-04-22T20:19:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2cd7b2e800 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-04-22T20:19:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2cd7b2e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2cd7b2e800 20 rados_obj.operate() r=0 bl.length=460 2024-04-22T20:19:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2cd7b2e800 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2024-04-22T20:19:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2cd7b2e800 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-22T20:19:15.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:15.113+0000 7f2cd7b2e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:16.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.799+0000 7f2cd7b2e800 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-04-22T20:19:16.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.799+0000 7f2cd7b2e800 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-22T20:19:16.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.799+0000 7f2cd7b2e800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-04-22T20:19:16.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.800+0000 7f2bb27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497215 cookie 94166774272944 notifier 7656 bl.length()=659 2024-04-22T20:19:16.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.800+0000 7f2bb27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-04-22T20:19:16.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.800+0000 7f2bb27fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-04-22T20:19:16.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.803+0000 7f2b89fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.805+0000 7f2cd7b2e800 20 remove_watcher() i=0 2024-04-22T20:19:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.805+0000 7f2cd7b2e800 2 removed watcher, disabling cache 2024-04-22T20:19:16.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.807+0000 7f2cd7b2e800 20 remove_watcher() i=1 2024-04-22T20:19:16.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.808+0000 7f2cd7b2e800 20 remove_watcher() i=2 2024-04-22T20:19:16.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.809+0000 7f2cd7b2e800 20 remove_watcher() i=3 2024-04-22T20:19:16.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.810+0000 7f2cd7b2e800 20 remove_watcher() i=4 2024-04-22T20:19:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.811+0000 7f2cd7b2e800 20 remove_watcher() i=5 2024-04-22T20:19:16.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.812+0000 7f2cd7b2e800 20 remove_watcher() i=6 2024-04-22T20:19:16.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.813+0000 7f2cd7b2e800 20 remove_watcher() i=7 2024-04-22T20:19:16.825 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2024-04-22T20:19:16.825 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name Mr.bar.client.0 --email bar+test@test.test --access-key GQWDPBJDURLWEJNDGGYO --secret BykXaAO3Z2IT/w02/imjwgC2394i1Gq42+0NHugHBOb8PPc2vLz9dA== --cluster ceph 2024-04-22T20:19:16.889 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:16.890 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:16.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.921+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.922+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.922+0000 7fa90d346800 20 realm 2024-04-22T20:19:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.922+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.922+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.922+0000 7fa90d346800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.922+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.922+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.922+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.923+0000 7fa90d346800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.923+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.924+0000 7fa90d346800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.924+0000 7fa90d346800 20 searching for the correct realm 2024-04-22T20:19:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:16.924+0000 7fa90ae4c640 0 -- 172.21.15.136:0/3863847356 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x5651b6167800 msgr2=0x5651b6169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:17.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.129+0000 7fa90ae4c640 0 -- 172.21.15.136:0/3863847356 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x5651b600e750 msgr2=0x7fa90408f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.338+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 20 zone default found 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 4 Realm: () 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 20 zonegroup default 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.339+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.340+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.340+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.340+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.340+0000 7fa90d346800 20 started sync module instance, tier type = 2024-04-22T20:19:17.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.340+0000 7fa90d346800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:17.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.350+0000 7fa90d346800 20 add_watcher() i=0 2024-04-22T20:19:17.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.350+0000 7fa90d346800 20 add_watcher() i=1 2024-04-22T20:19:17.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.350+0000 7fa90d346800 20 add_watcher() i=2 2024-04-22T20:19:17.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.350+0000 7fa90d346800 20 add_watcher() i=3 2024-04-22T20:19:17.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.350+0000 7fa90d346800 20 add_watcher() i=4 2024-04-22T20:19:17.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.350+0000 7fa90d346800 20 add_watcher() i=5 2024-04-22T20:19:17.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.350+0000 7fa90d346800 20 add_watcher() i=6 2024-04-22T20:19:17.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.351+0000 7fa90d346800 20 add_watcher() i=7 2024-04-22T20:19:17.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.351+0000 7fa90d346800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa7c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa90d346800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa90d346800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa90d346800 5 note: GC not initialized 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa73bfff640 20 reqs_thread_entry: start 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa90d346800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa90d346800 20 Filter name: none 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa739ffb640 20 reqs_thread_entry: start 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa90d346800 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-04-22T20:19:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.353+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2024-04-22T20:19:17.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 cache get: name=default.rgw.meta+users.email+bar+test@test.test : miss 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x0 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 adding default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 cache get: name=default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO : miss 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 cache put: name=default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO info.flags=0x0 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 adding default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO to cache LRU end 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 cache get: name=default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO : hit (negative entry) 2024-04-22T20:19:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.354+0000 7fa90d346800 10 cache get: name=default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO : hit (negative entry) 2024-04-22T20:19:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.356+0000 7fa90d346800 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-04-22T20:19:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.356+0000 7fa90d346800 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-22T20:19:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.356+0000 7fa90d346800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-04-22T20:19:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.356+0000 7fa7edffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497076 cookie 94908947283440 notifier 7241 bl.length()=645 2024-04-22T20:19:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.356+0000 7fa7edffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-04-22T20:19:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.356+0000 7fa7edffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-22T20:19:17.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.359+0000 7fa90d346800 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-04-22T20:19:17.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.359+0000 7fa90d346800 10 moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-04-22T20:19:17.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.359+0000 7fa90d346800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:bar+test@test.test, ofs0, ns] 2024-04-22T20:19:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.359+0000 7fa7edffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497099 cookie 94908947303216 notifier 7241 bl.length()=185 2024-04-22T20:19:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.359+0000 7fa7edffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-04-22T20:19:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.359+0000 7fa7edffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-04-22T20:19:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.362+0000 7fa90d346800 10 cache put: name=default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO info.flags=0x7 2024-04-22T20:19:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.362+0000 7fa90d346800 10 moving default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO to cache LRU end 2024-04-22T20:19:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.362+0000 7fa90d346800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:GQWDPBJDURLWEJNDGGYO, ofs0, ns] 2024-04-22T20:19:17.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.362+0000 7fa7edffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497044 cookie 94908947291072 notifier 7241 bl.length()=186 2024-04-22T20:19:17.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.362+0000 7fa7edffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO info.flags=0x7 2024-04-22T20:19:17.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.362+0000 7fa7edffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+GQWDPBJDURLWEJNDGGYO to cache LRU end 2024-04-22T20:19:17.575 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:17.575 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "bar.client.0", 2024-04-22T20:19:17.575 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr.bar.client.0", 2024-04-22T20:19:17.575 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "bar+test@test.test", 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "bar.client.0", 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "GQWDPBJDURLWEJNDGGYO", 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "BykXaAO3Z2IT/w02/imjwgC2394i1Gq42+0NHugHBOb8PPc2vLz9dA==", 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:17.356242Z" 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:19:17.576 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:17.577 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "rgw", 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "", 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:17.356228Z", 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:17.578 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:17.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.576+0000 7fa7c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:17.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.578+0000 7fa90d346800 20 remove_watcher() i=0 2024-04-22T20:19:17.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.578+0000 7fa90d346800 2 removed watcher, disabling cache 2024-04-22T20:19:17.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.580+0000 7fa90d346800 20 remove_watcher() i=1 2024-04-22T20:19:17.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.581+0000 7fa90d346800 20 remove_watcher() i=2 2024-04-22T20:19:17.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.582+0000 7fa90d346800 20 remove_watcher() i=3 2024-04-22T20:19:17.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.583+0000 7fa90d346800 20 remove_watcher() i=4 2024-04-22T20:19:17.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.584+0000 7fa90d346800 20 remove_watcher() i=5 2024-04-22T20:19:17.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.585+0000 7fa90d346800 20 remove_watcher() i=6 2024-04-22T20:19:17.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.586+0000 7fa90d346800 20 remove_watcher() i=7 2024-04-22T20:19:17.597 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 5953245061 --totp-seed DSQEYLUWVCKVLR6F654MMPCIKQNMURFY4LGVC2FRLNH4N7YCQDUY4HBRCERCFCLI --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-22T20:19:17.661 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:17.661 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:17.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.692+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.693+0000 7f4bcf30c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.693+0000 7f4bcf30c800 20 realm 2024-04-22T20:19:17.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.693+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.693+0000 7f4bcf30c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.693+0000 7f4bcf30c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:17.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.693+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.693+0000 7f4bcf30c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:17.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.693+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.694+0000 7f4bcf30c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.694+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.695+0000 7f4bcf30c800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.695+0000 7f4bcf30c800 20 searching for the correct realm 2024-04-22T20:19:17.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.697+0000 7f4bcd60c640 0 -- 172.21.15.136:0/2201479428 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f4bc406dc20 msgr2=0x7f4bc40580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:17.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:17.911+0000 7f4bcc60a640 0 -- 172.21.15.136:0/2201479428 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7f4bc4010730 msgr2=0x7f4bc40577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.121+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 20 zone default found 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 4 Realm: () 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 20 zonegroup default 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.122+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.123+0000 7f4bcf30c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.123+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.123+0000 7f4bcf30c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.123+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.123+0000 7f4bcf30c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.123+0000 7f4bcf30c800 20 started sync module instance, tier type = 2024-04-22T20:19:18.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.123+0000 7f4bcf30c800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:18.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.133+0000 7f4bcf30c800 20 add_watcher() i=0 2024-04-22T20:19:18.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.133+0000 7f4bcf30c800 20 add_watcher() i=1 2024-04-22T20:19:18.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.133+0000 7f4bcf30c800 20 add_watcher() i=2 2024-04-22T20:19:18.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.133+0000 7f4bcf30c800 20 add_watcher() i=3 2024-04-22T20:19:18.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.133+0000 7f4bcf30c800 20 add_watcher() i=4 2024-04-22T20:19:18.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.133+0000 7f4bcf30c800 20 add_watcher() i=5 2024-04-22T20:19:18.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.133+0000 7f4bcf30c800 20 add_watcher() i=6 2024-04-22T20:19:18.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.134+0000 7f4bcf30c800 20 add_watcher() i=7 2024-04-22T20:19:18.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.134+0000 7f4bcf30c800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:18.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.136+0000 7f4a82fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:18.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.136+0000 7f4bcf30c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:18.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.136+0000 7f4bcf30c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:18.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.137+0000 7f4bcf30c800 5 note: GC not initialized 2024-04-22T20:19:18.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.137+0000 7f4a81fe3640 20 reqs_thread_entry: start 2024-04-22T20:19:18.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.137+0000 7f4bcf30c800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:18.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.137+0000 7f4bcf30c800 20 Filter name: none 2024-04-22T20:19:18.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.137+0000 7f49f37fe640 20 reqs_thread_entry: start 2024-04-22T20:19:18.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.137+0000 7f4bcf30c800 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-04-22T20:19:18.138 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.137+0000 7f4bcf30c800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.138+0000 7f4bcf30c800 20 rados_obj.operate() r=0 bl.length=460 2024-04-22T20:19:18.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.138+0000 7f4bcf30c800 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2024-04-22T20:19:18.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.138+0000 7f4bcf30c800 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-22T20:19:18.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.138+0000 7f4bcf30c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:18.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.141+0000 7f4bcf30c800 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-04-22T20:19:18.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.141+0000 7f4bcf30c800 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-22T20:19:18.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.141+0000 7f4bcf30c800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-04-22T20:19:18.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.141+0000 7f4aa3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464373 cookie 93986808596352 notifier 7259 bl.length()=659 2024-04-22T20:19:18.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.141+0000 7f4aa3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-04-22T20:19:18.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.141+0000 7f4aa3fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-04-22T20:19:18.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.143+0000 7f4a82fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:18.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.146+0000 7f4bcf30c800 20 remove_watcher() i=0 2024-04-22T20:19:18.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.146+0000 7f4bcf30c800 2 removed watcher, disabling cache 2024-04-22T20:19:18.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.147+0000 7f4bcf30c800 20 remove_watcher() i=1 2024-04-22T20:19:18.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.148+0000 7f4bcf30c800 20 remove_watcher() i=2 2024-04-22T20:19:18.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.149+0000 7f4bcf30c800 20 remove_watcher() i=3 2024-04-22T20:19:18.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.150+0000 7f4bcf30c800 20 remove_watcher() i=4 2024-04-22T20:19:18.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.151+0000 7f4bcf30c800 20 remove_watcher() i=5 2024-04-22T20:19:18.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.152+0000 7f4bcf30c800 20 remove_watcher() i=6 2024-04-22T20:19:18.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.153+0000 7f4bcf30c800 20 remove_watcher() i=7 2024-04-22T20:19:18.164 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW33333333333333333 --account-name Mr.RGW33333333333333333 --email RGW33333333333333333+test@test.test --tenant testx 2024-04-22T20:19:18.223 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:18.223 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:18.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.456+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.458 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.458+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.458 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.458+0000 7fee4be19800 20 realm 2024-04-22T20:19:18.458 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.458+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.458+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.458+0000 7fee4be19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:18.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.458+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.458+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.458+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.459+0000 7fee4be19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:18.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.459+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.460+0000 7fee4be19800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:18.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.460+0000 7fee4be19800 20 searching for the correct realm 2024-04-22T20:19:18.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.461+0000 7fee49117640 0 -- 172.21.15.136:0/1371285204 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7fee3c060fa0 msgr2=0x7fee3c057710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:18.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.666+0000 7fee4a119640 0 -- 172.21.15.136:0/1371285204 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x5620d64be7d0 msgr2=0x5620d64c0be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:18.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:18.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:18.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:18.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:18.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:18.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:18.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:18.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.673+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:18.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 20 zone default found 2024-04-22T20:19:18.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 4 Realm: () 2024-04-22T20:19:18.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:18.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:18.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:18.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 20 zonegroup default 2024-04-22T20:19:18.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.674+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.675+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.675+0000 7fee4be19800 20 started sync module instance, tier type = 2024-04-22T20:19:18.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.675+0000 7fee4be19800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:18.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.684+0000 7fee4be19800 20 add_watcher() i=0 2024-04-22T20:19:18.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.684+0000 7fee4be19800 20 add_watcher() i=1 2024-04-22T20:19:18.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.684+0000 7fee4be19800 20 add_watcher() i=2 2024-04-22T20:19:18.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.684+0000 7fee4be19800 20 add_watcher() i=3 2024-04-22T20:19:18.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.684+0000 7fee4be19800 20 add_watcher() i=4 2024-04-22T20:19:18.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.684+0000 7fee4be19800 20 add_watcher() i=5 2024-04-22T20:19:18.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.684+0000 7fee4be19800 20 add_watcher() i=6 2024-04-22T20:19:18.685 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.685+0000 7fee4be19800 20 add_watcher() i=7 2024-04-22T20:19:18.685 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.685+0000 7fee4be19800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:18.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fecfb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fee4be19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fee4be19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fee4be19800 5 note: GC not initialized 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fecfa7fc640 20 reqs_thread_entry: start 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fee4be19800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fee4be19800 20 Filter name: none 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fec6bfff640 20 reqs_thread_entry: start 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fee4be19800 10 cache get: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 : miss 2024-04-22T20:19:18.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.687+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 10 cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x0 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 10 adding default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 20 failed to read name.testx$Mr.RGW33333333333333333 with: (2) No such file or directory 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 10 cache get: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test : miss 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 10 cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x0 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 10 adding default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-04-22T20:19:18.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.688+0000 7fee4be19800 20 failed to read rgw33333333333333333+test@test.test with: (2) No such file or directory 2024-04-22T20:19:18.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.690+0000 7fee4be19800 10 cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-04-22T20:19:18.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.690+0000 7fee4be19800 10 adding default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-04-22T20:19:18.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.690+0000 7fee4be19800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW33333333333333333, ofs0, ns] 2024-04-22T20:19:18.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.690+0000 7fed2d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464374 cookie 94699034274208 notifier 7713 bl.length()=389 2024-04-22T20:19:18.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.690+0000 7fed2d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-04-22T20:19:18.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.690+0000 7fed2d7fa640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-04-22T20:19:18.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.693+0000 7fee4be19800 10 cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x17 2024-04-22T20:19:18.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.693+0000 7fee4be19800 10 moving default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-04-22T20:19:18.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.693+0000 7fee4be19800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.testx$Mr.RGW33333333333333333, ofs0, ns] 2024-04-22T20:19:18.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.694+0000 7fed2d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464512 cookie 94699034281040 notifier 7713 bl.length()=230 2024-04-22T20:19:18.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.694+0000 7fed2d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x17 2024-04-22T20:19:18.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.694+0000 7fed2d7fa640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-04-22T20:19:18.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.697+0000 7fee4be19800 10 cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x17 2024-04-22T20:19:18.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.697+0000 7fee4be19800 10 moving default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-04-22T20:19:18.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.697+0000 7fee4be19800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw33333333333333333+test@test.test, ofs0, ns] 2024-04-22T20:19:18.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.697+0000 7fed2d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464337 cookie 94699034246720 notifier 7713 bl.length()=234 2024-04-22T20:19:18.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.697+0000 7fed2d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x17 2024-04-22T20:19:18.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.697+0000 7fed2d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-04-22T20:19:18.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.699+0000 7fecfb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:18.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.702+0000 7fee4be19800 20 remove_watcher() i=0 2024-04-22T20:19:18.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.702+0000 7fee4be19800 2 removed watcher, disabling cache 2024-04-22T20:19:18.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.703+0000 7fee4be19800 20 remove_watcher() i=1 2024-04-22T20:19:18.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.704+0000 7fee4be19800 20 remove_watcher() i=2 2024-04-22T20:19:18.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.705+0000 7fee4be19800 20 remove_watcher() i=3 2024-04-22T20:19:18.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.706+0000 7fee4be19800 20 remove_watcher() i=4 2024-04-22T20:19:18.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.707+0000 7fee4be19800 20 remove_watcher() i=5 2024-04-22T20:19:18.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.708+0000 7fee4be19800 20 remove_watcher() i=6 2024-04-22T20:19:18.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.709+0000 7fee4be19800 20 remove_watcher() i=7 2024-04-22T20:19:18.717 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:18.717 INFO:teuthology.orchestra.run.smithi136.stdout: "id": "RGW33333333333333333", 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "tenant": "testx", 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "name": "Mr.RGW33333333333333333", 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "RGW33333333333333333+test@test.test", 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "quota": { 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:18.718 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "max_users": 1000, 2024-04-22T20:19:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "max_roles": 1000, 2024-04-22T20:19:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "max_groups": 1000, 2024-04-22T20:19:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:18.719 INFO:teuthology.orchestra.run.smithi136.stdout: "max_access_keys": 4 2024-04-22T20:19:18.719 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:18.720 DEBUG:tasks.s3tests:Creating user RGW33333333333333333 on client.0 2024-04-22T20:19:18.721 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name Mr.RGW33333333333333333 --email 'testx$tenanteduser+test@test.test' --access-key ODVQGSVYVMVXQWDBEHSC --secret 4sKU7A51C5eZhqI0eof3OGKn3B4fVVR73jWAMTkIKuxFOJNx8zXIIw== --cluster ceph --account-id RGW33333333333333333 --account-root 2024-04-22T20:19:18.786 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:18.786 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:18.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.817+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.818+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.818+0000 7faf12211800 20 realm 2024-04-22T20:19:18.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.818+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.819+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.819+0000 7faf12211800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:18.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.819+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.819+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.819+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.820+0000 7faf12211800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:18.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.820+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.820+0000 7faf12211800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:18.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.820+0000 7faf12211800 20 searching for the correct realm 2024-04-22T20:19:18.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.822+0000 7faf0e148640 0 -- 172.21.15.136:0/2410736288 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x559901f0ebe0 msgr2=0x559901f10fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:18.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.826+0000 7faf0e148640 0 -- 172.21.15.136:0/2410736288 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7fadd807c6e0 msgr2=0x7fadd807eb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:18.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:18.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:18.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:18.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:18.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:18.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:18.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.831+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:18.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 20 zone default found 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 4 Realm: () 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 20 zonegroup default 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.832+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.833+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.833+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.833+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.833+0000 7faf12211800 20 started sync module instance, tier type = 2024-04-22T20:19:18.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.833+0000 7faf12211800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 20 add_watcher() i=0 2024-04-22T20:19:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 20 add_watcher() i=1 2024-04-22T20:19:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 20 add_watcher() i=2 2024-04-22T20:19:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 20 add_watcher() i=3 2024-04-22T20:19:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 20 add_watcher() i=4 2024-04-22T20:19:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 20 add_watcher() i=5 2024-04-22T20:19:18.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 20 add_watcher() i=6 2024-04-22T20:19:18.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 20 add_watcher() i=7 2024-04-22T20:19:18.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.842+0000 7faf12211800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.844+0000 7fadbf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.844+0000 7faf12211800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.844+0000 7faf12211800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:18.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.844+0000 7faf12211800 5 note: GC not initialized 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7fadbe7fc640 20 reqs_thread_entry: start 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 20 Filter name: none 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7fad2ffff640 20 reqs_thread_entry: start 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test : miss 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.845+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x0 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 adding default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-04-22T20:19:18.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 cache get: name=default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC : miss 2024-04-22T20:19:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 cache put: name=default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC info.flags=0x0 2024-04-22T20:19:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 adding default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC to cache LRU end 2024-04-22T20:19:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 cache get: name=default.rgw.meta+accounts+account.RGW33333333333333333 : miss 2024-04-22T20:19:18.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 20 rados_obj.operate() r=0 bl.length=189 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 adding default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 cache get: name=default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC : hit (negative entry) 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.846+0000 7faf12211800 10 cache get: name=default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC : hit (negative entry) 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.848+0000 7faf12211800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.848+0000 7faf12211800 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.848+0000 7faf12211800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.848+0000 7fadf17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464396 cookie 94115650972368 notifier 7728 bl.length()=732 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.848+0000 7fadf17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-04-22T20:19:18.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.848+0000 7fadf17fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-22T20:19:18.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.851+0000 7faf12211800 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-04-22T20:19:18.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.851+0000 7faf12211800 10 moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-04-22T20:19:18.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.851+0000 7faf12211800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.email:testx$tenanteduser+test@test.test, ofs0, ns] 2024-04-22T20:19:18.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.851+0000 7fadf17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464326 cookie 94115650977744 notifier 7728 bl.length()=215 2024-04-22T20:19:18.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.851+0000 7fadf17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-04-22T20:19:18.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.851+0000 7fadf17fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-04-22T20:19:18.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.854+0000 7faf12211800 10 cache put: name=default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC info.flags=0x7 2024-04-22T20:19:18.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.854+0000 7faf12211800 10 moving default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC to cache LRU end 2024-04-22T20:19:18.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.854+0000 7faf12211800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:ODVQGSVYVMVXQWDBEHSC, ofs0, ns] 2024-04-22T20:19:18.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.854+0000 7fadf17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464341 cookie 94115650962224 notifier 7728 bl.length()=201 2024-04-22T20:19:18.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.854+0000 7fadf17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC info.flags=0x7 2024-04-22T20:19:18.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.854+0000 7fadf17fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ODVQGSVYVMVXQWDBEHSC to cache LRU end 2024-04-22T20:19:18.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.857+0000 7faf12211800 20 linked user testx$tenanteduser.client.0 to account RGW33333333333333333 2024-04-22T20:19:18.858 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "testx$tenanteduser.client.0", 2024-04-22T20:19:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr.RGW33333333333333333", 2024-04-22T20:19:18.858 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "testx$tenanteduser+test@test.test", 2024-04-22T20:19:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:19:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:19:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:19:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "testx$tenanteduser.client.0", 2024-04-22T20:19:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "ODVQGSVYVMVXQWDBEHSC", 2024-04-22T20:19:18.859 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "4sKU7A51C5eZhqI0eof3OGKn3B4fVVR73jWAMTkIKuxFOJNx8zXIIw==", 2024-04-22T20:19:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:19:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:18.848328Z" 2024-04-22T20:19:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:19:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:19:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:19:18.860 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:19:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:19:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:19:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:19:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:18.861 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:18.862 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:19:18.863 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:18.864 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:19:18.865 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "root", 2024-04-22T20:19:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:19:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "RGW33333333333333333", 2024-04-22T20:19:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:19:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:18.848315Z", 2024-04-22T20:19:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:19:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:19:18.866 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:18.866 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:18.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.858+0000 7fadbf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:18.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.860+0000 7faf12211800 20 remove_watcher() i=0 2024-04-22T20:19:18.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.860+0000 7faf12211800 2 removed watcher, disabling cache 2024-04-22T20:19:18.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.861+0000 7faf12211800 20 remove_watcher() i=1 2024-04-22T20:19:18.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.862+0000 7faf12211800 20 remove_watcher() i=2 2024-04-22T20:19:18.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.864+0000 7faf12211800 20 remove_watcher() i=3 2024-04-22T20:19:18.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.865+0000 7faf12211800 20 remove_watcher() i=4 2024-04-22T20:19:18.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.866+0000 7faf12211800 20 remove_watcher() i=5 2024-04-22T20:19:18.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.867+0000 7faf12211800 20 remove_watcher() i=6 2024-04-22T20:19:18.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.867+0000 7faf12211800 20 remove_watcher() i=7 2024-04-22T20:19:18.879 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 6919758203 --totp-seed VRHJHJBMGHQAFQV4NCQF2GWPWX6CH7BLRYWN7I2ZB4RTHNWTB5XJPEQEQQX5WU7T --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-22T20:19:18.942 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:18.943 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:18.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.973+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.975+0000 7f0c3d462800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.975+0000 7f0c3d462800 20 realm 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.975+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.975+0000 7f0c3d462800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.975+0000 7f0c3d462800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.975+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.975+0000 7f0c3d462800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.975+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.976+0000 7f0c3d462800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:18.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.976+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:18.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.977+0000 7f0c3d462800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:18.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.977+0000 7f0c3d462800 20 searching for the correct realm 2024-04-22T20:19:18.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:18.977+0000 7f0c3af61640 0 -- 172.21.15.136:0/1213191381 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x55db75d16450 msgr2=0x55db75d18910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:19:19.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.182+0000 7f0c3af61640 0 -- 172.21.15.136:0/1213191381 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x55db75bbb800 msgr2=0x7f0c34095320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:19.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:19.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:19.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:19.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.392+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 zone default found 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 4 Realm: () 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:19.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:19.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 zonegroup default 2024-04-22T20:19:19.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.393+0000 7f0c3d462800 20 started sync module instance, tier type = 2024-04-22T20:19:19.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.394+0000 7f0c3d462800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:19.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.403+0000 7f0c3d462800 20 add_watcher() i=0 2024-04-22T20:19:19.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.403+0000 7f0c3d462800 20 add_watcher() i=1 2024-04-22T20:19:19.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.403+0000 7f0c3d462800 20 add_watcher() i=2 2024-04-22T20:19:19.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.403+0000 7f0c3d462800 20 add_watcher() i=3 2024-04-22T20:19:19.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.403+0000 7f0c3d462800 20 add_watcher() i=4 2024-04-22T20:19:19.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.403+0000 7f0c3d462800 20 add_watcher() i=5 2024-04-22T20:19:19.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.403+0000 7f0c3d462800 20 add_watcher() i=6 2024-04-22T20:19:19.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.404+0000 7f0c3d462800 20 add_watcher() i=7 2024-04-22T20:19:19.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.404+0000 7f0c3d462800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:19.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.406+0000 7f0af17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:19.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.406+0000 7f0c3d462800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:19.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.406+0000 7f0c3d462800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:19.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.406+0000 7f0c3d462800 5 note: GC not initialized 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.406+0000 7f0a6bfff640 20 reqs_thread_entry: start 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.406+0000 7f0c3d462800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.406+0000 7f0c3d462800 20 Filter name: none 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.406+0000 7f0a69ffb640 20 reqs_thread_entry: start 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.407+0000 7f0c3d462800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.407+0000 7f0c3d462800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.407+0000 7f0c3d462800 20 rados_obj.operate() r=0 bl.length=532 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.407+0000 7f0c3d462800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.407+0000 7f0c3d462800 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-22T20:19:19.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.407+0000 7f0c3d462800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:19.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.410+0000 7f0c3d462800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-04-22T20:19:19.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.410+0000 7f0c3d462800 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-22T20:19:19.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.410+0000 7f0c3d462800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-04-22T20:19:19.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.410+0000 7f0b1e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464397 cookie 94401062889952 notifier 7304 bl.length()=746 2024-04-22T20:19:19.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.410+0000 7f0b1e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-04-22T20:19:19.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.410+0000 7f0b1e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-04-22T20:19:19.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.412+0000 7f0af17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:19.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.415+0000 7f0c3d462800 20 remove_watcher() i=0 2024-04-22T20:19:19.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.415+0000 7f0c3d462800 2 removed watcher, disabling cache 2024-04-22T20:19:19.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.416+0000 7f0c3d462800 20 remove_watcher() i=1 2024-04-22T20:19:19.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.417+0000 7f0c3d462800 20 remove_watcher() i=2 2024-04-22T20:19:19.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.418+0000 7f0c3d462800 20 remove_watcher() i=3 2024-04-22T20:19:19.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.419+0000 7f0c3d462800 20 remove_watcher() i=4 2024-04-22T20:19:19.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.420+0000 7f0c3d462800 20 remove_watcher() i=5 2024-04-22T20:19:19.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.421+0000 7f0c3d462800 20 remove_watcher() i=6 2024-04-22T20:19:19.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.422+0000 7f0c3d462800 20 remove_watcher() i=7 2024-04-22T20:19:19.433 DEBUG:tasks.s3tests:Creating user foobar.client.0 on client.0 2024-04-22T20:19:19.433 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foobar.client.0 --display-name Mr.foobar.client.0 --email foobar+test@test.test --access-key BPUALREWTHMEQHIDNCWR --secret 6TIVwJmM+qrhapQTh0MrmsLSSGw6saQXeE4y+mW6bvHc2GHcLp9iRA== --cluster ceph 2024-04-22T20:19:19.495 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:19.495 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:19.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.526+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.527+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.527+0000 7ffb9bb2f800 20 realm 2024-04-22T20:19:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.527+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.527+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.527+0000 7ffb9bb2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.527+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.527+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.527+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.529 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.528+0000 7ffb9bb2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:19.529 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.528+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.529 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.529+0000 7ffb9bb2f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:19.529 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.529+0000 7ffb9bb2f800 20 searching for the correct realm 2024-04-22T20:19:19.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.531+0000 7ffb980c6640 0 -- 172.21.15.136:0/4071120564 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7ffb8803a070 msgr2=0x7ffb8802d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:19.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.737+0000 7ffb970c4640 0 -- 172.21.15.136:0/4071120564 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7ffb8801a040 msgr2=0x7ffb8803b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:19.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.943+0000 7ffb9bb2f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:19.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.943+0000 7ffb9bb2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:19.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.943+0000 7ffb9bb2f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:19.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.943+0000 7ffb9bb2f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:19.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.943+0000 7ffb9bb2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:19.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.943+0000 7ffb9bb2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:19.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.943+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.944+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.944+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.944+0000 7ffb9bb2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:19.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.944+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.944+0000 7ffb9bb2f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:19.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.944+0000 7ffb9bb2f800 20 zone default found 2024-04-22T20:19:19.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 4 Realm: () 2024-04-22T20:19:19.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:19.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 zonegroup default 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 started sync module instance, tier type = 2024-04-22T20:19:19.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.945+0000 7ffb9bb2f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:19.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.954+0000 7ffb9bb2f800 20 add_watcher() i=0 2024-04-22T20:19:19.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.954+0000 7ffb9bb2f800 20 add_watcher() i=1 2024-04-22T20:19:19.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.954+0000 7ffb9bb2f800 20 add_watcher() i=2 2024-04-22T20:19:19.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.954+0000 7ffb9bb2f800 20 add_watcher() i=3 2024-04-22T20:19:19.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.954+0000 7ffb9bb2f800 20 add_watcher() i=4 2024-04-22T20:19:19.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.954+0000 7ffb9bb2f800 20 add_watcher() i=5 2024-04-22T20:19:19.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.955+0000 7ffb9bb2f800 20 add_watcher() i=6 2024-04-22T20:19:19.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.955+0000 7ffb9bb2f800 20 add_watcher() i=7 2024-04-22T20:19:19.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.955+0000 7ffb9bb2f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.957+0000 7ffa4dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.957+0000 7ffb9bb2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.957+0000 7ffb9bb2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.957+0000 7ffb9bb2f800 5 note: GC not initialized 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffa4cfe1640 20 reqs_thread_entry: start 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 20 Filter name: none 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ff9c67fc640 20 reqs_thread_entry: start 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x0 2024-04-22T20:19:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 10 cache get: name=default.rgw.meta+users.email+foobar+test@test.test : miss 2024-04-22T20:19:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.958+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x0 2024-04-22T20:19:19.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 10 adding default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-04-22T20:19:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 10 cache get: name=default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR : miss 2024-04-22T20:19:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:19.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 10 cache put: name=default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR info.flags=0x0 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 10 adding default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR to cache LRU end 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 10 cache get: name=default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR : hit (negative entry) 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.959+0000 7ffb9bb2f800 10 cache get: name=default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR : hit (negative entry) 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.960+0000 7ffb9bb2f800 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.960+0000 7ffb9bb2f800 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.960+0000 7ffb9bb2f800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.960+0000 7ffa76ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464327 cookie 94721926101936 notifier 7764 bl.length()=660 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.960+0000 7ffa76ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-22T20:19:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.960+0000 7ffa76ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:19.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.964+0000 7ffb9bb2f800 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-04-22T20:19:19.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.964+0000 7ffb9bb2f800 10 moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-04-22T20:19:19.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.964+0000 7ffb9bb2f800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:foobar+test@test.test, ofs0, ns] 2024-04-22T20:19:19.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.964+0000 7ffa76ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464375 cookie 94721926076944 notifier 7764 bl.length()=191 2024-04-22T20:19:19.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.964+0000 7ffa76ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-04-22T20:19:19.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.964+0000 7ffa76ffd640 10 rgw watcher librados: moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-04-22T20:19:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.966+0000 7ffb9bb2f800 10 cache put: name=default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR info.flags=0x7 2024-04-22T20:19:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.967+0000 7ffb9bb2f800 10 moving default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR to cache LRU end 2024-04-22T20:19:19.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.967+0000 7ffb9bb2f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:BPUALREWTHMEQHIDNCWR, ofs0, ns] 2024-04-22T20:19:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.967+0000 7ffa76ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464398 cookie 94721926097984 notifier 7764 bl.length()=189 2024-04-22T20:19:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.967+0000 7ffa76ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR info.flags=0x7 2024-04-22T20:19:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.967+0000 7ffa76ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+BPUALREWTHMEQHIDNCWR to cache LRU end 2024-04-22T20:19:19.968 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "foobar.client.0", 2024-04-22T20:19:19.968 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr.foobar.client.0", 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "foobar+test@test.test", 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "foobar.client.0", 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "BPUALREWTHMEQHIDNCWR", 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "6TIVwJmM+qrhapQTh0MrmsLSSGw6saQXeE4y+mW6bvHc2GHcLp9iRA==", 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:19.960802Z" 2024-04-22T20:19:19.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:19.970 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:19.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "rgw", 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "", 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:19.960790Z", 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:19.972 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:19.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.969+0000 7ffa4dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:19.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.971+0000 7ffb9bb2f800 20 remove_watcher() i=0 2024-04-22T20:19:19.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.971+0000 7ffb9bb2f800 2 removed watcher, disabling cache 2024-04-22T20:19:19.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.973+0000 7ffb9bb2f800 20 remove_watcher() i=1 2024-04-22T20:19:19.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.974+0000 7ffb9bb2f800 20 remove_watcher() i=2 2024-04-22T20:19:19.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.975+0000 7ffb9bb2f800 20 remove_watcher() i=3 2024-04-22T20:19:19.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.976+0000 7ffb9bb2f800 20 remove_watcher() i=4 2024-04-22T20:19:19.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.977+0000 7ffb9bb2f800 20 remove_watcher() i=5 2024-04-22T20:19:19.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.978+0000 7ffb9bb2f800 20 remove_watcher() i=6 2024-04-22T20:19:19.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:19.979+0000 7ffb9bb2f800 20 remove_watcher() i=7 2024-04-22T20:19:19.990 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foobar.client.0 --totp-serial 7130237289 --totp-seed 6HU7TUQSCXSNQKWVM2K7KQ7XRA3ZOCXO2TB5FQQP7QYMKZTBFHAEJMOMIWCCLZ5B --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-22T20:19:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:20.053 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:20.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.084+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.085+0000 7f9a09134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.085+0000 7f9a09134800 20 realm 2024-04-22T20:19:20.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.085+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.085+0000 7f9a09134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.085+0000 7f9a09134800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:20.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.085+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.085+0000 7f9a09134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.085+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.086+0000 7f9a09134800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:20.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.086+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.087+0000 7f9a09134800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:20.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.087+0000 7f9a09134800 20 searching for the correct realm 2024-04-22T20:19:20.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.089+0000 7f9a056cd640 0 -- 172.21.15.136:0/2945975765 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x7f98d007c900 msgr2=0x7f98d007ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:20.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.293+0000 7f9a04ecc640 0 -- 172.21.15.136:0/2945975765 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x559df6e476f0 msgr2=0x7f98d0081a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.502+0000 7f9a09134800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.502+0000 7f9a09134800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.502+0000 7f9a09134800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.502+0000 7f9a09134800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.502+0000 7f9a09134800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.502+0000 7f9a09134800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.502+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.504 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 20 zone default found 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 4 Realm: () 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 20 zonegroup default 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.503+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.504+0000 7f9a09134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.504+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.504+0000 7f9a09134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.504+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.504+0000 7f9a09134800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.504+0000 7f9a09134800 20 started sync module instance, tier type = 2024-04-22T20:19:20.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.504+0000 7f9a09134800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:20.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.514+0000 7f9a09134800 20 add_watcher() i=0 2024-04-22T20:19:20.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.514+0000 7f9a09134800 20 add_watcher() i=1 2024-04-22T20:19:20.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.514+0000 7f9a09134800 20 add_watcher() i=2 2024-04-22T20:19:20.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.514+0000 7f9a09134800 20 add_watcher() i=3 2024-04-22T20:19:20.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.514+0000 7f9a09134800 20 add_watcher() i=4 2024-04-22T20:19:20.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.514+0000 7f9a09134800 20 add_watcher() i=5 2024-04-22T20:19:20.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.514+0000 7f9a09134800 20 add_watcher() i=6 2024-04-22T20:19:20.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.515+0000 7f9a09134800 20 add_watcher() i=7 2024-04-22T20:19:20.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.515+0000 7f9a09134800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.517+0000 7f98bafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.517+0000 7f9a09134800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.517+0000 7f9a09134800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.517+0000 7f9a09134800 5 note: GC not initialized 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.518+0000 7f98b9fe3640 20 reqs_thread_entry: start 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.518+0000 7f9a09134800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.518+0000 7f9a09134800 20 Filter name: none 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.518+0000 7f982b7fe640 20 reqs_thread_entry: start 2024-04-22T20:19:20.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.518+0000 7f9a09134800 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-04-22T20:19:20.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.518+0000 7f9a09134800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.519+0000 7f9a09134800 20 rados_obj.operate() r=0 bl.length=472 2024-04-22T20:19:20.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.519+0000 7f9a09134800 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-04-22T20:19:20.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.519+0000 7f9a09134800 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:20.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.519+0000 7f9a09134800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:20.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.521+0000 7f9a09134800 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-22T20:19:20.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.521+0000 7f9a09134800 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:20.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.521+0000 7f9a09134800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-04-22T20:19:20.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.522+0000 7f98dbfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464328 cookie 94136940442480 notifier 7337 bl.length()=674 2024-04-22T20:19:20.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.522+0000 7f98dbfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-22T20:19:20.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.522+0000 7f98dbfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:20.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.524+0000 7f98bafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:20.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.526+0000 7f9a09134800 20 remove_watcher() i=0 2024-04-22T20:19:20.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.526+0000 7f9a09134800 2 removed watcher, disabling cache 2024-04-22T20:19:20.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.528+0000 7f9a09134800 20 remove_watcher() i=1 2024-04-22T20:19:20.529 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.528+0000 7f9a09134800 20 remove_watcher() i=2 2024-04-22T20:19:20.530 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.530+0000 7f9a09134800 20 remove_watcher() i=3 2024-04-22T20:19:20.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.531+0000 7f9a09134800 20 remove_watcher() i=4 2024-04-22T20:19:20.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.532+0000 7f9a09134800 20 remove_watcher() i=5 2024-04-22T20:19:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.533+0000 7f9a09134800 20 remove_watcher() i=6 2024-04-22T20:19:20.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.533+0000 7f9a09134800 20 remove_watcher() i=7 2024-04-22T20:19:20.544 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'oidc-provider=*;roles=*;user-policy=*' --cluster ceph 2024-04-22T20:19:20.599 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:20.600 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:20.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.630+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.631+0000 7ff179c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.631+0000 7ff179c0e800 20 realm 2024-04-22T20:19:20.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.631+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.631+0000 7ff179c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.631+0000 7ff179c0e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:20.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.631+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.631+0000 7ff179c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:20.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.631+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.632+0000 7ff179c0e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:20.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.632+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:20.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.633+0000 7ff179c0e800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:20.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.633+0000 7ff179c0e800 20 searching for the correct realm 2024-04-22T20:19:20.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.635+0000 7ff177f0e640 0 -- 172.21.15.136:0/2039182897 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7ff170017070 msgr2=0x7ff17005f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:20.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:20.840+0000 7ff176f0c640 0 -- 172.21.15.136:0/2039182897 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7ff17005e330 msgr2=0x7ff170058ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:21.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:21.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:21.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:21.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:21.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:21.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:21.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.047+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:21.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:21.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 20 zone default found 2024-04-22T20:19:21.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 4 Realm: () 2024-04-22T20:19:21.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:21.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:21.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:21.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 20 zonegroup default 2024-04-22T20:19:21.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.048+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.049+0000 7ff179c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.049+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.049+0000 7ff179c0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.049+0000 7ff179c0e800 20 started sync module instance, tier type = 2024-04-22T20:19:21.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.049+0000 7ff179c0e800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:21.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.059+0000 7ff179c0e800 20 add_watcher() i=0 2024-04-22T20:19:21.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.059+0000 7ff179c0e800 20 add_watcher() i=1 2024-04-22T20:19:21.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.059+0000 7ff179c0e800 20 add_watcher() i=2 2024-04-22T20:19:21.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.059+0000 7ff179c0e800 20 add_watcher() i=3 2024-04-22T20:19:21.059 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.059+0000 7ff179c0e800 20 add_watcher() i=4 2024-04-22T20:19:21.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.059+0000 7ff179c0e800 20 add_watcher() i=5 2024-04-22T20:19:21.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.059+0000 7ff179c0e800 20 add_watcher() i=6 2024-04-22T20:19:21.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.060+0000 7ff179c0e800 20 add_watcher() i=7 2024-04-22T20:19:21.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.060+0000 7ff179c0e800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff02dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff179c0e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff179c0e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff179c0e800 5 note: GC not initialized 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff02cfe1640 20 reqs_thread_entry: start 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff179c0e800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff179c0e800 20 Filter name: none 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7fefa67fc640 20 reqs_thread_entry: start 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff179c0e800 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-04-22T20:19:21.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.062+0000 7ff179c0e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.063+0000 7ff179c0e800 20 rados_obj.operate() r=0 bl.length=486 2024-04-22T20:19:21.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.063+0000 7ff179c0e800 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-04-22T20:19:21.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.063+0000 7ff179c0e800 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:21.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.063+0000 7ff179c0e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:21.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.064+0000 7ff179c0e800 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-22T20:19:21.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.064+0000 7ff179c0e800 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:21.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.064+0000 7ff179c0e800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-04-22T20:19:21.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.065+0000 7ff05affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464329 cookie 94824545400864 notifier 7355 bl.length()=727 2024-04-22T20:19:21.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.065+0000 7ff05affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-04-22T20:19:21.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.065+0000 7ff05affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-04-22T20:19:21.067 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:21.067 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "foobar.client.0", 2024-04-22T20:19:21.067 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr.foobar.client.0", 2024-04-22T20:19:21.067 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "foobar+test@test.test", 2024-04-22T20:19:21.067 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:19:21.067 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:21.067 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:19:21.067 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "foobar.client.0", 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "BPUALREWTHMEQHIDNCWR", 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "6TIVwJmM+qrhapQTh0MrmsLSSGw6saQXeE4y+mW6bvHc2GHcLp9iRA==", 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:19.960802Z" 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [ 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "oidc-provider", 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: "perm": "*" 2024-04-22T20:19:21.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:21.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:21.069 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "roles", 2024-04-22T20:19:21.069 INFO:teuthology.orchestra.run.smithi136.stdout: "perm": "*" 2024-04-22T20:19:21.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:21.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:21.069 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "user-policy", 2024-04-22T20:19:21.069 INFO:teuthology.orchestra.run.smithi136.stdout: "perm": "*" 2024-04-22T20:19:21.069 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:21.070 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "rgw", 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [ 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "7130237289" 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "", 2024-04-22T20:19:21.071 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:19:21.072 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:19.960790Z", 2024-04-22T20:19:21.074 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:19:21.074 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:19:21.074 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:21.074 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.067+0000 7ff02dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.070+0000 7ff179c0e800 20 remove_watcher() i=0 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.070+0000 7ff179c0e800 2 removed watcher, disabling cache 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.071+0000 7ff179c0e800 20 remove_watcher() i=1 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.072+0000 7ff179c0e800 20 remove_watcher() i=2 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.073+0000 7ff179c0e800 20 remove_watcher() i=3 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.074+0000 7ff179c0e800 20 remove_watcher() i=4 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.075+0000 7ff179c0e800 20 remove_watcher() i=5 2024-04-22T20:19:21.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.076+0000 7ff179c0e800 20 remove_watcher() i=6 2024-04-22T20:19:21.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.077+0000 7ff179c0e800 20 remove_watcher() i=7 2024-04-22T20:19:21.091 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW88888888888888888 --account-name Mr.RGW88888888888888888 --email RGW88888888888888888+test@test.test 2024-04-22T20:19:21.150 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:21.151 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:21.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.180+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.181+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.181+0000 7f3a72575800 20 realm 2024-04-22T20:19:21.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.181+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.182+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.182+0000 7f3a72575800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:21.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.182+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.182+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.182+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.183+0000 7f3a72575800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:21.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.183+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.183+0000 7f3a72575800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:21.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.183+0000 7f3a72575800 20 searching for the correct realm 2024-04-22T20:19:21.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.184+0000 7f3a70074640 0 -- 172.21.15.136:0/206591419 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x56308bec7020 msgr2=0x56308bec94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:19:21.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.389+0000 7f3a70074640 0 -- 172.21.15.136:0/206591419 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x56308bd6a730 msgr2=0x7f3a68089610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:21.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:21.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:21.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:21.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:21.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:21.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:21.599 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.598+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:21.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:21.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 20 zone default found 2024-04-22T20:19:21.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 4 Realm: () 2024-04-22T20:19:21.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:21.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:21.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:21.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 20 zonegroup default 2024-04-22T20:19:21.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.599+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.600+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.600+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.600+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.600+0000 7f3a72575800 20 started sync module instance, tier type = 2024-04-22T20:19:21.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.600+0000 7f3a72575800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:21.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.609+0000 7f3a72575800 20 add_watcher() i=0 2024-04-22T20:19:21.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.609+0000 7f3a72575800 20 add_watcher() i=1 2024-04-22T20:19:21.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.609+0000 7f3a72575800 20 add_watcher() i=2 2024-04-22T20:19:21.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.609+0000 7f3a72575800 20 add_watcher() i=3 2024-04-22T20:19:21.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.609+0000 7f3a72575800 20 add_watcher() i=4 2024-04-22T20:19:21.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.609+0000 7f3a72575800 20 add_watcher() i=5 2024-04-22T20:19:21.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.609+0000 7f3a72575800 20 add_watcher() i=6 2024-04-22T20:19:21.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.610+0000 7f3a72575800 20 add_watcher() i=7 2024-04-22T20:19:21.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.610+0000 7f3a72575800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:21.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.612+0000 7f3926fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:21.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.612+0000 7f3a72575800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:21.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.612+0000 7f3a72575800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:21.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.612+0000 7f3a72575800 5 note: GC not initialized 2024-04-22T20:19:21.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3925fdb640 20 reqs_thread_entry: start 2024-04-22T20:19:21.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:21.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 20 Filter name: none 2024-04-22T20:19:21.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f38977fe640 20 reqs_thread_entry: start 2024-04-22T20:19:21.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 : miss 2024-04-22T20:19:21.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x0 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 10 adding default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 20 failed to read name.$Mr.RGW88888888888888888 with: (2) No such file or directory 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 10 cache get: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test : miss 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x0 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 10 adding default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-04-22T20:19:21.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.613+0000 7f3a72575800 20 failed to read rgw88888888888888888+test@test.test with: (2) No such file or directory 2024-04-22T20:19:21.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.615+0000 7f3a72575800 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-04-22T20:19:21.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.615+0000 7f3a72575800 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-04-22T20:19:21.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.615+0000 7f3a72575800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW88888888888888888, ofs0, ns] 2024-04-22T20:19:21.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.616+0000 7f39477fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464376 cookie 94766505963872 notifier 7370 bl.length()=384 2024-04-22T20:19:21.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.616+0000 7f39477fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-04-22T20:19:21.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.616+0000 7f39477fe640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-04-22T20:19:21.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.619+0000 7f3a72575800 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-04-22T20:19:21.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.619+0000 7f3a72575800 10 moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-04-22T20:19:21.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.619+0000 7f3a72575800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW88888888888888888, ofs0, ns] 2024-04-22T20:19:21.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.619+0000 7f39477fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464513 cookie 94766505978832 notifier 7370 bl.length()=225 2024-04-22T20:19:21.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.619+0000 7f39477fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-04-22T20:19:21.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.619+0000 7f39477fe640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-04-22T20:19:21.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.622+0000 7f3a72575800 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-04-22T20:19:21.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.622+0000 7f3a72575800 10 moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-04-22T20:19:21.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.622+0000 7f3a72575800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw88888888888888888+test@test.test, ofs0, ns] 2024-04-22T20:19:21.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.622+0000 7f39477fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464338 cookie 94766505976624 notifier 7370 bl.length()=234 2024-04-22T20:19:21.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.622+0000 7f39477fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-04-22T20:19:21.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.622+0000 7f39477fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-04-22T20:19:21.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.624+0000 7f3926fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:21.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.627+0000 7f3a72575800 20 remove_watcher() i=0 2024-04-22T20:19:21.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.627+0000 7f3a72575800 2 removed watcher, disabling cache 2024-04-22T20:19:21.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.628+0000 7f3a72575800 20 remove_watcher() i=1 2024-04-22T20:19:21.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.629+0000 7f3a72575800 20 remove_watcher() i=2 2024-04-22T20:19:21.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.630+0000 7f3a72575800 20 remove_watcher() i=3 2024-04-22T20:19:21.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.631+0000 7f3a72575800 20 remove_watcher() i=4 2024-04-22T20:19:21.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.632+0000 7f3a72575800 20 remove_watcher() i=5 2024-04-22T20:19:21.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.633+0000 7f3a72575800 20 remove_watcher() i=6 2024-04-22T20:19:21.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.634+0000 7f3a72575800 20 remove_watcher() i=7 2024-04-22T20:19:21.642 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "id": "RGW88888888888888888", 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "tenant": "", 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "name": "Mr.RGW88888888888888888", 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "RGW88888888888888888+test@test.test", 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "quota": { 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:21.643 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: "max_users": 1000, 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: "max_roles": 1000, 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: "max_groups": 1000, 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout: "max_access_keys": 4 2024-04-22T20:19:21.644 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:21.645 DEBUG:tasks.s3tests:Creating user RGW88888888888888888 on client.0 2024-04-22T20:19:21.645 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root1.client.0 --display-name Mr.RGW88888888888888888 --email root1+test@test.test --access-key TEPUMXQVMVAAGPIGRJXR --secret s+LATIWigBIEMfRGkxzAUyasEl3jjLXeR1rFwmJhtNRepEtJw5f2EA== --cluster ceph --account-id RGW88888888888888888 --account-root 2024-04-22T20:19:21.710 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:21.710 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:21.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.740+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.741+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.741+0000 7fe4ecc10800 20 realm 2024-04-22T20:19:21.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.741+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.741+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.741+0000 7fe4ecc10800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:21.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.741+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.741+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.741+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.742+0000 7fe4ecc10800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:21.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.742+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.743+0000 7fe4ecc10800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:21.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.743+0000 7fe4ecc10800 20 searching for the correct realm 2024-04-22T20:19:21.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.744+0000 7fe4e7afc640 0 -- 172.21.15.136:0/2768363147 >> [v2:172.21.15.164:6800/4018161608,v1:172.21.15.164:6801/4018161608] conn(0x7fe4e0018070 msgr2=0x7fe4e0081600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:19:21.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.950+0000 7fe4e8afe640 0 -- 172.21.15.136:0/2768363147 >> [v2:172.21.15.136:6808/253893352,v1:172.21.15.136:6809/253893352] conn(0x5557ae40fde0 msgr2=0x5557ae4121f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:21.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.956+0000 7fe4ecc10800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:21.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.956+0000 7fe4ecc10800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:21.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.956+0000 7fe4ecc10800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.956+0000 7fe4ecc10800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.956+0000 7fe4ecc10800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.956+0000 7fe4ecc10800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.956+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:21.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:21.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 20 zone default found 2024-04-22T20:19:21.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 4 Realm: () 2024-04-22T20:19:21.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 20 zonegroup default 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.957+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.958+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.958+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.958+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.958+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.958+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.958+0000 7fe4ecc10800 20 started sync module instance, tier type = 2024-04-22T20:19:21.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.958+0000 7fe4ecc10800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:21.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 20 add_watcher() i=0 2024-04-22T20:19:21.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 20 add_watcher() i=1 2024-04-22T20:19:21.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 20 add_watcher() i=2 2024-04-22T20:19:21.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 20 add_watcher() i=3 2024-04-22T20:19:21.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 20 add_watcher() i=4 2024-04-22T20:19:21.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 20 add_watcher() i=5 2024-04-22T20:19:21.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 20 add_watcher() i=6 2024-04-22T20:19:21.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 20 add_watcher() i=7 2024-04-22T20:19:21.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.968+0000 7fe4ecc10800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:21.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.970+0000 7fe39efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 5 note: GC not initialized 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe39dfdb640 20 reqs_thread_entry: start 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 20 Filter name: none 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe30f7fe640 20 reqs_thread_entry: start 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x0 2024-04-22T20:19:21.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 10 cache get: name=default.rgw.meta+users.email+root1+test@test.test : miss 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.971+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x0 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 10 adding default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 10 cache get: name=default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR : miss 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 10 cache put: name=default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR info.flags=0x0 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 10 adding default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR to cache LRU end 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 10 cache get: name=default.rgw.meta+accounts+account.RGW88888888888888888 : miss 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.972+0000 7fe4ecc10800 20 rados->read ofs=0 len=0 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.973+0000 7fe4ecc10800 20 rados_obj.operate() r=0 bl.length=184 2024-04-22T20:19:21.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.973+0000 7fe4ecc10800 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-04-22T20:19:21.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.973+0000 7fe4ecc10800 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-04-22T20:19:21.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.973+0000 7fe4ecc10800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:21.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.973+0000 7fe4ecc10800 10 cache get: name=default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR : hit (negative entry) 2024-04-22T20:19:21.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.973+0000 7fe4ecc10800 10 cache get: name=default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR : hit (negative entry) 2024-04-22T20:19:21.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.974+0000 7fe4ecc10800 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-04-22T20:19:21.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.974+0000 7fe4ecc10800 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-22T20:19:21.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.974+0000 7fe4ecc10800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-04-22T20:19:21.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.975+0000 7fe3bf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464377 cookie 93835074018896 notifier 7388 bl.length()=681 2024-04-22T20:19:21.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.975+0000 7fe3bf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-04-22T20:19:21.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.975+0000 7fe3bf7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-22T20:19:21.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.978+0000 7fe4ecc10800 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-04-22T20:19:21.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.978+0000 7fe4ecc10800 10 moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-04-22T20:19:21.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.978+0000 7fe4ecc10800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root1+test@test.test, ofs0, ns] 2024-04-22T20:19:21.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.978+0000 7fe3bf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464399 cookie 93835074046816 notifier 7388 bl.length()=189 2024-04-22T20:19:21.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.978+0000 7fe3bf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-04-22T20:19:21.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.978+0000 7fe3bf7fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-04-22T20:19:21.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.981+0000 7fe4ecc10800 10 cache put: name=default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR info.flags=0x7 2024-04-22T20:19:21.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.981+0000 7fe4ecc10800 10 moving default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR to cache LRU end 2024-04-22T20:19:21.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.981+0000 7fe4ecc10800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:TEPUMXQVMVAAGPIGRJXR, ofs0, ns] 2024-04-22T20:19:21.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.981+0000 7fe3beffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464514 cookie 93835074042944 notifier 7388 bl.length()=188 2024-04-22T20:19:21.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.981+0000 7fe3beffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR info.flags=0x7 2024-04-22T20:19:21.982 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.981+0000 7fe3beffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+TEPUMXQVMVAAGPIGRJXR to cache LRU end 2024-04-22T20:19:21.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.984+0000 7fe4ecc10800 20 linked user root1.client.0 to account RGW88888888888888888 2024-04-22T20:19:21.984 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:21.984 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "root1.client.0", 2024-04-22T20:19:21.984 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr.RGW88888888888888888", 2024-04-22T20:19:21.984 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "root1+test@test.test", 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "root1.client.0", 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "TEPUMXQVMVAAGPIGRJXR", 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "s+LATIWigBIEMfRGkxzAUyasEl3jjLXeR1rFwmJhtNRepEtJw5f2EA==", 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:21.974575Z" 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:19:21.985 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:21.986 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "root", 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "RGW88888888888888888", 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:21.974562Z", 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:19:21.987 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:19:21.988 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:21.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:21.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.984+0000 7fe39efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:21.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.987+0000 7fe4ecc10800 20 remove_watcher() i=0 2024-04-22T20:19:21.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.987+0000 7fe4ecc10800 2 removed watcher, disabling cache 2024-04-22T20:19:21.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.988+0000 7fe4ecc10800 20 remove_watcher() i=1 2024-04-22T20:19:21.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.989+0000 7fe4ecc10800 20 remove_watcher() i=2 2024-04-22T20:19:21.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.990+0000 7fe4ecc10800 20 remove_watcher() i=3 2024-04-22T20:19:21.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.991+0000 7fe4ecc10800 20 remove_watcher() i=4 2024-04-22T20:19:21.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.992+0000 7fe4ecc10800 20 remove_watcher() i=5 2024-04-22T20:19:21.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.993+0000 7fe4ecc10800 20 remove_watcher() i=6 2024-04-22T20:19:21.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:21.994+0000 7fe4ecc10800 20 remove_watcher() i=7 2024-04-22T20:19:22.004 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root1.client.0 --totp-serial 1681277293 --totp-seed ICAAOQU6ASX6RCPP36KMVVO3EWVR3IGNYIKT3V5U32RIWNRVJKG2344JZIPJHAYX --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-22T20:19:22.068 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:22.068 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:22.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.097+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.099+0000 7fc873d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.099+0000 7fc873d42800 20 realm 2024-04-22T20:19:22.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.099+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.099+0000 7fc873d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.099+0000 7fc873d42800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:22.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.099+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.099+0000 7fc873d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.099+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.100+0000 7fc873d42800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:22.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.100+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.101+0000 7fc873d42800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:22.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.101+0000 7fc873d42800 20 searching for the correct realm 2024-04-22T20:19:22.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.103+0000 7fc872042640 0 -- 172.21.15.136:0/3548481866 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fc86c028ba0 msgr2=0x7fc86c033410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:22.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.308+0000 7fc869040640 0 -- 172.21.15.136:0/3548481866 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fc86c00f3d0 msgr2=0x7fc86c00f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:22.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:22.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:22.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:22.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:22.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:22.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:22.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.515+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 20 zone default found 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 4 Realm: () 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:22.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 20 zonegroup default 2024-04-22T20:19:22.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.516+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.517+0000 7fc873d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.517+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.517+0000 7fc873d42800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.517+0000 7fc873d42800 20 started sync module instance, tier type = 2024-04-22T20:19:22.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.517+0000 7fc873d42800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:22.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.527+0000 7fc873d42800 20 add_watcher() i=0 2024-04-22T20:19:22.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.527+0000 7fc873d42800 20 add_watcher() i=1 2024-04-22T20:19:22.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.527+0000 7fc873d42800 20 add_watcher() i=2 2024-04-22T20:19:22.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.527+0000 7fc873d42800 20 add_watcher() i=3 2024-04-22T20:19:22.527 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.527+0000 7fc873d42800 20 add_watcher() i=4 2024-04-22T20:19:22.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.527+0000 7fc873d42800 20 add_watcher() i=5 2024-04-22T20:19:22.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.527+0000 7fc873d42800 20 add_watcher() i=6 2024-04-22T20:19:22.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.528+0000 7fc873d42800 20 add_watcher() i=7 2024-04-22T20:19:22.528 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.528+0000 7fc873d42800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:22.530 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.529+0000 7fc7237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:22.530 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.529+0000 7fc873d42800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:22.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.529+0000 7fc873d42800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:22.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.530+0000 7fc873d42800 5 note: GC not initialized 2024-04-22T20:19:22.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.530+0000 7fc7227fc640 20 reqs_thread_entry: start 2024-04-22T20:19:22.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.530+0000 7fc873d42800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:22.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.530+0000 7fc873d42800 20 Filter name: none 2024-04-22T20:19:22.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.530+0000 7fc693fff640 20 reqs_thread_entry: start 2024-04-22T20:19:22.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.530+0000 7fc873d42800 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-04-22T20:19:22.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.530+0000 7fc873d42800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.531+0000 7fc873d42800 20 rados_obj.operate() r=0 bl.length=494 2024-04-22T20:19:22.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.531+0000 7fc873d42800 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x13 2024-04-22T20:19:22.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.531+0000 7fc873d42800 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-22T20:19:22.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.531+0000 7fc873d42800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:22.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.533+0000 7fc873d42800 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-04-22T20:19:22.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.533+0000 7fc873d42800 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-22T20:19:22.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.533+0000 7fc873d42800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-04-22T20:19:22.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.533+0000 7fc7557fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464378 cookie 94134509845584 notifier 7851 bl.length()=695 2024-04-22T20:19:22.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.533+0000 7fc7557fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-04-22T20:19:22.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.533+0000 7fc7557fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-04-22T20:19:22.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.536+0000 7fc7237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:22.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.538+0000 7fc873d42800 20 remove_watcher() i=0 2024-04-22T20:19:22.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.538+0000 7fc873d42800 2 removed watcher, disabling cache 2024-04-22T20:19:22.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.539+0000 7fc873d42800 20 remove_watcher() i=1 2024-04-22T20:19:22.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.540+0000 7fc873d42800 20 remove_watcher() i=2 2024-04-22T20:19:22.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.541+0000 7fc873d42800 20 remove_watcher() i=3 2024-04-22T20:19:22.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.543+0000 7fc873d42800 20 remove_watcher() i=4 2024-04-22T20:19:22.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.544+0000 7fc873d42800 20 remove_watcher() i=5 2024-04-22T20:19:22.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.544+0000 7fc873d42800 20 remove_watcher() i=6 2024-04-22T20:19:22.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.545+0000 7fc873d42800 20 remove_watcher() i=7 2024-04-22T20:19:22.557 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW99999999999999999 --account-name Mr.RGW99999999999999999 --email RGW99999999999999999+test@test.test 2024-04-22T20:19:22.615 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:22.615 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:22.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.645+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.646+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.646+0000 7efddc12e800 20 realm 2024-04-22T20:19:22.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.646+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.647+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.647+0000 7efddc12e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:22.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.647+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.647+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:22.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.647+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.648+0000 7efddc12e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:22.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.648+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:22.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.649+0000 7efddc12e800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:22.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.649+0000 7efddc12e800 20 searching for the correct realm 2024-04-22T20:19:22.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.651+0000 7efdd868c640 0 -- 172.21.15.136:0/1190528729 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7efdc8072040 msgr2=0x7efdc8093470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:19:22.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:22.857+0000 7efdd768a640 0 -- 172.21.15.136:0/1190528729 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7efdc807e410 msgr2=0x7efdc807e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:23.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:23.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:23.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:23.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:23.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:23.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:23.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.063+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 20 zone default found 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 4 Realm: () 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 20 zonegroup default 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.064+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.065+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.065+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.065+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.065+0000 7efddc12e800 20 started sync module instance, tier type = 2024-04-22T20:19:23.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.065+0000 7efddc12e800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:23.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.075+0000 7efddc12e800 20 add_watcher() i=0 2024-04-22T20:19:23.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.075+0000 7efddc12e800 20 add_watcher() i=1 2024-04-22T20:19:23.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.075+0000 7efddc12e800 20 add_watcher() i=2 2024-04-22T20:19:23.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.075+0000 7efddc12e800 20 add_watcher() i=3 2024-04-22T20:19:23.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.075+0000 7efddc12e800 20 add_watcher() i=4 2024-04-22T20:19:23.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.075+0000 7efddc12e800 20 add_watcher() i=5 2024-04-22T20:19:23.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.075+0000 7efddc12e800 20 add_watcher() i=6 2024-04-22T20:19:23.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.076+0000 7efddc12e800 20 add_watcher() i=7 2024-04-22T20:19:23.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.076+0000 7efddc12e800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:23.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.077+0000 7efc8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:23.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 5 note: GC not initialized 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efc8cfe1640 20 reqs_thread_entry: start 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 20 Filter name: none 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efc067fc640 20 reqs_thread_entry: start 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 : miss 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x0 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 10 adding default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.078+0000 7efddc12e800 20 failed to read name.$Mr.RGW99999999999999999 with: (2) No such file or directory 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.079+0000 7efddc12e800 10 cache get: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test : miss 2024-04-22T20:19:23.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.079+0000 7efddc12e800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.079+0000 7efddc12e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.079+0000 7efddc12e800 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x0 2024-04-22T20:19:23.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.079+0000 7efddc12e800 10 adding default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-04-22T20:19:23.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.079+0000 7efddc12e800 20 failed to read rgw99999999999999999+test@test.test with: (2) No such file or directory 2024-04-22T20:19:23.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.080+0000 7efddc12e800 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-04-22T20:19:23.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.081+0000 7efddc12e800 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-04-22T20:19:23.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.081+0000 7efddc12e800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW99999999999999999, ofs0, ns] 2024-04-22T20:19:23.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.081+0000 7efcb67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464379 cookie 93998099992064 notifier 7421 bl.length()=384 2024-04-22T20:19:23.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.081+0000 7efcb67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-04-22T20:19:23.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.081+0000 7efcb67fc640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-04-22T20:19:23.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.084+0000 7efddc12e800 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-04-22T20:19:23.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.084+0000 7efddc12e800 10 moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-04-22T20:19:23.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.084+0000 7efddc12e800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW99999999999999999, ofs0, ns] 2024-04-22T20:19:23.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.084+0000 7efcb67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464515 cookie 93998100007984 notifier 7421 bl.length()=225 2024-04-22T20:19:23.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.084+0000 7efcb67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-04-22T20:19:23.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.084+0000 7efcb67fc640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-04-22T20:19:23.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.087+0000 7efddc12e800 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-04-22T20:19:23.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.087+0000 7efddc12e800 10 moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-04-22T20:19:23.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.087+0000 7efddc12e800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw99999999999999999+test@test.test, ofs0, ns] 2024-04-22T20:19:23.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.087+0000 7efcb67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464339 cookie 93998100001296 notifier 7421 bl.length()=234 2024-04-22T20:19:23.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.087+0000 7efcb67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-04-22T20:19:23.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.087+0000 7efcb67fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-04-22T20:19:23.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.089+0000 7efc8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:23.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.092+0000 7efddc12e800 20 remove_watcher() i=0 2024-04-22T20:19:23.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.092+0000 7efddc12e800 2 removed watcher, disabling cache 2024-04-22T20:19:23.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.093+0000 7efddc12e800 20 remove_watcher() i=1 2024-04-22T20:19:23.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.094+0000 7efddc12e800 20 remove_watcher() i=2 2024-04-22T20:19:23.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.095+0000 7efddc12e800 20 remove_watcher() i=3 2024-04-22T20:19:23.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.096+0000 7efddc12e800 20 remove_watcher() i=4 2024-04-22T20:19:23.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.097+0000 7efddc12e800 20 remove_watcher() i=5 2024-04-22T20:19:23.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.098+0000 7efddc12e800 20 remove_watcher() i=6 2024-04-22T20:19:23.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.099+0000 7efddc12e800 20 remove_watcher() i=7 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "id": "RGW99999999999999999", 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "tenant": "", 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "name": "Mr.RGW99999999999999999", 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "RGW99999999999999999+test@test.test", 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "quota": { 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:23.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:23.109 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:23.109 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:23.109 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:23.109 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:23.109 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:23.110 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:23.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:23.110 INFO:teuthology.orchestra.run.smithi136.stdout: "max_users": 1000, 2024-04-22T20:19:23.110 INFO:teuthology.orchestra.run.smithi136.stdout: "max_roles": 1000, 2024-04-22T20:19:23.110 INFO:teuthology.orchestra.run.smithi136.stdout: "max_groups": 1000, 2024-04-22T20:19:23.110 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:23.110 INFO:teuthology.orchestra.run.smithi136.stdout: "max_access_keys": 4 2024-04-22T20:19:23.110 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:23.111 DEBUG:tasks.s3tests:Creating user RGW99999999999999999 on client.0 2024-04-22T20:19:23.112 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root2.client.0 --display-name Mr.RGW99999999999999999 --email root2+test@test.test --access-key WXXVHRWYWNNRHMTNFUEC --secret X4xrt8Ql1LRS3T/WqN3XeCBJZP0VYY65aT44S7J+hD5TgW8YBUxjBA== --cluster ceph --account-id RGW99999999999999999 --account-root 2024-04-22T20:19:23.176 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:23.176 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:23.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.206+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.207+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.207+0000 7fbd6153f800 20 realm 2024-04-22T20:19:23.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.208+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.208+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.208+0000 7fbd6153f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:23.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.208+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.208+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.208+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.209+0000 7fbd6153f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:23.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.209+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.209+0000 7fbd6153f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:23.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.209+0000 7fbd6153f800 20 searching for the correct realm 2024-04-22T20:19:23.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.211+0000 7fbd5da9c640 0 -- 172.21.15.136:0/938703951 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7fbd5806dc20 msgr2=0x7fbd580719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:23.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.416+0000 7fbd5ca9a640 0 -- 172.21.15.136:0/938703951 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7fbd58081a40 msgr2=0x7fbd580418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:23.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:23.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.623+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 20 zone default found 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 4 Realm: () 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:23.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:23.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:23.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 20 zonegroup default 2024-04-22T20:19:23.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.624+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.625+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.625+0000 7fbd6153f800 20 started sync module instance, tier type = 2024-04-22T20:19:23.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.625+0000 7fbd6153f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:23.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.634+0000 7fbd6153f800 20 add_watcher() i=0 2024-04-22T20:19:23.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.634+0000 7fbd6153f800 20 add_watcher() i=1 2024-04-22T20:19:23.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.634+0000 7fbd6153f800 20 add_watcher() i=2 2024-04-22T20:19:23.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.634+0000 7fbd6153f800 20 add_watcher() i=3 2024-04-22T20:19:23.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.634+0000 7fbd6153f800 20 add_watcher() i=4 2024-04-22T20:19:23.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.634+0000 7fbd6153f800 20 add_watcher() i=5 2024-04-22T20:19:23.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.634+0000 7fbd6153f800 20 add_watcher() i=6 2024-04-22T20:19:23.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.635+0000 7fbd6153f800 20 add_watcher() i=7 2024-04-22T20:19:23.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.635+0000 7fbd6153f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.637+0000 7fbc137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.637+0000 7fbd6153f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.637+0000 7fbd6153f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.637+0000 7fbd6153f800 5 note: GC not initialized 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.637+0000 7fbc127e4640 20 reqs_thread_entry: start 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 20 Filter name: none 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbb83fff640 20 reqs_thread_entry: start 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-04-22T20:19:23.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 10 cache get: name=default.rgw.meta+users.email+root2+test@test.test : miss 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 10 adding default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 10 cache get: name=default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC : miss 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.638+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 10 cache put: name=default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC info.flags=0x0 2024-04-22T20:19:23.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 10 adding default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC to cache LRU end 2024-04-22T20:19:23.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 10 cache get: name=default.rgw.meta+accounts+account.RGW99999999999999999 : miss 2024-04-22T20:19:23.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 20 rados_obj.operate() r=0 bl.length=184 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 10 cache get: name=default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC : hit (negative entry) 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.639+0000 7fbd6153f800 10 cache get: name=default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC : hit (negative entry) 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.641+0000 7fbd6153f800 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.641+0000 7fbd6153f800 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.641+0000 7fbd6153f800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.641+0000 7fbc40ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464380 cookie 94542896089776 notifier 7439 bl.length()=681 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.641+0000 7fbc40ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-04-22T20:19:23.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.641+0000 7fbc40ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-22T20:19:23.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.644+0000 7fbd6153f800 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-04-22T20:19:23.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.644+0000 7fbd6153f800 10 moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-04-22T20:19:23.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.644+0000 7fbd6153f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root2+test@test.test, ofs0, ns] 2024-04-22T20:19:23.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.644+0000 7fbc2ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464400 cookie 94542896104016 notifier 7439 bl.length()=189 2024-04-22T20:19:23.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.644+0000 7fbc2ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-04-22T20:19:23.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.644+0000 7fbc2ffff640 10 rgw watcher librados: moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-04-22T20:19:23.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.647+0000 7fbd6153f800 10 cache put: name=default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC info.flags=0x7 2024-04-22T20:19:23.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.647+0000 7fbd6153f800 10 moving default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC to cache LRU end 2024-04-22T20:19:23.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.647+0000 7fbd6153f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:WXXVHRWYWNNRHMTNFUEC, ofs0, ns] 2024-04-22T20:19:23.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.648+0000 7fbc40ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464516 cookie 94542896100464 notifier 7439 bl.length()=188 2024-04-22T20:19:23.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.648+0000 7fbc40ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC info.flags=0x7 2024-04-22T20:19:23.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.648+0000 7fbc40ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+WXXVHRWYWNNRHMTNFUEC to cache LRU end 2024-04-22T20:19:23.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.650+0000 7fbd6153f800 20 linked user root2.client.0 to account RGW99999999999999999 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "user_id": "root2.client.0", 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "display_name": "Mr.RGW99999999999999999", 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "email": "root2+test@test.test", 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "suspended": 0, 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "max_buckets": 1000, 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "subusers": [], 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "keys": [ 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "user": "root2.client.0", 2024-04-22T20:19:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "access_key": "WXXVHRWYWNNRHMTNFUEC", 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "secret_key": "X4xrt8Ql1LRS3T/WqN3XeCBJZP0VYY65aT44S7J+hD5TgW8YBUxjBA==", 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "active": true, 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:23.641320Z" 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "swift_keys": [], 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": [], 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "op_mask": "read, write, delete", 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "default_placement": "", 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "default_storage_class": "", 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "placement_tags": [], 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "bucket_quota": { 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "user_quota": { 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "enabled": false, 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "check_on_raw": false, 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size": -1, 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "max_size_kb": 0, 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "max_objects": -1 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "temp_url_keys": [], 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "type": "root", 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "mfa_ids": [], 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "account_id": "RGW99999999999999999", 2024-04-22T20:19:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "path": "/", 2024-04-22T20:19:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "create_date": "2024-04-22T20:19:23.641304Z", 2024-04-22T20:19:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "tags": [], 2024-04-22T20:19:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "group_ids": [] 2024-04-22T20:19:23.654 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:19:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:19:23.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.651+0000 7fbc137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:23.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.653+0000 7fbd6153f800 20 remove_watcher() i=0 2024-04-22T20:19:23.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.653+0000 7fbd6153f800 2 removed watcher, disabling cache 2024-04-22T20:19:23.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.655+0000 7fbd6153f800 20 remove_watcher() i=1 2024-04-22T20:19:23.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.655+0000 7fbd6153f800 20 remove_watcher() i=2 2024-04-22T20:19:23.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.657+0000 7fbd6153f800 20 remove_watcher() i=3 2024-04-22T20:19:23.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.658+0000 7fbd6153f800 20 remove_watcher() i=4 2024-04-22T20:19:23.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.659+0000 7fbd6153f800 20 remove_watcher() i=5 2024-04-22T20:19:23.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.660+0000 7fbd6153f800 20 remove_watcher() i=6 2024-04-22T20:19:23.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.660+0000 7fbd6153f800 20 remove_watcher() i=7 2024-04-22T20:19:23.671 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root2.client.0 --totp-serial 0400038012 --totp-seed 6E3PPGMIRNO25R3NJM43WPFBO367Y3YHI7OFOEH4TARHS2IJFGMIIW55II3KELY2 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-04-22T20:19:23.734 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:19:23.734 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:19:23.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.764+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.766+0000 7ff64381f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.766+0000 7ff64381f800 20 realm 2024-04-22T20:19:23.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.766+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.766+0000 7ff64381f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.766+0000 7ff64381f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:19:23.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.766+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.766+0000 7ff64381f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:23.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.766+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.767+0000 7ff64381f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:23.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.767+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:23.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.768+0000 7ff64381f800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:19:23.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.768+0000 7ff64381f800 20 searching for the correct realm 2024-04-22T20:19:23.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.770+0000 7ff641b1f640 0 -- 172.21.15.136:0/851906449 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7ff63c018040 msgr2=0x7ff63c05d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:23.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:23.975+0000 7ff640b1d640 0 -- 172.21.15.136:0/851906449 >> [v2:172.21.15.136:6800/2747506017,v1:172.21.15.136:6801/2747506017] conn(0x7ff63c0668a0 msgr2=0x7ff63c078940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:19:24.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:19:24.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:19:24.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:19:24.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:19:24.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:19:24.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:19:24.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:24.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:24.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:24.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:19:24.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.182+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:24.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:19:24.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 20 zone default found 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 4 Realm: () 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 20 zonegroup default 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.183+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.184+0000 7ff64381f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.184+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.184+0000 7ff64381f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.184+0000 7ff64381f800 20 started sync module instance, tier type = 2024-04-22T20:19:24.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.184+0000 7ff64381f800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.193+0000 7ff64381f800 20 add_watcher() i=0 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.193+0000 7ff64381f800 20 add_watcher() i=1 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.193+0000 7ff64381f800 20 add_watcher() i=2 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.193+0000 7ff64381f800 20 add_watcher() i=3 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.193+0000 7ff64381f800 20 add_watcher() i=4 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.193+0000 7ff64381f800 20 add_watcher() i=5 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.193+0000 7ff64381f800 20 add_watcher() i=6 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.194+0000 7ff64381f800 20 add_watcher() i=7 2024-04-22T20:19:24.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.194+0000 7ff64381f800 2 all 8 watchers are set, enabling cache 2024-04-22T20:19:24.197 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.196+0000 7ff4f77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:24.197 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.196+0000 7ff64381f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:19:24.197 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.196+0000 7ff64381f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:19:24.197 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.196+0000 7ff64381f800 5 note: GC not initialized 2024-04-22T20:19:24.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff4f67e4640 20 reqs_thread_entry: start 2024-04-22T20:19:24.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff64381f800 20 init_complete bucket index max shards: 1 2024-04-22T20:19:24.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff64381f800 20 Filter name: none 2024-04-22T20:19:24.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff467fff640 20 reqs_thread_entry: start 2024-04-22T20:19:24.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff64381f800 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-04-22T20:19:24.198 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff64381f800 20 rados->read ofs=0 len=0 2024-04-22T20:19:24.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff64381f800 20 rados_obj.operate() r=0 bl.length=494 2024-04-22T20:19:24.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff64381f800 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x13 2024-04-22T20:19:24.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff64381f800 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-22T20:19:24.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.197+0000 7ff64381f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:19:24.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.200+0000 7ff64381f800 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-04-22T20:19:24.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.200+0000 7ff64381f800 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-22T20:19:24.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.200+0000 7ff64381f800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-04-22T20:19:24.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.201+0000 7ff524ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464381 cookie 94386118911232 notifier 7911 bl.length()=695 2024-04-22T20:19:24.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.201+0000 7ff524ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-04-22T20:19:24.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.201+0000 7ff524ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-04-22T20:19:24.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.203+0000 7ff4f77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:19:24.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.205+0000 7ff64381f800 20 remove_watcher() i=0 2024-04-22T20:19:24.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.205+0000 7ff64381f800 2 removed watcher, disabling cache 2024-04-22T20:19:24.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.206+0000 7ff64381f800 20 remove_watcher() i=1 2024-04-22T20:19:24.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.207+0000 7ff64381f800 20 remove_watcher() i=2 2024-04-22T20:19:24.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.209+0000 7ff64381f800 20 remove_watcher() i=3 2024-04-22T20:19:24.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.210+0000 7ff64381f800 20 remove_watcher() i=4 2024-04-22T20:19:24.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.211+0000 7ff64381f800 20 remove_watcher() i=5 2024-04-22T20:19:24.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.211+0000 7ff64381f800 20 remove_watcher() i=6 2024-04-22T20:19:24.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:19:24.213+0000 7ff64381f800 20 remove_watcher() i=7 2024-04-22T20:19:24.223 INFO:tasks.s3tests:Configuring s3-tests... 2024-04-22T20:19:24.224 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T20:19:24.224 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2024-04-22T20:19:24.252 INFO:tasks.s3tests:Configuring boto... 2024-04-22T20:19:24.252 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-22T20:19:24.252 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/boto-client.0.cfg 2024-04-22T20:19:24.320 DEBUG:teuthology.orchestra.run.smithi136:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-04-22T20:19:24.489 INFO:teuthology.orchestra.run.smithi136.stdout:GLOB sdist-make: /home/ubuntu/cephtest/s3-tests-client.0/setup.py 2024-04-22T20:19:24.668 INFO:teuthology.orchestra.run.smithi136.stdout:py create: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py 2024-04-22T20:19:24.905 INFO:teuthology.orchestra.run.smithi136.stdout:py installdeps: -rrequirements.txt 2024-04-22T20:21:11.876 INFO:teuthology.orchestra.run.smithi136.stdout:py inst: /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip 2024-04-22T20:21:13.576 INFO:teuthology.orchestra.run.smithi136.stdout:py installed: boto==2.49.0,boto3==1.24.96,botocore==1.27.96,cachetools==5.3.3,certifi==2024.2.2,chardet==5.2.0,charset-normalizer==3.3.2,colorama==0.4.6,distlib==0.3.8,exceptiongroup==1.2.1,filelock==3.13.4,gevent==24.2.1,greenlet==3.0.3,httplib2==0.22.0,idna==3.7,iniconfig==2.0.0,isodate==0.6.1,jmespath==1.0.1,lxml==5.2.1,munch==4.0.0,packaging==24.0,platformdirs==4.2.0,pluggy==1.5.0,pyparsing==3.1.2,pyproject-api==1.6.1,pytest==8.1.1,python-dateutil==2.9.0.post0,pytz==2024.1,PyYAML==6.0.1,requests==2.31.0,s3tests @ file:///home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip#sha256=e6ec1a856a6ed9003921b34135f79cf3fedea15dc047326f3024eefcc3c1bef6,s3transfer==0.6.2,six==1.16.0,tomli==2.0.1,tox==4.14.2,urllib3==1.26.18,virtualenv==20.25.3,zope.event==5.0,zope.interface==6.3 2024-04-22T20:21:13.577 INFO:teuthology.orchestra.run.smithi136.stdout:py run-test-pre: PYTHONHASHSEED='1081905096' 2024-04-22T20:21:13.577 INFO:teuthology.orchestra.run.smithi136.stdout:py run-test: commands[0] | pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-04-22T20:21:13.722 INFO:teuthology.orchestra.run.smithi136.stdout:============================= test session starts ============================== 2024-04-22T20:21:13.722 INFO:teuthology.orchestra.run.smithi136.stdout:platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.5.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python 2024-04-22T20:21:13.722 INFO:teuthology.orchestra.run.smithi136.stdout:cachedir: .tox/py/.pytest_cache 2024-04-22T20:21:13.722 INFO:teuthology.orchestra.run.smithi136.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0 2024-04-22T20:21:13.722 INFO:teuthology.orchestra.run.smithi136.stdout:configfile: pytest.ini 2024-04-22T20:21:15.498 INFO:teuthology.orchestra.run.smithi136.stdout:collecting ... collected 859 items / 127 deselected / 732 selected 2024-04-22T20:21:15.498 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:21:16.198 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_none PASSED [ 0%] 2024-04-22T20:21:16.531 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_empty PASSED [ 0%] 2024-04-22T20:21:16.831 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_date_and_amz_date PASSED [ 0%] 2024-04-22T20:21:17.128 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_amz_date_and_no_date PASSED [ 0%] 2024-04-22T20:21:17.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_none PASSED [ 0%] 2024-04-22T20:21:17.679 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_contentlength_none PASSED [ 0%] 2024-04-22T20:21:18.029 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_acl_create_contentlength_none PASSED [ 0%] 2024-04-22T20:21:18.251 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_contentlength_none PASSED [ 1%] 2024-04-22T20:21:18.377 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_empty PASSED [ 1%] 2024-04-22T20:21:18.502 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_none PASSED [ 1%] 2024-04-22T20:21:18.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws2 PASSED [ 1%] 2024-04-22T20:21:19.191 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws2 PASSED [ 1%] 2024-04-22T20:21:19.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws2 PASSED [ 1%] 2024-04-22T20:21:19.853 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws2 PASSED [ 1%] 2024-04-22T20:21:19.981 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws2 PASSED [ 2%] 2024-04-22T20:21:20.110 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws2 PASSED [ 2%] 2024-04-22T20:21:20.190 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws4 SKIPPED [ 2%] 2024-04-22T20:21:20.273 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws4 SKIPPED [ 2%] 2024-04-22T20:21:20.360 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws4 SKIPPED [ 2%] 2024-04-22T20:21:20.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws4 SKIPPED [ 2%] 2024-04-22T20:21:20.520 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_empty_aws4 SKIPPED [ 2%] 2024-04-22T20:21:20.602 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_none_aws4 SKIPPED [ 3%] 2024-04-22T20:21:20.682 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_invalid_aws4 SKIPPED [ 3%] 2024-04-22T20:21:20.762 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_invalid_aws4 SKIPPED [ 3%] 2024-04-22T20:21:20.842 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_empty_aws4 SKIPPED [ 3%] 2024-04-22T20:21:20.926 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_empty_aws4 SKIPPED [ 3%] 2024-04-22T20:21:21.005 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws4 SKIPPED [ 3%] 2024-04-22T20:21:21.086 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_none_aws4 SKIPPED [ 3%] 2024-04-22T20:21:21.165 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_today_aws4 SKIPPED [ 3%] 2024-04-22T20:21:21.246 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_today_aws4 SKIPPED [ 4%] 2024-04-22T20:21:21.326 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_today_aws4 SKIPPED [ 4%] 2024-04-22T20:21:21.407 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_today_aws4 SKIPPED [ 4%] 2024-04-22T20:21:21.488 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws4 SKIPPED [ 4%] 2024-04-22T20:21:21.571 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 4%] 2024-04-22T20:21:21.652 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_end_aws4 SKIPPED [ 4%] 2024-04-22T20:21:21.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_end_aws4 SKIPPED [ 4%] 2024-04-22T20:21:21.811 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_custom_header_aws4 SKIPPED [ 5%] 2024-04-22T20:21:21.890 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_header_aws4 SKIPPED [ 5%] 2024-04-22T20:21:21.972 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws4 SKIPPED [ 5%] 2024-04-22T20:21:22.052 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws4 SKIPPED [ 5%] 2024-04-22T20:21:22.133 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_none_aws4 SKIPPED [ 5%] 2024-04-22T20:21:22.212 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws4 SKIPPED [ 5%] 2024-04-22T20:21:22.292 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_invalid_aws4 SKIPPED [ 5%] 2024-04-22T20:21:22.372 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_empty_aws4 SKIPPED [ 6%] 2024-04-22T20:21:22.453 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_empty_aws4 SKIPPED [ 6%] 2024-04-22T20:21:22.532 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws4 SKIPPED [ 6%] 2024-04-22T20:21:22.613 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_none_aws4 SKIPPED [ 6%] 2024-04-22T20:21:22.693 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws4 SKIPPED [ 6%] 2024-04-22T20:21:22.773 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_today_aws4 SKIPPED [ 6%] 2024-04-22T20:21:22.854 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws4 SKIPPED [ 6%] 2024-04-22T20:21:22.934 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_after_today_aws4 SKIPPED [ 6%] 2024-04-22T20:21:23.014 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws4 SKIPPED [ 7%] 2024-04-22T20:21:23.095 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 7%] 2024-04-22T20:21:23.891 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_versioning_obj_read_not_exist_null PASSED [ 7%] 2024-04-22T20:21:32.261 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_object_storage_class PASSED [ 7%] 2024-04-22T20:21:39.926 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_object_storage_class_multipart PASSED [ 7%] 2024-04-22T20:21:55.416 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class PASSED [ 7%] 2024-04-22T20:22:16.318 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class_multipart PASSED [ 7%] 2024-04-22T20:22:22.555 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy PASSED [ 8%] 2024-04-22T20:22:28.210 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy_multipart PASSED [ 8%] 2024-04-22T20:22:29.690 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_atomic_dual_conditional_write_1mb PASSED [ 8%] 2024-04-22T20:22:29.992 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_atomic_write_bucket_gone PASSED [ 8%] 2024-04-22T20:22:30.885 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_1 PASSED [ 8%] 2024-04-22T20:22:32.127 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_2 PASSED [ 8%] 2024-04-22T20:22:33.222 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_bucket_policy_set_condition_operator_end_with_IfExists PASSED [ 8%] 2024-04-22T20:22:33.834 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_request_obj_tag PASSED [ 9%] 2024-04-22T20:22:34.346 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_nonexistant_bucket_rgw SKIPPED [ 9%] 2024-04-22T20:22:34.365 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_public_index SKIPPED [ 9%] 2024-04-22T20:22:34.365 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_public_index SKIPPED [ 9%] 2024-04-22T20:22:34.367 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty SKIPPED [ 9%] 2024-04-22T20:22:34.367 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty SKIPPED [ 9%] 2024-04-22T20:22:34.368 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index SKIPPED [ 9%] 2024-04-22T20:22:34.369 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index SKIPPED [ 9%] 2024-04-22T20:22:34.370 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-04-22T20:22:34.371 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-04-22T20:22:34.373 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-04-22T20:22:34.374 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-04-22T20:22:34.375 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-04-22T20:22:34.377 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_pubilc_errordoc SKIPPED [ 10%] 2024-04-22T20:22:34.378 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-04-22T20:22:34.380 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-04-22T20:22:34.381 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-04-22T20:22:34.382 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-04-22T20:22:34.383 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-04-22T20:22:34.385 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-04-22T20:22:34.386 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-04-22T20:22:34.388 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_base SKIPPED [ 11%] 2024-04-22T20:22:34.390 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path SKIPPED [ 12%] 2024-04-22T20:22:34.391 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path_upgrade SKIPPED [ 12%] 2024-04-22T20:22:34.392 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_nonwebsite SKIPPED [ 12%] 2024-04-22T20:22:34.394 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_relative SKIPPED [ 12%] 2024-04-22T20:22:34.395 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_abs SKIPPED [ 12%] 2024-04-22T20:22:34.397 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_relative SKIPPED [ 12%] 2024-04-22T20:22:34.398 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_abs SKIPPED [ 12%] 2024-04-22T20:22:34.529 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py::test_routing_generator XFAIL [ 12%] 2024-04-22T20:22:34.546 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_utils.py::test_generate PASSED [ 13%] 2024-04-22T20:22:35.792 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_short PASSED [ 13%] 2024-04-22T20:22:45.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_bad PASSED [ 13%] 2024-04-22T20:22:46.388 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_empty PASSED [ 13%] 2024-04-22T20:22:47.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_none PASSED [ 13%] 2024-04-22T20:22:48.275 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_mismatch PASSED [ 13%] 2024-04-22T20:22:49.222 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_empty PASSED [ 13%] 2024-04-22T20:22:50.171 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_none PASSED [ 14%] 2024-04-22T20:22:50.959 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contentlength_negative PASSED [ 14%] 2024-04-22T20:22:51.957 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_invalid PASSED [ 14%] 2024-04-22T20:22:52.807 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_empty PASSED [ 14%] 2024-04-22T20:22:53.652 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_none PASSED [ 14%] 2024-04-22T20:22:54.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_put_bad_canned_acl PASSED [ 14%] 2024-04-22T20:22:55.112 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_mismatch PASSED [ 14%] 2024-04-22T20:22:55.789 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_empty PASSED [ 15%] 2024-04-22T20:22:56.362 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_contentlength_negative PASSED [ 15%] 2024-04-22T20:22:57.146 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws2 PASSED [ 15%] 2024-04-22T20:22:58.018 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_empty_aws2 PASSED [ 15%] 2024-04-22T20:22:58.915 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_none_aws2 PASSED [ 15%] 2024-04-22T20:22:59.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_invalid_aws2 PASSED [ 15%] 2024-04-22T20:23:00.486 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_empty_aws2 PASSED [ 15%] 2024-04-22T20:23:01.267 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_today_aws2 PASSED [ 15%] 2024-04-22T20:23:02.046 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws2 PASSED [ 16%] 2024-04-22T20:23:02.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_after_end_aws2 PASSED [ 16%] 2024-04-22T20:23:03.505 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws2 PASSED [ 16%] 2024-04-22T20:23:04.186 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_none_aws2 PASSED [ 16%] 2024-04-22T20:23:04.760 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws2 PASSED [ 16%] 2024-04-22T20:23:05.359 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_empty_aws2 PASSED [ 16%] 2024-04-22T20:23:05.929 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws2 PASSED [ 16%] 2024-04-22T20:23:06.493 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws2 PASSED [ 17%] 2024-04-22T20:23:07.054 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws2 PASSED [ 17%] 2024-04-22T20:23:08.019 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy PASSED [ 17%] 2024-04-22T20:23:08.594 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_invalid_user PASSED [ 17%] 2024-04-22T20:23:09.173 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_parameter_limit PASSED [ 17%] 2024-04-22T20:23:09.893 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_put_existing_user_policy PASSED [ 17%] 2024-04-22T20:23:10.594 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy PASSED [ 17%] 2024-04-22T20:23:11.167 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy_invalid_user PASSED [ 18%] 2024-04-22T20:23:11.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy PASSED [ 18%] 2024-04-22T20:23:12.627 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_invalid_user PASSED [ 18%] 2024-04-22T20:23:13.482 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_from_multiple_policies PASSED [ 18%] 2024-04-22T20:23:14.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy PASSED [ 18%] 2024-04-22T20:23:14.828 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_user PASSED [ 18%] 2024-04-22T20:23:15.532 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_policy_name PASSED [ 18%] 2024-04-22T20:23:16.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_from_multiple_policies PASSED [ 18%] 2024-04-22T20:23:17.504 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_allow_bucket_actions_in_user_policy PASSED [ 19%] 2024-04-22T20:23:18.335 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_deny_bucket_actions_in_user_policy PASSED [ 19%] 2024-04-22T20:23:19.320 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_allow_object_actions_in_user_policy PASSED [ 19%] 2024-04-22T20:23:20.427 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_deny_object_actions_in_user_policy PASSED [ 19%] 2024-04-22T20:23:22.337 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_allow_multipart_actions_in_user_policy PASSED [ 19%] 2024-04-22T20:23:24.247 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_deny_multipart_actions_in_user_policy PASSED [ 19%] 2024-04-22T20:23:25.347 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_allow_tagging_actions_in_user_policy PASSED [ 19%] 2024-04-22T20:23:26.418 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_deny_tagging_actions_in_user_policy PASSED [ 20%] 2024-04-22T20:23:27.294 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policy_statements PASSED [ 20%] 2024-04-22T20:23:28.330 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policies PASSED [ 20%] 2024-04-22T20:23:29.137 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_verify_allow_iam_actions PASSED [ 20%] 2024-04-22T20:23:30.833 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_create PASSED [ 20%] 2024-04-22T20:23:31.979 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_case_insensitive_name PASSED [ 20%] 2024-04-22T20:23:33.323 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_delete PASSED [ 20%] 2024-04-22T20:23:35.848 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list PASSED [ 21%] 2024-04-22T20:23:39.114 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list_path_prefix PASSED [ 21%] 2024-04-22T20:23:41.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_name PASSED [ 21%] 2024-04-22T20:23:42.818 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_path PASSED [ 21%] 2024-04-22T20:23:44.204 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_create PASSED [ 21%] 2024-04-22T20:23:45.217 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_create PASSED [ 21%] 2024-04-22T20:23:46.837 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_update PASSED [ 21%] 2024-04-22T20:23:48.026 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_update PASSED [ 21%] 2024-04-22T20:23:49.565 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_delete PASSED [ 22%] 2024-04-22T20:23:50.711 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_delete PASSED [ 22%] 2024-04-22T20:23:52.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_list PASSED [ 22%] 2024-04-22T20:23:54.258 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_bucket_policy_allow PASSED [ 22%] 2024-04-22T20:23:56.384 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy PASSED [ 22%] 2024-04-22T20:23:58.508 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_managed PASSED [ 22%] 2024-04-22T20:24:00.050 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_allow PASSED [ 22%] 2024-04-22T20:24:01.465 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_create PASSED [ 23%] 2024-04-22T20:24:02.617 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_case_insensitive_name PASSED [ 23%] 2024-04-22T20:24:05.110 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_list PASSED [ 23%] 2024-04-22T20:24:06.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_update PASSED [ 23%] 2024-04-22T20:24:09.108 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy PASSED [ 23%] 2024-04-22T20:24:11.117 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy PASSED [ 23%] 2024-04-22T20:24:13.086 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy_allow PASSED [ 23%] 2024-04-22T20:24:15.001 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy_allow PASSED [ 24%] 2024-04-22T20:24:16.617 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_create PASSED [ 24%] 2024-04-22T20:24:17.820 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_case_insensitive_name PASSED [ 24%] 2024-04-22T20:24:19.151 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_delete PASSED [ 24%] 2024-04-22T20:24:21.453 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list PASSED [ 24%] 2024-04-22T20:24:24.382 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list_path_prefix PASSED [ 24%] 2024-04-22T20:24:25.753 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_update PASSED [ 24%] 2024-04-22T20:24:27.762 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy PASSED [ 25%] 2024-04-22T20:24:29.716 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_managed PASSED [ 25%] 2024-04-22T20:24:31.645 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow PASSED [ 25%] 2024-04-22T20:24:33.744 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_same_account_role_policy_allow PASSED [ 25%] 2024-04-22T20:24:35.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_role_policy_allow PASSED [ 25%] 2024-04-22T20:24:38.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_create_bucket PASSED [ 25%] 2024-04-22T20:24:40.120 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_get_role PASSED [ 25%] 2024-04-22T20:24:41.457 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_account_oidc_provider PASSED [ 25%] 2024-04-22T20:24:43.321 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_user_arn PASSED [ 26%] 2024-04-22T20:24:45.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_user_arn PASSED [ 26%] 2024-04-22T20:24:47.188 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_arn PASSED [ 26%] 2024-04-22T20:24:49.038 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_arn PASSED [ 26%] 2024-04-22T20:24:51.024 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_id PASSED [ 26%] 2024-04-22T20:24:52.880 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_id PASSED [ 26%] 2024-04-22T20:24:54.738 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_user_id PASSED [ 26%] 2024-04-22T20:24:56.668 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_user_id PASSED [ 27%] 2024-04-22T20:24:58.527 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_canonical_id PASSED [ 27%] 2024-04-22T20:25:00.486 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_canonical_id PASSED [ 27%] 2024-04-22T20:25:02.354 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_account_email PASSED [ 27%] 2024-04-22T20:25:04.271 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_account_email PASSED [ 27%] 2024-04-22T20:25:05.646 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_arn PASSED [ 27%] 2024-04-22T20:25:07.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_id PASSED [ 27%] 2024-04-22T20:25:08.348 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_canonical_id PASSED [ 28%] 2024-04-22T20:25:09.641 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_account_email PASSED [ 28%] 2024-04-22T20:25:10.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_empty PASSED [ 28%] 2024-04-22T20:25:11.732 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_distinct PASSED [ 28%] 2024-04-22T20:25:12.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_many PASSED [ 28%] 2024-04-22T20:25:13.739 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_many PASSED [ 28%] 2024-04-22T20:25:14.698 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_basic_key_count PASSED [ 28%] 2024-04-22T20:25:15.742 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_basic PASSED [ 28%] 2024-04-22T20:25:16.744 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_basic PASSED [ 29%] 2024-04-22T20:25:17.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_encoding_basic PASSED [ 29%] 2024-04-22T20:25:18.718 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_encoding_basic PASSED [ 29%] 2024-04-22T20:25:20.412 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix PASSED [ 29%] 2024-04-22T20:25:22.146 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix PASSED [ 29%] 2024-04-22T20:25:23.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-04-22T20:25:23.881 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-04-22T20:25:24.873 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_alt PASSED [ 30%] 2024-04-22T20:25:25.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_alt PASSED [ 30%] 2024-04-22T20:25:27.578 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_underscore PASSED [ 30%] 2024-04-22T20:25:29.300 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_underscore PASSED [ 30%] 2024-04-22T20:25:30.295 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_percentage PASSED [ 30%] 2024-04-22T20:25:31.275 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_percentage PASSED [ 30%] 2024-04-22T20:25:32.262 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_whitespace PASSED [ 30%] 2024-04-22T20:25:33.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_whitespace PASSED [ 31%] 2024-04-22T20:25:34.247 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_dot PASSED [ 31%] 2024-04-22T20:25:35.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_dot PASSED [ 31%] 2024-04-22T20:25:36.275 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_unreadable PASSED [ 31%] 2024-04-22T20:25:37.263 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_unreadable PASSED [ 31%] 2024-04-22T20:25:38.258 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_empty PASSED [ 31%] 2024-04-22T20:25:39.252 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_empty PASSED [ 31%] 2024-04-22T20:25:40.257 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_none PASSED [ 31%] 2024-04-22T20:25:41.288 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_none PASSED [ 32%] 2024-04-22T20:25:42.230 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_notempty PASSED [ 32%] 2024-04-22T20:25:43.169 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_defaultempty PASSED [ 32%] 2024-04-22T20:25:44.114 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_empty PASSED [ 32%] 2024-04-22T20:25:45.110 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_exist PASSED [ 32%] 2024-04-22T20:25:46.106 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_not_exist PASSED [ 32%] 2024-04-22T20:26:34.904 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_skip_special PASSED [ 32%] 2024-04-22T20:26:35.850 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_basic PASSED [ 33%] 2024-04-22T20:26:36.804 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_basic PASSED [ 33%] 2024-04-22T20:26:37.785 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_alt PASSED [ 33%] 2024-04-22T20:26:38.745 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_alt PASSED [ 33%] 2024-04-22T20:26:39.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_empty PASSED [ 33%] 2024-04-22T20:26:40.644 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_empty PASSED [ 33%] 2024-04-22T20:26:41.594 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_none PASSED [ 33%] 2024-04-22T20:26:42.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_none PASSED [ 34%] 2024-04-22T20:26:43.484 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_not_exist PASSED [ 34%] 2024-04-22T20:26:44.466 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_not_exist PASSED [ 34%] 2024-04-22T20:26:45.405 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_unreadable PASSED [ 34%] 2024-04-22T20:26:46.362 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_unreadable PASSED [ 34%] 2024-04-22T20:26:47.343 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_basic PASSED [ 34%] 2024-04-22T20:26:48.321 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_basic PASSED [ 34%] 2024-04-22T20:26:49.315 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt PASSED [ 34%] 2024-04-22T20:26:50.300 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt PASSED [ 35%] 2024-04-22T20:26:51.337 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-04-22T20:26:52.323 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-04-22T20:26:53.304 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-04-22T20:26:54.308 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-04-22T20:26:55.292 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-04-22T20:26:56.273 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-04-22T20:26:57.385 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_one PASSED [ 36%] 2024-04-22T20:26:58.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_one PASSED [ 36%] 2024-04-22T20:26:59.426 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_zero PASSED [ 36%] 2024-04-22T20:27:00.412 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_zero PASSED [ 36%] 2024-04-22T20:27:01.416 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_none PASSED [ 36%] 2024-04-22T20:27:02.401 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_none PASSED [ 36%] 2024-04-22T20:27:02.961 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_account_usage PASSED [ 36%] 2024-04-22T20:27:03.878 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_head_bucket_usage PASSED [ 37%] 2024-04-22T20:27:06.159 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_unordered PASSED [ 37%] 2024-04-22T20:27:08.414 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_unordered PASSED [ 37%] 2024-04-22T20:27:09.406 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_invalid PASSED [ 37%] 2024-04-22T20:27:10.399 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_none PASSED [ 37%] 2024-04-22T20:27:11.437 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_empty PASSED [ 37%] 2024-04-22T20:27:12.438 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken_empty PASSED [ 37%] 2024-04-22T20:27:13.484 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken PASSED [ 37%] 2024-04-22T20:27:14.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_both_continuationtoken_startafter PASSED [ 38%] 2024-04-22T20:27:15.520 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_unreadable PASSED [ 38%] 2024-04-22T20:27:16.516 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_unreadable PASSED [ 38%] 2024-04-22T20:27:17.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_not_in_list PASSED [ 38%] 2024-04-22T20:27:18.543 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list PASSED [ 38%] 2024-04-22T20:27:19.684 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_after_list PASSED [ 38%] 2024-04-22T20:27:20.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_after_list PASSED [ 38%] 2024-04-22T20:27:21.917 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data PASSED [ 39%] 2024-04-22T20:27:23.513 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning PASSED [ 39%] 2024-04-22T20:27:24.385 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous PASSED [ 39%] 2024-04-22T20:27:25.289 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous PASSED [ 39%] 2024-04-22T20:27:26.053 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous_fail PASSED [ 39%] 2024-04-22T20:27:26.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail PASSED [ 39%] 2024-04-22T20:27:27.403 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_notexist PASSED [ 39%] 2024-04-22T20:27:27.985 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_notexist PASSED [ 40%] 2024-04-22T20:27:28.553 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_notexist PASSED [ 40%] 2024-04-22T20:27:29.417 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_nonempty PASSED [ 40%] 2024-04-22T20:27:33.298 INFO:tasks.rgw.client.0.smithi136.stdout:2024-04-22T20:27:33.296+0000 64caf640 -1 req 8992558722512326851 0.039999109s s3:delete_bucket ERROR: could not remove bucket test-8v4uvy8gpgkx125ih4plr0qu-156 2024-04-22T20:27:33.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_concurrent_set_canned_acl PASSED [ 40%] 2024-04-22T20:27:34.010 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket PASSED [ 40%] 2024-04-22T20:27:34.010 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket ERROR [ 40%] 2024-04-22T20:27:34.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding PASSED [ 40%] 2024-04-22T20:27:34.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding ERROR [ 40%] 2024-04-22T20:27:35.034 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete PASSED [ 40%] 2024-04-22T20:27:35.034 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete ERROR [ 40%] 2024-04-22T20:27:35.541 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist PASSED [ 40%] 2024-04-22T20:27:35.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist ERROR [ 40%] 2024-04-22T20:27:36.095 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error PASSED [ 41%] 2024-04-22T20:27:36.095 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error ERROR [ 41%] 2024-04-22T20:27:38.074 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete PASSED [ 41%] 2024-04-22T20:27:38.074 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete ERROR [ 41%] 2024-04-22T20:27:38.934 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete PASSED [ 41%] 2024-04-22T20:27:38.934 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete ERROR [ 41%] 2024-04-22T20:27:39.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete PASSED [ 41%] 2024-04-22T20:27:39.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete ERROR [ 41%] 2024-04-22T20:28:29.438 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit PASSED [ 41%] 2024-04-22T20:28:29.439 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit ERROR [ 41%] 2024-04-22T20:29:17.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit PASSED [ 41%] 2024-04-22T20:29:17.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit ERROR [ 41%] 2024-04-22T20:29:18.756 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes PASSED [ 41%] 2024-04-22T20:29:18.756 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes ERROR [ 41%] 2024-04-22T20:29:19.335 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag PASSED [ 42%] 2024-04-22T20:29:19.336 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag ERROR [ 42%] 2024-04-22T20:29:19.939 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control PASSED [ 42%] 2024-04-22T20:29:19.940 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control ERROR [ 42%] 2024-04-22T20:29:20.588 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires PASSED [ 42%] 2024-04-22T20:29:20.588 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires ERROR [ 42%] 2024-04-22T20:29:21.320 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete PASSED [ 42%] 2024-04-22T20:29:21.320 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete ERROR [ 42%] 2024-04-22T20:29:21.934 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good PASSED [ 42%] 2024-04-22T20:29:21.934 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good ERROR [ 42%] 2024-04-22T20:29:22.540 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty PASSED [ 42%] 2024-04-22T20:29:22.541 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty ERROR [ 42%] 2024-04-22T20:29:23.352 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty PASSED [ 42%] 2024-04-22T20:29:23.352 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty ERROR [ 42%] 2024-04-22T20:29:24.004 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put PASSED [ 43%] 2024-04-22T20:29:24.004 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put ERROR [ 43%] 2024-04-22T20:29:24.645 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file PASSED [ 43%] 2024-04-22T20:29:24.645 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file ERROR [ 43%] 2024-04-22T20:29:25.309 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request PASSED [ 43%] 2024-04-22T20:29:25.310 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request ERROR [ 43%] 2024-04-22T20:29:26.122 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request PASSED [ 43%] 2024-04-22T20:29:26.123 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request ERROR [ 43%] 2024-04-22T20:29:26.740 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type PASSED [ 43%] 2024-04-22T20:29:26.740 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type ERROR [ 43%] 2024-04-22T20:29:27.265 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key PASSED [ 43%] 2024-04-22T20:29:27.265 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key ERROR [ 43%] 2024-04-22T20:29:27.854 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code PASSED [ 43%] 2024-04-22T20:29:27.854 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code ERROR [ 43%] 2024-04-22T20:29:28.485 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code PASSED [ 43%] 2024-04-22T20:29:28.485 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code ERROR [ 43%] 2024-04-22T20:29:29.993 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk PASSED [ 44%] 2024-04-22T20:29:29.993 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk ERROR [ 44%] 2024-04-22T20:29:30.688 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename PASSED [ 44%] 2024-04-22T20:29:30.688 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename ERROR [ 44%] 2024-04-22T20:29:31.276 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header PASSED [ 44%] 2024-04-22T20:29:31.276 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header ERROR [ 44%] 2024-04-22T20:29:31.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields PASSED [ 44%] 2024-04-22T20:29:31.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields ERROR [ 44%] 2024-04-22T20:29:32.611 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values PASSED [ 44%] 2024-04-22T20:29:32.611 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values ERROR [ 44%] 2024-04-22T20:29:33.281 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action PASSED [ 44%] 2024-04-22T20:29:33.281 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action ERROR [ 44%] 2024-04-22T20:29:33.811 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature PASSED [ 44%] 2024-04-22T20:29:33.811 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature ERROR [ 44%] 2024-04-22T20:29:34.347 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key PASSED [ 45%] 2024-04-22T20:29:34.347 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key ERROR [ 45%] 2024-04-22T20:29:34.920 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format PASSED [ 45%] 2024-04-22T20:29:34.920 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format ERROR [ 45%] 2024-04-22T20:29:35.448 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified PASSED [ 45%] 2024-04-22T20:29:35.448 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified ERROR [ 45%] 2024-04-22T20:29:35.979 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature PASSED [ 45%] 2024-04-22T20:29:35.979 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature ERROR [ 45%] 2024-04-22T20:29:36.522 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition PASSED [ 45%] 2024-04-22T20:29:36.523 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition ERROR [ 45%] 2024-04-22T20:29:37.250 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header PASSED [ 45%] 2024-04-22T20:29:37.250 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header ERROR [ 45%] 2024-04-22T20:29:37.781 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field PASSED [ 45%] 2024-04-22T20:29:37.782 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field ERROR [ 45%] 2024-04-22T20:29:38.316 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive PASSED [ 46%] 2024-04-22T20:29:38.316 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive ERROR [ 46%] 2024-04-22T20:29:38.845 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive PASSED [ 46%] 2024-04-22T20:29:38.846 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive ERROR [ 46%] 2024-04-22T20:29:39.420 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy PASSED [ 46%] 2024-04-22T20:29:39.420 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy ERROR [ 46%] 2024-04-22T20:29:40.163 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket PASSED [ 46%] 2024-04-22T20:29:40.163 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket ERROR [ 46%] 2024-04-22T20:29:40.694 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value PASSED [ 46%] 2024-04-22T20:29:40.694 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value ERROR [ 46%] 2024-04-22T20:29:41.227 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition PASSED [ 46%] 2024-04-22T20:29:41.227 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition ERROR [ 46%] 2024-04-22T20:29:41.800 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list PASSED [ 46%] 2024-04-22T20:29:41.800 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list ERROR [ 46%] 2024-04-22T20:29:42.349 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded PASSED [ 46%] 2024-04-22T20:29:42.349 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded ERROR [ 46%] 2024-04-22T20:29:42.881 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument PASSED [ 47%] 2024-04-22T20:29:42.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument ERROR [ 47%] 2024-04-22T20:29:43.416 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument PASSED [ 47%] 2024-04-22T20:29:43.416 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument ERROR [ 47%] 2024-04-22T20:29:43.991 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum PASSED [ 47%] 2024-04-22T20:29:43.991 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum ERROR [ 47%] 2024-04-22T20:29:45.850 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug PASSED [ 47%] 2024-04-22T20:29:45.850 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug ERROR [ 47%] 2024-04-22T20:29:46.377 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions PASSED [ 47%] 2024-04-22T20:29:46.377 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions ERROR [ 47%] 2024-04-22T20:29:47.028 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good PASSED [ 47%] 2024-04-22T20:29:47.028 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good ERROR [ 47%] 2024-04-22T20:29:47.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed PASSED [ 47%] 2024-04-22T20:29:47.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed ERROR [ 47%] 2024-04-22T20:29:48.343 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good PASSED [ 48%] 2024-04-22T20:29:48.343 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good ERROR [ 48%] 2024-04-22T20:29:48.988 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed PASSED [ 48%] 2024-04-22T20:29:48.988 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed ERROR [ 48%] 2024-04-22T20:29:49.647 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good PASSED [ 48%] 2024-04-22T20:29:49.647 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good ERROR [ 48%] 2024-04-22T20:29:51.396 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed PASSED [ 48%] 2024-04-22T20:29:51.396 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed ERROR [ 48%] 2024-04-22T20:29:52.047 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good PASSED [ 48%] 2024-04-22T20:29:52.047 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good ERROR [ 48%] 2024-04-22T20:29:52.706 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed PASSED [ 48%] 2024-04-22T20:29:52.706 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed ERROR [ 48%] 2024-04-22T20:29:53.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good PASSED [ 48%] 2024-04-22T20:29:53.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good ERROR [ 48%] 2024-04-22T20:29:54.295 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed PASSED [ 49%] 2024-04-22T20:29:54.296 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed ERROR [ 49%] 2024-04-22T20:29:55.043 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good PASSED [ 49%] 2024-04-22T20:29:55.044 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good ERROR [ 49%] 2024-04-22T20:29:55.658 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed PASSED [ 49%] 2024-04-22T20:29:55.658 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed ERROR [ 49%] 2024-04-22T20:29:56.410 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good PASSED [ 49%] 2024-04-22T20:29:56.411 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good ERROR [ 49%] 2024-04-22T20:29:57.246 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed PASSED [ 49%] 2024-04-22T20:29:57.246 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed ERROR [ 49%] 2024-04-22T20:29:57.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good PASSED [ 49%] 2024-04-22T20:29:57.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good ERROR [ 49%] 2024-04-22T20:29:58.693 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed PASSED [ 49%] 2024-04-22T20:29:58.693 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed ERROR [ 49%] 2024-04-22T20:29:59.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get PASSED [ 50%] 2024-04-22T20:29:59.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get ERROR [ 50%] 2024-04-22T20:29:59.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone PASSED [ 50%] 2024-04-22T20:29:59.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone ERROR [ 50%] 2024-04-22T20:30:00.434 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone PASSED [ 50%] 2024-04-22T20:30:00.434 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone ERROR [ 50%] 2024-04-22T20:30:01.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone PASSED [ 50%] 2024-04-22T20:30:01.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone ERROR [ 50%] 2024-04-22T20:30:01.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head PASSED [ 50%] 2024-04-22T20:30:01.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head ERROR [ 50%] 2024-04-22T20:30:01.905 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist PASSED [ 50%] 2024-04-22T20:30:01.905 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist ERROR [ 50%] 2024-04-22T20:30:02.711 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended PASSED [ 50%] 2024-04-22T20:30:02.712 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended ERROR [ 50%] 2024-04-22T20:30:03.306 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl PASSED [ 50%] 2024-04-22T20:30:03.307 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl ERROR [ 50%] 2024-04-22T20:30:03.915 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl PASSED [ 51%] 2024-04-22T20:30:03.915 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl ERROR [ 51%] 2024-04-22T20:30:07.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime PASSED [ 51%] 2024-04-22T20:30:07.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime ERROR [ 51%] 2024-04-22T20:30:07.673 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated PASSED [ 51%] 2024-04-22T20:30:07.673 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated ERROR [ 51%] 2024-04-22T20:30:08.271 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers PASSED [ 51%] 2024-04-22T20:30:08.271 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers ERROR [ 51%] 2024-04-22T20:30:08.913 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl PASSED [ 51%] 2024-04-22T20:30:08.914 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl ERROR [ 51%] 2024-04-22T20:30:09.506 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl PASSED [ 51%] 2024-04-22T20:30:09.506 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl ERROR [ 51%] 2024-04-22T20:30:10.042 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone PASSED [ 51%] 2024-04-22T20:30:10.045 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone ERROR [ 51%] 2024-04-22T20:30:10.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone PASSED [ 52%] 2024-04-22T20:30:10.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone ERROR [ 52%] 2024-04-22T20:30:11.418 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired PASSED [ 52%] 2024-04-22T20:30:11.419 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired ERROR [ 52%] 2024-04-22T20:30:11.984 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant PASSED [ 52%] 2024-04-22T20:30:11.984 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant ERROR [ 52%] 2024-04-22T20:30:12.618 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero PASSED [ 52%] 2024-04-22T20:30:12.618 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero ERROR [ 52%] 2024-04-22T20:30:13.234 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range PASSED [ 52%] 2024-04-22T20:30:13.234 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range ERROR [ 52%] 2024-04-22T20:30:13.861 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range PASSED [ 52%] 2024-04-22T20:30:13.861 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range ERROR [ 52%] 2024-04-22T20:30:14.571 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put PASSED [ 52%] 2024-04-22T20:30:14.571 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put ERROR [ 52%] 2024-04-22T20:30:15.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access PASSED [ 53%] 2024-04-22T20:30:15.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access ERROR [ 53%] 2024-04-22T20:30:15.827 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated PASSED [ 53%] 2024-04-22T20:30:15.827 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated ERROR [ 53%] 2024-04-22T20:30:16.580 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl PASSED [ 53%] 2024-04-22T20:30:16.581 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl ERROR [ 53%] 2024-04-22T20:30:17.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant PASSED [ 53%] 2024-04-22T20:30:17.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant ERROR [ 53%] 2024-04-22T20:30:17.812 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired PASSED [ 53%] 2024-04-22T20:30:17.812 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired ERROR [ 53%] 2024-04-22T20:30:18.122 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha PASSED [ 53%] 2024-04-22T20:30:18.123 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha ERROR [ 53%] 2024-04-22T20:30:18.419 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one PASSED [ 53%] 2024-04-22T20:30:18.420 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one ERROR [ 53%] 2024-04-22T20:30:18.767 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two PASSED [ 53%] 2024-04-22T20:30:18.768 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two ERROR [ 53%] 2024-04-22T20:30:19.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 PASSED [ 54%] 2024-04-22T20:30:19.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 ERROR [ 54%] 2024-04-22T20:30:19.598 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 PASSED [ 54%] 2024-04-22T20:30:19.598 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 ERROR [ 54%] 2024-04-22T20:30:20.007 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 PASSED [ 54%] 2024-04-22T20:30:20.007 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 ERROR [ 54%] 2024-04-22T20:30:20.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 PASSED [ 54%] 2024-04-22T20:30:20.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 ERROR [ 54%] 2024-04-22T20:30:20.951 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name PASSED [ 54%] 2024-04-22T20:30:20.951 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name ERROR [ 54%] 2024-04-22T20:30:21.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip PASSED [ 54%] 2024-04-22T20:30:21.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip ERROR [ 54%] 2024-04-22T20:30:21.541 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore PASSED [ 54%] 2024-04-22T20:30:21.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore ERROR [ 54%] 2024-04-22T20:30:22.006 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long PASSED [ 55%] 2024-04-22T20:30:22.007 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long ERROR [ 55%] 2024-04-22T20:30:22.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end PASSED [ 55%] 2024-04-22T20:30:22.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end ERROR [ 55%] 2024-04-22T20:30:22.598 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot PASSED [ 55%] 2024-04-22T20:30:22.598 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot ERROR [ 55%] 2024-04-22T20:30:22.894 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash PASSED [ 55%] 2024-04-22T20:30:22.894 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash ERROR [ 55%] 2024-04-22T20:30:23.187 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot PASSED [ 55%] 2024-04-22T20:30:23.188 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot ERROR [ 55%] 2024-04-22T20:30:23.693 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists PASSED [ 55%] 2024-04-22T20:30:23.693 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists ERROR [ 55%] 2024-04-22T20:30:24.173 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location PASSED [ 55%] 2024-04-22T20:30:24.173 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location ERROR [ 55%] 2024-04-22T20:30:24.691 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner PASSED [ 56%] 2024-04-22T20:30:24.691 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner ERROR [ 56%] 2024-04-22T20:30:25.167 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl PASSED [ 56%] 2024-04-22T20:30:25.167 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl ERROR [ 56%] 2024-04-22T20:30:25.686 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl PASSED [ 56%] 2024-04-22T20:30:25.686 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl ERROR [ 56%] 2024-04-22T20:30:26.187 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default PASSED [ 56%] 2024-04-22T20:30:26.187 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default ERROR [ 56%] 2024-04-22T20:30:26.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create PASSED [ 56%] 2024-04-22T20:30:26.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create ERROR [ 56%] 2024-04-22T20:30:27.212 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned PASSED [ 56%] 2024-04-22T20:30:27.212 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned ERROR [ 56%] 2024-04-22T20:30:27.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite PASSED [ 56%] 2024-04-22T20:30:27.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite ERROR [ 56%] 2024-04-22T20:30:28.198 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread PASSED [ 56%] 2024-04-22T20:30:28.198 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread ERROR [ 56%] 2024-04-22T20:30:28.848 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default PASSED [ 57%] 2024-04-22T20:30:28.848 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default ERROR [ 57%] 2024-04-22T20:30:29.487 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create PASSED [ 57%] 2024-04-22T20:30:29.487 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create ERROR [ 57%] 2024-04-22T20:30:30.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned PASSED [ 57%] 2024-04-22T20:30:30.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned ERROR [ 57%] 2024-04-22T20:30:30.922 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite PASSED [ 57%] 2024-04-22T20:30:30.922 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite ERROR [ 57%] 2024-04-22T20:30:31.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread PASSED [ 57%] 2024-04-22T20:30:31.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread ERROR [ 57%] 2024-04-22T20:30:32.295 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread PASSED [ 57%] 2024-04-22T20:30:32.295 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread ERROR [ 57%] 2024-04-22T20:30:33.041 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol PASSED [ 57%] 2024-04-22T20:30:33.041 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol ERROR [ 57%] 2024-04-22T20:30:33.751 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner PASSED [ 58%] 2024-04-22T20:30:33.751 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner ERROR [ 58%] 2024-04-22T20:30:34.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes PASSED [ 58%] 2024-04-22T20:30:34.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes ERROR [ 58%] 2024-04-22T20:30:35.006 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private PASSED [ 58%] 2024-04-22T20:30:35.006 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private ERROR [ 58%] 2024-04-22T20:30:35.791 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl PASSED [ 58%] 2024-04-22T20:30:35.792 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl ERROR [ 58%] 2024-04-22T20:30:36.532 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write PASSED [ 58%] 2024-04-22T20:30:36.532 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write ERROR [ 58%] 2024-04-22T20:30:37.288 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp PASSED [ 58%] 2024-04-22T20:30:37.288 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp ERROR [ 58%] 2024-04-22T20:30:38.025 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read PASSED [ 58%] 2024-04-22T20:30:38.026 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read ERROR [ 58%] 2024-04-22T20:30:38.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp PASSED [ 59%] 2024-04-22T20:30:38.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp ERROR [ 59%] 2024-04-22T20:30:40.018 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol PASSED [ 59%] 2024-04-22T20:30:40.018 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol ERROR [ 59%] 2024-04-22T20:30:41.040 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read PASSED [ 59%] 2024-04-22T20:30:41.041 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read ERROR [ 59%] 2024-04-22T20:30:42.120 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp PASSED [ 59%] 2024-04-22T20:30:42.120 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp ERROR [ 59%] 2024-04-22T20:30:43.236 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write PASSED [ 59%] 2024-04-22T20:30:43.236 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write ERROR [ 59%] 2024-04-22T20:30:44.267 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp PASSED [ 59%] 2024-04-22T20:30:44.267 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp ERROR [ 59%] 2024-04-22T20:30:44.916 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user PASSED [ 59%] 2024-04-22T20:30:44.916 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user ERROR [ 59%] 2024-04-22T20:30:45.553 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants PASSED [ 59%] 2024-04-22T20:30:45.553 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants ERROR [ 59%] 2024-04-22T20:30:46.221 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants PASSED [ 60%] 2024-04-22T20:30:46.221 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants ERROR [ 60%] 2024-04-22T20:30:46.878 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email PASSED [ 60%] 2024-04-22T20:30:46.878 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email ERROR [ 60%] 2024-04-22T20:30:47.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist PASSED [ 60%] 2024-04-22T20:30:47.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist ERROR [ 60%] 2024-04-22T20:30:48.295 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all PASSED [ 60%] 2024-04-22T20:30:48.295 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all ERROR [ 60%] 2024-04-22T20:30:49.434 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private PASSED [ 60%] 2024-04-22T20:30:49.434 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private ERROR [ 60%] 2024-04-22T20:30:50.562 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private PASSED [ 60%] 2024-04-22T20:30:50.562 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private ERROR [ 60%] 2024-04-22T20:30:51.734 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread PASSED [ 60%] 2024-04-22T20:30:51.734 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread ERROR [ 60%] 2024-04-22T20:30:52.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread PASSED [ 61%] 2024-04-22T20:30:52.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread ERROR [ 61%] 2024-04-22T20:30:53.985 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite PASSED [ 61%] 2024-04-22T20:30:53.986 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite ERROR [ 61%] 2024-04-22T20:30:55.161 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite PASSED [ 61%] 2024-04-22T20:30:55.161 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite ERROR [ 61%] 2024-04-22T20:30:56.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private PASSED [ 61%] 2024-04-22T20:30:56.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private ERROR [ 61%] 2024-04-22T20:30:57.435 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread PASSED [ 61%] 2024-04-22T20:30:57.436 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread ERROR [ 61%] 2024-04-22T20:30:58.608 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite PASSED [ 61%] 2024-04-22T20:30:58.608 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite ERROR [ 61%] 2024-04-22T20:30:59.621 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private PASSED [ 61%] 2024-04-22T20:30:59.621 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private ERROR [ 61%] 2024-04-22T20:31:00.626 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread PASSED [ 62%] 2024-04-22T20:31:00.626 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread ERROR [ 62%] 2024-04-22T20:31:02.087 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite PASSED [ 62%] 2024-04-22T20:31:02.087 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite ERROR [ 62%] 2024-04-22T20:31:03.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list PASSED [ 62%] 2024-04-22T20:31:03.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list ERROR [ 62%] 2024-04-22T20:31:04.430 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime PASSED [ 62%] 2024-04-22T20:31:04.430 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime ERROR [ 62%] 2024-04-22T20:31:04.721 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous PASSED [ 62%] 2024-04-22T20:31:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous ERROR [ 62%] 2024-04-22T20:31:05.075 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth PASSED [ 62%] 2024-04-22T20:31:05.075 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth ERROR [ 62%] 2024-04-22T20:31:05.367 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth PASSED [ 62%] 2024-04-22T20:31:05.367 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth ERROR [ 62%] 2024-04-22T20:31:06.019 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha PASSED [ 62%] 2024-04-22T20:31:06.019 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha ERROR [ 62%] 2024-04-22T20:31:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit PASSED [ 63%] 2024-04-22T20:31:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit ERROR [ 63%] 2024-04-22T20:31:07.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period PASSED [ 63%] 2024-04-22T20:31:07.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period ERROR [ 63%] 2024-04-22T20:31:07.504 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen PASSED [ 63%] 2024-04-22T20:31:07.504 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen ERROR [ 63%] 2024-04-22T20:31:08.327 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding PASSED [ 63%] 2024-04-22T20:31:08.327 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding ERROR [ 63%] 2024-04-22T20:31:10.624 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names PASSED [ 63%] 2024-04-22T20:31:10.624 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names ERROR [ 63%] 2024-04-22T20:31:11.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix PASSED [ 63%] 2024-04-22T20:31:11.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix ERROR [ 63%] 2024-04-22T20:31:12.270 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size PASSED [ 63%] 2024-04-22T20:31:12.270 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size ERROR [ 63%] 2024-04-22T20:31:15.679 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m PASSED [ 64%] 2024-04-22T20:31:15.679 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m ERROR [ 64%] 2024-04-22T20:31:16.430 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket PASSED [ 64%] 2024-04-22T20:31:16.430 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket ERROR [ 64%] 2024-04-22T20:31:17.228 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype PASSED [ 64%] 2024-04-22T20:31:17.228 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype ERROR [ 64%] 2024-04-22T20:31:17.922 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself PASSED [ 64%] 2024-04-22T20:31:17.923 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself ERROR [ 64%] 2024-04-22T20:31:18.617 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata PASSED [ 64%] 2024-04-22T20:31:18.617 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata ERROR [ 64%] 2024-04-22T20:31:19.688 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket PASSED [ 64%] 2024-04-22T20:31:19.688 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket ERROR [ 64%] 2024-04-22T20:31:20.318 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket PASSED [ 64%] 2024-04-22T20:31:20.318 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket ERROR [ 64%] 2024-04-22T20:31:21.348 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket PASSED [ 65%] 2024-04-22T20:31:21.348 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket ERROR [ 65%] 2024-04-22T20:31:22.235 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl PASSED [ 65%] 2024-04-22T20:31:22.235 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl ERROR [ 65%] 2024-04-22T20:31:23.745 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata PASSED [ 65%] 2024-04-22T20:31:23.745 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata ERROR [ 65%] 2024-04-22T20:31:25.064 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata PASSED [ 65%] 2024-04-22T20:31:25.064 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata ERROR [ 65%] 2024-04-22T20:31:25.573 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found PASSED [ 65%] 2024-04-22T20:31:25.573 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found ERROR [ 65%] 2024-04-22T20:31:26.083 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found PASSED [ 65%] 2024-04-22T20:31:26.083 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found ERROR [ 65%] 2024-04-22T20:31:28.398 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket PASSED [ 65%] 2024-04-22T20:31:28.399 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket ERROR [ 65%] 2024-04-22T20:31:29.254 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding PASSED [ 65%] 2024-04-22T20:31:29.254 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding ERROR [ 65%] 2024-04-22T20:31:53.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload PASSED [ 66%] 2024-04-22T20:31:53.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload ERROR [ 66%] 2024-04-22T20:31:53.990 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty PASSED [ 66%] 2024-04-22T20:31:53.990 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty ERROR [ 66%] 2024-04-22T20:31:54.814 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small PASSED [ 66%] 2024-04-22T20:31:54.814 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small ERROR [ 66%] 2024-04-22T20:31:57.713 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small PASSED [ 66%] 2024-04-22T20:31:57.713 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small ERROR [ 66%] 2024-04-22T20:31:58.551 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range PASSED [ 66%] 2024-04-22T20:31:58.551 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range ERROR [ 66%] 2024-04-22T20:31:59.953 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range PASSED [ 66%] 2024-04-22T20:31:59.953 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range ERROR [ 66%] 2024-04-22T20:32:10.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names PASSED [ 66%] 2024-04-22T20:32:10.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names ERROR [ 66%] 2024-04-22T20:32:29.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload PASSED [ 67%] 2024-04-22T20:32:29.416 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload ERROR [ 67%] 2024-04-22T20:32:56.553 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned PASSED [ 67%] 2024-04-22T20:32:56.553 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned ERROR [ 67%] 2024-04-22T20:34:50.864 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part PASSED [ 67%] 2024-04-22T20:34:50.864 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part ERROR [ 67%] 2024-04-22T20:35:02.042 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes PASSED [ 67%] 2024-04-22T20:35:02.043 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes ERROR [ 67%] 2024-04-22T20:35:23.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes PASSED [ 67%] 2024-04-22T20:35:23.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes ERROR [ 67%] 2024-04-22T20:35:24.769 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small PASSED [ 67%] 2024-04-22T20:35:24.769 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small ERROR [ 67%] 2024-04-22T20:35:30.880 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents PASSED [ 67%] 2024-04-22T20:35:30.880 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents ERROR [ 67%] 2024-04-22T20:35:35.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object PASSED [ 68%] 2024-04-22T20:35:35.584 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object ERROR [ 68%] 2024-04-22T20:35:38.980 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload PASSED [ 68%] 2024-04-22T20:35:38.980 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload ERROR [ 68%] 2024-04-22T20:35:39.625 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found PASSED [ 68%] 2024-04-22T20:35:39.625 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found ERROR [ 68%] 2024-04-22T20:35:44.398 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload PASSED [ 68%] 2024-04-22T20:35:44.398 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload ERROR [ 68%] 2024-04-22T20:35:45.295 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner PASSED [ 68%] 2024-04-22T20:35:45.295 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner ERROR [ 68%] 2024-04-22T20:35:45.926 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part PASSED [ 68%] 2024-04-22T20:35:45.926 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part ERROR [ 68%] 2024-04-22T20:35:46.563 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag PASSED [ 68%] 2024-04-22T20:35:46.563 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag ERROR [ 68%] 2024-04-22T20:36:08.178 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part PASSED [ 68%] 2024-04-22T20:36:08.178 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part ERROR [ 68%] 2024-04-22T20:36:08.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part PASSED [ 69%] 2024-04-22T20:36:08.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part ERROR [ 69%] 2024-04-22T20:36:09.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue PASSED [ 69%] 2024-04-22T20:36:09.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue ERROR [ 69%] 2024-04-22T20:36:10.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors PASSED [ 69%] 2024-04-22T20:36:10.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors ERROR [ 69%] 2024-04-22T20:36:17.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response PASSED [ 69%] 2024-04-22T20:36:17.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response ERROR [ 69%] 2024-04-22T20:36:21.456 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard PASSED [ 69%] 2024-04-22T20:36:21.456 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard ERROR [ 69%] 2024-04-22T20:36:25.114 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option PASSED [ 69%] 2024-04-22T20:36:25.115 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option ERROR [ 69%] 2024-04-22T20:36:25.871 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object PASSED [ 69%] 2024-04-22T20:36:25.871 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object ERROR [ 69%] 2024-04-22T20:36:26.459 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant PASSED [ 70%] 2024-04-22T20:36:26.459 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant ERROR [ 70%] 2024-04-22T20:36:27.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object PASSED [ 70%] 2024-04-22T20:36:27.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object ERROR [ 70%] 2024-04-22T20:36:28.039 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl PASSED [ 70%] 2024-04-22T20:36:28.040 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl ERROR [ 70%] 2024-04-22T20:36:28.621 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant PASSED [ 70%] 2024-04-22T20:36:28.621 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant ERROR [ 70%] 2024-04-22T20:36:29.205 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl PASSED [ 70%] 2024-04-22T20:36:29.205 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl ERROR [ 70%] 2024-04-22T20:36:29.944 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging PASSED [ 70%] 2024-04-22T20:36:29.944 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging ERROR [ 70%] 2024-04-22T20:36:31.426 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb PASSED [ 70%] 2024-04-22T20:36:31.426 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb ERROR [ 70%] 2024-04-22T20:36:34.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb PASSED [ 71%] 2024-04-22T20:36:34.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb ERROR [ 71%] 2024-04-22T20:36:40.934 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb PASSED [ 71%] 2024-04-22T20:36:40.935 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb ERROR [ 71%] 2024-04-22T20:36:42.859 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb PASSED [ 71%] 2024-04-22T20:36:42.859 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb ERROR [ 71%] 2024-04-22T20:36:47.189 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb PASSED [ 71%] 2024-04-22T20:36:47.190 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb ERROR [ 71%] 2024-04-22T20:36:54.634 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb PASSED [ 71%] 2024-04-22T20:36:54.635 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb ERROR [ 71%] 2024-04-22T20:36:56.214 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb PASSED [ 71%] 2024-04-22T20:36:56.215 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb ERROR [ 71%] 2024-04-22T20:37:00.419 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb PASSED [ 71%] 2024-04-22T20:37:00.419 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb ERROR [ 71%] 2024-04-22T20:37:06.572 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb PASSED [ 71%] 2024-04-22T20:37:06.572 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb ERROR [ 71%] 2024-04-22T20:37:08.254 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb PASSED [ 72%] 2024-04-22T20:37:08.254 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb ERROR [ 72%] 2024-04-22T20:37:09.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write PASSED [ 72%] 2024-04-22T20:37:09.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write ERROR [ 72%] 2024-04-22T20:37:10.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last PASSED [ 72%] 2024-04-22T20:37:10.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last ERROR [ 72%] 2024-04-22T20:37:10.931 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code PASSED [ 72%] 2024-04-22T20:37:10.931 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code ERROR [ 72%] 2024-04-22T20:37:17.723 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code PASSED [ 72%] 2024-04-22T20:37:17.723 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code ERROR [ 72%] 2024-04-22T20:37:18.365 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code PASSED [ 72%] 2024-04-22T20:37:18.365 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code ERROR [ 72%] 2024-04-22T20:37:19.013 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code PASSED [ 72%] 2024-04-22T20:37:19.013 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code ERROR [ 72%] 2024-04-22T20:37:19.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range PASSED [ 73%] 2024-04-22T20:37:19.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range ERROR [ 73%] 2024-04-22T20:37:20.350 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object PASSED [ 73%] 2024-04-22T20:37:20.350 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object ERROR [ 73%] 2024-04-22T20:37:21.450 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend PASSED [ 73%] 2024-04-22T20:37:21.450 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend ERROR [ 73%] 2024-04-22T20:37:32.865 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove PASSED [ 73%] 2024-04-22T20:37:32.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove ERROR [ 73%] 2024-04-22T20:37:34.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head PASSED [ 73%] 2024-04-22T20:37:34.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head ERROR [ 73%] 2024-04-22T20:37:35.033 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal PASSED [ 73%] 2024-04-22T20:37:35.033 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal ERROR [ 73%] 2024-04-22T20:37:36.242 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite PASSED [ 73%] 2024-04-22T20:37:36.242 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite ERROR [ 73%] 2024-04-22T20:37:37.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended PASSED [ 74%] 2024-04-22T20:37:37.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended ERROR [ 74%] 2024-04-22T20:37:42.502 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions PASSED [ 74%] 2024-04-22T20:37:42.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions ERROR [ 74%] 2024-04-22T20:37:43.713 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy PASSED [ 74%] 2024-04-22T20:37:43.713 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy ERROR [ 74%] 2024-04-22T20:37:49.601 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all PASSED [ 74%] 2024-04-22T20:37:49.602 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all ERROR [ 74%] 2024-04-22T20:38:11.306 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names PASSED [ 74%] 2024-04-22T20:38:11.307 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names ERROR [ 74%] 2024-04-22T20:38:47.595 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart PASSED [ 74%] 2024-04-22T20:38:47.595 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart ERROR [ 74%] 2024-04-22T20:38:49.953 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker PASSED [ 74%] 2024-04-22T20:38:49.953 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker ERROR [ 74%] 2024-04-22T20:38:52.309 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version PASSED [ 75%] 2024-04-22T20:38:52.309 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version ERROR [ 75%] 2024-04-22T20:38:53.368 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete PASSED [ 75%] 2024-04-22T20:38:53.368 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete ERROR [ 75%] 2024-04-22T20:38:54.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker PASSED [ 75%] 2024-04-22T20:38:54.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker ERROR [ 75%] 2024-04-22T20:38:55.210 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create PASSED [ 75%] 2024-04-22T20:38:55.211 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create ERROR [ 75%] 2024-04-22T20:38:56.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl PASSED [ 75%] 2024-04-22T20:38:56.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl ERROR [ 75%] 2024-04-22T20:38:57.684 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified PASSED [ 75%] 2024-04-22T20:38:57.684 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified ERROR [ 75%] 2024-04-22T20:38:59.933 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove PASSED [ 75%] 2024-04-22T20:38:59.933 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove ERROR [ 75%] 2024-04-22T20:39:00.495 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set PASSED [ 75%] 2024-04-22T20:39:00.495 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set ERROR [ 75%] 2024-04-22T20:39:01.156 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get PASSED [ 76%] 2024-04-22T20:39:01.157 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get ERROR [ 76%] 2024-04-22T20:39:01.726 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id PASSED [ 76%] 2024-04-22T20:39:01.726 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id ERROR [ 76%] 2024-04-22T20:39:02.230 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long PASSED [ 76%] 2024-04-22T20:39:02.230 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long ERROR [ 76%] 2024-04-22T20:39:02.739 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id PASSED [ 76%] 2024-04-22T20:39:02.739 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id ERROR [ 76%] 2024-04-22T20:39:03.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status PASSED [ 76%] 2024-04-22T20:39:03.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status ERROR [ 76%] 2024-04-22T20:39:03.932 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date PASSED [ 76%] 2024-04-22T20:39:03.932 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date ERROR [ 76%] 2024-04-22T20:39:04.439 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date PASSED [ 76%] 2024-04-22T20:39:04.440 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date ERROR [ 76%] 2024-04-22T20:39:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent PASSED [ 77%] 2024-04-22T20:39:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent ERROR [ 77%] 2024-04-22T20:39:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker PASSED [ 77%] 2024-04-22T20:39:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker ERROR [ 77%] 2024-04-22T20:39:06.185 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter PASSED [ 77%] 2024-04-22T20:39:06.185 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter ERROR [ 77%] 2024-04-22T20:39:06.700 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter PASSED [ 77%] 2024-04-22T20:39:06.700 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter ERROR [ 77%] 2024-04-22T20:39:07.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart PASSED [ 77%] 2024-04-22T20:39:07.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart ERROR [ 77%] 2024-04-22T20:39:07.766 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date PASSED [ 77%] 2024-04-22T20:39:07.766 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date ERROR [ 77%] 2024-04-22T20:39:08.417 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b PASSED [ 77%] 2024-04-22T20:39:08.417 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b ERROR [ 77%] 2024-04-22T20:39:09.069 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb PASSED [ 78%] 2024-04-22T20:39:09.069 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb ERROR [ 78%] 2024-04-22T20:39:10.130 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB PASSED [ 78%] 2024-04-22T20:39:10.131 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB ERROR [ 78%] 2024-04-22T20:39:10.861 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b PASSED [ 78%] 2024-04-22T20:39:10.862 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b ERROR [ 78%] 2024-04-22T20:39:11.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head PASSED [ 78%] 2024-04-22T20:39:11.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head ERROR [ 78%] 2024-04-22T20:39:12.170 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present PASSED [ 78%] 2024-04-22T20:39:12.170 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present ERROR [ 78%] 2024-04-22T20:39:12.819 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key PASSED [ 78%] 2024-04-22T20:39:12.819 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key ERROR [ 78%] 2024-04-22T20:39:13.380 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 PASSED [ 78%] 2024-04-22T20:39:13.380 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 ERROR [ 78%] 2024-04-22T20:39:13.891 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 PASSED [ 78%] 2024-04-22T20:39:13.891 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 ERROR [ 78%] 2024-04-22T20:39:14.397 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key PASSED [ 79%] 2024-04-22T20:39:14.398 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key ERROR [ 79%] 2024-04-22T20:39:14.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c PASSED [ 79%] 2024-04-22T20:39:14.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c ERROR [ 79%] 2024-04-22T20:40:01.433 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload PASSED [ 79%] 2024-04-22T20:40:01.433 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload ERROR [ 79%] 2024-04-22T20:40:48.117 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload PASSED [ 79%] 2024-04-22T20:40:48.117 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload ERROR [ 79%] 2024-04-22T20:40:57.544 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download PASSED [ 79%] 2024-04-22T20:40:57.545 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download ERROR [ 79%] 2024-04-22T20:40:58.217 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request PASSED [ 79%] 2024-04-22T20:40:58.217 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request ERROR [ 79%] 2024-04-22T20:40:58.937 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head PASSED [ 79%] 2024-04-22T20:40:58.937 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head ERROR [ 79%] 2024-04-22T20:40:59.586 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present PASSED [ 80%] 2024-04-22T20:40:59.586 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present ERROR [ 80%] 2024-04-22T20:41:00.086 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key PASSED [ 80%] 2024-04-22T20:41:00.086 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key ERROR [ 80%] 2024-04-22T20:41:00.592 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared PASSED [ 80%] 2024-04-22T20:41:00.592 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared ERROR [ 80%] 2024-04-22T20:41:30.481 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload PASSED [ 80%] 2024-04-22T20:41:30.482 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload ERROR [ 80%] 2024-04-22T20:41:39.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 PASSED [ 80%] 2024-04-22T20:41:39.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 ERROR [ 80%] 2024-04-22T20:41:47.537 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 PASSED [ 80%] 2024-04-22T20:41:47.537 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 ERROR [ 80%] 2024-04-22T20:41:48.213 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request PASSED [ 80%] 2024-04-22T20:41:48.213 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request ERROR [ 80%] 2024-04-22T20:41:48.925 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b PASSED [ 81%] 2024-04-22T20:41:48.925 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b ERROR [ 81%] 2024-04-22T20:41:49.582 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb PASSED [ 81%] 2024-04-22T20:41:49.582 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb ERROR [ 81%] 2024-04-22T20:41:50.637 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB PASSED [ 81%] 2024-04-22T20:41:50.637 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB ERROR [ 81%] 2024-04-22T20:41:51.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b PASSED [ 81%] 2024-04-22T20:41:51.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b ERROR [ 81%] 2024-04-22T20:41:51.977 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare PASSED [ 81%] 2024-04-22T20:41:51.978 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare ERROR [ 81%] 2024-04-22T20:41:52.686 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy PASSED [ 81%] 2024-04-22T20:41:52.686 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy ERROR [ 81%] 2024-04-22T20:41:53.403 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy PASSED [ 81%] 2024-04-22T20:41:53.403 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy ERROR [ 81%] 2024-04-22T20:41:54.222 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl PASSED [ 81%] 2024-04-22T20:41:54.223 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl ERROR [ 81%] 2024-04-22T20:41:55.062 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl PASSED [ 82%] 2024-04-22T20:41:55.062 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl ERROR [ 82%] 2024-04-22T20:41:56.272 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket PASSED [ 82%] 2024-04-22T20:41:56.272 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket ERROR [ 82%] 2024-04-22T20:41:57.469 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket PASSED [ 82%] 2024-04-22T20:41:57.470 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket ERROR [ 82%] 2024-04-22T20:42:00.116 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging PASSED [ 82%] 2024-04-22T20:42:00.116 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging ERROR [ 82%] 2024-04-22T20:42:02.100 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging PASSED [ 82%] 2024-04-22T20:42:02.101 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging ERROR [ 82%] 2024-04-22T20:42:04.770 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags PASSED [ 82%] 2024-04-22T20:42:04.771 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags ERROR [ 82%] 2024-04-22T20:42:06.962 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags PASSED [ 82%] 2024-04-22T20:42:06.962 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags ERROR [ 82%] 2024-04-22T20:42:09.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags PASSED [ 83%] 2024-04-22T20:42:09.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags ERROR [ 83%] 2024-04-22T20:42:12.136 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags PASSED [ 83%] 2024-04-22T20:42:12.137 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags ERROR [ 83%] 2024-04-22T20:42:14.729 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags PASSED [ 83%] 2024-04-22T20:42:14.729 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags ERROR [ 83%] 2024-04-22T20:42:16.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags PASSED [ 83%] 2024-04-22T20:42:16.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags ERROR [ 83%] 2024-04-22T20:42:19.049 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags PASSED [ 83%] 2024-04-22T20:42:19.049 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags ERROR [ 83%] 2024-04-22T20:42:19.755 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request PASSED [ 83%] 2024-04-22T20:42:19.755 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request ERROR [ 83%] 2024-04-22T20:42:20.433 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request PASSED [ 83%] 2024-04-22T20:42:20.433 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request ERROR [ 83%] 2024-04-22T20:42:21.202 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags PASSED [ 84%] 2024-04-22T20:42:21.202 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags ERROR [ 84%] 2024-04-22T20:42:23.353 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public PASSED [ 84%] 2024-04-22T20:42:23.353 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public ERROR [ 84%] 2024-04-22T20:42:25.575 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public PASSED [ 84%] 2024-04-22T20:42:25.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public ERROR [ 84%] 2024-04-22T20:42:27.799 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public PASSED [ 84%] 2024-04-22T20:42:27.799 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public ERROR [ 84%] 2024-04-22T20:42:29.381 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id PASSED [ 84%] 2024-04-22T20:42:29.381 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id ERROR [ 84%] 2024-04-22T20:42:51.607 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id PASSED [ 84%] 2024-04-22T20:42:51.607 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id ERROR [ 84%] 2024-04-22T20:42:52.643 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag PASSED [ 84%] 2024-04-22T20:42:52.643 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag ERROR [ 84%] 2024-04-22T20:42:53.781 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag PASSED [ 84%] 2024-04-22T20:42:53.781 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag ERROR [ 84%] 2024-04-22T20:42:54.808 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag PASSED [ 85%] 2024-04-22T20:42:54.808 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag ERROR [ 85%] 2024-04-22T20:42:56.196 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source PASSED [ 85%] 2024-04-22T20:42:56.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source ERROR [ 85%] 2024-04-22T20:42:57.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta PASSED [ 85%] 2024-04-22T20:42:57.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta ERROR [ 85%] 2024-04-22T20:42:58.171 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl PASSED [ 85%] 2024-04-22T20:42:58.172 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl ERROR [ 85%] 2024-04-22T20:42:59.307 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant PASSED [ 85%] 2024-04-22T20:42:59.307 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant ERROR [ 85%] 2024-04-22T20:42:59.826 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 PASSED [ 85%] 2024-04-22T20:42:59.827 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 ERROR [ 85%] 2024-04-22T20:43:00.404 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms PASSED [ 85%] 2024-04-22T20:43:00.404 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms ERROR [ 85%] 2024-04-22T20:43:00.910 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms PASSED [ 86%] 2024-04-22T20:43:00.910 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms ERROR [ 86%] 2024-04-22T20:43:01.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms PASSED [ 86%] 2024-04-22T20:43:01.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms ERROR [ 86%] 2024-04-22T20:43:02.083 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc PASSED [ 86%] 2024-04-22T20:43:02.083 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc ERROR [ 86%] 2024-04-22T20:43:02.707 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 PASSED [ 86%] 2024-04-22T20:43:02.707 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 ERROR [ 86%] 2024-04-22T20:43:03.795 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag PASSED [ 86%] 2024-04-22T20:43:03.795 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag ERROR [ 86%] 2024-04-22T20:43:04.337 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock PASSED [ 86%] 2024-04-22T20:43:04.337 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock ERROR [ 86%] 2024-04-22T20:43:04.812 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket PASSED [ 86%] 2024-04-22T20:43:04.812 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket ERROR [ 86%] 2024-04-22T20:43:05.345 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years PASSED [ 87%] 2024-04-22T20:43:05.345 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years ERROR [ 87%] 2024-04-22T20:43:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days PASSED [ 87%] 2024-04-22T20:43:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days ERROR [ 87%] 2024-04-22T20:43:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years PASSED [ 87%] 2024-04-22T20:43:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years ERROR [ 87%] 2024-04-22T20:43:06.824 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode PASSED [ 87%] 2024-04-22T20:43:06.824 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode ERROR [ 87%] 2024-04-22T20:43:07.366 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status PASSED [ 87%] 2024-04-22T20:43:07.366 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status ERROR [ 87%] 2024-04-22T20:43:07.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning PASSED [ 87%] 2024-04-22T20:43:07.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning ERROR [ 87%] 2024-04-22T20:43:08.343 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock PASSED [ 87%] 2024-04-22T20:43:08.343 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock ERROR [ 87%] 2024-04-22T20:43:08.812 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket PASSED [ 87%] 2024-04-22T20:43:08.812 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket ERROR [ 87%] 2024-04-22T20:43:09.449 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention PASSED [ 88%] 2024-04-22T20:43:09.449 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention ERROR [ 88%] 2024-04-22T20:43:10.074 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket PASSED [ 88%] 2024-04-22T20:43:10.075 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket ERROR [ 88%] 2024-04-22T20:43:10.749 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode PASSED [ 88%] 2024-04-22T20:43:10.749 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode ERROR [ 88%] 2024-04-22T20:43:11.369 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention PASSED [ 88%] 2024-04-22T20:43:11.369 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention ERROR [ 88%] 2024-04-22T20:43:11.953 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 PASSED [ 88%] 2024-04-22T20:43:11.953 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 ERROR [ 88%] 2024-04-22T20:43:12.584 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket PASSED [ 88%] 2024-04-22T20:43:12.584 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket ERROR [ 88%] 2024-04-22T20:43:13.315 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid PASSED [ 88%] 2024-04-22T20:43:13.315 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid ERROR [ 88%] 2024-04-22T20:43:13.962 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention PASSED [ 89%] 2024-04-22T20:43:13.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention ERROR [ 89%] 2024-04-22T20:43:14.619 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period PASSED [ 89%] 2024-04-22T20:43:14.620 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period ERROR [ 89%] 2024-04-22T20:43:15.340 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period PASSED [ 89%] 2024-04-22T20:43:15.341 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period ERROR [ 89%] 2024-04-22T20:43:16.004 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass PASSED [ 89%] 2024-04-22T20:43:16.004 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass ERROR [ 89%] 2024-04-22T20:43:16.625 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention PASSED [ 89%] 2024-04-22T20:43:16.625 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention ERROR [ 89%] 2024-04-22T20:43:17.376 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention PASSED [ 89%] 2024-04-22T20:43:17.376 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention ERROR [ 89%] 2024-04-22T20:43:18.202 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker PASSED [ 89%] 2024-04-22T20:43:18.202 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker ERROR [ 89%] 2024-04-22T20:43:18.892 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention PASSED [ 90%] 2024-04-22T20:43:18.892 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention ERROR [ 90%] 2024-04-22T20:43:19.502 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold PASSED [ 90%] 2024-04-22T20:43:19.502 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold ERROR [ 90%] 2024-04-22T20:43:20.100 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket PASSED [ 90%] 2024-04-22T20:43:20.100 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket ERROR [ 90%] 2024-04-22T20:43:20.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status PASSED [ 90%] 2024-04-22T20:43:20.690 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status ERROR [ 90%] 2024-04-22T20:43:21.484 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold PASSED [ 90%] 2024-04-22T20:43:21.484 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold ERROR [ 90%] 2024-04-22T20:43:22.050 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket PASSED [ 90%] 2024-04-22T20:43:22.050 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket ERROR [ 90%] 2024-04-22T20:43:22.718 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on PASSED [ 90%] 2024-04-22T20:43:22.718 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on ERROR [ 90%] 2024-04-22T20:43:23.512 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on PASSED [ 90%] 2024-04-22T20:43:23.512 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on ERROR [ 90%] 2024-04-22T20:43:24.120 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off PASSED [ 91%] 2024-04-22T20:43:24.120 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off ERROR [ 91%] 2024-04-22T20:43:24.779 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata PASSED [ 91%] 2024-04-22T20:43:24.779 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata ERROR [ 91%] 2024-04-22T20:43:25.373 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj PASSED [ 91%] 2024-04-22T20:43:25.373 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj ERROR [ 91%] 2024-04-22T20:43:35.685 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass PASSED [ 91%] 2024-04-22T20:43:35.685 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass ERROR [ 91%] 2024-04-22T20:43:36.342 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass PASSED [ 91%] 2024-04-22T20:43:36.342 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass ERROR [ 91%] 2024-04-22T20:43:46.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance PASSED [ 91%] 2024-04-22T20:43:46.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance ERROR [ 91%] 2024-04-22T20:43:46.942 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good PASSED [ 91%] 2024-04-22T20:43:46.942 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good ERROR [ 91%] 2024-04-22T20:43:47.664 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed PASSED [ 92%] 2024-04-22T20:43:47.665 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed ERROR [ 92%] 2024-04-22T20:43:48.226 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status PASSED [ 92%] 2024-04-22T20:43:48.226 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status ERROR [ 92%] 2024-04-22T20:43:48.789 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status PASSED [ 92%] 2024-04-22T20:43:48.790 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status ERROR [ 92%] 2024-04-22T20:43:49.365 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status PASSED [ 92%] 2024-04-22T20:43:49.365 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status ERROR [ 92%] 2024-04-22T20:43:49.990 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-04-22T20:43:49.990 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status ERROR [ 92%] 2024-04-22T20:43:50.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-04-22T20:43:50.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status ERROR [ 92%] 2024-04-22T20:43:51.241 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status PASSED [ 92%] 2024-04-22T20:43:51.242 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status ERROR [ 92%] 2024-04-22T20:43:51.809 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block PASSED [ 93%] 2024-04-22T20:43:51.809 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block ERROR [ 93%] 2024-04-22T20:43:52.468 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy PASSED [ 93%] 2024-04-22T20:43:52.468 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy ERROR [ 93%] 2024-04-22T20:43:53.090 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block PASSED [ 93%] 2024-04-22T20:43:53.090 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block ERROR [ 93%] 2024-04-22T20:43:53.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls PASSED [ 93%] 2024-04-22T20:43:53.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls ERROR [ 93%] 2024-04-22T20:43:54.501 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls PASSED [ 93%] 2024-04-22T20:43:54.501 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls ERROR [ 93%] 2024-04-22T20:43:55.064 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy PASSED [ 93%] 2024-04-22T20:43:55.064 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy ERROR [ 93%] 2024-04-22T20:43:56.041 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls PASSED [ 93%] 2024-04-22T20:43:56.041 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls ERROR [ 93%] 2024-04-22T20:44:07.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy PASSED [ 93%] 2024-04-22T20:44:07.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy ERROR [ 93%] 2024-04-22T20:44:07.700 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms PASSED [ 94%] 2024-04-22T20:44:07.701 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms ERROR [ 94%] 2024-04-22T20:44:08.298 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms PASSED [ 94%] 2024-04-22T20:44:08.299 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms ERROR [ 94%] 2024-04-22T20:44:08.972 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms PASSED [ 94%] 2024-04-22T20:44:08.973 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms ERROR [ 94%] 2024-04-22T20:44:09.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request PASSED [ 94%] 2024-04-22T20:44:09.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request ERROR [ 94%] 2024-04-22T20:44:11.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent PASSED [ 94%] 2024-04-22T20:44:11.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent ERROR [ 94%] 2024-04-22T20:44:22.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_where_clause PASSED [ 94%] 2024-04-22T20:44:32.386 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_projection PASSED [ 94%] 2024-04-22T20:44:33.233 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_count_operation PASSED [ 95%] 2024-04-22T20:44:34.449 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_count_json_operation PASSED [ 95%] 2024-04-22T20:44:48.116 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_json_column_sum_min_max PASSED [ 95%] 2024-04-22T20:45:05.380 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_json_nullif_expressions PASSED [ 95%] 2024-04-22T20:45:09.452 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_column_sum_min_max PASSED [ 95%] 2024-04-22T20:45:11.756 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_nullif_expressions PASSED [ 95%] 2024-04-22T20:45:14.058 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_nulliftrue_expressions PASSED [ 95%] 2024-04-22T20:45:16.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_is_not_null_expressions PASSED [ 96%] 2024-04-22T20:45:16.973 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_lowerupper_expressions PASSED [ 96%] 2024-04-22T20:45:21.175 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_in_expressions PASSED [ 96%] 2024-04-22T20:45:26.112 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_true_false_in_expressions PASSED [ 96%] 2024-04-22T20:46:23.580 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_like_expressions PASSED [ 96%] 2024-04-22T20:46:26.960 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_truefalselike_expressions PASSED [ 96%] 2024-04-22T20:46:29.162 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_complex_expressions PASSED [ 96%] 2024-04-22T20:46:31.191 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_alias PASSED [ 96%] 2024-04-22T20:46:32.209 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_alias_cyclic_refernce PASSED [ 97%] 2024-04-22T20:46:37.677 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_datetime PASSED [ 97%] 2024-04-22T20:46:40.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_true_false_datetime PASSED [ 97%] 2024-04-22T20:46:42.371 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_csv_parser PASSED [ 97%] 2024-04-22T20:46:43.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_csv_definition PASSED [ 97%] 2024-04-22T20:46:49.268 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_schema_definition PASSED [ 97%] 2024-04-22T20:46:52.833 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_when_then_else_expressions PASSED [ 97%] 2024-04-22T20:46:57.311 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_coalesce_expressions PASSED [ 98%] 2024-04-22T20:47:00.463 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_cast_expressions PASSED [ 98%] 2024-04-22T20:47:00.933 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_version PASSED [ 98%] 2024-04-22T20:47:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_trim_expressions PASSED [ 98%] 2024-04-22T20:47:09.163 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_truefalse_trim_expressions PASSED [ 98%] 2024-04-22T20:47:11.772 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_escape_expressions PASSED [ 98%] 2024-04-22T20:47:16.212 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_case_value_expressions PASSED [ 98%] 2024-04-22T20:47:17.677 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_bool_cast_expressions PASSED [ 99%] 2024-04-22T20:47:59.746 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_progress_expressions PASSED [ 99%] 2024-04-22T20:48:00.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py::test_output_serial_expressions PASSED [ 99%] 2024-04-22T20:48:01.481 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic PASSED [ 99%] 2024-04-22T20:48:02.471 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_sns.py::test_cross_account_topic PASSED [ 99%] 2024-04-22T20:48:03.349 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic_publish PASSED [ 99%] 2024-04-22T20:48:04.446 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_sns.py::test_cross_account_topic_publish PASSED [ 99%] 2024-04-22T20:48:04.472 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_utils.py::test_generate PASSED [100%] 2024-04-22T20:48:04.472 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.472 INFO:teuthology.orchestra.run.smithi136.stdout:==================================== ERRORS ==================================== 2024-04-22T20:48:04.472 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_object_write_to_nonexist_bucket ___________ 2024-04-22T20:48:04.472 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.473 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.474 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.475 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.476 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.477 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.478 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_object_write_with_chunked_transfer_encoding _____ 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.479 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.480 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.481 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.482 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.483 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.484 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.485 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_bucket_create_delete ________________ 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.486 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.487 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.487 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.487 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.487 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.492 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.493 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.495 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.496 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.497 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_object_read_not_exist ________________ 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.498 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.499 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.500 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.501 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.502 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.503 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_requestid_matches_header_on_error ______ 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.504 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.505 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.506 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.507 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.508 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.509 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.510 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_versioning_concurrent_multi_object_delete ______ 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.511 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.512 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.513 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.514 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.516 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_multi_object_delete _________________ 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.519 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.521 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.522 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.523 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_multi_objectv2_delete ________________ 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.524 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.525 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.527 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.528 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.529 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_multi_object_delete_key_limit ____________ 2024-04-22T20:48:04.530 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.531 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.532 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.533 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.534 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.535 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.536 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_multi_objectv2_delete_key_limit ___________ 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.537 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.538 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.540 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.541 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.542 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_object_head_zero_bytes _______________ 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.543 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.544 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.545 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.546 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.547 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.548 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.549 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_object_write_check_etag _______________ 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.550 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.551 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.553 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.554 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.555 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_write_cache_control _____________ 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.556 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.558 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.559 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.560 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.561 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.562 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_object_write_expires ________________ 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.563 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.564 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.567 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.568 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_write_read_update_read_delete ________ 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.569 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.570 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.571 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.573 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.574 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.575 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_set_get_metadata_none_to_good ________ 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.576 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.577 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.578 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.580 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.581 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.582 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_object_set_get_metadata_none_to_empty ________ 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.583 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.584 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.585 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.586 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.587 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.588 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_object_set_get_metadata_overwrite_to_empty _____ 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.589 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.590 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.591 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.592 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.593 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.594 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.595 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_object_metadata_replaced_on_put ___________ 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.597 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.599 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.600 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.601 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_object_write_file __________________ 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.602 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.603 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.606 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_post_object_anonymous_request ____________ 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.609 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.610 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.611 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.612 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.613 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.614 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_post_object_authenticated_request __________ 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.615 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.617 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.619 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.620 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.621 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_post_object_authenticated_no_content_type ______ 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.622 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.623 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.624 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.625 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.626 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.627 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout:__ ERROR at teardown of test_post_object_authenticated_request_bad_access_key __ 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.628 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.630 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.632 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.633 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.634 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_post_object_set_success_code ____________ 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.635 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.636 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.637 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.640 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_post_object_set_invalid_success_code ________ 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.641 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.642 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.645 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.646 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.647 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_post_object_upload_larger_than_chunk ________ 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.648 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.650 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.652 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.653 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.654 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_post_object_set_key_from_filename __________ 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.655 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.656 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.657 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.658 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.659 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.660 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.661 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_post_object_ignored_header _____________ 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.663 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.664 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.665 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.666 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.667 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.668 INFO:teuthology.orchestra.run.smithi136.stdout:___ ERROR at teardown of test_post_object_case_insensitive_condition_fields ____ 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.669 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.670 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.671 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.672 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.673 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.674 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_post_object_escaped_field_values __________ 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.679 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.680 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.681 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_post_object_success_redirect_action _________ 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.682 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.683 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.684 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.685 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.686 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.687 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_post_object_invalid_signature ____________ 2024-04-22T20:48:04.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.689 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.690 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.691 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.692 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.693 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.694 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_post_object_invalid_access_key ___________ 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.695 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.697 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.699 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.700 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_post_object_invalid_date_format ___________ 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.702 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.703 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.704 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.705 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.706 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.707 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_post_object_no_key_specified ____________ 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.708 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.709 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.710 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.712 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.713 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_post_object_missing_signature ____________ 2024-04-22T20:48:04.714 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.715 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.716 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.717 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.718 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.719 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.720 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_post_object_missing_policy_condition ________ 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.722 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.724 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.725 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.726 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_post_object_user_specified_header __________ 2024-04-22T20:48:04.727 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.728 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.729 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.730 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.731 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.732 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.733 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_post_object_request_missing_policy_specified_field _ 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.734 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.735 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.736 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.738 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.739 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.740 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_post_object_condition_is_case_sensitive _______ 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.741 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.742 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.743 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.744 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.745 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.746 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_post_object_expires_is_case_sensitive ________ 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.747 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.748 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.749 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.750 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.751 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.752 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.753 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_post_object_expired_policy _____________ 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.754 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.755 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.756 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.757 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.758 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_post_object_wrong_bucket ______________ 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.760 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.761 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.763 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.764 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.765 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_post_object_invalid_request_field_value _______ 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.766 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.767 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.768 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.770 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.771 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.772 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_post_object_missing_expires_condition ________ 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.773 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.774 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.775 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.777 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.779 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_post_object_missing_conditions_list _________ 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.780 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.781 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.782 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.783 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.784 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.785 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_post_object_upload_size_limit_exceeded _______ 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.786 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.787 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.790 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.791 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.792 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_post_object_missing_content_length_argument _____ 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.793 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.794 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.795 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.796 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.797 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.798 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_post_object_invalid_content_length_argument _____ 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.799 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.800 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.801 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.802 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.803 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.804 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.805 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_post_object_upload_size_below_minimum ________ 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.806 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.807 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.809 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.810 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.811 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_post_object_upload_size_rgw_chunk_size_bug _____ 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.812 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.813 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.814 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.815 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.816 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.817 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.818 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_post_object_empty_conditions ____________ 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.819 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.820 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.821 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.822 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.823 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.824 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_get_object_ifmatch_good _______________ 2024-04-22T20:48:04.825 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.826 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.827 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.828 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.829 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.830 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.831 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_get_object_ifmatch_failed ______________ 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.832 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.833 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.834 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.835 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.836 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.837 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.838 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_get_object_ifnonematch_good _____________ 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.839 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.840 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.841 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.843 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.844 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_get_object_ifnonematch_failed ____________ 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.845 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.846 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.847 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.848 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.849 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.850 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.851 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_get_object_ifmodifiedsince_good ___________ 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.852 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.853 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.855 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.856 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.857 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_get_object_ifmodifiedsince_failed __________ 2024-04-22T20:48:04.858 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.859 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.860 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.861 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.862 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.863 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.864 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_get_object_ifunmodifiedsince_good __________ 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.865 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.866 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.867 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.867 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.867 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.867 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.867 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.869 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.869 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.869 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.871 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.872 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.873 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_get_object_ifunmodifiedsince_failed _________ 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.874 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.875 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.876 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.878 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.880 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_put_object_ifmatch_good _______________ 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.881 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.882 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.883 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.884 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.885 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.886 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_put_object_ifmatch_failed ______________ 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.887 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.888 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.889 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.890 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.892 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.893 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_put_object_ifmatch_overwrite_existed_good ______ 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.895 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.898 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.899 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.900 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_put_object_ifmatch_nonexisted_failed ________ 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.901 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.902 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T20:48:04.923 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.923 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_put_object_ifnonmatch_good _____________ 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.929 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.930 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_put_object_ifnonmatch_failed ____________ 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_put_object_ifnonmatch_nonexisted_good ________ 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.948 INFO:teuthology.orchestra.run.smithi136.stdout:___ ERROR at teardown of test_put_object_ifnonmatch_overwrite_existed_failed ___ 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout:___________________ ERROR at teardown of test_object_raw_get ___________________ 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.961 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_raw_get_bucket_gone _____________ 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.963 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_object_delete_key_bucket_gone ____________ 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.974 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_raw_get_object_gone _____________ 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.975 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.981 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout:____________________ ERROR at teardown of test_bucket_head _____________________ 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.982 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_bucket_head_notexist ________________ 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_bucket_head_extended ________________ 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_raw_get_bucket_acl ______________ 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.001 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_raw_get_object_acl ______________ 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.012 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.013 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.014 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_object_put_acl_mtime ________________ 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.015 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.017 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.018 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.019 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.020 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_object_raw_authenticated ______________ 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.022 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.023 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.025 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.026 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.027 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_object_raw_response_headers _____________ 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.028 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.029 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.031 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.032 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.033 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_raw_authenticated_bucket_acl _________ 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.036 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.038 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.039 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.040 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.041 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_raw_authenticated_object_acl _________ 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.042 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.043 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.044 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.045 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.046 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.047 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_raw_authenticated_bucket_gone ________ 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.048 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.049 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.050 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.051 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.052 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.053 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.054 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_raw_authenticated_object_gone ________ 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.055 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.056 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.057 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.058 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.059 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.060 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.061 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_raw_get_x_amz_expires_not_expired ______ 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.062 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.063 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.064 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.065 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.066 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.067 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout:__ ERROR at teardown of test_object_raw_get_x_amz_expires_not_expired_tenant ___ 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.068 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.069 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.070 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.071 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.072 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.073 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.074 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_object_raw_get_x_amz_expires_out_range_zero _____ 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.075 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.077 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.077 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.077 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.077 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.078 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.081 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.082 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.083 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_object_raw_get_x_amz_expires_out_max_range _____ 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.084 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.085 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.086 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.088 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.089 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.090 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout:__ ERROR at teardown of test_object_raw_get_x_amz_expires_out_positive_range ___ 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.091 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.092 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.093 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.094 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.095 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.096 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_object_anon_put ___________________ 2024-04-22T20:48:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.098 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.099 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.100 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.101 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.102 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_object_anon_put_write_access ____________ 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.104 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.105 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.107 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.108 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.109 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_object_put_authenticated ______________ 2024-04-22T20:48:05.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.111 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.112 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.113 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.114 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.115 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.116 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_presigned_put_object_with_acl ________ 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.117 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.118 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.120 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.121 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.122 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_object_presigned_put_object_with_acl_tenant _____ 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.123 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.124 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.125 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.127 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.128 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.129 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_raw_put_authenticated_expired ________ 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.130 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.131 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.132 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.133 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.134 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.135 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.136 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_bucket_create_naming_bad_starts_nonalpha ______ 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.137 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.138 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.139 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.140 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.142 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_create_naming_bad_short_one _________ 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.143 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.144 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.145 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.146 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.147 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.148 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.149 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_create_naming_bad_short_two _________ 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.150 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.151 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.152 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.153 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.154 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.155 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_60 __________ 2024-04-22T20:48:05.156 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.157 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.158 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.159 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.161 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.162 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_61 __________ 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.163 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.164 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.165 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.167 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.168 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.169 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_62 __________ 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.170 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.171 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.172 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.173 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.174 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.175 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_63 __________ 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.176 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.177 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.178 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.179 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.180 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.181 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_bucket_list_long_name ________________ 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.183 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.184 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.185 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.187 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.188 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.189 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_bucket_create_naming_bad_ip _____________ 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.190 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.191 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.192 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.193 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.194 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.195 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_bucket_create_naming_dns_underscore _________ 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.196 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.197 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.198 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.199 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.200 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.201 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.202 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_bucket_create_naming_dns_long ____________ 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.204 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.205 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.206 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.207 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.208 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_bucket_create_naming_dns_dash_at_end ________ 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.209 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.210 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.211 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.212 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.213 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.214 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.215 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_bucket_create_naming_dns_dot_dot __________ 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.216 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.217 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.219 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.220 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.221 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_create_naming_dns_dot_dash __________ 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.222 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.223 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.224 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.225 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.226 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.227 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.228 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_create_naming_dns_dash_dot __________ 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.229 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.230 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.231 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.232 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.233 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.234 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_bucket_create_exists ________________ 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.235 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.236 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.238 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.239 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.240 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.241 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_bucket_get_location _________________ 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.242 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.243 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.244 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.245 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.246 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.247 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_bucket_create_exists_nonowner ____________ 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.248 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.249 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.250 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.251 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.253 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.254 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.255 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_bucket_recreate_overwrite_acl ____________ 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.256 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.257 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.258 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.259 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.260 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.261 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.262 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_bucket_recreate_new_acl _______________ 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.263 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.264 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.265 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.266 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.267 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.268 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_bucket_acl_default _________________ 2024-04-22T20:48:05.269 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.270 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.271 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.272 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.273 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.274 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.275 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_bucket_acl_canned_during_create ___________ 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.276 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.277 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.279 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.280 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.281 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_bucket_acl_canned __________________ 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.282 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.283 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.284 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.285 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.286 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.287 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.288 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_acl_canned_publicreadwrite __________ 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.289 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.290 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.291 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.292 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.293 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.294 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.295 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_bucket_acl_canned_authenticatedread _________ 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.296 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.297 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.299 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.300 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.301 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_object_acl_default _________________ 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.302 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.303 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.304 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.306 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.308 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_object_acl_canned_during_create ___________ 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.309 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.310 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.311 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.312 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.313 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.314 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_object_acl_canned __________________ 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.315 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.316 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.317 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.318 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.319 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.320 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.321 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_object_acl_canned_publicreadwrite __________ 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.322 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.323 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.325 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.326 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.327 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_acl_canned_authenticatedread _________ 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.328 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.329 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.330 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.331 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.332 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.333 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_object_acl_canned_bucketownerread __________ 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.334 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.335 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.336 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.337 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.338 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.339 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.340 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_acl_canned_bucketownerfullcontrol ______ 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.341 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.342 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.345 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.346 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.347 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_acl_full_control_verify_owner ________ 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.348 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.349 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.350 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.351 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.352 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.353 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.354 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_object_acl_full_control_verify_attributes ______ 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.355 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.356 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.357 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.358 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.359 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.360 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_bucket_acl_canned_private_to_private ________ 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.361 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.362 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.363 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.364 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.365 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.366 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.367 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout:_____________________ ERROR at teardown of test_object_acl _____________________ 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.368 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.369 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.370 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.371 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.372 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.373 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_object_acl_write __________________ 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.374 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.375 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.376 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.377 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.378 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.379 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.380 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_object_acl_writeacp _________________ 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.381 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.382 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.385 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.386 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.387 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_object_acl_read ___________________ 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.388 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.389 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.390 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.391 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.393 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_object_acl_readacp _________________ 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.394 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.395 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.396 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.397 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.398 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.399 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.400 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_bucket_acl_grant_userid_fullcontrol _________ 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.401 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.402 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.403 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.404 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.405 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.406 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_bucket_acl_grant_userid_read ____________ 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.407 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.408 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.409 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.410 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.411 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.412 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.413 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_bucket_acl_grant_userid_readacp ___________ 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.414 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.415 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.416 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.417 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.418 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.419 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.420 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_bucket_acl_grant_userid_write ____________ 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.421 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.422 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.423 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.424 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.425 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.426 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_bucket_acl_grant_userid_writeacp __________ 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.427 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.428 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.429 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.430 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.431 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.432 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.433 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_bucket_acl_grant_nonexist_user ___________ 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.434 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.435 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.436 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.437 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.438 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.439 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_object_header_acl_grants ______________ 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.440 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.441 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.442 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.443 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.445 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.446 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_bucket_header_acl_grants ______________ 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.447 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.448 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.449 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.450 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.451 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.452 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_bucket_acl_grant_email _______________ 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.453 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.454 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.455 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.456 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.457 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.458 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.459 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_bucket_acl_grant_email_not_exist __________ 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.460 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.461 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.462 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_bucket_acl_revoke_all ________________ 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.466 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_access_bucket_private_object_private ________ 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.473 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.474 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_access_bucket_private_objectv2_private _______ 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.480 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.481 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_access_bucket_private_object_publicread _______ 2024-04-22T20:48:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.489 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_access_bucket_private_objectv2_publicread ______ 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_access_bucket_private_object_publicreadwrite ____ 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.502 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.508 INFO:teuthology.orchestra.run.smithi136.stdout:___ ERROR at teardown of test_access_bucket_private_objectv2_publicreadwrite ___ 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.509 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.510 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_access_bucket_publicread_object_private _______ 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.516 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.521 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_access_bucket_publicread_object_publicread _____ 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.522 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.523 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout:__ ERROR at teardown of test_access_bucket_publicread_object_publicreadwrite ___ 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_access_bucket_publicreadwrite_object_private ____ 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout:__ ERROR at teardown of test_access_bucket_publicreadwrite_object_publicread ___ 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.541 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_access_bucket_publicreadwrite_object_publicreadwrite _ 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.548 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.549 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.550 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.551 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.552 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.553 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_buckets_create_then_list ______________ 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.554 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.555 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.557 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.558 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.559 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_buckets_list_ctime _________________ 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.560 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.561 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.562 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.563 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.564 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.565 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.566 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_list_buckets_anonymous _______________ 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.567 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.568 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.569 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.570 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.571 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.572 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_list_buckets_invalid_auth ______________ 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.573 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.574 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.576 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.577 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.578 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_list_buckets_bad_auth ________________ 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.579 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_bucket_create_naming_good_starts_alpha _______ 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.587 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:---------------------------- Captured stdout setup ----------------------------- 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:Done with cleanup of buckets in tests. 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:Done with cleanup of buckets in tests. 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_bucket_create_naming_good_starts_digit _______ 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.592 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:---------------------------- Captured stdout setup ----------------------------- 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:Done with cleanup of buckets in tests. 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:Done with cleanup of buckets in tests. 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_bucket_create_naming_good_contains_period ______ 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.605 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_bucket_create_naming_good_contains_hyphen ______ 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.606 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.607 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_bucket_recreate_not_overriding ___________ 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.618 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_bucket_create_special_key_names ___________ 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.619 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_bucket_list_special_prefix _____________ 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_object_copy_zero_size ________________ 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.632 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.633 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_object_copy_16m ___________________ 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.638 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.644 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_object_copy_same_bucket _______________ 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.645 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.646 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_object_copy_verify_contenttype ___________ 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.657 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_object_copy_to_itself ________________ 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.658 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.659 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_copy_to_itself_with_metadata _________ 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.664 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_object_copy_diff_bucket _______________ 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.671 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.672 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_object_copy_not_owned_bucket ____________ 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.677 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.678 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_object_copy_not_owned_object_bucket _________ 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.685 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.690 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.691 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.691 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_object_copy_canned_acl _______________ 2024-04-22T20:48:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.693 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.693 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.693 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.694 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_object_copy_retaining_metadata ___________ 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.699 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.701 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.702 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.703 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.704 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.705 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_object_copy_replacing_metadata ___________ 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.706 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.707 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.709 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.710 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.711 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.712 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_object_copy_bucket_not_found ____________ 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.713 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.714 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.715 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.716 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.717 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.718 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_copy_key_not_found ______________ 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.719 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.720 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.721 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.722 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.723 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.725 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_object_copy_versioned_bucket ____________ 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.726 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.728 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.730 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.731 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_object_copy_versioned_url_encoding _________ 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.732 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.733 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.734 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.735 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.736 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.737 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.738 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_copy_versioning_multipart_upload _______ 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.739 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.740 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.741 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.742 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.743 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_multipart_upload_empty _______________ 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.745 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.746 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.747 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.748 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.749 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.750 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.751 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_multipart_upload_small _______________ 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.752 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.753 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.754 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.755 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.756 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.757 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_multipart_copy_small ________________ 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.758 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.759 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.760 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.761 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.762 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.763 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.764 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_multipart_copy_invalid_range ____________ 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.765 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.766 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.767 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.768 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.769 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.770 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_multipart_copy_without_range ____________ 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.771 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.772 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.773 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.774 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.775 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.776 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_multipart_copy_special_names ____________ 2024-04-22T20:48:05.777 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.778 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.779 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.780 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.781 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.782 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.783 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_multipart_upload __________________ 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.785 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.787 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.788 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.789 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_multipart_copy_versioned ______________ 2024-04-22T20:48:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.791 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.792 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.794 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.795 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.796 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_multipart_upload_resend_part ____________ 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.797 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.798 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.800 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.801 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.802 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_multipart_upload_multiple_sizes ___________ 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.803 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.804 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.805 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.806 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.807 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.808 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.809 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_multipart_copy_multiple_sizes ____________ 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.810 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.811 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.813 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.814 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.815 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_multipart_upload_size_too_small ___________ 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.816 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.818 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.820 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.821 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.822 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_multipart_upload_contents ______________ 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.824 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.825 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.826 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.827 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.828 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_multipart_upload_overwrite_existing_object _____ 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.829 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.830 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.833 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.834 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.835 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_abort_multipart_upload _______________ 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.836 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.837 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.838 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.839 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.840 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.841 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_abort_multipart_upload_not_found __________ 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.842 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.844 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.846 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.847 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.848 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_list_multipart_upload ________________ 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.849 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.850 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.851 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.852 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.853 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.854 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_list_multipart_upload_owner _____________ 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.855 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.856 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.857 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.859 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.860 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.861 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_multipart_upload_missing_part ____________ 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.862 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.863 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.864 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.865 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.866 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.867 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_multipart_upload_incorrect_etag ___________ 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.868 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.870 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.872 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.873 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.874 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_multipart_get_part _________________ 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.875 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.878 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.879 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.880 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_non_multipart_get_part _______________ 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.881 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.886 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout:____________________ ERROR at teardown of test_100_continue ____________________ 2024-04-22T20:48:05.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.889 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.896 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:______________________ ERROR at teardown of test_set_cors ______________________ 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.898 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.899 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.900 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.901 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.902 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_cors_origin_response ________________ 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.903 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.904 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.905 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.906 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.908 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_cors_origin_wildcard ________________ 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.910 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.911 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.912 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.913 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.914 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.915 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_cors_header_option _________________ 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.916 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.917 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.918 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.919 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.920 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.921 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.922 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_cors_presigned_get_object ______________ 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.923 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.924 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.925 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.926 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.927 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.928 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_cors_presigned_get_object_tenant __________ 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.929 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.930 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.931 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.933 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.934 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.935 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_cors_presigned_put_object ______________ 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.936 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.937 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.939 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.940 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.941 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_cors_presigned_put_object_with_acl _________ 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.942 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.943 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.944 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.947 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_cors_presigned_put_object_tenant __________ 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.950 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.953 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.954 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.955 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_cors_presigned_put_object_tenant_with_acl ______ 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.956 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.957 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.959 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.960 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.961 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_set_bucket_tagging _________________ 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.962 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.963 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.965 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.966 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.967 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_atomic_read_1mb ___________________ 2024-04-22T20:48:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.969 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.970 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.971 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.972 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.973 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_atomic_read_4mb ___________________ 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.975 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.976 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.979 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.980 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_atomic_read_8mb ___________________ 2024-04-22T20:48:05.981 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.982 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.983 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.984 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.986 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.987 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_atomic_write_1mb __________________ 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.989 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.990 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.991 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.992 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.993 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.994 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_atomic_write_4mb __________________ 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:05.995 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:05.996 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:05.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:05.998 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:05.999 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.000 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_atomic_write_8mb __________________ 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.001 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.002 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.003 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.005 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.006 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.007 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_atomic_dual_write_1mb ________________ 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.008 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.009 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.010 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.011 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.012 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.013 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_atomic_dual_write_4mb ________________ 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.014 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.015 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.016 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.017 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.018 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.019 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.020 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_atomic_dual_write_8mb ________________ 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.021 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.023 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.024 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.025 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.026 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.027 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_atomic_conditional_write_1mb ____________ 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.028 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.029 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.030 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.031 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.032 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.033 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_atomic_multipart_upload_write ____________ 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.034 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.035 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.036 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.038 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.039 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.040 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_multipart_resend_first_finishes_last ________ 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.041 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.042 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.043 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.044 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.045 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.046 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_ranged_request_response_code ____________ 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.047 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.048 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.049 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.050 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.051 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.052 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.053 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_ranged_big_request_response_code __________ 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.054 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.055 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.057 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.059 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout:__ ERROR at teardown of test_ranged_request_skip_leading_bytes_response_code ___ 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.060 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.061 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.062 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.063 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.064 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.065 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.066 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_ranged_request_return_trailing_bytes_response_code _ 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.067 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.068 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.069 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.070 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.071 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.072 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_ranged_request_invalid_range ____________ 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.073 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.074 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.075 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.076 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.077 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.078 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.079 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_ranged_request_empty_object _____________ 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.080 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.081 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.084 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.086 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_versioning_bucket_create_suspend __________ 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.087 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.088 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.089 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.090 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.091 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.092 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_versioning_obj_create_read_remove __________ 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.093 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.094 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.095 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.096 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.099 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_versioning_obj_create_read_remove_head _______ 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.107 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_versioning_obj_plain_null_version_removal ______ 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.108 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.109 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_versioning_obj_plain_null_version_overwrite _____ 2024-04-22T20:48:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.115 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_versioning_obj_plain_null_version_overwrite_suspended _ 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.122 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_versioning_obj_suspend_versions ___________ 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.128 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.129 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_versioning_obj_suspended_copy ____________ 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.134 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_versioning_obj_create_versions_remove_all ______ 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.141 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_versioning_obj_create_versions_remove_special_names _ 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.149 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_versioning_obj_create_overwrite_multipart ______ 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.154 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_versioning_obj_list_marker _____________ 2024-04-22T20:48:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.161 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_versioning_copy_obj_version _____________ 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_versioning_multi_object_delete ___________ 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.179 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_versioning_multi_object_delete_with_marker _____ 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.180 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.181 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_versioning_multi_object_delete_with_marker_create __ 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.186 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_versioned_object_acl ________________ 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.194 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.197 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.198 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.199 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_versioned_object_acl_no_version_specified ______ 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.200 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.201 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.202 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.203 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.204 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.205 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout:___ ERROR at teardown of test_versioned_concurrent_object_create_and_remove ____ 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.206 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.207 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.208 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.209 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.210 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.211 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.212 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout:------------------------------ Captured log call ------------------------------- 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout:WARNING urllib3.connectionpool:connectionpool.py:323 Connection pool is full, discarding connection: smithi136.front.sepia.ceph.com. Connection pool size: 10 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout:WARNING urllib3.connectionpool:connectionpool.py:323 Connection pool is full, discarding connection: smithi136.front.sepia.ceph.com. Connection pool size: 10 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout:WARNING urllib3.connectionpool:connectionpool.py:323 Connection pool is full, discarding connection: smithi136.front.sepia.ceph.com. Connection pool size: 10 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout:___________________ ERROR at teardown of test_lifecycle_set ____________________ 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.213 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.214 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.215 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.216 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.217 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.218 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.219 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:___________________ ERROR at teardown of test_lifecycle_get ____________________ 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.220 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.221 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.222 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.223 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.224 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.225 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_lifecycle_get_no_id _________________ 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.226 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.227 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.229 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.230 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.231 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.232 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_lifecycle_id_too_long ________________ 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.233 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.234 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.235 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.236 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.237 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.238 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_lifecycle_same_id __________________ 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.239 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.240 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.241 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.242 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.243 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.244 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.245 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_lifecycle_invalid_status ______________ 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.246 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.247 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.249 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.250 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.251 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_lifecycle_set_date _________________ 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.252 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.253 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.254 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.255 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.256 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.257 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.258 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_lifecycle_set_invalid_date _____________ 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.259 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.260 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.261 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.263 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.264 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_lifecycle_set_noncurrent ______________ 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.265 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.266 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.267 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.268 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.269 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.270 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.271 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_lifecycle_set_deletemarker _____________ 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.272 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.273 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.275 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.276 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.277 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.278 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_lifecycle_set_filter ________________ 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.279 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.280 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.281 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.282 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.283 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.284 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_lifecycle_set_empty_filter _____________ 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.285 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.286 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.287 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.289 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.290 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.291 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_lifecycle_set_multipart _______________ 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.292 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.293 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.295 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.296 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.297 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_lifecycle_transition_set_invalid_date ________ 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.298 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.299 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.300 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.301 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.302 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.303 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.304 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1b ________________ 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.305 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.306 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.307 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.308 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.309 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.310 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.311 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1kb _______________ 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.312 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.313 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.315 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.316 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.317 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1MB _______________ 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.318 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.319 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.320 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.321 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_encrypted_transfer_13b _______________ 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.325 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.326 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.328 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.329 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.330 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.331 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_encryption_sse_c_method_head ____________ 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.332 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.333 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.334 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.335 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.336 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.337 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_encryption_sse_c_present ______________ 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.338 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.339 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.340 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.341 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.342 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.343 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.344 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_encryption_sse_c_other_key _____________ 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.345 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.346 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.347 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.349 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.350 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.351 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_encryption_sse_c_invalid_md5 ____________ 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.352 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.353 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.354 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.355 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.356 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.357 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_encryption_sse_c_no_md5 _______________ 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.358 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.359 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.360 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.361 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.362 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.363 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.364 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_encryption_sse_c_no_key _______________ 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.365 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.366 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.367 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.368 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.369 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.370 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_encryption_key_no_sse_c _______________ 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.371 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.372 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.373 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.374 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.375 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.376 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.377 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_encryption_sse_c_multipart_upload __________ 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.378 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.379 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.380 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.381 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.382 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.383 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_encryption_sse_c_unaligned_multipart_upload _____ 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.384 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.385 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.386 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.387 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.388 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.389 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.390 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_encryption_sse_c_multipart_bad_download _______ 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.391 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.392 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.393 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.395 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.396 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_encryption_sse_c_post_object_authenticated_request _ 2024-04-22T20:48:06.397 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.398 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.400 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.401 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.402 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.403 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_sse_kms_method_head _________________ 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.404 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.405 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.407 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.408 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.409 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.410 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_sse_kms_present ___________________ 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.411 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.413 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.414 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.415 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.416 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout:___________________ ERROR at teardown of test_sse_kms_no_key ___________________ 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.417 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.418 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.419 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.420 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.421 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.423 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_sse_kms_not_declared ________________ 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.424 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.425 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.427 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.428 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.429 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_sse_kms_multipart_upload ______________ 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.430 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.431 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.432 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.433 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.434 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.435 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.436 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_sse_kms_multipart_invalid_chunks_1 _________ 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.437 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.438 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.440 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.441 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.442 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.443 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_sse_kms_multipart_invalid_chunks_2 _________ 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.444 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.445 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.446 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.447 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.448 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.449 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_sse_kms_post_object_authenticated_request ______ 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.450 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.451 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.452 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.453 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.454 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.456 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1b _________________ 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.457 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.458 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.460 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.461 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.462 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1kb ________________ 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.463 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.464 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.465 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.466 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.467 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.469 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1MB ________________ 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.470 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.471 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.472 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.473 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.474 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.475 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_sse_kms_transfer_13b ________________ 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.476 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.477 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.478 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.479 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.480 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.481 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.482 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_sse_kms_read_declare ________________ 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.483 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.484 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.485 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.486 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.487 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.488 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout:___________________ ERROR at teardown of test_bucket_policy ____________________ 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.489 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.490 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.492 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.493 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.494 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.495 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_bucketv2_policy ___________________ 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.496 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.497 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.499 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.500 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.501 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_bucket_policy_acl __________________ 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.502 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.503 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.504 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.505 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.506 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.507 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.509 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_bucketv2_policy_acl _________________ 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.510 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.511 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.512 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.513 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.514 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.515 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.516 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_bucket_policy_another_bucket ____________ 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.517 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.518 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.519 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.520 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.521 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.522 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_bucketv2_policy_another_bucket ___________ 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.523 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.524 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.526 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.527 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.528 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.529 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_get_obj_tagging ___________________ 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.530 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.531 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.532 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.533 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.534 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.535 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_get_obj_head_tagging ________________ 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.536 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.537 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.539 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.540 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.541 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.542 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout:____________________ ERROR at teardown of test_put_max_tags ____________________ 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.543 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.544 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.545 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.546 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.547 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.548 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_put_excess_tags ___________________ 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.549 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.550 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.552 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.553 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.554 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.555 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_put_max_kvsize_tags _________________ 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.556 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.557 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.558 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.559 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.560 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.561 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_put_excess_key_tags _________________ 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.562 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.563 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.564 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.565 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.566 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.568 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_put_excess_val_tags _________________ 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.569 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.570 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.571 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.572 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.573 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.574 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_put_modify_tags ___________________ 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.575 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.576 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.577 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.578 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.579 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.580 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.581 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_put_delete_tags ___________________ 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.582 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.583 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.584 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.586 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.587 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.588 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_post_object_tags_anonymous_request _________ 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.590 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.591 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.592 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.593 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_post_object_tags_authenticated_request _______ 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.595 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.596 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.597 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.598 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.599 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.600 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.601 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_put_obj_with_tags __________________ 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.602 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.603 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.605 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_get_tags_acl_public _________________ 2024-04-22T20:48:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_put_tags_acl_public _________________ 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.616 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout:_______________ ERROR at teardown of test_delete_tags_obj_public _______________ 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.622 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.623 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_versioning_bucket_atomic_upload_return_version_id __ 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.628 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_versioning_bucket_multipart_upload_return_version_id _ 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.636 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_policy_get_obj_existing_tag _________ 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_bucket_policy_get_obj_tagging_existing_tag _____ 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.649 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_bucket_policy_put_obj_tagging_existing_tag _____ 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.655 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.656 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout:_________ ERROR at teardown of test_bucket_policy_put_obj_copy_source __________ 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.662 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.663 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.668 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_bucket_policy_put_obj_copy_source_meta _______ 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.670 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_bucket_policy_put_obj_acl ______________ 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.676 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_bucket_policy_put_obj_grant _____________ 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.682 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_put_obj_enc_conflict_c_s3 ______________ 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_put_obj_enc_conflict_c_kms _____________ 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_put_obj_enc_conflict_s3_kms _____________ 2024-04-22T20:48:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.705 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.706 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.707 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_put_obj_enc_conflict_bad_enc_kms __________ 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.708 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.709 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.710 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.713 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.714 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.715 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.716 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_bucket_policy_put_obj_kms_noenc ___________ 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.717 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.718 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.719 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.720 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.721 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.722 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.723 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_bucket_policy_put_obj_kms_s3 ____________ 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.724 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.725 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.727 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.728 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.729 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_bucket_policy_get_obj_acl_existing_tag _______ 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.730 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.731 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.732 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.733 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.734 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.735 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.736 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_object_lock_put_obj_lock ______________ 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.737 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.738 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.739 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.740 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.741 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.742 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_bucket _______ 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.743 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.744 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.745 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.746 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.747 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.749 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_object_lock_put_obj_lock_with_days_and_years ____ 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.750 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.751 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.752 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.753 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.754 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.755 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_days ________ 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.756 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.757 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.758 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.759 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.760 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.761 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.762 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_years _______ 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.763 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.764 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.765 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.766 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.767 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.768 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.769 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_mode ________ 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.770 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.771 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.772 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.773 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.774 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.775 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_status _______ 2024-04-22T20:48:06.776 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.777 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.778 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.779 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.780 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.781 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.782 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_object_lock_suspend_versioning ___________ 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.783 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.784 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.785 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.786 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.787 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.788 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_object_lock_get_obj_lock ______________ 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.789 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.790 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.791 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.792 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.793 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.794 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.795 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_lock_get_obj_lock_invalid_bucket _______ 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.796 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.797 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.798 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.799 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.801 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_object_lock_put_obj_retention ____________ 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.802 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.803 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.804 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.805 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.806 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.807 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.808 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_object_lock_put_obj_retention_invalid_bucket ____ 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.809 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.810 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.811 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.812 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.813 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.814 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_object_lock_put_obj_retention_invalid_mode _____ 2024-04-22T20:48:06.815 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.816 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.817 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.818 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.819 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.820 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.821 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_object_lock_get_obj_retention ____________ 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.822 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.823 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.824 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.825 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.826 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.827 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout:_______ ERROR at teardown of test_object_lock_get_obj_retention_iso8601 ________ 2024-04-22T20:48:06.828 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.829 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.830 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.831 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.832 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.833 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.834 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_object_lock_get_obj_retention_invalid_bucket ____ 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.835 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.836 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.837 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.838 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.839 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.841 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_lock_put_obj_retention_versionid _______ 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.842 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.843 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.845 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.846 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.847 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_object_lock_put_obj_retention_override_default_retention _ 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.848 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.849 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.850 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.851 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.852 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.853 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.854 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout:___ ERROR at teardown of test_object_lock_put_obj_retention_increase_period ____ 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.855 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.856 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.858 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.859 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.860 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_object_lock_put_obj_retention_shorten_period ____ 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.861 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.862 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.863 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.864 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.865 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.866 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.867 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_object_lock_put_obj_retention_shorten_period_bypass _ 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.868 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.869 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.870 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.871 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.872 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.873 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_object_lock_delete_object_with_retention ______ 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.874 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.876 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.877 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.878 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.879 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.880 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_object_lock_delete_multipart_object_with_retention _ 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.881 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.882 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.884 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.885 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.886 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.887 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_object_lock_delete_object_with_retention_and_marker _ 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.888 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.889 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.890 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.891 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.892 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.893 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout:___ ERROR at teardown of test_object_lock_multi_delete_object_with_retention ___ 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.894 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.895 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.896 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.897 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.898 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.899 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.900 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_lock_put_legal_hold _____________ 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.901 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.902 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.903 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.904 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.905 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.906 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_object_lock_put_legal_hold_invalid_bucket ______ 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.907 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.908 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.909 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.911 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.912 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_object_lock_put_legal_hold_invalid_status ______ 2024-04-22T20:48:06.913 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.914 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.915 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.916 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.917 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.918 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.919 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_lock_get_legal_hold _____________ 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.920 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.921 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.922 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.923 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.924 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.925 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.926 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_object_lock_get_legal_hold_invalid_bucket ______ 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.927 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.928 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.929 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.930 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.931 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.932 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.933 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_object_lock_delete_object_with_legal_hold_on ____ 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.934 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.935 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.937 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.938 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.939 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_object_lock_delete_multipart_object_with_legal_hold_on _ 2024-04-22T20:48:06.940 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.941 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.942 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.943 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.944 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.945 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.946 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout:___ ERROR at teardown of test_object_lock_delete_object_with_legal_hold_off ____ 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.947 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.948 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.949 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.951 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.952 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.953 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_object_lock_get_obj_metadata ____________ 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.954 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.955 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.957 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.958 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.959 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_object_lock_uploading_obj ______________ 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.960 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.961 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.962 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.963 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.964 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.965 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.966 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_object_lock_changing_mode_from_governance_with_bypass _ 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.967 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.968 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.969 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.970 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.971 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.972 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout:nuke_bucket test-8v4uvy8gpgkx125ih4plr0qu-550 waiting 9.567509 seconds for object locks to expire 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_object_lock_changing_mode_from_governance_without_bypass _ 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.973 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.974 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.975 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.976 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.977 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.978 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.979 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_object_lock_changing_mode_from_compliance ______ 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.980 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.981 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.982 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.983 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.984 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.986 INFO:teuthology.orchestra.run.smithi136.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout:nuke_bucket test-8v4uvy8gpgkx125ih4plr0qu-552 waiting 9.100873 seconds for object locks to expire 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_copy_object_ifmatch_good ______________ 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.987 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.988 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.989 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.990 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.991 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.992 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout:___________ ERROR at teardown of test_copy_object_ifnonematch_failed ___________ 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.993 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:06.994 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:06.995 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:06.996 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.997 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:06.998 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout:______________ ERROR at teardown of test_get_bucket_policy_status ______________ 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:06.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.000 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.001 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.002 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.003 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.004 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_get_public_acl_bucket_policy_status _________ 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.006 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.007 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.008 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.009 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.010 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.011 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_get_authpublic_acl_bucket_policy_status _______ 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.012 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.013 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.014 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.015 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.016 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.017 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.018 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout:_____ ERROR at teardown of test_get_publicpolicy_acl_bucket_policy_status ______ 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.019 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.020 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.021 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.022 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.023 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.024 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout:____ ERROR at teardown of test_get_nonpublicpolicy_acl_bucket_policy_status ____ 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.025 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.026 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.027 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.028 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.030 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.031 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout:___ ERROR at teardown of test_get_nonpublicpolicy_deny_bucket_policy_status ____ 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.032 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.033 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.034 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.035 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.036 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.037 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_get_undefined_public_block _____________ 2024-04-22T20:48:07.038 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.039 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.040 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.041 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.042 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.043 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.044 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout:________ ERROR at teardown of test_get_public_block_deny_bucket_policy _________ 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.045 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.046 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.048 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.049 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.050 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout:__________________ ERROR at teardown of test_put_public_block __________________ 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.051 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.052 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.053 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.054 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.055 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.056 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.057 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_block_public_put_bucket_acls ____________ 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.059 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.060 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.061 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.062 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.063 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout:__________ ERROR at teardown of test_block_public_object_canned_acls ___________ 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.064 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.065 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.066 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.069 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.070 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout:________________ ERROR at teardown of test_block_public_policy _________________ 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.071 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.072 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.073 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.074 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.075 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.076 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_ignore_public_acls _________________ 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.077 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.078 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.079 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.080 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.081 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.082 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.083 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout:______ ERROR at teardown of test_multipart_upload_on_a_bucket_with_policy ______ 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.084 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.085 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.086 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.087 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.088 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.089 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.090 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_put_bucket_encryption_kms ______________ 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.091 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.093 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.094 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.095 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.096 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout:_____________ ERROR at teardown of test_get_bucket_encryption_kms ______________ 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.097 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.098 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.099 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.100 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.101 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.102 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout:____________ ERROR at teardown of test_delete_bucket_encryption_kms ____________ 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.103 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.104 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.105 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.106 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.107 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.108 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.109 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout:_ ERROR at teardown of test_sse_kms_default_post_object_authenticated_request __ 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.110 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.111 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.112 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.113 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.114 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.115 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout:_________________ ERROR at teardown of test_get_object_torrent _________________ 2024-04-22T20:48:07.116 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout:configfile = 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.fixture(autouse=True) 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout: def setup_teardown(configfile): 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout: yield 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout:> teardown() 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:332: 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-04-22T20:48:07.117 INFO:teuthology.orchestra.run.smithi136.stdout: raise err 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout: nuke_bucket(client, bucket_name) 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout: for objects in list_versions(client, bucket, batch_size): 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout: listing = client.list_object_versions(**kwargs) 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:514: in _api_call 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout: return self._make_api_call(operation_name, kwargs) 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout:self = 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout:operation_name = 'ListObjectVersions' 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout:api_params = {'Bucket': 'test-8v4uvy8gpgkx125ih4plr0qu-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-04-22T20:48:07.118 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: def _make_api_call(self, operation_name, api_params): 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: service_name = self._service_model.service_name 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: history_recorder.record( 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'API_CALL', 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'service': service_name, 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'operation': operation_name, 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'params': api_params, 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: if operation_model.deprecated: 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: logger.debug( 2024-04-22T20:48:07.119 INFO:teuthology.orchestra.run.smithi136.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: request_context = { 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_region': self.meta.region_name, 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'client_config': self.meta.config, 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: 'auth_type': operation_model.auth_type, 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: request_dict = self._convert_to_request_dict( 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: api_params, operation_model, context=request_context 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: service_id = self._service_model.service_id.hyphenize() 2024-04-22T20:48:07.120 INFO:teuthology.orchestra.run.smithi136.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: params=request_dict, 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: request_signer=self._request_signer, 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: if event_response is not None: 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = event_response 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: else: 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: apply_request_checksum(request_dict) 2024-04-22T20:48:07.121 INFO:teuthology.orchestra.run.smithi136.stdout: http, parsed_response = self._make_request( 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: operation_model, request_dict, request_context 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: self.meta.events.emit( 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: service_id=service_id, operation_name=operation_name 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: ), 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: http_response=http, 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: parsed=parsed_response, 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: model=operation_model, 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: context=request_context, 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: ) 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.122 INFO:teuthology.orchestra.run.smithi136.stdout: if http.status_code >= 300: 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout: error_class = self.exceptions.from_code(error_code) 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout:> raise error_class(parsed_response, operation_name) 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/botocore/client.py:938: NoSuchBucket 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout:=============================== warnings summary =============================== 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout:.tox/py/lib/python3.9/site-packages/boto/plugin.py:40 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/lib/python3.9/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout: import imp 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.123 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py:120 2024-04-22T20:48:07.124 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py:120: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-04-22T20:48:07.124 INFO:teuthology.orchestra.run.smithi136.stdout: if boto_type is 'S3Connection': 2024-04-22T20:48:07.125 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.125 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_headers.py:124 2024-04-22T20:48:07.125 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py:124: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-04-22T20:48:07.125 INFO:teuthology.orchestra.run.smithi136.stdout: elif boto_type is 'HTTPRequest': 2024-04-22T20:48:07.125 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.125 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3.py:359 2024-04-22T20:48:07.125 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3.py:359: DeprecationWarning: invalid escape sequence \W 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: extra_sc = re.split('\W+', config['main']['storage_classes']) 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py:176 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py:176: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: if k is 'BucketName': 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py:253 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py:253: PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.mark.timeout(10) 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests/functional/test_s3_website.py:1077 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: s3tests/functional/test_s3_website.py:1077: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_routing_generator will be ignored 2024-04-22T20:48:07.126 INFO:teuthology.orchestra.run.smithi136.stdout: @pytest.mark.s3website_routing_rules 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2213 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2213: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2228 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2228: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2234 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2234: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: response = client.get_object(Bucket=bucket_name, Key='\$foo.txt') 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.127 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2291 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2291: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2306 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2306: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2324 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2324: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2339 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2339: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.128 INFO:teuthology.orchestra.run.smithi136.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id[::-1]),\ 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2357 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2357: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2372 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2372: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2422 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2422: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.129 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py:2454 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2454: DeprecationWarning: invalid escape sequence \$ 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout: ["starts-with", "$key", "\$foo"],\ 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_headers.py: 29 warnings 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_iam.py: 79 warnings 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3.py: 486 warnings 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_s3select.py: 33 warnings 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout:s3tests_boto3/functional/test_sns.py: 3 warnings 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/lib/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'smithi136.front.sepia.ceph.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout: warnings.warn( 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout:-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html 2024-04-22T20:48:07.130 INFO:teuthology.orchestra.run.smithi136.stdout:=========================== short test summary info ============================ 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_delete - botoco... 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_read_not_exist - botoc... 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_object_delete - botocor... 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete - botoc... 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes - boto... 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_check_etag - bot... 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_cache_control - ... 2024-04-22T20:48:07.131 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_expires - botoco... 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_file - botocore.... 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code 2024-04-22T20:48:07.132 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header - ... 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field 2024-04-22T20:48:07.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy - ... 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket - bo... 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good - bot... 2024-04-22T20:48:07.134 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed - b... 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good - bot... 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed - b... 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good - ... 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed 2024-04-22T20:48:07.135 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get - botocore.err... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone - ... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone - ... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head - botocore.errorf... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist - botoco... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head_extended - botoco... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl - b... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl - b... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime - botoco... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated - bo... 2024-04-22T20:48:07.136 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_anon_put - botocore.er... 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_put_authenticated - bo... 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl 2024-04-22T20:48:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name - botoc... 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long 2024-04-22T20:48:07.138 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_exists - botoco... 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_get_location - botocor... 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl - bot... 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_default - botocore... 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned - botocore.... 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite 2024-04-22T20:48:07.139 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_default - botocore... 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned - botocore.... 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl - botocore.errorfa... 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_write - botocore.e... 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp - botocor... 2024-04-22T20:48:07.140 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_read - botocore.er... 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_readacp - botocore... 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants - bo... 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants - bo... 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email - boto... 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all - botoc... 2024-04-22T20:48:07.141 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list - bo... 2024-04-22T20:48:07.142 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime - botocore... 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous - boto... 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth - b... 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth - botoc... 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix - ... 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size - botoc... 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_16m - botocore.er... 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket - bot... 2024-04-22T20:48:07.143 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself - botoc... 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket - bot... 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl - boto... 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found - b... 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding 2024-04-22T20:48:07.144 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty - boto... 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_small - boto... 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_small - botoco... 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload - botocore.e... 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned - bo... 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents - b... 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object 2024-04-22T20:48:07.145 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload - boto... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_multipart_upload - botoc... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_get_part - botocore... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part - boto... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_100_continue - botocore.error... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_set_cors - botocore.errorfact... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_origin_response - botoco... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard - botoco... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_header_option - botocore... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object - b... 2024-04-22T20:48:07.146 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object - b... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging - botocore... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb - botocore.er... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb - botocore.er... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb - botocore.er... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb - botocore.e... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb - botocore.e... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb - botocore.e... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb - botoc... 2024-04-22T20:48:07.147 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb - botoc... 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb - botoc... 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove 2024-04-22T20:48:07.148 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker - ... 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create 2024-04-22T20:48:07.149 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_object_acl - botoco... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set - botocore.erro... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_get - botocore.erro... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id - botocor... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long - botoc... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id - botocore.... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status - bo... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date - botocore... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date - ... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent - bo... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker - ... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter - botoco... 2024-04-22T20:48:07.150 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter - ... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart - bot... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b - botoc... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb - boto... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB - boto... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b - boto... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present - bo... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key - ... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 - bot... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key - bot... 2024-04-22T20:48:07.151 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c - bot... 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head - botocor... 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_present - botocore.er... 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key - botocore.err... 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared - botoco... 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload - bo... 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b - botocor... 2024-04-22T20:48:07.152 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb - botoco... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB - botoco... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b - botoco... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare - botoco... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy - botocore.erro... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy - botocore.er... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl - botocore.... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl - botocor... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_obj_tagging - botocore.er... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging - botoco... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_max_tags - botocore.error... 2024-04-22T20:48:07.153 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_tags - botocore.er... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags - botocor... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags - botocor... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags - botocor... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_modify_tags - botocore.er... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_delete_tags - botocore.er... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags - botocore.... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public - botocor... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public - botocor... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public - boto... 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id 2024-04-22T20:48:07.154 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl - b... 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 - b... 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms - ... 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 2024-04-22T20:48:07.155 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock - bo... 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock - bo... 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket 2024-04-22T20:48:07.156 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold - ... 2024-04-22T20:48:07.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold - ... 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj - b... 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good - bo... 2024-04-22T20:48:07.158 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status - bo... 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block - ... 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_public_block - botocore.e... 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_policy - botocor... 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ignore_public_acls - botocore... 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy 2024-04-22T20:48:07.159 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms - b... 2024-04-22T20:48:07.160 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms - b... 2024-04-22T20:48:07.160 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms 2024-04-22T20:48:07.160 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request 2024-04-22T20:48:07.160 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_torrent - botocore... 2024-04-22T20:48:07.160 INFO:teuthology.orchestra.run.smithi136.stdout:= 666 passed, 65 skipped, 127 deselected, 1 xfailed, 648 warnings, 397 errors in 1610.89s (0:26:50) = 2024-04-22T20:48:07.160 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: InvocationError for command /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' (exited with code 1) 2024-04-22T20:48:07.160 INFO:teuthology.orchestra.run.smithi136.stdout:___________________________________ summary ____________________________________ 2024-04-22T20:48:07.160 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: py: commands failed 2024-04-22T20:48:07.160 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi136 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-04-22T20:48:07.162 INFO:tasks.s3tests:Cleaning up boto... 2024-04-22T20:48:07.162 DEBUG:teuthology.orchestra.run.smithi136:> rm /home/ubuntu/cephtest/boto-client.0.cfg 2024-04-22T20:48:07.196 INFO:tasks.s3tests:Removing s3-tests... 2024-04-22T20:48:07.196 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/s3-tests-client.0 2024-04-22T20:48:07.392 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi136 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-04-22T20:48:07.640 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=174e96b77bb14192bf6d271a752eadf8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_9ed8263edff7e87907bf24b054e5c7144c194dbe/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi136 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-04-22T20:48:07.644 DEBUG:teuthology.run_tasks:Unwinding manager s3tests 2024-04-22T20:48:07.656 DEBUG:teuthology.run_tasks:Unwinding manager s3tests-java 2024-04-22T20:48:07.704 DEBUG:tasks.s3tests_java:S3 Tests Java: END 2024-04-22T20:48:07.704 INFO:tasks.s3tests_java:S3 Tests Java: Cleaning up s3-tests-java... 2024-04-22T20:48:07.705 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/s3-tests-java 2024-04-22T20:48:07.770 INFO:tasks.s3tests_java:S3 Tests Java: Deleting S3 users... 2024-04-22T20:48:07.770 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid tester.client.0 --purge-data --cluster ceph 2024-04-22T20:48:07.825 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:48:07.825 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:48:07.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.859+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:07.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.860+0000 7f4e69f27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:07.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.860+0000 7f4e69f27800 20 realm 2024-04-22T20:48:07.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.860+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:07.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.860+0000 7f4e69f27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:07.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.860+0000 7f4e69f27800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:48:07.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.860+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:07.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.861+0000 7f4e69f27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:07.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.861+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:07.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.861+0000 7f4e69f27800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:48:07.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.861+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:07.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.862+0000 7f4e69f27800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:48:07.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.862+0000 7f4e69f27800 20 searching for the correct realm 2024-04-22T20:48:07.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:07.864+0000 7f4e66461640 0 -- 172.21.15.136:0/679455553 >> [v2:172.21.15.164:6808/3291801237,v1:172.21.15.164:6809/3291801237] conn(0x7f4e6003f060 msgr2=0x7f4e60096120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-04-22T20:48:08.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.471+0000 7f4e6545f640 0 -- 172.21.15.136:0/679455553 >> [v2:172.21.15.164:6824/100873265,v1:172.21.15.164:6825/100873265] conn(0x55fa5f4473c0 msgr2=0x55fa5f449810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:48:08.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:48:08.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:48:08.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:48:08.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:48:08.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:48:08.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:48:08.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:08.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:08.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.477+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:08.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:48:08.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:08.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:48:08.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 20 zone default found 2024-04-22T20:48:08.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 4 Realm: () 2024-04-22T20:48:08.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:48:08.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:48:08.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:48:08.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 20 zonegroup default 2024-04-22T20:48:08.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:08.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:08.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.478+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:08.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.479+0000 7f4e69f27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:08.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.479+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:08.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.479+0000 7f4e69f27800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:08.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.479+0000 7f4e69f27800 20 started sync module instance, tier type = 2024-04-22T20:48:08.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.479+0000 7f4e69f27800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:48:08.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.491+0000 7f4e69f27800 20 add_watcher() i=0 2024-04-22T20:48:08.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.491+0000 7f4e69f27800 20 add_watcher() i=1 2024-04-22T20:48:08.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.491+0000 7f4e69f27800 20 add_watcher() i=2 2024-04-22T20:48:08.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.491+0000 7f4e69f27800 20 add_watcher() i=3 2024-04-22T20:48:08.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.491+0000 7f4e69f27800 20 add_watcher() i=4 2024-04-22T20:48:08.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.491+0000 7f4e69f27800 20 add_watcher() i=5 2024-04-22T20:48:08.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.491+0000 7f4e69f27800 20 add_watcher() i=6 2024-04-22T20:48:08.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.494+0000 7f4e69f27800 20 add_watcher() i=7 2024-04-22T20:48:08.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.494+0000 7f4e69f27800 2 all 8 watchers are set, enabling cache 2024-04-22T20:48:08.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.496+0000 7f4d177fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:48:08.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.496+0000 7f4e69f27800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:48:08.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.496+0000 7f4e69f27800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:48:08.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.535+0000 7f4d167fc640 20 reqs_thread_entry: start 2024-04-22T20:48:08.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.535+0000 7f4e69f27800 20 init_complete bucket index max shards: 1 2024-04-22T20:48:08.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.535+0000 7f4e69f27800 20 Filter name: none 2024-04-22T20:48:08.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.535+0000 7f4c87fff640 20 reqs_thread_entry: start 2024-04-22T20:48:08.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.535+0000 7f4e69f27800 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-04-22T20:48:08.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.535+0000 7f4e69f27800 20 rados->read ofs=0 len=0 2024-04-22T20:48:08.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.536+0000 7f4e69f27800 20 rados_obj.operate() r=0 bl.length=482 2024-04-22T20:48:08.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.536+0000 7f4e69f27800 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x13 2024-04-22T20:48:08.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.536+0000 7f4e69f27800 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-04-22T20:48:08.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.536+0000 7f4e69f27800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-22T20:48:08.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.537+0000 7f4e69f27800 10 removing key index: CMFMMIFZSZAWERQOBMJF 2024-04-22T20:48:08.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.538+0000 7f4e69f27800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:CMFMMIFZSZAWERQOBMJF, ofs0, ns] 2024-04-22T20:48:08.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.538+0000 7f4d497fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464520 cookie 94533828563168 notifier 7469 bl.length()=170 2024-04-22T20:48:08.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.539+0000 7f4e69f27800 10 removing email index: tester.client.0_test@test.test 2024-04-22T20:48:08.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.540+0000 7f4e69f27800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-04-22T20:48:08.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.541+0000 7f4d497fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464521 cookie 94533828563168 notifier 7469 bl.length()=181 2024-04-22T20:48:08.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.542+0000 7f4e69f27800 10 removing user buckets index 2024-04-22T20:48:08.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.543+0000 7f4e69f27800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:tester.client.0.buckets, ofs0, ns] 2024-04-22T20:48:08.540 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.543+0000 7f4d497fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986464509 cookie 94533828567888 notifier 7469 bl.length()=172 2024-04-22T20:48:08.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.544+0000 7f4e69f27800 10 removing user index: tester.client.0 2024-04-22T20:48:08.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.546+0000 7f4e69f27800 10 removing default.rgw.meta+users.uid+tester.client.0 from cache 2024-04-22T20:48:08.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.546+0000 7f4e69f27800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-04-22T20:48:08.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.546+0000 7f4d497fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1120986465730 cookie 94533828574224 notifier 7469 bl.length()=164 2024-04-22T20:48:08.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.548+0000 7f4d177fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:48:08.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.550+0000 7f4e69f27800 20 remove_watcher() i=0 2024-04-22T20:48:08.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.550+0000 7f4e69f27800 2 removed watcher, disabling cache 2024-04-22T20:48:08.547 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.551+0000 7f4e69f27800 20 remove_watcher() i=1 2024-04-22T20:48:08.548 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.552+0000 7f4e69f27800 20 remove_watcher() i=2 2024-04-22T20:48:08.549 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.553+0000 7f4e69f27800 20 remove_watcher() i=3 2024-04-22T20:48:08.550 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.554+0000 7f4e69f27800 20 remove_watcher() i=4 2024-04-22T20:48:08.551 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.555+0000 7f4e69f27800 20 remove_watcher() i=5 2024-04-22T20:48:08.552 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.556+0000 7f4e69f27800 20 remove_watcher() i=6 2024-04-22T20:48:08.553 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:08.556+0000 7f4e69f27800 20 remove_watcher() i=7 2024-04-22T20:48:08.564 DEBUG:teuthology.run_tasks:Unwinding manager ragweed 2024-04-22T20:48:08.610 DEBUG:teuthology.run_tasks:Unwinding manager tox 2024-04-22T20:48:08.621 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/tox-venv 2024-04-22T20:48:08.675 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-04-22T20:48:08.686 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-04-22T20:48:08.687 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:48:08.695 INFO:tasks.rgw.client.0.smithi136.stdout:2024-04-22T20:48:08.694+0000 f708640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast ssl_certificate=/home/ubuntu/cephtest/ca/rgw.client.0.crt ssl_port=443 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 44638) UID: 0 2024-04-22T20:48:08.695 INFO:tasks.rgw.client.0.smithi136.stdout:2024-04-22T20:48:08.697+0000 a052100 -1 shutting down 2024-04-22T20:48:12.792 INFO:tasks.rgw.client.0.smithi136.stderr:==00:00:56:32.475 44641== could not unlink /tmp/vgdb-pipe-from-vgdb-to-44641-by-root-on-??? 2024-04-22T20:48:12.792 INFO:tasks.rgw.client.0.smithi136.stderr:==00:00:56:32.475 44641== could not unlink /tmp/vgdb-pipe-to-vgdb-from-44641-by-root-on-??? 2024-04-22T20:48:12.792 INFO:tasks.rgw.client.0.smithi136.stderr:==00:00:56:32.475 44641== could not unlink /tmp/vgdb-pipe-shared-mem-vgdb-44641-by-root-on-??? 2024-04-22T20:48:14.788 INFO:tasks.rgw.client.0:Stopped 2024-04-22T20:48:14.788 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-04-22T20:48:14.822 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/vault-root-token 2024-04-22T20:48:14.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-04-22T20:48:14.984 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-04-22T20:48:14.984 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-04-22T20:48:14.984 DEBUG:teuthology.orchestra.run.smithi136:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-04-22T20:48:15.081 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setuser ceph since I am not root 2024-04-22T20:48:15.081 INFO:teuthology.orchestra.run.smithi136.stderr:ignoring --setgroup ceph since I am not root 2024-04-22T20:48:15.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.116+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.117+0000 7f2f5fb46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.117+0000 7f2f5fb46800 20 realm 2024-04-22T20:48:15.114 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.117+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.117+0000 7f2f5fb46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:15.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.117+0000 7f2f5fb46800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-22T20:48:15.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.117+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.117+0000 7f2f5fb46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:15.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.117+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.118+0000 7f2f5fb46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:48:15.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.118+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.119+0000 7f2f5fb46800 20 rados_obj.operate() r=0 bl.length=1139 2024-04-22T20:48:15.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.119+0000 7f2f5fb46800 20 searching for the correct realm 2024-04-22T20:48:15.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.119+0000 7f2f5d645640 0 -- 172.21.15.136:0/416170015 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x55df04f75e60 msgr2=0x55df04f78320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:48:15.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.324+0000 7f2f5d645640 0 -- 172.21.15.136:0/416170015 >> [v2:172.21.15.164:6816/2410758924,v1:172.21.15.164:6817/2410758924] conn(0x55df04e1af40 msgr2=0x7f2f5805d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:48:15.531 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 RGWRados::pool_iterate: got zone_info.19263407-9cd1-4c73-8de4-71c0779d4a94 2024-04-22T20:48:15.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-22T20:48:15.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 RGWRados::pool_iterate: got zonegroup_info.d57a4bf0-d9f1-478e-92bc-d3bb9c9570de 2024-04-22T20:48:15.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 RGWRados::pool_iterate: got default.zone. 2024-04-22T20:48:15.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-22T20:48:15.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-22T20:48:15.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:15.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-22T20:48:15.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.533+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 20 rados_obj.operate() r=0 bl.length=470 2024-04-22T20:48:15.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 20 zone default found 2024-04-22T20:48:15.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 4 Realm: () 2024-04-22T20:48:15.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 4 ZoneGroup: default (d57a4bf0-d9f1-478e-92bc-d3bb9c9570de) 2024-04-22T20:48:15.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 4 Zone: default (19263407-9cd1-4c73-8de4-71c0779d4a94) 2024-04-22T20:48:15.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-22T20:48:15.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 20 zonegroup default 2024-04-22T20:48:15.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:15.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.534+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.535+0000 7f2f5fb46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:15.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.535+0000 7f2f5fb46800 20 rados->read ofs=0 len=0 2024-04-22T20:48:15.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.535+0000 7f2f5fb46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-22T20:48:15.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.535+0000 7f2f5fb46800 20 started sync module instance, tier type = 2024-04-22T20:48:15.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.535+0000 7f2f5fb46800 20 started zone id=19263407-9cd1-4c73-8de4-71c0779d4a94 (name=default) with tier type = 2024-04-22T20:48:15.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.544+0000 7f2f5fb46800 20 add_watcher() i=0 2024-04-22T20:48:15.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.544+0000 7f2f5fb46800 20 add_watcher() i=1 2024-04-22T20:48:15.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.544+0000 7f2f5fb46800 20 add_watcher() i=2 2024-04-22T20:48:15.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.544+0000 7f2f5fb46800 20 add_watcher() i=3 2024-04-22T20:48:15.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.544+0000 7f2f5fb46800 20 add_watcher() i=4 2024-04-22T20:48:15.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.544+0000 7f2f5fb46800 20 add_watcher() i=5 2024-04-22T20:48:15.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.544+0000 7f2f5fb46800 20 add_watcher() i=6 2024-04-22T20:48:15.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.545+0000 7f2f5fb46800 20 add_watcher() i=7 2024-04-22T20:48:15.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.545+0000 7f2f5fb46800 2 all 8 watchers are set, enabling cache 2024-04-22T20:48:15.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.547+0000 7f2e137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:48:15.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.547+0000 7f2f5fb46800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-22T20:48:15.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.547+0000 7f2f5fb46800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-22T20:48:15.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.585+0000 7f2e127e4640 20 reqs_thread_entry: start 2024-04-22T20:48:15.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.585+0000 7f2f5fb46800 20 init_complete bucket index max shards: 1 2024-04-22T20:48:15.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.585+0000 7f2f5fb46800 20 Filter name: none 2024-04-22T20:48:15.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.585+0000 7f2d83fff640 20 reqs_thread_entry: start 2024-04-22T20:48:15.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.585+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-22T20:48:15.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:15.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.587+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:15.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42355' 2024-04-22T20:48:15.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_8zvOKijGqT1UiN3Cflw0_hh7rqiCdHX', time=2024-04-22T22:22:14.741044+0000, chain.objs.size()=3 2024-04-22T20:48:15.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.-Rldb7LFCVIz9AWuulhDA49mCuKhzUf_0 2024-04-22T20:48:15.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.-Rldb7LFCVIz9AWuulhDA49mCuKhzUf_1 2024-04-22T20:48:15.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.-Rldb7LFCVIz9AWuulhDA49mCuKhzUf_2 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9439890628248749257', time=2024-04-22T22:22:53.204532+0000, chain.objs.size()=1 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.172__shadow_.zg1KqbhLFka0kv8m8dhlJ6Ch7droibd_0 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8138677976179935091', time=2024-04-22T22:25:21.699815+0000, chain.objs.size()=1 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.227__shadow_.UcpTt6nino2T1Yds3981J3GVXP03W6d_0 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9697172248603730788', time=2024-04-22T22:25:32.803671+0000, chain.objs.size()=1 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.237__shadow_.pkipszK-9wmZyeK2AQkDtp9YbuhaR29_0 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13072206015574896898', time=2024-04-22T22:25:44.662755+0000, chain.objs.size()=1 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.249__shadow_.QjOlzCmEwYZQ37CDfYCTWcnOrTuqWuV_0 2024-04-22T20:48:15.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3803676901836665144', time=2024-04-22T22:26:22.464623+0000, chain.objs.size()=1 2024-04-22T20:48:15.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.F_EtL8gRQlZyJLiVeqdiO6bssIJfebZ_0 2024-04-22T20:48:15.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13496667705063080770', time=2024-04-22T22:26:22.986731+0000, chain.objs.size()=1 2024-04-22T20:48:15.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.18k-wzmbb1slR05wtp6c60o-QetGhwd_0 2024-04-22T20:48:15.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14743520738291774226', time=2024-04-22T22:26:23.010522+0000, chain.objs.size()=1 2024-04-22T20:48:15.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-TZXB9o56KU1CVkJTP05n3yHP_rKWOF_0 2024-04-22T20:48:15.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5117118978423429199', time=2024-04-22T22:26:24.048756+0000, chain.objs.size()=1 2024-04-22T20:48:15.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LM5etShDsBbOXZXfpwa_ZODEpBsQmNY_0 2024-04-22T20:48:15.588 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5200966114965514209', time=2024-04-22T22:26:24.537116+0000, chain.objs.size()=1 2024-04-22T20:48:15.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.588+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.IawbQ6qmH60bG6iU5w2xADC6C_3RR6g_0 2024-04-22T20:48:15.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14401488149993983633', time=2024-04-22T22:26:25.101270+0000, chain.objs.size()=1 2024-04-22T20:48:15.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oxdqmh23qg1C8Z3U1K9r9sRHIVOzZRU_0 2024-04-22T20:48:15.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7810353012124510978', time=2024-04-22T22:26:25.815129+0000, chain.objs.size()=1 2024-04-22T20:48:15.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Bcm7WrEWR-oZK-5ja38pSbbLT5XVMDE_0 2024-04-22T20:48:15.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14249517686247580885', time=2024-04-22T22:26:25.824217+0000, chain.objs.size()=1 2024-04-22T20:48:15.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.deoiI6rAkFiqc9rWBYeFcG_oulBqo2b_0 2024-04-22T20:48:15.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5668723397879252001', time=2024-04-22T22:26:26.973818+0000, chain.objs.size()=1 2024-04-22T20:48:15.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ClfC2GG_XNtMCZz7mOKNIM7zDv4j36v_0 2024-04-22T20:48:15.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11838387330836530369', time=2024-04-22T22:26:27.502629+0000, chain.objs.size()=1 2024-04-22T20:48:15.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dEaSlNn2Egw1i5QkOJJfJRWOkaJ9Uv8_0 2024-04-22T20:48:15.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15390942451077958043', time=2024-04-22T22:26:27.813534+0000, chain.objs.size()=1 2024-04-22T20:48:15.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SoqAH22wTmgiCUpcgl3I4en8IaSnYeg_0 2024-04-22T20:48:15.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7839179780444578073', time=2024-04-22T22:26:27.976409+0000, chain.objs.size()=1 2024-04-22T20:48:15.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BiTXye_ndTurEYvn_8VWyiTZcQqm4iK_0 2024-04-22T20:48:15.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8468932555292532272', time=2024-04-22T22:26:28.604339+0000, chain.objs.size()=1 2024-04-22T20:48:15.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jrgICiiuvgGnlyWvte4PtPanud6gcDH_0 2024-04-22T20:48:15.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9739927112152687961', time=2024-04-22T22:26:28.619416+0000, chain.objs.size()=1 2024-04-22T20:48:15.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SDuG6iTjk_u3SUn4rRTOOv9EMpZsoBs_0 2024-04-22T20:48:15.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7574630376453555333', time=2024-04-22T22:26:29.574206+0000, chain.objs.size()=1 2024-04-22T20:48:15.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.596+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7Kr5eiHOXAPxjI4H-ai-iOGOalJm6Wh_0 2024-04-22T20:48:15.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.597+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13463331104186427565', time=2024-04-22T22:26:32.026016+0000, chain.objs.size()=1 2024-04-22T20:48:15.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.597+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EPKx7xxnj8MZ6kvZvq_GBbCyGqX3LoS_0 2024-04-22T20:48:15.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1709693145699869679', time=2024-04-22T22:26:32.712658+0000, chain.objs.size()=1 2024-04-22T20:48:15.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ad4lr0gNstMpj2CZukmHnEyYFTfnTny_0 2024-04-22T20:48:15.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5159734552945487533', time=2024-04-22T22:26:32.842734+0000, chain.objs.size()=1 2024-04-22T20:48:15.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZrERJaHcfksv-BAJu09BYvvgJ762mTC_0 2024-04-22T20:48:15.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10929797165757713282', time=2024-04-22T22:26:32.985695+0000, chain.objs.size()=1 2024-04-22T20:48:15.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HuZD_GuMcFAekch8lFOeZQK6Gs99mHh_0 2024-04-22T20:48:15.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16449721565067245502', time=2024-04-22T22:26:33.044748+0000, chain.objs.size()=1 2024-04-22T20:48:15.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.U6SiWIOvifB7J2DLnoahfMnzT5L640Q_0 2024-04-22T20:48:15.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.607557883067254606', time=2024-04-22T22:26:33.078215+0000, chain.objs.size()=1 2024-04-22T20:48:15.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jXy4ga01FbpsncAMhdpRlkJLGXqpheN_0 2024-04-22T20:48:15.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15837058914507890672', time=2024-04-22T22:26:33.162262+0000, chain.objs.size()=1 2024-04-22T20:48:15.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.QiJyUYOs4Te28UL5JKOxV4y0PqfqxNl_0 2024-04-22T20:48:15.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1655035818871949579', time=2024-04-22T22:26:33.839204+0000, chain.objs.size()=1 2024-04-22T20:48:15.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mXiC1hxGJfphQGlcP6wFGhZSzj7z0Ow_0 2024-04-22T20:48:15.601 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2144630099730874990', time=2024-04-22T22:26:34.291925+0000, chain.objs.size()=1 2024-04-22T20:48:15.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Iilqsj1IN4gsg1x7Nw2ZIoBaOUm1sNy_0 2024-04-22T20:48:15.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6114051786625395443', time=2024-04-22T22:26:34.439908+0000, chain.objs.size()=1 2024-04-22T20:48:15.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.yhTgx0dly4XMmOLXaxQjjGJmfgb97Bz_0 2024-04-22T20:48:15.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6161787113897748647', time=2024-04-22T22:26:47.881309+0000, chain.objs.size()=1 2024-04-22T20:48:15.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.602+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.265__shadow_.l3x1stsY9_2Giq0QGzUCW5ymFX51KCh_0 2024-04-22T20:48:15.602 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.605+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.477990612203024704', time=2024-04-22T22:26:52.867377+0000, chain.objs.size()=1 2024-04-22T20:48:15.603 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.605+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.270__shadow_.nXp346E9_39hgDMDONXm83ISHFBzB7X_0 2024-04-22T20:48:15.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.607+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17587905384491283331', time=2024-04-22T22:27:05.590647+0000, chain.objs.size()=1 2024-04-22T20:48:15.604 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.607+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.kZ5_oHWPVBXEjXJ3fTmIaOVQNY3RBwD_0 2024-04-22T20:48:15.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4191216916684214626', time=2024-04-22T22:27:05.675671+0000, chain.objs.size()=1 2024-04-22T20:48:15.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.A28j-D2S9eiTIstMNL2_8xzVgEYWjcN_0 2024-04-22T20:48:15.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11740327368422959758', time=2024-04-22T22:27:07.858264+0000, chain.objs.size()=1 2024-04-22T20:48:15.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.9QuMd0dULLXDdBk1uFZTxLme1siZZF2_0 2024-04-22T20:48:15.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2912869906687674104', time=2024-04-22T22:27:07.866648+0000, chain.objs.size()=1 2024-04-22T20:48:15.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.GA4A2l3AZ6GyD9vbJ7w2j37MMbvSPXj_0 2024-04-22T20:48:15.605 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9432709736776037334', time=2024-04-22T22:28:17.332354+0000, chain.objs.size()=1 2024-04-22T20:48:15.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wMOedK2Uwsz2xpqnBSa8hnJHC3hM7pR_0 2024-04-22T20:48:15.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1247420675580967198', time=2024-04-22T22:28:17.690712+0000, chain.objs.size()=1 2024-04-22T20:48:15.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uWSG1uh1-Y3QswFyd40cqX_KAiDgQCS_0 2024-04-22T20:48:15.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8396085687024048129', time=2024-04-22T22:28:17.962999+0000, chain.objs.size()=1 2024-04-22T20:48:15.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.EjPZ7l-SjWobZ_Im4D09tC9hVqe4dHY_0 2024-04-22T20:48:15.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5282010507863712252', time=2024-04-22T22:28:18.221192+0000, chain.objs.size()=1 2024-04-22T20:48:15.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0nKXGRj6QyusTGzhRZNSmVZsJbPuT-6_0 2024-04-22T20:48:15.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3354302328029255551', time=2024-04-22T22:28:18.381597+0000, chain.objs.size()=1 2024-04-22T20:48:15.607 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.608+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dBIseT_kAIJVUVi4hgua1EFLFZm6v6T_0 2024-04-22T20:48:15.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.610+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8968532483147699823', time=2024-04-22T22:28:18.525412+0000, chain.objs.size()=1 2024-04-22T20:48:15.608 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.610+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GTSJvT8KqUgdoFc0dvcAvstcJqfqka2_0 2024-04-22T20:48:15.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.610+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7123103607757021034', time=2024-04-22T22:28:19.942084+0000, chain.objs.size()=1 2024-04-22T20:48:15.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.610+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SdS3wpS9RPLSz68giAc_yuXhSJrlTSJ_0 2024-04-22T20:48:15.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.611+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5958647731649482483', time=2024-04-22T22:28:20.091516+0000, chain.objs.size()=1 2024-04-22T20:48:15.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.611+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BZjISWk8Qq683OB3kd1BoAlCyyO7D12_0 2024-04-22T20:48:15.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.613+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10641703320571229035', time=2024-04-22T22:28:20.176474+0000, chain.objs.size()=1 2024-04-22T20:48:15.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.613+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.V8tr9XNKRTXjbcU6r5xYiLYUonGtFXK_0 2024-04-22T20:48:15.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.613+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6063736009923546661', time=2024-04-22T22:28:21.175289+0000, chain.objs.size()=1 2024-04-22T20:48:15.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.613+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Ke6cvGUsjUzn-sd0r1OOXG4nZ7Doy8j_0 2024-04-22T20:48:15.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.613+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11700077010097478271', time=2024-04-22T22:28:21.276587+0000, chain.objs.size()=1 2024-04-22T20:48:15.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.613+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.N6XZyepAOjEQZ6OhvZYb5lflHwzd--t_0 2024-04-22T20:48:15.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.615+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5862742603403110121', time=2024-04-22T22:28:21.439845+0000, chain.objs.size()=1 2024-04-22T20:48:15.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.615+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uSowDC_zCMe4CUnGykP2uyY03Ml3Hbl_0 2024-04-22T20:48:15.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.615+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10057832513532057355', time=2024-04-22T22:28:21.526850+0000, chain.objs.size()=1 2024-04-22T20:48:15.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.615+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.O-9JKRyn0-_gqst44rvlV4zLnU8cmCP_0 2024-04-22T20:48:15.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11072701346451093280', time=2024-04-22T22:28:23.236332+0000, chain.objs.size()=1 2024-04-22T20:48:15.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rZpVqX2e_smDQcNAvrnfm5S2xeaXUy3_0 2024-04-22T20:48:15.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17293387667511808709', time=2024-04-22T22:28:24.603360+0000, chain.objs.size()=1 2024-04-22T20:48:15.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.eHHePabgy8ecXZ8UJvFkNBYyeGizN8y_0 2024-04-22T20:48:15.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16964778342183883795', time=2024-04-22T22:28:24.676496+0000, chain.objs.size()=1 2024-04-22T20:48:15.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3g_pwSDZ4ZP2HfaosiDTrnwYTJWcq5J_0 2024-04-22T20:48:15.617 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13575163137330942380', time=2024-04-22T22:28:25.720631+0000, chain.objs.size()=1 2024-04-22T20:48:15.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.capQZmPJBCtzf28rdbIN_m1xfpwHNE__0 2024-04-22T20:48:15.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13563063548542880418', time=2024-04-22T22:28:26.206532+0000, chain.objs.size()=1 2024-04-22T20:48:15.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.618+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.L4LcPNOz3bWfThYTUI4cav_jm_Pl-xx_0 2024-04-22T20:48:15.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.619+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6332927508876572134', time=2024-04-22T22:28:27.138309+0000, chain.objs.size()=1 2024-04-22T20:48:15.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.620+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3jlxuZ-6w_EuMux6tzTa_hWSFndZXxK_0 2024-04-22T20:48:15.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.620+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9963341488564047281', time=2024-04-22T22:28:28.037434+0000, chain.objs.size()=1 2024-04-22T20:48:15.618 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.620+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wjov1iYX8ubT6NWwSCSWZuE_7ZIp1f5_0 2024-04-22T20:48:15.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.620+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14042834597536593287', time=2024-04-22T22:28:28.328629+0000, chain.objs.size()=1 2024-04-22T20:48:15.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.620+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6M73pM4TYiEgUYlUiIKuDvBx9o-FQU__0 2024-04-22T20:48:15.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.620+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3479910467058099369', time=2024-04-22T22:28:28.950082+0000, chain.objs.size()=1 2024-04-22T20:48:15.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.620+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tAMupHZxooxlZsxHbSBUv45GRygooyQ_0 2024-04-22T20:48:15.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.621+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4170007246571390979', time=2024-04-22T22:29:06.228565+0000, chain.objs.size()=1 2024-04-22T20:48:15.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.621+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YYQa94eYwZhKRjl2fxkAZDmAuYGbS2U_0 2024-04-22T20:48:15.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.622+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11582058111900001585', time=2024-04-22T22:29:06.501816+0000, chain.objs.size()=1 2024-04-22T20:48:15.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.622+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.vhNGo59TDfXeCzIm4kZvexhMR865yvD_0 2024-04-22T20:48:15.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11732861222317575392', time=2024-04-22T22:29:06.616465+0000, chain.objs.size()=1 2024-04-22T20:48:15.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LVhGPWStdCoxDXzMxtsWg-8NdaZhuap_0 2024-04-22T20:48:15.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5328694655139276068', time=2024-04-22T22:29:06.681123+0000, chain.objs.size()=1 2024-04-22T20:48:15.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Zue7-Ow7pRW3LRN-t0UAkF2HF-sBpE6_0 2024-04-22T20:48:15.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15520103032601382905', time=2024-04-22T22:29:07.215573+0000, chain.objs.size()=1 2024-04-22T20:48:15.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0fv6nsLf4x1cy0pp95GG5b8tPGXPAtR_0 2024-04-22T20:48:15.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3458039671750765630', time=2024-04-22T22:29:07.287948+0000, chain.objs.size()=1 2024-04-22T20:48:15.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YqPAwTpBOrCuHVm9d_B3Vz5SrRstXw__0 2024-04-22T20:48:15.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1760399934989216601', time=2024-04-22T22:29:08.015435+0000, chain.objs.size()=1 2024-04-22T20:48:15.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Qhhh3m2ABLKJLRQAgUy9qQMWWs0ev_L_0 2024-04-22T20:48:15.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1764083334208938844', time=2024-04-22T22:29:08.770459+0000, chain.objs.size()=1 2024-04-22T20:48:15.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KzBHwLAgFFZz6ZTLwrUZ4Giq6WSKIdf_0 2024-04-22T20:48:15.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15236382857705245497', time=2024-04-22T22:29:08.847696+0000, chain.objs.size()=1 2024-04-22T20:48:15.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OoCBhc7o-Y8uvvFBRdcXsJlnvQ1byTx_0 2024-04-22T20:48:15.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.179195832388581901', time=2024-04-22T22:29:09.917868+0000, chain.objs.size()=1 2024-04-22T20:48:15.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WfG3MAt1fM8quglojH8KxVEqBLHUVFI_0 2024-04-22T20:48:15.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1798799618226710459', time=2024-04-22T22:29:11.041563+0000, chain.objs.size()=1 2024-04-22T20:48:15.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.otO3WW9NM4eUvN9NVll-AOwsVHCTWsl_0 2024-04-22T20:48:15.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9228333366349286863', time=2024-04-22T22:29:11.692995+0000, chain.objs.size()=1 2024-04-22T20:48:15.624 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PF4ks_zxc2xL0GXy22jHLnHW6RUYVeq_0 2024-04-22T20:48:15.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15835436293230509378', time=2024-04-22T22:29:12.562056+0000, chain.objs.size()=1 2024-04-22T20:48:15.625 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.627+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8llOeBMsQFigsYxDZafCkW1mYMW3zJ6_0 2024-04-22T20:48:15.626 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.629+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8949238119296497293', time=2024-04-22T22:29:12.739246+0000, chain.objs.size()=1 2024-04-22T20:48:15.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.629+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Ty9AWPPjxmrz9ffd6OnpsysqJThXhio_0 2024-04-22T20:48:15.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.629+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13293921470017779834', time=2024-04-22T22:29:13.054405+0000, chain.objs.size()=1 2024-04-22T20:48:15.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.629+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NhpgMvsWTEzNEpwPGMn41bJe527ql7Y_0 2024-04-22T20:48:15.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.630+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2561005376262799988', time=2024-04-22T22:29:13.960757+0000, chain.objs.size()=1 2024-04-22T20:48:15.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.630+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GBhzbQtp5lYa7X8EoxTyu3mRp1VifEj_0 2024-04-22T20:48:15.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.630+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15118885417715283530', time=2024-04-22T22:29:14.702704+0000, chain.objs.size()=1 2024-04-22T20:48:15.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.630+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pSmCtIUkzzQMJpgjBaU3sEUADJ7Omah_0 2024-04-22T20:48:15.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.630+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2717690694434053562', time=2024-04-22T22:29:14.838383+0000, chain.objs.size()=1 2024-04-22T20:48:15.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.630+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hA91ljIrC14dobZD5MNq0VMTTug_IrJ_0 2024-04-22T20:48:15.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.630+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17567990910556739362', time=2024-04-22T22:29:14.924148+0000, chain.objs.size()=1 2024-04-22T20:48:15.628 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.630+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HMkCPpcxjuwm2AbAqEbN4fZZ6gvRoVZ_0 2024-04-22T20:48:15.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1885992526114780699', time=2024-04-22T22:29:15.365349+0000, chain.objs.size()=1 2024-04-22T20:48:15.629 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Z01zhLMPnQwxN5CPD0adrSIfke5m2At_0 2024-04-22T20:48:15.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11707560729199069536', time=2024-04-22T22:29:15.947871+0000, chain.objs.size()=1 2024-04-22T20:48:15.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TKxIL2kGg6WQNEfzixXCNahBoznaqeu_0 2024-04-22T20:48:15.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13181617332521144340', time=2024-04-22T22:29:15.993462+0000, chain.objs.size()=1 2024-04-22T20:48:15.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._Ri3bA7Fr8wJdCoC21eInqV69Av8GfW_0 2024-04-22T20:48:15.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5327306680498908625', time=2024-04-22T22:29:16.231897+0000, chain.objs.size()=1 2024-04-22T20:48:15.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.AQOCRqfupZD57U4FPEQ4bqaGct3IwfN_0 2024-04-22T20:48:15.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1047824404937813793', time=2024-04-22T22:29:16.979558+0000, chain.objs.size()=1 2024-04-22T20:48:15.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bf7-Z-e245DDoMBgqLxt4Z73a5N_Cz9_0 2024-04-22T20:48:15.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12061033705250834693', time=2024-04-22T22:29:17.658672+0000, chain.objs.size()=1 2024-04-22T20:48:15.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.632+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gGUsraTFHWiOGFytSbJD8oXNMZeOwt5_0 2024-04-22T20:48:15.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.634+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16039893985204430859', time=2024-04-22T22:29:17.662084+0000, chain.objs.size()=1 2024-04-22T20:48:15.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.634+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Mk9oknz9DWb8eSZ7ziIXbm6g69eBXNL_0 2024-04-22T20:48:15.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.634+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12999022206632820702', time=2024-04-22T22:29:31.148634+0000, chain.objs.size()=1 2024-04-22T20:48:15.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.634+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.332__shadow_.xOJt_6PqHrb2nVmBUzLqeOepRAKcOa8_0 2024-04-22T20:48:15.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.634+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14976999358396736244', time=2024-04-22T22:29:47.559799+0000, chain.objs.size()=1 2024-04-22T20:48:15.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.634+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.359__shadow_.nvPXlY9Nu7cBGNvfQQ31yiZT6YtOOmJ_0 2024-04-22T20:48:15.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.634+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9290172123672515783', time=2024-04-22T22:29:57.767164+0000, chain.objs.size()=1 2024-04-22T20:48:15.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.634+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.372__shadow_.UcpVL0FRqpGA9Mdj8onmAdYGl-v7Z-p_0 2024-04-22T20:48:15.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5358825458749487389', time=2024-04-22T22:30:33.622266+0000, chain.objs.size()=1 2024-04-22T20:48:15.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.425__shadow_.ZO9gJxexjI4jZhgQI2zTmCS_Gd9LNxU_0 2024-04-22T20:48:15.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.637+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8835086201542468008', time=2024-04-22T22:30:50.439554+0000, chain.objs.size()=1 2024-04-22T20:48:15.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.637+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.445__shadow_.N3J3e7XISk5zMLq23QnA0CemjUsXU-C_0 2024-04-22T20:48:15.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.637+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10188215506000908366', time=2024-04-22T22:30:52.754661+0000, chain.objs.size()=1 2024-04-22T20:48:15.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.637+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.447__shadow_.ljLsAuwMmAS7--HrWWGrzPA4gXVJkx7_0 2024-04-22T20:48:15.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.638+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3642795026327408647', time=2024-04-22T22:30:55.039759+0000, chain.objs.size()=1 2024-04-22T20:48:15.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.638+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.449__shadow_.mm6Rso0OJaDNZT_FjYAwZ_BlzkGHHZE_0 2024-04-22T20:48:15.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.638+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15366524889885750837', time=2024-04-22T22:30:59.140412+0000, chain.objs.size()=1 2024-04-22T20:48:15.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.638+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.453__shadow_.gVe0eorQdARFEsH7arPVkonrjVpr15o_0 2024-04-22T20:48:15.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.638+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.477__shadow_.k143HZyb7gHF87tUUMqDsGcUp9OasYi_1_ff74c95c7f912b05fc87451d89c7055b', time=2024-04-22T22:31:17.106914+0000, chain.objs.size()=1 2024-04-22T20:48:15.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.638+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.477__shadow_.k143HZyb7gHF87tUUMqDsGcUp9OasYi_0 2024-04-22T20:48:15.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.638+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4999781293139683161', time=2024-04-22T22:31:17.789912+0000, chain.objs.size()=1 2024-04-22T20:48:15.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.638+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.478__shadow_.7Og9vnTibCncpTg6XOCuYo9_mvhPCA1_0 2024-04-22T20:48:15.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.639+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.484__shadow_.Dpq0vGsicQtqYEHswgZySKlYud89EuE_1_5ded2c95fd38bf7d79e0e13a3719b88e', time=2024-04-22T22:31:21.230689+0000, chain.objs.size()=1 2024-04-22T20:48:15.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.639+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.484__shadow_.Dpq0vGsicQtqYEHswgZySKlYud89EuE_0 2024-04-22T20:48:15.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.640+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12887255206244507280', time=2024-04-22T22:36:39.567755+0000, chain.objs.size()=1 2024-04-22T20:48:15.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.640+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.540__shadow_.eRtIGMZveZ0bQAF3Mcbn1dsQj4cfyrR_0 2024-04-22T20:48:15.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.641+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17812465022894294775', time=2024-04-22T22:36:58.880004+0000, chain.objs.size()=1 2024-04-22T20:48:15.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.641+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.545__shadow_.9_5k_1UE72XlnfXwhqgNoEpfzDRn23c_0 2024-04-22T20:48:15.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.641+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13190559814320552093', time=2024-04-22T22:37:18.240086+0000, chain.objs.size()=1 2024-04-22T20:48:15.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.641+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.552__shadow_.hRiSU7KsGYTTURNzt8Lfxf0jS_ZG_Dh_0 2024-04-22T20:48:15.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.641+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13896296088008858902', time=2024-04-22T22:37:33.357857+0000, chain.objs.size()=1 2024-04-22T20:48:15.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.641+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.558__shadow:GFGyB9mwVZlWPxZV.PxAWx03rJfntmS_.rhq2Xos_huy1tjdYucLCcX6SwuK6--e_0 2024-04-22T20:48:15.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.643+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11649604274735020172', time=2024-04-22T22:37:44.656183+0000, chain.objs.size()=1 2024-04-22T20:48:15.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.643+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:QxOIj8lv5SVRFlIvPZHr-44LTQFzdLD_.JWs0IXUcqdJW5gES4iXMrftT_LP076J_0 2024-04-22T20:48:15.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.648+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/42355 2024-04-22T20:48:15.650 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:48:15.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10836789874622803986', time=2024-04-22T22:37:46.158695+0000, chain.objs.size()=1 2024-04-22T20:48:15.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:n4BevW6B6wVuYFe2VxW6CgdRJLU5Wie_.7114W_TfT0BcAXe5yTNJzDscv19yWhY_0 2024-04-22T20:48:15.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.31728924054941144', time=2024-04-22T22:37:49.269227+0000, chain.objs.size()=1 2024-04-22T20:48:15.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:2PucJOHsNt9MeeAoOMT63Z8nvLYIgiS_.USjsvXpC4Yu5ua1IgXV_CBQ-L57OUdq_0 2024-04-22T20:48:15.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10373484319345895483', time=2024-04-22T22:37:54.914932+0000, chain.objs.size()=1 2024-04-22T20:48:15.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:-tUzBc-vKOvm73gF2-zL8uPAoEn-U1a_.bpqbN1LbFV6tE0zHxISiRo28eNK_JWV_0 2024-04-22T20:48:15.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3942114384667541184', time=2024-04-22T22:37:56.599166+0000, chain.objs.size()=1 2024-04-22T20:48:15.651 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:1O0hxdGsThNruVjhkOgclDFHs3tyexT_.LuTHc5EE9Apvk0-sBOj_YtP7TydL0qc_0 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15696134911769832821', time=2024-04-22T22:38:01.036907+0000, chain.objs.size()=1 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:6.tCNCSgtXaMHNG6zz-3K5rGH8wT-zP_.WsylIwhxdsRom1G0JxOlNob6JrEEzl8_0 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.190074489859013025', time=2024-04-22T22:40:01.259865+0000, chain.objs.size()=12 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__multipart_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.1 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__shadow_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.1_1 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__multipart_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.2 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__shadow_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.2_1 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__multipart_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.3 2024-04-22T20:48:15.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__shadow_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.3_1 2024-04-22T20:48:15.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.653+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__multipart_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.4 2024-04-22T20:48:15.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.656+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__shadow_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.4_1 2024-04-22T20:48:15.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.656+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__multipart_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.5 2024-04-22T20:48:15.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.657+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__shadow_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.5_1 2024-04-22T20:48:15.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.658+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__multipart_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.6 2024-04-22T20:48:15.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.658+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.601__shadow_multipart_enc.2~IqUFNwmy0D3psrGjzWathO3bWqVkZYa.6_1 2024-04-22T20:48:15.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.659+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1947934168037229202', time=2024-04-22T22:42:09.473582+0000, chain.objs.size()=1 2024-04-22T20:48:15.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.659+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.630__shadow_.7wh2FjHITrqRO7D3--OYlstJwhOS7e2_0 2024-04-22T20:48:15.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.659+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14286741456219456950', time=2024-04-22T22:43:23.398364+0000, chain.objs.size()=1 2024-04-22T20:48:15.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.659+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.695__multipart_file1.2~qVqkSmOWxlh6nKgw5okOGGHpvQg2yxa.1 2024-04-22T20:48:15.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.659+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4091668210319020000', time=2024-04-22T22:43:35.561906+0000, chain.objs.size()=1 2024-04-22T20:48:15.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.659+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.699__shadow:bx8sxSfWkAyNSHS2jitq.-v0WTlGQCT_.AJ3U5WTQjSI4RDwPWpby0hYdD_Ov7Lu_0 2024-04-22T20:48:15.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.663+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:15.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.665+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-22T20:48:15.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.666+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:15.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:15.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45150' 2024-04-22T20:48:15.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1379670305508689793', time=2024-04-22T22:21:17.044823+0000, chain.objs.size()=1 2024-04-22T20:48:15.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.137__shadow_.j3DilkamrT8kflIQ-oInPWEA2RVKtX0_0 2024-04-22T20:48:15.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18198904164584555643', time=2024-04-22T22:22:27.970385+0000, chain.objs.size()=2 2024-04-22T20:48:15.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.153__multipart_foo.2~nEFSZCe-fIKfGOMCdZy2C5mXPlf5Fk3.1 2024-04-22T20:48:15.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.153__multipart_foo.2~nEFSZCe-fIKfGOMCdZy2C5mXPlf5Fk3.2 2024-04-22T20:48:15.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10649852249413175192', time=2024-04-22T22:25:27.140518+0000, chain.objs.size()=1 2024-04-22T20:48:15.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.232__shadow_.j0LAzcoNiBAaq0yVaJ6OHa57YhkojE9_0 2024-04-22T20:48:15.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18293880367887427454', time=2024-04-22T22:25:31.829223+0000, chain.objs.size()=1 2024-04-22T20:48:15.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.236__shadow_.Jy1oULA_rahtb5EKiw8xO5dUSUDB6EY_0 2024-04-22T20:48:15.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9824246252114372359', time=2024-04-22T22:25:42.733912+0000, chain.objs.size()=1 2024-04-22T20:48:15.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.247__shadow_.UZvPF9Mc0C1Oy1ZyG84WsLmhFH3z_Wp_0 2024-04-22T20:48:15.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7393016157904697564', time=2024-04-22T22:26:22.394021+0000, chain.objs.size()=1 2024-04-22T20:48:15.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vtGz83m8fDJNMd2s5FP8KnJ7Lt4qhzR_0 2024-04-22T20:48:15.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9749765898363715195', time=2024-04-22T22:26:22.394728+0000, chain.objs.size()=1 2024-04-22T20:48:15.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GC_c66BvjV012z06WxJC4XosyWgj_LJ_0 2024-04-22T20:48:15.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11297026997499746750', time=2024-04-22T22:26:22.778696+0000, chain.objs.size()=1 2024-04-22T20:48:15.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qB4inVrogCSAPZdtiZ5Gx_z-0-kTuWK_0 2024-04-22T20:48:15.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.668+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7881790959033209116', time=2024-04-22T22:26:22.918742+0000, chain.objs.size()=1 2024-04-22T20:48:15.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.668+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2qV6lJGZbxqnOkrVkQj8uuDerRRwnkB_0 2024-04-22T20:48:15.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.668+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4436575794913455551', time=2024-04-22T22:26:23.048778+0000, chain.objs.size()=1 2024-04-22T20:48:15.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.668+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.H5JCke-A20S5VzhjnrRrtoIueYWSzLS_0 2024-04-22T20:48:15.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.668+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16131434718214912889', time=2024-04-22T22:26:24.263473+0000, chain.objs.size()=1 2024-04-22T20:48:15.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.668+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MNrk0hg9aXRY-3htISwjFxlJ2hd1Glb_0 2024-04-22T20:48:15.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17976361475787123065', time=2024-04-22T22:26:24.615158+0000, chain.objs.size()=1 2024-04-22T20:48:15.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.p02hy5H-Nee0BOtpT88iyE5b6Q2QdXK_0 2024-04-22T20:48:15.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10054357728948857697', time=2024-04-22T22:26:24.665430+0000, chain.objs.size()=1 2024-04-22T20:48:15.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6W5CBKUtngWG-fyRtcfcsH30EmJja7a_0 2024-04-22T20:48:15.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15549418576319570512', time=2024-04-22T22:26:25.124285+0000, chain.objs.size()=1 2024-04-22T20:48:15.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0RncF_7ZShjNfWFBPWFHMA0JmweOGOx_0 2024-04-22T20:48:15.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.671+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12125613914407538313', time=2024-04-22T22:26:26.018185+0000, chain.objs.size()=1 2024-04-22T20:48:15.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.671+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FE9KMW6d24D0B-bFLL4Du7uumnisHQT_0 2024-04-22T20:48:15.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11866459450317359222', time=2024-04-22T22:26:26.161371+0000, chain.objs.size()=1 2024-04-22T20:48:15.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.b6gKjht91bkbM11JkkKIrl6qAlqWTrD_0 2024-04-22T20:48:15.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5152677713983960414', time=2024-04-22T22:26:26.458447+0000, chain.objs.size()=1 2024-04-22T20:48:15.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gmS0hsHCpnrr0kLSYNvhNDhYn775ZDx_0 2024-04-22T20:48:15.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10163415369750466051', time=2024-04-22T22:26:26.813309+0000, chain.objs.size()=1 2024-04-22T20:48:15.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PRmn6mwRMHznHcXjlPhZg2lXVnoZ5r4_0 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2563163891245757873', time=2024-04-22T22:26:26.827960+0000, chain.objs.size()=1 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8tAZSTngKcT91EP14LqrHCY9PlPkmAl_0 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.407142699633806934', time=2024-04-22T22:26:27.646776+0000, chain.objs.size()=1 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aNLBU402BTm4cyhqZOvKHrIxSSVapEK_0 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8516865650176619958', time=2024-04-22T22:26:27.654316+0000, chain.objs.size()=1 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.672+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GfLIRTJdNLg7HGuNC3lVjDr5zvDTR4z_0 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.673+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12278697824926365184', time=2024-04-22T22:26:28.212080+0000, chain.objs.size()=1 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.673+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eHNIEB-30azF71dFRlb5o5WdOWmoflG_0 2024-04-22T20:48:15.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.673+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11773296854097172613', time=2024-04-22T22:26:28.347221+0000, chain.objs.size()=1 2024-04-22T20:48:15.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.673+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6iHC-gX4ohMQhIcM8cgL3arvSyvnMpZ_0 2024-04-22T20:48:15.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.675+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7355754637022358212', time=2024-04-22T22:26:28.470446+0000, chain.objs.size()=1 2024-04-22T20:48:15.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.675+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XnvUsn5DLtuLtWdXwKLQMe9WBGIxsPp_0 2024-04-22T20:48:15.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.675+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13406313094535386901', time=2024-04-22T22:26:28.538364+0000, chain.objs.size()=1 2024-04-22T20:48:15.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.675+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.f-IB8zIHpcE9AXVvRpLaEnL5-0jMcxv_0 2024-04-22T20:48:15.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.675+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3012880044327737999', time=2024-04-22T22:26:28.608992+0000, chain.objs.size()=1 2024-04-22T20:48:15.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.675+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pPtkmskm01kI0jzE5fGtmkOD0QtEm0E_0 2024-04-22T20:48:15.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.677+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14136048652162634682', time=2024-04-22T22:26:29.074200+0000, chain.objs.size()=1 2024-04-22T20:48:15.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.677+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.W3nLnDiC9Phv5yhhwccUBbj_u9ZxKyy_0 2024-04-22T20:48:15.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1144999174107091939', time=2024-04-22T22:26:29.351663+0000, chain.objs.size()=1 2024-04-22T20:48:15.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GGh797xsBjsBAz_YGSXogzlXo2YWlH8_0 2024-04-22T20:48:15.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3194315968798969090', time=2024-04-22T22:26:30.960158+0000, chain.objs.size()=1 2024-04-22T20:48:15.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.TxzXhzwjQYqz2Ezmr5vmiECBnW_JkJB_0 2024-04-22T20:48:15.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16581587949916608458', time=2024-04-22T22:26:31.540434+0000, chain.objs.size()=1 2024-04-22T20:48:15.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZlP2fW6V399CBdmjGiy1TnKkqmi1RGk_0 2024-04-22T20:48:15.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9631703575559578396', time=2024-04-22T22:26:31.692368+0000, chain.objs.size()=1 2024-04-22T20:48:15.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eVI-zdmWHxaRWn4N1XQKS9mIUd6W03a_0 2024-04-22T20:48:15.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6534530749690699314', time=2024-04-22T22:26:32.024371+0000, chain.objs.size()=1 2024-04-22T20:48:15.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aIVuW3a7HWxa_gVAs1HztBiQB1CxDPL_0 2024-04-22T20:48:15.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10784946305968594290', time=2024-04-22T22:26:32.025230+0000, chain.objs.size()=1 2024-04-22T20:48:15.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oZZqbSODsbZShlr5q8F_g4djHRy3lLJ_0 2024-04-22T20:48:15.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3896756664669378637', time=2024-04-22T22:26:32.088400+0000, chain.objs.size()=1 2024-04-22T20:48:15.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Q8rBwu9c8dJTzXMD2qqcBNLfhVFRaz0_0 2024-04-22T20:48:15.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.679+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5669803405922139267', time=2024-04-22T22:26:32.386055+0000, chain.objs.size()=1 2024-04-22T20:48:15.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.679+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Z6V0mfOY3iXpODRmF69z2Z9xHCK2oKW_0 2024-04-22T20:48:15.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.679+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13091437945145412553', time=2024-04-22T22:26:33.172653+0000, chain.objs.size()=1 2024-04-22T20:48:15.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.679+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dlRZSdV0r8Bvne2vN1-bka6SBwsNZJe_0 2024-04-22T20:48:15.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.680+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6730814079974809759', time=2024-04-22T22:26:33.733412+0000, chain.objs.size()=1 2024-04-22T20:48:15.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.680+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.USbvLC-1tLoo7Ko45CYaiUMnqjJSiPP_0 2024-04-22T20:48:15.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8675130923086237816', time=2024-04-22T22:26:33.980543+0000, chain.objs.size()=1 2024-04-22T20:48:15.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rPaMmQNS6fghUMimhvNWF_UKNUgKpmt_0 2024-04-22T20:48:15.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17589781586917155771', time=2024-04-22T22:26:34.176946+0000, chain.objs.size()=1 2024-04-22T20:48:15.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DHQYh2GFOcZr_0ECPJpxEMuMPb9jh4i_0 2024-04-22T20:48:15.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6596724280531298805', time=2024-04-22T22:26:44.962627+0000, chain.objs.size()=1 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.262__shadow_.eBayRr3LIVlrKXw4OimQvE8ghEFgTBg_0 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5551786427590067563', time=2024-04-22T22:26:55.840248+0000, chain.objs.size()=1 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.273__shadow_.qjrWzch07qgVpjonwlahEB7pMg1tFIL_0 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9637458039248794652', time=2024-04-22T22:26:59.978179+0000, chain.objs.size()=1 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.682+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.277__shadow_.59y-B0IGinyDaELgdMlyh5yO0ZvuTwc_0 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15753402884031118132', time=2024-04-22T22:27:05.567573+0000, chain.objs.size()=1 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.pne_3UCMGeO9VTaNiVxYjntlHDXpVj8_0 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6881089300420995306', time=2024-04-22T22:27:05.635394+0000, chain.objs.size()=1 2024-04-22T20:48:15.683 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.pY6M_74ZXj98KEKTwCdpcF4RQV0dv4K_0 2024-04-22T20:48:15.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4743684395097177629', time=2024-04-22T22:27:05.714544+0000, chain.objs.size()=1 2024-04-22T20:48:15.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.uavMVDcbPYCJuHP5wwPyvDy9OzuIhx__0 2024-04-22T20:48:15.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.959068329322420370', time=2024-04-22T22:27:07.856326+0000, chain.objs.size()=1 2024-04-22T20:48:15.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.hSTCZwtBaHxbRByA04bxdEabMhbmAOW_0 2024-04-22T20:48:15.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2785384500072733158', time=2024-04-22T22:27:09.959025+0000, chain.objs.size()=1 2024-04-22T20:48:15.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.683+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.284__shadow_.ZlWDfO8b_bPzXkOYxao26ONz_KZOeXd_0 2024-04-22T20:48:15.684 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.684+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14451104788595803770', time=2024-04-22T22:27:10.984049+0000, chain.objs.size()=1 2024-04-22T20:48:15.685 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.684+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.285__shadow_.HoLkZ75RhPrsyu0MIpY8UtinqWM1z9e_0 2024-04-22T20:48:15.686 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.685+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.997901637998901247', time=2024-04-22T22:27:37.382995+0000, chain.objs.size()=1 2024-04-22T20:48:15.686 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.685+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.308__shadow:Te8QJnCiLkZjTXQw9HMVEBLnDSA73AJ_.3m7nG3zjViMQejHqAufJilN-7WOeNIj_0 2024-04-22T20:48:15.686 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.686+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11519579242674195178', time=2024-04-22T22:28:17.521069+0000, chain.objs.size()=1 2024-04-22T20:48:15.686 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.686+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fkzLAJJUgr1oEPgN8bgt0Axv6-RAsEf_0 2024-04-22T20:48:15.686 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.686+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9544346555686873090', time=2024-04-22T22:28:17.889457+0000, chain.objs.size()=1 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.686+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gr467HQ2dfi1ZWxOpvltuNaKC-P2lSw_0 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.686+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1752160650065693663', time=2024-04-22T22:28:18.478269+0000, chain.objs.size()=1 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.686+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6X-NVYMRCgVyEJk2YBASfshFnQHfHYP_0 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11771312009623349291', time=2024-04-22T22:28:19.594781+0000, chain.objs.size()=1 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.b7flvxgsNPeZRBb5zZxGBSsbsnzI3b1_0 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13637424746083402424', time=2024-04-22T22:28:19.595571+0000, chain.objs.size()=1 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.U-azrHNucyyGffgWLphveI4Mqq_SIAd_0 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5832708705750823619', time=2024-04-22T22:28:19.769559+0000, chain.objs.size()=1 2024-04-22T20:48:15.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-8TDfBz7ZM4EGU8zcQJE7Sz7nt5timL_0 2024-04-22T20:48:15.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8275089711095893130', time=2024-04-22T22:28:20.845341+0000, chain.objs.size()=1 2024-04-22T20:48:15.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Zjq6WWIdfO0VVW6sNJVlOJas-dglX-W_0 2024-04-22T20:48:15.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15035659039801676765', time=2024-04-22T22:28:21.027119+0000, chain.objs.size()=1 2024-04-22T20:48:15.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zOSZ14tiQP5r8iGHUXzVRAfaUCCKcFc_0 2024-04-22T20:48:15.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12040379477540211296', time=2024-04-22T22:28:21.550311+0000, chain.objs.size()=1 2024-04-22T20:48:15.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.--FbOhFGUca_EtfCvKMpxNOXibugI2a_0 2024-04-22T20:48:15.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10857368445892451387', time=2024-04-22T22:28:21.857876+0000, chain.objs.size()=1 2024-04-22T20:48:15.688 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.687+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.w40NxvjBOsXvHizMhGZcwYRxYsKTYzL_0 2024-04-22T20:48:15.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.690+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6804005506100992746', time=2024-04-22T22:28:23.339116+0000, chain.objs.size()=1 2024-04-22T20:48:15.689 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.690+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.2GvzF_69x-Mmp94GsMtjsSvqiPGaYKe_0 2024-04-22T20:48:15.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.690+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16230090345714151634', time=2024-04-22T22:28:23.441193+0000, chain.objs.size()=1 2024-04-22T20:48:15.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.690+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._DomGBCT65zvt27G0e6LQYeSdHA3gU5_0 2024-04-22T20:48:15.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.690+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4601145897223435829', time=2024-04-22T22:28:24.266717+0000, chain.objs.size()=1 2024-04-22T20:48:15.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.690+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.okaa310MEqEyviHrGGvrNS26UrFxdH7_0 2024-04-22T20:48:15.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.691+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2471544327788647119', time=2024-04-22T22:28:24.880611+0000, chain.objs.size()=1 2024-04-22T20:48:15.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.691+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.DLcm7WI0pDxPr2HUkA2gXg0hPB4fAtP_0 2024-04-22T20:48:15.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7067343582396427270', time=2024-04-22T22:28:24.967461+0000, chain.objs.size()=1 2024-04-22T20:48:15.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-8wTzQR5uh4jCv3n0tfSn6siGEi9-ur_0 2024-04-22T20:48:15.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4029730972920017406', time=2024-04-22T22:28:24.968254+0000, chain.objs.size()=1 2024-04-22T20:48:15.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zkEEz3Q0v0uhHb1bJET8m_55GZUAoWB_0 2024-04-22T20:48:15.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.967622500089883051', time=2024-04-22T22:28:25.030725+0000, chain.objs.size()=1 2024-04-22T20:48:15.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nZldIx016zKsG5MNO2fhCStIErTXtrP_0 2024-04-22T20:48:15.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6264147478181210643', time=2024-04-22T22:28:25.576414+0000, chain.objs.size()=1 2024-04-22T20:48:15.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BRpKB8oh-Zy17gyFkgjU8EPt6ymMImI_0 2024-04-22T20:48:15.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1952176540149879621', time=2024-04-22T22:28:25.639693+0000, chain.objs.size()=1 2024-04-22T20:48:15.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FgsQKzWFpnFpqpzTgLfoUV-0NrZrdEj_0 2024-04-22T20:48:15.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5992633975035723589', time=2024-04-22T22:28:25.909138+0000, chain.objs.size()=1 2024-04-22T20:48:15.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bpJJsdXz6diMoRA06z1kt2KMFf2p1NN_0 2024-04-22T20:48:15.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17985541877014235041', time=2024-04-22T22:28:26.100348+0000, chain.objs.size()=1 2024-04-22T20:48:15.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Mjcjm6p_IPBoLzrAthRlUzRitAtr31v_0 2024-04-22T20:48:15.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17314435435417980233', time=2024-04-22T22:28:26.417905+0000, chain.objs.size()=1 2024-04-22T20:48:15.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.692+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.eWhBdADbKftmiKgqWlE6MQ_UvmCVJhE_0 2024-04-22T20:48:15.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.695+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.78230793639537967', time=2024-04-22T22:28:27.782821+0000, chain.objs.size()=1 2024-04-22T20:48:15.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.695+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NipXPt_a5-1I9XAArYwbzE0tuMFZx0Q_0 2024-04-22T20:48:15.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.695+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8050227000248917090', time=2024-04-22T22:28:28.529165+0000, chain.objs.size()=1 2024-04-22T20:48:15.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.695+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.naK8beZHFtBEx2FFYGAIxJa4NjvA0Ow_0 2024-04-22T20:48:15.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.695+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1232570082877432398', time=2024-04-22T22:29:06.881954+0000, chain.objs.size()=1 2024-04-22T20:48:15.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.695+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3y4kLls4YKXKFZi6Xm2ofdYjXJw4i2d_0 2024-04-22T20:48:15.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.696+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13728012643880323395', time=2024-04-22T22:29:07.290131+0000, chain.objs.size()=1 2024-04-22T20:48:15.694 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.696+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HC1oxR5PtZ_SqEm3zXCsAo3lobvzJaD_0 2024-04-22T20:48:15.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.696+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16556398154003953393', time=2024-04-22T22:29:08.050387+0000, chain.objs.size()=1 2024-04-22T20:48:15.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.696+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6nip36iiPG5QbqORKGbp86BbG23vC_J_0 2024-04-22T20:48:15.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5589314532736140976', time=2024-04-22T22:29:08.126164+0000, chain.objs.size()=1 2024-04-22T20:48:15.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.L-tGe8XCQ1d3QUQ7ZXFCSNBVaQx7moC_0 2024-04-22T20:48:15.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2933782586077832857', time=2024-04-22T22:29:08.172764+0000, chain.objs.size()=1 2024-04-22T20:48:15.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.opLa5C8ViS1WEs9KCfxokXXfR8cykT0_0 2024-04-22T20:48:15.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4764927995717158676', time=2024-04-22T22:29:08.325106+0000, chain.objs.size()=1 2024-04-22T20:48:15.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.cdS3mqCs5aferJkScNZJsvVUy2WGSx-_0 2024-04-22T20:48:15.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4005533337249201930', time=2024-04-22T22:29:08.616777+0000, chain.objs.size()=1 2024-04-22T20:48:15.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6t0q5HIkLbXebNVeJYRmQNlGYmoYKgE_0 2024-04-22T20:48:15.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16950197275819703666', time=2024-04-22T22:29:09.413418+0000, chain.objs.size()=1 2024-04-22T20:48:15.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1KCUtSfrUsroqhl6YwOadZodPmBF1JS_0 2024-04-22T20:48:15.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.589764340710240569', time=2024-04-22T22:29:09.901108+0000, chain.objs.size()=1 2024-04-22T20:48:15.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.697+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.J3cEYuTsGexnQWsFjni6fEW6uzu84je_0 2024-04-22T20:48:15.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.699+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.402917290627234762', time=2024-04-22T22:29:11.059471+0000, chain.objs.size()=1 2024-04-22T20:48:15.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.699+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.CGQ-Qbd4GTCVYQnq0zv7kVo63z9sMbQ_0 2024-04-22T20:48:15.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.699+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5257958573656165079', time=2024-04-22T22:29:11.105799+0000, chain.objs.size()=1 2024-04-22T20:48:15.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.699+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oUrsdvPgy5I5342XNOl1IXXVHQ7ghrO_0 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.699+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3534894716469998277', time=2024-04-22T22:29:11.334908+0000, chain.objs.size()=1 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.699+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kLrrre7cgA9GaJDoDvJaUUkxhFWuW6D_0 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.699+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11720093112343460012', time=2024-04-22T22:29:11.511604+0000, chain.objs.size()=1 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.699+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TgTlSUn77TeGh4gjMXDiFl9-yZvmFMF_0 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.700+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.869779100212011578', time=2024-04-22T22:29:11.685613+0000, chain.objs.size()=1 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.700+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pXfrVLT4lQrhmX7m5DaqAQwKoPvzxFJ_0 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4920164777587769754', time=2024-04-22T22:29:12.570505+0000, chain.objs.size()=1 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OOp9Sh28IU-J7nyzTmqUFuBeIgTSn6z_0 2024-04-22T20:48:15.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3201689387982293369', time=2024-04-22T22:29:12.571340+0000, chain.objs.size()=1 2024-04-22T20:48:15.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wnq5c-0Ly-FWa8zR9Q2wrpHTyWA0t6u_0 2024-04-22T20:48:15.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.34965409102856398', time=2024-04-22T22:29:12.710287+0000, chain.objs.size()=1 2024-04-22T20:48:15.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ws0670gFkqxaKJv3nfSJyEh4eeW_r58_0 2024-04-22T20:48:15.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12989317957030173012', time=2024-04-22T22:29:12.905303+0000, chain.objs.size()=1 2024-04-22T20:48:15.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MoyBtoJnk1Cbm7jH1Ock43MVw0D1Lqz_0 2024-04-22T20:48:15.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16324946157994127812', time=2024-04-22T22:29:12.920191+0000, chain.objs.size()=1 2024-04-22T20:48:15.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.y8vsoCLqBsG8e9KA426cqrrjZXb0LpF_0 2024-04-22T20:48:15.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5897594044656909003', time=2024-04-22T22:29:13.080408+0000, chain.objs.size()=1 2024-04-22T20:48:15.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.h8FZ8jhohM0sOgiHoK0xgu0pDhayH12_0 2024-04-22T20:48:15.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5828120485132702103', time=2024-04-22T22:29:13.236313+0000, chain.objs.size()=1 2024-04-22T20:48:15.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TE4WjQhkgSUe8FxIELzv_i_Yb9OVvl1_0 2024-04-22T20:48:15.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1395267010737145354', time=2024-04-22T22:29:13.339723+0000, chain.objs.size()=1 2024-04-22T20:48:15.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.704+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.C4YHLIirlAb0HAT7ZgdiDOf-cCLqZgn_0 2024-04-22T20:48:15.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.704+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4464487083790358328', time=2024-04-22T22:29:13.907891+0000, chain.objs.size()=1 2024-04-22T20:48:15.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.704+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pCyl3GBGr8SNl9nfYDt9c0MaFwl86bu_0 2024-04-22T20:48:15.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.704+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8862882416607596780', time=2024-04-22T22:29:14.200419+0000, chain.objs.size()=1 2024-04-22T20:48:15.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.704+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.x_FFYGoIT435cOAsu9D5cr5tamKhYa3_0 2024-04-22T20:48:15.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.704+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12133136780294887315', time=2024-04-22T22:29:14.789164+0000, chain.objs.size()=1 2024-04-22T20:48:15.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.704+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.u3AcgGUnbrm324yTPmBywCi_joa3zyx_0 2024-04-22T20:48:15.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.705+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13566469924499149761', time=2024-04-22T22:29:14.945796+0000, chain.objs.size()=1 2024-04-22T20:48:15.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.705+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.stRyh0nhbQLMeoTQLQFX2Lv2puGbNca_0 2024-04-22T20:48:15.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.705+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2250998485845052874', time=2024-04-22T22:29:15.475615+0000, chain.objs.size()=1 2024-04-22T20:48:15.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.705+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.s34T5UJOZJKZMBygPzcvmwIuVOizYh6_0 2024-04-22T20:48:15.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.708+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/45150 2024-04-22T20:48:15.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=17, truncated=0, next_marker='' 2024-04-22T20:48:15.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14301895988957122552', time=2024-04-22T22:29:15.595902+0000, chain.objs.size()=1 2024-04-22T20:48:15.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XvXEaCmnqWuW9gurEWzhcG8VebXRq2y_0 2024-04-22T20:48:15.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12560780698232024590', time=2024-04-22T22:29:15.856217+0000, chain.objs.size()=1 2024-04-22T20:48:15.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.toWariz8bYpW66mpllk3H1tU2NNMkYm_0 2024-04-22T20:48:15.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10613863412461883525', time=2024-04-22T22:29:16.435567+0000, chain.objs.size()=1 2024-04-22T20:48:15.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.K-ikaYwsUVBHkdRfZ-QR2Gk58klyjHG_0 2024-04-22T20:48:15.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12063555592342960155', time=2024-04-22T22:29:17.107524+0000, chain.objs.size()=1 2024-04-22T20:48:15.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.710+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.csGqD4-j2UabUaS4mXupywBMVNDJtOk_0 2024-04-22T20:48:15.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.984277437706649264', time=2024-04-22T22:29:20.996340+0000, chain.objs.size()=1 2024-04-22T20:48:15.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.318__shadow_.o1WYA02f1ah3gfL_6ElCf9hR6h3NgNp_0 2024-04-22T20:48:15.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9617790317411318095', time=2024-04-22T22:29:30.560076+0000, chain.objs.size()=1 2024-04-22T20:48:15.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.331__shadow_.A4nf7Z3DsOF055F0DC6Qoxh5E7PkS8w_0 2024-04-22T20:48:15.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15128718667670142930', time=2024-04-22T22:30:30.096007+0000, chain.objs.size()=1 2024-04-22T20:48:15.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.420__shadow_.QAAt-JVH6zr4u3w9coj4M-nGSZoxVzw_0 2024-04-22T20:48:15.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10370944174204918615', time=2024-04-22T22:30:48.168014+0000, chain.objs.size()=1 2024-04-22T20:48:15.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.443__shadow_.OVF-KJzRmfvJszwJZRwzucmfd_cO1Nf_0 2024-04-22T20:48:15.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9345107171701536008', time=2024-04-22T22:31:10.510098+0000, chain.objs.size()=1 2024-04-22T20:48:15.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.vetDiX1C3ElwewSMkcLXrhHisG2XeCm_0 2024-04-22T20:48:15.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1788019470897059351', time=2024-04-22T22:31:54.688623+0000, chain.objs.size()=1 2024-04-22T20:48:15.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.500__multipart_mymultipart.2~sB_7OkhhoFMasQk0D9GcW_XEew_Rljh.1 2024-04-22T20:48:15.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14154326872486921713', time=2024-04-22T22:33:28.143490+0000, chain.objs.size()=6 2024-04-22T20:48:15.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.9Og2O5BOJZU97nEUTlY87Hmr7N-icjEl.1 2024-04-22T20:48:15.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~sT2ZJgR2b-RyAXhQK2XBFx84aKM79sZ.2 2024-04-22T20:48:15.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~sT2ZJgR2b-RyAXhQK2XBFx84aKM79sZ.3 2024-04-22T20:48:15.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~sT2ZJgR2b-RyAXhQK2XBFx84aKM79sZ.4 2024-04-22T20:48:15.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~sT2ZJgR2b-RyAXhQK2XBFx84aKM79sZ.5 2024-04-22T20:48:15.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~sT2ZJgR2b-RyAXhQK2XBFx84aKM79sZ.6 2024-04-22T20:48:15.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11770638377386217733', time=2024-04-22T22:37:05.296611+0000, chain.objs.size()=2 2024-04-22T20:48:15.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.546__shadow_.UfoIlDscdI1Re3Ho1OgRPCW_6AoXAXm_0 2024-04-22T20:48:15.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.546__shadow_.UfoIlDscdI1Re3Ho1OgRPCW_6AoXAXm_1 2024-04-22T20:48:15.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9685898672242460711', time=2024-04-22T22:37:33.634643+0000, chain.objs.size()=1 2024-04-22T20:48:15.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.558__shadow:nn1Z8qiYbVL1bmAoo8weWa34ck3vSCH_.wzD2rSy30BHRJAsRjSmlcyVmTP2isg-_0 2024-04-22T20:48:15.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2460206886911405275', time=2024-04-22T22:37:40.741749+0000, chain.objs.size()=1 2024-04-22T20:48:15.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow:3Gy72GUewTZX9T1ReL9cDvhhoUxKwmA_.SFnS4hZph-zemfoQfF4xjScdCOv0R6e_0 2024-04-22T20:48:15.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10680208501915976971', time=2024-04-22T22:38:10.677630+0000, chain.objs.size()=1 2024-04-22T20:48:15.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:z.KvkiYfkNWTsbIM44NoXNFNa1rCxhA_.C-RxcreMLZoqcmiu5_8FYkwyk9my7ZY_0 2024-04-22T20:48:15.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10969855002815083772', time=2024-04-22T22:38:57.556958+0000, chain.objs.size()=1 2024-04-22T20:48:15.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.574__shadow:pZf9A5i3ZJ5ReU.G6alF0aQgjR0A8vF_.KGCG4pS18eUvZIm-QrTQgPp9qoZdWvq_0 2024-04-22T20:48:15.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12179403842179936312', time=2024-04-22T22:42:55.899392+0000, chain.objs.size()=1 2024-04-22T20:48:15.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.650__shadow_.95k9zvUGpE3oAKTBbZW2p_g59KuqGoZ_0 2024-04-22T20:48:15.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.726+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:15.726 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.730+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-22T20:48:15.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.731+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:15.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.731+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:15.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.732+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45326' 2024-04-22T20:48:15.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.732+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1586358294815711137', time=2024-04-22T22:21:23.802608+0000, chain.objs.size()=1 2024-04-22T20:48:15.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.732+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.146__shadow:yql9P8c7BpEEd8KdYH2IZ381DfYWogG_.QRnnvvSTT7UZGhywjw3RTHg8g-OhpxN_0 2024-04-22T20:48:15.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.732+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7634516403024751899', time=2024-04-22T22:21:39.819659+0000, chain.objs.size()=5 2024-04-22T20:48:15.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.732+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-LUKEWARM.2~jQ-bBhKqCBWPlKous7e3Uugz_LQPFkh.1 2024-04-22T20:48:15.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.732+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__shadow_mymultipart-LUKEWARM.2~jQ-bBhKqCBWPlKous7e3Uugz_LQPFkh.1_1 2024-04-22T20:48:15.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.732+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-LUKEWARM.2~jQ-bBhKqCBWPlKous7e3Uugz_LQPFkh.2 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.732+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__shadow_mymultipart-LUKEWARM.2~jQ-bBhKqCBWPlKous7e3Uugz_LQPFkh.2_1 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-LUKEWARM.2~jQ-bBhKqCBWPlKous7e3Uugz_LQPFkh.3 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9835884800472664304', time=2024-04-22T22:25:22.564633+0000, chain.objs.size()=1 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.228__shadow_.sB1KQllYMt0kV9JI3HtrVP2U0X401ss_0 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12200983535210830477', time=2024-04-22T22:25:24.429850+0000, chain.objs.size()=1 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.230__shadow_.FfSeCCtR3pZQZwzg-DV5c9Pko9ZEaXS_0 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4528462700701977081', time=2024-04-22T22:26:22.335260+0000, chain.objs.size()=1 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.76BMK4-oM-vYVf8tbOUU2SMQ9gpIRo5_0 2024-04-22T20:48:15.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3157792384907342734', time=2024-04-22T22:26:22.587330+0000, chain.objs.size()=1 2024-04-22T20:48:15.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.i0ioRF6A4_AmxIKvIXmhopzgxZoHiMl_0 2024-04-22T20:48:15.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9359611248026958828', time=2024-04-22T22:26:23.048220+0000, chain.objs.size()=1 2024-04-22T20:48:15.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Zp4bhS-8ttHhIrxfN44PWAUTnTUmuet_0 2024-04-22T20:48:15.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10202381353152450527', time=2024-04-22T22:26:24.312982+0000, chain.objs.size()=1 2024-04-22T20:48:15.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cjnSfCJW44vJTwA6IG-RnGwloZ2u2pr_0 2024-04-22T20:48:15.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.735+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5792701309184109999', time=2024-04-22T22:26:24.763295+0000, chain.objs.size()=1 2024-04-22T20:48:15.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.735+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gr1U6xqZ8hnh3X7B-Uo00nBCilzHMiG_0 2024-04-22T20:48:15.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.736+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2707806179348380146', time=2024-04-22T22:26:25.017426+0000, chain.objs.size()=1 2024-04-22T20:48:15.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.736+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.H6yHU9w8tPiIV9wdcenHnStTFvF7RfX_0 2024-04-22T20:48:15.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.736+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6365815883458658373', time=2024-04-22T22:26:25.800739+0000, chain.objs.size()=1 2024-04-22T20:48:15.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.736+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MyelfvMfD8mLJ61zCT1rK_2rMvFsB4x_0 2024-04-22T20:48:15.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.737+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15241214020754630803', time=2024-04-22T22:26:26.448077+0000, chain.objs.size()=1 2024-04-22T20:48:15.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.737+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.n6qOhtD3Ex982d25yIFow3-uH5mY-lb_0 2024-04-22T20:48:15.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.737+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10105544108776904456', time=2024-04-22T22:26:26.829399+0000, chain.objs.size()=1 2024-04-22T20:48:15.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.737+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.TXVqZ8OD5zVgpx9vcS_UUF8yLMF1goW_0 2024-04-22T20:48:15.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.737+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2829646346294661675', time=2024-04-22T22:26:26.830060+0000, chain.objs.size()=1 2024-04-22T20:48:15.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.737+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xe_eXFWqlJgk1EfzssGz8m0K2F-nL_E_0 2024-04-22T20:48:15.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.737+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9535804926322055929', time=2024-04-22T22:26:26.978461+0000, chain.objs.size()=1 2024-04-22T20:48:15.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.737+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Mf3tiDzr6geyskfbOAAlUB2VxlSYHy0_0 2024-04-22T20:48:15.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14234805375393215661', time=2024-04-22T22:26:27.494630+0000, chain.objs.size()=1 2024-04-22T20:48:15.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mbxhqXrxJtiX8VVObgyb_EK7AdTK9i0_0 2024-04-22T20:48:15.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11866974410188519969', time=2024-04-22T22:26:27.888024+0000, chain.objs.size()=1 2024-04-22T20:48:15.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.G8omFhm8Q3Lq4JIOZGriIisNzH2Kxkn_0 2024-04-22T20:48:15.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2133277087327848443', time=2024-04-22T22:26:27.889162+0000, chain.objs.size()=1 2024-04-22T20:48:15.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PG4jHWkl11eTqOVIxCuOfKzuZUjZT0B_0 2024-04-22T20:48:15.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3141802476640910167', time=2024-04-22T22:26:28.347613+0000, chain.objs.size()=1 2024-04-22T20:48:15.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vhqA1QY8k2osoqqUVXFaTT7Rtm-SJ6b_0 2024-04-22T20:48:15.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7324277919715635100', time=2024-04-22T22:26:28.550057+0000, chain.objs.size()=1 2024-04-22T20:48:15.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.738+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.kNAHhcumvNrAtf5gRbHNGQUlZDvRoaz_0 2024-04-22T20:48:15.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.740+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5236484971442650020', time=2024-04-22T22:26:29.050690+0000, chain.objs.size()=1 2024-04-22T20:48:15.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.740+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.d5W3hwfcjwoQrRA2MYhc4t38kNijBeI_0 2024-04-22T20:48:15.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.740+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2935822662115806886', time=2024-04-22T22:26:29.234223+0000, chain.objs.size()=1 2024-04-22T20:48:15.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.740+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4N_lnsDn6xGNLzKpBbSGjp0FVW_6JvI_0 2024-04-22T20:48:15.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12286397724308591759', time=2024-04-22T22:26:29.656426+0000, chain.objs.size()=1 2024-04-22T20:48:15.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vR1USRfLpyEEnxUGTj1LciSDx13bzBD_0 2024-04-22T20:48:15.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13852097893700357305', time=2024-04-22T22:26:29.735054+0000, chain.objs.size()=1 2024-04-22T20:48:15.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JRn_IUsnvPNruVebNi0Axmwp64r2xSM_0 2024-04-22T20:48:15.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11717321294853596698', time=2024-04-22T22:26:30.117821+0000, chain.objs.size()=1 2024-04-22T20:48:15.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BYUm_sItLaUqN42lsEGLeKi1wG5ZTFH_0 2024-04-22T20:48:15.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6932425719663191816', time=2024-04-22T22:26:30.124457+0000, chain.objs.size()=1 2024-04-22T20:48:15.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BxzHMQIJTiAkbfhMb_FJANs6Cx64Do1_0 2024-04-22T20:48:15.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10604397827469614055', time=2024-04-22T22:26:30.729573+0000, chain.objs.size()=1 2024-04-22T20:48:15.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eIscWojyU7cl8osUpxYEJCYSZVaWZGU_0 2024-04-22T20:48:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9357196798234996503', time=2024-04-22T22:26:31.370133+0000, chain.objs.size()=1 2024-04-22T20:48:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.P_eFsC-4dttsxRqk7u97npExlIzqfb7_0 2024-04-22T20:48:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17848554210394254679', time=2024-04-22T22:26:31.510570+0000, chain.objs.size()=1 2024-04-22T20:48:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.89Gut2Fw2axmDlQys6cjp4g16HrzfwM_0 2024-04-22T20:48:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9831819721702412659', time=2024-04-22T22:26:32.211080+0000, chain.objs.size()=1 2024-04-22T20:48:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.csCFPhvBLDHGwcpDcjBLFrtNj7Ze326_0 2024-04-22T20:48:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6167612885339506768', time=2024-04-22T22:26:32.843130+0000, chain.objs.size()=1 2024-04-22T20:48:15.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.y-nsMTkiui2pidgqvxnRpSx-fKGLkRX_0 2024-04-22T20:48:15.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7395865373458387451', time=2024-04-22T22:26:33.168974+0000, chain.objs.size()=1 2024-04-22T20:48:15.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-RY792nn-xLZK5I4IbYrJIMCQ5a4eME_0 2024-04-22T20:48:15.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.743+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1656414262905306608', time=2024-04-22T22:26:33.656939+0000, chain.objs.size()=1 2024-04-22T20:48:15.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.743+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FxQmoVeuhfMUDk530mFj4gf3oQ4yFgd_0 2024-04-22T20:48:15.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3383772794577639837', time=2024-04-22T22:26:34.059810+0000, chain.objs.size()=1 2024-04-22T20:48:15.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VHlEoGe3zW4WgYfOpn0BxUBvKjxkxkm_0 2024-04-22T20:48:15.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16856459377113127463', time=2024-04-22T22:26:42.098538+0000, chain.objs.size()=1 2024-04-22T20:48:15.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.259__shadow_.QcUTHuJx7g_GVdgPaiKafilpKGOwYSa_0 2024-04-22T20:48:15.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.671665311409178046', time=2024-04-22T22:26:42.105528+0000, chain.objs.size()=1 2024-04-22T20:48:15.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.259__shadow_.JwJwYSSteDbm9AkHpdnos7eNTIzPuz3_0 2024-04-22T20:48:15.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2564889437924850551', time=2024-04-22T22:26:43.039197+0000, chain.objs.size()=1 2024-04-22T20:48:15.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.260__shadow_.dzO-CbtdJ0TiaoolCL250eLwfRIOAcR_0 2024-04-22T20:48:15.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1590968486510984102', time=2024-04-22T22:26:45.902956+0000, chain.objs.size()=1 2024-04-22T20:48:15.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.263__shadow_.TxqaDpWv1cXUN5kKT7ko6DF4q0ooScp_0 2024-04-22T20:48:15.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6326956742499228359', time=2024-04-22T22:27:07.854973+0000, chain.objs.size()=1 2024-04-22T20:48:15.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.O_Qm3a3U7KxIOqMlzWCfRhEIZFjHlCt_0 2024-04-22T20:48:15.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15942850335970311534', time=2024-04-22T22:27:13.041793+0000, chain.objs.size()=1 2024-04-22T20:48:15.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.287__shadow_.KJ0nhs_lVZit53FPDB04RUT7M1u8yg0_0 2024-04-22T20:48:15.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1085163278786470205', time=2024-04-22T22:27:16.065290+0000, chain.objs.size()=1 2024-04-22T20:48:15.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.290__shadow_.ZfN0UNFfIn9ztKPbqya6Jc9XmRprzJm_0 2024-04-22T20:48:15.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5457023335789650113', time=2024-04-22T22:27:37.290709+0000, chain.objs.size()=1 2024-04-22T20:48:15.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.308__shadow:DvBWUiAjDU97MW6RNFxqe9kNNHNdevL_.rB_IbSE57gHz1HXYt0me8OGvahE-1Hf_0 2024-04-22T20:48:15.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11247968438430683147', time=2024-04-22T22:28:17.352821+0000, chain.objs.size()=1 2024-04-22T20:48:15.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FIm51e49CAmrSruauV7SrhjayfUYUe0_0 2024-04-22T20:48:15.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6261521241894556652', time=2024-04-22T22:28:17.844126+0000, chain.objs.size()=1 2024-04-22T20:48:15.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.747+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Yn5CpokmnN_ECmn3v6UPba3qZhnx7Wz_0 2024-04-22T20:48:15.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.894806882247990964', time=2024-04-22T22:28:17.844997+0000, chain.objs.size()=1 2024-04-22T20:48:15.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WMigptfHaNxiMTfwhAkXNDBvuUExP-9_0 2024-04-22T20:48:15.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7733646714024155376', time=2024-04-22T22:28:17.988238+0000, chain.objs.size()=1 2024-04-22T20:48:15.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.t_87Wa3coA1hPrj-Ur59zZxHWVwTb4x_0 2024-04-22T20:48:15.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15753278386987339557', time=2024-04-22T22:28:18.266391+0000, chain.objs.size()=1 2024-04-22T20:48:15.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xI8HEUQx6jFX34rFcODlf1CVkGGfmb9_0 2024-04-22T20:48:15.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15616715216781954123', time=2024-04-22T22:28:18.267300+0000, chain.objs.size()=1 2024-04-22T20:48:15.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.E3gy7_EmgRJPgUwKvCUnmq414z1sc9G_0 2024-04-22T20:48:15.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6386903030080636558', time=2024-04-22T22:28:19.477746+0000, chain.objs.size()=1 2024-04-22T20:48:15.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GOw7Gv96v6ql6uuRBZuoygMx2tc0ilz_0 2024-04-22T20:48:15.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7716643516258909347', time=2024-04-22T22:28:19.721123+0000, chain.objs.size()=1 2024-04-22T20:48:15.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.g9xk6aqK7u-zwcUoswguWIRV9vCgqsO_0 2024-04-22T20:48:15.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6544851626130953677', time=2024-04-22T22:28:19.797603+0000, chain.objs.size()=1 2024-04-22T20:48:15.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Uo3hDzW6Zb3sPJlM1KHcqGcgeboost2_0 2024-04-22T20:48:15.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6897784295363711328', time=2024-04-22T22:28:20.975325+0000, chain.objs.size()=1 2024-04-22T20:48:15.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.jW4MkPyV4MmlHGR4qpUrSWmJl9aSRxL_0 2024-04-22T20:48:15.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17869405520663968051', time=2024-04-22T22:28:21.001272+0000, chain.objs.size()=1 2024-04-22T20:48:15.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bg2Ekh1RW6j9MUW0BSAxNmKyqt17CKi_0 2024-04-22T20:48:15.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11297159713027742150', time=2024-04-22T22:28:21.645070+0000, chain.objs.size()=1 2024-04-22T20:48:15.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dJ9G7edeFW06ktY9F3CPulGVvkvs5mZ_0 2024-04-22T20:48:15.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18125909436614864738', time=2024-04-22T22:28:21.803931+0000, chain.objs.size()=1 2024-04-22T20:48:15.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RFQ4KOmUSgP84yftW9Q2ao4AOBmX76y_0 2024-04-22T20:48:15.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5317296981169311674', time=2024-04-22T22:28:22.868054+0000, chain.objs.size()=1 2024-04-22T20:48:15.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.alqF-8IOTn49pt9NC47xk2_yws42ImG_0 2024-04-22T20:48:15.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14505115769327331324', time=2024-04-22T22:28:24.176063+0000, chain.objs.size()=1 2024-04-22T20:48:15.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.y8ILkJmwcwNLdE-ve7tMJCSA5gYCrOK_0 2024-04-22T20:48:15.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6552363539765992379', time=2024-04-22T22:28:24.188475+0000, chain.objs.size()=1 2024-04-22T20:48:15.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pbBoOT0o-qrwVG3JpdnWp1wv5161tGL_0 2024-04-22T20:48:15.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5936106171018121941', time=2024-04-22T22:28:24.603549+0000, chain.objs.size()=1 2024-04-22T20:48:15.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5Az8X3T0JXEvkxCH_PMM-0jn2fP-p7B_0 2024-04-22T20:48:15.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14953083318017854911', time=2024-04-22T22:28:24.696557+0000, chain.objs.size()=1 2024-04-22T20:48:15.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.755+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.DD1mKAaLFPpDR-NBEET4l6WOGldGP0O_0 2024-04-22T20:48:15.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1499042850551792717', time=2024-04-22T22:28:24.897539+0000, chain.objs.size()=1 2024-04-22T20:48:15.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.YtsDsYd-Sy4ZKkpq99ZwJ43Jefg_RLq_0 2024-04-22T20:48:15.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3498969862147203292', time=2024-04-22T22:28:25.784737+0000, chain.objs.size()=1 2024-04-22T20:48:15.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UBIINjhvtp8iXsKWOfFUufTgI-q7tB7_0 2024-04-22T20:48:15.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18092025766440277530', time=2024-04-22T22:28:25.941498+0000, chain.objs.size()=1 2024-04-22T20:48:15.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NNkyA0dH_qvooP7wrbMIFwySgTqlhMq_0 2024-04-22T20:48:15.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6977221885472881681', time=2024-04-22T22:28:26.217652+0000, chain.objs.size()=1 2024-04-22T20:48:15.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.A4rnWwJNJPUz8_9P2FgZUe_MpSmtAD8_0 2024-04-22T20:48:15.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11851418327078616550', time=2024-04-22T22:28:26.489902+0000, chain.objs.size()=1 2024-04-22T20:48:15.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.jUaOIYiZHy8MXhAu0SuT_H2uAEQmkmi_0 2024-04-22T20:48:15.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13622324210097127054', time=2024-04-22T22:28:26.984236+0000, chain.objs.size()=1 2024-04-22T20:48:15.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Blt4_Pg0qh1A1BWpksRFa3MiLqBOrqh_0 2024-04-22T20:48:15.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6565159792733561130', time=2024-04-22T22:28:27.882081+0000, chain.objs.size()=1 2024-04-22T20:48:15.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RqL8V9dXgIEPhLJ2JiRFZ6-7HrtDH_i_0 2024-04-22T20:48:15.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8009965960256165575', time=2024-04-22T22:28:28.641627+0000, chain.objs.size()=1 2024-04-22T20:48:15.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lee9e50Rh-34UbxBpYwLgT-acTuXyDI_0 2024-04-22T20:48:15.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10322487123570272565', time=2024-04-22T22:28:28.794322+0000, chain.objs.size()=1 2024-04-22T20:48:15.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ftdv2nNW2L1DMXgHOTy4lRtGP09dtL1_0 2024-04-22T20:48:15.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4231440972424164575', time=2024-04-22T22:29:06.637313+0000, chain.objs.size()=1 2024-04-22T20:48:15.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KDlrYW0Ro0kS9IQTKuijFv71oCdyFaX_0 2024-04-22T20:48:15.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10642729215897684801', time=2024-04-22T22:29:06.774845+0000, chain.objs.size()=1 2024-04-22T20:48:15.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rY3Xa60zLf26zU_8mhhPgKOc8Q1V9XM_0 2024-04-22T20:48:15.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1875593530050547555', time=2024-04-22T22:29:06.976536+0000, chain.objs.size()=1 2024-04-22T20:48:15.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.biHPbwcVT93KFlkCn0g7RTVV3yT5IC__0 2024-04-22T20:48:15.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13467372148131592428', time=2024-04-22T22:29:07.007523+0000, chain.objs.size()=1 2024-04-22T20:48:15.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MzD8zpHF9vY0pHIRGwAMx1uRFyjDCDZ_0 2024-04-22T20:48:15.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16571991908566803338', time=2024-04-22T22:29:07.025671+0000, chain.objs.size()=1 2024-04-22T20:48:15.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6eqALpznnyi6VtbazpXj5t6CRAENeh2_0 2024-04-22T20:48:15.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8014152014169452875', time=2024-04-22T22:29:07.808314+0000, chain.objs.size()=1 2024-04-22T20:48:15.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.h4bF-Wpemhr8qRg6McaQRqN6bnLJaZM_0 2024-04-22T20:48:15.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8034698587682350576', time=2024-04-22T22:29:08.904487+0000, chain.objs.size()=1 2024-04-22T20:48:15.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.d4UgqY2ZFyeLlQ00FLncApVxjMC_sXD_0 2024-04-22T20:48:15.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2918620809821139286', time=2024-04-22T22:29:09.486318+0000, chain.objs.size()=1 2024-04-22T20:48:15.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.761+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.orA0hqtvl4etkInykw3ou_f-9zjo03-_0 2024-04-22T20:48:15.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.764+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1897439079342555826', time=2024-04-22T22:29:09.753163+0000, chain.objs.size()=1 2024-04-22T20:48:15.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.764+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gLqwKOvU6sLUPArASjF_Yfxi6BBzn7U_0 2024-04-22T20:48:15.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.242488574221102578', time=2024-04-22T22:29:11.061934+0000, chain.objs.size()=1 2024-04-22T20:48:15.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JkaGejlPkP-tFSLE00uERp5NDiia9g7_0 2024-04-22T20:48:15.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10276079247933019905', time=2024-04-22T22:29:11.112081+0000, chain.objs.size()=1 2024-04-22T20:48:15.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ptrF15kF5bIdrv-bhkC5MFUyENgpQwK_0 2024-04-22T20:48:15.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17718095310226891477', time=2024-04-22T22:29:11.225007+0000, chain.objs.size()=1 2024-04-22T20:48:15.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.o7WCYTWKVn9yZlkdUsMtDVEGr25OoHK_0 2024-04-22T20:48:15.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.407090274852748298', time=2024-04-22T22:29:11.393869+0000, chain.objs.size()=1 2024-04-22T20:48:15.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wmS9aQsnHaeG6zNx99HAes43lAEMPhP_0 2024-04-22T20:48:15.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4556945396499447710', time=2024-04-22T22:29:11.994196+0000, chain.objs.size()=1 2024-04-22T20:48:15.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2_K1eKSRdvSFF1XPgQaw_d8AWhVs2-u_0 2024-04-22T20:48:15.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6833576170725529912', time=2024-04-22T22:29:12.927671+0000, chain.objs.size()=1 2024-04-22T20:48:15.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HhZZhJAq7mCLSsQuhmrC88BOwOew86Z_0 2024-04-22T20:48:15.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5953708459132275445', time=2024-04-22T22:29:13.236018+0000, chain.objs.size()=1 2024-04-22T20:48:15.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hKhI68N2SyXNLPuWrmT3G3xhYx01SzW_0 2024-04-22T20:48:15.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14795044009400902619', time=2024-04-22T22:29:13.237380+0000, chain.objs.size()=1 2024-04-22T20:48:15.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PJp7yUQylsJBBCIJHv_XzARmDIpa_IG_0 2024-04-22T20:48:15.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.972060457953807704', time=2024-04-22T22:29:14.060781+0000, chain.objs.size()=1 2024-04-22T20:48:15.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OghRpyJIPtmJEVsjte8pLlPI9ZRajiM_0 2024-04-22T20:48:15.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12009171298722651281', time=2024-04-22T22:29:14.221366+0000, chain.objs.size()=1 2024-04-22T20:48:15.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Gqv-oMJ7y8ZilMBlRZxB-vi_xgo0EtT_0 2024-04-22T20:48:15.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.929819020960687836', time=2024-04-22T22:29:14.488217+0000, chain.objs.size()=1 2024-04-22T20:48:15.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.lmzp7ezz77jv8ThQgtvGvV4DrJZ1afH_0 2024-04-22T20:48:15.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16374517404878856082', time=2024-04-22T22:29:14.594928+0000, chain.objs.size()=1 2024-04-22T20:48:15.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gxD9lJnyQhuhV04ATt9pGokk2a5gxbt_0 2024-04-22T20:48:15.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8256930666309245334', time=2024-04-22T22:29:14.686254+0000, chain.objs.size()=1 2024-04-22T20:48:15.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7PB0f3viWtWzbku1ulOv4yBw_Tvn5Wb_0 2024-04-22T20:48:15.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.556513691391718236', time=2024-04-22T22:29:14.811933+0000, chain.objs.size()=1 2024-04-22T20:48:15.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RHcrC8Qziiqe8iBJBsWnGomxVMbBBcu_0 2024-04-22T20:48:15.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13217421989096084462', time=2024-04-22T22:29:16.155953+0000, chain.objs.size()=1 2024-04-22T20:48:15.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HvF7wQVIRL64UPOCjUXj7Km44vclccA_0 2024-04-22T20:48:15.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2034120308385361741', time=2024-04-22T22:29:16.343663+0000, chain.objs.size()=1 2024-04-22T20:48:15.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.saPSC_PKMVryBAtrTlI_qjPIYm4DIkq_0 2024-04-22T20:48:15.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7288410326344935713', time=2024-04-22T22:29:16.344901+0000, chain.objs.size()=1 2024-04-22T20:48:15.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.e9r2enf0iWoNRC-3dpJAHyiWBA6n0pG_0 2024-04-22T20:48:15.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17644856048528101109', time=2024-04-22T22:29:16.439150+0000, chain.objs.size()=1 2024-04-22T20:48:15.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YBoXwSFX2e4JJP_1YikTcEpM-YRVldo_0 2024-04-22T20:48:15.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10246804038568323738', time=2024-04-22T22:29:16.774470+0000, chain.objs.size()=1 2024-04-22T20:48:15.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Vel58JaMtmN0GQtllWziFbt8T7tnDSP_0 2024-04-22T20:48:15.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5536574528672784166', time=2024-04-22T22:29:17.257800+0000, chain.objs.size()=1 2024-04-22T20:48:15.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tTVgKg9jQAMKjhsoscBITctAKp6cBxx_0 2024-04-22T20:48:15.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12669395157217630075', time=2024-04-22T22:29:17.307713+0000, chain.objs.size()=1 2024-04-22T20:48:15.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.68PmekAjAUhir5R4xKTg2A4JKwHxqFp_0 2024-04-22T20:48:15.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16114697153370717048', time=2024-04-22T22:29:17.489572+0000, chain.objs.size()=1 2024-04-22T20:48:15.772 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.772+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GAW61QCyE2VTynJZpEdWG7ddgKll1vq_0 2024-04-22T20:48:15.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.782+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/45326 2024-04-22T20:48:15.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=15, truncated=0, next_marker='' 2024-04-22T20:48:15.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9396272125917563186', time=2024-04-22T22:30:15.699466+0000, chain.objs.size()=1 2024-04-22T20:48:15.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.397__shadow_.Gn4ekP3blSQX5TE0u-cTdTzjUVnbVgZ_0 2024-04-22T20:48:15.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12702336542848398619', time=2024-04-22T22:30:16.403097+0000, chain.objs.size()=1 2024-04-22T20:48:15.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.398__shadow_.mDH-urB5PQAvIvFI_hRwSIlg0XVHt9Q_0 2024-04-22T20:48:15.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14480937274328070243', time=2024-04-22T22:31:10.510497+0000, chain.objs.size()=1 2024-04-22T20:48:15.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.59nvZVcFbETcqjXkeCkGwWdzNjk5lBX_0 2024-04-22T20:48:15.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.475__shadow_.Cw43RM8kCmoHPhj_R79zmKaFTLJQkNI_1_802a09705d993a59cb18d622a9027339', time=2024-04-22T22:31:15.560868+0000, chain.objs.size()=1 2024-04-22T20:48:15.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.475__shadow_.Cw43RM8kCmoHPhj_R79zmKaFTLJQkNI_0 2024-04-22T20:48:15.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17906354286022817036', time=2024-04-22T22:37:00.295597+0000, chain.objs.size()=1 2024-04-22T20:48:15.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.545__shadow_.r_JYsAsXw4XooeMsRolvJekWvICB_E__0 2024-04-22T20:48:15.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12413518828946668979', time=2024-04-22T22:37:48.712679+0000, chain.objs.size()=1 2024-04-22T20:48:15.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:.OFYZ8UX5tOX1E0xbPjuWN6oUyRNFwm_.ZoGrdh0Ia5a24kG1nPIPYxtWb_pW_g1_0 2024-04-22T20:48:15.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11624006032826172707', time=2024-04-22T22:37:57.310192+0000, chain.objs.size()=1 2024-04-22T20:48:15.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:0Mv2xQ4a3.af1OcA7FKLnPLEjFUu3rH_.SXYbBFr_B9b3g0Xljcv3Z8JLhkvSlUR_0 2024-04-22T20:48:15.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12485800324770858200', time=2024-04-22T22:38:03.165917+0000, chain.objs.size()=1 2024-04-22T20:48:15.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:CQsFTteY2DNA8SNZ9Rr5Vwy1Utb0AL-_.Bvd_GzU58fOU2HDoXz1-fHppnXoI2B5_0 2024-04-22T20:48:15.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11857733436563630675', time=2024-04-22T22:38:03.740877+0000, chain.objs.size()=1 2024-04-22T20:48:15.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:jlR1JntUDftxSrRoIs50NrqdTzw6Z-A_.qIRSKAPmjaiAB0DgOrKEfk8G-fPqlT7_0 2024-04-22T20:48:15.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17743891613955018679', time=2024-04-22T22:38:49.721665+0000, chain.objs.size()=1 2024-04-22T20:48:15.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:bSIc2NeOukJ2v9sDuDulhHber9X6dpU_.peocg6z2kLtDLSGgn3rJZ3vPMGflbWH_0 2024-04-22T20:48:15.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:qVjJ5iTGVxG9xodzTPYYDMlFBVXKTEK_.IzdXg9s7RTEeaFnlFpPPoAEZ3j1jBIA_1_4432d06e3ab3f4bd902e31b399227a44', time=2024-04-22T22:38:52.193331+0000, chain.objs.size()=1 2024-04-22T20:48:15.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:qVjJ5iTGVxG9xodzTPYYDMlFBVXKTEK_.IzdXg9s7RTEeaFnlFpPPoAEZ3j1jBIA_0 2024-04-22T20:48:15.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17963694483019337643', time=2024-04-22T22:38:56.361823+0000, chain.objs.size()=1 2024-04-22T20:48:15.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.573__shadow:GvassFhzY4Yp3t1FrLd9vtXTx2cMijM_.NLSL_xAoDQKmUkrIVdZwf5c9nZ_bMXb_0 2024-04-22T20:48:15.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.788+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16956546113946849415', time=2024-04-22T22:42:12.013338+0000, chain.objs.size()=1 2024-04-22T20:48:15.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.788+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.631__shadow_.VEUNB6o-kxzVQcTBpmjkYZIKw9lHYSU_0 2024-04-22T20:48:15.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.788+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.652__shadow_.9-wutzRoPo9HiH2AFTpwHTFHeiHtdg__1_fc39144e6ffc089371f8f93ba536a825', time=2024-04-22T22:42:57.253197+0000, chain.objs.size()=1 2024-04-22T20:48:15.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.788+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.652__shadow_.9-wutzRoPo9HiH2AFTpwHTFHeiHtdg__0 2024-04-22T20:48:15.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.788+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14534450902544668003', time=2024-04-22T22:43:19.976430+0000, chain.objs.size()=1 2024-04-22T20:48:15.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.788+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.690__shadow_.9r4l8h3Zb4Z_pLIEWy3t_FuUklgrQKi_0 2024-04-22T20:48:15.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.791+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:15.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.793+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-22T20:48:15.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.794+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:15.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:15.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43451' 2024-04-22T20:48:15.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17274728740454697485', time=2024-04-22T22:21:17.946152+0000, chain.objs.size()=1 2024-04-22T20:48:15.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.140__shadow_.qEaAI6Kq9Z76aH3rbaleTK9fh3Dbd5P_0 2024-04-22T20:48:15.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.294272193252666326', time=2024-04-22T22:22:52.358494+0000, chain.objs.size()=1 2024-04-22T20:48:15.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.171__shadow_.yQKNAs2mz1wUJ1zdNnpBaF7LnrGwz3B_0 2024-04-22T20:48:15.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4714951612134436079', time=2024-04-22T22:22:58.467214+0000, chain.objs.size()=1 2024-04-22T20:48:15.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.178__shadow_.sKx5H5IJMqGjHgKrH8cd0dvDY8yZEvT_0 2024-04-22T20:48:15.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8571692096960214714', time=2024-04-22T22:26:22.330107+0000, chain.objs.size()=1 2024-04-22T20:48:15.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pi7O8B0J8T-Ej3wbTebwVx2ytoprfQJ_0 2024-04-22T20:48:15.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2001060309862055842', time=2024-04-22T22:26:22.335251+0000, chain.objs.size()=1 2024-04-22T20:48:15.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Zg5phGqLHVUDR5IzpgKZTAb-K0N4QtO_0 2024-04-22T20:48:15.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13975468788232118999', time=2024-04-22T22:26:23.198452+0000, chain.objs.size()=1 2024-04-22T20:48:15.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ELquRqvEkHfcrBWhsfefWUUHswXmbZv_0 2024-04-22T20:48:15.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2025748509798869179', time=2024-04-22T22:26:23.389827+0000, chain.objs.size()=1 2024-04-22T20:48:15.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uoxneJ-gOeG1veA2oKh7Ro5qOp_iwXR_0 2024-04-22T20:48:15.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2923950100633151531', time=2024-04-22T22:26:24.007792+0000, chain.objs.size()=1 2024-04-22T20:48:15.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.opkAmqjtHRkcnPBF-8OBF9mSkTsxFJ3_0 2024-04-22T20:48:15.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1873438212695764187', time=2024-04-22T22:26:24.150909+0000, chain.objs.size()=1 2024-04-22T20:48:15.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.e3hnJkg2mRD_i7RyLKdh__akOJb0ldZ_0 2024-04-22T20:48:15.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.283845550267601260', time=2024-04-22T22:26:24.494713+0000, chain.objs.size()=1 2024-04-22T20:48:15.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gzPtTEwKw98FyyMOyx-dTUD6cd_jy_t_0 2024-04-22T20:48:15.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8311100708104359304', time=2024-04-22T22:26:24.534678+0000, chain.objs.size()=1 2024-04-22T20:48:15.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.M7xwYUW1rLBm0lU7Ggg0B3yl3sU3s5Y_0 2024-04-22T20:48:15.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1261011626123705382', time=2024-04-22T22:26:24.802855+0000, chain.objs.size()=1 2024-04-22T20:48:15.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3IyShhUWebT-3wmwXdo8GE9NBoBX_-f_0 2024-04-22T20:48:15.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2133084319097168545', time=2024-04-22T22:26:25.887037+0000, chain.objs.size()=1 2024-04-22T20:48:15.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SpMFuRx_3qTd7ZyXVdzNpiGxrYANsGo_0 2024-04-22T20:48:15.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7812809413337822658', time=2024-04-22T22:26:25.973634+0000, chain.objs.size()=1 2024-04-22T20:48:15.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lPP6j_wYC0ub_iY4UmyiY-CsE_yEg67_0 2024-04-22T20:48:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9541841406213278310', time=2024-04-22T22:26:26.301102+0000, chain.objs.size()=1 2024-04-22T20:48:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.t1s-aEYMtFzwONJ935C1ZmjiSvc5ZJg_0 2024-04-22T20:48:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8701775062140777095', time=2024-04-22T22:26:26.829966+0000, chain.objs.size()=1 2024-04-22T20:48:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.y6jfqaPXjb5Umex2MdYKJrm_sUtKcjC_0 2024-04-22T20:48:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2562943233426694864', time=2024-04-22T22:26:27.722418+0000, chain.objs.size()=1 2024-04-22T20:48:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XGR-q6BMx3xmjv2uIFNszzjXlhSguVW_0 2024-04-22T20:48:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10966514028985127350', time=2024-04-22T22:26:27.988374+0000, chain.objs.size()=1 2024-04-22T20:48:15.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OYaE-evw6Fif7RvQQ0TceeLnacVVv2L_0 2024-04-22T20:48:15.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7064900669633460883', time=2024-04-22T22:26:28.204097+0000, chain.objs.size()=1 2024-04-22T20:48:15.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zTZFzomNjDPEkEPpmSjmpemfFTxJExB_0 2024-04-22T20:48:15.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5172277645674637120', time=2024-04-22T22:26:28.496326+0000, chain.objs.size()=1 2024-04-22T20:48:15.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hkwCiU7h1er5P33RMiLoCOO1pBOZ66t_0 2024-04-22T20:48:15.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8402902158407337799', time=2024-04-22T22:26:29.791035+0000, chain.objs.size()=1 2024-04-22T20:48:15.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hGetgR-T4pOVYcG1sDa6v57cDOhfoJW_0 2024-04-22T20:48:15.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3317674966178237970', time=2024-04-22T22:26:30.562415+0000, chain.objs.size()=1 2024-04-22T20:48:15.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.inyM654CCZxUaBuyzMzPGL6kgUxHDSK_0 2024-04-22T20:48:15.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12766091119685890535', time=2024-04-22T22:26:30.729547+0000, chain.objs.size()=1 2024-04-22T20:48:15.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.94VxWR0IpQPaP3OROTnt9my_bN1ufSs_0 2024-04-22T20:48:15.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.803+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10357551130861427552', time=2024-04-22T22:26:30.921598+0000, chain.objs.size()=1 2024-04-22T20:48:15.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.803+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.i2-PtVfaixnuqT4AxCD7NOu6_6CHKK__0 2024-04-22T20:48:15.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.803+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15633996247418323833', time=2024-04-22T22:26:31.186825+0000, chain.objs.size()=1 2024-04-22T20:48:15.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.803+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._S0kD00AK7jOf1AP2HWyZNMaLDTC--R_0 2024-04-22T20:48:15.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4392184082896632765', time=2024-04-22T22:26:32.501563+0000, chain.objs.size()=1 2024-04-22T20:48:15.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7CdyVqwpCXm9GdTimI_iAAlZ8Lq0IN2_0 2024-04-22T20:48:15.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1200180967790678217', time=2024-04-22T22:26:32.606233+0000, chain.objs.size()=1 2024-04-22T20:48:15.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.t2RgPoktzdZY3U_hlSZdtroXwVFHCQI_0 2024-04-22T20:48:15.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16695830642514786053', time=2024-04-22T22:26:34.043218+0000, chain.objs.size()=1 2024-04-22T20:48:15.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.G1gOWsd3xEW0BtGID5vEge0H6DxnG-I_0 2024-04-22T20:48:15.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11160985921162010123', time=2024-04-22T22:26:34.090571+0000, chain.objs.size()=1 2024-04-22T20:48:15.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8XjhixzEz64IAIjjZxJK5YZU0AtlNcL_0 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9031815567670683359', time=2024-04-22T22:26:35.415194+0000, chain.objs.size()=1 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.252__shadow_.g4pWjYVHZ1JvKu08H13wVBd6WTQ9l1-_0 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.517405307234678524', time=2024-04-22T22:26:36.369209+0000, chain.objs.size()=1 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.253__shadow_.NCH5opHaT2hNoi8u1yg2m3VTgaoBXkk_0 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5822437054315969728', time=2024-04-22T22:26:39.258428+0000, chain.objs.size()=1 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.256__shadow_.CqAvYDltqtyIf_OfcqZFlZwu6WS3lfu_0 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11407780998301791059', time=2024-04-22T22:26:47.893945+0000, chain.objs.size()=1 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.265__shadow_.wWCrV3gv3aa4SqVsD2iWz8vpbEP9_H2_0 2024-04-22T20:48:15.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9867406406981356475', time=2024-04-22T22:26:48.871055+0000, chain.objs.size()=1 2024-04-22T20:48:15.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.266__shadow_.xWWhJ2eA99WPJGwot2Gbo1VUC9qvFWy_0 2024-04-22T20:48:15.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5723330144629002592', time=2024-04-22T22:26:53.870314+0000, chain.objs.size()=1 2024-04-22T20:48:15.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.271__shadow_.lUQGAi8Po-RPGtqMIpxNc48QLs1sY5u_0 2024-04-22T20:48:15.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7102035822999127097', time=2024-04-22T22:27:05.567594+0000, chain.objs.size()=1 2024-04-22T20:48:15.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.upNeHqLSWy8etCbgy4m3foyLR3JWdwM_0 2024-04-22T20:48:15.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.809+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11405240853160814191', time=2024-04-22T22:27:05.707247+0000, chain.objs.size()=1 2024-04-22T20:48:15.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.809+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.5j9qtNC7vVmLb6Qm37ukK33bpS8Kpfc_0 2024-04-22T20:48:15.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.809+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2291721562707801404', time=2024-04-22T22:27:15.080349+0000, chain.objs.size()=1 2024-04-22T20:48:15.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.809+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.289__shadow_.7KpNZC3cFpTVSbXanC4UrUfiWO8YUvf_0 2024-04-22T20:48:15.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7763192413955926417', time=2024-04-22T22:28:18.437897+0000, chain.objs.size()=1 2024-04-22T20:48:15.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BtD19wUF1Hznc-4IwhFjdE0_gU3u744_0 2024-04-22T20:48:15.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3191599150943207559', time=2024-04-22T22:28:18.529433+0000, chain.objs.size()=1 2024-04-22T20:48:15.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Lcz-8d8ZwLUlLmZ7K1bB_5eoNM5rMOO_0 2024-04-22T20:48:15.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14086875193229333767', time=2024-04-22T22:28:19.082998+0000, chain.objs.size()=1 2024-04-22T20:48:15.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.aSDcPxWaIUu1usHbrEK4VfZYqljUvVE_0 2024-04-22T20:48:15.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16451999938493003482', time=2024-04-22T22:28:19.188203+0000, chain.objs.size()=1 2024-04-22T20:48:15.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RTcR7sJ1kwl66G6UVNC04_nHV2cUxUc_0 2024-04-22T20:48:15.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1337568286392479138', time=2024-04-22T22:28:19.242598+0000, chain.objs.size()=1 2024-04-22T20:48:15.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SWzOgYFgngd_A52UN_LpyEB6jQRbiWV_0 2024-04-22T20:48:15.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15317210838392776112', time=2024-04-22T22:28:20.716254+0000, chain.objs.size()=1 2024-04-22T20:48:15.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0_0InUsZ5cRq1QauNWhfUgrkoQdzgNE_0 2024-04-22T20:48:15.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12786694436603274661', time=2024-04-22T22:28:20.928027+0000, chain.objs.size()=1 2024-04-22T20:48:15.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.QQlB1IolOiBD0D1aoMy5qs5tfMKR5Ps_0 2024-04-22T20:48:15.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7462502340436567198', time=2024-04-22T22:28:21.239929+0000, chain.objs.size()=1 2024-04-22T20:48:15.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XchaSt3x43jmazPoJ7EoiIlxLH1bN79_0 2024-04-22T20:48:15.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.811+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12665884215169320062', time=2024-04-22T22:28:21.327194+0000, chain.objs.size()=1 2024-04-22T20:48:15.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.811+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6sLbFFyjd9H1_uPs9ZUORhFFyOnZL2s_0 2024-04-22T20:48:15.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8928897299949501975', time=2024-04-22T22:28:21.505585+0000, chain.objs.size()=1 2024-04-22T20:48:15.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Et_sHdw-_M9Ib8UOaz1v_EPeE4wlxYt_0 2024-04-22T20:48:15.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.813+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9181456233945497395', time=2024-04-22T22:28:21.675702+0000, chain.objs.size()=1 2024-04-22T20:48:15.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.813+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.85kDEdp2go8EYqFLd9R39a384sl2Z_g_0 2024-04-22T20:48:15.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.814+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17142945554067901461', time=2024-04-22T22:28:21.802748+0000, chain.objs.size()=1 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gnE_K-IWI68vDZvj4fYKyW4OBtRLHy1_0 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8802318838610925125', time=2024-04-22T22:28:21.855390+0000, chain.objs.size()=1 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WmC-I8Smx-GshdLClqNn4GGkEC1LWLq_0 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15980870021150130689', time=2024-04-22T22:28:22.457149+0000, chain.objs.size()=1 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZbSy8iI41HADDHauedUEi3RSbzzOwL5_0 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5550166636718283323', time=2024-04-22T22:28:22.491554+0000, chain.objs.size()=1 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3I_t7fdKiE5puWxDQWTMe8uYhGflHYt_0 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5133996931814739009', time=2024-04-22T22:28:22.881675+0000, chain.objs.size()=1 2024-04-22T20:48:15.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ESwj5VxHgAPIUFtnq09_hMYI3PozBUl_0 2024-04-22T20:48:15.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6839921141204343002', time=2024-04-22T22:28:23.198499+0000, chain.objs.size()=1 2024-04-22T20:48:15.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3JRhNDpEXaraqJjbYhhFf_CSaRQMhzl_0 2024-04-22T20:48:15.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5548149614097672249', time=2024-04-22T22:28:23.866020+0000, chain.objs.size()=1 2024-04-22T20:48:15.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4xrKO1v0KiixPzlR8K-_yJfrWCOJQ9c_0 2024-04-22T20:48:15.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15441776537751657446', time=2024-04-22T22:28:23.882094+0000, chain.objs.size()=1 2024-04-22T20:48:15.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SYrxWMxdJR1ceAqFbB-SWJbiuKPYbi3_0 2024-04-22T20:48:15.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14288809628005840772', time=2024-04-22T22:28:24.897480+0000, chain.objs.size()=1 2024-04-22T20:48:15.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.VAZJSiJnCW5X-9RnWxRUVmxWEulCcaM_0 2024-04-22T20:48:15.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10443102117885545387', time=2024-04-22T22:28:25.403406+0000, chain.objs.size()=1 2024-04-22T20:48:15.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.s95ui-WPOxpdcJdt_dr6ks1V9b5YsJf_0 2024-04-22T20:48:15.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1242019666843826565', time=2024-04-22T22:28:25.519987+0000, chain.objs.size()=1 2024-04-22T20:48:15.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.815+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.r3XvfpG3XcgFaMWOzuBUiuUzc6ijBTP_0 2024-04-22T20:48:15.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11939367324778728995', time=2024-04-22T22:28:25.642910+0000, chain.objs.size()=1 2024-04-22T20:48:15.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.DHzrq0EExxvvdxqXFWbze59THvI-3Es_0 2024-04-22T20:48:15.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9625645004331196004', time=2024-04-22T22:28:26.246424+0000, chain.objs.size()=1 2024-04-22T20:48:15.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SLd5FEZ2KApnfyO6cbXqGEF2HgYlD0W_0 2024-04-22T20:48:15.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12551625966077844129', time=2024-04-22T22:28:26.547378+0000, chain.objs.size()=1 2024-04-22T20:48:15.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TGKQ3OAoMCyevkU-mvqJke5Dj0dXb2x_0 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4169250658539243678', time=2024-04-22T22:28:26.879301+0000, chain.objs.size()=1 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qDyTPiVYABs_msWGVhag_idE4yL3WvL_0 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16550176982631271653', time=2024-04-22T22:28:27.092942+0000, chain.objs.size()=1 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LOgjfJI-d_jXZvoh7Q1jqOmqSh-aWQu_0 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17749413314365118365', time=2024-04-22T22:28:27.739479+0000, chain.objs.size()=1 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zgl0C2W9rLJSXjPOR3jxzfId-i2LYga_0 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8672596468821461398', time=2024-04-22T22:28:28.518028+0000, chain.objs.size()=1 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1cacGiej6cglQEKhXEO3iQPbwNtNrqO_0 2024-04-22T20:48:15.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.742662376474858478', time=2024-04-22T22:28:28.520154+0000, chain.objs.size()=1 2024-04-22T20:48:15.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fOA39CvwWbVYEwyMfkc0UNlX0UBAaOe_0 2024-04-22T20:48:15.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7857482716351578681', time=2024-04-22T22:28:28.948673+0000, chain.objs.size()=1 2024-04-22T20:48:15.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dsSGb7TqbDZ6aFJ7OkUrNH90edx_BPt_0 2024-04-22T20:48:15.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16610243667678076844', time=2024-04-22T22:29:06.177645+0000, chain.objs.size()=1 2024-04-22T20:48:15.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.V6dvIW6yEumSJMwru2yqWCHX_8fgAvP_0 2024-04-22T20:48:15.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3138905569397143250', time=2024-04-22T22:29:06.245952+0000, chain.objs.size()=1 2024-04-22T20:48:15.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nvMQefgCprxxBUZ2outDsKaLDgjakwi_0 2024-04-22T20:48:15.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9459565172695148189', time=2024-04-22T22:29:06.459290+0000, chain.objs.size()=1 2024-04-22T20:48:15.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wo9EQYC6CHbA5iEVmop9uSI3t79J6FW_0 2024-04-22T20:48:15.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.825+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14605808161418845572', time=2024-04-22T22:29:06.953245+0000, chain.objs.size()=1 2024-04-22T20:48:15.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.825+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wr6RuRlmJNd4-aLPSnyR_IWAiaJuG0d_0 2024-04-22T20:48:15.824 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11253497481144831640', time=2024-04-22T22:29:07.297768+0000, chain.objs.size()=1 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Tw5Kl5VTL_v5yFjOKACrndDNmtJxUgV_0 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10478835032912835332', time=2024-04-22T22:29:07.962393+0000, chain.objs.size()=1 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KUtuoIdvCDDR5YRFhsr_SLuKx067390_0 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11834198167342587125', time=2024-04-22T22:29:08.052920+0000, chain.objs.size()=1 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kJdnCxlmcOOj8BXZul9DcgtF7sNGz6P_0 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.352280630112681008', time=2024-04-22T22:29:08.791062+0000, chain.objs.size()=1 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PkYL_Dr6XVtIeK3qJ1w-_l1UUjg3j13_0 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2450719006569415953', time=2024-04-22T22:29:08.853637+0000, chain.objs.size()=1 2024-04-22T20:48:15.825 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tzr2T0vwTA3unB6E8kpNxRQcmECZhT8_0 2024-04-22T20:48:15.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5901501088868807572', time=2024-04-22T22:29:09.413193+0000, chain.objs.size()=1 2024-04-22T20:48:15.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gqFcGRbDhwdIMJqjrhmJBiXlOEU4yaC_0 2024-04-22T20:48:15.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11035311686364223871', time=2024-04-22T22:29:09.717115+0000, chain.objs.size()=1 2024-04-22T20:48:15.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YRbV50G6aebJMqqLmFSQOdOwRXgwVMv_0 2024-04-22T20:48:15.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2299272043303475533', time=2024-04-22T22:29:09.888054+0000, chain.objs.size()=1 2024-04-22T20:48:15.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MP7BXUlItPxvHmurPplGTBag2GyTGWc_0 2024-04-22T20:48:15.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5001300108540150535', time=2024-04-22T22:29:10.070262+0000, chain.objs.size()=1 2024-04-22T20:48:15.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.b1Romn7P-NjyZmU81zVa-GGhq3jo7dr_0 2024-04-22T20:48:15.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8511704916675180489', time=2024-04-22T22:29:10.071055+0000, chain.objs.size()=1 2024-04-22T20:48:15.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.828+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Yd6LTMSniWaCF9e1Oi4K4CfTaVsSjjC_0 2024-04-22T20:48:15.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.831+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.348185417616072867', time=2024-04-22T22:29:10.349436+0000, chain.objs.size()=1 2024-04-22T20:48:15.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.831+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0HlCMW-v2ykzOKS0opkfi8IJ6l9BraZ_0 2024-04-22T20:48:15.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.831+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9505538312635679629', time=2024-04-22T22:29:10.385429+0000, chain.objs.size()=1 2024-04-22T20:48:15.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.831+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mChf5uykrLojmykVQE07hGMpQCY0BMg_0 2024-04-22T20:48:15.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.831+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7769439776244149751', time=2024-04-22T22:29:10.876637+0000, chain.objs.size()=1 2024-04-22T20:48:15.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.831+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.e64prH2JJsL2jc7mzqhm5G0AuPrCiNj_0 2024-04-22T20:48:15.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2967952041317738011', time=2024-04-22T22:29:10.967469+0000, chain.objs.size()=1 2024-04-22T20:48:15.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QuQpovnFs-wcyyOtbcZyPo4IDZR3jaE_0 2024-04-22T20:48:15.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17167662846193351589', time=2024-04-22T22:29:11.269179+0000, chain.objs.size()=1 2024-04-22T20:48:15.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.a3JJz4Obcy5vXMJqrAvghXzIgkTnb1L_0 2024-04-22T20:48:15.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15540704897808460803', time=2024-04-22T22:29:11.269916+0000, chain.objs.size()=1 2024-04-22T20:48:15.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.eubMI94K6srG33TU8VTKa0QUrYLHfR5_0 2024-04-22T20:48:15.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1328624599845334058', time=2024-04-22T22:29:11.393899+0000, chain.objs.size()=1 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DE2q06xTgpgm9W60DfJWu0rxSXoLs4q_0 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17860255375570755198', time=2024-04-22T22:29:11.527473+0000, chain.objs.size()=1 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WgDVMyxL6vOZxM2waLsK2jpTARVLMJ2_0 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2821392393943044747', time=2024-04-22T22:29:11.725469+0000, chain.objs.size()=1 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.D1YiPcEO5PFsWxGW8rP-eePLrsQXs6b_0 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13943714293889283984', time=2024-04-22T22:29:11.800810+0000, chain.objs.size()=1 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yzMLS5bZLkKiM9BGuqZK7uqfO4Eqo9e_0 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12348269849493859393', time=2024-04-22T22:29:11.801490+0000, chain.objs.size()=1 2024-04-22T20:48:15.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.834+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1x_WlFqipz3-u6-XnsLBnvRU5Fdr_QN_0 2024-04-22T20:48:15.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.836+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.79820865252536334', time=2024-04-22T22:29:11.951672+0000, chain.objs.size()=1 2024-04-22T20:48:15.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.836+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ripLEk73Zdk7JuagiCSO_V-2CquPiOR_0 2024-04-22T20:48:15.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.836+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6462129395423709998', time=2024-04-22T22:29:12.991408+0000, chain.objs.size()=1 2024-04-22T20:48:15.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.836+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ekO3XZ2CfBCAVyohfvw6TLQBF-UqT3v_0 2024-04-22T20:48:15.833 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.836+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9697657044683528697', time=2024-04-22T22:29:13.199933+0000, chain.objs.size()=1 2024-04-22T20:48:15.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.836+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iE8EZqF911X8LUvoB-0EXyTTybRkdSn_0 2024-04-22T20:48:15.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.839+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7608477822590501337', time=2024-04-22T22:29:13.269084+0000, chain.objs.size()=1 2024-04-22T20:48:15.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.839+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7N7wGYt5a9Zs9-_ttCTQ-bDRR84Qe_Z_0 2024-04-22T20:48:15.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.839+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11553942365947453231', time=2024-04-22T22:29:13.918292+0000, chain.objs.size()=1 2024-04-22T20:48:15.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.839+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iP7NTQnrsUccahg2yMraBAdbDkL9c8D_0 2024-04-22T20:48:15.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.839+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.211663357011859212', time=2024-04-22T22:29:14.220126+0000, chain.objs.size()=1 2024-04-22T20:48:15.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.839+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LeulrRTVE3-N-BVukCv9pU2Vn6na2Wo_0 2024-04-22T20:48:15.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.844+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/43451 2024-04-22T20:48:15.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=30, truncated=0, next_marker='' 2024-04-22T20:48:15.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15063313741029437390', time=2024-04-22T22:29:14.530219+0000, chain.objs.size()=1 2024-04-22T20:48:15.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bg7DDB6JM9wPt4wYSbsldaXdtna9RP-_0 2024-04-22T20:48:15.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13017682755052773941', time=2024-04-22T22:29:14.633423+0000, chain.objs.size()=1 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.c6u9OHTN_LYJCaMWSxHcqwRnEGjewrL_0 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8559830283717326624', time=2024-04-22T22:29:14.799965+0000, chain.objs.size()=1 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ujoOoJnRfN5LyYndTYGVcilw16UKDQd_0 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17008879473766893892', time=2024-04-22T22:29:14.885902+0000, chain.objs.size()=1 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.c_crTy7LRVjHbRBAz1iIAMrc3clvvKf_0 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12032926023162504160', time=2024-04-22T22:29:14.886630+0000, chain.objs.size()=1 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PM5KSlfC0l6KqDuBTW-HYI7RWjb_C8W_0 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1707188844488953834', time=2024-04-22T22:29:14.979318+0000, chain.objs.size()=1 2024-04-22T20:48:15.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SolLIErgnVBJF4Qgz37htRN0r2PEg_5_0 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6643814932462611897', time=2024-04-22T22:29:15.746723+0000, chain.objs.size()=1 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.05N8ZhdOQgVSboc5cxZX3JiQxuKbQOB_0 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18359991334694617581', time=2024-04-22T22:29:15.996548+0000, chain.objs.size()=1 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-Baxwi9eQ8NjfhgNso3HkY1jOfa0b2j_0 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4911795448626798142', time=2024-04-22T22:29:16.003627+0000, chain.objs.size()=1 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RpwOPXChcPDZ81xuYObVNpt0qm5uYLl_0 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10600076309227751515', time=2024-04-22T22:29:16.407908+0000, chain.objs.size()=1 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OzxvfSL4iPfiVBZxyrrKTkQ4zVTJTUx_0 2024-04-22T20:48:15.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2701519481294359211', time=2024-04-22T22:29:16.759773+0000, chain.objs.size()=1 2024-04-22T20:48:15.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.q74TrX8wSDA1n8nnsLWGf8TTbBTBEGQ_0 2024-04-22T20:48:15.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11981584437034217732', time=2024-04-22T22:29:16.965291+0000, chain.objs.size()=1 2024-04-22T20:48:15.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tNKDyB3cMHdOotuMrEZ-TEOboHN7uJ1_0 2024-04-22T20:48:15.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.852+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3738983957496180836', time=2024-04-22T22:29:17.206247+0000, chain.objs.size()=1 2024-04-22T20:48:15.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.852+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.a_9U6SZvsFlo_AnI15gz0NTFUfko2iL_0 2024-04-22T20:48:15.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15656804813223524030', time=2024-04-22T22:29:17.422254+0000, chain.objs.size()=1 2024-04-22T20:48:15.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TJFCh8MliWy0OCFgy7d7NXulTfN6r6K_0 2024-04-22T20:48:15.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17507188957862267448', time=2024-04-22T22:29:32.483147+0000, chain.objs.size()=1 2024-04-22T20:48:15.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.334__shadow_.D799s9K68otQQjlB-_ZOXGA-r9OrQ4H_0 2024-04-22T20:48:15.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.89908591863592587', time=2024-04-22T22:29:48.212185+0000, chain.objs.size()=1 2024-04-22T20:48:15.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.360__shadow_.F6sj9HZKGIWwatWC1uEjT5zp8zgmEwl_0 2024-04-22T20:48:15.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2265016164055339847', time=2024-04-22T22:31:52.840539+0000, chain.objs.size()=6 2024-04-22T20:48:15.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.1 2024-04-22T20:48:15.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2 2024-04-22T20:48:15.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.3 2024-04-22T20:48:15.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.4 2024-04-22T20:48:15.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.5 2024-04-22T20:48:15.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.6 2024-04-22T20:48:15.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1783347753313807936', time=2024-04-22T22:35:01.918670+0000, chain.objs.size()=2 2024-04-22T20:48:15.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~9O2-otNxGGvwQiPg9v6WtUqUskBEOCq.1 2024-04-22T20:48:15.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~9O2-otNxGGvwQiPg9v6WtUqUskBEOCq.2 2024-04-22T20:48:15.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.862+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17982149788378479507', time=2024-04-22T22:37:27.437322+0000, chain.objs.size()=1 2024-04-22T20:48:15.859 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.862+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:hEbbxrrmSUP7dvXilQb7df6Gh4cO.lX_.wZSkkriDDv8Zk9Pb_MNDIeVKbyuPy3k_0 2024-04-22T20:48:15.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.325780279098506409', time=2024-04-22T22:37:28.570153+0000, chain.objs.size()=1 2024-04-22T20:48:15.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:hnm86hHQVNuy7FKQQXmdQnULBSfGt63_.N0bSTKjzwRflcbNCXHuqcR4X3RoWsn3_0 2024-04-22T20:48:15.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18276943575131205897', time=2024-04-22T22:37:52.057132+0000, chain.objs.size()=1 2024-04-22T20:48:15.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:AJ8GpkRGB5AllqtTn3-wu38t1ipjMMr_.5VzuRbL_iVHg9yysLRAlx1OXVoxMTVi_0 2024-04-22T20:48:15.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.408718288503767075', time=2024-04-22T22:37:54.224591+0000, chain.objs.size()=1 2024-04-22T20:48:15.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:oAXtplpGaTin0J2veQO6caodth2KSx8_.jr4O1bHloqM9NCGorwWqYShWigDu52h_0 2024-04-22T20:48:15.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1308564514642557618', time=2024-04-22T22:38:04.697231+0000, chain.objs.size()=1 2024-04-22T20:48:15.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:rb8zOTHw2uxqwY2f2xbtll3Egpt4l-U_.iVkUGEJ_i42ruJ3ZoyVLVKgeF52KUpk_0 2024-04-22T20:48:15.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15527190997434459000', time=2024-04-22T22:38:07.780152+0000, chain.objs.size()=1 2024-04-22T20:48:15.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:iwInQsTbOP1YzIjfMWcTrjdd41dyTcZ_.SQRgPJLBeXPwk3qJLtun9DGyQXULasc_0 2024-04-22T20:48:15.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14020113278232297121', time=2024-04-22T22:38:59.166392+0000, chain.objs.size()=1 2024-04-22T20:48:15.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.863+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:YKuaKg-5lAE-5d4sqXbIJ6v8AafRjAb_.beVowjIv807d1IRKPcHba0eWyVmervL_0 2024-04-22T20:48:15.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6341436567102018644', time=2024-04-22T22:38:59.478874+0000, chain.objs.size()=1 2024-04-22T20:48:15.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:waxyOEw3DU1n.Acp0LgjdjgNMwrlQef_.0_2JjgcXkJHazBMUMTnKTzhXhue019G_0 2024-04-22T20:48:15.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4344560406320331484', time=2024-04-22T22:39:04.977122+0000, chain.objs.size()=1 2024-04-22T20:48:15.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.584__shadow_.DoKfgLhJnZTERxALweUaIDpu3DZrKF9_0 2024-04-22T20:48:15.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.941154486137512920', time=2024-04-22T22:41:48.088918+0000, chain.objs.size()=1 2024-04-22T20:48:15.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.612__shadow_.JfUY7ThWdswu0Ej1G4IKdAHhodKZhrW_0 2024-04-22T20:48:15.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.555884338466410279', time=2024-04-22T22:43:14.954521+0000, chain.objs.size()=1 2024-04-22T20:48:15.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.683__shadow:yhOdHl83fBCBuuy9YzFQDTadYaZp1Tc_.jJt_hOrWDgjD2ugRmOfhwgmr0_usdKc_0 2024-04-22T20:48:15.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.868+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1529075829073868529', time=2024-04-22T22:43:18.497642+0000, chain.objs.size()=1 2024-04-22T20:48:15.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.868+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.688__shadow:.rXmroS6aM7YD4SwgEHjgbwM1-KNf8p_.A0dxsKzJNcqRsERIVK7AzpsbKY2O6IB_0 2024-04-22T20:48:15.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:15.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.880+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-22T20:48:15.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.882+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:15.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.882+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:15.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/46944' 2024-04-22T20:48:15.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_pTXlnZ6pVoPpEVKLsFujR7L2Hi9iImt', time=2024-04-22T22:22:08.028774+0000, chain.objs.size()=3 2024-04-22T20:48:15.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.77BBU1FGM7Raw2EnwcFt64DCwzleeOx_0 2024-04-22T20:48:15.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.77BBU1FGM7Raw2EnwcFt64DCwzleeOx_1 2024-04-22T20:48:15.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.77BBU1FGM7Raw2EnwcFt64DCwzleeOx_2 2024-04-22T20:48:15.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_mMfVsoCX4YkLzDvagI5d_d7Rcadng2f', time=2024-04-22T22:22:16.184120+0000, chain.objs.size()=3 2024-04-22T20:48:15.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.z7mubPbeas-J5Ngx4rju9gssj4WS6Qc_0 2024-04-22T20:48:15.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.z7mubPbeas-J5Ngx4rju9gssj4WS6Qc_1 2024-04-22T20:48:15.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.z7mubPbeas-J5Ngx4rju9gssj4WS6Qc_2 2024-04-22T20:48:15.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11296900131762968516', time=2024-04-22T22:25:12.292068+0000, chain.objs.size()=1 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.219__shadow_.zVnT1aVEbjHY068dps3dUU1ieFXsSyw_0 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11005163792945390770', time=2024-04-22T22:25:13.299790+0000, chain.objs.size()=1 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.220__shadow_._S7R_2WqAM1T2g3R1odbba7BwR3znqZ_0 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11513711509073568631', time=2024-04-22T22:25:19.951049+0000, chain.objs.size()=1 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.226__shadow_.hESKnVXABGpi2Fel6KNkJKx0q4haPdc_0 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11229704331479373232', time=2024-04-22T22:25:24.437949+0000, chain.objs.size()=1 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.230__shadow_.TP8bBJQaxxqeUVU4Vg6t9hCjzmiCwbI_0 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1408506849250818944', time=2024-04-22T22:25:45.673118+0000, chain.objs.size()=1 2024-04-22T20:48:15.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.250__shadow_.R7PvmZLspVB2qKTSBvTyYCYYbFhlvVQ_0 2024-04-22T20:48:15.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5004265641308393520', time=2024-04-22T22:26:22.345735+0000, chain.objs.size()=1 2024-04-22T20:48:15.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.884+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CliKcxbuRFZXuf-BCVOZIed6tBAaAyA_0 2024-04-22T20:48:15.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15858861659820852941', time=2024-04-22T22:26:22.588212+0000, chain.objs.size()=1 2024-04-22T20:48:15.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.p-WVCNThtrJYzFh8UlIRh2g0bmOhWvO_0 2024-04-22T20:48:15.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17760688922261816980', time=2024-04-22T22:26:23.077675+0000, chain.objs.size()=1 2024-04-22T20:48:15.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pNuOHJR2mUzDagQ30MsuCdiumLrMYQJ_0 2024-04-22T20:48:15.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3932267775112363671', time=2024-04-22T22:26:23.078818+0000, chain.objs.size()=1 2024-04-22T20:48:15.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JHLUYjGHsCEfLZYWkyorg_2e669S875_0 2024-04-22T20:48:15.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4050097948822530264', time=2024-04-22T22:26:23.079410+0000, chain.objs.size()=1 2024-04-22T20:48:15.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CSBKfh_uXeEE1PQ_MGadW4FuMW6vy0Q_0 2024-04-22T20:48:15.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18224349197368567923', time=2024-04-22T22:26:23.431361+0000, chain.objs.size()=1 2024-04-22T20:48:15.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XGXuOXxWh1Tfl0s4OKOI0d3Cvm_sxNU_0 2024-04-22T20:48:15.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3169717944553677676', time=2024-04-22T22:26:23.470219+0000, chain.objs.size()=1 2024-04-22T20:48:15.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jajRv5RjhC2JJ1BUJzzUPxzEy-PBRoj_0 2024-04-22T20:48:15.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15373155896687418068', time=2024-04-22T22:26:24.401509+0000, chain.objs.size()=1 2024-04-22T20:48:15.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.fX4N3DijBNbr3PTSMunm1WW8XiFU-Nh_0 2024-04-22T20:48:15.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2461366550041095217', time=2024-04-22T22:26:24.733938+0000, chain.objs.size()=1 2024-04-22T20:48:15.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ISImra70_WB7aXwYsyXJHG_9ShzJjWk_0 2024-04-22T20:48:15.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15894795546530553244', time=2024-04-22T22:26:25.118119+0000, chain.objs.size()=1 2024-04-22T20:48:15.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.S_x18dtaS2DhTRVMlQXPdSysr6SmxzU_0 2024-04-22T20:48:15.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13516966931621302384', time=2024-04-22T22:26:26.447192+0000, chain.objs.size()=1 2024-04-22T20:48:15.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.IXmbfRIKtuV2Wr_fpKL1-cqRsSdH1kJ_0 2024-04-22T20:48:15.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15862454385128931359', time=2024-04-22T22:26:26.462575+0000, chain.objs.size()=1 2024-04-22T20:48:15.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DgqHNy9mA6M9CLFDWqMdV0CiGjiFK2u_0 2024-04-22T20:48:15.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1477767922195186500', time=2024-04-22T22:26:27.634479+0000, chain.objs.size()=1 2024-04-22T20:48:15.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Gw30Kdpf2wEcWwVM8kENHxJQFVlnaTP_0 2024-04-22T20:48:15.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10347189791217637763', time=2024-04-22T22:26:27.814960+0000, chain.objs.size()=1 2024-04-22T20:48:15.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bsve9SpxTEbHtHbDk0x6hIKuQ7p8bv7_0 2024-04-22T20:48:15.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10543149460970940701', time=2024-04-22T22:26:27.908055+0000, chain.objs.size()=1 2024-04-22T20:48:15.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jo3Zg58j9vmL2aGqLqVQgHhgjlUPMJo_0 2024-04-22T20:48:15.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16566176223934672958', time=2024-04-22T22:26:28.080957+0000, chain.objs.size()=1 2024-04-22T20:48:15.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vwQtEbJfccU_EmtaX5IzYZMWn38CTvw_0 2024-04-22T20:48:15.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.892+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12504497539613545961', time=2024-04-22T22:26:28.346430+0000, chain.objs.size()=1 2024-04-22T20:48:15.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.892+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5Nxq6VuBrMG6zjr71ULza-4y-Y8A9md_0 2024-04-22T20:48:15.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.892+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3594234421073418484', time=2024-04-22T22:26:29.576879+0000, chain.objs.size()=1 2024-04-22T20:48:15.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.892+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CGPdqow8wx2LjOVhXk_kOPq_3g3Di3b_0 2024-04-22T20:48:15.893 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.894+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5390901204127604839', time=2024-04-22T22:26:30.834482+0000, chain.objs.size()=1 2024-04-22T20:48:15.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.894+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PUIkarSY548973A-JXzSQcPzDOOuPtc_0 2024-04-22T20:48:15.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.895+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14475955339566863907', time=2024-04-22T22:26:30.835185+0000, chain.objs.size()=1 2024-04-22T20:48:15.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.895+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oEXysJj9S5H2H_V2uBcKF_uQrMN6ftQ_0 2024-04-22T20:48:15.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.895+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8025396659404620136', time=2024-04-22T22:26:30.858062+0000, chain.objs.size()=1 2024-04-22T20:48:15.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.895+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wBgwEWzQd84hTFqrPD1LvAENSaMXgN4_0 2024-04-22T20:48:15.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.895+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.858719910968481750', time=2024-04-22T22:26:30.920558+0000, chain.objs.size()=1 2024-04-22T20:48:15.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.895+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OMKwlsjOMMPaNZaC5ilxW_mBT8ZC3sS_0 2024-04-22T20:48:15.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5387986416795083050', time=2024-04-22T22:26:30.999392+0000, chain.objs.size()=1 2024-04-22T20:48:15.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HO4rweOGKjyXMjSwwKlG4z2-NKkk63K_0 2024-04-22T20:48:15.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16269402128229096788', time=2024-04-22T22:26:31.329561+0000, chain.objs.size()=1 2024-04-22T20:48:15.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XOV8w8F3wRkad4nqGVpgf7Ql6H-Rynr_0 2024-04-22T20:48:15.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17426714543796836012', time=2024-04-22T22:26:32.061697+0000, chain.objs.size()=1 2024-04-22T20:48:15.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pKC8luEMFmq5DAerCefKYQJ0ZnCKR5t_0 2024-04-22T20:48:15.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16230860778681719426', time=2024-04-22T22:26:32.236624+0000, chain.objs.size()=1 2024-04-22T20:48:15.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dKxolYV2K32edbjovV8h3n0UMxxrMrf_0 2024-04-22T20:48:15.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2831825750014525891', time=2024-04-22T22:26:32.354277+0000, chain.objs.size()=1 2024-04-22T20:48:15.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aQdLBcY2iPYkavgmy8J2M8BBPcFx7OT_0 2024-04-22T20:48:15.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13568859946473442405', time=2024-04-22T22:26:32.524385+0000, chain.objs.size()=1 2024-04-22T20:48:15.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.I7B4-5cyf7juuAzx9XuKawLRQ3SO_Sw_0 2024-04-22T20:48:15.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.897+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7330560060773694769', time=2024-04-22T22:26:33.838729+0000, chain.objs.size()=1 2024-04-22T20:48:15.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.897+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.s6JlY6elP33g0W_HpC6FZocdamuT_Md_0 2024-04-22T20:48:15.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.898+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4728845405338573582', time=2024-04-22T22:26:34.058559+0000, chain.objs.size()=1 2024-04-22T20:48:15.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.898+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DSbRWDmJsb8OAOsdEyStXcdpFQr1RK8_0 2024-04-22T20:48:15.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.898+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13860462768745372589', time=2024-04-22T22:26:34.090480+0000, chain.objs.size()=1 2024-04-22T20:48:15.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.898+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MNjFb9wCpX1jX3o5lIPDqQgacftVfWr_0 2024-04-22T20:48:15.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.899+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16824221358010762715', time=2024-04-22T22:26:34.136121+0000, chain.objs.size()=1 2024-04-22T20:48:15.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.899+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JRYk4P0nAB9N4VK3HzujytbXPrPOGhM_0 2024-04-22T20:48:15.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.899+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2433801889212327976', time=2024-04-22T22:26:43.034389+0000, chain.objs.size()=1 2024-04-22T20:48:15.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.899+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.260__shadow_.g3jeic5QeDmMtcfTNTQmWLYyEkv5Gbc_0 2024-04-22T20:48:15.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15040507294807184304', time=2024-04-22T22:26:46.914394+0000, chain.objs.size()=1 2024-04-22T20:48:15.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.264__shadow_.55uGUlA_gDTrYtBDB6ISwn9xU3yXkNW_0 2024-04-22T20:48:15.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10427337924801725838', time=2024-04-22T22:26:54.857503+0000, chain.objs.size()=1 2024-04-22T20:48:15.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.272__shadow_.i17t-5rBV59kvA4mq8Vc6Py0pXpMjMd_0 2024-04-22T20:48:15.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10941686312386383962', time=2024-04-22T22:27:15.084127+0000, chain.objs.size()=1 2024-04-22T20:48:15.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.289__shadow_.Z3wM-mG00712BjQnid_g4wD5oe8H4uo_0 2024-04-22T20:48:15.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15317839364530392858', time=2024-04-22T22:27:17.061039+0000, chain.objs.size()=1 2024-04-22T20:48:15.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.291__shadow_.8tw625BXjGZuSvNR48dU6lzlDq4bp4n_0 2024-04-22T20:48:15.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17409620764462482443', time=2024-04-22T22:28:17.775680+0000, chain.objs.size()=1 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bQeLIqV4FxJnGmGhgIl_T93-x2LQ8c5_0 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6376273882097837692', time=2024-04-22T22:28:18.068864+0000, chain.objs.size()=1 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.O1TvLnwB8o5fhY9qq9KpClzrRtfDo4Y_0 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16300625088292274802', time=2024-04-22T22:28:18.133458+0000, chain.objs.size()=1 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.93cUPr_XLKfk7XrSr8kZbjnuQeg3QZp_0 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9465473163339266674', time=2024-04-22T22:28:19.407153+0000, chain.objs.size()=1 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rNDet38vj4d3yqT5RHDS7w8X5mu9mwK_0 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1848760441021150618', time=2024-04-22T22:28:19.942944+0000, chain.objs.size()=1 2024-04-22T20:48:15.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.901+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vIc_Ui0r8sPgNNX6YZIqUiw1K0qdQi4_0 2024-04-22T20:48:15.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.903+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13546270472959474980', time=2024-04-22T22:28:20.116726+0000, chain.objs.size()=1 2024-04-22T20:48:15.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.903+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rzWgEVeCIgBMOIcp3WGLgo2D88AE6tH_0 2024-04-22T20:48:15.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.903+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1217130662729638204', time=2024-04-22T22:28:21.442280+0000, chain.objs.size()=1 2024-04-22T20:48:15.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.903+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Eth26QixmWlfTOW7pEDYkxDROw510_m_0 2024-04-22T20:48:15.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4542196773559922709', time=2024-04-22T22:28:21.592147+0000, chain.objs.size()=1 2024-04-22T20:48:15.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FdwVY3IeWGW6nqajG1IQQPM7sg3TATE_0 2024-04-22T20:48:15.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13827238117067317467', time=2024-04-22T22:28:21.645069+0000, chain.objs.size()=1 2024-04-22T20:48:15.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wnsC7mQODGgwcZhHQNMO2dgHILxV7Lc_0 2024-04-22T20:48:15.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.905+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12175170356424143601', time=2024-04-22T22:28:22.324410+0000, chain.objs.size()=1 2024-04-22T20:48:15.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.905+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zYYc9Whiuy7garbyghAkE6oG6yUgmV1_0 2024-04-22T20:48:15.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.905+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6813568415813808982', time=2024-04-22T22:28:22.799799+0000, chain.objs.size()=1 2024-04-22T20:48:15.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.905+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-QOmv3kCDnt5403VkS_mWvGxHipoOqW_0 2024-04-22T20:48:15.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.905+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6236231295150775479', time=2024-04-22T22:28:22.938140+0000, chain.objs.size()=1 2024-04-22T20:48:15.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.905+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.T0iHE0ENEjkX5zWoIL3tD4voEYOCrQv_0 2024-04-22T20:48:15.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.905+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7373443392094768114', time=2024-04-22T22:28:23.126321+0000, chain.objs.size()=1 2024-04-22T20:48:15.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.905+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1NMJaMdPZI98IqKd-wKF9NNYMSDw2gg_0 2024-04-22T20:48:15.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.907+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6710866111475700349', time=2024-04-22T22:28:23.856713+0000, chain.objs.size()=1 2024-04-22T20:48:15.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.907+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5tNG_7JC8PyF3Kak3k1upuAsI8lIJYc_0 2024-04-22T20:48:15.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.907+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15422268596390402254', time=2024-04-22T22:28:24.086927+0000, chain.objs.size()=1 2024-04-22T20:48:15.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.907+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OUlVOtSCof0tsUCcfggmNlM8c_2SFDr_0 2024-04-22T20:48:15.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.907+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1479675284852648776', time=2024-04-22T22:28:24.120997+0000, chain.objs.size()=1 2024-04-22T20:48:15.905 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.907+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.237P-hRey_7nFSNLO32v5oduze3briM_0 2024-04-22T20:48:15.905 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.217656294183896565', time=2024-04-22T22:28:24.123513+0000, chain.objs.size()=1 2024-04-22T20:48:15.905 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FP4KcCG810WgtFIzxi8T2qXyDozU8Sn_0 2024-04-22T20:48:15.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17644301703498632342', time=2024-04-22T22:28:24.476986+0000, chain.objs.size()=1 2024-04-22T20:48:15.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bizwT1zr_E1S9sP7v8RlDk3L2ZaISAo_0 2024-04-22T20:48:15.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13775465506123075797', time=2024-04-22T22:28:25.022522+0000, chain.objs.size()=1 2024-04-22T20:48:15.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JpXbgLZcuUg4CMsbEBMQe-XpBG_ryPr_0 2024-04-22T20:48:15.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9197249190650367620', time=2024-04-22T22:28:25.600193+0000, chain.objs.size()=1 2024-04-22T20:48:15.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gIHY7NFOZ8wMh_7W6sY2hCySkIwwkao_0 2024-04-22T20:48:15.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13970407859814249511', time=2024-04-22T22:28:25.604323+0000, chain.objs.size()=1 2024-04-22T20:48:15.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tIShREOsET7_LWkPns_jXFY26ZmnAYZ_0 2024-04-22T20:48:15.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16999040526493332683', time=2024-04-22T22:28:25.710567+0000, chain.objs.size()=1 2024-04-22T20:48:15.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LG7sqpLSyS_pu8RTwQLjitJgNZaZTQV_0 2024-04-22T20:48:15.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9707094808807429552', time=2024-04-22T22:28:25.797565+0000, chain.objs.size()=1 2024-04-22T20:48:15.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1e5t296p6P6-qQL9piEpwFyXnO2azn1_0 2024-04-22T20:48:15.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14160312634859945593', time=2024-04-22T22:28:25.941625+0000, chain.objs.size()=1 2024-04-22T20:48:15.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.d6DPXBWiStQ5l1cTTQpijRpuFkXdKIq_0 2024-04-22T20:48:15.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17925245435188930674', time=2024-04-22T22:28:26.247608+0000, chain.objs.size()=1 2024-04-22T20:48:15.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3CpBaGRsEkayIPOWxE6cMBBQ52IVvea_0 2024-04-22T20:48:15.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7048877535240019500', time=2024-04-22T22:28:26.329976+0000, chain.objs.size()=1 2024-04-22T20:48:15.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.911+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bIGbDvxr-o8OEAt2cgtKja1Z-rC3-hs_0 2024-04-22T20:48:15.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.913+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12155841542370690851', time=2024-04-22T22:28:27.514828+0000, chain.objs.size()=1 2024-04-22T20:48:15.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.913+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BJGBqpeE9hw2wVUv6D69oiv4S-1VFiK_0 2024-04-22T20:48:15.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.913+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13772465344910028187', time=2024-04-22T22:28:28.041323+0000, chain.objs.size()=1 2024-04-22T20:48:15.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.913+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JZhBgJ_sdr9Zc2g1OPU0lbNX_eWj5Lb_0 2024-04-22T20:48:15.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9908736835627238365', time=2024-04-22T22:28:28.644088+0000, chain.objs.size()=1 2024-04-22T20:48:15.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.2WAAU0MEPWEbIiurHvVUu88VkATLqS6_0 2024-04-22T20:48:15.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6208450892131498952', time=2024-04-22T22:28:29.254164+0000, chain.objs.size()=1 2024-04-22T20:48:15.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gtmjVUUuyzm-gPpU4M1Tz8om-kfUa1I_0 2024-04-22T20:48:15.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7511652243437399156', time=2024-04-22T22:29:06.177590+0000, chain.objs.size()=1 2024-04-22T20:48:15.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WnDrA9XPp11qhGfyNoim2KJXb0AobKv_0 2024-04-22T20:48:15.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12883535125569564150', time=2024-04-22T22:29:06.365620+0000, chain.objs.size()=1 2024-04-22T20:48:15.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.9b4uj_WyLcdEU9erHBqmurof2HwwSb8_0 2024-04-22T20:48:15.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.916+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.64324435410233155', time=2024-04-22T22:29:06.366809+0000, chain.objs.size()=1 2024-04-22T20:48:15.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.916+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jdwvc0YMLWiMJqSD5BJe7F_qGBoHMFT_0 2024-04-22T20:48:15.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.916+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8032980759119885980', time=2024-04-22T22:29:06.500599+0000, chain.objs.size()=1 2024-04-22T20:48:15.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.916+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.lhINjX0rAOJL93SsRCMsuvYR9TsqdyX_0 2024-04-22T20:48:15.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17278240895840780249', time=2024-04-22T22:29:06.501297+0000, chain.objs.size()=1 2024-04-22T20:48:15.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8hSu427N6uY4Xp5bIpy5vYD-Sqg-uaE_0 2024-04-22T20:48:15.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8554808957355310909', time=2024-04-22T22:29:07.040526+0000, chain.objs.size()=1 2024-04-22T20:48:15.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SO3tBiyK3Mm1YhaSKHRXVUtUrjcFskC_0 2024-04-22T20:48:15.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14978606221315793424', time=2024-04-22T22:29:07.201986+0000, chain.objs.size()=1 2024-04-22T20:48:15.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ML0guP3R5Jjh8D_CvEXOHqdjCAdqNrF_0 2024-04-22T20:48:15.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5020831350281337583', time=2024-04-22T22:29:08.433463+0000, chain.objs.size()=1 2024-04-22T20:48:15.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.K0w8GoVu5XQgAs6VB7-wtOQ6rraAgzQ_0 2024-04-22T20:48:15.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17976463039673564053', time=2024-04-22T22:29:08.679319+0000, chain.objs.size()=1 2024-04-22T20:48:15.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dTZ5X9K9wdI8PDCy8AF0Tv-thhtW4GD_0 2024-04-22T20:48:15.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5258683073736526848', time=2024-04-22T22:29:08.884424+0000, chain.objs.size()=1 2024-04-22T20:48:15.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6xGMCENxrfIC_qsrIMzGFVLnz2bC00D_0 2024-04-22T20:48:15.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10841112591169902494', time=2024-04-22T22:29:10.045269+0000, chain.objs.size()=1 2024-04-22T20:48:15.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nodjXNMFG5uh_c4mvas6dnVCT9Zwm9A_0 2024-04-22T20:48:15.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11609108270275187762', time=2024-04-22T22:29:10.068501+0000, chain.objs.size()=1 2024-04-22T20:48:15.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Lcei4SyU_QGneXoqNZtCcnmud4KDne7_0 2024-04-22T20:48:15.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5067526308074164995', time=2024-04-22T22:29:10.069284+0000, chain.objs.size()=1 2024-04-22T20:48:15.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yP0StskvfkfRCDj5B96GtXFeXWOpSn__0 2024-04-22T20:48:15.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.921+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9870276972984550088', time=2024-04-22T22:29:10.896285+0000, chain.objs.size()=1 2024-04-22T20:48:15.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.921+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yQOqxUjny0dPNRxhkMvXqCsOrY-mp23_0 2024-04-22T20:48:15.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.921+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13339017673853277400', time=2024-04-22T22:29:11.112082+0000, chain.objs.size()=1 2024-04-22T20:48:15.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.921+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Wu1joj-gBLWVsy3gZPnuNtDseFFHlSs_0 2024-04-22T20:48:15.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.922+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4472067852732438375', time=2024-04-22T22:29:11.779142+0000, chain.objs.size()=1 2024-04-22T20:48:15.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.922+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6u-Ce44g3zqnuv1g0kFROGkAhdvL7a5_0 2024-04-22T20:48:15.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8948241683381255781', time=2024-04-22T22:29:12.658526+0000, chain.objs.size()=1 2024-04-22T20:48:15.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3d8neURMgZ8xZ2u0nFw4zcc98nfiqkd_0 2024-04-22T20:48:15.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18007978813571009070', time=2024-04-22T22:29:12.792533+0000, chain.objs.size()=1 2024-04-22T20:48:15.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.lTwk8wN-ANZQlSOk-s6XHYoG7mTb3NE_0 2024-04-22T20:48:15.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17720267932140325119', time=2024-04-22T22:29:13.237337+0000, chain.objs.size()=1 2024-04-22T20:48:15.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Avh9y6lHaYW_IfUd7QJuCvLDS4IjfSp_0 2024-04-22T20:48:15.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3791365513668908798', time=2024-04-22T22:29:13.238194+0000, chain.objs.size()=1 2024-04-22T20:48:15.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Z3TNJuxvswvISlTf6LhjTExB8P6JEHz_0 2024-04-22T20:48:15.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6907224234905571072', time=2024-04-22T22:29:13.408899+0000, chain.objs.size()=1 2024-04-22T20:48:15.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.j_jKXfQaVTPA3LefICJB-sKiBuoSr4d_0 2024-04-22T20:48:15.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14185404788221986848', time=2024-04-22T22:29:14.114233+0000, chain.objs.size()=1 2024-04-22T20:48:15.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Mb6l5Vl38CwkyPSIbhl9qu3htEc1fF8_0 2024-04-22T20:48:15.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3041703333697071234', time=2024-04-22T22:29:14.354668+0000, chain.objs.size()=1 2024-04-22T20:48:15.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.adR8vdl8nZB2MDJhK6OUP82s8h6ItZ2_0 2024-04-22T20:48:15.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1623003334890457814', time=2024-04-22T22:29:14.665921+0000, chain.objs.size()=1 2024-04-22T20:48:15.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ugM_A8dIclMBhkXfM3-mF9ZC5pt8KuF_0 2024-04-22T20:48:15.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.926+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11790278323452818382', time=2024-04-22T22:29:14.755732+0000, chain.objs.size()=1 2024-04-22T20:48:15.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.926+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GadxXDrBcg118-sNp_ShD3qRyJxlZwv_0 2024-04-22T20:48:15.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.926+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8312233516740884481', time=2024-04-22T22:29:14.818363+0000, chain.objs.size()=1 2024-04-22T20:48:15.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mgtIdz_cuiQcCYLRDruOXCZO9nsCXtt_0 2024-04-22T20:48:15.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.930+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/46944 2024-04-22T20:48:15.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=21, truncated=0, next_marker='' 2024-04-22T20:48:15.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11121295719002206415', time=2024-04-22T22:29:15.902328+0000, chain.objs.size()=1 2024-04-22T20:48:15.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OPPq904FNQfnBX2uDMt3Am4gW3lUHgj_0 2024-04-22T20:48:15.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.866578699614224889', time=2024-04-22T22:29:16.003604+0000, chain.objs.size()=1 2024-04-22T20:48:15.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-AdAXbNZ3yIRnpISNP2YS0ltuuLXbLe_0 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16360626871580406233', time=2024-04-22T22:29:16.105221+0000, chain.objs.size()=1 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OcTy01kP28REDW3MF6YvXI7T7jDY5Et_0 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16361277589987704281', time=2024-04-22T22:29:16.155954+0000, chain.objs.size()=1 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1J4_fMPbKNAPnHvIHsloTjiTYRgRVMe_0 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11281630868813601389', time=2024-04-22T22:29:16.423277+0000, chain.objs.size()=1 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TCq1gxR8FTX9Sp2m9Zo3I0dvKRlil2w_0 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1168755034574684455', time=2024-04-22T22:29:56.284026+0000, chain.objs.size()=1 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.370__shadow_.L7dWIADYKW31muLnW4WsDJL276XLgxC_0 2024-04-22T20:48:15.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16167099033487715396', time=2024-04-22T22:30:02.594295+0000, chain.objs.size()=1 2024-04-22T20:48:15.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.379__shadow_.BdvtCqoRwK5LUN2btXRFnUtcgr_g_cc_0 2024-04-22T20:48:15.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2238105741387129927', time=2024-04-22T22:30:15.123224+0000, chain.objs.size()=1 2024-04-22T20:48:15.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.396__shadow_.BRcKQ8b0ScwTgfIO-yifop3NfuJs7uF_0 2024-04-22T20:48:15.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18152121885128308345', time=2024-04-22T22:31:01.156017+0000, chain.objs.size()=1 2024-04-22T20:48:15.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.455__shadow_._WKRpMsx_vCUiBrQK3NyjouiftGrNOQ_0 2024-04-22T20:48:15.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3439564529929939405', time=2024-04-22T22:31:10.502155+0000, chain.objs.size()=1 2024-04-22T20:48:15.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.ENTvSomT-QX0iw-W9CI_0c7oHBvuWNh_0 2024-04-22T20:48:15.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.486__shadow_.HUx_WbM68znI7__I0BODOKzAROauthW_1_24cd523eed9038687c773bacb1a1b260', time=2024-04-22T22:31:23.447434+0000, chain.objs.size()=1 2024-04-22T20:48:15.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.486__shadow_.HUx_WbM68znI7__I0BODOKzAROauthW_0 2024-04-22T20:48:15.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2_374288145088ae027eeee8295e3d6ae3', time=2024-04-22T22:31:52.840466+0000, chain.objs.size()=6 2024-04-22T20:48:15.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.1 2024-04-22T20:48:15.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.942+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2 2024-04-22T20:48:15.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.3 2024-04-22T20:48:15.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.4 2024-04-22T20:48:15.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.5 2024-04-22T20:48:15.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.6 2024-04-22T20:48:15.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13009934283505301412', time=2024-04-22T22:32:56.204437+0000, chain.objs.size()=4 2024-04-22T20:48:15.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.509__shadow:HotXA5t9QF5d.sqdgJvRZG75E85eQzG_.Sw3grBkOBjok746rPc5Rew4DEZWn9ny_0 2024-04-22T20:48:15.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.509__shadow:HotXA5t9QF5d.sqdgJvRZG75E85eQzG_.Sw3grBkOBjok746rPc5Rew4DEZWn9ny_1 2024-04-22T20:48:15.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.946+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.509__shadow:HotXA5t9QF5d.sqdgJvRZG75E85eQzG_.Sw3grBkOBjok746rPc5Rew4DEZWn9ny_2 2024-04-22T20:48:15.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.946+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.509__shadow:HotXA5t9QF5d.sqdgJvRZG75E85eQzG_.Sw3grBkOBjok746rPc5Rew4DEZWn9ny_3 2024-04-22T20:48:15.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.946+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9558289898168979468', time=2024-04-22T22:37:26.999219+0000, chain.objs.size()=1 2024-04-22T20:48:15.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.946+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:wDiKJDhNMHyDFjlQx0VvMPs.3hdJyxa_.5-ZjZqJthIW_XbBHry12LFlZrS40420_0 2024-04-22T20:48:15.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.946+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2723670261268270537', time=2024-04-22T22:37:58.523975+0000, chain.objs.size()=1 2024-04-22T20:48:15.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.946+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:bzLnY2EiJU7amMFwVB.1eUWMb2sdFTM_.vjoczgrPj-eMgH3jXrreFaNVfXqo_6U_0 2024-04-22T20:48:15.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.948+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:qVjJ5iTGVxG9xodzTPYYDMlFBVXKTEK_.IzdXg9s7RTEeaFnlFpPPoAEZ3j1jBIA_1_ad323b8c3bcd6c2132a11eba8b2600ed', time=2024-04-22T22:38:52.190069+0000, chain.objs.size()=1 2024-04-22T20:48:15.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.948+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:qVjJ5iTGVxG9xodzTPYYDMlFBVXKTEK_.IzdXg9s7RTEeaFnlFpPPoAEZ3j1jBIA_0 2024-04-22T20:48:15.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1260097400820805616', time=2024-04-22T22:38:56.292268+0000, chain.objs.size()=1 2024-04-22T20:48:15.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.573__shadow:fj11.cEV652ZAGcrhLT4tcekwCxu7pG_.4tPaJBUJNfS6OYApx4fHM3THi9cv_2E_0 2024-04-22T20:48:15.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15360681646380307108', time=2024-04-22T22:39:08.293393+0000, chain.objs.size()=1 2024-04-22T20:48:15.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.590__shadow_.gR-jcsr7e0teyUer6SOi-xxGhEbXpAL_0 2024-04-22T20:48:15.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18162113345736541161', time=2024-04-22T22:41:51.159635+0000, chain.objs.size()=1 2024-04-22T20:48:15.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.616__shadow_.1tx-dQVMnjotR2P_N3mALVvMeT5URHc_0 2024-04-22T20:48:15.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3365591551078826186', time=2024-04-22T22:43:13.637263+0000, chain.objs.size()=1 2024-04-22T20:48:15.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.681__shadow:-Yf8XorFmbM94iugyyLrA046qr-xZqF_.gZiklAPVlm_W3vMwlxuTnAPa-HNkCb2_0 2024-04-22T20:48:15.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16976196103873068745', time=2024-04-22T22:43:25.040226+0000, chain.objs.size()=1 2024-04-22T20:48:15.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.950+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.698__shadow:r8O7CiER9T4ZF5-IaqdWJgq..OXYBLE_.PgvbptkpqpHP3yPmbOP_qklTBky7RWT_0 2024-04-22T20:48:15.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.958+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:15.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.966+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-22T20:48:15.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.967+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:15.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:15.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/46510' 2024-04-22T20:48:15.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9558674221292566513', time=2024-04-22T22:25:15.297357+0000, chain.objs.size()=1 2024-04-22T20:48:15.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.222__shadow_.9rTm04odODN4ASww4vX-15W30_H9w0m_0 2024-04-22T20:48:15.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18445297452979565285', time=2024-04-22T22:25:18.280791+0000, chain.objs.size()=1 2024-04-22T20:48:15.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.225__shadow_.S3lSHg4ANtTqtbw0UXbnyToMC2V824U_0 2024-04-22T20:48:15.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16366181061524859356', time=2024-04-22T22:25:41.795000+0000, chain.objs.size()=1 2024-04-22T20:48:15.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.246__shadow_.ygw3X_-VNJiyq9K9Fa3myVrtZ0sSs96_0 2024-04-22T20:48:15.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2143036811162198029', time=2024-04-22T22:26:22.685884+0000, chain.objs.size()=1 2024-04-22T20:48:15.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.15WjoitsuDxsWJi4Up27tx-yVoydSM-_0 2024-04-22T20:48:15.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11037502916369082758', time=2024-04-22T22:26:22.785003+0000, chain.objs.size()=1 2024-04-22T20:48:15.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pBdIgpleRVvPTTGgq4kKEktO8Fkvs68_0 2024-04-22T20:48:15.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12688916215511652191', time=2024-04-22T22:26:22.918998+0000, chain.objs.size()=1 2024-04-22T20:48:15.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Bq30jKS-KGwg2z2Zcf0FXANhtWs4Uq__0 2024-04-22T20:48:15.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6313746986171917199', time=2024-04-22T22:26:23.298355+0000, chain.objs.size()=1 2024-04-22T20:48:15.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.968+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.e4Pb6vWhtrHdqngMOMxif19L-QSvMbz_0 2024-04-22T20:48:15.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6166571070592176107', time=2024-04-22T22:26:24.352924+0000, chain.objs.size()=1 2024-04-22T20:48:15.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.As_ztqLOzjytqHd5TZpRscxvOLQ2fO7_0 2024-04-22T20:48:15.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2283338992259036594', time=2024-04-22T22:26:24.353768+0000, chain.objs.size()=1 2024-04-22T20:48:15.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rIJ14Zz2i8Y1NrefrhKwRjUi1Cp1jGD_0 2024-04-22T20:48:15.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16145209846986518971', time=2024-04-22T22:26:25.044950+0000, chain.objs.size()=1 2024-04-22T20:48:15.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.u5QMOpwt2lqmYa-RIb5CnLmyOqZNA9a_0 2024-04-22T20:48:15.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5127760322061283846', time=2024-04-22T22:26:25.664890+0000, chain.objs.size()=1 2024-04-22T20:48:15.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zvl28jx8f0dnuoQ0EjoZlGvVGmPvRpQ_0 2024-04-22T20:48:15.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9792743874324753156', time=2024-04-22T22:26:26.057726+0000, chain.objs.size()=1 2024-04-22T20:48:15.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.969+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ROr593NUzgIoV4Kdjo4rgifZ8OZjZEs_0 2024-04-22T20:48:15.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.971+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6728250073956282802', time=2024-04-22T22:26:26.829572+0000, chain.objs.size()=1 2024-04-22T20:48:15.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.971+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.v8VjfQK8F4saNCBuKsvqzz3M7XQyZJ3_0 2024-04-22T20:48:15.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.971+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4074826556127867249', time=2024-04-22T22:26:27.663995+0000, chain.objs.size()=1 2024-04-22T20:48:15.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.971+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CL_NWq2Ojpi4lwm2Fy_BZIZFysD-l26_0 2024-04-22T20:48:15.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.971+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13634373343362045781', time=2024-04-22T22:26:28.288346+0000, chain.objs.size()=1 2024-04-22T20:48:15.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.971+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HKxzNIsYrCfoHKtlPchF2lqgZirtIOE_0 2024-04-22T20:48:15.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.972+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8972362908838436258', time=2024-04-22T22:26:29.079560+0000, chain.objs.size()=1 2024-04-22T20:48:15.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.972+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.j-4uMhju42UGTdd5ICDc35f8BcSsvdO_0 2024-04-22T20:48:15.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.972+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8976435954835368533', time=2024-04-22T22:26:29.183101+0000, chain.objs.size()=1 2024-04-22T20:48:15.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.972+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5tfHsRIAVea_9-M6HgHXsSbdTGP0S_K_0 2024-04-22T20:48:15.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.972+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5085022675023475180', time=2024-04-22T22:26:29.410138+0000, chain.objs.size()=1 2024-04-22T20:48:15.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.972+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.17twLI8C97crjk9KBbKMNHwlc70p_uy_0 2024-04-22T20:48:15.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.973+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7127092729436250069', time=2024-04-22T22:26:29.542375+0000, chain.objs.size()=1 2024-04-22T20:48:15.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.973+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.M5S21c4vsF3fMwmlNcSebpjlUldYOO-_0 2024-04-22T20:48:15.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14479865083186831897', time=2024-04-22T22:26:29.834102+0000, chain.objs.size()=1 2024-04-22T20:48:15.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oGI0E4tLLPyaZx90yW7V7I2QErioxa-_0 2024-04-22T20:48:15.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1831722702858618537', time=2024-04-22T22:26:29.939022+0000, chain.objs.size()=1 2024-04-22T20:48:15.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VcUpyffmfDuQSzVoxwLG0IMlM8iCqZB_0 2024-04-22T20:48:15.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5036214357739715114', time=2024-04-22T22:26:30.011447+0000, chain.objs.size()=1 2024-04-22T20:48:15.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.nvhC5bUa5F7N2zjvatntyjL9f6ltu92_0 2024-04-22T20:48:15.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18011328362370291127', time=2024-04-22T22:26:30.552429+0000, chain.objs.size()=1 2024-04-22T20:48:15.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.L_79Gf2cD4TsJsjPJg-huDWEpWfWy2n_0 2024-04-22T20:48:15.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3468393778613029715', time=2024-04-22T22:26:31.001531+0000, chain.objs.size()=1 2024-04-22T20:48:15.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8OEiRe88hGjR7_elIqrJWP0yhOu2Kh1_0 2024-04-22T20:48:15.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13052933242693265636', time=2024-04-22T22:26:31.002215+0000, chain.objs.size()=1 2024-04-22T20:48:15.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.r1nODvPpycOquHZAFTj6AXDsKXzYten_0 2024-04-22T20:48:15.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1740941313123365007', time=2024-04-22T22:26:31.047399+0000, chain.objs.size()=1 2024-04-22T20:48:15.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.976+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5xx-OTZCzVFtpgrfnW-Wv6ozOZyHy8g_0 2024-04-22T20:48:15.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9105526789839064055', time=2024-04-22T22:26:31.156915+0000, chain.objs.size()=1 2024-04-22T20:48:15.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UQuN20IwJhlfi1x228ip8gK590BT4-n_0 2024-04-22T20:48:15.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13140703074430567485', time=2024-04-22T22:26:31.570982+0000, chain.objs.size()=1 2024-04-22T20:48:15.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wZQjTUD6JpnQDxviyA2O1-YpBPFJawe_0 2024-04-22T20:48:15.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12586147602802056277', time=2024-04-22T22:26:31.623219+0000, chain.objs.size()=1 2024-04-22T20:48:15.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jYjMt9RwEH3moYIS9iuOQCzIotsY7kp_0 2024-04-22T20:48:15.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1170643258708109595', time=2024-04-22T22:26:31.696720+0000, chain.objs.size()=1 2024-04-22T20:48:15.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YcO56i6xQCyDcW2Jsbf2KbSiPrj4vlz_0 2024-04-22T20:48:15.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2243736646490282375', time=2024-04-22T22:26:32.242842+0000, chain.objs.size()=1 2024-04-22T20:48:15.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0ZdwGbfX_zWS31MVhyWSyR5hs7pKx5D_0 2024-04-22T20:48:15.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18064227867734793606', time=2024-04-22T22:26:32.390482+0000, chain.objs.size()=1 2024-04-22T20:48:15.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.71DxX6CS_xXRYStme56ZITtwFYN_jLX_0 2024-04-22T20:48:15.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9714070734246888537', time=2024-04-22T22:26:32.607043+0000, chain.objs.size()=1 2024-04-22T20:48:15.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7TmUgvT02ap1knNrXgrsvOOVfhx2akm_0 2024-04-22T20:48:15.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9570945751851915179', time=2024-04-22T22:26:33.098788+0000, chain.objs.size()=1 2024-04-22T20:48:15.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Agv4Tj-Pz8uvMtFuSBomUh81dEmRYx0_0 2024-04-22T20:48:15.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13845807948625651434', time=2024-04-22T22:26:35.409416+0000, chain.objs.size()=1 2024-04-22T20:48:15.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.252__shadow_.j6opE5GeoLMPqGRFj9cUUpaoMhYWP06_0 2024-04-22T20:48:15.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5605171850029604417', time=2024-04-22T22:26:46.899639+0000, chain.objs.size()=1 2024-04-22T20:48:15.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.264__shadow_.Pn5-ZQKm8fZozMu1aIUtn_BHieXx-wd_0 2024-04-22T20:48:15.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2008050602079545397', time=2024-04-22T22:26:51.882090+0000, chain.objs.size()=1 2024-04-22T20:48:15.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.269__shadow_.gEebj0ztquGj44nMTbryrWBoB58wOOU_0 2024-04-22T20:48:15.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.984+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.292208217682150860', time=2024-04-22T22:26:56.946702+0000, chain.objs.size()=1 2024-04-22T20:48:15.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.984+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.274__shadow_.YTXoN-rkSlAnOZNGFg69Qz_AaMcGtV4_0 2024-04-22T20:48:15.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.984+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13120439341759833055', time=2024-04-22T22:27:05.635565+0000, chain.objs.size()=1 2024-04-22T20:48:15.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.984+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.wd28BLxyJB84PdZ_f2rVoGAEcsW3hOL_0 2024-04-22T20:48:15.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.984+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7684864115205837244', time=2024-04-22T22:27:07.853131+0000, chain.objs.size()=1 2024-04-22T20:48:15.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.984+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.W71eXD0aH3tz7dT8fyUURwgSJN8dJZE_0 2024-04-22T20:48:15.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.984+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2073542571208611933', time=2024-04-22T22:27:07.974286+0000, chain.objs.size()=1 2024-04-22T20:48:15.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.984+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.itwP1aEQaHrkoUxjGcl0ymSToUimKCP_0 2024-04-22T20:48:15.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1243534977523554643', time=2024-04-22T22:28:17.373462+0000, chain.objs.size()=1 2024-04-22T20:48:15.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uWvF32ZoXI-u2diHT6bsNVT000UpmyG_0 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14276232673489548281', time=2024-04-22T22:28:17.551853+0000, chain.objs.size()=1 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KJoGEVmJeQnMffd-EOZ4WVaQKN-VYaF_0 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13922167919327918114', time=2024-04-22T22:28:18.180435+0000, chain.objs.size()=1 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.34tX5PxcnR44KWW1TUyXqmqw3Eh4Ykk_0 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11438190019185723954', time=2024-04-22T22:28:18.227148+0000, chain.objs.size()=1 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7P1PhP3CndN4e3pahjfhiVMPWH2rVwm_0 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2064644053038494802', time=2024-04-22T22:28:18.327351+0000, chain.objs.size()=1 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.l2UJusFVq1ibA83TgITG_on9zVFT9S2_0 2024-04-22T20:48:15.984 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15696200434156154304', time=2024-04-22T22:28:18.495348+0000, chain.objs.size()=1 2024-04-22T20:48:15.985 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.986+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MUQUcvIKpy43bH6tJjybnJO_KSYaQdS_0 2024-04-22T20:48:15.985 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.987+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11465504212741640093', time=2024-04-22T22:28:19.246941+0000, chain.objs.size()=1 2024-04-22T20:48:15.985 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.987+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JUIFuMC0MYfqm-dJP8hYG1wtdTXtH-F_0 2024-04-22T20:48:15.985 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5617661384097456184', time=2024-04-22T22:28:19.593559+0000, chain.objs.size()=1 2024-04-22T20:48:15.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9HsAWs6jhDQZb-C08e11B9bxVvBUPgU_0 2024-04-22T20:48:15.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12566524357846461803', time=2024-04-22T22:28:19.594358+0000, chain.objs.size()=1 2024-04-22T20:48:15.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5roqrWY45TPdPQQUMRnHa1uB7Beg4Ag_0 2024-04-22T20:48:15.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4774207536302683397', time=2024-04-22T22:28:19.755527+0000, chain.objs.size()=1 2024-04-22T20:48:15.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.J5DZixrbDwNl-sAbGqE069Z5clUkYdC_0 2024-04-22T20:48:15.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12638519278325199986', time=2024-04-22T22:28:20.718022+0000, chain.objs.size()=1 2024-04-22T20:48:15.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.M9q1_GpoBBAlmFI8OJF-LKO4nqISU6o_0 2024-04-22T20:48:15.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11522323471225738801', time=2024-04-22T22:28:21.423235+0000, chain.objs.size()=1 2024-04-22T20:48:15.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.i7Io5RN7uDOKtDV7p5pOGRc9Z5q59xZ_0 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.962353992130709324', time=2024-04-22T22:28:21.443142+0000, chain.objs.size()=1 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wgOI6eQ9JrKgWcA7eDGbiT8Mr84Cn28_0 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17803627059941951338', time=2024-04-22T22:28:22.776297+0000, chain.objs.size()=1 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qjamsnHdy8tlYjbx5DIsXCaWHMOyo4V_0 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14851906348891588960', time=2024-04-22T22:28:26.095398+0000, chain.objs.size()=1 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1vx89Ip08LmTUjNFSXgWnBjCSEV3BjM_0 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9531462064073191022', time=2024-04-22T22:28:26.154489+0000, chain.objs.size()=1 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._BzkeXPGjaAvhwuUZjGs_644swXpROv_0 2024-04-22T20:48:15.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14862825654250611274', time=2024-04-22T22:28:26.251855+0000, chain.objs.size()=1 2024-04-22T20:48:15.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6NqyPeovaKtLPU-rrSDW2ZmOOPNXlCf_0 2024-04-22T20:48:15.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14275720062983883116', time=2024-04-22T22:28:26.523733+0000, chain.objs.size()=1 2024-04-22T20:48:15.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.992+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3Fe0dB4W3g44S53oudJzx3qUjtpDA5F_0 2024-04-22T20:48:15.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12540580340447175449', time=2024-04-22T22:28:26.968590+0000, chain.objs.size()=1 2024-04-22T20:48:15.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MiXSWt6o0SX1_fDLmNagT-e_RLQ_3Z1_0 2024-04-22T20:48:15.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8573017785497443345', time=2024-04-22T22:28:27.639433+0000, chain.objs.size()=1 2024-04-22T20:48:15.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.psy3XVjPXDmk6K2ZNjcVmcwcvd6TMMC_0 2024-04-22T20:48:15.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13618148483513535037', time=2024-04-22T22:28:27.836112+0000, chain.objs.size()=1 2024-04-22T20:48:15.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.b1J2AeRnO0nXXHfBUk8DFOfE9eqYeem_0 2024-04-22T20:48:15.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8261308687435526757', time=2024-04-22T22:28:28.071216+0000, chain.objs.size()=1 2024-04-22T20:48:15.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ya4LpKZg6JtVYf9dCSzC1FwoT79aX96_0 2024-04-22T20:48:15.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.996+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14795758764124225480', time=2024-04-22T22:28:28.526760+0000, chain.objs.size()=1 2024-04-22T20:48:15.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.996+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wNOJcwzzN-lW1rPwIrWLcKSl5pDhghh_0 2024-04-22T20:48:15.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.996+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15770724652751081252', time=2024-04-22T22:28:29.006574+0000, chain.objs.size()=1 2024-04-22T20:48:15.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.996+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.E23ydRFSOBWeLIDBlfLzE9HLlPKihJT_0 2024-04-22T20:48:15.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.996+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17035729343364830571', time=2024-04-22T22:28:29.072293+0000, chain.objs.size()=1 2024-04-22T20:48:15.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.996+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.px7CDxKD20KNfKatncEorKCta1gZxmK_0 2024-04-22T20:48:15.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.996+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13308535071013822913', time=2024-04-22T22:28:29.257819+0000, chain.objs.size()=1 2024-04-22T20:48:15.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.996+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.EZPs0iKMLB7BnEvO5yxGaTmlM1DgTQV_0 2024-04-22T20:48:15.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17379478355759806304', time=2024-04-22T22:29:08.446924+0000, chain.objs.size()=1 2024-04-22T20:48:15.994 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.cAPK9a_HVFFHjB0gWMtSliA1m326gny_0 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1361164048553783245', time=2024-04-22T22:29:08.896502+0000, chain.objs.size()=1 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.K30jx8rrNVHLiZl2xAZRdVRr467xxiW_0 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15557120172227138151', time=2024-04-22T22:29:09.733574+0000, chain.objs.size()=1 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.i2hn2aAHVsrmWYhVqdg57RPo_Mfw1at_0 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8772185183120095414', time=2024-04-22T22:29:09.734093+0000, chain.objs.size()=1 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Uke9mQWnYo6_--n_w8IfwyV_L54lOIV_0 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13463760597834543012', time=2024-04-22T22:29:09.769098+0000, chain.objs.size()=1 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ua01FEOneSlMvlQOsbDVFdlKD-TNczn_0 2024-04-22T20:48:15.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18399376679837597056', time=2024-04-22T22:29:09.915369+0000, chain.objs.size()=1 2024-04-22T20:48:15.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zqS95ttBqA3Or4diQBjrJu--qUuyupO_0 2024-04-22T20:48:15.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11753633283378071705', time=2024-04-22T22:29:10.066101+0000, chain.objs.size()=1 2024-04-22T20:48:15.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:15.997+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TIIe-jifLjbhk9LJKR26WnBkDnXBx7h_0 2024-04-22T20:48:15.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.000+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3632262292524964475', time=2024-04-22T22:29:10.066828+0000, chain.objs.size()=1 2024-04-22T20:48:15.996 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.000+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hZmvHbd7GngP_U7b53mJ8hYOQjSiOIr_0 2024-04-22T20:48:15.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4968460490808366014', time=2024-04-22T22:29:10.238500+0000, chain.objs.size()=1 2024-04-22T20:48:15.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PLkOHXJ9e2vIO5_KAt3QFj6mims4-fj_0 2024-04-22T20:48:15.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12177057345744808266', time=2024-04-22T22:29:11.182691+0000, chain.objs.size()=1 2024-04-22T20:48:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jlc6aLCT4T5y2b2jfrTqWmI6ea4q8Oi_0 2024-04-22T20:48:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14209304664364958450', time=2024-04-22T22:29:11.218455+0000, chain.objs.size()=1 2024-04-22T20:48:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IiLIZRs6AJ4QW-1okHNb__e9JsnpshY_0 2024-04-22T20:48:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5364425981016230793', time=2024-04-22T22:29:11.392605+0000, chain.objs.size()=1 2024-04-22T20:48:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.uqEuVV8Xj0pV_iebBNvbe7mkq65VsMT_0 2024-04-22T20:48:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2474606626927302284', time=2024-04-22T22:29:11.643213+0000, chain.objs.size()=1 2024-04-22T20:48:15.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.j2WEvQQvVbpCuqadqJrzqAkpv4yXVvU_0 2024-04-22T20:48:16.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8407054801508560519', time=2024-04-22T22:29:11.922316+0000, chain.objs.size()=1 2024-04-22T20:48:16.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.vgw3hdMSPByPK597fBrIeVmJG130vXl_0 2024-04-22T20:48:16.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16971230183022740430', time=2024-04-22T22:29:12.718834+0000, chain.objs.size()=1 2024-04-22T20:48:16.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.UTBlLnT48AvfnnwdZEMGrswcTZmGVM8_0 2024-04-22T20:48:16.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15723371492511860281', time=2024-04-22T22:29:13.018277+0000, chain.objs.size()=1 2024-04-22T20:48:16.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wQe12Y3lb8xbPMsN8aM7EvShrlUt6Fj_0 2024-04-22T20:48:16.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16384643773613063617', time=2024-04-22T22:29:13.086209+0000, chain.objs.size()=1 2024-04-22T20:48:16.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XO3bJN1vNVwdqELUwG5730aMDIlpESa_0 2024-04-22T20:48:16.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9000378269882292321', time=2024-04-22T22:29:13.086956+0000, chain.objs.size()=1 2024-04-22T20:48:16.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.003+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WfcOZQ2xuHCfzhOyzLooP7qNM4y2LuA_0 2024-04-22T20:48:16.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.004+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2962964645462879696', time=2024-04-22T22:29:13.520596+0000, chain.objs.size()=1 2024-04-22T20:48:16.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.004+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sOar8-JZTR81rf9bI3jVlqBMl_H7C2e_0 2024-04-22T20:48:16.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12568809115791066552', time=2024-04-22T22:29:14.336300+0000, chain.objs.size()=1 2024-04-22T20:48:16.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.aUT0xCOUS0pGMq8t20FuRgcI5zgLm1d_0 2024-04-22T20:48:16.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3748341276253112311', time=2024-04-22T22:29:14.505131+0000, chain.objs.size()=1 2024-04-22T20:48:16.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ScqcQMCJz0S5yNdAGfMpQQIVyrQj1Wa_0 2024-04-22T20:48:16.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10520667300202110139', time=2024-04-22T22:29:14.735591+0000, chain.objs.size()=1 2024-04-22T20:48:16.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.aexENlnxDUJy28MDkgMi_RC8Svr-46R_0 2024-04-22T20:48:16.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8242975596327181929', time=2024-04-22T22:29:15.304391+0000, chain.objs.size()=1 2024-04-22T20:48:16.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZNQfqvf0GtaEwaW9JmmDX_8ssHpF3zY_0 2024-04-22T20:48:16.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7753692232731342311', time=2024-04-22T22:29:15.475401+0000, chain.objs.size()=1 2024-04-22T20:48:16.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jZua-6bFoY6ak9EXOIj9mWewchLY3Rs_0 2024-04-22T20:48:16.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8894544443875909715', time=2024-04-22T22:29:15.823352+0000, chain.objs.size()=1 2024-04-22T20:48:16.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Lp_TARRNacX2ynjBJyOyZ9RpuhJhhs6_0 2024-04-22T20:48:16.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8083989643740474577', time=2024-04-22T22:29:15.880031+0000, chain.objs.size()=1 2024-04-22T20:48:16.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oNDi2EjnUXCOQ6qkybULFuKZuBBaGJd_0 2024-04-22T20:48:16.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.007+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16715097808272751338', time=2024-04-22T22:29:16.184113+0000, chain.objs.size()=1 2024-04-22T20:48:16.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.007+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4vtsDA-7LTwVyqqe09e5ZXmLyEaQUCc_0 2024-04-22T20:48:16.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.008+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15721602310771063096', time=2024-04-22T22:29:16.325248+0000, chain.objs.size()=1 2024-04-22T20:48:16.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.008+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.f2aQIkKETY-wmCg1cNzBIgbX8Rx74F0_0 2024-04-22T20:48:16.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.008+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17019773664108118371', time=2024-04-22T22:29:16.759832+0000, chain.objs.size()=1 2024-04-22T20:48:16.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.008+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ojU78dr8J8tWtbBtcP_o_98JS0ELIjh_0 2024-04-22T20:48:16.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.008+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7448094341133927951', time=2024-04-22T22:29:16.762339+0000, chain.objs.size()=1 2024-04-22T20:48:16.006 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.008+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7cHNO-N3MHRQaAxUkauhuH33elwzZq4_0 2024-04-22T20:48:16.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.322803942255484381', time=2024-04-22T22:29:17.187859+0000, chain.objs.size()=1 2024-04-22T20:48:16.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.amCLe6aQjELyMAh6M5DP7FSlmHzRbDv_0 2024-04-22T20:48:16.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5350608477765669309', time=2024-04-22T22:29:17.209909+0000, chain.objs.size()=1 2024-04-22T20:48:16.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qockacnpn6HsOI3Y1I9sXvmbzudJ1q1_0 2024-04-22T20:48:16.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16118992149981968232', time=2024-04-22T22:29:17.619816+0000, chain.objs.size()=1 2024-04-22T20:48:16.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.75EOeTlbiR8sF587RyqF7Fy2AQ_dyPS_0 2024-04-22T20:48:16.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.019+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/46510 2024-04-22T20:48:16.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=17, truncated=0, next_marker='' 2024-04-22T20:48:16.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.750337530209348328', time=2024-04-22T22:29:17.622016+0000, chain.objs.size()=1 2024-04-22T20:48:16.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jcaVCb6K0X5c5v6JTJNdvxceVilTyLK_0 2024-04-22T20:48:16.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11227492556337680494', time=2024-04-22T22:29:17.630858+0000, chain.objs.size()=1 2024-04-22T20:48:16.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.z8gzyhi5_eYdEkj3nxU1c-omYEeRuWz_0 2024-04-22T20:48:16.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15738693135144793337', time=2024-04-22T22:30:02.596183+0000, chain.objs.size()=1 2024-04-22T20:48:16.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.379__shadow_.NJzCda3gX63UUFqWnyw6t1LPIf82poe_0 2024-04-22T20:48:16.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3965603394581326661', time=2024-04-22T22:30:28.718176+0000, chain.objs.size()=1 2024-04-22T20:48:16.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.418__shadow_.HOIZRLaALtfJwbMNP8u4OxYFtl9bAL7_0 2024-04-22T20:48:16.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6752564941676827548', time=2024-04-22T22:31:10.500818+0000, chain.objs.size()=1 2024-04-22T20:48:16.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.ZE6KmSl9D2o7sYzQq6BeBHAlbdOVDWr_0 2024-04-22T20:48:16.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1930041668859324119', time=2024-04-22T22:31:11.418718+0000, chain.objs.size()=1 2024-04-22T20:48:16.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.473__shadow_.1oj4SM0lq-eLBCA52Wngd5rsHX_lMP4_0 2024-04-22T20:48:16.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_1_efecb81ce2564c38f45807aee53fd4b3', time=2024-04-22T22:31:28.105623+0000, chain.objs.size()=1 2024-04-22T20:48:16.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_0 2024-04-22T20:48:16.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1531796306270401972', time=2024-04-22T22:31:57.586068+0000, chain.objs.size()=1 2024-04-22T20:48:16.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.502__multipart_mymultipart.2~078mf1tN4KXD-sl10aMjtoKifVEKcXR.1 2024-04-22T20:48:16.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1783531023149967409', time=2024-04-22T22:34:39.478879+0000, chain.objs.size()=6 2024-04-22T20:48:16.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~dxnRni6wrKi9l2LSRLFxcAZ_ydgo8Ts.1 2024-04-22T20:48:16.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.wSNIkj7tn2-n5dZxXjP46MMtl7QvjRAy.2 2024-04-22T20:48:16.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.-JAVKivw7jHsHx11PnC4fj7SL3Ewyd6e.3 2024-04-22T20:48:16.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~dxnRni6wrKi9l2LSRLFxcAZ_ydgo8Ts.4 2024-04-22T20:48:16.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.023+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~dxnRni6wrKi9l2LSRLFxcAZ_ydgo8Ts.5 2024-04-22T20:48:16.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.026+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~dxnRni6wrKi9l2LSRLFxcAZ_ydgo8Ts.6 2024-04-22T20:48:16.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7741339883139945443', time=2024-04-22T22:36:30.777747+0000, chain.objs.size()=1 2024-04-22T20:48:16.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.538__shadow_.XAYTfDHD1KpVsGbvCTfLj1NoxkXLIQn_0 2024-04-22T20:48:16.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8775871910965519807', time=2024-04-22T22:37:22.820390+0000, chain.objs.size()=1 2024-04-22T20:48:16.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:rkAjX4-F5Zp.U4OWaJiT6gmGlI6lg3G_.KNPOv5Il-aCfhYobTMMBJg6aA3Y5Gmm_0 2024-04-22T20:48:16.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14104342878162593684', time=2024-04-22T22:37:33.684311+0000, chain.objs.size()=1 2024-04-22T20:48:16.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.558__shadow:GWOvjYvf7Se6XgcpPuOdDFHBwm0EBtx_.c58t47T_ndvSNjZIiwLDj0rplAmcJqN_0 2024-04-22T20:48:16.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11674396173173707973', time=2024-04-22T22:38:52.838446+0000, chain.objs.size()=1 2024-04-22T20:48:16.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.570__shadow:IMO6VjrdfqZL6Uc79gPs5ZOcxgsYOx1_.iZ_AYXp-ATfcyn33wcn3yetyd_YJ1YL_0 2024-04-22T20:48:16.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16685944886173850803', time=2024-04-22T22:38:56.344006+0000, chain.objs.size()=1 2024-04-22T20:48:16.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.573__shadow:uW.wrftrCGwBDV4CsbCIXrRpPj8MQkr_.Z0otsJA6Ay3qlsaFJSosUcX9T34Ie32_0 2024-04-22T20:48:16.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3359809253636238597', time=2024-04-22T22:42:16.720826+0000, chain.objs.size()=1 2024-04-22T20:48:16.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.633__shadow_.UaNXDJsEfSBGf7PqqVno50aQU0_7mek_0 2024-04-22T20:48:16.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13244300514940065973', time=2024-04-22T22:42:51.303524+0000, chain.objs.size()=6 2024-04-22T20:48:16.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.645__multipart_baz.2~3A2gPjZvRNVek3OXMJbCEsPEUO9J2h7.1 2024-04-22T20:48:16.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.645__multipart_baz.2~3A2gPjZvRNVek3OXMJbCEsPEUO9J2h7.2 2024-04-22T20:48:16.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.645__multipart_baz.2~3A2gPjZvRNVek3OXMJbCEsPEUO9J2h7.3 2024-04-22T20:48:16.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.029+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.645__multipart_baz.2~3A2gPjZvRNVek3OXMJbCEsPEUO9J2h7.4 2024-04-22T20:48:16.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.032+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.645__multipart_baz.2~3A2gPjZvRNVek3OXMJbCEsPEUO9J2h7.5 2024-04-22T20:48:16.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.032+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.645__multipart_baz.2~3A2gPjZvRNVek3OXMJbCEsPEUO9J2h7.6 2024-04-22T20:48:16.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.032+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16266588507495415376', time=2024-04-22T22:42:58.043255+0000, chain.objs.size()=1 2024-04-22T20:48:16.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.032+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.654__shadow_.qlfWBQPMCmtPsr1RTXtKp-jEq2zJD1p_0 2024-04-22T20:48:16.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.038+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.040+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-22T20:48:16.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.041+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.038 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/40894' 2024-04-22T20:48:16.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2850027839300236813', time=2024-04-22T22:22:49.723134+0000, chain.objs.size()=1 2024-04-22T20:48:16.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.168__shadow_.pNqnbpAwIYeqzaTRQjyRQW4ZwOkloff_0 2024-04-22T20:48:16.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15641679447837011074', time=2024-04-22T22:25:19.960944+0000, chain.objs.size()=1 2024-04-22T20:48:16.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.226__shadow_.zt9N8bPP3YmKGpeHDhg3l-XDiSz_NQY_0 2024-04-22T20:48:16.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16801535549773974163', time=2024-04-22T22:25:25.434896+0000, chain.objs.size()=1 2024-04-22T20:48:16.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.231__shadow_.YDGAWx_nF0wZcKGEDjZwKQJuGw-cwHs_0 2024-04-22T20:48:16.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16619187004182664891', time=2024-04-22T22:25:38.801776+0000, chain.objs.size()=1 2024-04-22T20:48:16.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.243__shadow_.9Py5gA2ZQkVdPDRFJDGaqHn464AIHjS_0 2024-04-22T20:48:16.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1998117675891160002', time=2024-04-22T22:25:43.673768+0000, chain.objs.size()=1 2024-04-22T20:48:16.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.248__shadow_.uky_F_02-rMnW-rvj9u9OnrEEL6686p_0 2024-04-22T20:48:16.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6923152112621418734', time=2024-04-22T22:26:22.759067+0000, chain.objs.size()=1 2024-04-22T20:48:16.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.IBqgdz_zoj57yTSFD7BzRWuKODb_4v5_0 2024-04-22T20:48:16.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15466754079014522091', time=2024-04-22T22:26:23.137245+0000, chain.objs.size()=1 2024-04-22T20:48:16.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.I0F2H0ibn13a6ymhM1kjMe6oeJbynm__0 2024-04-22T20:48:16.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10780910519104521565', time=2024-04-22T22:26:23.168288+0000, chain.objs.size()=1 2024-04-22T20:48:16.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4dXngxgNxi0DFtAPpeaIbHBv-Ft_P4k_0 2024-04-22T20:48:16.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10067244625763047125', time=2024-04-22T22:26:23.261860+0000, chain.objs.size()=1 2024-04-22T20:48:16.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.042+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.nrLcGBHLbSvblIPoGFFqVGTEQNrhTSc_0 2024-04-22T20:48:16.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18139877838279757173', time=2024-04-22T22:26:23.484723+0000, chain.objs.size()=1 2024-04-22T20:48:16.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hrI9Y-D_jbTMZCuiHnfuVqfIgqu-v8Q_0 2024-04-22T20:48:16.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14979470141420581665', time=2024-04-22T22:26:24.238626+0000, chain.objs.size()=1 2024-04-22T20:48:16.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PwQhtCorqP7gICqxwYWB4IV3MLpAGfS_0 2024-04-22T20:48:16.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7186969717178039958', time=2024-04-22T22:26:24.445489+0000, chain.objs.size()=1 2024-04-22T20:48:16.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.R3NjJUQqZe-3p-CmNKtBGI1hbrscwKL_0 2024-04-22T20:48:16.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.047+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17206575395489064519', time=2024-04-22T22:26:24.678357+0000, chain.objs.size()=1 2024-04-22T20:48:16.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.047+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.QSMfd4zhYmD0qfD-pvf3hIulMxSATVM_0 2024-04-22T20:48:16.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.047+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11148195608180694370', time=2024-04-22T22:26:26.300924+0000, chain.objs.size()=1 2024-04-22T20:48:16.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.047+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dMw7gOLU5gz1NJCDseFc1TXlPjICHGD_0 2024-04-22T20:48:16.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18218675060933796132', time=2024-04-22T22:26:26.443510+0000, chain.objs.size()=1 2024-04-22T20:48:16.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YM2OkIZEZHQ37aSPx3KQ8xtNKU91ZHz_0 2024-04-22T20:48:16.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8488562962162137501', time=2024-04-22T22:26:27.009603+0000, chain.objs.size()=1 2024-04-22T20:48:16.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ytBqiR8_VuNeR0D3Y3alqLLo_1rpMMM_0 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5226082840027755496', time=2024-04-22T22:26:27.485429+0000, chain.objs.size()=1 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2dIWePBLdq4UXeSPkwCfAD_xuGV7_p4_0 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2748867818247562744', time=2024-04-22T22:26:27.816182+0000, chain.objs.size()=1 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Q4CCe7mEgXxYOg6_pHQF8BAOvI3zsiA_0 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14356707409680064473', time=2024-04-22T22:26:27.891285+0000, chain.objs.size()=1 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FVUYOF9dIosdtv7AysfjpDMRtgXh0Ig_0 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8598570347924138241', time=2024-04-22T22:26:28.489110+0000, chain.objs.size()=1 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YPQUDriAE2M5UZXVsgnKuEEo95aPwg6_0 2024-04-22T20:48:16.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15486566054899928017', time=2024-04-22T22:26:28.609337+0000, chain.objs.size()=1 2024-04-22T20:48:16.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.D3N-qyYFaxy0HTF8zaq2SO29Z7biZWw_0 2024-04-22T20:48:16.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14703507165513757778', time=2024-04-22T22:26:29.272885+0000, chain.objs.size()=1 2024-04-22T20:48:16.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wJqUM281D3dtV3bG0WEUEZrUYNqhoWy_0 2024-04-22T20:48:16.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3687748137912705111', time=2024-04-22T22:26:29.324652+0000, chain.objs.size()=1 2024-04-22T20:48:16.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4cZLDZFkEpNpA5a1pquc1z0wvzsujiQ_0 2024-04-22T20:48:16.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.052+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6545035999782345992', time=2024-04-22T22:26:30.119209+0000, chain.objs.size()=1 2024-04-22T20:48:16.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hrtloXdnKgp6VHyLg1Ux8yuypNJz6jB_0 2024-04-22T20:48:16.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.052+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3595326753621405517', time=2024-04-22T22:26:31.041754+0000, chain.objs.size()=1 2024-04-22T20:48:16.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CrVoTnRRKAYFHluMbFax0N2TuLd_nGR_0 2024-04-22T20:48:16.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.052+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13170226824760208413', time=2024-04-22T22:26:31.096769+0000, chain.objs.size()=1 2024-04-22T20:48:16.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.KF_EpBZWXuj0XMAnG-O63E00doagz-j_0 2024-04-22T20:48:16.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.055+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1321111895930111846', time=2024-04-22T22:26:31.329550+0000, chain.objs.size()=1 2024-04-22T20:48:16.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.055+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-ptZtN0bHcRtEjVPbgcL2FdDqvRlrNE_0 2024-04-22T20:48:16.054 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.057+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3047070222946997797', time=2024-04-22T22:26:32.719695+0000, chain.objs.size()=1 2024-04-22T20:48:16.054 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.057+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0yCpAQMTEjHa9WbaIJhrAmEuUt1nxz8_0 2024-04-22T20:48:16.054 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.057+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5559222976844208683', time=2024-04-22T22:26:33.101247+0000, chain.objs.size()=1 2024-04-22T20:48:16.054 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.057+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.H5B4oibqrj04J3xBPDW0TWd6Kca4fb6_0 2024-04-22T20:48:16.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.057+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3157515567803933322', time=2024-04-22T22:26:33.576653+0000, chain.objs.size()=1 2024-04-22T20:48:16.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.057+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mXWIDnGZP2xXcaCi6ET9v-8-r2CSuyv_0 2024-04-22T20:48:16.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.057+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4344150171456261735', time=2024-04-22T22:26:33.918271+0000, chain.objs.size()=1 2024-04-22T20:48:16.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.057+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CCKQnTDbW2Z1EAkcXEcZqtmF4y0w-sT_0 2024-04-22T20:48:16.055 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10228787031609171427', time=2024-04-22T22:26:34.364780+0000, chain.objs.size()=1 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MP5PuO7X3Ug2zU8QjbMlVIA7QjlfehV_0 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13105747550272313052', time=2024-04-22T22:26:34.419164+0000, chain.objs.size()=1 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tsycEB6sDGDq4JnRfMfAfMQE_i2-Vqe_0 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18309295259285047482', time=2024-04-22T22:26:37.351538+0000, chain.objs.size()=1 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.254__shadow_.vBZRyXIVwEXUlSCcIK1ODXeJzyIA-KU_0 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.592642631272563623', time=2024-04-22T22:26:50.898800+0000, chain.objs.size()=1 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.268__shadow_.JyWKAF-YWOTaUzgWgJ515V61FAz4xV7_0 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2307723726759319242', time=2024-04-22T22:26:54.857588+0000, chain.objs.size()=1 2024-04-22T20:48:16.056 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.272__shadow_.eUnu973TtB45y-x8GrzDx5cx_csYPpe_0 2024-04-22T20:48:16.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2021213381834447182', time=2024-04-22T22:27:05.567972+0000, chain.objs.size()=1 2024-04-22T20:48:16.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.058+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.rMeGjhagvwn-UG_tvFDuyfb87HHSqFV_0 2024-04-22T20:48:16.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.060+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3349718056664450296', time=2024-04-22T22:27:05.683961+0000, chain.objs.size()=1 2024-04-22T20:48:16.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.060+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.bQ4sZ1L4KtWPICjy1RFbmL338pWNkig_0 2024-04-22T20:48:16.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10561399061556690601', time=2024-04-22T22:27:07.858154+0000, chain.objs.size()=1 2024-04-22T20:48:16.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.J91HHaFxaEWP1uhfgGDWjszU4krlQu9_0 2024-04-22T20:48:16.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7784192868132117445', time=2024-04-22T22:27:14.099523+0000, chain.objs.size()=1 2024-04-22T20:48:16.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.288__shadow_.wkm-d_1VXir9CbNUZu_lYMeitCcw5Mg_0 2024-04-22T20:48:16.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1216012920956291275', time=2024-04-22T22:27:18.101848+0000, chain.objs.size()=1 2024-04-22T20:48:16.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.292__shadow_.kR3pcIz6cLO9-DFsXhymD5_BkBf4We9_0 2024-04-22T20:48:16.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9054147583797841973', time=2024-04-22T22:28:17.504755+0000, chain.objs.size()=1 2024-04-22T20:48:16.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zTiY0S4PIZ_ATrcxpbwAbIO1X9jsTVT_0 2024-04-22T20:48:16.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2562734264452597534', time=2024-04-22T22:28:17.648322+0000, chain.objs.size()=1 2024-04-22T20:48:16.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.S4MBf6Ptxpknc2aUVk3ol_rSLsqInGN_0 2024-04-22T20:48:16.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5525873188167256799', time=2024-04-22T22:28:18.178765+0000, chain.objs.size()=1 2024-04-22T20:48:16.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BhqosQ1t-8yRgop6uUGDHUsjGH8tdvw_0 2024-04-22T20:48:16.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6581257239232678125', time=2024-04-22T22:28:18.485741+0000, chain.objs.size()=1 2024-04-22T20:48:16.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.hfFf_4HAGbndiCeJJHvcrsb45vJU1xa_0 2024-04-22T20:48:16.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.064+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12371660171024310449', time=2024-04-22T22:28:18.530035+0000, chain.objs.size()=1 2024-04-22T20:48:16.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.064+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CInrHcOpycJXSGRpYro1EELOGMrXHiO_0 2024-04-22T20:48:16.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.064+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8148627408056819050', time=2024-04-22T22:28:18.530800+0000, chain.objs.size()=1 2024-04-22T20:48:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.064+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.u0zl_u6_46lLvSuGG6HPMqTbIf9SmxN_0 2024-04-22T20:48:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.064+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2439032508821009424', time=2024-04-22T22:28:19.239602+0000, chain.objs.size()=1 2024-04-22T20:48:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.064+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IkioSCE5nTCziwqogFkLTNWUR6eaJpB_0 2024-04-22T20:48:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.064+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13655905228737872066', time=2024-04-22T22:28:19.419807+0000, chain.objs.size()=1 2024-04-22T20:48:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.064+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RN1rTGt7AM8sKtmwzWBKcECapl-ZP6T_0 2024-04-22T20:48:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.067+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1532406597981672295', time=2024-04-22T22:28:19.518778+0000, chain.objs.size()=1 2024-04-22T20:48:16.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.067+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xa8-dHTctD_lR0pxESyk6H7IrF6SGmp_0 2024-04-22T20:48:16.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14967380856339693197', time=2024-04-22T22:28:19.598314+0000, chain.objs.size()=1 2024-04-22T20:48:16.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wyJV7e-Fid5h_NuG5eyYt1GKlbsvcpD_0 2024-04-22T20:48:16.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2090276407992021366', time=2024-04-22T22:28:19.941575+0000, chain.objs.size()=1 2024-04-22T20:48:16.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IaM7HvQuK4NaftJKb4jJbhZ6XWCpKDm_0 2024-04-22T20:48:16.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14541575931088079231', time=2024-04-22T22:28:20.089446+0000, chain.objs.size()=1 2024-04-22T20:48:16.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yFsHz0UdQNrtQ_jZVJ09ro-2_puvg7-_0 2024-04-22T20:48:16.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6329065907921759498', time=2024-04-22T22:28:20.716235+0000, chain.objs.size()=1 2024-04-22T20:48:16.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.s82QpufJwLkC33lh0uxo_COm3syLGkh_0 2024-04-22T20:48:16.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17609890287235187225', time=2024-04-22T22:28:21.078509+0000, chain.objs.size()=1 2024-04-22T20:48:16.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.068+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kZy4oIcsAMQ1PKqHwPE3x5BnIGyqLBB_0 2024-04-22T20:48:16.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4729708094925619060', time=2024-04-22T22:28:21.850792+0000, chain.objs.size()=1 2024-04-22T20:48:16.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uZ102U7WZ7OscVMF1S6B5JwQM8BZy4X_0 2024-04-22T20:48:16.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14075669153609810087', time=2024-04-22T22:28:22.940893+0000, chain.objs.size()=1 2024-04-22T20:48:16.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Lt8hlIYWT7Wm0sR-9HsmHKpKWzhvrmW_0 2024-04-22T20:48:16.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5396613216027866191', time=2024-04-22T22:28:22.957519+0000, chain.objs.size()=1 2024-04-22T20:48:16.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.llPraQDvGrxBqYPHjDTAEqJWYJEZFMQ_0 2024-04-22T20:48:16.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15494371259114367900', time=2024-04-22T22:28:23.081103+0000, chain.objs.size()=1 2024-04-22T20:48:16.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IexqHYAugZqe9cpKSJmgRSYb_szD83Z_0 2024-04-22T20:48:16.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5183387464388846876', time=2024-04-22T22:28:23.882070+0000, chain.objs.size()=1 2024-04-22T20:48:16.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vVCHuRi8rxBd8FXQwH2oq0QRP49_0Zm_0 2024-04-22T20:48:16.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13365732798535273788', time=2024-04-22T22:28:23.987585+0000, chain.objs.size()=1 2024-04-22T20:48:16.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uERpwgmQTL_cHnwzAkwsuhQmEukfa-v_0 2024-04-22T20:48:16.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.072+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6268969995393772171', time=2024-04-22T22:28:24.122217+0000, chain.objs.size()=1 2024-04-22T20:48:16.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.072+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.sEGCtCNJLkG_RnFJzMRqBe1aqBrdm8E_0 2024-04-22T20:48:16.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.072+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3051317721143246333', time=2024-04-22T22:28:25.457304+0000, chain.objs.size()=1 2024-04-22T20:48:16.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.072+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZPIi8rJbwIBQLgg0BZFt5rSgZRIAYHv_0 2024-04-22T20:48:16.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.072+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3517272748661693481', time=2024-04-22T22:28:25.601391+0000, chain.objs.size()=1 2024-04-22T20:48:16.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.072+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FYhX5M9WFLN2otEdG718F-jWOBAGWS4_0 2024-04-22T20:48:16.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.074+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3108301871154644546', time=2024-04-22T22:28:26.426164+0000, chain.objs.size()=1 2024-04-22T20:48:16.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.074+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4U4hm-p8E-uq-scPN28WcrSfEQBJtQL_0 2024-04-22T20:48:16.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.074+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7468844916452375193', time=2024-04-22T22:28:26.970329+0000, chain.objs.size()=1 2024-04-22T20:48:16.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.074+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.s7Y4h7i8GVX58cjMX3fl8w2z-0Xk0jF_0 2024-04-22T20:48:16.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.074+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12241651975367036975', time=2024-04-22T22:28:27.215077+0000, chain.objs.size()=1 2024-04-22T20:48:16.071 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.074+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.m9kJGuGqr_HNJBPqgovKcl-V_KLVY12_0 2024-04-22T20:48:16.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9001845656941911251', time=2024-04-22T22:28:27.802682+0000, chain.objs.size()=1 2024-04-22T20:48:16.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Un625bpV8HEm_t1tIeFQAHVxqjOO7hI_0 2024-04-22T20:48:16.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8240369033292177871', time=2024-04-22T22:28:28.343099+0000, chain.objs.size()=1 2024-04-22T20:48:16.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZnP3Jn250WfufWD8QPpJ_MkzQHV2K9v_0 2024-04-22T20:48:16.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15135890685670966251', time=2024-04-22T22:28:28.518033+0000, chain.objs.size()=1 2024-04-22T20:48:16.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RoSRNELHXa2FMR7Qzl96CkYg8iefPSC_0 2024-04-22T20:48:16.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7794901065606591241', time=2024-04-22T22:28:29.253246+0000, chain.objs.size()=1 2024-04-22T20:48:16.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Kz_XWFfW3sHFG4rFovnWHcF8aNso_hT_0 2024-04-22T20:48:16.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5702759684468549366', time=2024-04-22T22:28:29.279991+0000, chain.objs.size()=1 2024-04-22T20:48:16.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FOkEHKRgzZrs4RrNtYfcNiFdKHwCMYS_0 2024-04-22T20:48:16.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7811431437459564383', time=2024-04-22T22:29:06.313473+0000, chain.objs.size()=1 2024-04-22T20:48:16.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xkpmLVeV3TOhfTfbPDtGQdYSZ3d7BfI_0 2024-04-22T20:48:16.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17908254074446181581', time=2024-04-22T22:29:07.258780+0000, chain.objs.size()=1 2024-04-22T20:48:16.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sDYPPSl23pYFqwcibL4CJoUz56Lgr0m_0 2024-04-22T20:48:16.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8555573748839188670', time=2024-04-22T22:29:07.810077+0000, chain.objs.size()=1 2024-04-22T20:48:16.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.K0XI-yRiAyb8kYb0SunK1OZ2A6HKOPs_0 2024-04-22T20:48:16.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.079+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14314642031216582904', time=2024-04-22T22:29:07.811277+0000, chain.objs.size()=1 2024-04-22T20:48:16.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.079+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ERoU1odOBqHnOQrXCaaXOGpuWmsl_F8_0 2024-04-22T20:48:16.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.079+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15712840934127374998', time=2024-04-22T22:29:07.971599+0000, chain.objs.size()=1 2024-04-22T20:48:16.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.079+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2nlGdr2AaKojQ1zLk2j5xFCSf-MfLA__0 2024-04-22T20:48:16.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.079+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10101357818638567326', time=2024-04-22T22:29:08.148605+0000, chain.objs.size()=1 2024-04-22T20:48:16.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.079+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8JFX7QUYNvtENuT6irEo_PA4d9eIb8k_0 2024-04-22T20:48:16.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.081+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1518743538668706471', time=2024-04-22T22:29:08.151982+0000, chain.objs.size()=1 2024-04-22T20:48:16.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.081+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.moB0lxP3Je9x5ECLQeOjFSSxt1oP_6x_0 2024-04-22T20:48:16.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14936807478656090400', time=2024-04-22T22:29:08.461296+0000, chain.objs.size()=1 2024-04-22T20:48:16.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yLjsF_LDK5hb5pZf75bD9tvfmBKm61C_0 2024-04-22T20:48:16.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12251308196122053320', time=2024-04-22T22:29:08.884892+0000, chain.objs.size()=1 2024-04-22T20:48:16.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.x36_3N1pNMD2pjjpZnamJS1fNotU9Uo_0 2024-04-22T20:48:16.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3966050027430574307', time=2024-04-22T22:29:09.652558+0000, chain.objs.size()=1 2024-04-22T20:48:16.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.eVmIuFDMmMrYyjMR9-aPQEwz5Bktfr0_0 2024-04-22T20:48:16.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13771863499318039513', time=2024-04-22T22:29:09.889740+0000, chain.objs.size()=1 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.66LV5-uNbGW3BpF_wdSvWsgv-JEDNBe_0 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12038955233764693136', time=2024-04-22T22:29:10.349450+0000, chain.objs.size()=1 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.p6nd-EWYWmizX9gKBjo-GCnC52Cuw9R_0 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16176817304120820723', time=2024-04-22T22:29:11.061893+0000, chain.objs.size()=1 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.r-n-Omm2bGI6NgO5vwvTQmoekX7ohq2_0 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4293270435365432911', time=2024-04-22T22:29:11.159267+0000, chain.objs.size()=1 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SubyEhBqDKtEt2jqJifTepaVPib6zv0_0 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7041629175745575676', time=2024-04-22T22:29:11.392855+0000, chain.objs.size()=1 2024-04-22T20:48:16.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-fZqjBBlCL5OKwhBxs7dZZ7UN096MJT_0 2024-04-22T20:48:16.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13516068484004230809', time=2024-04-22T22:29:11.825571+0000, chain.objs.size()=1 2024-04-22T20:48:16.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HdIN_2_rAin0grJiLPw6nPAzd1Uxy_n_0 2024-04-22T20:48:16.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17331971853250738868', time=2024-04-22T22:29:11.950814+0000, chain.objs.size()=1 2024-04-22T20:48:16.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Kgix7vFKqhfrxoISkZjLV7UlZBR33N9_0 2024-04-22T20:48:16.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.085+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16697565026158259632', time=2024-04-22T22:29:13.386624+0000, chain.objs.size()=1 2024-04-22T20:48:16.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.085+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jPYCt6kUlhvCdsCwUH1wCCAThJoT-uA_0 2024-04-22T20:48:16.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.086+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7864421627473212952', time=2024-04-22T22:29:13.909200+0000, chain.objs.size()=1 2024-04-22T20:48:16.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.086+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2BeqnnYCCw1nE8iluA4X3k_FKvBAcMm_0 2024-04-22T20:48:16.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.086+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16811804856138333378', time=2024-04-22T22:29:14.384637+0000, chain.objs.size()=1 2024-04-22T20:48:16.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.086+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PsS_NX_B03DbZj1XuQ4OZIW8x9kNYrB_0 2024-04-22T20:48:16.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11642851388692626573', time=2024-04-22T22:29:14.494831+0000, chain.objs.size()=1 2024-04-22T20:48:16.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BU-2KV5FSBmU1CrH58oBe2UajhjkoHH_0 2024-04-22T20:48:16.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17954587418932712423', time=2024-04-22T22:29:14.963485+0000, chain.objs.size()=1 2024-04-22T20:48:16.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mrG8mzqWO6jn19yXG0qASeWBPXycHxk_0 2024-04-22T20:48:16.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6728082120093354125', time=2024-04-22T22:29:15.996508+0000, chain.objs.size()=1 2024-04-22T20:48:16.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gMK0P38vQbwYUCaXzPZgN262IEpcYK6_0 2024-04-22T20:48:16.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3665394401965981661', time=2024-04-22T22:29:16.105208+0000, chain.objs.size()=1 2024-04-22T20:48:16.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ki1UWSfjpTlW2asoCE_D1-T1Z7q0luj_0 2024-04-22T20:48:16.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3324329400062346978', time=2024-04-22T22:29:16.965213+0000, chain.objs.size()=1 2024-04-22T20:48:16.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SKh8IWWZTqvfCzGHDHyWdw5wfaB4_E7_0 2024-04-22T20:48:16.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16432315485479497602', time=2024-04-22T22:29:17.522716+0000, chain.objs.size()=1 2024-04-22T20:48:16.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.088+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zoJEcwq-w4_4eA3wLru-BTJAC2RvfNN_0 2024-04-22T20:48:16.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.091+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14950110994853179540', time=2024-04-22T22:29:17.604590+0000, chain.objs.size()=1 2024-04-22T20:48:16.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.091+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iv5sNHCv5CQYtktSVu77zrfR4VCreot_0 2024-04-22T20:48:16.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.091+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17464186137468505536', time=2024-04-22T22:29:17.661863+0000, chain.objs.size()=1 2024-04-22T20:48:16.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.091+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.r2NQF4hhxnmeecPMZC3B4l7izWqqNH9_0 2024-04-22T20:48:16.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.094+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/40894 2024-04-22T20:48:16.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=21, truncated=0, next_marker='' 2024-04-22T20:48:16.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5392112667115738400', time=2024-04-22T22:29:22.894927+0000, chain.objs.size()=1 2024-04-22T20:48:16.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.321__shadow_.e7V8fMBQqB6CUh6XkDrrWMZPmFVTalo_0 2024-04-22T20:48:16.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18018064693331675743', time=2024-04-22T22:29:29.862997+0000, chain.objs.size()=1 2024-04-22T20:48:16.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.330__shadow_.PebUBo4R2r9BhzjsWQNzFc6LnF56hib_0 2024-04-22T20:48:16.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2381263505376518330', time=2024-04-22T22:29:33.154264+0000, chain.objs.size()=1 2024-04-22T20:48:16.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.335__shadow_.ViVm1Z9-rpjJ6USh8Zm-WbnvRfJJsCE_0 2024-04-22T20:48:16.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4331306815213773688', time=2024-04-22T22:29:54.916236+0000, chain.objs.size()=1 2024-04-22T20:48:16.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.368__shadow_.68bes6ypV56TydJ4G2JLbFXqwbtvnEu_0 2024-04-22T20:48:16.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16576668563253596200', time=2024-04-22T22:30:56.166051+0000, chain.objs.size()=1 2024-04-22T20:48:16.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.450__shadow_.Mr8W6U7_uvWIx3lKPWZ65zqSr8J2GYP_0 2024-04-22T20:48:16.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4574363512827234675', time=2024-04-22T22:31:01.917687+0000, chain.objs.size()=1 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.455__shadow_.KtiegKM8guPpY5vQbPtZ2rX3uU2Fmqt_0 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13419002174345331428', time=2024-04-22T22:31:18.486266+0000, chain.objs.size()=1 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.479__shadow_.h-sIQFelhSzme_tuL40BrFsKDIPC3yE_0 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4123118927837758641', time=2024-04-22T22:31:20.190506+0000, chain.objs.size()=1 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.482__shadow_.QPmggrc4NM5O281cgb8KvWrlrjTCemE_0 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1974714254607928840', time=2024-04-22T22:35:23.371793+0000, chain.objs.size()=1 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.513__shadow_.fdwBHLI3392gv7Tzy9aeDZXhW6xG1rA_0 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5407590475981613616', time=2024-04-22T22:35:30.760595+0000, chain.objs.size()=4 2024-04-22T20:48:16.096 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.516__multipart_mymultipart.2~IsOj3Sd2fws7NJNSXAFneAPQUxgwX2n.1 2024-04-22T20:48:16.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.516__multipart_mymultipart.2~IsOj3Sd2fws7NJNSXAFneAPQUxgwX2n.2 2024-04-22T20:48:16.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.516__multipart_mymultipart.2~IsOj3Sd2fws7NJNSXAFneAPQUxgwX2n.3 2024-04-22T20:48:16.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.099+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.516__multipart_mymultipart.2~IsOj3Sd2fws7NJNSXAFneAPQUxgwX2n.4 2024-04-22T20:48:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.099+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2637746339315467916', time=2024-04-22T22:37:23.130353+0000, chain.objs.size()=1 2024-04-22T20:48:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.099+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:w5M0u0TIZPf4OhUvLbundG5IgoEmQru_.btp0awecwnVgIcujTV0e5tCWlrcA9lq_0 2024-04-22T20:48:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5859679642834315432', time=2024-04-22T22:37:24.936345+0000, chain.objs.size()=1 2024-04-22T20:48:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:6.SXdfnfYcqnCo0ax1inDZAl5MUMjqS_.wGtAys8iFnd0S6U0S72Zi1Q-WUryib9_0 2024-04-22T20:48:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.110802701817334249', time=2024-04-22T22:37:47.378962+0000, chain.objs.size()=1 2024-04-22T20:48:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:yGpzbh4VYz8oOvqNCnaDBDAxNLsOtz._.T5FpOT2vZJpXxekKc6_e_-ZPhhBWn7c_0 2024-04-22T20:48:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13414479918498624713', time=2024-04-22T22:37:50.575913+0000, chain.objs.size()=1 2024-04-22T20:48:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:gWXIHrVnjuRJyrA0OmJj4lA5vfik143_.PpYNcKAJ-uwYOAZWVE3nTRU39DVLJAJ_0 2024-04-22T20:48:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16225452238660490122', time=2024-04-22T22:37:59.852703+0000, chain.objs.size()=1 2024-04-22T20:48:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:Dj8XAX6haVI-YBtIVcrCjc18ylCuLBo_.R1d9UePNIKKoJZKqTvbr1h0Q4_KB1Yx_0 2024-04-22T20:48:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11130006927887811980', time=2024-04-22T22:38:49.776677+0000, chain.objs.size()=1 2024-04-22T20:48:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:d8STIagVJg5zHdpVhtVEgczqMhDrdZp_.L-XVayu8wC3PrzbnBdLM_fPjZOKUC4B_0 2024-04-22T20:48:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14546865750841765336', time=2024-04-22T22:42:51.494163+0000, chain.objs.size()=12 2024-04-22T20:48:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__multipart_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.1 2024-04-22T20:48:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__shadow_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.1_1 2024-04-22T20:48:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.100+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__multipart_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.2 2024-04-22T20:48:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.102+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__shadow_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.2_1 2024-04-22T20:48:16.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.104+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__multipart_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.3 2024-04-22T20:48:16.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.104+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__shadow_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.3_1 2024-04-22T20:48:16.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__multipart_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.4 2024-04-22T20:48:16.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__shadow_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.4_1 2024-04-22T20:48:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__multipart_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.5 2024-04-22T20:48:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__shadow_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.5_1 2024-04-22T20:48:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__multipart_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.6 2024-04-22T20:48:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.646__shadow_foo.2~mFXyKlEanxtELeJ4UTJRW2nCVkRnref.6_1 2024-04-22T20:48:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18219047693064933851', time=2024-04-22T22:42:53.610353+0000, chain.objs.size()=1 2024-04-22T20:48:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.648__shadow_.b8Gqxp_cn1KSCGUerFfdQpbxgONj9rU_0 2024-04-22T20:48:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7821720554515602523', time=2024-04-22T22:42:53.616714+0000, chain.objs.size()=1 2024-04-22T20:48:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.648__shadow_.nLPJXM0DV4u-dhyYt15f_lrW2Flhrtu_0 2024-04-22T20:48:16.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14824290575404556672', time=2024-04-22T22:42:54.693230+0000, chain.objs.size()=1 2024-04-22T20:48:16.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.649__shadow_.s3_1JlM6m-x2ZC-n5RUdc-R1SkfSXT2_0 2024-04-22T20:48:16.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.107+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6707253843913499631', time=2024-04-22T22:43:55.855423+0000, chain.objs.size()=1 2024-04-22T20:48:16.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.107+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.716__shadow_.hJ2saw0LAcf82B56w3G1VPlvC_EV6-S_0 2024-04-22T20:48:16.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.110+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.110 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.113+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-22T20:48:16.111 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.115+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.112 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.115+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/54310' 2024-04-22T20:48:16.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6437146294282172536', time=2024-04-22T22:25:13.294513+0000, chain.objs.size()=1 2024-04-22T20:48:16.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.220__shadow_.1zpLfrX6smPJJc0YUa4gs1nQD6IQI8I_0 2024-04-22T20:48:16.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4778049644395884152', time=2024-04-22T22:25:27.132571+0000, chain.objs.size()=1 2024-04-22T20:48:16.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.232__shadow_.I-HSZdbzmUC6clOWTY8Awll21ZoaS4J_0 2024-04-22T20:48:16.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17044328776999341133', time=2024-04-22T22:25:36.827155+0000, chain.objs.size()=1 2024-04-22T20:48:16.116 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.241__shadow_.PPS_B6XXRFlD-gtRs5V1hbBLKW3kSyD_0 2024-04-22T20:48:16.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17470360718818859441', time=2024-04-22T22:25:39.799062+0000, chain.objs.size()=1 2024-04-22T20:48:16.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.244__shadow_.g4cKx4treZVVKmkqqpkYAg-gJjdKPR0_0 2024-04-22T20:48:16.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9269345692195550416', time=2024-04-22T22:25:44.663084+0000, chain.objs.size()=1 2024-04-22T20:48:16.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.249__shadow_.ZDvf-NVGdykTSrAYk6zAkhZmbdszCmf_0 2024-04-22T20:48:16.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9005881162483424633', time=2024-04-22T22:26:22.849622+0000, chain.objs.size()=1 2024-04-22T20:48:16.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JdK9_PkFZ4oUPTvvW-iHl4HzKv2xXHH_0 2024-04-22T20:48:16.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4655346573112528651', time=2024-04-22T22:26:24.077254+0000, chain.objs.size()=1 2024-04-22T20:48:16.118 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8B50I-hVmR6izpGNFatgEgmIdoFZux-_0 2024-04-22T20:48:16.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.379460789575306799', time=2024-04-22T22:26:24.699478+0000, chain.objs.size()=1 2024-04-22T20:48:16.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.RKVgm6nbuR-Q6H1zcr4qSdmy_SgobGn_0 2024-04-22T20:48:16.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2930873846106586716', time=2024-04-22T22:26:25.091810+0000, chain.objs.size()=1 2024-04-22T20:48:16.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.TT5Ql5CH2T39GYUyqpuGZdxC3X-lOyF_0 2024-04-22T20:48:16.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10564633538691682300', time=2024-04-22T22:26:25.809103+0000, chain.objs.size()=1 2024-04-22T20:48:16.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ac9fDS16AI9AB0wBGZ5DIHPwkZ9guD2_0 2024-04-22T20:48:16.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15877754290762279149', time=2024-04-22T22:26:25.809759+0000, chain.objs.size()=1 2024-04-22T20:48:16.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YbQcYKo_BT_R3sDMqjj_NC8NjyrqJld_0 2024-04-22T20:48:16.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6555412517959337585', time=2024-04-22T22:26:25.972316+0000, chain.objs.size()=1 2024-04-22T20:48:16.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hMGmRLsDVHYmnMTzQjmvx3ob3lvZ9fF_0 2024-04-22T20:48:16.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4792952064875351654', time=2024-04-22T22:26:26.120696+0000, chain.objs.size()=1 2024-04-22T20:48:16.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-GYaNg0z3PDAlYjUA_gdGroBsmyO4-F_0 2024-04-22T20:48:16.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2318108959493149109', time=2024-04-22T22:26:26.992376+0000, chain.objs.size()=1 2024-04-22T20:48:16.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hmEu5V1g8MJ6S7e-PG6ypwogPvQW18u_0 2024-04-22T20:48:16.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11131379599439951345', time=2024-04-22T22:26:27.474975+0000, chain.objs.size()=1 2024-04-22T20:48:16.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.898AtYQ4mCGKndRT1agGxFNGsl4pZqS_0 2024-04-22T20:48:16.126 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7433093038996085258', time=2024-04-22T22:26:27.970624+0000, chain.objs.size()=1 2024-04-22T20:48:16.126 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Ak1Y96SOKFxyPYjEr2Ubt0YFc9TGzo6_0 2024-04-22T20:48:16.126 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.212312129783746842', time=2024-04-22T22:26:28.343290+0000, chain.objs.size()=1 2024-04-22T20:48:16.126 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rCuSpoWcnunkX3KiRF20eR7wI_Ac2Sl_0 2024-04-22T20:48:16.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3225744412443009215', time=2024-04-22T22:26:29.055937+0000, chain.objs.size()=1 2024-04-22T20:48:16.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0TQOHwGtyYZrnoJWuocPrNgbL0H2SrG_0 2024-04-22T20:48:16.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2595887531068077676', time=2024-04-22T22:26:29.056640+0000, chain.objs.size()=1 2024-04-22T20:48:16.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JV8nZ_QhXGYbMmEp6guMd5s5BtNLGDL_0 2024-04-22T20:48:16.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12282056569322586987', time=2024-04-22T22:26:29.271993+0000, chain.objs.size()=1 2024-04-22T20:48:16.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.119+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EPEuEDLz1vBoYNxjBD86oy3eCDDNRBh_0 2024-04-22T20:48:16.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.121+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6754172246980978046', time=2024-04-22T22:26:29.993560+0000, chain.objs.size()=1 2024-04-22T20:48:16.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.121+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.e6iCDkQvS-GxITHQGsUtq3J-hCEwdOa_0 2024-04-22T20:48:16.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.121+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4164084120344079750', time=2024-04-22T22:26:30.031835+0000, chain.objs.size()=1 2024-04-22T20:48:16.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.121+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.v5vzTVW0Jf7yBCNILoDStWwnLS1lzgR_0 2024-04-22T20:48:16.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.121+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10819318206076203754', time=2024-04-22T22:26:30.157547+0000, chain.objs.size()=1 2024-04-22T20:48:16.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.121+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OAbBRr7W2QJTTdUpfL94pTC7xfrtMgO_0 2024-04-22T20:48:16.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.122+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9828655160739669281', time=2024-04-22T22:26:30.687606+0000, chain.objs.size()=1 2024-04-22T20:48:16.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HmxuXB2xYFoaCfyd0LAcleUrPdvE7qb_0 2024-04-22T20:48:16.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.123+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18313310956282365586', time=2024-04-22T22:26:31.098407+0000, chain.objs.size()=1 2024-04-22T20:48:16.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.123+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JtNOSFH_ZzslabVR8YeP782NDwb8-v-_0 2024-04-22T20:48:16.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.123+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10508966283203157683', time=2024-04-22T22:26:31.525661+0000, chain.objs.size()=1 2024-04-22T20:48:16.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.123+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mzDYweFES8uubZYIdrkuIRQx3Oq9PdQ_0 2024-04-22T20:48:16.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.123+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2352336489587787936', time=2024-04-22T22:26:32.052755+0000, chain.objs.size()=1 2024-04-22T20:48:16.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.123+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ck46t0EqJa_TTPCxLW03VxgWc4Dp7-7_0 2024-04-22T20:48:16.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.123+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1975227893766310992', time=2024-04-22T22:26:32.147640+0000, chain.objs.size()=1 2024-04-22T20:48:16.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.123+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qwozBH93NXsQ84gSIzc2jxYRmAvxk8q_0 2024-04-22T20:48:16.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.124+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11136948774642565502', time=2024-04-22T22:26:32.353061+0000, chain.objs.size()=1 2024-04-22T20:48:16.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.124+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tB-gnNUT5735v8PBzh0jB5TJovfvb2m_0 2024-04-22T20:48:16.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.124+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4793761215128586583', time=2024-04-22T22:26:32.842534+0000, chain.objs.size()=1 2024-04-22T20:48:16.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.124+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Z5W3uvriYK5Vk6sEwpQk1mdz4hpt1ww_0 2024-04-22T20:48:16.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.124+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16270473765315218260', time=2024-04-22T22:26:33.481831+0000, chain.objs.size()=1 2024-04-22T20:48:16.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.124+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.L4yQ05MjVMVTGeG6wCs5Tvj7F-rPKrd_0 2024-04-22T20:48:16.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.125+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4142558424064575817', time=2024-04-22T22:26:33.841782+0000, chain.objs.size()=1 2024-04-22T20:48:16.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.125+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Vl1AdreVkDr6g9Su_820MaJAqULYP9l_0 2024-04-22T20:48:16.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7132527694450553538', time=2024-04-22T22:26:33.997526+0000, chain.objs.size()=1 2024-04-22T20:48:16.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._H0P2QB3E4-yn7vsS9vEVeIO0q784we_0 2024-04-22T20:48:16.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18310094743638726652', time=2024-04-22T22:26:34.166947+0000, chain.objs.size()=1 2024-04-22T20:48:16.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6FTuuBqewLU1Jd7SgQ9khRWrQh3zjvm_0 2024-04-22T20:48:16.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511750068803909939', time=2024-04-22T22:26:34.241011+0000, chain.objs.size()=1 2024-04-22T20:48:16.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FN-Au3eOippdLbxmcvdxqzaWoZaScUD_0 2024-04-22T20:48:16.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1669515815274792910', time=2024-04-22T22:26:34.424549+0000, chain.objs.size()=1 2024-04-22T20:48:16.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Jxho2lI9jKAORrXB8o-2pUZwKoLlWIM_0 2024-04-22T20:48:16.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8772710154638368047', time=2024-04-22T22:26:49.865499+0000, chain.objs.size()=1 2024-04-22T20:48:16.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.267__shadow_.agTMPqioKdno9QU7ILlfht8qCwuZ47T_0 2024-04-22T20:48:16.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7171912747533454599', time=2024-04-22T22:26:51.887868+0000, chain.objs.size()=1 2024-04-22T20:48:16.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.269__shadow_.eGB0Qw5bz465oS4p9R485RCfoRhyFhV_0 2024-04-22T20:48:16.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9638656593611688424', time=2024-04-22T22:27:01.963998+0000, chain.objs.size()=1 2024-04-22T20:48:16.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.279__shadow_.yKSVWqWJOpFipUXeKvpgor09W4LS-jB_0 2024-04-22T20:48:16.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2799425923195528952', time=2024-04-22T22:27:07.854404+0000, chain.objs.size()=1 2024-04-22T20:48:16.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.KWNqGeIMTgKLsABeHeK1FpUx8Xqay_2_0 2024-04-22T20:48:16.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.385483336483590116', time=2024-04-22T22:27:17.063936+0000, chain.objs.size()=1 2024-04-22T20:48:16.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.291__shadow_.YoV1FJfCdkmrqoLyRoK-ZFLqnZqC7Z2_0 2024-04-22T20:48:16.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8207499427864050852', time=2024-04-22T22:27:18.109582+0000, chain.objs.size()=1 2024-04-22T20:48:16.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.292__shadow_.F6TBedeuYDRmNBpH26k0884OkTMr3c0_0 2024-04-22T20:48:16.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13515660686553175215', time=2024-04-22T22:28:18.438077+0000, chain.objs.size()=1 2024-04-22T20:48:16.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wZpWOEc-PuWY6NSwPB3bKvktWlyArJh_0 2024-04-22T20:48:16.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4282776330801137303', time=2024-04-22T22:28:20.091009+0000, chain.objs.size()=1 2024-04-22T20:48:16.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NBlfHThoZs_-moAStxo5KDB6jH9aYmh_0 2024-04-22T20:48:16.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14175230669815206858', time=2024-04-22T22:28:20.179750+0000, chain.objs.size()=1 2024-04-22T20:48:16.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dnHiBTFjkeCGiilxuAiEecQjjYLq6px_0 2024-04-22T20:48:16.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9386546444260394556', time=2024-04-22T22:28:20.890170+0000, chain.objs.size()=1 2024-04-22T20:48:16.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cl7vmoNocwjz9faRIy1BXhLfe7--uf9_0 2024-04-22T20:48:16.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1613568712559719698', time=2024-04-22T22:28:21.076935+0000, chain.objs.size()=1 2024-04-22T20:48:16.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nI7k3WQEUn-DfzkvJY8qOkYp7zbnhbb_0 2024-04-22T20:48:16.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5405655530828333008', time=2024-04-22T22:28:21.257038+0000, chain.objs.size()=1 2024-04-22T20:48:16.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.132+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1CsPZLaXOyezNtbrGYf3zSVvxuAJb7O_0 2024-04-22T20:48:16.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.134+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16202780219524110283', time=2024-04-22T22:28:21.257771+0000, chain.objs.size()=1 2024-04-22T20:48:16.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.134+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lw2BuKtUHtrV7QSQcYY_k9Ycokh8a_d_0 2024-04-22T20:48:16.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13851274037364020232', time=2024-04-22T22:28:22.943633+0000, chain.objs.size()=1 2024-04-22T20:48:16.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fhY5bRvyJ5V1pOnHVHOBlLMyZ8t_Ujt_0 2024-04-22T20:48:16.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17306565993022489761', time=2024-04-22T22:28:23.365498+0000, chain.objs.size()=1 2024-04-22T20:48:16.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tMrKjJVAgLDrEV8655wHoE2_HxOEUYb_0 2024-04-22T20:48:16.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2870097345587483758', time=2024-04-22T22:28:23.409712+0000, chain.objs.size()=1 2024-04-22T20:48:16.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.URDahfo2NnRXbyCFOUNVVj-JetBCMyt_0 2024-04-22T20:48:16.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5424670386758041711', time=2024-04-22T22:28:23.960643+0000, chain.objs.size()=1 2024-04-22T20:48:16.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pDv1jRd20TYb2T032ZE52gFKaddpIa2_0 2024-04-22T20:48:16.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7039716291864448737', time=2024-04-22T22:28:23.986640+0000, chain.objs.size()=1 2024-04-22T20:48:16.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mjM4Okl04BdNN0vh6WSS6FBCTc7huRP_0 2024-04-22T20:48:16.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.137+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14715446170660428953', time=2024-04-22T22:28:24.015484+0000, chain.objs.size()=1 2024-04-22T20:48:16.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.137+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xkZ8u7NLur572E2ppipnfVHd5IYU81Z_0 2024-04-22T20:48:16.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.137+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11075525264449463633', time=2024-04-22T22:28:24.130589+0000, chain.objs.size()=1 2024-04-22T20:48:16.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.137+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LBqOC76XkkKa9XyxkipnomnH3VmZost_0 2024-04-22T20:48:16.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.137+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9625933278349522171', time=2024-04-22T22:28:25.030462+0000, chain.objs.size()=1 2024-04-22T20:48:16.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.137+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.In9knkvDrTJ2UqcUKkJBiRqtMjFjria_0 2024-04-22T20:48:16.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.137+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18000001016892662030', time=2024-04-22T22:28:25.415893+0000, chain.objs.size()=1 2024-04-22T20:48:16.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.137+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-TYysysW6vME29W15yw8KCqqvovwejF_0 2024-04-22T20:48:16.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.138+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7525758070098797226', time=2024-04-22T22:28:26.085116+0000, chain.objs.size()=1 2024-04-22T20:48:16.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.138+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MXHMfldeAVVkWZejZ_AoLUCsuOp9exR_0 2024-04-22T20:48:16.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.138+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12907836841997317918', time=2024-04-22T22:28:26.933776+0000, chain.objs.size()=1 2024-04-22T20:48:16.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.138+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.67iFv7q12usKVOJvko2RqAb0IBl65AF_0 2024-04-22T20:48:16.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.138+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11301437265492830421', time=2024-04-22T22:28:27.247808+0000, chain.objs.size()=1 2024-04-22T20:48:16.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.138+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LHjwEd7JNz9xpB3g_hh_vy433SSmshr_0 2024-04-22T20:48:16.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.138+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14001425894646683194', time=2024-04-22T22:28:27.390641+0000, chain.objs.size()=1 2024-04-22T20:48:16.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.138+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lA3nenGhzUiem8V_d-pUWG__LdAuYgm_0 2024-04-22T20:48:16.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13565728976712801659', time=2024-04-22T22:28:27.391402+0000, chain.objs.size()=1 2024-04-22T20:48:16.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4XTFsdgf4WyejNm7cQeQqMzRM151IyS_0 2024-04-22T20:48:16.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11214959640613194480', time=2024-04-22T22:28:27.557241+0000, chain.objs.size()=1 2024-04-22T20:48:16.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pcJkGNVosJq8Sqyy-SdVp43c9EfkttI_0 2024-04-22T20:48:16.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5475371672003727594', time=2024-04-22T22:28:27.726995+0000, chain.objs.size()=1 2024-04-22T20:48:16.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IS4_5IAVSaDlqB4APOmucp14QmJkC56_0 2024-04-22T20:48:16.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6579867218024378973', time=2024-04-22T22:28:27.727768+0000, chain.objs.size()=1 2024-04-22T20:48:16.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9WElX5rlVJ9V7piDK11fQ7tZmt8K4Gs_0 2024-04-22T20:48:16.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14337357515762148269', time=2024-04-22T22:28:27.955320+0000, chain.objs.size()=1 2024-04-22T20:48:16.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.141+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.m3PIaQHjhALAV5rB06PBKV4KsK_N5bb_0 2024-04-22T20:48:16.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.143+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16678953006953543287', time=2024-04-22T22:28:28.402115+0000, chain.objs.size()=1 2024-04-22T20:48:16.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.143+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TBDQ2Z_9vZ39sHv-bcB6ITMyeod8nxf_0 2024-04-22T20:48:16.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.143+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12597290452220750432', time=2024-04-22T22:28:28.460060+0000, chain.objs.size()=1 2024-04-22T20:48:16.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.143+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7wtjqxgP8L2aRKpr1_Gt1XWalrZm2Pk_0 2024-04-22T20:48:16.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.143+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10293602531653007992', time=2024-04-22T22:28:28.939984+0000, chain.objs.size()=1 2024-04-22T20:48:16.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.143+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.aXwqzftkR2C0mY7S96VNVTjZPJ7huXG_0 2024-04-22T20:48:16.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.143+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16139486028128767025', time=2024-04-22T22:28:29.080663+0000, chain.objs.size()=1 2024-04-22T20:48:16.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.143+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vPLLyLn7C4lhl0sjj8faeRba0rj6NK8_0 2024-04-22T20:48:16.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.144+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1884445750258398459', time=2024-04-22T22:29:06.183166+0000, chain.objs.size()=1 2024-04-22T20:48:16.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.144+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rDKa0Z0LwLoibLZ64VpKQv1hCOiuzOZ_0 2024-04-22T20:48:16.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.144+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.186967114685849390', time=2024-04-22T22:29:06.184498+0000, chain.objs.size()=1 2024-04-22T20:48:16.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.144+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kEa7hDf-4xH-j2kFHN_Ukb9J8mRH3-b_0 2024-04-22T20:48:16.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13173618155328287232', time=2024-04-22T22:29:06.236663+0000, chain.objs.size()=1 2024-04-22T20:48:16.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZX9MEXUEm9oq6ZjSbAz5RdiUg3TXDcA_0 2024-04-22T20:48:16.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15268964483322591604', time=2024-04-22T22:29:06.608569+0000, chain.objs.size()=1 2024-04-22T20:48:16.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.L0TfJuX-YrtyicMUPVEYMW9oQOPWfFD_0 2024-04-22T20:48:16.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16646334461508392398', time=2024-04-22T22:29:06.679967+0000, chain.objs.size()=1 2024-04-22T20:48:16.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._SBHVuY3QB05yHgVWiTQgrqD2nL4i18_0 2024-04-22T20:48:16.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12288363595615896020', time=2024-04-22T22:29:06.773307+0000, chain.objs.size()=1 2024-04-22T20:48:16.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RKSV3K8uMy4ME_qEnTgP6bslt3dKVvE_0 2024-04-22T20:48:16.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1713030677086616748', time=2024-04-22T22:29:06.898027+0000, chain.objs.size()=1 2024-04-22T20:48:16.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.146+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LmrptBLBkTnu6Cm-rSqIPeVS-xlERqk_0 2024-04-22T20:48:16.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2702411409100055478', time=2024-04-22T22:29:06.976571+0000, chain.objs.size()=1 2024-04-22T20:48:16.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DtPD15hQm1rnaFfZx23C4AxsaqBiGwx_0 2024-04-22T20:48:16.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5687753101627092746', time=2024-04-22T22:29:07.959856+0000, chain.objs.size()=1 2024-04-22T20:48:16.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZMYNO34RIrsdkMbkRyn-5vffkFVGZNp_0 2024-04-22T20:48:16.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14251886866729993970', time=2024-04-22T22:29:08.437458+0000, chain.objs.size()=1 2024-04-22T20:48:16.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.CsbCKM2eLckvAB847Da8ImMF_btsA1t_0 2024-04-22T20:48:16.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1012595379416849380', time=2024-04-22T22:29:08.461359+0000, chain.objs.size()=1 2024-04-22T20:48:16.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RTdhDwXJC351ac-hv_Zl1prP9rcZdGq_0 2024-04-22T20:48:16.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.172520999397653040', time=2024-04-22T22:29:08.604144+0000, chain.objs.size()=1 2024-04-22T20:48:16.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gj2FuAJ5tbXWjs3x6AdJD7qUv2hpr5s_0 2024-04-22T20:48:16.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6458091791774270841', time=2024-04-22T22:29:08.731182+0000, chain.objs.size()=1 2024-04-22T20:48:16.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.149+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NkF-eUWacHwbV5biJFib5muocxs6YKS_0 2024-04-22T20:48:16.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.151+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3363635953413689077', time=2024-04-22T22:29:08.885476+0000, chain.objs.size()=1 2024-04-22T20:48:16.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.151+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TNbcy5TddmdEVnOq4uacfl2_-9gj-Jc_0 2024-04-22T20:48:16.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.151+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11692298486428666460', time=2024-04-22T22:29:09.357995+0000, chain.objs.size()=1 2024-04-22T20:48:16.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.151+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.d0sCL0s34zyLoxAcVBe-QsSzxcldpR9_0 2024-04-22T20:48:16.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.151+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2118717992040819763', time=2024-04-22T22:29:09.506827+0000, chain.objs.size()=1 2024-04-22T20:48:16.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.151+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.b4PFPo3JRWTj4mc1pJD5WXHb8b0mdaI_0 2024-04-22T20:48:16.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.151+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14880425337446715289', time=2024-04-22T22:29:09.554650+0000, chain.objs.size()=1 2024-04-22T20:48:16.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.151+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3UDUSkOs3ZxiZ6OXSff_taa-GsgF-2X_0 2024-04-22T20:48:16.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.152+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4011857884998706990', time=2024-04-22T22:29:10.079193+0000, chain.objs.size()=1 2024-04-22T20:48:16.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.152+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0KyAqjFWM-PgNssgN_yUPLhsPe4gyd__0 2024-04-22T20:48:16.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.152+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12413834045505678484', time=2024-04-22T22:29:10.479639+0000, chain.objs.size()=1 2024-04-22T20:48:16.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.152+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.V8387LMo6P7vRNj--zyilXUZRtGokct_0 2024-04-22T20:48:16.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.152+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4219642437429624583', time=2024-04-22T22:29:10.895753+0000, chain.objs.size()=1 2024-04-22T20:48:16.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.152+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YkNjMJONuGhDFTwkvxFGegmOYLvHNRU_0 2024-04-22T20:48:16.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.154+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15622520994994494814', time=2024-04-22T22:29:10.897523+0000, chain.objs.size()=1 2024-04-22T20:48:16.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2vPMGlvYY0Ig7F0dPu8pvuqtvROJefK_0 2024-04-22T20:48:16.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.154+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14686044198638993102', time=2024-04-22T22:29:11.415370+0000, chain.objs.size()=1 2024-04-22T20:48:16.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kvhxNk8W_fqzo444w2pysyaEpXkSbk8_0 2024-04-22T20:48:16.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.154+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16487596835136601313', time=2024-04-22T22:29:13.076708+0000, chain.objs.size()=1 2024-04-22T20:48:16.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iPTnRPEA8_taiotcJaa4y2WwTvV-8ZC_0 2024-04-22T20:48:16.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3526280035582271415', time=2024-04-22T22:29:13.363327+0000, chain.objs.size()=1 2024-04-22T20:48:16.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.B6iM4iYtz3rGOmZw0GjQsQ83Y13Mun3_0 2024-04-22T20:48:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17258390432452255657', time=2024-04-22T22:29:14.070608+0000, chain.objs.size()=1 2024-04-22T20:48:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.O9uA2VZNQ8LYDHsxqmU7mHtRH6Icz-i_0 2024-04-22T20:48:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13509915328265896485', time=2024-04-22T22:29:14.182161+0000, chain.objs.size()=1 2024-04-22T20:48:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.t400m8gd718xGqQ_lylZUhJu9JD6z74_0 2024-04-22T20:48:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10771840445713199270', time=2024-04-22T22:29:14.185045+0000, chain.objs.size()=1 2024-04-22T20:48:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RnDEaHHNe637dUyFz_DZYtc6XNnjAG1_0 2024-04-22T20:48:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11265082116504234214', time=2024-04-22T22:29:14.336484+0000, chain.objs.size()=1 2024-04-22T20:48:16.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.155+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NhV7sg1E7xuyWvcVLvrKP9TgyVaLDB2_0 2024-04-22T20:48:16.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.158+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7435237204381016197', time=2024-04-22T22:29:14.801466+0000, chain.objs.size()=1 2024-04-22T20:48:16.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.158+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4_gumYSkcEvva8J0nKAavfKk89r-W7b_0 2024-04-22T20:48:16.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.162+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/54310 2024-04-22T20:48:16.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=31, truncated=0, next_marker='' 2024-04-22T20:48:16.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10490401646448436330', time=2024-04-22T22:29:14.961446+0000, chain.objs.size()=1 2024-04-22T20:48:16.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZuQCZqfmfwCaxXzV8he2Ng43bYIIaRa_0 2024-04-22T20:48:16.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11726736473715233352', time=2024-04-22T22:29:15.351993+0000, chain.objs.size()=1 2024-04-22T20:48:16.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.adfMwF5iiefR0r-aE92BszSijXIiz26_0 2024-04-22T20:48:16.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8334506906105659389', time=2024-04-22T22:29:15.479620+0000, chain.objs.size()=1 2024-04-22T20:48:16.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oBWqXWRBiI_uoDkaNfIbat100A100SZ_0 2024-04-22T20:48:16.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18276678159107701462', time=2024-04-22T22:29:15.480602+0000, chain.objs.size()=1 2024-04-22T20:48:16.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Q0Jp9j0exgNrCtY9Uqu9cSQa-cd2uI7_0 2024-04-22T20:48:16.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6569003565139612960', time=2024-04-22T22:29:15.638285+0000, chain.objs.size()=1 2024-04-22T20:48:16.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xENK3MhHu2AqZP88RIS7DWb0AuoO47O_0 2024-04-22T20:48:16.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2587757304807559890', time=2024-04-22T22:29:15.790541+0000, chain.objs.size()=1 2024-04-22T20:48:16.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Kct86HQj4fX1jJWMedUA48qw81gNWXV_0 2024-04-22T20:48:16.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1273380371881060905', time=2024-04-22T22:29:16.100675+0000, chain.objs.size()=1 2024-04-22T20:48:16.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.StV9r8nuQq9bmmuOY9rDzSfSqdvN1um_0 2024-04-22T20:48:16.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.133003082609208516', time=2024-04-22T22:29:17.170218+0000, chain.objs.size()=1 2024-04-22T20:48:16.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zacJgfbS_2HP07Nzp5jQ_d1Wq0gLQWS_0 2024-04-22T20:48:16.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12872477160401275726', time=2024-04-22T22:29:17.433054+0000, chain.objs.size()=1 2024-04-22T20:48:16.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.UC9z68g88bE2cPtNFrJPhYQ7zyE-Y1__0 2024-04-22T20:48:16.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9340916801312354984', time=2024-04-22T22:29:17.588407+0000, chain.objs.size()=1 2024-04-22T20:48:16.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-apV_dfn5nh96m6ntsmuUDYshciseYI_0 2024-04-22T20:48:16.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2304051554671965283', time=2024-04-22T22:29:17.589198+0000, chain.objs.size()=1 2024-04-22T20:48:16.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.uNGFWasA1dC_P8fNte9XV-f5ui9eO6M_0 2024-04-22T20:48:16.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4171122643274243232', time=2024-04-22T22:29:23.176632+0000, chain.objs.size()=1 2024-04-22T20:48:16.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.321__shadow_.RHlxwDFDl7K2SLIanIYmpZ-p-OkIwnL_0 2024-04-22T20:48:16.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.167+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10432497174380352284', time=2024-04-22T22:29:25.180385+0000, chain.objs.size()=1 2024-04-22T20:48:16.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.167+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.324__shadow_.nPT84TLVMCMYBcphfzGceNvuwvcMv2y_0 2024-04-22T20:48:16.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.167+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11212096975264150648', time=2024-04-22T22:29:28.310087+0000, chain.objs.size()=1 2024-04-22T20:48:16.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.167+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.329__shadow_.HZJfJxKRonN5fxCzSQoP0BJuDrmzdd8_0 2024-04-22T20:48:16.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.167+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.932902060658718075', time=2024-04-22T22:29:53.012041+0000, chain.objs.size()=1 2024-04-22T20:48:16.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.167+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.366__shadow_.6tYRyCuOBq2Lns7o7bYZPqHA6TMU6eW_0 2024-04-22T20:48:16.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.168+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1961796444524718477', time=2024-04-22T22:29:55.967082+0000, chain.objs.size()=1 2024-04-22T20:48:16.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.168+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.370__shadow_.X8nXUG98G9HGtfluQHNrRF5qV5ZXbvw_0 2024-04-22T20:48:16.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.168+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3007997392807474759', time=2024-04-22T22:31:24.763714+0000, chain.objs.size()=1 2024-04-22T20:48:16.173 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.168+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.488__shadow_.TmHVafHNVDGOusYYgOk7_Sq4e2YLYoU_0 2024-04-22T20:48:16.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.168+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_1_41d9f8754e6b4cc1e7d867047fc6a0bd', time=2024-04-22T22:31:27.907195+0000, chain.objs.size()=1 2024-04-22T20:48:16.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.168+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_0 2024-04-22T20:48:16.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.168+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13353382330154273748', time=2024-04-22T22:33:48.317503+0000, chain.objs.size()=12 2024-04-22T20:48:16.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.168+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.1 2024-04-22T20:48:16.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.169+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__shadow_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.1_1 2024-04-22T20:48:16.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.169+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.Q9EoVliFxOnq9Z4YKEcWFTeobHDZxRBb.2 2024-04-22T20:48:16.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.169+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__shadow_mymultipart.Q9EoVliFxOnq9Z4YKEcWFTeobHDZxRBb.2_1 2024-04-22T20:48:16.175 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.169+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.3 2024-04-22T20:48:16.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.171+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__shadow_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.3_1 2024-04-22T20:48:16.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.171+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.4 2024-04-22T20:48:16.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.171+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__shadow_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.4_1 2024-04-22T20:48:16.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.172+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.5 2024-04-22T20:48:16.177 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.172+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__shadow_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.5_1 2024-04-22T20:48:16.177 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.172+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.6 2024-04-22T20:48:16.177 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.172+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__shadow_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.6_1 2024-04-22T20:48:16.178 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.172+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13277916220008111035', time=2024-04-22T22:35:34.051702+0000, chain.objs.size()=1 2024-04-22T20:48:16.178 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.172+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.517__shadow_.i0K46gPCx4C6yL-GfliNq-GuP2fIq-Y_0 2024-04-22T20:48:16.178 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.172+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10242892836795548161', time=2024-04-22T22:37:28.816239+0000, chain.objs.size()=1 2024-04-22T20:48:16.178 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.172+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:zEwVhPrOtgzX4mh4LOWE5QXfVpzcIho_.zR4MSryqy1TyojmeAzgWHLoUO7hGV_k_0 2024-04-22T20:48:16.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.174+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3542313879560471909', time=2024-04-22T22:37:31.098316+0000, chain.objs.size()=1 2024-04-22T20:48:16.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.174+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:enRif2vYWhcF-m7GPwOEH3yl7xXRa5B_.t0njhy_yQ7qj08HvAOGtUr0SLr-pxV7_0 2024-04-22T20:48:16.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.174+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4371961678872116320', time=2024-04-22T22:37:37.060847+0000, chain.objs.size()=1 2024-04-22T20:48:16.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.174+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.561__shadow_.pTUETrw0pYogpnDoEHyJ67bcIwC8Iji_0 2024-04-22T20:48:16.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2530427823838299753', time=2024-04-22T22:37:41.870840+0000, chain.objs.size()=1 2024-04-22T20:48:16.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow:.m4b3BgBMR0oXG.lndTUKn715mT2mAp_.eZGnEclvnHDqgrVR39uQ__Y1RElzJTg_0 2024-04-22T20:48:16.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7433218691395090741', time=2024-04-22T22:37:55.160140+0000, chain.objs.size()=1 2024-04-22T20:48:16.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:8xgE5ahaOGWNslmQrMpqBjzmgiMu2Ev_.1DEicdagLGaARWYM8yaaLvbX7Ovy_B7_0 2024-04-22T20:48:16.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4794104452273506832', time=2024-04-22T22:38:49.720284+0000, chain.objs.size()=1 2024-04-22T20:48:16.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:NuB7f65NhWLNgfXT1yKSwFu-Kn6yZgQ_.raUnSL4BlBi-561xzo-ukRZQ6cDF0B3_0 2024-04-22T20:48:16.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:qVjJ5iTGVxG9xodzTPYYDMlFBVXKTEK_.IzdXg9s7RTEeaFnlFpPPoAEZ3j1jBIA_1_a0a3ec9c0f7f23d456e90322e82c20c0', time=2024-04-22T22:38:51.924368+0000, chain.objs.size()=1 2024-04-22T20:48:16.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:qVjJ5iTGVxG9xodzTPYYDMlFBVXKTEK_.IzdXg9s7RTEeaFnlFpPPoAEZ3j1jBIA_0 2024-04-22T20:48:16.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1238247674640416260', time=2024-04-22T22:39:10.655523+0000, chain.objs.size()=1 2024-04-22T20:48:16.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.593__shadow_.vB7ssTDXEnl8Yl-7S0VrHMYeuSfiOVz_0 2024-04-22T20:48:16.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17666594903555086175', time=2024-04-22T22:42:04.629288+0000, chain.objs.size()=1 2024-04-22T20:48:16.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.628__shadow_.Jr5KAqim_BE0IYnNlpuOlWkObPCtWgq_0 2024-04-22T20:48:16.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1534445937426990945', time=2024-04-22T22:43:11.630119+0000, chain.objs.size()=1 2024-04-22T20:48:16.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.678__shadow:IX0ZNsFGYhzlUTPzV2Pk8aeF76BXWKo_.9kkfSSY0ffu-i0ZIxBNoMNULwpU5vW0_0 2024-04-22T20:48:16.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13401341912308948736', time=2024-04-22T22:43:12.927061+0000, chain.objs.size()=1 2024-04-22T20:48:16.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.177+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.680__shadow:3dHQZ2qgAV6whZZZRrsB3Il8EoLjeMx_.V3eQn8CGXN-ODRE-N4NTxtYbzj8xGPk_0 2024-04-22T20:48:16.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.188 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.185+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-22T20:48:16.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.188+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.188+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/48687' 2024-04-22T20:48:16.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2299752331779822595', time=2024-04-22T22:21:32.124279+0000, chain.objs.size()=3 2024-04-22T20:48:16.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.147__shadow_.lnxL_ihEJM-UGwMtWxSvHmSIhm4sL9S_0 2024-04-22T20:48:16.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.147__shadow_.lnxL_ihEJM-UGwMtWxSvHmSIhm4sL9S_1 2024-04-22T20:48:16.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.147__shadow_.lnxL_ihEJM-UGwMtWxSvHmSIhm4sL9S_2 2024-04-22T20:48:16.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4939174591186414813', time=2024-04-22T22:22:48.770504+0000, chain.objs.size()=1 2024-04-22T20:48:16.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.167__shadow_.wUj4ocOiweaZtgFYu6Tjd2yF_cEeREc_0 2024-04-22T20:48:16.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10389436339261835771', time=2024-04-22T22:25:16.305604+0000, chain.objs.size()=1 2024-04-22T20:48:16.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.223__shadow_.-RNqRoPueprgK6p8rYb1001SDldXqLL_0 2024-04-22T20:48:16.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8367904480962691429', time=2024-04-22T22:25:31.824776+0000, chain.objs.size()=1 2024-04-22T20:48:16.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.236__shadow_.BVEeBeWGtJiExkasX_uAPJ2EpMslutA_0 2024-04-22T20:48:16.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3857821699763937293', time=2024-04-22T22:25:31.827483+0000, chain.objs.size()=1 2024-04-22T20:48:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.236__shadow_.itH6eujV2_CE-r4-uA7i3aA8WvizJlR_0 2024-04-22T20:48:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2986186789790338157', time=2024-04-22T22:25:37.813814+0000, chain.objs.size()=1 2024-04-22T20:48:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.242__shadow_.44CikyjnkDsbG8r1NAyO-1A5gHyN4Rv_0 2024-04-22T20:48:16.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12109147793620618478', time=2024-04-22T22:25:39.804145+0000, chain.objs.size()=1 2024-04-22T20:48:16.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.244__shadow_.aA88Ja8-lGx5Rkmy0UgUDgQITb6gDL6_0 2024-04-22T20:48:16.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11497428439077778869', time=2024-04-22T22:25:40.801922+0000, chain.objs.size()=1 2024-04-22T20:48:16.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.245__shadow_.Fgs2t7alToPgWoVAZmAxfricbC0Lwid_0 2024-04-22T20:48:16.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6968337807095793716', time=2024-04-22T22:26:23.050709+0000, chain.objs.size()=1 2024-04-22T20:48:16.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6ByeBDzwTFMVADag-G-rFej8fJ5Awxu_0 2024-04-22T20:48:16.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7526446595251570339', time=2024-04-22T22:26:24.011532+0000, chain.objs.size()=1 2024-04-22T20:48:16.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.189+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sP-mucZTrC8cBc1jo3HZtP_hqxhZ2w-_0 2024-04-22T20:48:16.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.193+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16076144516560941912', time=2024-04-22T22:26:24.269406+0000, chain.objs.size()=1 2024-04-22T20:48:16.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.193+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.kYKvRujDYxFvWhYNR5Z85vZglpX67W2_0 2024-04-22T20:48:16.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.193+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14240666052307353650', time=2024-04-22T22:26:24.353707+0000, chain.objs.size()=1 2024-04-22T20:48:16.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.193+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wCC95EWZV7dwuagaAKn9BA8wpn48cfT_0 2024-04-22T20:48:16.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.193+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12205940348511737619', time=2024-04-22T22:26:24.646170+0000, chain.objs.size()=1 2024-04-22T20:48:16.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.193+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.TW8sWyOETtDmUKlTqJ7iU0r8YbhjER6_0 2024-04-22T20:48:16.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6833847490227779644', time=2024-04-22T22:26:24.786106+0000, chain.objs.size()=1 2024-04-22T20:48:16.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dj0Ihiu569czUdosprlU6Z_9e0J1TUe_0 2024-04-22T20:48:16.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6934529166292033422', time=2024-04-22T22:26:25.047482+0000, chain.objs.size()=1 2024-04-22T20:48:16.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8kdgUD4IAW2OTOclPm4saynDs_4ZUfD_0 2024-04-22T20:48:16.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7227426233399525798', time=2024-04-22T22:26:25.720340+0000, chain.objs.size()=1 2024-04-22T20:48:16.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VttBnKZg62_KgpBmkfZ4bu0ZuLD4x2D_0 2024-04-22T20:48:16.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12638354537122932191', time=2024-04-22T22:26:25.789200+0000, chain.objs.size()=1 2024-04-22T20:48:16.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qA2hi98ZIp0w3LAbX09sv1aFXQS0c9h_0 2024-04-22T20:48:16.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2386741993782502750', time=2024-04-22T22:26:26.020208+0000, chain.objs.size()=1 2024-04-22T20:48:16.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WK2BD-4r1-y7RhIZ7T-H5YD82z09BkJ_0 2024-04-22T20:48:16.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4761255291049709179', time=2024-04-22T22:26:26.161245+0000, chain.objs.size()=1 2024-04-22T20:48:16.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MxotFUSAoT5Nt5tQnMeZNrjBJqhDbCM_0 2024-04-22T20:48:16.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3212339257196963232', time=2024-04-22T22:26:26.942364+0000, chain.objs.size()=1 2024-04-22T20:48:16.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3NbrQBozvnM7K7k7aHJD97AtjTJ-y2k_0 2024-04-22T20:48:16.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11410273204790172414', time=2024-04-22T22:26:27.500700+0000, chain.objs.size()=1 2024-04-22T20:48:16.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rfumzOlQQLUT_y5oGmLlv6lEXydxwFN_0 2024-04-22T20:48:16.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.198+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14470914694290773004', time=2024-04-22T22:26:28.167690+0000, chain.objs.size()=1 2024-04-22T20:48:16.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.198+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YwdPkZhGKV8MymxmknLHdqagNRzIUzF_0 2024-04-22T20:48:16.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.198+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13872663646539817555', time=2024-04-22T22:26:28.633327+0000, chain.objs.size()=1 2024-04-22T20:48:16.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.198+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bOeAK9qq-9QFKdRtg0S_vo6AEFmebCc_0 2024-04-22T20:48:16.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.198+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13931604431459054870', time=2024-04-22T22:26:29.021876+0000, chain.objs.size()=1 2024-04-22T20:48:16.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.198+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._POGKWUHyMb44skLbTgKbEY1TWOWzIu_0 2024-04-22T20:48:16.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.199+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12030896177816688223', time=2024-04-22T22:26:29.939429+0000, chain.objs.size()=1 2024-04-22T20:48:16.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.199+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mpRPTNe-vHhn0_uxXHLhaSZ4WPCjnr0_0 2024-04-22T20:48:16.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.199+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9855468537862408131', time=2024-04-22T22:26:30.883713+0000, chain.objs.size()=1 2024-04-22T20:48:16.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.199+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Fl7l02s0cWL789hBbf42C_f_a8echMp_0 2024-04-22T20:48:16.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.199+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6184541328613983867', time=2024-04-22T22:26:31.001152+0000, chain.objs.size()=1 2024-04-22T20:48:16.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.199+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zSGoZi-IUwicspS_ilKaQ8bOEZzKdOd_0 2024-04-22T20:48:16.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12449779571667834047', time=2024-04-22T22:26:31.644889+0000, chain.objs.size()=1 2024-04-22T20:48:16.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UZsJE05Ucg3SVbUyvhGocpiuRHC_9fY_0 2024-04-22T20:48:16.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3113800318429839148', time=2024-04-22T22:26:32.088280+0000, chain.objs.size()=1 2024-04-22T20:48:16.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rM5zdmyGoYq0rnlo7W_vIeYHAqWTsCE_0 2024-04-22T20:48:16.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4952481509024238311', time=2024-04-22T22:26:32.090076+0000, chain.objs.size()=1 2024-04-22T20:48:16.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.i7DsTRaHFUj_w3EwN9zIDJaqSIOseir_0 2024-04-22T20:48:16.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2055221267730253622', time=2024-04-22T22:26:32.246839+0000, chain.objs.size()=1 2024-04-22T20:48:16.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.z30kHjSdrsDeqTJpCW2FhReAJO-oqCg_0 2024-04-22T20:48:16.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3176985927907261014', time=2024-04-22T22:26:32.332296+0000, chain.objs.size()=1 2024-04-22T20:48:16.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.200+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ICfbJOpb2COcErdcTE7p8uuZSiE4WpN_0 2024-04-22T20:48:16.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.201+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7040735521349515197', time=2024-04-22T22:26:33.758029+0000, chain.objs.size()=1 2024-04-22T20:48:16.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.201+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3cGlYhyJb7JWEtesdjxgOEbSNvfsk9i_0 2024-04-22T20:48:16.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.202+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11428539794251989191', time=2024-04-22T22:26:33.844092+0000, chain.objs.size()=1 2024-04-22T20:48:16.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.202+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PWI8dgdEQPUIaW9O-oAC2Gjtmn1VXLh_0 2024-04-22T20:48:16.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.202+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16259098212747437025', time=2024-04-22T22:26:41.156229+0000, chain.objs.size()=1 2024-04-22T20:48:16.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.202+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.258__shadow_.FnripJRTDB1iG-K9ge0thaExL1HWPNB_0 2024-04-22T20:48:16.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.202+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2336822748744333124', time=2024-04-22T22:26:44.023834+0000, chain.objs.size()=1 2024-04-22T20:48:16.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.202+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.261__shadow_.rxXMaRBGWYzKfCaPExo6TnpqSVWi8EB_0 2024-04-22T20:48:16.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12076870795237531327', time=2024-04-22T22:26:45.905459+0000, chain.objs.size()=1 2024-04-22T20:48:16.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.263__shadow_.houv9AELxh_v7f_05mu7Q_9dAIA5MV3_0 2024-04-22T20:48:16.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1695525982130549829', time=2024-04-22T22:26:53.863175+0000, chain.objs.size()=1 2024-04-22T20:48:16.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.271__shadow_.dEbQQZQg5UP5xP321Y9c9zMDjVsWIwE_0 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9605939449978238946', time=2024-04-22T22:27:07.856302+0000, chain.objs.size()=1 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.lZVFQuBnUTCGhKz1Brs6LIkSpHOG0SB_0 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11457460528601877754', time=2024-04-22T22:27:07.960154+0000, chain.objs.size()=1 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.8oaGIADYSUtA8bT2ZsijD8lZWp8l0-G_0 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7140541165406425814', time=2024-04-22T22:28:17.521088+0000, chain.objs.size()=1 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yQd7rGWlex4uNlDShuAk7CQL9YZZ5jI_0 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15969525096233793334', time=2024-04-22T22:28:17.713611+0000, chain.objs.size()=1 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xgWs-8uhEGhGSmZBIab_yl3aYSGUgYH_0 2024-04-22T20:48:16.211 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12658003863471528350', time=2024-04-22T22:28:17.930642+0000, chain.objs.size()=1 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rVSNol5KiyEr_JcAfojLuSzVCFji3gk_0 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.205+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5957420944098133120', time=2024-04-22T22:28:19.092983+0000, chain.objs.size()=1 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.205+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fKAP8NsfqYrlom9O0iy-KqJeGPiyZK6_0 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15552212128661535445', time=2024-04-22T22:28:20.058791+0000, chain.objs.size()=1 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8wSQZHql8k1LqYB8PfavQcyIqBcJa1X_0 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14261631931726234572', time=2024-04-22T22:28:21.113930+0000, chain.objs.size()=1 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Wmr7AfiuOnxvp5n8cEP9nmGMCSIy1h3_0 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.207+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11004424316196852973', time=2024-04-22T22:28:21.676284+0000, chain.objs.size()=1 2024-04-22T20:48:16.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.207+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xu8Ovixyqf5uhLfY8sLKz_vR5ePSiVK_0 2024-04-22T20:48:16.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.208+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7663681121343623957', time=2024-04-22T22:28:21.677038+0000, chain.objs.size()=1 2024-04-22T20:48:16.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.208+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bSUo4tbjNWJPSA4Iqc81I8puIc5CaOg_0 2024-04-22T20:48:16.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.208+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11880625226601403240', time=2024-04-22T22:28:22.325671+0000, chain.objs.size()=1 2024-04-22T20:48:16.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.208+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rv4okUJ3xPOwNG4ljGC_t4vrttGv2YG_0 2024-04-22T20:48:16.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.208+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12567213611001905289', time=2024-04-22T22:28:22.500930+0000, chain.objs.size()=1 2024-04-22T20:48:16.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.208+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._Narq-HlkQKoOQqJHSgHt6IAWnrTATX_0 2024-04-22T20:48:16.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.208+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4483220722388229422', time=2024-04-22T22:28:22.552268+0000, chain.objs.size()=1 2024-04-22T20:48:16.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.208+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0QFDfZA86xcENOl7MO9D6v-6uVfOM5U_0 2024-04-22T20:48:16.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15381058801434967186', time=2024-04-22T22:28:23.046897+0000, chain.objs.size()=1 2024-04-22T20:48:16.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TiwuxenIhAtGz2InwO-s4G3KN6TpwyK_0 2024-04-22T20:48:16.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3410412657958777193', time=2024-04-22T22:28:23.104177+0000, chain.objs.size()=1 2024-04-22T20:48:16.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.U9r48LvjBcpy1jnG7FeENEa9nhQgGzV_0 2024-04-22T20:48:16.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7840360568536224176', time=2024-04-22T22:28:23.481940+0000, chain.objs.size()=1 2024-04-22T20:48:16.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7KrH9ddidr_I__GCgSmUVL868RXriFC_0 2024-04-22T20:48:16.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4161852714900746139', time=2024-04-22T22:28:24.018792+0000, chain.objs.size()=1 2024-04-22T20:48:16.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._opVcflhtNLxeuWGHvS7dH6icWz03bY_0 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1466858933428340398', time=2024-04-22T22:28:24.551893+0000, chain.objs.size()=1 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._YhXqhnSyjv8TlX5mNuoxKbiwvOqwlC_0 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7631163566402293105', time=2024-04-22T22:28:24.993991+0000, chain.objs.size()=1 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BKPIA_ojNlzGAfP7l5tbLdg2ANevVys_0 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18156672103578413573', time=2024-04-22T22:28:25.639911+0000, chain.objs.size()=1 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8SBCvd_xXlJpC0W1g2glZ1nco-7AH9t_0 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6721762165847318032', time=2024-04-22T22:28:25.778862+0000, chain.objs.size()=1 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.I8WzcN1beOJXXvFkym9JRFrwv35IDHK_0 2024-04-22T20:48:16.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.212+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14952425542924068305', time=2024-04-22T22:28:25.779526+0000, chain.objs.size()=1 2024-04-22T20:48:16.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.212+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rIZ5WgkBAFv426kGg_nNnuz_FxYVax2_0 2024-04-22T20:48:16.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.212+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2438379896318311830', time=2024-04-22T22:28:25.954283+0000, chain.objs.size()=1 2024-04-22T20:48:16.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.212+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UBCeuEhhVPhn_XGQm1hCVFkYU49AsO4_0 2024-04-22T20:48:16.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.212+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10581841071125031279', time=2024-04-22T22:28:26.987135+0000, chain.objs.size()=1 2024-04-22T20:48:16.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.212+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.H14meL9zWdCzsrDEmlmF5la2DnzfH6M_0 2024-04-22T20:48:16.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.212+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18140190486428224890', time=2024-04-22T22:28:27.333517+0000, chain.objs.size()=1 2024-04-22T20:48:16.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.212+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RUxYpaK3Bv3240Ri5caDDBTrSXn7vJO_0 2024-04-22T20:48:16.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.213+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17654873751591886391', time=2024-04-22T22:28:27.894619+0000, chain.objs.size()=1 2024-04-22T20:48:16.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.213+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mSQWF1gMQ34j4xB6fScJflKpxfxWRS1_0 2024-04-22T20:48:16.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.213+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6323640890129571131', time=2024-04-22T22:28:27.994082+0000, chain.objs.size()=1 2024-04-22T20:48:16.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.213+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Pb93DYKFEAU31TKFU5ybk8r535STo0T_0 2024-04-22T20:48:16.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.214+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3447162287748086329', time=2024-04-22T22:28:28.581820+0000, chain.objs.size()=1 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.214+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.g9L132iHc3GusyyHKZD_59nrKQm6OMd_0 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.214+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18203825234887923053', time=2024-04-22T22:28:28.705854+0000, chain.objs.size()=1 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.214+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.O61sAcMpqxpaHFJw1G4XOlKnz69mMJL_0 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.216+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4535189329256331849', time=2024-04-22T22:28:28.821811+0000, chain.objs.size()=1 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.216+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XNhp87G9bX9rEm8qSy4xiRApbZTibcP_0 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8027479144281523203', time=2024-04-22T22:28:28.822329+0000, chain.objs.size()=1 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.jt_mdYDWDWFkatA6DN8Jv0lyXw2KaJ2_0 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9782224800772673711', time=2024-04-22T22:28:28.971057+0000, chain.objs.size()=1 2024-04-22T20:48:16.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0Get-LQO8lTbdnKhlDcgL56oD1NAvJT_0 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3866658964778633505', time=2024-04-22T22:28:29.191300+0000, chain.objs.size()=1 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tcM160qiZu7sy2_lxxTz61HMrBiNVEJ_0 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1089339484262680047', time=2024-04-22T22:29:06.457840+0000, chain.objs.size()=1 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.fxthM2E7dKgySoRBgP7npjReSOw3iJx_0 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4480297630671455882', time=2024-04-22T22:29:07.116511+0000, chain.objs.size()=1 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gMVvl4gIRQUO5gOiKMSK2xpGRL4FRjV_0 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2915112844208440325', time=2024-04-22T22:29:07.295339+0000, chain.objs.size()=1 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OBSV-3bsGphcKdT9SpmQrCdHLcqpcH7_0 2024-04-22T20:48:16.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5654117300249808510', time=2024-04-22T22:29:07.850062+0000, chain.objs.size()=1 2024-04-22T20:48:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1oltp3xSTBmQA00w87lx-tnOrj0aala_0 2024-04-22T20:48:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17810590076756170534', time=2024-04-22T22:29:08.275323+0000, chain.objs.size()=1 2024-04-22T20:48:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XQ9J4cMVFIqe_dCncJYNQ3H21Ee0N39_0 2024-04-22T20:48:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15044754677038391572', time=2024-04-22T22:29:08.276072+0000, chain.objs.size()=1 2024-04-22T20:48:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.q7PC5dnIPv8bV4XvSVWSObBEzTiefmp_0 2024-04-22T20:48:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13921970551151267890', time=2024-04-22T22:29:08.281898+0000, chain.objs.size()=1 2024-04-22T20:48:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8v8LFWQFph767CmCQg8K6qxTo0VB6Iq_0 2024-04-22T20:48:16.221 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12425535822719808108', time=2024-04-22T22:29:08.284817+0000, chain.objs.size()=1 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KllydxoZTvlqKFVHHgZyNZEAlRsT4YJ_0 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17910821447745402036', time=2024-04-22T22:29:08.661633+0000, chain.objs.size()=1 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.lWm7mmQreAZVpoNheOhpJZRnpzc-yyN_0 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7368867940514323330', time=2024-04-22T22:29:08.793658+0000, chain.objs.size()=1 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JMJ0ZIKzFrmyVpFapqiII7JJ3YreWd7_0 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1620076878040474423', time=2024-04-22T22:29:09.796614+0000, chain.objs.size()=1 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HR6HVSBPpnLAvWeBwtvtanynKuKRMCz_0 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9186856022902363918', time=2024-04-22T22:29:11.923691+0000, chain.objs.size()=1 2024-04-22T20:48:16.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.W0T3FHNtdfgn21xtYtINl09CvZtG_Ty_0 2024-04-22T20:48:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13611689682625693423', time=2024-04-22T22:29:12.453251+0000, chain.objs.size()=1 2024-04-22T20:48:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PXtdJbmKDYkBbwKrgDut75T7NhYv9dg_0 2024-04-22T20:48:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10072329118703387382', time=2024-04-22T22:29:12.651533+0000, chain.objs.size()=1 2024-04-22T20:48:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.79PRz3ESVWiDtaRtbCDi91gNcKWjVwt_0 2024-04-22T20:48:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13072267882915464181', time=2024-04-22T22:29:13.080409+0000, chain.objs.size()=1 2024-04-22T20:48:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.cCGUMD2ST8L9TEhtfnNBwGE2_JmCXEU_0 2024-04-22T20:48:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14957307449261657548', time=2024-04-22T22:29:13.301286+0000, chain.objs.size()=1 2024-04-22T20:48:16.223 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4pkJFVxTrRiJHe--tXpZXlyt4w5gMzr_0 2024-04-22T20:48:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10559041101764144441', time=2024-04-22T22:29:13.480973+0000, chain.objs.size()=1 2024-04-22T20:48:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.CkQgemr-4lhV8DD0iLTfaN_OKT9ia06_0 2024-04-22T20:48:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10918845287111836670', time=2024-04-22T22:29:13.503959+0000, chain.objs.size()=1 2024-04-22T20:48:16.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.221+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Mha7U79wqVlCpMivtnMjJznN1En2-RD_0 2024-04-22T20:48:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.225+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18056598387039954966', time=2024-04-22T22:29:14.360098+0000, chain.objs.size()=1 2024-04-22T20:48:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.225+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oRVBEM5FfdgbkjiGLz9EZcS2zwkCHiV_0 2024-04-22T20:48:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.225+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3676304668495597298', time=2024-04-22T22:29:14.379553+0000, chain.objs.size()=1 2024-04-22T20:48:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.225+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PWUCdeaS7WjjSUtS58vl3crYEPDC7L7_0 2024-04-22T20:48:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6841931882386112245', time=2024-04-22T22:29:15.640915+0000, chain.objs.size()=1 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.q0lvLOucdPvdeV2CaicwDvu8k3U2CQS_0 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8801469749144265302', time=2024-04-22T22:29:15.817131+0000, chain.objs.size()=1 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.fsFSM9PZr4dssg05c4cDv4ltT_NLP8Z_0 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6665661330017299103', time=2024-04-22T22:29:16.096508+0000, chain.objs.size()=1 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._4gnTEDxlL9kl3tBhg8g-mOMFro1YyA_0 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15030371169978418790', time=2024-04-22T22:29:16.338825+0000, chain.objs.size()=1 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Huy_QZ494RwWsFsFr15chgT5-_fKk9E_0 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1901240670544693024', time=2024-04-22T22:29:16.370249+0000, chain.objs.size()=1 2024-04-22T20:48:16.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1vYpcLcriIZjnpVPHlj4HHoAVnHhThM_0 2024-04-22T20:48:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1301608367682287569', time=2024-04-22T22:29:16.403426+0000, chain.objs.size()=1 2024-04-22T20:48:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2FA3gc9ytS1b1AnLkAwyDrDvx2oMyVv_0 2024-04-22T20:48:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17833795592987638222', time=2024-04-22T22:29:16.960596+0000, chain.objs.size()=1 2024-04-22T20:48:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gFWW_C4JmapQ6TPMa5cgu4SFO_lueUi_0 2024-04-22T20:48:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11494831666497356844', time=2024-04-22T22:29:17.055336+0000, chain.objs.size()=1 2024-04-22T20:48:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.uu5eE7B8f9iYoZ3AiDdWfIFaSPU3gUP_0 2024-04-22T20:48:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14413942514446899813', time=2024-04-22T22:29:17.067163+0000, chain.objs.size()=1 2024-04-22T20:48:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BNpkPFCTSmGAj-ReKDo61s9aMPaZfSg_0 2024-04-22T20:48:16.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.233+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/48687 2024-04-22T20:48:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=27, truncated=0, next_marker='' 2024-04-22T20:48:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14987743960298889296', time=2024-04-22T22:29:17.108862+0000, chain.objs.size()=1 2024-04-22T20:48:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.i3yuF3KBVNuuDxV9e8noMyUGW1eCTcL_0 2024-04-22T20:48:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6181645505857945715', time=2024-04-22T22:29:17.257305+0000, chain.objs.size()=1 2024-04-22T20:48:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1sytat-OwO9NFclKyq1j7NbM5eqhKsC_0 2024-04-22T20:48:16.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14432520815747003605', time=2024-04-22T22:29:17.502416+0000, chain.objs.size()=1 2024-04-22T20:48:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GAo_WJPZOBLaO9wNYx16eYnoZJWbigQ_0 2024-04-22T20:48:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14904900889741155307', time=2024-04-22T22:29:24.517748+0000, chain.objs.size()=1 2024-04-22T20:48:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.323__shadow_.KpI34Hf4HFX8a8XkY4S175Js1uJNbEK_0 2024-04-22T20:48:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5582982820925124708', time=2024-04-22T22:29:25.951054+0000, chain.objs.size()=1 2024-04-22T20:48:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.325__shadow_.rZAOBJ-NroHn_uNZb57arSqA2krJfQa_0 2024-04-22T20:48:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17278161365309003847', time=2024-04-22T22:30:30.793818+0000, chain.objs.size()=1 2024-04-22T20:48:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.421__shadow_.eWxuhmWwBoSr2PX32EF9cikXpb1GP3R_0 2024-04-22T20:48:16.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4297675429097404014', time=2024-04-22T22:31:01.049266+0000, chain.objs.size()=1 2024-04-22T20:48:16.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.455__shadow_.kk-Y3BMKdgU5jPWEZsgl_ad8cECSkkp_0 2024-04-22T20:48:16.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1985402441673181582', time=2024-04-22T22:31:10.481482+0000, chain.objs.size()=1 2024-04-22T20:48:16.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.5wjHyz4i7_2ua28Bi0D9QYekVitteaC_0 2024-04-22T20:48:16.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17567778960852013268', time=2024-04-22T22:31:17.110695+0000, chain.objs.size()=1 2024-04-22T20:48:16.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.477__shadow_.k143HZyb7gHF87tUUMqDsGcUp9OasYi_0 2024-04-22T20:48:16.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.485__shadow_.w2l_3sOFmMOJVmjO1JZYrPIBbJRaO_4_1_63773a5062a8befe5f34b311fd0a243b', time=2024-04-22T22:31:22.118844+0000, chain.objs.size()=1 2024-04-22T20:48:16.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.485__shadow_.w2l_3sOFmMOJVmjO1JZYrPIBbJRaO_4_0 2024-04-22T20:48:16.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10813508250975281978', time=2024-04-22T22:31:23.450157+0000, chain.objs.size()=1 2024-04-22T20:48:16.236 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.486__shadow_.HUx_WbM68znI7__I0BODOKzAROauthW_0 2024-04-22T20:48:16.236 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12182099885116298715', time=2024-04-22T22:31:27.905158+0000, chain.objs.size()=1 2024-04-22T20:48:16.236 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.236+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_0 2024-04-22T20:48:16.236 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.239+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7720405952085303802', time=2024-04-22T22:32:56.428504+0000, chain.objs.size()=3 2024-04-22T20:48:16.236 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.239+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~nYGH7ryZe5Q0Od3BAHNuC38g6hq70Ap.1 2024-04-22T20:48:16.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.240+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~nYGH7ryZe5Q0Od3BAHNuC38g6hq70Ap.2 2024-04-22T20:48:16.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~nYGH7ryZe5Q0Od3BAHNuC38g6hq70Ap.3 2024-04-22T20:48:16.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5176649982367202646', time=2024-04-22T22:35:09.493717+0000, chain.objs.size()=2 2024-04-22T20:48:16.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~oFuNqyp7DeI77E1XE9DPkSK59kixmQD.1 2024-04-22T20:48:16.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~oFuNqyp7DeI77E1XE9DPkSK59kixmQD.2 2024-04-22T20:48:16.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14367074740850101326', time=2024-04-22T22:36:55.525808+0000, chain.objs.size()=1 2024-04-22T20:48:16.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.544__shadow_.z3u69lMl8nEMgkZy1DlMggnzn9bRdPY_0 2024-04-22T20:48:16.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10456735563167247535', time=2024-04-22T22:37:23.376574+0000, chain.objs.size()=1 2024-04-22T20:48:16.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:1lWr5SjN1Erj2-iAJYPSIEfUPE1bH0p_.QQGoSz4xczucvSscODjspdyjPEakxHL_0 2024-04-22T20:48:16.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7061839268149995072', time=2024-04-22T22:37:25.195501+0000, chain.objs.size()=1 2024-04-22T20:48:16.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:uCgBTQAt7E0paIDWIi4eHKlHL6Kp-ia_.DZVHTdWKmQgauN5xSRVsrAyJyBMHI8A_0 2024-04-22T20:48:16.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4471357848280117550', time=2024-04-22T22:37:36.885438+0000, chain.objs.size()=1 2024-04-22T20:48:16.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.561__shadow_.CrGEedtX216p-DmJiROhgLstPnhrbfy_0 2024-04-22T20:48:16.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6362952455746970532', time=2024-04-22T22:37:42.114228+0000, chain.objs.size()=1 2024-04-22T20:48:16.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow:DE.xwcBp22fN2SQrOu1LpTJGBG.J73J_.urbu5tL_0vofnMAtuq8s3Q4TTfacoB2_0 2024-04-22T20:48:16.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8687169677481467194', time=2024-04-22T22:37:59.477325+0000, chain.objs.size()=1 2024-04-22T20:48:16.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:mxmePj5CwsVQCRxO1sK3my2KxwNKt5S_.Q6756ZIhlVm_7ZpHOH6TjxJzU7GW_yq_0 2024-04-22T20:48:16.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3510261378727257731', time=2024-04-22T22:38:00.410946+0000, chain.objs.size()=1 2024-04-22T20:48:16.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.241+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:LCebvXObWbcvCEW91Mgqfnp1cBJx4Yv_.Cpsiu8ET_XBxWAGRUUxFbkbeasxjOVL_0 2024-04-22T20:48:16.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.246+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3174150228721405503', time=2024-04-22T22:38:05.413110+0000, chain.objs.size()=1 2024-04-22T20:48:16.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.246+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:hZ8AJfvOg5f5VCz7Ad.8G-2VjlDj2F-_.FjjNG-esl0Dc-ZhG_4oiAoWQHPS0DtY_0 2024-04-22T20:48:16.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.246+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:-dVByXoDlV-EFnizGQqWv8SzVp2FAWe_.kWtQjnq6VIeWmMQahTWNPn-ZtLhy_ww_1_5e2792e2c0e0ab0c99de0fe79cdc8b4c', time=2024-04-22T22:38:52.199587+0000, chain.objs.size()=1 2024-04-22T20:48:16.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.246+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:-dVByXoDlV-EFnizGQqWv8SzVp2FAWe_.kWtQjnq6VIeWmMQahTWNPn-ZtLhy_ww_0 2024-04-22T20:48:16.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.246+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10193534800198295090', time=2024-04-22T22:41:52.561096+0000, chain.objs.size()=1 2024-04-22T20:48:16.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.246+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.618__shadow_.kDy-MEJTuwFCDAC3LDGiob6DnHt2wex_0 2024-04-22T20:48:16.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.246+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5085738808442072061', time=2024-04-22T22:43:03.680500+0000, chain.objs.size()=1 2024-04-22T20:48:16.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.663__shadow_.4fur7zFcu4PjLp7190QWx707PpwGo5f_0 2024-04-22T20:48:16.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8271539261481732470', time=2024-04-22T22:43:14.295271+0000, chain.objs.size()=1 2024-04-22T20:48:16.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.682__shadow:p8E87JLfrsLWDavHErkYF3Sagha6Gx7_.xLXQ6AQ0J-3xT-_brmTBIlPDoZNdFUO_0 2024-04-22T20:48:16.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.703__shadow_.jfSzKLFSNJUO2rjPebH1vRTtIzB_CX2_1_ee771184f592437ef50fac3673346166', time=2024-04-22T22:43:47.547122+0000, chain.objs.size()=1 2024-04-22T20:48:16.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.703__shadow_.jfSzKLFSNJUO2rjPebH1vRTtIzB_CX2_0 2024-04-22T20:48:16.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.254+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-22T20:48:16.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.256+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45358' 2024-04-22T20:48:16.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.256+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9803799960430128399', time=2024-04-22T22:21:39.790801+0000, chain.objs.size()=5 2024-04-22T20:48:16.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.256+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-FROZEN.2~2tWJyXY925HEO55iyBRmpnZUbGIlfBv.1 2024-04-22T20:48:16.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__shadow_mymultipart-FROZEN.2~2tWJyXY925HEO55iyBRmpnZUbGIlfBv.1_1 2024-04-22T20:48:16.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-FROZEN.2~2tWJyXY925HEO55iyBRmpnZUbGIlfBv.2 2024-04-22T20:48:16.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__shadow_mymultipart-FROZEN.2~2tWJyXY925HEO55iyBRmpnZUbGIlfBv.2_1 2024-04-22T20:48:16.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-FROZEN.2~2tWJyXY925HEO55iyBRmpnZUbGIlfBv.3 2024-04-22T20:48:16.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_NJXhmm7tJHeHD3OK4FBnt-pGjiDjOaM', time=2024-04-22T22:22:28.128400+0000, chain.objs.size()=1 2024-04-22T20:48:16.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.154__shadow_.6mNb_DrDaXiCo7lnpK-asWUogjx7HoR_0 2024-04-22T20:48:16.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17238155237915041144', time=2024-04-22T22:25:15.284500+0000, chain.objs.size()=1 2024-04-22T20:48:16.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.222__shadow_.6cc7N7gU2rGZTXXctapyNsCNZq_1vRW_0 2024-04-22T20:48:16.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15336238192775121275', time=2024-04-22T22:25:17.293668+0000, chain.objs.size()=1 2024-04-22T20:48:16.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.224__shadow_.2i2DE0TlDHUhvDTOFBlyCKjE1cDCAsN_0 2024-04-22T20:48:16.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5676143419969906833', time=2024-04-22T22:25:25.429798+0000, chain.objs.size()=1 2024-04-22T20:48:16.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.231__shadow_.do2fMtoKyUXwdy7_HCZVTKs0MqqALXT_0 2024-04-22T20:48:16.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1450969094388016725', time=2024-04-22T22:25:29.852207+0000, chain.objs.size()=1 2024-04-22T20:48:16.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.234__shadow_.f7dunoCIcoPZfigQT8AjjUbq_9_tTf0_0 2024-04-22T20:48:16.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2756920448874195475', time=2024-04-22T22:25:29.862432+0000, chain.objs.size()=1 2024-04-22T20:48:16.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.234__shadow_.yCcXAYQTDc--xzbhTFwqlI45BeCIy3G_0 2024-04-22T20:48:16.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18152790449264370823', time=2024-04-22T22:25:30.839465+0000, chain.objs.size()=1 2024-04-22T20:48:16.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.235__shadow_.HGAPTZq8Csas31tTf_5cqon3RG8jAml_0 2024-04-22T20:48:16.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.260+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12592657389645653017', time=2024-04-22T22:25:30.840388+0000, chain.objs.size()=1 2024-04-22T20:48:16.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.260+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.235__shadow_.yHC1uHddeB0Mck8d-rd51hFj2hbOYrv_0 2024-04-22T20:48:16.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.260+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4625864314639140136', time=2024-04-22T22:25:33.800223+0000, chain.objs.size()=1 2024-04-22T20:48:16.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.260+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.238__shadow_.iuWK9G_ynSI3FTiv8YJaVgorl-0chUu_0 2024-04-22T20:48:16.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.260+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16264117806076587169', time=2024-04-22T22:25:40.800665+0000, chain.objs.size()=1 2024-04-22T20:48:16.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.260+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.245__shadow_.u3Tr_bzf8q3PvtwPXjaXAqUt_nyr7xz_0 2024-04-22T20:48:16.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17059489833843399755', time=2024-04-22T22:25:45.673041+0000, chain.objs.size()=1 2024-04-22T20:48:16.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.250__shadow_.8v6B98pJTY_iR0ouFdkwnxwzK-qPOIS_0 2024-04-22T20:48:16.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9763295713863557375', time=2024-04-22T22:26:22.780418+0000, chain.objs.size()=1 2024-04-22T20:48:16.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wZ9GuLSptwfjQpuWeP4ePCLPDyKJaPV_0 2024-04-22T20:48:16.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16068258342989422117', time=2024-04-22T22:26:22.974577+0000, chain.objs.size()=1 2024-04-22T20:48:16.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ka-Jdgbo-RgItx5QBCjKZguc5DlOrbx_0 2024-04-22T20:48:16.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10207129837434412139', time=2024-04-22T22:26:23.081373+0000, chain.objs.size()=1 2024-04-22T20:48:16.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.yt6y-4xzLSvR8jQv4ZRauux2wz8niO-_0 2024-04-22T20:48:16.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10084682615060026152', time=2024-04-22T22:26:23.132406+0000, chain.objs.size()=1 2024-04-22T20:48:16.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uGxEYVcp4pZqbcv1jvpQ-B1UwA1yzW-_0 2024-04-22T20:48:16.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10749140096750826649', time=2024-04-22T22:26:23.437216+0000, chain.objs.size()=1 2024-04-22T20:48:16.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aaVzvtX0sWwhbra3H0slBEDOGMim7wN_0 2024-04-22T20:48:16.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18172975027215908126', time=2024-04-22T22:26:23.489405+0000, chain.objs.size()=1 2024-04-22T20:48:16.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.k7S1o7OP9yUt31lZGNGhTLTG9CtkZVA_0 2024-04-22T20:48:16.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17305407433708032661', time=2024-04-22T22:26:23.943052+0000, chain.objs.size()=1 2024-04-22T20:48:16.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.myc_Tj6uKW1bSYOzvabXzF78-UAFkEA_0 2024-04-22T20:48:16.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.263+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17568720637653220539', time=2024-04-22T22:26:25.064646+0000, chain.objs.size()=1 2024-04-22T20:48:16.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.263+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.88-ivzsFmxPX_GIK68KmC4ZP8Mj-JUo_0 2024-04-22T20:48:16.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.263+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.888093705636473788', time=2024-04-22T22:26:25.092605+0000, chain.objs.size()=1 2024-04-22T20:48:16.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.263+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.kDkU2W3G8-nmq5Cp0sL19OForEN0NyL_0 2024-04-22T20:48:16.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.263+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7538003459135114717', time=2024-04-22T22:26:26.409968+0000, chain.objs.size()=1 2024-04-22T20:48:16.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.263+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SzeyhQW2l1EqqABIlBEcVJxuIIDjM1a_0 2024-04-22T20:48:16.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.264+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11920967081331202051', time=2024-04-22T22:26:27.728224+0000, chain.objs.size()=1 2024-04-22T20:48:16.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.264+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.whkBhp4yWnuBE2bGRs-jLR96xKFNKTV_0 2024-04-22T20:48:16.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.264+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17325919428168652962', time=2024-04-22T22:26:27.866563+0000, chain.objs.size()=1 2024-04-22T20:48:16.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.264+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZRBqpDTMmxmPMvOl1RXh9F4EbnJluYb_0 2024-04-22T20:48:16.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.264+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7388849800865581986', time=2024-04-22T22:26:28.025460+0000, chain.objs.size()=1 2024-04-22T20:48:16.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.264+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PTaQqctowHOug934xxQJ9NK18ilEfaQ_0 2024-04-22T20:48:16.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.265+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9374689492958532058', time=2024-04-22T22:26:28.493005+0000, chain.objs.size()=1 2024-04-22T20:48:16.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.265+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Sd9wbBJjGuOzeDCt9X1AJ0FN8MSAwJN_0 2024-04-22T20:48:16.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4633211334599395686', time=2024-04-22T22:26:28.990207+0000, chain.objs.size()=1 2024-04-22T20:48:16.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dfGA6GYYumlNAs3F6mUmgpvq4OCT-Fh_0 2024-04-22T20:48:16.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5397606888110041498', time=2024-04-22T22:26:29.220812+0000, chain.objs.size()=1 2024-04-22T20:48:16.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xIPdGg1i3hy-YNDqvJwHrq5QSqIIdqC_0 2024-04-22T20:48:16.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3106289068372423617', time=2024-04-22T22:26:29.832848+0000, chain.objs.size()=1 2024-04-22T20:48:16.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ccZIEEuc9lDrxF7Cp8ki2UGC9zEmsoq_0 2024-04-22T20:48:16.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3461755921284917414', time=2024-04-22T22:26:30.126444+0000, chain.objs.size()=1 2024-04-22T20:48:16.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JFDwWYecAk7RwPPAzraBrKiVApKLS45_0 2024-04-22T20:48:16.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2975776115618658405', time=2024-04-22T22:26:30.126925+0000, chain.objs.size()=1 2024-04-22T20:48:16.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.266+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.A33jhASOSadezzqtSjs56K57a8jfYPY_0 2024-04-22T20:48:16.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9695181723991226087', time=2024-04-22T22:26:30.157072+0000, chain.objs.size()=1 2024-04-22T20:48:16.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SR8R5gPIjFBWe8KkCBK8fghorLAxs92_0 2024-04-22T20:48:16.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.268+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12561152570148001947', time=2024-04-22T22:26:30.545785+0000, chain.objs.size()=1 2024-04-22T20:48:16.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.268+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LG2_u-1-44IfFXhT-u7IMcp_FUbx--b_0 2024-04-22T20:48:16.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.268+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11765584220037259748', time=2024-04-22T22:26:31.147752+0000, chain.objs.size()=1 2024-04-22T20:48:16.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.268+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.F-WgxGr3uDvyE2wrAYdwicqZ-q8R2OI_0 2024-04-22T20:48:16.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.268+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10272626144514393735', time=2024-04-22T22:26:31.159783+0000, chain.objs.size()=1 2024-04-22T20:48:16.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.268+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.r4XNFW_XdAKuri1nLXthVZlhGoJ9GJR_0 2024-04-22T20:48:16.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.268+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14409199426833593396', time=2024-04-22T22:26:31.376492+0000, chain.objs.size()=1 2024-04-22T20:48:16.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.268+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Hsw0IDobL0YR-5wj9o4Q3eu2t75onSy_0 2024-04-22T20:48:16.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12779795881310153166', time=2024-04-22T22:26:32.306838+0000, chain.objs.size()=1 2024-04-22T20:48:16.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qumC1NcFCeFL14FtkFLMVIeCJGCQezO_0 2024-04-22T20:48:16.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.431009512859804148', time=2024-04-22T22:26:32.564895+0000, chain.objs.size()=1 2024-04-22T20:48:16.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UCa5UphMSxY7AuWsffdnznJQDU9rbLg_0 2024-04-22T20:48:16.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.479577728855399743', time=2024-04-22T22:26:33.128802+0000, chain.objs.size()=1 2024-04-22T20:48:16.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.AR4uO0A_ajYr5Zz0ayGgb3KsmVkYwH1_0 2024-04-22T20:48:16.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3397974620951788366', time=2024-04-22T22:26:33.170212+0000, chain.objs.size()=1 2024-04-22T20:48:16.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sxXJJafBkQqTDm7vWmP_StFLKOlN3bE_0 2024-04-22T20:48:16.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5592428507178213486', time=2024-04-22T22:26:33.638137+0000, chain.objs.size()=1 2024-04-22T20:48:16.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.270+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.50VYQgy_nylUI1viVWApdPhIf1FcHoy_0 2024-04-22T20:48:16.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.271+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9911936362192357489', time=2024-04-22T22:26:33.672180+0000, chain.objs.size()=1 2024-04-22T20:48:16.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.271+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4wgoMeZ2PX30cI_F8kLS9MiXxkn2B4t_0 2024-04-22T20:48:16.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.271+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15523682741909215690', time=2024-04-22T22:26:34.172081+0000, chain.objs.size()=1 2024-04-22T20:48:16.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.271+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BlGRcH88jJZ24qadT4tgoDLp_3gF8rI_0 2024-04-22T20:48:16.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.271+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15475846338233024329', time=2024-04-22T22:26:37.350877+0000, chain.objs.size()=1 2024-04-22T20:48:16.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.271+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.254__shadow_.pnOipcVgBnKtDuOqoxtxk0AEd9eL8Kj_0 2024-04-22T20:48:16.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.272+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.783427374693950021', time=2024-04-22T22:27:05.710381+0000, chain.objs.size()=1 2024-04-22T20:48:16.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.272+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.seqekVC4hRdY7r5QkJulOz2PHivf4hv_0 2024-04-22T20:48:16.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.273+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6289200278597450673', time=2024-04-22T22:27:09.959573+0000, chain.objs.size()=1 2024-04-22T20:48:16.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.273+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.284__shadow_.VFr38UgRhIo_9Pb4-gW21SIB3u2rds3_0 2024-04-22T20:48:16.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.273+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4810015903758389497', time=2024-04-22T22:27:37.261729+0000, chain.objs.size()=1 2024-04-22T20:48:16.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.273+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.308__shadow:kHAJkpm8nfMF2ip9YfkrfzjKIDJU8Rc_.FAXq4VNYaq_9rjR_xqOeXzu9CmUDSus_0 2024-04-22T20:48:16.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.273+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7664498968973693536', time=2024-04-22T22:28:18.414536+0000, chain.objs.size()=1 2024-04-22T20:48:16.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.273+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZJHJ9qCeb4ORNoisERrFQx-vqkpLsiq_0 2024-04-22T20:48:16.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14555334762478764302', time=2024-04-22T22:28:18.525466+0000, chain.objs.size()=1 2024-04-22T20:48:16.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tIqqHsPd9uNKswThsgLIo_OI29NnOgA_0 2024-04-22T20:48:16.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4714221704647916850', time=2024-04-22T22:28:19.089184+0000, chain.objs.size()=1 2024-04-22T20:48:16.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XD81bA2SE2ou3ZPwjeb0oDamGg1EXlv_0 2024-04-22T20:48:16.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9962185746770184517', time=2024-04-22T22:28:19.230044+0000, chain.objs.size()=1 2024-04-22T20:48:16.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lAc9lqF6Kyd514oITAuoX1SToOAvM_Y_0 2024-04-22T20:48:16.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7280634158484120843', time=2024-04-22T22:28:19.342597+0000, chain.objs.size()=1 2024-04-22T20:48:16.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dZb2jPDDCtV9yL30mfhnJzGtniBlscR_0 2024-04-22T20:48:16.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.275+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16863322580253791672', time=2024-04-22T22:28:19.422134+0000, chain.objs.size()=1 2024-04-22T20:48:16.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.275+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.s6-fTB8VInYGFfL_muMqkaHkKFmGWBX_0 2024-04-22T20:48:16.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18275914190116497200', time=2024-04-22T22:28:19.569261+0000, chain.objs.size()=1 2024-04-22T20:48:16.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.iZV0_QFPSqmiocP9bNWvuPKEnpPDMnA_0 2024-04-22T20:48:16.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7264559552799821956', time=2024-04-22T22:28:19.942849+0000, chain.objs.size()=1 2024-04-22T20:48:16.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZkGC3sanE6PS0jREJOZ-27s7IWXfj1C_0 2024-04-22T20:48:16.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15247828453147801780', time=2024-04-22T22:28:19.943513+0000, chain.objs.size()=1 2024-04-22T20:48:16.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5_aHGk0qbrSk78VxDzvtHjEW_4jw3C5_0 2024-04-22T20:48:16.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15483634785501429453', time=2024-04-22T22:28:20.658129+0000, chain.objs.size()=1 2024-04-22T20:48:16.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.m3dX6n8V2EKtPMrXTR5H8L2_shr4wU__0 2024-04-22T20:48:16.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17574853770097331091', time=2024-04-22T22:28:20.978832+0000, chain.objs.size()=1 2024-04-22T20:48:16.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Y75uT6B0vsc-VsgylnHv7BwRdIQSGUX_0 2024-04-22T20:48:16.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.197146906788826206', time=2024-04-22T22:28:21.266110+0000, chain.objs.size()=1 2024-04-22T20:48:16.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JusI_pWZqysUZYqInKUaejbqTwKiZXn_0 2024-04-22T20:48:16.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17391414893428456982', time=2024-04-22T22:28:21.328608+0000, chain.objs.size()=1 2024-04-22T20:48:16.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.277+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xOMAIqaRhMXO2izi6z3fJBuaFA4m3v2_0 2024-04-22T20:48:16.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11904819325338955819', time=2024-04-22T22:28:21.393088+0000, chain.objs.size()=1 2024-04-22T20:48:16.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7YAs4ufQpzBs4YwAcnEe7yggm6JVmEk_0 2024-04-22T20:48:16.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.279+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12164113978258706750', time=2024-04-22T22:28:21.441477+0000, chain.objs.size()=1 2024-04-22T20:48:16.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.279+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.VqeCttGcSL8gS9Nr4NTf7SmLlXGGFs5_0 2024-04-22T20:48:16.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.279+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15243719520270799422', time=2024-04-22T22:28:22.330162+0000, chain.objs.size()=1 2024-04-22T20:48:16.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.279+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.t4IVgzW-D4KTnkv40KhTn_zd-UvNuXI_0 2024-04-22T20:48:16.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.279+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17175074697044782756', time=2024-04-22T22:28:22.356202+0000, chain.objs.size()=1 2024-04-22T20:48:16.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.279+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-qIVh4PFuWV1QOTA9VC0tvMsKjRfopZ_0 2024-04-22T20:48:16.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12335121470876995426', time=2024-04-22T22:28:22.640051+0000, chain.objs.size()=1 2024-04-22T20:48:16.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-iBD5nkvNyn915BWYA5Yl9cxXlG2Q34_0 2024-04-22T20:48:16.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15713108250688780338', time=2024-04-22T22:28:22.728157+0000, chain.objs.size()=1 2024-04-22T20:48:16.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rquWpjgDddXELI3YtGhpvqd8lvxPvKc_0 2024-04-22T20:48:16.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12674766425220997111', time=2024-04-22T22:28:24.851055+0000, chain.objs.size()=1 2024-04-22T20:48:16.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.hkEwc1B1AjYqbirAHZCOoBbS-IsgjuH_0 2024-04-22T20:48:16.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12994742884627359159', time=2024-04-22T22:28:24.934915+0000, chain.objs.size()=1 2024-04-22T20:48:16.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PROi1bPgk_w7uRPnft5Jt7XWB9SSYB0_0 2024-04-22T20:48:16.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6497879088872056038', time=2024-04-22T22:28:25.737419+0000, chain.objs.size()=1 2024-04-22T20:48:16.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.leBHOYqmadC9YBwloB7Sd5K9EKSywYS_0 2024-04-22T20:48:16.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11156529973686790689', time=2024-04-22T22:28:26.426381+0000, chain.objs.size()=1 2024-04-22T20:48:16.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.281+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.A-BvVOK_nebV0D7CCaXH8YEEwsbtc-R_0 2024-04-22T20:48:16.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13764090290712865683', time=2024-04-22T22:28:26.560338+0000, chain.objs.size()=1 2024-04-22T20:48:16.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.s-QAQV6CoKT7iJSwkAz5oebBp9ooqiI_0 2024-04-22T20:48:16.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14144222122639774529', time=2024-04-22T22:28:26.985386+0000, chain.objs.size()=1 2024-04-22T20:48:16.280 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ewW-dKqBk49xDHJjax0OHQOy8354ffX_0 2024-04-22T20:48:16.281 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16264683670298485828', time=2024-04-22T22:28:27.139251+0000, chain.objs.size()=1 2024-04-22T20:48:16.281 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NNJrb2aeLTJUxidOWCLAFF_t8sSuFNF_0 2024-04-22T20:48:16.281 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12173631251439396283', time=2024-04-22T22:28:27.516129+0000, chain.objs.size()=1 2024-04-22T20:48:16.281 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qgnPr6WQZBrn2_MsQg1wSmW_ktxbcm6_0 2024-04-22T20:48:16.281 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.888843947804782474', time=2024-04-22T22:28:27.787480+0000, chain.objs.size()=1 2024-04-22T20:48:16.281 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.283+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cn6F8zyUYOI5GM3mS25rEe1znl7DTl6_0 2024-04-22T20:48:16.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8131559899095490224', time=2024-04-22T22:28:28.039934+0000, chain.objs.size()=1 2024-04-22T20:48:16.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.284+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vrtXccGXhw1PR_kk9eMT-wEGJ2SpHas_0 2024-04-22T20:48:16.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.286+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16308125001694782140', time=2024-04-22T22:28:28.523557+0000, chain.objs.size()=1 2024-04-22T20:48:16.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.286+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lw8bT_P9iqeLrEhjG19aebHwytOBI03_0 2024-04-22T20:48:16.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.286+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9406188015984854932', time=2024-04-22T22:28:28.688850+0000, chain.objs.size()=1 2024-04-22T20:48:16.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.286+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.U13Cd5DJi0v9B7DsnAtSBcZ4Ca6myEg_0 2024-04-22T20:48:16.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.286+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7644687641633424416', time=2024-04-22T22:29:07.175824+0000, chain.objs.size()=1 2024-04-22T20:48:16.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.286+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rdpPwIv5S3zNAWFjWmAuGy2-5hZ7EvV_0 2024-04-22T20:48:16.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.287+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16521958867479629242', time=2024-04-22T22:29:07.286695+0000, chain.objs.size()=1 2024-04-22T20:48:16.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.287+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.c1C0YlxLGGDiFoCVpF9VZ_aDHrur7Q0_0 2024-04-22T20:48:16.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.287+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16117110340432396203', time=2024-04-22T22:29:08.436039+0000, chain.objs.size()=1 2024-04-22T20:48:16.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.287+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1oMM1a6fX6XzlZvgkp1YGzQZkSSNPwn_0 2024-04-22T20:48:16.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.287+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2638494496260819933', time=2024-04-22T22:29:09.551132+0000, chain.objs.size()=1 2024-04-22T20:48:16.285 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.287+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OtS8-97xJiQA3_UEZI-q0FoaprLEfHz_0 2024-04-22T20:48:16.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12136553343993288884', time=2024-04-22T22:29:11.692562+0000, chain.objs.size()=1 2024-04-22T20:48:16.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IH1n6puCuZscUrJ7YNZQAfyx4OSNgDx_0 2024-04-22T20:48:16.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12904194490101271123', time=2024-04-22T22:29:11.906147+0000, chain.objs.size()=1 2024-04-22T20:48:16.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OXTtCbn53ykaDdHtsx7JzKMqnfPk4DI_0 2024-04-22T20:48:16.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3463695629594202776', time=2024-04-22T22:29:12.006098+0000, chain.objs.size()=1 2024-04-22T20:48:16.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6nEoCDDybbfcjOp7sYBCDPh1mP5Q3Kq_0 2024-04-22T20:48:16.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17174677849698553815', time=2024-04-22T22:29:12.609867+0000, chain.objs.size()=1 2024-04-22T20:48:16.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-Fdtrz2pYURwPU3yvOHddbkxM5jEjaU_0 2024-04-22T20:48:16.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4300889017495489382', time=2024-04-22T22:29:13.386050+0000, chain.objs.size()=1 2024-04-22T20:48:16.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dXIzJzVBhg7-2NWrR1dLPG2DZ9PhAym_0 2024-04-22T20:48:16.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.356230690906170426', time=2024-04-22T22:29:14.204574+0000, chain.objs.size()=1 2024-04-22T20:48:16.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mgMnV8SdeczWAsVVBq4akKqLAVaBB2m_0 2024-04-22T20:48:16.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9268525377087460340', time=2024-04-22T22:29:14.400682+0000, chain.objs.size()=1 2024-04-22T20:48:16.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BqlGarcWqnQH4WmxkGt7OhnKsFmBheT_0 2024-04-22T20:48:16.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4958557623183821260', time=2024-04-22T22:29:14.663167+0000, chain.objs.size()=1 2024-04-22T20:48:16.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tZx_sifJr7bUudlgpL39m4r-b6v6xR0_0 2024-04-22T20:48:16.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.851979903486304792', time=2024-04-22T22:29:14.663767+0000, chain.objs.size()=1 2024-04-22T20:48:16.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rW788IgjMqMHieDJleUpxDfzAvy_gp3_0 2024-04-22T20:48:16.290 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4631838587884740466', time=2024-04-22T22:29:14.810964+0000, chain.objs.size()=1 2024-04-22T20:48:16.290 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7A91JLVdFczBFtQrctDhsYADEGlnWRc_0 2024-04-22T20:48:16.290 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5926492749947150692', time=2024-04-22T22:29:14.869246+0000, chain.objs.size()=1 2024-04-22T20:48:16.290 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RF41MGYzk5dFy0o4YFJ4p4vpQqqLDM__0 2024-04-22T20:48:16.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5817002327674216542', time=2024-04-22T22:29:15.790882+0000, chain.objs.size()=1 2024-04-22T20:48:16.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jnLMAEhFA1BA6cWLKZWxAu3XL-zGlra_0 2024-04-22T20:48:16.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1905559053435894184', time=2024-04-22T22:29:16.004235+0000, chain.objs.size()=1 2024-04-22T20:48:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7YMsIBIDRUD2PXKB9E8HiuXZdbiW8QQ_0 2024-04-22T20:48:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6429075403459187946', time=2024-04-22T22:29:16.398732+0000, chain.objs.size()=1 2024-04-22T20:48:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-715swStCwgljJzt57jrefBVCigb8v5_0 2024-04-22T20:48:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3611915120871629183', time=2024-04-22T22:30:34.374972+0000, chain.objs.size()=1 2024-04-22T20:48:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.426__shadow_.7yNz-C1tfGBaw7OltwGsT0tmUirUSKB_0 2024-04-22T20:48:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13062449300388698771', time=2024-04-22T22:31:57.412800+0000, chain.objs.size()=1 2024-04-22T20:48:16.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.294+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.501__shadow_.qML6dHE1L4Cc9t8Bp4LyCH0gY9xL9QI_0 2024-04-22T20:48:16.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.297+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1874066427445815248', time=2024-04-22T22:32:10.175192+0000, chain.objs.size()=1 2024-04-22T20:48:16.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.297+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.506__shadow_.1xrnZ2g5pP_ZZw2bmqWBvN_Oybg7PTo_0 2024-04-22T20:48:16.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.297+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17197576318462718404', time=2024-04-22T22:32:10.337533+0000, chain.objs.size()=1 2024-04-22T20:48:16.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.297+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.507__multipart_mymultipart.2~lAJA-HclsOs4n6Z49qs7_-OMumvRwL3.1 2024-04-22T20:48:16.299 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.303+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/45358 2024-04-22T20:48:16.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.308+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-04-22T20:48:16.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.308+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7083133060150580441', time=2024-04-22T22:32:56.249087+0000, chain.objs.size()=1 2024-04-22T20:48:16.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.308+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.509__shadow:qdCCLmM33gdFHcTVPwaxzxKphhTnwNo_.1il5VlZvE-sbfuhYZX39nwpR3pPtbPf_0 2024-04-22T20:48:16.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.308+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9428454161830601871', time=2024-04-22T22:35:23.546672+0000, chain.objs.size()=2 2024-04-22T20:48:16.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.308+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~OL9DYnddbwsN1dY__LKLyp0og2Y-eoq.1 2024-04-22T20:48:16.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.308+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~OL9DYnddbwsN1dY__LKLyp0og2Y-eoq.2 2024-04-22T20:48:16.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.308+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5204150929005029830', time=2024-04-22T22:35:35.459070+0000, chain.objs.size()=2 2024-04-22T20:48:16.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.308+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.517__multipart_mymultipart.2~FxmpNdteT50cwZFNxxOyiJj_bK6M645.1 2024-04-22T20:48:16.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.517__multipart_mymultipart.2~FxmpNdteT50cwZFNxxOyiJj_bK6M645.2 2024-04-22T20:48:16.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2039795817072906193', time=2024-04-22T22:37:19.542409+0000, chain.objs.size()=1 2024-04-22T20:48:16.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.554__shadow_.7YZ3fCm_Kh8SEPZJu7WjCsuQeGTXUPR_0 2024-04-22T20:48:16.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18346703226813278716', time=2024-04-22T22:38:10.921784+0000, chain.objs.size()=1 2024-04-22T20:48:16.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:7iHk1JkAug1b7cIk-SjxKEclIor.vBC_.0APe7byZkmIV8xlz5mX_UT-3vzhqjXc_0 2024-04-22T20:48:16.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3938655624732738618', time=2024-04-22T22:38:49.549147+0000, chain.objs.size()=1 2024-04-22T20:48:16.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:.HcEbMtKyLD.DT6AthEdihlsR5FMNs._.P3ObusTYAUcDLClAt6UPrZ-BrJP6A08_0 2024-04-22T20:48:16.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9829901743950211204', time=2024-04-22T22:42:14.605493+0000, chain.objs.size()=1 2024-04-22T20:48:16.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.632__shadow_.bB4JVoHc1tYPXVvfSuWDjXSCNGWthMW_0 2024-04-22T20:48:16.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12028356425668294254', time=2024-04-22T22:42:55.904434+0000, chain.objs.size()=1 2024-04-22T20:48:16.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.650__shadow_.IpZmspcWjK9KvmU6Q4Cr0LQd0LrIB6y_0 2024-04-22T20:48:16.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7960918649794491592', time=2024-04-22T22:43:12.402351+0000, chain.objs.size()=1 2024-04-22T20:48:16.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.679__shadow_.CtvLgmk1PgkiDk2RRpMS4c0mcN8Gtu__0 2024-04-22T20:48:16.310 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.313+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.316+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-22T20:48:16.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.318+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/40309' 2024-04-22T20:48:16.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.745624924217471793', time=2024-04-22T22:25:18.283809+0000, chain.objs.size()=1 2024-04-22T20:48:16.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.225__shadow_.-_J3b4TvkyEpeC0dM5zao76cbNcQ35g_0 2024-04-22T20:48:16.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3151779656371130533', time=2024-04-22T22:25:28.861128+0000, chain.objs.size()=1 2024-04-22T20:48:16.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.233__shadow_._z_e09Y5LkN2tt-_mLPuxyqY5y27PIg_0 2024-04-22T20:48:16.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4616801173379280125', time=2024-04-22T22:25:43.668779+0000, chain.objs.size()=1 2024-04-22T20:48:16.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.248__shadow_.qJ6_ru_vo8tmz4rkJrI7QfGjXTavLYf_0 2024-04-22T20:48:16.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15780842113653409743', time=2024-04-22T22:26:24.333152+0000, chain.objs.size()=1 2024-04-22T20:48:16.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.9wgGaBsTYq8IDbV7HI15V6QWViUuKGa_0 2024-04-22T20:48:16.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1681261221108125682', time=2024-04-22T22:26:24.733307+0000, chain.objs.size()=1 2024-04-22T20:48:16.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lm9Imz85NjSL3SIABU5RmTR1ykZxL3p_0 2024-04-22T20:48:16.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.425883029843106382', time=2024-04-22T22:26:24.763371+0000, chain.objs.size()=1 2024-04-22T20:48:16.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vXKzzNqCuVYI1qxYt6TQ5K7unZS59fr_0 2024-04-22T20:48:16.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4033009342816767972', time=2024-04-22T22:26:25.684652+0000, chain.objs.size()=1 2024-04-22T20:48:16.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xXE8iraS2UuzeKjsZTFHwI8MYxLaXgL_0 2024-04-22T20:48:16.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16989624286766472247', time=2024-04-22T22:26:25.685228+0000, chain.objs.size()=1 2024-04-22T20:48:16.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6hQBBEJpsigONp5E1ccI0DVF1oP6aBF_0 2024-04-22T20:48:16.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17227119997596563172', time=2024-04-22T22:26:26.126805+0000, chain.objs.size()=1 2024-04-22T20:48:16.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.TYVm-hwY7CM7aU6u0nLORGJqWcMOu5M_0 2024-04-22T20:48:16.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10149823085609946554', time=2024-04-22T22:26:27.751226+0000, chain.objs.size()=1 2024-04-22T20:48:16.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8M94SQWpDMi3vcoNCYCVOYtaSOjZIUv_0 2024-04-22T20:48:16.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12316487409727629489', time=2024-04-22T22:26:29.492839+0000, chain.objs.size()=1 2024-04-22T20:48:16.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8vY2s9FRGJyMx8jMV5mXOEWHTIEbPnt_0 2024-04-22T20:48:16.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2964589785758292359', time=2024-04-22T22:26:30.034828+0000, chain.objs.size()=1 2024-04-22T20:48:16.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bfbqB4oRtVYTOzD8k7hu6gEOaHVvpen_0 2024-04-22T20:48:16.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.321+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4137779737336243213', time=2024-04-22T22:26:30.562464+0000, chain.objs.size()=1 2024-04-22T20:48:16.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.321+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.W-wv-0LAnRzg-YVTgSsQm4b178ht9UF_0 2024-04-22T20:48:16.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17713045520893871700', time=2024-04-22T22:26:31.041319+0000, chain.objs.size()=1 2024-04-22T20:48:16.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-1bi1_6iTO0KO68qAeQNCCDBArjMtn__0 2024-04-22T20:48:16.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12688852799823275101', time=2024-04-22T22:26:31.493382+0000, chain.objs.size()=1 2024-04-22T20:48:16.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Sl8U4gETNbL-ngwvQvCxoJm62cZxtoO_0 2024-04-22T20:48:16.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10927656346178504923', time=2024-04-22T22:26:31.510598+0000, chain.objs.size()=1 2024-04-22T20:48:16.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wnzirKZhJLnCD-oZ_nKipb53QKKBui1_0 2024-04-22T20:48:16.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10366505986558445491', time=2024-04-22T22:26:32.060990+0000, chain.objs.size()=1 2024-04-22T20:48:16.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.TATPc_qWm9NWRD5j_P2Ei4jncNaH0WB_0 2024-04-22T20:48:16.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11095090163965233744', time=2024-04-22T22:26:32.747235+0000, chain.objs.size()=1 2024-04-22T20:48:16.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xmbb9ToOe1cKSRPHTtYxv2edMWteMcC_0 2024-04-22T20:48:16.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.323+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2331922262232807719', time=2024-04-22T22:26:32.968412+0000, chain.objs.size()=1 2024-04-22T20:48:16.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.323+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5XdaMx-i9JYZXiMaBqPxhXoewpXLO4q_0 2024-04-22T20:48:16.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8553323331574344650', time=2024-04-22T22:26:33.458646+0000, chain.objs.size()=1 2024-04-22T20:48:16.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.yEkP7gtuEycGNaQImnv5jcjPbw2w3x8_0 2024-04-22T20:48:16.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12396466273031011198', time=2024-04-22T22:26:33.512352+0000, chain.objs.size()=1 2024-04-22T20:48:16.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.277YFLEYDp0lC8c39zpiK-AOGdWSjBN_0 2024-04-22T20:48:16.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13318879718136574727', time=2024-04-22T22:26:33.609012+0000, chain.objs.size()=1 2024-04-22T20:48:16.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MtNNcU6caR6pAfoRm0obhA6Tq85qEkb_0 2024-04-22T20:48:16.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8523180237819024004', time=2024-04-22T22:26:33.838013+0000, chain.objs.size()=1 2024-04-22T20:48:16.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.yNH57GQYW-914WWMomy8ZkMvKpt7CUT_0 2024-04-22T20:48:16.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1667688406206696277', time=2024-04-22T22:26:34.059874+0000, chain.objs.size()=1 2024-04-22T20:48:16.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.324+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1QBsb5ECLmUS5foqVYsKemAJMVEEdob_0 2024-04-22T20:48:16.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.327+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2931116240925234991', time=2024-04-22T22:26:34.390003+0000, chain.objs.size()=1 2024-04-22T20:48:16.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.327+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WrW63t4ZqqZDl5EXhfG-SVMR5pzaUlU_0 2024-04-22T20:48:16.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.327+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3774265867363317633', time=2024-04-22T22:26:36.368990+0000, chain.objs.size()=1 2024-04-22T20:48:16.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.327+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.253__shadow_.6G0mzQ5r_rWCTSuZoTkkcq8ooGB9UZf_0 2024-04-22T20:48:16.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9514826151162954742', time=2024-04-22T22:27:05.642075+0000, chain.objs.size()=1 2024-04-22T20:48:16.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.d_9JneeFbfoj3Xb_zHJhFqJIQWh0qFh_0 2024-04-22T20:48:16.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11927303423153194987', time=2024-04-22T22:27:07.953418+0000, chain.objs.size()=1 2024-04-22T20:48:16.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.C3UbUok6f-5xEvzD4t4vS2KwFOYfuXH_0 2024-04-22T20:48:16.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10024056473932674054', time=2024-04-22T22:27:10.984375+0000, chain.objs.size()=1 2024-04-22T20:48:16.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.285__shadow_.kQsWt77Xuh10DMvPoVWZ_hqAhrcBEAn_0 2024-04-22T20:48:16.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3541330571315499586', time=2024-04-22T22:27:21.474784+0000, chain.objs.size()=1 2024-04-22T20:48:16.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.295__shadow_.dLBIdbObmeriOdCo2TMHLlCif64B-GU_0 2024-04-22T20:48:16.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16765198029616596633', time=2024-04-22T22:28:17.504827+0000, chain.objs.size()=1 2024-04-22T20:48:16.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.t3_mLeJ_U3HfrbDr1TDoe3XTzOTINV6_0 2024-04-22T20:48:16.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5346643423124033851', time=2024-04-22T22:28:17.896241+0000, chain.objs.size()=1 2024-04-22T20:48:16.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OeIoqspw27lAZfE7a6bIOFl3FEa096k_0 2024-04-22T20:48:16.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6236364300591348840', time=2024-04-22T22:28:18.076587+0000, chain.objs.size()=1 2024-04-22T20:48:16.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WZWHVauPU_1lCTFzF1l8sBTpyW5qvXe_0 2024-04-22T20:48:16.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6920798072017216105', time=2024-04-22T22:28:18.391871+0000, chain.objs.size()=1 2024-04-22T20:48:16.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.K0uIYPgjSkvbLjhRR-QnhqwFafqVciU_0 2024-04-22T20:48:16.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8726773754135603337', time=2024-04-22T22:28:19.408379+0000, chain.objs.size()=1 2024-04-22T20:48:16.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.329+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8X2qrwxaO4A9SDDANUV-qsvrg5ELRWA_0 2024-04-22T20:48:16.328 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.331+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11565518971802662038', time=2024-04-22T22:28:19.419961+0000, chain.objs.size()=1 2024-04-22T20:48:16.328 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.331+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.sz72rZur1mE7fbPPgB6ZJnhrJFR11Tg_0 2024-04-22T20:48:16.328 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.332+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6232931974239605445', time=2024-04-22T22:28:20.089405+0000, chain.objs.size()=1 2024-04-22T20:48:16.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.332+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.X_7anTnYGb8Kh_ST4gAL3RzANmNBhkY_0 2024-04-22T20:48:16.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.332+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14839270466962866149', time=2024-04-22T22:28:20.687660+0000, chain.objs.size()=1 2024-04-22T20:48:16.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.332+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.33N8NGpSfntjRgSExmWnBmatvfanHVk_0 2024-04-22T20:48:16.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.332+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15092160575593737317', time=2024-04-22T22:28:20.848496+0000, chain.objs.size()=1 2024-04-22T20:48:16.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.332+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qGwqpyX4jXRh7rX80u9_WTQjo36VZF6_0 2024-04-22T20:48:16.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3366135753463655434', time=2024-04-22T22:28:20.927981+0000, chain.objs.size()=1 2024-04-22T20:48:16.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4APIarVRwA97yyqPYPia4-kM-TljdNl_0 2024-04-22T20:48:16.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8499816940434273500', time=2024-04-22T22:28:20.987089+0000, chain.objs.size()=1 2024-04-22T20:48:16.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3xb7JZicM3Fl4OsoQwZvlCPUQDwlpqQ_0 2024-04-22T20:48:16.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15224343991639992563', time=2024-04-22T22:28:21.675191+0000, chain.objs.size()=1 2024-04-22T20:48:16.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.b2np-Z_EbzfWVoVN6NMU4CH5BPMIZOD_0 2024-04-22T20:48:16.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16058548947975705845', time=2024-04-22T22:28:21.875393+0000, chain.objs.size()=1 2024-04-22T20:48:16.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5q40KemRwqXEEkrMPaHtjpvVMlSQXOK_0 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15436390458437666069', time=2024-04-22T22:28:22.727922+0000, chain.objs.size()=1 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.2420_TYaYVJkLERsJQIK9MlRg-GEyIv_0 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8470953625951480445', time=2024-04-22T22:28:22.939260+0000, chain.objs.size()=1 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8FVHUEbusG18JoJnVBBdHNZMvpflckL_0 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4976670795213275717', time=2024-04-22T22:28:23.005578+0000, chain.objs.size()=1 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XOp25kymroHCc12TH7mDozSFAiCXTUd_0 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17721543401379460345', time=2024-04-22T22:28:23.461974+0000, chain.objs.size()=1 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BjbLAGhhmi4iNJY6ngH0dFnGJcwZbla_0 2024-04-22T20:48:16.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6446145888856127824', time=2024-04-22T22:28:24.671199+0000, chain.objs.size()=1 2024-04-22T20:48:16.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fxX6VS1hqCiaxqB-qWQKbIeBYPW7o8y_0 2024-04-22T20:48:16.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.337+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10851199132360708691', time=2024-04-22T22:28:24.676534+0000, chain.objs.size()=1 2024-04-22T20:48:16.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.337+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.aDLV-etkrBrmPLH_fFIQYB2g-ThXtMJ_0 2024-04-22T20:48:16.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.337+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6077573360373294771', time=2024-04-22T22:28:24.940299+0000, chain.objs.size()=1 2024-04-22T20:48:16.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.337+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LGInNUGSThNcNRTNQ7pRaZEG514Mw9l_0 2024-04-22T20:48:16.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.340+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5705262065814060229', time=2024-04-22T22:28:24.965493+0000, chain.objs.size()=1 2024-04-22T20:48:16.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.340+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FbbpQp5a8PIImQ--Tp9eNScMZti_01p_0 2024-04-22T20:48:16.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.340+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16602326617345060621', time=2024-04-22T22:28:25.403414+0000, chain.objs.size()=1 2024-04-22T20:48:16.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.340+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.sDmft4LmXF7YfUzJL66mY7w7Wkvlx_C_0 2024-04-22T20:48:16.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14794544633940529293', time=2024-04-22T22:28:25.941055+0000, chain.objs.size()=1 2024-04-22T20:48:16.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IepKD02kbVbXpvlTd8mKc5oRfKnKCYB_0 2024-04-22T20:48:16.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.549293870030794329', time=2024-04-22T22:28:26.317096+0000, chain.objs.size()=1 2024-04-22T20:48:16.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.49INTXd0LV45-BF3WM_uW36AcFAUt1W_0 2024-04-22T20:48:16.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4996030683598283377', time=2024-04-22T22:28:26.330660+0000, chain.objs.size()=1 2024-04-22T20:48:16.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.46zBf3eUyW6Z6R0J0dnchM17KuXj0jq_0 2024-04-22T20:48:16.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6429610822567164424', time=2024-04-22T22:28:26.969007+0000, chain.objs.size()=1 2024-04-22T20:48:16.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mxRDvIyAT6GKi2lC_LXsOkpCKA_kh0L_0 2024-04-22T20:48:16.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5949125787655324481', time=2024-04-22T22:28:27.118355+0000, chain.objs.size()=1 2024-04-22T20:48:16.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KCofjKMAaS3Y3o_t28RUrEUvNSIRCtZ_0 2024-04-22T20:48:16.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4573444930245915772', time=2024-04-22T22:28:27.134066+0000, chain.objs.size()=1 2024-04-22T20:48:16.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8dYgwkgR_-fuL1uiKTkOlqZ9BH6XEIU_0 2024-04-22T20:48:16.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17252674144878592382', time=2024-04-22T22:28:27.215152+0000, chain.objs.size()=1 2024-04-22T20:48:16.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MuwC0kkohg5YRFAMDIX5cDdh1GdG0Oj_0 2024-04-22T20:48:16.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.342+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10691755566754954310', time=2024-04-22T22:28:28.405739+0000, chain.objs.size()=1 2024-04-22T20:48:16.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.342+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wU-MpHExzyahTVX2E_G8TKACfYD__i0_0 2024-04-22T20:48:16.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.343+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5593479727977528287', time=2024-04-22T22:28:28.949857+0000, chain.objs.size()=1 2024-04-22T20:48:16.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.343+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CoPJlB1T1NBA3Jr4YmetsAcp8MCnRbL_0 2024-04-22T20:48:16.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18346199730299536083', time=2024-04-22T22:28:29.081677+0000, chain.objs.size()=1 2024-04-22T20:48:16.342 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KbA45L0tsstisVbDk28V7KNrXeFe5ys_0 2024-04-22T20:48:16.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17704271370917489158', time=2024-04-22T22:28:29.250952+0000, chain.objs.size()=1 2024-04-22T20:48:16.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.014_CuivkXEU5fwf4pFS8C8alEFNqxr_0 2024-04-22T20:48:16.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12018951400188799384', time=2024-04-22T22:28:29.254713+0000, chain.objs.size()=1 2024-04-22T20:48:16.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.EZ805QybgJOvWA72mc_yHkXDqiJ8Srf_0 2024-04-22T20:48:16.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2647899818095394123', time=2024-04-22T22:28:29.294885+0000, chain.objs.size()=1 2024-04-22T20:48:16.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IIQ4B1682cyrmcSOYuA8k--rVCKbnKj_0 2024-04-22T20:48:16.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12744218503298172652', time=2024-04-22T22:28:29.308537+0000, chain.objs.size()=1 2024-04-22T20:48:16.343 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.aSAoabR91zAokLsJQtUsiO7Y4A_Lrgb_0 2024-04-22T20:48:16.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8547060277500311245', time=2024-04-22T22:29:06.637349+0000, chain.objs.size()=1 2024-04-22T20:48:16.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PU3uQAHtt4vzd-ENd3LSi7dj_r1y94I_0 2024-04-22T20:48:16.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15192012373730383810', time=2024-04-22T22:29:06.669369+0000, chain.objs.size()=1 2024-04-22T20:48:16.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.J_LHCaAGDE82ctx1DaGVbKRjabGFyXX_0 2024-04-22T20:48:16.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4612276437169550195', time=2024-04-22T22:29:06.820346+0000, chain.objs.size()=1 2024-04-22T20:48:16.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Oa5wJ7XPOE7LKydbu1yrSOmHBn0Do_R_0 2024-04-22T20:48:16.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6807589080909640931', time=2024-04-22T22:29:06.953260+0000, chain.objs.size()=1 2024-04-22T20:48:16.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QjldD-2khXtfwoU9rBrywzOCnrGMzR6_0 2024-04-22T20:48:16.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6042502184725627670', time=2024-04-22T22:29:06.976516+0000, chain.objs.size()=1 2024-04-22T20:48:16.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Xt6wtH3Q1HMQzAggWYjNg695ySs4l61_0 2024-04-22T20:48:16.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12845985274833886853', time=2024-04-22T22:29:07.007577+0000, chain.objs.size()=1 2024-04-22T20:48:16.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.348+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1GoweitSNW2k6qUnad62Zleta4TQXtj_0 2024-04-22T20:48:16.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14704277675791325072', time=2024-04-22T22:29:07.041081+0000, chain.objs.size()=1 2024-04-22T20:48:16.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6nPaGKrdM4xih7ZCefVwa72x-r9yPJI_0 2024-04-22T20:48:16.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17791155311472851317', time=2024-04-22T22:29:08.116856+0000, chain.objs.size()=1 2024-04-22T20:48:16.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DpaorvvPXK3S7zQ4hmU9IMkZd_QC4Vd_0 2024-04-22T20:48:16.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6045832947190930077', time=2024-04-22T22:29:08.609537+0000, chain.objs.size()=1 2024-04-22T20:48:16.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jEKdNOBv4reK56Ux1R1CLxPmgzygCiX_0 2024-04-22T20:48:16.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4551315618834033087', time=2024-04-22T22:29:08.792826+0000, chain.objs.size()=1 2024-04-22T20:48:16.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.uTMNShEXpcx14s6W5TpA29AEy5xrTRT_0 2024-04-22T20:48:16.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.864114387009244357', time=2024-04-22T22:29:08.862596+0000, chain.objs.size()=1 2024-04-22T20:48:16.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.351+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zUCL6D6qsXDXYoswV-S7R0WX4o7cjiR_0 2024-04-22T20:48:16.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6273189049946520722', time=2024-04-22T22:29:08.884963+0000, chain.objs.size()=1 2024-04-22T20:48:16.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xQuXCOA264aMuaPR4Jm9q3nyGL_Mgqz_0 2024-04-22T20:48:16.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14176823026374250', time=2024-04-22T22:29:09.547215+0000, chain.objs.size()=1 2024-04-22T20:48:16.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1U84FZIIVCBFna60Poa7Cnl8yvU8m_m_0 2024-04-22T20:48:16.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12117157051180601968', time=2024-04-22T22:29:10.067780+0000, chain.objs.size()=1 2024-04-22T20:48:16.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2C8ILG8kDi4_T-d_dO6KpPQKPmwtZIV_0 2024-04-22T20:48:16.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9399733784977977302', time=2024-04-22T22:29:10.385464+0000, chain.objs.size()=1 2024-04-22T20:48:16.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.aKagiQTjOoCHXNDwcM76X3XXWs5Xtnr_0 2024-04-22T20:48:16.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.355+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16433275315102034082', time=2024-04-22T22:29:11.061942+0000, chain.objs.size()=1 2024-04-22T20:48:16.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.355+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Ts-nL0MvCYgtQYA-terTN38eHCruey4_0 2024-04-22T20:48:16.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.355+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8951960759025986907', time=2024-04-22T22:29:11.204275+0000, chain.objs.size()=1 2024-04-22T20:48:16.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.355+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zWfgJtHHBQSt6P6xSYRGf6-ovUOFxCk_0 2024-04-22T20:48:16.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.356+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13523393498527344468', time=2024-04-22T22:29:11.342968+0000, chain.objs.size()=1 2024-04-22T20:48:16.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.356+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6n0gfmyMPcZrmtbU1taTzthQGmFv92F_0 2024-04-22T20:48:16.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.356+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4160786738625818775', time=2024-04-22T22:29:11.343464+0000, chain.objs.size()=1 2024-04-22T20:48:16.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.356+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nY19GshIVsAfxF58NUx1cyJ5hNBMhl9_0 2024-04-22T20:48:16.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.357+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8863465630485655226', time=2024-04-22T22:29:13.195011+0000, chain.objs.size()=1 2024-04-22T20:48:16.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.357+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ImRyx7aSBZEWQOTQkfbizkgYiCEM0cW_0 2024-04-22T20:48:16.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.357+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5851658024261724474', time=2024-04-22T22:29:13.200034+0000, chain.objs.size()=1 2024-04-22T20:48:16.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.357+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Qdtchm0xv8J65gUCoMmgVk3TUQJqLYh_0 2024-04-22T20:48:16.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7666865050612952942', time=2024-04-22T22:29:13.918932+0000, chain.objs.size()=1 2024-04-22T20:48:16.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nKl8DJn6XEZvVDFxPFnXIKIspwCKvO3_0 2024-04-22T20:48:16.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3262519429982690523', time=2024-04-22T22:29:14.384641+0000, chain.objs.size()=1 2024-04-22T20:48:16.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BVJ3RcsGrr_iVlRB3jlgTzJzGzWbj5C_0 2024-04-22T20:48:16.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6224847182646188586', time=2024-04-22T22:29:14.906759+0000, chain.objs.size()=1 2024-04-22T20:48:16.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.46ERlZ23m_ybW0SY1vGmqA_pF56koyM_0 2024-04-22T20:48:16.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1559982249717757990', time=2024-04-22T22:29:15.630849+0000, chain.objs.size()=1 2024-04-22T20:48:16.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1S8bewh4mV_LXBYf1POvJEZdzKBTtI-_0 2024-04-22T20:48:16.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3525878976987637335', time=2024-04-22T22:29:15.651971+0000, chain.objs.size()=1 2024-04-22T20:48:16.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.359+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.42aa_B02L13hDftaoMheqJiMSplwv4j_0 2024-04-22T20:48:16.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.360+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17410139652111971727', time=2024-04-22T22:29:15.996516+0000, chain.objs.size()=1 2024-04-22T20:48:16.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.360+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qvOSRhR0DgWrxT-7iTrgd7bitw-UrT__0 2024-04-22T20:48:16.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17534588697098728842', time=2024-04-22T22:29:16.186385+0000, chain.objs.size()=1 2024-04-22T20:48:16.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.K9hve8DhZGCd8LngPDECB4SOMHy7huz_0 2024-04-22T20:48:16.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4136889978506004476', time=2024-04-22T22:29:17.100095+0000, chain.objs.size()=1 2024-04-22T20:48:16.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7RaG_Us0luPSAZKwChdWD62a1Mjt9eT_0 2024-04-22T20:48:16.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16039287745826515347', time=2024-04-22T22:29:17.628563+0000, chain.objs.size()=1 2024-04-22T20:48:16.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.djYMobXZIsbYcApcQre4aGt0VtMhO6C_0 2024-04-22T20:48:16.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.363+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5753112093233108616', time=2024-04-22T22:29:23.875557+0000, chain.objs.size()=1 2024-04-22T20:48:16.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.363+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.322__shadow_.EG9FXLvJkoaQ4SuGLoj7ItsiUfPO0_g_0 2024-04-22T20:48:16.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.363+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4719912759091257375', time=2024-04-22T22:29:51.915912+0000, chain.objs.size()=1 2024-04-22T20:48:16.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.363+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.364__shadow_.KWydIZHnvnCyLTfqk4n1eiLgOmwX57S_0 2024-04-22T20:48:16.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.363+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5855162820163738001', time=2024-04-22T22:29:54.118666+0000, chain.objs.size()=1 2024-04-22T20:48:16.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.363+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.367__shadow_.xPdu9y9fgwJTqxLTBO0chpkuiKBesQF_0 2024-04-22T20:48:16.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.363+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13250141252312481051', time=2024-04-22T22:30:49.314686+0000, chain.objs.size()=1 2024-04-22T20:48:16.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.363+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.444__shadow_.ehK6iINMIVcKpz9aam-cP6JV8_sKB-7_0 2024-04-22T20:48:16.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/40309 2024-04-22T20:48:16.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=18, truncated=0, next_marker='' 2024-04-22T20:48:16.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4943526947534782433', time=2024-04-22T22:30:59.501916+0000, chain.objs.size()=1 2024-04-22T20:48:16.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.453__shadow_.i7e27YoB7JqT45Itk3M7we7RIihXUoz_0 2024-04-22T20:48:16.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8839806658796017517', time=2024-04-22T22:31:10.479909+0000, chain.objs.size()=1 2024-04-22T20:48:16.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.cDphoUFgXAD3dsbDHcNuZJDDdwQkBkj_0 2024-04-22T20:48:16.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13897249256084973370', time=2024-04-22T22:31:10.480902+0000, chain.objs.size()=1 2024-04-22T20:48:16.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.Hk5pX3PWOe2YrYUy9wuVXcPhuu8fK6h_0 2024-04-22T20:48:16.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_1_d9591eebd65b0461a88bc96153b2b040', time=2024-04-22T22:31:27.902504+0000, chain.objs.size()=1 2024-04-22T20:48:16.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_0 2024-04-22T20:48:16.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18034218034972600228', time=2024-04-22T22:35:16.824841+0000, chain.objs.size()=3 2024-04-22T20:48:16.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~zEEHU_1YDCouex70y6pmTevduOiMymC.1 2024-04-22T20:48:16.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~zEEHU_1YDCouex70y6pmTevduOiMymC.2 2024-04-22T20:48:16.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~zEEHU_1YDCouex70y6pmTevduOiMymC.3 2024-04-22T20:48:16.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17810102040098189048', time=2024-04-22T22:35:20.642869+0000, chain.objs.size()=3 2024-04-22T20:48:16.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~RYW-5OjDQnsS7LVo9L9t2Cx9QFG7Rzk.1 2024-04-22T20:48:16.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~RYW-5OjDQnsS7LVo9L9t2Cx9QFG7Rzk.2 2024-04-22T20:48:16.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~RYW-5OjDQnsS7LVo9L9t2Cx9QFG7Rzk.3 2024-04-22T20:48:16.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~EASfRP7dFEOi3yQ3v-fA15tyaGXIgEe', time=2024-04-22T22:35:38.582205+0000, chain.objs.size()=2 2024-04-22T20:48:16.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.518__multipart_mymultipart.2~EASfRP7dFEOi3yQ3v-fA15tyaGXIgEe.1 2024-04-22T20:48:16.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.518__multipart_mymultipart.2~EASfRP7dFEOi3yQ3v-fA15tyaGXIgEe.2 2024-04-22T20:48:16.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.375+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15127341333894520102', time=2024-04-22T22:37:05.058475+0000, chain.objs.size()=1 2024-04-22T20:48:16.373 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.375+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.546__shadow_.NIJRpEVa6dgaYYpxb8CdT0FSVhIFJom_0 2024-04-22T20:48:16.373 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17924446071095260066', time=2024-04-22T22:37:26.755133+0000, chain.objs.size()=1 2024-04-22T20:48:16.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:080KoHCUg1vBkfTGJaww2BEpzq4Ghxa_.IyVn04YQUtEZuGOyQ89d57WBR0FwKQ-_0 2024-04-22T20:48:16.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15183092002595640145', time=2024-04-22T22:37:35.671439+0000, chain.objs.size()=1 2024-04-22T20:48:16.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.560__shadow:R8VbOGFTCeOmpjmlFCR5Q49ECAXurje_.6t0QA-z8ZDmS9CCV0iZNobepE5vDYKi_0 2024-04-22T20:48:16.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11531701560607051692', time=2024-04-22T22:38:05.076301+0000, chain.objs.size()=1 2024-04-22T20:48:16.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:m5pCBLMBCm6pm04gNOaYB.44qt0ukc3_.6n_02HGKcbfzDwqJpzAdRVx3UBb8lgr_0 2024-04-22T20:48:16.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9978009496484067115', time=2024-04-22T22:38:59.450495+0000, chain.objs.size()=1 2024-04-22T20:48:16.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:Tz4VLDjVQBj2ypi-oM2-ZAv-h0RHMl0_.6gUtnz8db9E4iVia2057jpo1qYhSBTA_0 2024-04-22T20:48:16.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1401082106954168418', time=2024-04-22T22:39:12.040593+0000, chain.objs.size()=1 2024-04-22T20:48:16.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.595__shadow_.3E7gPdzw-dsNWw-Bi8oVRFDc0PcjMXJ_0 2024-04-22T20:48:16.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10577172295617400474', time=2024-04-22T22:40:59.461308+0000, chain.objs.size()=1 2024-04-22T20:48:16.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.606__shadow_.IUeoRrwC0-2Ia8QB7ugONWHLXOL0hJl_0 2024-04-22T20:48:16.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8752248901318468036', time=2024-04-22T22:41:55.968791+0000, chain.objs.size()=1 2024-04-22T20:48:16.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.622__shadow_.8p2Yq5l_gKiUAlEW4o2ypSh2S40yjZ1_0 2024-04-22T20:48:16.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4948558702158997916', time=2024-04-22T22:42:53.616755+0000, chain.objs.size()=1 2024-04-22T20:48:16.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.648__shadow_.PiK_72Rdpl9LyoI8iAhyzmUnczuOpvr_0 2024-04-22T20:48:16.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17168011787099475006', time=2024-04-22T22:42:57.080035+0000, chain.objs.size()=1 2024-04-22T20:48:16.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.652__shadow_.zYUSxkPTXH_lk-Klenrla3snW0mBbX1_0 2024-04-22T20:48:16.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.307307551068172550', time=2024-04-22T22:42:58.996806+0000, chain.objs.size()=1 2024-04-22T20:48:16.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.380+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.655__shadow_.WzUsLnYImUSM_j95WwrLzf0C38KGLXC_0 2024-04-22T20:48:16.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.385+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.388+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-22T20:48:16.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.389+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.390+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/39872' 2024-04-22T20:48:16.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12127129910139910898', time=2024-04-22T22:25:30.844949+0000, chain.objs.size()=1 2024-04-22T20:48:16.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.235__shadow_.kN1uBEW0Edv2jvr3Ix7yY_q_1n7NCxd_0 2024-04-22T20:48:16.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.249767834099564619', time=2024-04-22T22:25:35.833605+0000, chain.objs.size()=1 2024-04-22T20:48:16.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.240__shadow_.PiWhw_AqGHston437-kcS0MAlo-9jW__0 2024-04-22T20:48:16.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8964318595481041190', time=2024-04-22T22:25:42.735434+0000, chain.objs.size()=1 2024-04-22T20:48:16.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.247__shadow_.oVebMPFpw0-0GijJVtaKFtFdrFokaTX_0 2024-04-22T20:48:16.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12109352551346310669', time=2024-04-22T22:26:22.473391+0000, chain.objs.size()=1 2024-04-22T20:48:16.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.o7S-HGgg5Mkn7T6I982HcVLxrDZZqiH_0 2024-04-22T20:48:16.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10472954480882021634', time=2024-04-22T22:26:22.505527+0000, chain.objs.size()=1 2024-04-22T20:48:16.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._7CLPA3Vmq6CiMlDb-DuZXX-ahYiMYx_0 2024-04-22T20:48:16.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9933592944329487672', time=2024-04-22T22:26:22.605350+0000, chain.objs.size()=1 2024-04-22T20:48:16.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jiKuMzYgVkLMN01rhrz7h-Ofu8WZ_R0_0 2024-04-22T20:48:16.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9772768110727280567', time=2024-04-22T22:26:22.849640+0000, chain.objs.size()=1 2024-04-22T20:48:16.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WwsWCgUX6B910bvArwKEZd611iFR04s_0 2024-04-22T20:48:16.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17436636494110406124', time=2024-04-22T22:26:22.929963+0000, chain.objs.size()=1 2024-04-22T20:48:16.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wLKdpd78qk0iqtQEA2p_fzwehuL34tk_0 2024-04-22T20:48:16.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8067836626372118495', time=2024-04-22T22:26:23.943294+0000, chain.objs.size()=1 2024-04-22T20:48:16.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cIF4SAmKdGjFFOpKsaE0It8BiXat21X_0 2024-04-22T20:48:16.391 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3049610022340418538', time=2024-04-22T22:26:23.991192+0000, chain.objs.size()=1 2024-04-22T20:48:16.391 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.fiwcZ1XrK4NvYQ-WUeijKQIIw8xP2qW_0 2024-04-22T20:48:16.391 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10695411458641357270', time=2024-04-22T22:26:24.402708+0000, chain.objs.size()=1 2024-04-22T20:48:16.391 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YD2ycdXZPFf7aID8HRCNglRl69o_Sg7_0 2024-04-22T20:48:16.391 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11315775752353306511', time=2024-04-22T22:26:24.964544+0000, chain.objs.size()=1 2024-04-22T20:48:16.391 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Q7reCzk-iG5gjYcaJEi3yZre_N-n3Zp_0 2024-04-22T20:48:16.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3442786701887335414', time=2024-04-22T22:26:26.120738+0000, chain.objs.size()=1 2024-04-22T20:48:16.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LhYXMn4vvdptW_TRXhY3jI4ox8z4BoV_0 2024-04-22T20:48:16.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13888909322110219228', time=2024-04-22T22:26:26.543785+0000, chain.objs.size()=1 2024-04-22T20:48:16.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ddMCMwIJMiG0QWW-1bY8AXbwGNWy9FI_0 2024-04-22T20:48:16.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14568608825707140547', time=2024-04-22T22:26:26.927847+0000, chain.objs.size()=1 2024-04-22T20:48:16.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Z_uOTREzKac5tVU2TwsQeu1rH75Usug_0 2024-04-22T20:48:16.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9842949430565834907', time=2024-04-22T22:26:27.439344+0000, chain.objs.size()=1 2024-04-22T20:48:16.393 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YKz4fF-xvZva9QG5Tc6aCSgyaCliT7d_0 2024-04-22T20:48:16.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1643721467919578391', time=2024-04-22T22:26:28.439171+0000, chain.objs.size()=1 2024-04-22T20:48:16.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.394+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qrDKq5sauaj21MiTLWHkhg5aVBhypg1_0 2024-04-22T20:48:16.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14087815592267771503', time=2024-04-22T22:26:28.608358+0000, chain.objs.size()=1 2024-04-22T20:48:16.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7a-1wBZCyGghJsYowtiF6nMY-kA28e__0 2024-04-22T20:48:16.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18444653679588732239', time=2024-04-22T22:26:29.007594+0000, chain.objs.size()=1 2024-04-22T20:48:16.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.c2CEJ4oAZAhmQxjV04JFzvui8uLQzkT_0 2024-04-22T20:48:16.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10064275601159067562', time=2024-04-22T22:26:29.055970+0000, chain.objs.size()=1 2024-04-22T20:48:16.394 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.9p8iOqp8L9g2Ny6GntPmpwcDlrycAjT_0 2024-04-22T20:48:16.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1722974748802353213', time=2024-04-22T22:26:29.230276+0000, chain.objs.size()=1 2024-04-22T20:48:16.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.whe7heejCvJcpXuuD_a6tMDzpU7DGrP_0 2024-04-22T20:48:16.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15851262731614942326', time=2024-04-22T22:26:29.345975+0000, chain.objs.size()=1 2024-04-22T20:48:16.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ApZ9OgYggWi0qc6rEZNHbGRidlKpKHD_0 2024-04-22T20:48:16.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4615554869343813898', time=2024-04-22T22:26:29.602134+0000, chain.objs.size()=1 2024-04-22T20:48:16.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.395+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GDKFC7pdyBGRWlFQ1UJK7Tra5VMnAtU_0 2024-04-22T20:48:16.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.399+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7247429089862713435', time=2024-04-22T22:26:29.750450+0000, chain.objs.size()=1 2024-04-22T20:48:16.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.399+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1fnzXMBC4KKKwfxRTxHS301mgs-95B6_0 2024-04-22T20:48:16.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.399+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10195468748911365506', time=2024-04-22T22:26:29.935277+0000, chain.objs.size()=1 2024-04-22T20:48:16.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.399+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JacqicXJJxX5gZhgtvng3Zns5vzHGbz_0 2024-04-22T20:48:16.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.400+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12483914898830523319', time=2024-04-22T22:26:30.566147+0000, chain.objs.size()=1 2024-04-22T20:48:16.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.400+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Ab6ao3vGJqX0NsE1uxSDOZ7Q7rEyzjK_0 2024-04-22T20:48:16.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12968475062814734947', time=2024-04-22T22:26:30.662854+0000, chain.objs.size()=1 2024-04-22T20:48:16.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.891jk4fMbWoqnXF-gpDBiUh6A8bKBU8_0 2024-04-22T20:48:16.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11036373704795816356', time=2024-04-22T22:26:30.687511+0000, chain.objs.size()=1 2024-04-22T20:48:16.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0uRHhh8Af-b-naM5l3rm5ZEHaXhdIiE_0 2024-04-22T20:48:16.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9888930209165460862', time=2024-04-22T22:26:31.042924+0000, chain.objs.size()=1 2024-04-22T20:48:16.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3rPV5wwGdn70DZpr7J1H2cwILvWea3G_0 2024-04-22T20:48:16.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15108283515070748681', time=2024-04-22T22:26:31.098407+0000, chain.objs.size()=1 2024-04-22T20:48:16.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Pxm_HzAAhSbjOEFvA7FI8OpgG1wLGFz_0 2024-04-22T20:48:16.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.495465903582872341', time=2024-04-22T22:26:31.188736+0000, chain.objs.size()=1 2024-04-22T20:48:16.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4aJqJdtyXI5sY-IDf-fBCvvErjTeX_b_0 2024-04-22T20:48:16.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1160123598888406481', time=2024-04-22T22:26:31.521757+0000, chain.objs.size()=1 2024-04-22T20:48:16.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vWiKMEZNApEdsknmkwEsTSed181pqG1_0 2024-04-22T20:48:16.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7191434604697645231', time=2024-04-22T22:26:32.201280+0000, chain.objs.size()=1 2024-04-22T20:48:16.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zsY5KT_Z9xXLRaB1TIK8tzeZjuRDWzl_0 2024-04-22T20:48:16.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11782623334747599008', time=2024-04-22T22:26:32.353016+0000, chain.objs.size()=1 2024-04-22T20:48:16.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.afxFo6k0zBTv1mSYGr0PDr9MLhO_eVG_0 2024-04-22T20:48:16.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.402+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1121146375469051003', time=2024-04-22T22:26:32.748559+0000, chain.objs.size()=1 2024-04-22T20:48:16.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.402+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dTw7lrA2p9i-BY3c0SvL59W0asoRGT5_0 2024-04-22T20:48:16.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.403+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6844692758361184330', time=2024-04-22T22:26:33.119607+0000, chain.objs.size()=1 2024-04-22T20:48:16.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.403+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.k_pVQgH1ae0rkmPsNJFrpKbmR98sXwe_0 2024-04-22T20:48:16.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.403+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5810121538652342716', time=2024-04-22T22:26:33.623339+0000, chain.objs.size()=1 2024-04-22T20:48:16.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.403+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wy4KWCoffuGLLkr7Up-RNVE0axyBRjt_0 2024-04-22T20:48:16.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.404+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12189373778391128709', time=2024-04-22T22:26:33.919872+0000, chain.objs.size()=1 2024-04-22T20:48:16.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.404+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.f1yH41yM-c5FjM2rhhef9emR7jqkRVl_0 2024-04-22T20:48:16.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.404+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9333593768932159444', time=2024-04-22T22:26:33.920532+0000, chain.objs.size()=1 2024-04-22T20:48:16.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.404+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jLLzIgQwydsGYJzkqg9dtnbVY3ryNfK_0 2024-04-22T20:48:16.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2098290856060616563', time=2024-04-22T22:26:34.205622+0000, chain.objs.size()=1 2024-04-22T20:48:16.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.69SA363NsAXHcj_piJyTfL0FrxSDCux_0 2024-04-22T20:48:16.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7715354816269499994', time=2024-04-22T22:26:34.437309+0000, chain.objs.size()=1 2024-04-22T20:48:16.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tDtd2bzKUIYc7Rr8haEIF7f47j6dzhH_0 2024-04-22T20:48:16.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2177442261669384712', time=2024-04-22T22:26:49.855037+0000, chain.objs.size()=1 2024-04-22T20:48:16.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.267__shadow_.tlqkJXDfd2AVZBrPCNYXuMFAvVWvUQi_0 2024-04-22T20:48:16.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3303414562434232960', time=2024-04-22T22:26:51.886139+0000, chain.objs.size()=1 2024-04-22T20:48:16.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.269__shadow_.nG3jpBH_16Id_I3RWQaUEe6XczUkqZG_0 2024-04-22T20:48:16.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10515432005484942510', time=2024-04-22T22:26:57.989000+0000, chain.objs.size()=1 2024-04-22T20:48:16.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.275__shadow_.kdv28psCdmqE4xX9NF3pfiQOUlguaKc_0 2024-04-22T20:48:16.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7419193862008295278', time=2024-04-22T22:26:59.975767+0000, chain.objs.size()=1 2024-04-22T20:48:16.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.277__shadow_.4Y3KyVJrSItKjSsJFt-plaEbtjynZWs_0 2024-04-22T20:48:16.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15478781999269966084', time=2024-04-22T22:27:07.853775+0000, chain.objs.size()=1 2024-04-22T20:48:16.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.405+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.chFOSLToWiNiEse4Vz-y3bltP1kFn62_0 2024-04-22T20:48:16.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.407+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11204244068622165082', time=2024-04-22T22:27:23.052953+0000, chain.objs.size()=1 2024-04-22T20:48:16.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.407+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.296__shadow:vaD6gxtEB4X0uAY2aTOwjv3RnmQTkAD_.bOmFgbXlyPMjNYi2v7wa3OTUDNKGGcH_0 2024-04-22T20:48:16.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.408+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6484377644000681635', time=2024-04-22T22:27:37.314189+0000, chain.objs.size()=1 2024-04-22T20:48:16.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.408+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.308__shadow:TC4DLR3p1mMkiLpAV6d.MrheTypX9Ca_.-W7HuvvFS3BOjTkYvI6WEEJd0ySGMXC_0 2024-04-22T20:48:16.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.408+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3330555811336150844', time=2024-04-22T22:28:19.061072+0000, chain.objs.size()=1 2024-04-22T20:48:16.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.408+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UvSM2SBXNwL6KgiU-E1B7m1O9D8BjB0_0 2024-04-22T20:48:16.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11419556331701436839', time=2024-04-22T22:28:19.363140+0000, chain.objs.size()=1 2024-04-22T20:48:16.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1dqlsCtn6669C3Z3cX1CndZvL-dtaom_0 2024-04-22T20:48:16.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12721322864939605117', time=2024-04-22T22:28:19.677394+0000, chain.objs.size()=1 2024-04-22T20:48:16.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JrrGFoqigsOR4vvy0eY04ko9eaUFvmu_0 2024-04-22T20:48:16.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6807112604246614750', time=2024-04-22T22:28:19.869182+0000, chain.objs.size()=1 2024-04-22T20:48:16.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xK0FXSBQ63MsJDdHRbBPcpC0vrzgWC0_0 2024-04-22T20:48:16.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.124454409442905515', time=2024-04-22T22:28:22.327447+0000, chain.objs.size()=1 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.j2R2QGklNt9EVQSZvEb-5WSwMCkzWBE_0 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13323108210953814213', time=2024-04-22T22:28:22.328194+0000, chain.objs.size()=1 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.L6aQeTT14Mie1wVFq1aTFqr0C1wLEc6_0 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8005400823120413356', time=2024-04-22T22:28:22.638557+0000, chain.objs.size()=1 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.biiYx9c7vnHOswKJjFy13mQ5Gl4Wwbz_0 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7052885821213274301', time=2024-04-22T22:28:22.868028+0000, chain.objs.size()=1 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yixfxxoxD58r8tbZkQHivfPYn_SDbJ2_0 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8470917925903949773', time=2024-04-22T22:28:22.891574+0000, chain.objs.size()=1 2024-04-22T20:48:16.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.409+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Qs_4vqwl5nqc4eJoCNm2LCWZJCgEEFd_0 2024-04-22T20:48:16.409 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.411+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4958195449379889651', time=2024-04-22T22:28:23.076757+0000, chain.objs.size()=1 2024-04-22T20:48:16.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.411+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NK_aMJ03EmzlSNPUT-fE5HZ89vKs_6i_0 2024-04-22T20:48:16.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.412+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12262488356213601091', time=2024-04-22T22:28:23.077390+0000, chain.objs.size()=1 2024-04-22T20:48:16.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.412+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vYp_9ViL6IQcMCdXg5VUjVzIowvaULr_0 2024-04-22T20:48:16.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.412+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7343866804663981574', time=2024-04-22T22:28:23.275866+0000, chain.objs.size()=1 2024-04-22T20:48:16.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.412+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tjLmMCWc0zI3VhuOohQ6EagyrBFnR5-_0 2024-04-22T20:48:16.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.413+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12786217077317545491', time=2024-04-22T22:28:23.276447+0000, chain.objs.size()=1 2024-04-22T20:48:16.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.413+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JDUnM4MSMSpZRsHRVCja9n9gCYvGq0L_0 2024-04-22T20:48:16.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.413+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8804905212994153724', time=2024-04-22T22:28:23.481420+0000, chain.objs.size()=1 2024-04-22T20:48:16.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.413+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.hvJwrwWDZt7gcfVHnRZQayxSOG33MM5_0 2024-04-22T20:48:16.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.413+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16222741599163531872', time=2024-04-22T22:28:23.482343+0000, chain.objs.size()=1 2024-04-22T20:48:16.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.413+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XZn-xeS_szwhaisPc1K7SJYc4e1RQ6q_0 2024-04-22T20:48:16.411 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3682702905575646600', time=2024-04-22T22:28:23.890618+0000, chain.objs.size()=1 2024-04-22T20:48:16.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8v1hH-dWlk6HzUfNlJtM9g_Zz2q__W0_0 2024-04-22T20:48:16.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8662375124732219485', time=2024-04-22T22:28:24.391700+0000, chain.objs.size()=1 2024-04-22T20:48:16.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.08L2OaGRtkfCpf4BS3E5rKH7hNP14EM_0 2024-04-22T20:48:16.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3559888864353865971', time=2024-04-22T22:28:24.566966+0000, chain.objs.size()=1 2024-04-22T20:48:16.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zNi_bW_cv9vhNAM3SJQwb_--6IozXNo_0 2024-04-22T20:48:16.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1134956360320526534', time=2024-04-22T22:28:24.695453+0000, chain.objs.size()=1 2024-04-22T20:48:16.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.e27d-aI_kJmSsKGQE7-oUr2vyItPS64_0 2024-04-22T20:48:16.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10146660762342675202', time=2024-04-22T22:28:25.606116+0000, chain.objs.size()=1 2024-04-22T20:48:16.413 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.414+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qbes3p3cBBjeL0Aw7T4XaZ75kJGmzNE_0 2024-04-22T20:48:16.413 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.415+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8386065615267263188', time=2024-04-22T22:28:26.174259+0000, chain.objs.size()=1 2024-04-22T20:48:16.413 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.415+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.DbXJMqlCLP9D9syCaI8RIv_Em8J7dXB_0 2024-04-22T20:48:16.413 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.416+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12944350109296768062', time=2024-04-22T22:28:26.933781+0000, chain.objs.size()=1 2024-04-22T20:48:16.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.416+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tAIQThoxjq2t7HDRbhVc2cHhWwBFAYx_0 2024-04-22T20:48:16.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.416+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15489962307539061918', time=2024-04-22T22:28:27.073402+0000, chain.objs.size()=1 2024-04-22T20:48:16.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.416+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.oqmGGUtWpKoA6eoRvAws22bKpj0bJbr_0 2024-04-22T20:48:16.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.419+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7652947166748716879', time=2024-04-22T22:28:27.142451+0000, chain.objs.size()=1 2024-04-22T20:48:16.415 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.On4lLn9kFFty7AcwUlNspbnS8zr5KxI_0 2024-04-22T20:48:16.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.419+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16639672816317762216', time=2024-04-22T22:28:27.390689+0000, chain.objs.size()=1 2024-04-22T20:48:16.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kZmB8Z4NdaAwLtCJczIydhOfofkjPhp_0 2024-04-22T20:48:16.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13754310612304306821', time=2024-04-22T22:28:27.400649+0000, chain.objs.size()=1 2024-04-22T20:48:16.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6Fe55MIGP4zAiL20BGd23CuFG5s7ckq_0 2024-04-22T20:48:16.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16564914711300981779', time=2024-04-22T22:28:27.476240+0000, chain.objs.size()=1 2024-04-22T20:48:16.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tQ-Jal-FES_RG1ch0fjUlpBLa0ZT3A5_0 2024-04-22T20:48:16.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3890104320303210398', time=2024-04-22T22:28:27.488283+0000, chain.objs.size()=1 2024-04-22T20:48:16.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gG65Z4Nm7a8S43Kzq-0GRW2WJjOODyx_0 2024-04-22T20:48:16.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16896639338226259157', time=2024-04-22T22:28:27.783729+0000, chain.objs.size()=1 2024-04-22T20:48:16.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6iuQx3gw-nUyzv28n1BuQykQReKRYDW_0 2024-04-22T20:48:16.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11964106871451373562', time=2024-04-22T22:28:27.955326+0000, chain.objs.size()=1 2024-04-22T20:48:16.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PCPRVNGOSQ3SlhWcThFo5giN78cjnBC_0 2024-04-22T20:48:16.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15427111626527307644', time=2024-04-22T22:28:27.978551+0000, chain.objs.size()=1 2024-04-22T20:48:16.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kOJwH56Pt1UZx2fynLsXBJ2rHP7kAgt_0 2024-04-22T20:48:16.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11433452042623495677', time=2024-04-22T22:28:28.797647+0000, chain.objs.size()=1 2024-04-22T20:48:16.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.420+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.o2VMxsy_1DQOlG5zJ4rTUvpah7CLlGE_0 2024-04-22T20:48:16.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.421+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4018382925538021389', time=2024-04-22T22:28:29.144154+0000, chain.objs.size()=1 2024-04-22T20:48:16.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.421+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wOZwGCV2_YKUB6a4LqlpruSegSuxKMz_0 2024-04-22T20:48:16.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.421+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15209281623179268362', time=2024-04-22T22:28:29.202803+0000, chain.objs.size()=1 2024-04-22T20:48:16.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.421+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.oMJdQHIKucoPV0QGlwBxPyoIf0b0LAU_0 2024-04-22T20:48:16.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.422+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6206045549888957800', time=2024-04-22T22:29:06.490838+0000, chain.objs.size()=1 2024-04-22T20:48:16.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.422+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.96n7KhE1QLC4OviFpBzCji_m4IxW0CW_0 2024-04-22T20:48:16.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.422+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1522799142666959480', time=2024-04-22T22:29:06.680000+0000, chain.objs.size()=1 2024-04-22T20:48:16.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.422+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qb5nA8kxIJrdoJIbGneBtQeRAZB379U_0 2024-04-22T20:48:16.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.422+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10835182238603583726', time=2024-04-22T22:29:06.822348+0000, chain.objs.size()=1 2024-04-22T20:48:16.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.422+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.O9xoS0XUi75LSy_FXiM1_ivm3NYRZFs_0 2024-04-22T20:48:16.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.919854165735567237', time=2024-04-22T22:29:06.908866+0000, chain.objs.size()=1 2024-04-22T20:48:16.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tkgNoq2oVgKL56hN4PyE6OyCLD2DJ-P_0 2024-04-22T20:48:16.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1250364132044536537', time=2024-04-22T22:29:07.042420+0000, chain.objs.size()=1 2024-04-22T20:48:16.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8AvVctUUFl28XLx84VzT6h9mYoGkMLo_0 2024-04-22T20:48:16.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2179507938060258954', time=2024-04-22T22:29:07.169576+0000, chain.objs.size()=1 2024-04-22T20:48:16.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.x4NZ8-OZ1oDLbx9NkB5q7s1NJPo5y9L_0 2024-04-22T20:48:16.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4285464193908291035', time=2024-04-22T22:29:07.177803+0000, chain.objs.size()=1 2024-04-22T20:48:16.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.cuKm_xPjR7pgF260XZeb53taNFczRtj_0 2024-04-22T20:48:16.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9636997043916517342', time=2024-04-22T22:29:07.959237+0000, chain.objs.size()=1 2024-04-22T20:48:16.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.424+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tYftALv8KUwDV8tExizWNIyCp5VB6qd_0 2024-04-22T20:48:16.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.425+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3438779502549293025', time=2024-04-22T22:29:08.056687+0000, chain.objs.size()=1 2024-04-22T20:48:16.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.425+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8uA9rrl33-EH4rVx6No8cczcRoaJq0m_0 2024-04-22T20:48:16.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.425+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10390990639959805323', time=2024-04-22T22:29:08.794192+0000, chain.objs.size()=1 2024-04-22T20:48:16.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.425+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.9ajnB69GC-88eKpkshkyyqgPYvyngX-_0 2024-04-22T20:48:16.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.425+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1176980206125230277', time=2024-04-22T22:29:09.368060+0000, chain.objs.size()=1 2024-04-22T20:48:16.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.425+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.5vKBgvdDaa-tdAHUTwXT95gLedr00X8_0 2024-04-22T20:48:16.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.425+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8898859541091417785', time=2024-04-22T22:29:09.412116+0000, chain.objs.size()=1 2024-04-22T20:48:16.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.425+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.5c59ISexxqoeBBP76TNijFOaE8xg902_0 2024-04-22T20:48:16.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.426+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13763847440624121372', time=2024-04-22T22:29:09.751834+0000, chain.objs.size()=1 2024-04-22T20:48:16.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.426+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.l9M7M6ocNFO6zaTb_b5Y3bQtM2rBTbS_0 2024-04-22T20:48:16.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.427+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5005825484705032265', time=2024-04-22T22:29:10.183686+0000, chain.objs.size()=1 2024-04-22T20:48:16.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.427+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.B0JsyzeyL0iNEq-iOktvc8r59CdBLQE_0 2024-04-22T20:48:16.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.428+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11641722789156989276', time=2024-04-22T22:29:10.295861+0000, chain.objs.size()=1 2024-04-22T20:48:16.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.428+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iAvgx8BuRviLDkkWsbUWQ2CGrNa3xEW_0 2024-04-22T20:48:16.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.428+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3410330170318860204', time=2024-04-22T22:29:10.966223+0000, chain.objs.size()=1 2024-04-22T20:48:16.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.428+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.paWeccaC5Pa33O7KxUxKXO-x-pI9POr_0 2024-04-22T20:48:16.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.428+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11375850651589361233', time=2024-04-22T22:29:11.114555+0000, chain.objs.size()=1 2024-04-22T20:48:16.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.428+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.aiW_knhOMZyCgH_8P4okNUOmTLabuwI_0 2024-04-22T20:48:16.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.428+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2151449979197400377', time=2024-04-22T22:29:11.494456+0000, chain.objs.size()=1 2024-04-22T20:48:16.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.428+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TmYJ9o6r-d66CzTieWIIdldnPT-9ZKX_0 2024-04-22T20:48:16.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.431+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/39872 2024-04-22T20:48:16.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=39, truncated=0, next_marker='' 2024-04-22T20:48:16.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18315159827469989501', time=2024-04-22T22:29:11.688169+0000, chain.objs.size()=1 2024-04-22T20:48:16.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sU5zLVQaH_9bUHzQhYtHQaDQc_vuoci_0 2024-04-22T20:48:16.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10282555759257097410', time=2024-04-22T22:29:11.702229+0000, chain.objs.size()=1 2024-04-22T20:48:16.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7-YSt-Kr_Yzp6lnsrqj7WHFTsaj3o_M_0 2024-04-22T20:48:16.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7055744741321528101', time=2024-04-22T22:29:11.758466+0000, chain.objs.size()=1 2024-04-22T20:48:16.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qorr8sBfsKBu2yobnG1V45PuMHvUWgm_0 2024-04-22T20:48:16.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14413882010769136991', time=2024-04-22T22:29:12.472387+0000, chain.objs.size()=1 2024-04-22T20:48:16.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.X_Mnc8-FqHjMfP6izngHaYW6i9UDIOU_0 2024-04-22T20:48:16.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2011599099615794280', time=2024-04-22T22:29:12.612629+0000, chain.objs.size()=1 2024-04-22T20:48:16.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wIKODcY6B-P6CtvqfEL1qGFSWHbj5VJ_0 2024-04-22T20:48:16.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17549384601555727544', time=2024-04-22T22:29:12.718488+0000, chain.objs.size()=1 2024-04-22T20:48:16.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2OXkr_GjzJ2Slmr5wsh_LrZH6dn4PjY_0 2024-04-22T20:48:16.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11266014885473562441', time=2024-04-22T22:29:12.792621+0000, chain.objs.size()=1 2024-04-22T20:48:16.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tBDDXW3oissFWNjh0fD5joNhIUyr7kd_0 2024-04-22T20:48:16.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12529040083184634120', time=2024-04-22T22:29:12.876116+0000, chain.objs.size()=1 2024-04-22T20:48:16.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.EMhTx_ATl0f93GsNi7Z9vltRnoFAZHm_0 2024-04-22T20:48:16.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4786527428572233623', time=2024-04-22T22:29:12.876806+0000, chain.objs.size()=1 2024-04-22T20:48:16.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YZhWrI2jBOl8Sfudqm6Eom9JdcNsbJ__0 2024-04-22T20:48:16.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14548641789551508950', time=2024-04-22T22:29:12.914266+0000, chain.objs.size()=1 2024-04-22T20:48:16.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3f2kQoFHCqhZWalWrysaYYOiXx-YqXi_0 2024-04-22T20:48:16.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3753341831990746610', time=2024-04-22T22:29:14.070507+0000, chain.objs.size()=1 2024-04-22T20:48:16.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Mty5drHSZBs3szw3emHo23FkKpW3D-b_0 2024-04-22T20:48:16.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16174800902127857372', time=2024-04-22T22:29:14.071272+0000, chain.objs.size()=1 2024-04-22T20:48:16.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6RYISbt78BnRJQ9EN0WPoxhfK0BSSwZ_0 2024-04-22T20:48:16.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.436+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12352564575520070305', time=2024-04-22T22:29:14.294865+0000, chain.objs.size()=1 2024-04-22T20:48:16.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.436+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2GYfcMVv9U1nB-P8V8oWPJyAU0Y0zAq_0 2024-04-22T20:48:16.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.437+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2444834284092722529', time=2024-04-22T22:29:14.592560+0000, chain.objs.size()=1 2024-04-22T20:48:16.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.437+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.CfPsasy9BqbelPWZl0f0d6d8zKT6G1g_0 2024-04-22T20:48:16.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.437+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12575232392726397834', time=2024-04-22T22:29:15.479087+0000, chain.objs.size()=1 2024-04-22T20:48:16.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.437+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3E2bkDTOWMBFUl46KQFai0GhgVZ12s__0 2024-04-22T20:48:16.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.437+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14871429812386118436', time=2024-04-22T22:29:16.156870+0000, chain.objs.size()=1 2024-04-22T20:48:16.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.437+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DOu7Y78rPzI-Fu9HbRBoHC4cQaIabs7_0 2024-04-22T20:48:16.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.437+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12004671104148098235', time=2024-04-22T22:29:16.190901+0000, chain.objs.size()=1 2024-04-22T20:48:16.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.437+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3DgQhqmJiMvTWV_buRBA5uaiydte9yU_0 2024-04-22T20:48:16.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3535245141298934717', time=2024-04-22T22:29:16.320172+0000, chain.objs.size()=1 2024-04-22T20:48:16.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RIaawmATa9B2c02laFuP8LRhTnr5Ezd_0 2024-04-22T20:48:16.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1819740386922794452', time=2024-04-22T22:29:17.636217+0000, chain.objs.size()=1 2024-04-22T20:48:16.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QorZVLxloDmsIvheM_1twXdfyt8TUsF_0 2024-04-22T20:48:16.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557928608572769017', time=2024-04-22T22:29:17.659949+0000, chain.objs.size()=1 2024-04-22T20:48:16.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.L8NUtpemTH_71s0V5os0coiacOVeJ8P_0 2024-04-22T20:48:16.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5912624658028434693', time=2024-04-22T22:29:46.901001+0000, chain.objs.size()=1 2024-04-22T20:48:16.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.358__shadow_.is6WEArBZz2U8dM_bnORx7WpJZ5svSq_0 2024-04-22T20:48:16.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14244533294745856317', time=2024-04-22T22:29:57.082817+0000, chain.objs.size()=1 2024-04-22T20:48:16.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.371__shadow_.4VLWSPlvZ-9mTgx3dgr-OZ6hVAU3qz9_0 2024-04-22T20:48:16.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14221987445556091134', time=2024-04-22T22:30:06.946040+0000, chain.objs.size()=1 2024-04-22T20:48:16.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.438+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.383__shadow:NiIX1AfzgAricIF.iV-cTGgKemS3v3q_.1FOdzHfqC2MZhLcGm_yQw0L2jSSefPy_0 2024-04-22T20:48:16.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.441+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14427577621893969389', time=2024-04-22T22:30:37.141454+0000, chain.objs.size()=1 2024-04-22T20:48:16.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.441+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.430__shadow_.93Mu8vPUpko_ge5Mri7KTcQaEBkBKNR_0 2024-04-22T20:48:16.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7467700726063449116', time=2024-04-22T22:30:53.864580+0000, chain.objs.size()=1 2024-04-22T20:48:16.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.448__shadow_.tvcVfCL23bR6h7pcgEnFiH8xOxwSRZ6_0 2024-04-22T20:48:16.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5944362223625181903', time=2024-04-22T22:31:16.311359+0000, chain.objs.size()=1 2024-04-22T20:48:16.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.476__shadow_.ZGAd0XLAkcZi9s6EFonT6Hhv6FfNYeY_0 2024-04-22T20:48:16.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2_76468172b2cadbf1717a866185092c84', time=2024-04-22T22:31:53.218171+0000, chain.objs.size()=6 2024-04-22T20:48:16.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.1 2024-04-22T20:48:16.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2 2024-04-22T20:48:16.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.3 2024-04-22T20:48:16.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.4 2024-04-22T20:48:16.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.5 2024-04-22T20:48:16.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.6 2024-04-22T20:48:16.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18185996628143512432', time=2024-04-22T22:31:59.651162+0000, chain.objs.size()=1 2024-04-22T20:48:16.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.504__shadow_.NL1EznOI4UYXLEXvh9LR572tMLBzn8c_0 2024-04-22T20:48:16.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16063692127808442492', time=2024-04-22T22:32:29.247574+0000, chain.objs.size()=12 2024-04-22T20:48:16.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.442+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__multipart_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.1 2024-04-22T20:48:16.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.443+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__shadow_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.1_1 2024-04-22T20:48:16.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__multipart_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.2 2024-04-22T20:48:16.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__shadow_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.2_1 2024-04-22T20:48:16.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__multipart_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.3 2024-04-22T20:48:16.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__shadow_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.3_1 2024-04-22T20:48:16.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__multipart_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.4 2024-04-22T20:48:16.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__shadow_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.4_1 2024-04-22T20:48:16.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__multipart_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.5 2024-04-22T20:48:16.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__shadow_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.5_1 2024-04-22T20:48:16.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__multipart_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.6 2024-04-22T20:48:16.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.508__shadow_mymultipart.2~0uvr7v_xOjZqhvZXkBqdCcudej932zN.6_1 2024-04-22T20:48:16.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6931031747054059144', time=2024-04-22T22:34:50.741990+0000, chain.objs.size()=6 2024-04-22T20:48:16.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.449+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.0XohQrci2FOkgVISYsQ1c44ufZKIQxBd.1 2024-04-22T20:48:16.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.454+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.AsaLGfnofqWttDqmynPaWW35pCfldtMG.2 2024-04-22T20:48:16.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.454+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.byf3vdQBXYhdfN2NBkZ-64IZRqtjtYnH.3 2024-04-22T20:48:16.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.454+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.hADptLuWuSfS4O9KlN0_77E8YkY1qxRA.4 2024-04-22T20:48:16.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.pUKfMtPC9uccrSY2ivWon-J9aqVSgVN2.5 2024-04-22T20:48:16.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.lNjbrkuoli-NcOIZMRX2GLIvxYKi9C8c.6 2024-04-22T20:48:16.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10592431514707165726', time=2024-04-22T22:36:40.810272+0000, chain.objs.size()=1 2024-04-22T20:48:16.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.540__shadow_.Ik0wuxbkimbreBVOSkTtjrxicHg8AKu_0 2024-04-22T20:48:16.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5965215024260209657', time=2024-04-22T22:37:30.415425+0000, chain.objs.size()=1 2024-04-22T20:48:16.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:XiDUu0eKrHaBD4xHGnxcPQPxNZXCpUD_.xfWVXa3fPDXBNtCUe0OlVuNeutPIWCT_0 2024-04-22T20:48:16.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14218812754833710955', time=2024-04-22T22:37:46.801079+0000, chain.objs.size()=1 2024-04-22T20:48:16.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:JYxK973CeLlJNXw.QYsPgPpeL1ucDXO_.GwyKNGa3Ex9ba2xCZ2DurZBTgQSqrcJ_0 2024-04-22T20:48:16.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.457+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1158095858093051290', time=2024-04-22T22:37:52.697057+0000, chain.objs.size()=1 2024-04-22T20:48:16.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.457+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:FUl9xYPhJiE-4wN9QLjdo3J8pvbEoc7_.lUME3sJ-5NSyLFFw4ezJc4HnucqPJra_0 2024-04-22T20:48:16.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.457+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5783786124262960329', time=2024-04-22T22:38:09.007955+0000, chain.objs.size()=1 2024-04-22T20:48:16.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.457+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:B3h7Z2s7YQLsDUqW6htlGKcF7J-w80P_.Z4rv5Zx4n_MwNHeQbBsOKZjRgCqEba1_0 2024-04-22T20:48:16.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.457+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12472732114493421982', time=2024-04-22T22:38:59.520675+0000, chain.objs.size()=1 2024-04-22T20:48:16.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.457+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:9MFwHR4xKk6G-WPmpxgsu9VK-A4O.87_._g9sWGwtHdxDsGU1ijUMDhDJRRHmQ_u_0 2024-04-22T20:48:16.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.459+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12858997269991964890', time=2024-04-22T22:40:58.758994+0000, chain.objs.size()=1 2024-04-22T20:48:16.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.459+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.605__shadow_.Iz81R-b0M3oA0qhVfl-4o_ilPy3Tgyh_0 2024-04-22T20:48:16.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.459+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9738421525355011067', time=2024-04-22T22:42:57.076261+0000, chain.objs.size()=1 2024-04-22T20:48:16.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.459+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.652__shadow_.9-wutzRoPo9HiH2AFTpwHTFHeiHtdg__0 2024-04-22T20:48:16.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.459+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7791857593604390721', time=2024-04-22T22:43:21.370404+0000, chain.objs.size()=1 2024-04-22T20:48:16.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.459+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.692__shadow:zoMBzE4RIrT-zrIFeI-MivoWzo26jHc_.VgrKWJ1I8o13Ek3TlnLHMys8ty-2ZQ6_0 2024-04-22T20:48:16.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.468+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.476+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-22T20:48:16.474 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.477+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.474 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.478+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.476 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.479+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/56095' 2024-04-22T20:48:16.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.479+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16539544637020298270', time=2024-04-22T22:21:42.608814+0000, chain.objs.size()=1 2024-04-22T20:48:16.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.479+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.eEyG8LBCPahXYblTtWOVgF8AeuXhhtL_0 2024-04-22T20:48:16.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.479+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14879417625006595074', time=2024-04-22T22:21:47.603520+0000, chain.objs.size()=3 2024-04-22T20:48:16.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.479+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.wZJYQnW9-UgwsmoCBnwT9IaeKvcksV4_0 2024-04-22T20:48:16.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.479+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.wZJYQnW9-UgwsmoCBnwT9IaeKvcksV4_1 2024-04-22T20:48:16.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.479+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.wZJYQnW9-UgwsmoCBnwT9IaeKvcksV4_2 2024-04-22T20:48:16.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.479+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11245667451825171883', time=2024-04-22T22:22:33.079642+0000, chain.objs.size()=1 2024-04-22T20:48:16.477 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.159__shadow_.O894xcpvxgTVVT0M82x8bfmVHIxqAss_0 2024-04-22T20:48:16.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2196426367540106908', time=2024-04-22T22:22:51.496467+0000, chain.objs.size()=1 2024-04-22T20:48:16.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.170__shadow_.6jkaLMJ5zcAA6Foni7HUTA9CzHAp-8E_0 2024-04-22T20:48:16.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5217624037607910092', time=2024-04-22T22:25:34.842923+0000, chain.objs.size()=1 2024-04-22T20:48:16.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.239__shadow_.L4w-RjUKYChz0OIkf4q8RIuQbBrumMH_0 2024-04-22T20:48:16.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.140149448717120752', time=2024-04-22T22:25:34.843725+0000, chain.objs.size()=1 2024-04-22T20:48:16.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.239__shadow_.GBnuM7p4CIp6r5nz1bgzBnMzwB4OTM0_0 2024-04-22T20:48:16.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7352349444338855278', time=2024-04-22T22:25:36.817831+0000, chain.objs.size()=1 2024-04-22T20:48:16.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.241__shadow_.6ItXvSWcVRsTZIGVOD_qkm8X-zrWkYv_0 2024-04-22T20:48:16.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9006212156728262329', time=2024-04-22T22:25:36.827776+0000, chain.objs.size()=1 2024-04-22T20:48:16.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.241__shadow_.ajUFLwCItsQVbRBR--EikHgoInVKiQJ_0 2024-04-22T20:48:16.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2259535133008819075', time=2024-04-22T22:25:44.672961+0000, chain.objs.size()=1 2024-04-22T20:48:16.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.249__shadow_.7w7zvX47RbaFwjMcV011yDYypt8ifcd_0 2024-04-22T20:48:16.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13723067611766901411', time=2024-04-22T22:26:22.929183+0000, chain.objs.size()=1 2024-04-22T20:48:16.479 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.480+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.L3kGYoQEoY0iOTVjflXiNI-aaSSsSZS_0 2024-04-22T20:48:16.480 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.484+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2164449716422909496', time=2024-04-22T22:26:22.974620+0000, chain.objs.size()=1 2024-04-22T20:48:16.481 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.484+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.whHzsksZtXVv28Zr-HCUlXG0UnceZoJ_0 2024-04-22T20:48:16.483 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.486+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16587990600110088675', time=2024-04-22T22:26:23.040580+0000, chain.objs.size()=1 2024-04-22T20:48:16.483 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.487+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.m5KRKgQKVWh6d0o45d9_HHMdec2tXW9_0 2024-04-22T20:48:16.486 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4557169638496766776', time=2024-04-22T22:26:23.340516+0000, chain.objs.size()=1 2024-04-22T20:48:16.486 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1wXl87xXSn-fFcjL2wvH0tvxn-hWDIk_0 2024-04-22T20:48:16.486 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16916886651108529949', time=2024-04-22T22:26:24.211121+0000, chain.objs.size()=1 2024-04-22T20:48:16.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.38f-GIqpmnidv3R9ckDaAT-1M-iL5jf_0 2024-04-22T20:48:16.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2720337291172502446', time=2024-04-22T22:26:24.857534+0000, chain.objs.size()=1 2024-04-22T20:48:16.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZQp7lGifqy5V5XqyYLTKK-HRshHjERX_0 2024-04-22T20:48:16.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17157202485303673970', time=2024-04-22T22:26:24.965586+0000, chain.objs.size()=1 2024-04-22T20:48:16.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pOU8zUQ2hRASaeM-VVpQkZZTTFGMeBv_0 2024-04-22T20:48:16.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2887567530211270967', time=2024-04-22T22:26:25.047950+0000, chain.objs.size()=1 2024-04-22T20:48:16.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pDzEIpug_g0_sIKskZDWbc6A7k7ZAn8_0 2024-04-22T20:48:16.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4142532853776681946', time=2024-04-22T22:26:25.833106+0000, chain.objs.size()=1 2024-04-22T20:48:16.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.fgxwvFjCHsXqbjOZz2yK_kEIscotbMK_0 2024-04-22T20:48:16.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12187133533835930550', time=2024-04-22T22:26:26.124116+0000, chain.objs.size()=1 2024-04-22T20:48:16.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.J8iU-9ciNyD4LH41leSKeHo4jAquEl5_0 2024-04-22T20:48:16.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13232207448110862966', time=2024-04-22T22:26:26.134791+0000, chain.objs.size()=1 2024-04-22T20:48:16.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Uc3agGciVWtTKiQLQ_giJ1fQCmwdDmY_0 2024-04-22T20:48:16.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15234201995832573694', time=2024-04-22T22:26:26.457534+0000, chain.objs.size()=1 2024-04-22T20:48:16.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EP9e1VDFYghxgBP4-9KNUmvukC-Evdg_0 2024-04-22T20:48:16.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12139956060608786315', time=2024-04-22T22:26:26.821655+0000, chain.objs.size()=1 2024-04-22T20:48:16.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.489+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5RyfPsCpv3jPpVnQ7bCZ-t_nOPiycax_0 2024-04-22T20:48:16.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.492+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9365335073327212133', time=2024-04-22T22:26:27.803881+0000, chain.objs.size()=1 2024-04-22T20:48:16.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.492+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aepE1ySUF9zy6Map_Az-clP5UpnVsPe_0 2024-04-22T20:48:16.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.494+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10546922829722141379', time=2024-04-22T22:26:27.908608+0000, chain.objs.size()=1 2024-04-22T20:48:16.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.494+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.nMK_WtWJgFWgs24d2YvAOQC6ZIxjps4_0 2024-04-22T20:48:16.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10079576423775108783', time=2024-04-22T22:26:28.174014+0000, chain.objs.size()=1 2024-04-22T20:48:16.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3AnzAsgglf7angqbM-c52DV_0IPxtgl_0 2024-04-22T20:48:16.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3320582653874262141', time=2024-04-22T22:26:28.345353+0000, chain.objs.size()=1 2024-04-22T20:48:16.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.U1W7J0v1IjJW9fkT64V5tMNwB8iuQog_0 2024-04-22T20:48:16.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15589609474652526141', time=2024-04-22T22:26:29.389597+0000, chain.objs.size()=1 2024-04-22T20:48:16.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dVY7sSRqqgYwD7ZxoBFZBJmyjvg-OcM_0 2024-04-22T20:48:16.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5862225573341512682', time=2024-04-22T22:26:29.944832+0000, chain.objs.size()=1 2024-04-22T20:48:16.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rl8yj84wzkzZVwmZ9y_gWOogDlGYFv1_0 2024-04-22T20:48:16.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13193695770577244705', time=2024-04-22T22:26:29.965913+0000, chain.objs.size()=1 2024-04-22T20:48:16.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.s43_af0ocKhOUfZJVEBxBjbd1Y7Bd3M_0 2024-04-22T20:48:16.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8249560655209156772', time=2024-04-22T22:26:30.180975+0000, chain.objs.size()=1 2024-04-22T20:48:16.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Uf4PieN64_WAgLK3IBzrwHDzYUA1Xst_0 2024-04-22T20:48:16.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.733373996777469209', time=2024-04-22T22:26:30.541882+0000, chain.objs.size()=1 2024-04-22T20:48:16.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Nxo9tONXuLs8Gb05Yfne2rP6oYi1Mi1_0 2024-04-22T20:48:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12359363080914570177', time=2024-04-22T22:26:30.548955+0000, chain.objs.size()=1 2024-04-22T20:48:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1LlVuTlXWS09pL65s2VIzx3-n5wb80-_0 2024-04-22T20:48:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.625532850557538989', time=2024-04-22T22:26:30.728377+0000, chain.objs.size()=1 2024-04-22T20:48:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Nh07VHEOwtEPosyW99TWVX1RViZUFY4_0 2024-04-22T20:48:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18203257294033086343', time=2024-04-22T22:26:30.729029+0000, chain.objs.size()=1 2024-04-22T20:48:16.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.495+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.n3wQUKejlCvlEOtqBmOnbumUO0dohT3_0 2024-04-22T20:48:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.498+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6988912156379567325', time=2024-04-22T22:26:31.191203+0000, chain.objs.size()=1 2024-04-22T20:48:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.498+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aPgfatG14rOy4UjzH2iya8ZqHf1YH72_0 2024-04-22T20:48:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.498+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.509751860434767598', time=2024-04-22T22:26:31.201156+0000, chain.objs.size()=1 2024-04-22T20:48:16.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.498+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZXCyot4VfDJCI80ZtsUpEIO5LN6MoV1_0 2024-04-22T20:48:16.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.499+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11540911745517946325', time=2024-04-22T22:26:31.375159+0000, chain.objs.size()=1 2024-04-22T20:48:16.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.499+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.U_PvZBkPfsajHifF7-h0IonWfwK6gq7_0 2024-04-22T20:48:16.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.499+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17532023623772493550', time=2024-04-22T22:26:31.685679+0000, chain.objs.size()=1 2024-04-22T20:48:16.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.499+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lVkVVTwYadimZPBqTB1GjgoZxUXxZAE_0 2024-04-22T20:48:16.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.499+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4762037022017143398', time=2024-04-22T22:26:31.710003+0000, chain.objs.size()=1 2024-04-22T20:48:16.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.499+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5bZQivnprGTxZMeC2otyP6P4B0fwmwj_0 2024-04-22T20:48:16.497 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16645314377318145644', time=2024-04-22T22:26:32.385339+0000, chain.objs.size()=1 2024-04-22T20:48:16.497 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sXjhDu5gyhS2Fwkyr_y6Fp9sHSoGnQk_0 2024-04-22T20:48:16.497 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12688958583548106234', time=2024-04-22T22:26:32.600977+0000, chain.objs.size()=1 2024-04-22T20:48:16.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2NKJQVVPs9vn44JoGCf6QM3Be8t47uM_0 2024-04-22T20:48:16.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4331061652267041496', time=2024-04-22T22:26:32.747075+0000, chain.objs.size()=1 2024-04-22T20:48:16.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.b4vw8NHv6Lw7a3yq6FeFyzLElweyriH_0 2024-04-22T20:48:16.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1289643710642705309', time=2024-04-22T22:26:32.944285+0000, chain.objs.size()=1 2024-04-22T20:48:16.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.26-n4cAsXTLglo3Fm3xZjaw6t9GFcWJ_0 2024-04-22T20:48:16.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7616427785515477044', time=2024-04-22T22:26:33.152162+0000, chain.objs.size()=1 2024-04-22T20:48:16.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.500+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.erPhobgvDXdb4EfyPghLK1uLFIWHYuQ_0 2024-04-22T20:48:16.499 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.501+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9729242545773198977', time=2024-04-22T22:26:33.839555+0000, chain.objs.size()=1 2024-04-22T20:48:16.499 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.501+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VdEcY4AS5rewikXZdrc-4sr17cWDwZw_0 2024-04-22T20:48:16.499 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.501+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12706251925804572993', time=2024-04-22T22:26:33.921649+0000, chain.objs.size()=1 2024-04-22T20:48:16.499 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.501+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.k_w5TqC0LyUPC28wfHh_qjUZc_kYnOr_0 2024-04-22T20:48:16.499 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4415051219629188084', time=2024-04-22T22:26:34.275357+0000, chain.objs.size()=1 2024-04-22T20:48:16.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bCuCVsCQnOevjBH48PWpsqJlxx6nnCV_0 2024-04-22T20:48:16.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10924279723841011895', time=2024-04-22T22:26:38.306042+0000, chain.objs.size()=1 2024-04-22T20:48:16.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.255__shadow_.bXLphu7_wxz0V5475MPhC5mV0s5hOsO_0 2024-04-22T20:48:16.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16132020779678572013', time=2024-04-22T22:26:38.315707+0000, chain.objs.size()=1 2024-04-22T20:48:16.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.255__shadow_.BqQ3FYVfXM6SukaDVB5jZq9aiPSJvcd_0 2024-04-22T20:48:16.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6388227496690107432', time=2024-04-22T22:26:44.969357+0000, chain.objs.size()=1 2024-04-22T20:48:16.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.262__shadow_.QGkEu8KiXH68HPP5j9fDkuk5JotqdHt_0 2024-04-22T20:48:16.501 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2942455812122926211', time=2024-04-22T22:27:05.593368+0000, chain.objs.size()=1 2024-04-22T20:48:16.501 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.503+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.C5ZbbX2K5IsiywPIdXgGSz6f1YXULyA_0 2024-04-22T20:48:16.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4384304641739033125', time=2024-04-22T22:27:10.985339+0000, chain.objs.size()=1 2024-04-22T20:48:16.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.285__shadow_.WiyxD-OGJcTqPcnQsxuL7VtubvHkVLj_0 2024-04-22T20:48:16.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4469114199936701386', time=2024-04-22T22:28:17.646728+0000, chain.objs.size()=1 2024-04-22T20:48:16.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.99EbO72KKm71BWGAMVIWChhqEWufh3J_0 2024-04-22T20:48:16.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5822775603467401078', time=2024-04-22T22:28:18.161233+0000, chain.objs.size()=1 2024-04-22T20:48:16.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ACaeQR9dH0YaYbdzknaHYS2EnYIXf-l_0 2024-04-22T20:48:16.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14916224788480557339', time=2024-04-22T22:28:18.300390+0000, chain.objs.size()=1 2024-04-22T20:48:16.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zB0AuE4vuiynvkWPn0xDs6vyU8qqSdM_0 2024-04-22T20:48:16.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16497913717225996290', time=2024-04-22T22:28:19.518810+0000, chain.objs.size()=1 2024-04-22T20:48:16.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.n2qspnhIJyS4DBsm-8hF2BjJCOeQmjX_0 2024-04-22T20:48:16.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13057761860954630943', time=2024-04-22T22:28:20.181081+0000, chain.objs.size()=1 2024-04-22T20:48:16.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.505+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GRPvyF5Qljp0SMOKoUh9lXDhqx4JwQt_0 2024-04-22T20:48:16.504 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4429057099471986275', time=2024-04-22T22:28:20.894271+0000, chain.objs.size()=1 2024-04-22T20:48:16.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4ch0_pB-D2r3VIpMqk5dKcySiUNHST5_0 2024-04-22T20:48:16.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7659935234155737126', time=2024-04-22T22:28:21.572925+0000, chain.objs.size()=1 2024-04-22T20:48:16.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.YxIwatCLIL7INp5iEkGg1LnhJEuViT1_0 2024-04-22T20:48:16.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7758651122665187504', time=2024-04-22T22:28:22.490196+0000, chain.objs.size()=1 2024-04-22T20:48:16.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Fl6OLDOqlHtGC2F8HeS3Sv8JXWttSkh_0 2024-04-22T20:48:16.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13268809624076584569', time=2024-04-22T22:28:22.543807+0000, chain.objs.size()=1 2024-04-22T20:48:16.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FIfd6x5eRevOrJeBBNegSkdWO4mfIwl_0 2024-04-22T20:48:16.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4269911052874008226', time=2024-04-22T22:28:22.545275+0000, chain.objs.size()=1 2024-04-22T20:48:16.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.508+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WgJbTPuSfPaB-SoD8WT3ugI6L1GKSZI_0 2024-04-22T20:48:16.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.510+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4990941259996903032', time=2024-04-22T22:28:22.547582+0000, chain.objs.size()=1 2024-04-22T20:48:16.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.510+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.oPjMcTaryloCabhMquWZMa-uymjQgl__0 2024-04-22T20:48:16.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.510+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8288877138602593543', time=2024-04-22T22:28:22.890298+0000, chain.objs.size()=1 2024-04-22T20:48:16.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.510+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.M8-Q-CbVLHfEJgEXE83YSZXStUdQhuc_0 2024-04-22T20:48:16.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.510+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16353313140024672636', time=2024-04-22T22:28:23.079922+0000, chain.objs.size()=1 2024-04-22T20:48:16.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.510+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7_1EFM1mAH8QLTdtnzYzlXZ6uYT-MCM_0 2024-04-22T20:48:16.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3618601769410033644', time=2024-04-22T22:28:23.411133+0000, chain.objs.size()=1 2024-04-22T20:48:16.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZHqQt43hl3VHxm-rrixUPH-qZftlIFo_0 2024-04-22T20:48:16.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8105388010423119234', time=2024-04-22T22:28:23.880777+0000, chain.objs.size()=1 2024-04-22T20:48:16.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Wi_q5jt3wQ7dKL2L5hL2OgpjrYf4mNM_0 2024-04-22T20:48:16.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11180573386612540358', time=2024-04-22T22:28:24.253040+0000, chain.objs.size()=1 2024-04-22T20:48:16.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1TPsfLj3_KLNZqBNTnRLumu3BZ3y2lI_0 2024-04-22T20:48:16.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13122120012121959392', time=2024-04-22T22:28:24.476415+0000, chain.objs.size()=1 2024-04-22T20:48:16.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.YmGVv8qo6i5QRQCEu9l-l-rs3wiGPpt_0 2024-04-22T20:48:16.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5947662517354057146', time=2024-04-22T22:28:24.476969+0000, chain.objs.size()=1 2024-04-22T20:48:16.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.e8Z-lVKz9PDQUfWsNplQwsYvQiDJMf2_0 2024-04-22T20:48:16.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2670220085247557568', time=2024-04-22T22:28:25.007826+0000, chain.objs.size()=1 2024-04-22T20:48:16.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.39aIDl4GX6JZ40-t6JcOsghvSpW8Gq__0 2024-04-22T20:48:16.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13423773005517006958', time=2024-04-22T22:28:25.523789+0000, chain.objs.size()=1 2024-04-22T20:48:16.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cUoHFLl-usjChbdTC6SlJm23qX9A7gb_0 2024-04-22T20:48:16.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5778538953868287610', time=2024-04-22T22:28:25.759038+0000, chain.objs.size()=1 2024-04-22T20:48:16.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.513+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Nn1oZoCzuhOHfoPcRiAeh8wk2QRc8qT_0 2024-04-22T20:48:16.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.516+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14754518133209768979', time=2024-04-22T22:28:26.105972+0000, chain.objs.size()=1 2024-04-22T20:48:16.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.516+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FW0hr-pBM87ekx7jjXFdnSu-J227qLS_0 2024-04-22T20:48:16.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.516+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7021614639027424172', time=2024-04-22T22:28:26.371870+0000, chain.objs.size()=1 2024-04-22T20:48:16.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.516+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.58fFP49vxjhZnQZocOtHBPlFUZNPTSL_0 2024-04-22T20:48:16.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.516+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5692495710347798144', time=2024-04-22T22:28:27.055816+0000, chain.objs.size()=1 2024-04-22T20:48:16.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.516+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4MOe1K2CVtgj1g03-suPlzBpW0ui7TO_0 2024-04-22T20:48:16.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.516+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5307166143486661446', time=2024-04-22T22:28:27.137696+0000, chain.objs.size()=1 2024-04-22T20:48:16.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.516+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.L_j3E4Hyzj0FSItujwSXh5HfhHzuc_8_0 2024-04-22T20:48:16.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.517+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9845250853113938130', time=2024-04-22T22:28:27.290149+0000, chain.objs.size()=1 2024-04-22T20:48:16.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.517+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.B8BFQBVv1HFxbNVJ2M6gFKRxcJDwQaz_0 2024-04-22T20:48:16.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16357225114897611293', time=2024-04-22T22:28:27.461482+0000, chain.objs.size()=1 2024-04-22T20:48:16.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WzoE05oeZbsbfoBct2QUF0951I48Y4X_0 2024-04-22T20:48:16.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1624533791890897707', time=2024-04-22T22:28:27.477789+0000, chain.objs.size()=1 2024-04-22T20:48:16.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OGX4B0KgNrsnhy-dhiGAqxvrP3YmARn_0 2024-04-22T20:48:16.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14527314996911462330', time=2024-04-22T22:28:27.478485+0000, chain.objs.size()=1 2024-04-22T20:48:16.515 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qZBFQ5ghxGZx-BBajvu5Dw9jPtr1sKw_0 2024-04-22T20:48:16.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14069999185473434375', time=2024-04-22T22:28:28.638597+0000, chain.objs.size()=1 2024-04-22T20:48:16.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Uo0T2zrZv35vvy-ZHGyXkwHvhDY1qh7_0 2024-04-22T20:48:16.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8139257197854668216', time=2024-04-22T22:28:28.785180+0000, chain.objs.size()=1 2024-04-22T20:48:16.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.518+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.c9HsNW5HqU7nQy4WW5og4K-vXFEdbDN_0 2024-04-22T20:48:16.516 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.519+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5799993119055202566', time=2024-04-22T22:28:29.257559+0000, chain.objs.size()=1 2024-04-22T20:48:16.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.519+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FtCODwoNjcIjES2-b1uAHVHoU64pUjI_0 2024-04-22T20:48:16.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.519+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17859357531401310916', time=2024-04-22T22:29:06.178909+0000, chain.objs.size()=1 2024-04-22T20:48:16.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.519+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._fY_qwcn3vr0qXSlpWgwyCtmpy-bqI-_0 2024-04-22T20:48:16.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.519+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2141000196894954236', time=2024-04-22T22:29:06.526358+0000, chain.objs.size()=1 2024-04-22T20:48:16.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.519+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qad5u3gg_qIVHVa7S1QvTyNaF4sEvU0_0 2024-04-22T20:48:16.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.521+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6437788064055099739', time=2024-04-22T22:29:07.236052+0000, chain.objs.size()=1 2024-04-22T20:48:16.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.521+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kWJODHEx9caiTIMbjvPFEVN-86-d4Dt_0 2024-04-22T20:48:16.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.521+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4119272236978591184', time=2024-04-22T22:29:08.145178+0000, chain.objs.size()=1 2024-04-22T20:48:16.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.521+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WWsSUJMNlS4x92zVYG2-aOMG3c5xk2K_0 2024-04-22T20:48:16.518 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.522+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12803215737414334331', time=2024-04-22T22:29:08.188283+0000, chain.objs.size()=1 2024-04-22T20:48:16.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.522+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.w7jfSKvcsaeEb1_WLT7nTuLDIydbj2w_0 2024-04-22T20:48:16.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.522+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13733299296070841325', time=2024-04-22T22:29:08.716669+0000, chain.objs.size()=1 2024-04-22T20:48:16.519 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.522+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZCsrKDnZ28bzqW_Hx7QlZ5bcFEWFluQ_0 2024-04-22T20:48:16.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11653479265355157263', time=2024-04-22T22:29:08.804605+0000, chain.objs.size()=1 2024-04-22T20:48:16.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7f0AHY5fWk4QYJaozTrSfw9WG89NWr3_0 2024-04-22T20:48:16.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12838547984692089448', time=2024-04-22T22:29:09.410599+0000, chain.objs.size()=1 2024-04-22T20:48:16.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.lA-KzlrZhv54ToVQj-5nlIfhVOUKOGL_0 2024-04-22T20:48:16.521 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8985774255328288240', time=2024-04-22T22:29:09.652844+0000, chain.objs.size()=1 2024-04-22T20:48:16.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pXfDhvUX5VtbRYykDROcBHHGiisGa41_0 2024-04-22T20:48:16.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12811447467431263615', time=2024-04-22T22:29:09.760627+0000, chain.objs.size()=1 2024-04-22T20:48:16.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mWJztAmzR0hwBMVD19Ty2iuYRaWbzdx_0 2024-04-22T20:48:16.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5726225694652466407', time=2024-04-22T22:29:10.215583+0000, chain.objs.size()=1 2024-04-22T20:48:16.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.D0Dr_x5QC3rwc1a_mc08B718GqaIkIF_0 2024-04-22T20:48:16.522 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9775705801152133601', time=2024-04-22T22:29:10.892032+0000, chain.objs.size()=1 2024-04-22T20:48:16.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.524+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1x150bSAudcnf4MPJYaPjbiDfvkVsNz_0 2024-04-22T20:48:16.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.525+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11219577013549982277', time=2024-04-22T22:29:11.164664+0000, chain.objs.size()=1 2024-04-22T20:48:16.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.525+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ShDCNLoZRxBDU6W-thwYvmaQR8hVMru_0 2024-04-22T20:48:16.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.525+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8571546827144571076', time=2024-04-22T22:29:11.365842+0000, chain.objs.size()=1 2024-04-22T20:48:16.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.525+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4L2Qmb1a8PePty3te-qjZMuyDcj1aYV_0 2024-04-22T20:48:16.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.525+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18109067301383610515', time=2024-04-22T22:29:11.509432+0000, chain.objs.size()=1 2024-04-22T20:48:16.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.525+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.vJFC3WVnfPzGX8IrNyG1rGQBeOC7-m4_0 2024-04-22T20:48:16.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.526+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7865085604548307822', time=2024-04-22T22:29:12.611873+0000, chain.objs.size()=1 2024-04-22T20:48:16.525 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.526+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.a40ZaMVnrdo6oiQcz7OwxqoRFoAasHf_0 2024-04-22T20:48:16.526 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.529+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/56095 2024-04-22T20:48:16.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=26, truncated=0, next_marker='' 2024-04-22T20:48:16.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9317527091819533033', time=2024-04-22T22:29:12.921610+0000, chain.objs.size()=1 2024-04-22T20:48:16.532 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TX3fH7P2ndHRUacO6CKikgO_7QkU3DB_0 2024-04-22T20:48:16.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14247037048344156647', time=2024-04-22T22:29:13.338181+0000, chain.objs.size()=1 2024-04-22T20:48:16.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hLeR9uPmaZww96Glc_s6d1WQTSN5Qim_0 2024-04-22T20:48:16.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15951434625431644346', time=2024-04-22T22:29:13.471852+0000, chain.objs.size()=1 2024-04-22T20:48:16.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.VrxAEsnluwvfJZRTW7Am753HzU-LKjr_0 2024-04-22T20:48:16.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8622679942518865270', time=2024-04-22T22:29:14.060437+0000, chain.objs.size()=1 2024-04-22T20:48:16.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.w8UNa2aif6coCvWySxNtk8APV3zTcQU_0 2024-04-22T20:48:16.533 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14387962927997610420', time=2024-04-22T22:29:14.481827+0000, chain.objs.size()=1 2024-04-22T20:48:16.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tDZG_cGqT6ocWFy6ffMBJdzGKxzesp3_0 2024-04-22T20:48:16.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1109894067952267438', time=2024-04-22T22:29:14.663317+0000, chain.objs.size()=1 2024-04-22T20:48:16.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4KCDA6Q0kK_k1mOLSK0wRmOSHPq9Dj3_0 2024-04-22T20:48:16.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13768724450660694581', time=2024-04-22T22:29:15.296262+0000, chain.objs.size()=1 2024-04-22T20:48:16.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.h2w8B-rGto9nt0mMEMqcLrGZf1m2iSb_0 2024-04-22T20:48:16.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17093921784506176458', time=2024-04-22T22:29:15.364578+0000, chain.objs.size()=1 2024-04-22T20:48:16.534 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HHRaJ_Ftu_-_w21eFglaxnJR6ksIpjh_0 2024-04-22T20:48:16.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3027298222782524466', time=2024-04-22T22:29:16.773606+0000, chain.objs.size()=1 2024-04-22T20:48:16.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nrUJRGsQ6P1YfmoJn9IHzG7WwbFvmYD_0 2024-04-22T20:48:16.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10837787519580800537', time=2024-04-22T22:29:17.105953+0000, chain.objs.size()=1 2024-04-22T20:48:16.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6hrcU12VlXzyr35TXfvPb-sOusxw21i_0 2024-04-22T20:48:16.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.498603043112331362', time=2024-04-22T22:29:45.718028+0000, chain.objs.size()=1 2024-04-22T20:48:16.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.356__shadow_.Sk-uNIFTNQZHL9C_xniIefRZmt1xl4W_0 2024-04-22T20:48:16.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2911450116834633', time=2024-04-22T22:29:52.576308+0000, chain.objs.size()=1 2024-04-22T20:48:16.535 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.535+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.365__shadow_.CE64b9qx4e5za0vz-bUD8Uup8EJqFRZ_0 2024-04-22T20:48:16.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.538+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7322639533637428333', time=2024-04-22T22:30:58.490605+0000, chain.objs.size()=1 2024-04-22T20:48:16.536 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.538+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.452__shadow_.mq67iqHeiW-KK4EMfZDLc6ToWz9-HVn_0 2024-04-22T20:48:16.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.538+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1392338565301742485', time=2024-04-22T22:30:59.498392+0000, chain.objs.size()=1 2024-04-22T20:48:16.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.538+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.453__shadow_.eVVUN_Fai8yws1Dk-GBCyUZ38Io10-R_0 2024-04-22T20:48:16.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12369603780425411785', time=2024-04-22T22:31:00.040829+0000, chain.objs.size()=1 2024-04-22T20:48:16.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.454__shadow_.0EUPp1Y3KLFfpGh-F_0j-tnDYLd5cEZ_0 2024-04-22T20:48:16.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8610010535960570676', time=2024-04-22T22:31:21.230571+0000, chain.objs.size()=1 2024-04-22T20:48:16.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.484__shadow_.Dpq0vGsicQtqYEHswgZySKlYud89EuE_0 2024-04-22T20:48:16.537 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14935419160415650477', time=2024-04-22T22:37:08.937024+0000, chain.objs.size()=1 2024-04-22T20:48:16.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.548__shadow_._8A2LcC5hYvhxnz4l5s1vy48HC7m4iH_0 2024-04-22T20:48:16.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1620851861561502122', time=2024-04-22T22:37:23.810457+0000, chain.objs.size()=1 2024-04-22T20:48:16.538 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:qINit1YrvnYVmTZMn.JUHIo7MDCbuLK_.xRGFkMStqYf28OyM3Ba1E5v_nMBSzjE_0 2024-04-22T20:48:16.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5976816569038195959', time=2024-04-22T22:37:26.067014+0000, chain.objs.size()=1 2024-04-22T20:48:16.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:CS84pjBhMzcfXlfQ1.zCFyUXB36O8Oy_.HUe7rfj6kVVfRP8hIumBmJngIfTrlW6_0 2024-04-22T20:48:16.539 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3272658647212091551', time=2024-04-22T22:37:45.451325+0000, chain.objs.size()=1 2024-04-22T20:48:16.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:2bzyAQo8Rf54F7aAGmKv4H-9WBg0Eww_.nudFgkaaVkjY0KBsX_g9zpseDv9At4f_0 2024-04-22T20:48:16.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18368513284365283450', time=2024-04-22T22:37:48.335322+0000, chain.objs.size()=1 2024-04-22T20:48:16.541 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:8Xzu0fhbGpYy6B2OtV89TRqY-Plmzs1_.pBw43_d7WqL5HvY8egMgLwujqih1KkE_0 2024-04-22T20:48:16.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6043236337820542480', time=2024-04-22T22:37:57.947333+0000, chain.objs.size()=1 2024-04-22T20:48:16.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.540+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:T7Zri4-wv.-relkG5jsM1VdxKlWQKWY_.liZFht4V1Cn8RLAvCaVYAg30OG07pYk_0 2024-04-22T20:48:16.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.541+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3283588992870942738', time=2024-04-22T22:38:52.855932+0000, chain.objs.size()=1 2024-04-22T20:48:16.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.541+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.570__shadow:1JT2f9QtvtAajM69zEAKxTkRKoV4TGX_.J04UY0UHGp1KuNibfXbCr5eU6slJdPZ_0 2024-04-22T20:48:16.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.541+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14057872349199624749', time=2024-04-22T22:40:57.429656+0000, chain.objs.size()=12 2024-04-22T20:48:16.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.541+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__multipart_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.1 2024-04-22T20:48:16.542 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.541+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__shadow_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.1_1 2024-04-22T20:48:16.543 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__multipart_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.2 2024-04-22T20:48:16.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__shadow_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.2_1 2024-04-22T20:48:16.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__multipart_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.3 2024-04-22T20:48:16.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__shadow_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.3_1 2024-04-22T20:48:16.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__multipart_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.4 2024-04-22T20:48:16.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__shadow_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.4_1 2024-04-22T20:48:16.544 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__multipart_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.5 2024-04-22T20:48:16.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__shadow_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.5_1 2024-04-22T20:48:16.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__multipart_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.6 2024-04-22T20:48:16.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.603__shadow_multipart_enc.2~pUY-pSLyO2VKcloqxVJDntBSwnpWBRi.6_1 2024-04-22T20:48:16.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14949112664842538482', time=2024-04-22T22:41:53.275886+0000, chain.objs.size()=1 2024-04-22T20:48:16.545 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.546+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.619__shadow_.dwHyFv5Gx-JOSvxlqcE3jNTZg0828gn_0 2024-04-22T20:48:16.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.549+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6522014882972309667', time=2024-04-22T22:43:19.385918+0000, chain.objs.size()=1 2024-04-22T20:48:16.546 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.549+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.689__shadow:SQf9CCQoi4ZVfVT8pRI9v-SECz2v.x0_.qpHMtDF30BpurwMgPZNZbhbCoywh3wh_0 2024-04-22T20:48:16.552 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.556+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.555 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.558+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-22T20:48:16.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.559+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.556 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.560+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/50416' 2024-04-22T20:48:16.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13920003825138783335', time=2024-04-22T22:21:32.152271+0000, chain.objs.size()=3 2024-04-22T20:48:16.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.147__shadow_.ZEt0uQWgXnoJVUuOmmRR-cd-7PORsQU_0 2024-04-22T20:48:16.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.147__shadow_.ZEt0uQWgXnoJVUuOmmRR-cd-7PORsQU_1 2024-04-22T20:48:16.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.147__shadow_.ZEt0uQWgXnoJVUuOmmRR-cd-7PORsQU_2 2024-04-22T20:48:16.558 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_Whi4wXTgum0gig-_Zn5gnl6FZ7qN5Qp', time=2024-04-22T22:21:55.335889+0000, chain.objs.size()=1 2024-04-22T20:48:16.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.Tuy3aNkekVI-iSkj3UV_82E5qKSj9ss_0 2024-04-22T20:48:16.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10627899599978574905', time=2024-04-22T22:25:24.432453+0000, chain.objs.size()=1 2024-04-22T20:48:16.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.230__shadow_.47XkXIxS_8296bzzdrgAOix4s1j6VRZ_0 2024-04-22T20:48:16.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2692759470460959221', time=2024-04-22T22:25:32.812673+0000, chain.objs.size()=1 2024-04-22T20:48:16.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.237__shadow_.1Zo-QVB9jAAaYlzoZqUWiEgzSfQSTz0_0 2024-04-22T20:48:16.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3869401614917344173', time=2024-04-22T22:26:22.837450+0000, chain.objs.size()=1 2024-04-22T20:48:16.559 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4o8bT75OtsdphoXGb_g9YJD95et2693_0 2024-04-22T20:48:16.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16489064389701255577', time=2024-04-22T22:26:23.091160+0000, chain.objs.size()=1 2024-04-22T20:48:16.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mUWY_WHIPjNgiyvrNnIWH8a9UtfmhUm_0 2024-04-22T20:48:16.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10100714474747824880', time=2024-04-22T22:26:24.007668+0000, chain.objs.size()=1 2024-04-22T20:48:16.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0OQS8JFK_zfH01uQ1zaMUac4cPnPgxI_0 2024-04-22T20:48:16.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7406446372333529604', time=2024-04-22T22:26:24.868288+0000, chain.objs.size()=1 2024-04-22T20:48:16.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qrHVx3VXaUtxLLjwev3qynnAKSpzNm1_0 2024-04-22T20:48:16.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.808655589389709256', time=2024-04-22T22:26:24.975508+0000, chain.objs.size()=1 2024-04-22T20:48:16.560 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.RDDTn9jONBDnJz1x2Y2gshjjCVbpqA9_0 2024-04-22T20:48:16.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5997921630108953070', time=2024-04-22T22:26:25.102862+0000, chain.objs.size()=1 2024-04-22T20:48:16.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.561+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ND7cmmU19Y47a6ua3jkpUJyeOhJxAxz_0 2024-04-22T20:48:16.561 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8439362500558123758', time=2024-04-22T22:26:26.462582+0000, chain.objs.size()=1 2024-04-22T20:48:16.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.IpCaI3pKCF3GkV0HJEhjULKyaqkPeha_0 2024-04-22T20:48:16.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2900703180592950003', time=2024-04-22T22:26:27.573802+0000, chain.objs.size()=1 2024-04-22T20:48:16.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Pj7xDlSaSBS5Knyo71dPIJ9INPt1FSx_0 2024-04-22T20:48:16.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10314096147804751519', time=2024-04-22T22:26:27.646665+0000, chain.objs.size()=1 2024-04-22T20:48:16.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sA4mk0Ioh6HafRoV5p0O5g4RZAnXSzW_0 2024-04-22T20:48:16.562 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13737698579599107901', time=2024-04-22T22:26:27.650871+0000, chain.objs.size()=1 2024-04-22T20:48:16.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.e5R5wuVwiTVG7HkTN3yTXXdlYuvjMc-_0 2024-04-22T20:48:16.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7608966937293710229', time=2024-04-22T22:26:27.729258+0000, chain.objs.size()=1 2024-04-22T20:48:16.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.s6hT1bPpPmE7dx1Sp9ScYm6sDHCUSVx_0 2024-04-22T20:48:16.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6034331151661501978', time=2024-04-22T22:26:28.537682+0000, chain.objs.size()=1 2024-04-22T20:48:16.563 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.564+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.iTJvV23cbyglTp4Or34jwQrG2BvrF0S_0 2024-04-22T20:48:16.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6677167525775155399', time=2024-04-22T22:26:29.554022+0000, chain.objs.size()=1 2024-04-22T20:48:16.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OiE7eGqFDGYMsDpZDpAMEKpvY4zGgzy_0 2024-04-22T20:48:16.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9916996268400011349', time=2024-04-22T22:26:29.612549+0000, chain.objs.size()=1 2024-04-22T20:48:16.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2n6DGm72xblmXIHApsT-UhnvBf8R8pf_0 2024-04-22T20:48:16.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16367280077094256125', time=2024-04-22T22:26:29.935914+0000, chain.objs.size()=1 2024-04-22T20:48:16.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3Ly8h5KLGGDrUGAXGBuYnjV1Ne5GZn0_0 2024-04-22T20:48:16.564 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17518622426369782446', time=2024-04-22T22:26:30.838829+0000, chain.objs.size()=1 2024-04-22T20:48:16.565 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ym2xg0SgeD3B1BxATnpzI7x4HvKkFg4_0 2024-04-22T20:48:16.565 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10850373231603940362', time=2024-04-22T22:26:30.886664+0000, chain.objs.size()=1 2024-04-22T20:48:16.565 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.565+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PwRc4sjWLAYlBq4LR68m0mlUsmYaPzU_0 2024-04-22T20:48:16.565 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.567+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15135618301008491831', time=2024-04-22T22:26:31.156285+0000, chain.objs.size()=1 2024-04-22T20:48:16.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.567+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cMnTE_AFRny0_Ifux30_SuoG4QEQx5p_0 2024-04-22T20:48:16.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.567+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9499301428002166482', time=2024-04-22T22:26:31.266472+0000, chain.objs.size()=1 2024-04-22T20:48:16.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.567+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gPAMtV3cn-zh637VEZemVaairh3jE2s_0 2024-04-22T20:48:16.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.567+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10493553109805916861', time=2024-04-22T22:26:31.340887+0000, chain.objs.size()=1 2024-04-22T20:48:16.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.567+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ovxmgzqfprPJMFAPf3UwojIQHoH0cWm_0 2024-04-22T20:48:16.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.568+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4796227703151526004', time=2024-04-22T22:26:31.541729+0000, chain.objs.size()=1 2024-04-22T20:48:16.566 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.568+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3By9Qj-3GhQvgjO4jFq0GFZVd0odTuK_0 2024-04-22T20:48:16.567 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.568+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6414185837917432102', time=2024-04-22T22:26:32.027545+0000, chain.objs.size()=1 2024-04-22T20:48:16.567 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.568+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Nl51Iz6mQGVOhyMp5Uokr-w1Gbi2gT3_0 2024-04-22T20:48:16.567 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.568+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10752220348993268894', time=2024-04-22T22:26:32.525830+0000, chain.objs.size()=1 2024-04-22T20:48:16.567 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.568+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sL5i92rHIjLGTkkB_mjqejmOWHhp0DZ_0 2024-04-22T20:48:16.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6899505334439353159', time=2024-04-22T22:26:32.605001+0000, chain.objs.size()=1 2024-04-22T20:48:16.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.H70s6jRjzpcU7eD92gcE2_NpTQsAtcr_0 2024-04-22T20:48:16.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17408486130450572216', time=2024-04-22T22:26:32.783698+0000, chain.objs.size()=1 2024-04-22T20:48:16.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Zsk9ZyuLpTKrTitEA977oCQs18MHLG4_0 2024-04-22T20:48:16.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11427161537051171573', time=2024-04-22T22:26:32.857766+0000, chain.objs.size()=1 2024-04-22T20:48:16.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YBUc55FasaYDk2w97fKQc82eJZI7QPq_0 2024-04-22T20:48:16.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16662062894414651785', time=2024-04-22T22:26:33.023236+0000, chain.objs.size()=1 2024-04-22T20:48:16.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2L7kiJSpozLniy-gblaBG38Sq4LA0KL_0 2024-04-22T20:48:16.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5573883045455036970', time=2024-04-22T22:26:33.547456+0000, chain.objs.size()=1 2024-04-22T20:48:16.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cC6HVqSSnQ7UeIsxryXDF9wKUq8QNFA_0 2024-04-22T20:48:16.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1609873261567472025', time=2024-04-22T22:26:40.198297+0000, chain.objs.size()=1 2024-04-22T20:48:16.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.257__shadow_.Ng2r2NGHZbpe9qZKh1nXItuMyaXVpX5_0 2024-04-22T20:48:16.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14436499584326531921', time=2024-04-22T22:26:41.155625+0000, chain.objs.size()=1 2024-04-22T20:48:16.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.258__shadow_.lKHlkjqC14q1_QJcnBZviNdKFPvi1aT_0 2024-04-22T20:48:16.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6916691183560645003', time=2024-04-22T22:26:45.899278+0000, chain.objs.size()=1 2024-04-22T20:48:16.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.570+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.263__shadow_.QcsI6svHgsTNsA1NeMARSOesPuHckag_0 2024-04-22T20:48:16.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.572+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12517080047566009910', time=2024-04-22T22:27:05.635358+0000, chain.objs.size()=1 2024-04-22T20:48:16.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.572+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.i4-TkISi0dv5BtxyNWSRpuL9SI1uAYK_0 2024-04-22T20:48:16.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.572+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7223266409056788246', time=2024-04-22T22:27:07.952726+0000, chain.objs.size()=1 2024-04-22T20:48:16.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.572+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.7lE1tNTNr95sOP8NDfIS4nE52IeAAe__0 2024-04-22T20:48:16.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.573+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1347738346023402958', time=2024-04-22T22:27:11.990989+0000, chain.objs.size()=1 2024-04-22T20:48:16.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.573+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.286__shadow_.jFwUddqN0A06RTsqKRoIT7RIHL0ikTd_0 2024-04-22T20:48:16.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.573+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16351920569733335773', time=2024-04-22T22:27:16.063277+0000, chain.objs.size()=1 2024-04-22T20:48:16.572 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.573+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.290__shadow_.HFjZg5VZwE_4uAmIg7DgCcFtm1Rduo4_0 2024-04-22T20:48:16.572 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.574+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.759787543675833095', time=2024-04-22T22:27:19.246526+0000, chain.objs.size()=1 2024-04-22T20:48:16.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.574+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.293__shadow_.ZxMH8gH2yQCbjfcKToy-Qb2q6_zjfz-_0 2024-04-22T20:48:16.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12611960620526209178', time=2024-04-22T22:28:17.369642+0000, chain.objs.size()=1 2024-04-22T20:48:16.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NCK3yaMunr7ZYebXh4quAwObcVLg7EW_0 2024-04-22T20:48:16.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1888984790968662628', time=2024-04-22T22:28:17.521893+0000, chain.objs.size()=1 2024-04-22T20:48:16.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zmQwuwkBLg3Bs7sz9hFyQtAoFbx17rC_0 2024-04-22T20:48:16.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7205620329113397166', time=2024-04-22T22:28:17.612803+0000, chain.objs.size()=1 2024-04-22T20:48:16.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qqo702KwDxS9OkhPARKOVdKF6p9wsmU_0 2024-04-22T20:48:16.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13025346082502269120', time=2024-04-22T22:28:17.712819+0000, chain.objs.size()=1 2024-04-22T20:48:16.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yTNULLcp2CNLnZn7srd2Fnxn0rIoOJ7_0 2024-04-22T20:48:16.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18191240912297576319', time=2024-04-22T22:28:18.273753+0000, chain.objs.size()=1 2024-04-22T20:48:16.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.j93iqGZRFg1uedY9dMeq8EZ8f9MdbUL_0 2024-04-22T20:48:16.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16956023474614068740', time=2024-04-22T22:28:18.533008+0000, chain.objs.size()=1 2024-04-22T20:48:16.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.eENOzuz8q-_H7-sdJ5WZZpKbnYMuWZO_0 2024-04-22T20:48:16.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11703299203529862004', time=2024-04-22T22:28:19.195599+0000, chain.objs.size()=1 2024-04-22T20:48:16.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.575+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Gf7iNuwpUrzKecuREgF7wx2bImOotkY_0 2024-04-22T20:48:16.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.577+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12350987541382305295', time=2024-04-22T22:28:19.439919+0000, chain.objs.size()=1 2024-04-22T20:48:16.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.577+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rgcX0rFSv-bB_8drjkYhlsJ7HaGDU3V_0 2024-04-22T20:48:16.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.578+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10747948979182001616', time=2024-04-22T22:28:19.595581+0000, chain.objs.size()=1 2024-04-22T20:48:16.575 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.578+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4kDEVY_mDoMx6chQCPKnLxqt_Bj8Hwl_0 2024-04-22T20:48:16.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.578+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10902296085974874818', time=2024-04-22T22:28:19.834742+0000, chain.objs.size()=1 2024-04-22T20:48:16.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.578+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.41hbNOjjnhhBlUUsO-E0VY34rUnFfrD_0 2024-04-22T20:48:16.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13354362357018570693', time=2024-04-22T22:28:20.685549+0000, chain.objs.size()=1 2024-04-22T20:48:16.576 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rv2quwNZrj1-z730y1aT0a-LLOv71oP_0 2024-04-22T20:48:16.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13508972691531986723', time=2024-04-22T22:28:21.157721+0000, chain.objs.size()=1 2024-04-22T20:48:16.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NrFZNpIUrYcW854dMGJBR6XQLQ2fhsF_0 2024-04-22T20:48:16.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2830036822007046565', time=2024-04-22T22:28:21.188099+0000, chain.objs.size()=1 2024-04-22T20:48:16.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wArgMv7qd6bjO1jxODB-H3BrCEtiyJQ_0 2024-04-22T20:48:16.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13125526680138290343', time=2024-04-22T22:28:21.224201+0000, chain.objs.size()=1 2024-04-22T20:48:16.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.QRmUcFjQIdFwZpgq_iJKMzuTJvkOXof_0 2024-04-22T20:48:16.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8838556697567263025', time=2024-04-22T22:28:21.865741+0000, chain.objs.size()=1 2024-04-22T20:48:16.577 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.G5ja-nNPXWr97L2SqCSTgPl01mMNGLj_0 2024-04-22T20:48:16.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15818301342017454087', time=2024-04-22T22:28:22.476069+0000, chain.objs.size()=1 2024-04-22T20:48:16.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.579+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Gl4FI0xr50j0hXvP_jgClqTzMPWaWkQ_0 2024-04-22T20:48:16.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.581+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17987126019378482675', time=2024-04-22T22:28:22.500146+0000, chain.objs.size()=1 2024-04-22T20:48:16.578 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.581+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.c5au4clSR_GnMh6Ij7jm6deNk8aCLag_0 2024-04-22T20:48:16.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.581+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11775981633393659031', time=2024-04-22T22:28:22.501665+0000, chain.objs.size()=1 2024-04-22T20:48:16.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.581+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cui907zZjieeOefGNjjqxXPCr0q1f5m_0 2024-04-22T20:48:16.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.581+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16649789759672481534', time=2024-04-22T22:28:22.645926+0000, chain.objs.size()=1 2024-04-22T20:48:16.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.581+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._AHZTDJZTV2lGLvRjdE0pxrqwBjFqt8_0 2024-04-22T20:48:16.579 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.582+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14401925827766325463', time=2024-04-22T22:28:22.767110+0000, chain.objs.size()=1 2024-04-22T20:48:16.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.582+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wr_5QuYog-8A9bLiB1LIgEmA0mu27Vt_0 2024-04-22T20:48:16.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.582+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3118080959000562027', time=2024-04-22T22:28:22.891018+0000, chain.objs.size()=1 2024-04-22T20:48:16.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.582+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.i2QgRNnEvtvQEbUbId8EqFcB4P21YLz_0 2024-04-22T20:48:16.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.582+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9123603963318424873', time=2024-04-22T22:28:23.448328+0000, chain.objs.size()=1 2024-04-22T20:48:16.580 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.582+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.I0qMBLlnAtNMbtN4CezlWYC9IrDpFd7_0 2024-04-22T20:48:16.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.584+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17638102788820628688', time=2024-04-22T22:28:24.088854+0000, chain.objs.size()=1 2024-04-22T20:48:16.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.584+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ELVEevgxfiuw-dUMnUJgqEjqB5GodSp_0 2024-04-22T20:48:16.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.584+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9870642788949233179', time=2024-04-22T22:28:24.313525+0000, chain.objs.size()=1 2024-04-22T20:48:16.581 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.584+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.k1cD9wgbIwDzje7A8FlgnXD-IJM-YRR_0 2024-04-22T20:48:16.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8530873348952301840', time=2024-04-22T22:28:24.314163+0000, chain.objs.size()=1 2024-04-22T20:48:16.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lmB5uQY6vB-egD1N0kPCba8JnEuBWD3_0 2024-04-22T20:48:16.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1828104892597748626', time=2024-04-22T22:28:24.451013+0000, chain.objs.size()=1 2024-04-22T20:48:16.583 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0P2KI8h_8KbpCPeJYq-qqdnJAcen47f_0 2024-04-22T20:48:16.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18073450121598234099', time=2024-04-22T22:28:24.491873+0000, chain.objs.size()=1 2024-04-22T20:48:16.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3NEJiaYYytNjpPw-cDFPBpIZRJBsF_J_0 2024-04-22T20:48:16.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12847101887021995504', time=2024-04-22T22:28:24.619085+0000, chain.objs.size()=1 2024-04-22T20:48:16.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.x9sT1_OwYy1NhQ3aJIudEwsd2oFX-EH_0 2024-04-22T20:48:16.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12771122530710033538', time=2024-04-22T22:28:24.880606+0000, chain.objs.size()=1 2024-04-22T20:48:16.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ydXkfrvOhYMvKLqjryS91kxTSrYK8pN_0 2024-04-22T20:48:16.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4307821389478244268', time=2024-04-22T22:28:25.409360+0000, chain.objs.size()=1 2024-04-22T20:48:16.584 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CyS4pZyQZQO8d72DHKi37G8nxh9TRRv_0 2024-04-22T20:48:16.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18103535814813946508', time=2024-04-22T22:28:25.576469+0000, chain.objs.size()=1 2024-04-22T20:48:16.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.AR6JT7zgsR7Ck0B7JIWRp1lsawZ_prV_0 2024-04-22T20:48:16.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15480427094139588023', time=2024-04-22T22:28:25.954867+0000, chain.objs.size()=1 2024-04-22T20:48:16.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.hct81Gy-tvBW1CpXDQfaNTD42n03ZAj_0 2024-04-22T20:48:16.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2037388889975047533', time=2024-04-22T22:28:26.426689+0000, chain.objs.size()=1 2024-04-22T20:48:16.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.efNW0r6VC1jj0xR080BmM1Wx00dEF1w_0 2024-04-22T20:48:16.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.586+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6128099748887070622', time=2024-04-22T22:28:26.562352+0000, chain.objs.size()=1 2024-04-22T20:48:16.585 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.587+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3hFkum_JaEBQpnAsZ6XMQNghjtmp086_0 2024-04-22T20:48:16.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.589+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9342585546604465812', time=2024-04-22T22:28:26.920934+0000, chain.objs.size()=1 2024-04-22T20:48:16.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.589+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ktC7YlkhkFXsDitVvb4tHUOv_fQPisB_0 2024-04-22T20:48:16.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.590+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12709930177465692589', time=2024-04-22T22:28:27.310608+0000, chain.objs.size()=1 2024-04-22T20:48:16.587 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.590+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TJGuulkrRqVKTIkUkt-8WmBNxct1Gq4_0 2024-04-22T20:48:16.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.592+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18282807389543488913', time=2024-04-22T22:28:27.311388+0000, chain.objs.size()=1 2024-04-22T20:48:16.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.592+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._PxCg6lp5QiRPE47RQuOi3Cc2AJ4Oq2_0 2024-04-22T20:48:16.589 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.592+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1257914455872694403', time=2024-04-22T22:28:27.651063+0000, chain.objs.size()=1 2024-04-22T20:48:16.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.592+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6VMh86g2eq7RfbhBzA8NMaBUGE_U1k2_0 2024-04-22T20:48:16.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.592+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14748044151641241914', time=2024-04-22T22:28:27.992564+0000, chain.objs.size()=1 2024-04-22T20:48:16.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.592+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.n9OZrCquILDzEC5xYP4B7e1dWGGtDv1_0 2024-04-22T20:48:16.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.592+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12157297047310303332', time=2024-04-22T22:28:28.051006+0000, chain.objs.size()=1 2024-04-22T20:48:16.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.592+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dVWh7gpubPxB-u3owSTpVLIKtPRS_Qw_0 2024-04-22T20:48:16.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16496886956456841159', time=2024-04-22T22:28:28.063415+0000, chain.objs.size()=1 2024-04-22T20:48:16.590 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vS7SYWpwmd8COqSL7342qh4iRF9cXl__0 2024-04-22T20:48:16.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1136164602057963789', time=2024-04-22T22:28:28.450778+0000, chain.objs.size()=1 2024-04-22T20:48:16.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JIeDfuC46W_2MnTlIjWzjmrhLqvOJ8U_0 2024-04-22T20:48:16.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11166244411031570605', time=2024-04-22T22:28:28.493718+0000, chain.objs.size()=1 2024-04-22T20:48:16.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Mm7n0_VxMor9n7hsMUe1E84G5fEB4FR_0 2024-04-22T20:48:16.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5987052793158076709', time=2024-04-22T22:28:28.647815+0000, chain.objs.size()=1 2024-04-22T20:48:16.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-SkS3JvI32bTGs_chnfehPat2wW6-8M_0 2024-04-22T20:48:16.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6122936820640168187', time=2024-04-22T22:28:29.103672+0000, chain.objs.size()=1 2024-04-22T20:48:16.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.593+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RgPNHskjkxaw4fpHXwHKR3oMpXDbPFL_0 2024-04-22T20:48:16.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.595+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2930381039850099117', time=2024-04-22T22:28:29.218477+0000, chain.objs.size()=1 2024-04-22T20:48:16.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.595+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fIHNmub49ms8blm4YRYnmUD4fru3ILe_0 2024-04-22T20:48:16.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.595+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9606956538405336959', time=2024-04-22T22:29:06.209363+0000, chain.objs.size()=1 2024-04-22T20:48:16.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.595+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wnxzqGebQOhpFZtFgJziFjnorqiyNJh_0 2024-04-22T20:48:16.594 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13649404306776145310', time=2024-04-22T22:29:07.152672+0000, chain.objs.size()=1 2024-04-22T20:48:16.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.K8FP-rLhDTFTHYKxNDzBqFmJwhOBoRK_0 2024-04-22T20:48:16.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11858611394878880177', time=2024-04-22T22:29:07.809395+0000, chain.objs.size()=1 2024-04-22T20:48:16.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qlV7bPHQ78RwiKoU_Gk5HPi3BSIeQ-P_0 2024-04-22T20:48:16.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17116541039556442893', time=2024-04-22T22:29:08.324453+0000, chain.objs.size()=1 2024-04-22T20:48:16.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.EXCZ9fzgEI3nsd5XZ6GoCVZ1w1VVL1l_0 2024-04-22T20:48:16.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9611367345421051139', time=2024-04-22T22:29:08.431368+0000, chain.objs.size()=1 2024-04-22T20:48:16.595 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xJVUm1-CqHHHZ-W_wkiy5QuD_21D1ZW_0 2024-04-22T20:48:16.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5111441352300421283', time=2024-04-22T22:29:08.538676+0000, chain.objs.size()=1 2024-04-22T20:48:16.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1SE-c_NoZbXenEmjRJL7u62T-K0wPqe_0 2024-04-22T20:48:16.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16378623638347789019', time=2024-04-22T22:29:08.603302+0000, chain.objs.size()=1 2024-04-22T20:48:16.596 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.598+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DLbLty5FNoGZ8-5kolDrGSHpC1i-4g5_0 2024-04-22T20:48:16.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.599+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14271543632020142515', time=2024-04-22T22:29:08.764346+0000, chain.objs.size()=1 2024-04-22T20:48:16.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.599+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JlAh6TSKT03ZVAiRmPnjCqBvCxTE0-c_0 2024-04-22T20:48:16.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.599+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9017225342217087991', time=2024-04-22T22:29:09.888109+0000, chain.objs.size()=1 2024-04-22T20:48:16.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.599+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HTg1kpXx-HOTVjGWWaihsC4hHvb6V8N_0 2024-04-22T20:48:16.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.599+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12860468288474866649', time=2024-04-22T22:29:10.064894+0000, chain.objs.size()=1 2024-04-22T20:48:16.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.599+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zWGPVPVLbl0JpAjuWFx9ikSXCan9gp4_0 2024-04-22T20:48:16.597 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.599+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11565221865114972682', time=2024-04-22T22:29:10.455752+0000, chain.objs.size()=1 2024-04-22T20:48:16.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.599+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.AnEK0pfqfvgGyExgSUkRaqRWpY_2uRo_0 2024-04-22T20:48:16.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.600+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17472072326072511696', time=2024-04-22T22:29:10.967088+0000, chain.objs.size()=1 2024-04-22T20:48:16.598 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.600+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2vGWZElpbVaZyf9vZg0iF07V0SqW4Tt_0 2024-04-22T20:48:16.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.603+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7354155167947360711', time=2024-04-22T22:29:11.105737+0000, chain.objs.size()=1 2024-04-22T20:48:16.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.603+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.5oZzx7QOvknrmlccxb431lDog0XK8VN_0 2024-04-22T20:48:16.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.603+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2599451986680192802', time=2024-04-22T22:29:11.221946+0000, chain.objs.size()=1 2024-04-22T20:48:16.600 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.603+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0ardwvxOtrl1urKs1Er6MeFZM55V3qY_0 2024-04-22T20:48:16.606 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.609+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/50416 2024-04-22T20:48:16.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=31, truncated=0, next_marker='' 2024-04-22T20:48:16.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17807785039581799763', time=2024-04-22T22:29:11.823614+0000, chain.objs.size()=1 2024-04-22T20:48:16.609 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.W0S48fNBW77u8-ZQnNbmdLc1pDDPKMG_0 2024-04-22T20:48:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10105355504549602925', time=2024-04-22T22:29:11.951763+0000, chain.objs.size()=1 2024-04-22T20:48:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.68X6uJA_TSc3M4sIyfYXtCgL8kMEowI_0 2024-04-22T20:48:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8170169642237311536', time=2024-04-22T22:29:12.650932+0000, chain.objs.size()=1 2024-04-22T20:48:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.unxhPpOXueko-IZa__FDWlAbdWjaRrE_0 2024-04-22T20:48:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3451748616418325558', time=2024-04-22T22:29:12.861146+0000, chain.objs.size()=1 2024-04-22T20:48:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JNWdwkxscJuoxbbECuOar4CgOtuKlc9_0 2024-04-22T20:48:16.610 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17227055007790337227', time=2024-04-22T22:29:12.914724+0000, chain.objs.size()=1 2024-04-22T20:48:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.i0ax27YOPyxnjA0HitMuMAsPR_8jFEV_0 2024-04-22T20:48:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15817003796473678827', time=2024-04-22T22:29:13.052558+0000, chain.objs.size()=1 2024-04-22T20:48:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iABmvw4yalnIu-HNJxmT1IQEhRCsI98_0 2024-04-22T20:48:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16519652417632948740', time=2024-04-22T22:29:13.924860+0000, chain.objs.size()=1 2024-04-22T20:48:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.5f1pRPYiq1lnHVBAD0Ee2FrZyYV5iCa_0 2024-04-22T20:48:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8665744318865628780', time=2024-04-22T22:29:14.064059+0000, chain.objs.size()=1 2024-04-22T20:48:16.611 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZyqSuMUhJl6-7AbefL5VBpeCiSuNqKO_0 2024-04-22T20:48:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7592975102719371321', time=2024-04-22T22:29:14.802920+0000, chain.objs.size()=1 2024-04-22T20:48:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.lk8L3osEfm2MuqSgW7XQviAXnCivLKz_0 2024-04-22T20:48:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1072812793225466541', time=2024-04-22T22:29:14.883106+0000, chain.objs.size()=1 2024-04-22T20:48:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oXDrf5FAOGE2nGF4eU1WR8Fk1J8LaAK_0 2024-04-22T20:48:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18068435053052538426', time=2024-04-22T22:29:14.984920+0000, chain.objs.size()=1 2024-04-22T20:48:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dzbktu1fBn6AdK6rz3FIaz5xMMptnzs_0 2024-04-22T20:48:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16931960371256648708', time=2024-04-22T22:29:15.816492+0000, chain.objs.size()=1 2024-04-22T20:48:16.612 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.612+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YikW_JOOrzU_9CxS9DUYX7w7J5MC2H6_0 2024-04-22T20:48:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.615+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9033558560643473015', time=2024-04-22T22:29:16.248246+0000, chain.objs.size()=1 2024-04-22T20:48:16.613 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.615+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._1g44m3tFY3scYB18ur-xuAHBExiYbz_0 2024-04-22T20:48:16.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17039923340882276712', time=2024-04-22T22:29:17.071109+0000, chain.objs.size()=1 2024-04-22T20:48:16.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gqXY61BVjqyIGieyde2Pb0iXa3qBx-k_0 2024-04-22T20:48:16.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5955597980355974197', time=2024-04-22T22:29:17.107494+0000, chain.objs.size()=1 2024-04-22T20:48:16.614 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.A9ip6MwktuuJ7VBhjxnTYHXRN4AmNhy_0 2024-04-22T20:48:16.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12387369864841616958', time=2024-04-22T22:29:17.108289+0000, chain.objs.size()=1 2024-04-22T20:48:16.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RPXi3ggXRoRYST2c56Z6RlbE_HH_oHD_0 2024-04-22T20:48:16.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8954467212032356658', time=2024-04-22T22:29:17.446290+0000, chain.objs.size()=1 2024-04-22T20:48:16.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XrqNIFvEUPNuCq3iM-N54dCfhklUwFK_0 2024-04-22T20:48:16.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5490382655073612411', time=2024-04-22T22:29:51.221333+0000, chain.objs.size()=1 2024-04-22T20:48:16.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.363__shadow_.aMdC18GRvPzj6TPvvHveyorSg0hcB_g_0 2024-04-22T20:48:16.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17969427701486572370', time=2024-04-22T22:30:56.165297+0000, chain.objs.size()=1 2024-04-22T20:48:16.615 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.450__shadow_.kA2Sg-t5hnvXuWm4IXenb2a1bGipIeg_0 2024-04-22T20:48:16.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14172080400467995001', time=2024-04-22T22:31:11.415341+0000, chain.objs.size()=1 2024-04-22T20:48:16.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.473__shadow_.KEqAvYCMQmQ-YmnALIJ2Szwaf8ISRpc_0 2024-04-22T20:48:16.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11229321498721100569', time=2024-04-22T22:34:55.097525+0000, chain.objs.size()=3 2024-04-22T20:48:16.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~UKZrbHIDgp8zasi3yaScRclBN21sBcS.1 2024-04-22T20:48:16.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__shadow_mymultipart.2~UKZrbHIDgp8zasi3yaScRclBN21sBcS.1_1 2024-04-22T20:48:16.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~UKZrbHIDgp8zasi3yaScRclBN21sBcS.2 2024-04-22T20:48:16.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8861158837038902244', time=2024-04-22T22:36:31.299847+0000, chain.objs.size()=1 2024-04-22T20:48:16.616 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.617+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.538__shadow_.VznV4vU3WpnLXjI4oZeWPqm2d7PSMEp_0 2024-04-22T20:48:16.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14184761899601340438', time=2024-04-22T22:36:42.733248+0000, chain.objs.size()=1 2024-04-22T20:48:16.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.541__shadow_.OOgxtwH7thh_dR70A9cFAmFJlchmhTu_0 2024-04-22T20:48:16.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12172359795878607714', time=2024-04-22T22:37:22.446465+0000, chain.objs.size()=1 2024-04-22T20:48:16.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:MJUV5uhSzh.7k40HO88S53e.BRS.YzG_.E-ykoAoKvJ8faZnFA26ukNhl9mWwL8n_0 2024-04-22T20:48:16.620 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4559085827381090882', time=2024-04-22T22:37:27.884310+0000, chain.objs.size()=1 2024-04-22T20:48:16.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:VRFW-tRc5sbcJrlxF8Bwn3.SVrYpzng_.4evbjOnbK9rWI1fZpM7EfFGaBaghJ7a_0 2024-04-22T20:48:16.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16285635728404451242', time=2024-04-22T22:37:43.197754+0000, chain.objs.size()=1 2024-04-22T20:48:16.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.563__shadow_.uXfK6eZrss11FPud7BnQMsQ5t9xXxH__0 2024-04-22T20:48:16.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4192815425826509636', time=2024-04-22T22:38:07.063331+0000, chain.objs.size()=1 2024-04-22T20:48:16.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:NnttkDZlo3uT51VururxhfFPG-B77dv_.0Jn14egIZ-EJIL7P8ME8L4xeA7H8-cx_0 2024-04-22T20:48:16.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15733291559467533112', time=2024-04-22T22:38:09.961100+0000, chain.objs.size()=1 2024-04-22T20:48:16.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.623+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:3DypBFdwVOoVhQ67Nz0a7JtHXVt0Xaq_.XRMk2ErgejJL1o40tWsN30v12JOS3PT_0 2024-04-22T20:48:16.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7227555935984402445', time=2024-04-22T22:38:51.990773+0000, chain.objs.size()=1 2024-04-22T20:48:16.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:qVjJ5iTGVxG9xodzTPYYDMlFBVXKTEK_.IzdXg9s7RTEeaFnlFpPPoAEZ3j1jBIA_0 2024-04-22T20:48:16.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9743022444143311172', time=2024-04-22T22:39:10.005153+0000, chain.objs.size()=1 2024-04-22T20:48:16.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.592__shadow_.p7333vLDklft2id2yXSHuGmXg6jIkDr_0 2024-04-22T20:48:16.622 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.624+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10091858240861627319', time=2024-04-22T22:44:09.550069+0000, chain.objs.size()=1 2024-04-22T20:48:16.623 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.624+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.721__shadow_.S8m1pJHk3cKln8wBykIMzmmtgAQW2fB_0 2024-04-22T20:48:16.627 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.631+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.630 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.633+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-22T20:48:16.631 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.635+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.635+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/54094' 2024-04-22T20:48:16.633 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_X8QlVmqXyA7X2Eq30yqQbwUZHhMNhAj', time=2024-04-22T22:22:16.237845+0000, chain.objs.size()=3 2024-04-22T20:48:16.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.ksoXJS8Vc6DLsenxC3zi8moAwEyxLm-_0 2024-04-22T20:48:16.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.ksoXJS8Vc6DLsenxC3zi8moAwEyxLm-_1 2024-04-22T20:48:16.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.ksoXJS8Vc6DLsenxC3zi8moAwEyxLm-_2 2024-04-22T20:48:16.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9718282298318040218', time=2024-04-22T22:25:38.808647+0000, chain.objs.size()=1 2024-04-22T20:48:16.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.243__shadow_.R4P1XBTdksC_vi0Yz1gTY9UgXFBJmbW_0 2024-04-22T20:48:16.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13681077017859211920', time=2024-04-22T22:26:22.473315+0000, chain.objs.size()=1 2024-04-22T20:48:16.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4eCVtuFYZmZ_Q3F5MO6pJ6s2whJnyWj_0 2024-04-22T20:48:16.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14012888966447942797', time=2024-04-22T22:26:22.851897+0000, chain.objs.size()=1 2024-04-22T20:48:16.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wgBoI13tKyH6zuPRcmS_UdtLsaADc3M_0 2024-04-22T20:48:16.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11814007372377255193', time=2024-04-22T22:26:22.916138+0000, chain.objs.size()=1 2024-04-22T20:48:16.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6l400VcaeCxAb3-eOhbeIO9sYiYuOtB_0 2024-04-22T20:48:16.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13287472335029493979', time=2024-04-22T22:26:23.434063+0000, chain.objs.size()=1 2024-04-22T20:48:16.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ba_VDCA7VzeaW6bb8uQNjUAJXkaiho-_0 2024-04-22T20:48:16.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16345107248733193848', time=2024-04-22T22:26:23.462862+0000, chain.objs.size()=1 2024-04-22T20:48:16.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4fr7d1z2BhqJpqOFklHmTXykybriPVm_0 2024-04-22T20:48:16.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7404590856779504107', time=2024-04-22T22:26:23.493839+0000, chain.objs.size()=1 2024-04-22T20:48:16.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UVLm0EpXFcGCiTBdnM8PACcGikVKNco_0 2024-04-22T20:48:16.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13951794451176740075', time=2024-04-22T22:26:24.139971+0000, chain.objs.size()=1 2024-04-22T20:48:16.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3uoRIR5Z2MX5MNRurnUK2L3BSVM6p7V_0 2024-04-22T20:48:16.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12647112211475839317', time=2024-04-22T22:26:24.534838+0000, chain.objs.size()=1 2024-04-22T20:48:16.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.636+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mJ4aU3oN04b-LDEVh6Rz1VCwIP5QHld_0 2024-04-22T20:48:16.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.641+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11431561698907123426', time=2024-04-22T22:26:24.615042+0000, chain.objs.size()=1 2024-04-22T20:48:16.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.641+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OkttyO3vZ0bbk3adyvlZNADNTd5U8Z4_0 2024-04-22T20:48:16.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.641+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14576520451454029298', time=2024-04-22T22:26:24.615528+0000, chain.objs.size()=1 2024-04-22T20:48:16.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.641+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._YnI8DQkTvcDKk69RdM7pvDgb6RfGBJ_0 2024-04-22T20:48:16.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.641+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9567843781937391745', time=2024-04-22T22:26:24.798271+0000, chain.objs.size()=1 2024-04-22T20:48:16.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.641+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VxOsZbeEbDWvsFLo5ktozxL9YhOJDBR_0 2024-04-22T20:48:16.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.642+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5187052414432160590', time=2024-04-22T22:26:24.857297+0000, chain.objs.size()=1 2024-04-22T20:48:16.639 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.642+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.I0znekOakio9dyUeQS1Zzmw0O02fWfr_0 2024-04-22T20:48:16.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.642+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17581842065139382550', time=2024-04-22T22:26:24.861407+0000, chain.objs.size()=1 2024-04-22T20:48:16.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.642+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XtnwZ1qQP2c33p61XOF-VqaOQRfmXK6_0 2024-04-22T20:48:16.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.642+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9809601948304369545', time=2024-04-22T22:26:25.828316+0000, chain.objs.size()=1 2024-04-22T20:48:16.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.642+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hwjOXg5U-b3PKVvHi2Fsiyk4XFexLaz_0 2024-04-22T20:48:16.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.642+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15297286674936205013', time=2024-04-22T22:26:25.829487+0000, chain.objs.size()=1 2024-04-22T20:48:16.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.642+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.T1w1iBilk8UDeJbtEZzAj4wMX27bJu3_0 2024-04-22T20:48:16.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.644+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5071690528350329838', time=2024-04-22T22:26:25.975239+0000, chain.objs.size()=1 2024-04-22T20:48:16.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.644+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.a9j2isVpSVKjSLZGkfJJ3EssNqGjRFJ_0 2024-04-22T20:48:16.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.644+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10354252969600502138', time=2024-04-22T22:26:26.124083+0000, chain.objs.size()=1 2024-04-22T20:48:16.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.644+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gFDesdDEWD9xuIheurLL42-miE4QLna_0 2024-04-22T20:48:16.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.644+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1090175108901499560', time=2024-04-22T22:26:26.304187+0000, chain.objs.size()=1 2024-04-22T20:48:16.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.644+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.iYdmuvLEhL-A9IdoByuPcxziX27qGGP_0 2024-04-22T20:48:16.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.644+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9957036175988032', time=2024-04-22T22:26:26.304822+0000, chain.objs.size()=1 2024-04-22T20:48:16.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.644+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VmT3qINLFMNmw_OiJmwtVEIpca_wRan_0 2024-04-22T20:48:16.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15116454262517297318', time=2024-04-22T22:26:26.793889+0000, chain.objs.size()=1 2024-04-22T20:48:16.644 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8zPgYZ8bJlpo-TTTv4DVgXQrMeemhC4_0 2024-04-22T20:48:16.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1089113670295029385', time=2024-04-22T22:26:26.827739+0000, chain.objs.size()=1 2024-04-22T20:48:16.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OFLP2vCbW2_oSAygpsFp_pS9jmYbs8W_0 2024-04-22T20:48:16.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11450295468502522153', time=2024-04-22T22:26:27.760856+0000, chain.objs.size()=1 2024-04-22T20:48:16.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.IJIXMAzN-gFUE_ZeS6MCFeFz1Sc28tC_0 2024-04-22T20:48:16.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13271379323118795623', time=2024-04-22T22:26:28.605350+0000, chain.objs.size()=1 2024-04-22T20:48:16.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.s85tPV7ZNetm11EfWTJFYN2E15sl-55_0 2024-04-22T20:48:16.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4287309221070087274', time=2024-04-22T22:26:29.494422+0000, chain.objs.size()=1 2024-04-22T20:48:16.645 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VKTieUThXoJ3dHg1xh-FqayDHup5f4S_0 2024-04-22T20:48:16.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11652477707651318540', time=2024-04-22T22:26:29.832334+0000, chain.objs.size()=1 2024-04-22T20:48:16.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.647+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Wb78HIyLB1gkKylyoOVlGkAzFB2StR6_0 2024-04-22T20:48:16.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.648+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15578223732195078615', time=2024-04-22T22:26:30.125176+0000, chain.objs.size()=1 2024-04-22T20:48:16.646 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.648+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4OAfQAPYUj3mH-iz1V8EXXLpJ0__APv_0 2024-04-22T20:48:16.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.650+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5128288946477827032', time=2024-04-22T22:26:30.543340+0000, chain.objs.size()=1 2024-04-22T20:48:16.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.650+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0vR68RwRpBvUU4itAcJPMBwM0K6QYJC_0 2024-04-22T20:48:16.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.650+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14789721536902845455', time=2024-04-22T22:26:30.728935+0000, chain.objs.size()=1 2024-04-22T20:48:16.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.650+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Y6fQCqFsvSgWjsBg4ajTsjcHcag6ub6_0 2024-04-22T20:48:16.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.650+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16407613477114771475', time=2024-04-22T22:26:31.405776+0000, chain.objs.size()=1 2024-04-22T20:48:16.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.650+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.X-tSEt_PWr10HaTcHfrHTFVqzcBZdXI_0 2024-04-22T20:48:16.647 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.650+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6138611606837611181', time=2024-04-22T22:26:31.449527+0000, chain.objs.size()=1 2024-04-22T20:48:16.648 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.650+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.T4RWgvCSt3Nxzz7AIEvJ1A0VHVnJdVh_0 2024-04-22T20:48:16.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.652+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5590904763264189671', time=2024-04-22T22:26:31.685638+0000, chain.objs.size()=1 2024-04-22T20:48:16.649 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.652+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.b74wiLEgHJLwbPrkODFByALS9n02CWE_0 2024-04-22T20:48:16.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11433292562797337732', time=2024-04-22T22:26:32.064646+0000, chain.objs.size()=1 2024-04-22T20:48:16.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.i5UkWS8Dla7TIE4VkVFhVA3CJbpsXOs_0 2024-04-22T20:48:16.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3597427009064573965', time=2024-04-22T22:26:33.594805+0000, chain.objs.size()=1 2024-04-22T20:48:16.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.e7Apnm4HP1cemFr3GQ0RyBtxW_KRv5m_0 2024-04-22T20:48:16.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1840437306942440295', time=2024-04-22T22:26:33.669375+0000, chain.objs.size()=1 2024-04-22T20:48:16.652 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ORxXTqHlS6a6gdPBkVNHyMwnyHeB6V0_0 2024-04-22T20:48:16.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16968169073599336227', time=2024-04-22T22:26:33.671593+0000, chain.objs.size()=1 2024-04-22T20:48:16.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.capW_pkorDC0XOg2_tVnzD5EmhivODJ_0 2024-04-22T20:48:16.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6279633758936372944', time=2024-04-22T22:26:33.733953+0000, chain.objs.size()=1 2024-04-22T20:48:16.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OknM2G0F6WEGNyAgWrhcogM3IgJwmEg_0 2024-04-22T20:48:16.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10382556791495223779', time=2024-04-22T22:26:34.405290+0000, chain.objs.size()=1 2024-04-22T20:48:16.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lSqK9zerpR9Tfe9LbOxbLohVPndEffl_0 2024-04-22T20:48:16.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8329023313091098188', time=2024-04-22T22:26:44.026392+0000, chain.objs.size()=1 2024-04-22T20:48:16.653 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.261__shadow_.ir2pfV8JPCg0XNOaelmOYbwSqXGzBIU_0 2024-04-22T20:48:16.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17919020731177986943', time=2024-04-22T22:26:46.903791+0000, chain.objs.size()=1 2024-04-22T20:48:16.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.264__shadow_.kveuI4zkVv-viDtzA3LvJZ6gXryHjyU_0 2024-04-22T20:48:16.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15299109591433337164', time=2024-04-22T22:26:52.867421+0000, chain.objs.size()=1 2024-04-22T20:48:16.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.270__shadow_.wa9omYgnOfug8XjE9aCI6EJdvgvWrAb_0 2024-04-22T20:48:16.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2781148016406308059', time=2024-04-22T22:26:57.996790+0000, chain.objs.size()=1 2024-04-22T20:48:16.654 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.655+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.275__shadow_.1WgNjh6YIQr868xdiDcxDTUkmGb668y_0 2024-04-22T20:48:16.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.657+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6451234891547436187', time=2024-04-22T22:27:01.957587+0000, chain.objs.size()=1 2024-04-22T20:48:16.655 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.657+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.279__shadow_.RF83ENEE5A-Y9nO2hEmMw2V9D1pHxZW_0 2024-04-22T20:48:16.656 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.660+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15775807986041176889', time=2024-04-22T22:27:07.858916+0000, chain.objs.size()=1 2024-04-22T20:48:16.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.660+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.tztkD-yXHGMlL9FeLuxflPesTuTTk3Y_0 2024-04-22T20:48:16.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2451078270510216941', time=2024-04-22T22:27:07.958133+0000, chain.objs.size()=1 2024-04-22T20:48:16.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.uBPOo103lv7QYLFrV1l4CZgagQ-Nnwm_0 2024-04-22T20:48:16.657 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14841477262051008380', time=2024-04-22T22:27:39.276488+0000, chain.objs.size()=1 2024-04-22T20:48:16.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.311__shadow_.-YNAM2eiUpc4HA3ywqBE4GZ639muhwO_0 2024-04-22T20:48:16.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8393808385201016196', time=2024-04-22T22:28:17.363903+0000, chain.objs.size()=1 2024-04-22T20:48:16.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.odkf6SmUE9Gf166aG_480y3t5mA0W7n_0 2024-04-22T20:48:16.658 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18139091876736413738', time=2024-04-22T22:28:17.673576+0000, chain.objs.size()=1 2024-04-22T20:48:16.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.oJDJVFqcKxpiQT4QbTssyK-9RodzN-M_0 2024-04-22T20:48:16.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3923778409466071171', time=2024-04-22T22:28:17.850530+0000, chain.objs.size()=1 2024-04-22T20:48:16.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WQOvYN9HrN406v6F4OleZsD8oIrwBbS_0 2024-04-22T20:48:16.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17249715299426760039', time=2024-04-22T22:28:17.928977+0000, chain.objs.size()=1 2024-04-22T20:48:16.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3jGIgYyURo-kw4GMGix9f_hmh0WFDC9_0 2024-04-22T20:48:16.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11985209166541547209', time=2024-04-22T22:28:17.987885+0000, chain.objs.size()=1 2024-04-22T20:48:16.659 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gu36iAql2DCcn_neRa3c-vD8Yyiog0Y_0 2024-04-22T20:48:16.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15475139990973991769', time=2024-04-22T22:28:18.327265+0000, chain.objs.size()=1 2024-04-22T20:48:16.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kwGy4mtHZF78dTorG5TmFlxhCX9Cnhl_0 2024-04-22T20:48:16.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.394206487619214814', time=2024-04-22T22:28:18.526148+0000, chain.objs.size()=1 2024-04-22T20:48:16.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.og0tPSzeVasgPT5brf91Czh6urP7ek1_0 2024-04-22T20:48:16.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7977789565826816921', time=2024-04-22T22:28:19.363298+0000, chain.objs.size()=1 2024-04-22T20:48:16.660 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.661+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mQigb9NNg-grb8gqfymhIcZZC9M6IWw_0 2024-04-22T20:48:16.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.663+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15642404328024953024', time=2024-04-22T22:28:19.568082+0000, chain.objs.size()=1 2024-04-22T20:48:16.661 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.663+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.g0cdwxO8JaVqQjfnbgRRbuvUBCGB9n__0 2024-04-22T20:48:16.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2305414420737036250', time=2024-04-22T22:28:19.590413+0000, chain.objs.size()=1 2024-04-22T20:48:16.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bcI3jXcmXSXPxw16rTZm5GJSR8VPISS_0 2024-04-22T20:48:16.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10876409063237611084', time=2024-04-22T22:28:21.023842+0000, chain.objs.size()=1 2024-04-22T20:48:16.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bdCKofp7fa-eKFezWhGyS47Np8JLLMl_0 2024-04-22T20:48:16.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17679274411955528487', time=2024-04-22T22:28:21.271271+0000, chain.objs.size()=1 2024-04-22T20:48:16.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nKlAE5exWO5p0l68YCr1mUf4M8cYyT3_0 2024-04-22T20:48:16.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4136003498908957470', time=2024-04-22T22:28:21.440925+0000, chain.objs.size()=1 2024-04-22T20:48:16.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.i-Fu7hNQk-KeSjCm0mSnhV55zS1Nqaq_0 2024-04-22T20:48:16.663 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7963248915421179058', time=2024-04-22T22:28:21.639691+0000, chain.objs.size()=1 2024-04-22T20:48:16.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.W0thOjlVg978xbfVN5cDCX7SJBss4Vw_0 2024-04-22T20:48:16.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10539962817261055930', time=2024-04-22T22:28:21.790124+0000, chain.objs.size()=1 2024-04-22T20:48:16.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6umPJoRgq8dtqPfqi1BYm-IV8HiLIdS_0 2024-04-22T20:48:16.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8664858910871307821', time=2024-04-22T22:28:21.842733+0000, chain.objs.size()=1 2024-04-22T20:48:16.664 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.666+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.D38QB6ZVhbnJmJY1J4f9FhD5wC8LqZX_0 2024-04-22T20:48:16.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2077019243444709778', time=2024-04-22T22:28:22.640830+0000, chain.objs.size()=1 2024-04-22T20:48:16.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pWuSBPtVZ4g5GWxUWbDmt5pzqsJ9lJT_0 2024-04-22T20:48:16.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7145001925430160421', time=2024-04-22T22:28:22.727568+0000, chain.objs.size()=1 2024-04-22T20:48:16.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.01KoSJHQZsjHF_YW7UoCsKZsjNu7C-y_0 2024-04-22T20:48:16.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.667+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7724505751642879551', time=2024-04-22T22:28:23.238487+0000, chain.objs.size()=1 2024-04-22T20:48:16.665 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.667+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9VLEDCVBbEKhPs0bIKUtoen67c4916M_0 2024-04-22T20:48:16.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.668+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12149857326197993131', time=2024-04-22T22:28:23.480738+0000, chain.objs.size()=1 2024-04-22T20:48:16.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.668+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.H5XnlBbqcmKqS7Bj4yUgsk_oaC9QDO7_0 2024-04-22T20:48:16.666 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17441304129295296386', time=2024-04-22T22:28:23.880028+0000, chain.objs.size()=1 2024-04-22T20:48:16.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lZj39M8uL1RlE20_d8RS4ds-4UoKrsd_0 2024-04-22T20:48:16.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10838456418726950489', time=2024-04-22T22:28:23.880732+0000, chain.objs.size()=1 2024-04-22T20:48:16.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.hQ4AwB_ssgs73Z995AxFBv8uUKUh-WC_0 2024-04-22T20:48:16.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3351162276489199068', time=2024-04-22T22:28:23.881314+0000, chain.objs.size()=1 2024-04-22T20:48:16.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-KBldBG0cS5UckALL_lAEtnb6kTmOmx_0 2024-04-22T20:48:16.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14463700587428571336', time=2024-04-22T22:28:24.188396+0000, chain.objs.size()=1 2024-04-22T20:48:16.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.O5vjknaAGfwTckl7ISthhIfAicqU08r_0 2024-04-22T20:48:16.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15380710053803884539', time=2024-04-22T22:28:25.584285+0000, chain.objs.size()=1 2024-04-22T20:48:16.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LwsHrW4zKujgKzcYrBzPrOcFML6EZdz_0 2024-04-22T20:48:16.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.476982315642764467', time=2024-04-22T22:28:25.865982+0000, chain.objs.size()=1 2024-04-22T20:48:16.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tY9PjkbfHprRNTd7GII3tDhNn8x7QNb_0 2024-04-22T20:48:16.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12209419477488353696', time=2024-04-22T22:28:25.942379+0000, chain.objs.size()=1 2024-04-22T20:48:16.668 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.670+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.aFh6x3jqlqljOOO4LTGht2TLhrdttvw_0 2024-04-22T20:48:16.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.673+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6153253015817020157', time=2024-04-22T22:28:25.951576+0000, chain.objs.size()=1 2024-04-22T20:48:16.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.673+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.M7z9iuVBYvikDTYQRlzdEeWvCM1I1kS_0 2024-04-22T20:48:16.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.673+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11477812684603781265', time=2024-04-22T22:28:26.213337+0000, chain.objs.size()=1 2024-04-22T20:48:16.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.673+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LK5-8XVyXx85-yJFiFe9a0PIiHGnI3T_0 2024-04-22T20:48:16.670 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.673+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6343501869827814064', time=2024-04-22T22:28:26.546491+0000, chain.objs.size()=1 2024-04-22T20:48:16.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.673+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kNWIjD9ut9EMwrkkWNNhiqmjOxcQ8Dx_0 2024-04-22T20:48:16.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.673+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1084179678481436274', time=2024-04-22T22:28:28.040656+0000, chain.objs.size()=1 2024-04-22T20:48:16.671 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.673+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.w57j_H7O8f-ui64Yya5b060fF0j7rWL_0 2024-04-22T20:48:16.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.675+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.645097810203338590', time=2024-04-22T22:28:28.330066+0000, chain.objs.size()=1 2024-04-22T20:48:16.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.675+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lNT1qK7eRCdrEg9OfOTyK2G8sdSMYHd_0 2024-04-22T20:48:16.672 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.675+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9522410293828229646', time=2024-04-22T22:28:29.257602+0000, chain.objs.size()=1 2024-04-22T20:48:16.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.675+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.oc8HQx-JI8m_M2jsTXmwFUJNbPdLGdd_0 2024-04-22T20:48:16.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15499064471029653392', time=2024-04-22T22:29:06.179949+0000, chain.objs.size()=1 2024-04-22T20:48:16.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4mfPG81aq-W2a3Ik1xfkt2G4I67ftn5_0 2024-04-22T20:48:16.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18212463259105590831', time=2024-04-22T22:29:06.674996+0000, chain.objs.size()=1 2024-04-22T20:48:16.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PLQT71jR_iWkgTl3Tp2rXUXh1H-j6Si_0 2024-04-22T20:48:16.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13088656571278533389', time=2024-04-22T22:29:06.815809+0000, chain.objs.size()=1 2024-04-22T20:48:16.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.R0F5iXhkOuYV2BWSwZcePXW4lN8oyk5_0 2024-04-22T20:48:16.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3973779014706419961', time=2024-04-22T22:29:06.881620+0000, chain.objs.size()=1 2024-04-22T20:48:16.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.fjjkAmzJ___ePH58N078G51Z323Y798_0 2024-04-22T20:48:16.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6312799749492037111', time=2024-04-22T22:29:07.062054+0000, chain.objs.size()=1 2024-04-22T20:48:16.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.676+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.vTsY6_-29zXWQVwd9vgzXfOx1nnoTHp_0 2024-04-22T20:48:16.674 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17935994256929473871', time=2024-04-22T22:29:07.213401+0000, chain.objs.size()=1 2024-04-22T20:48:16.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.I1xrL3eaNBhn_EtanX81_BEptxxVN7Y_0 2024-04-22T20:48:16.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8887768107128569356', time=2024-04-22T22:29:07.250429+0000, chain.objs.size()=1 2024-04-22T20:48:16.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Id7YZOm5XWnHwthc1_N_G1ndTvas3Pk_0 2024-04-22T20:48:16.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4238458147052307978', time=2024-04-22T22:29:08.564447+0000, chain.objs.size()=1 2024-04-22T20:48:16.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.v362kXxattwGVkifINHsCbxfP1rPJtK_0 2024-04-22T20:48:16.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13794368698471729859', time=2024-04-22T22:29:09.359148+0000, chain.objs.size()=1 2024-04-22T20:48:16.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bVBsFqeIifEdKsZ0_FvQON5mhnH_euE_0 2024-04-22T20:48:16.675 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.549928140732139656', time=2024-04-22T22:29:09.553996+0000, chain.objs.size()=1 2024-04-22T20:48:16.676 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.678+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.V8lSGIq7zkyVa19-zGolCZVQcyo6d1j_0 2024-04-22T20:48:16.677 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17682761896856390381', time=2024-04-22T22:29:10.935408+0000, chain.objs.size()=1 2024-04-22T20:48:16.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QjM2_YGk8ls9bcEoskJKZ5BNoqLtSoG_0 2024-04-22T20:48:16.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9215006341464705119', time=2024-04-22T22:29:10.936260+0000, chain.objs.size()=1 2024-04-22T20:48:16.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jsclHCho4Fp6mah-PiF0DixxqB8zhHa_0 2024-04-22T20:48:16.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16437650407037680807', time=2024-04-22T22:29:11.497468+0000, chain.objs.size()=1 2024-04-22T20:48:16.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yfh7fTxPVB8un-SopUM_govfwLx-9Ge_0 2024-04-22T20:48:16.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13982904525731088885', time=2024-04-22T22:29:11.924630+0000, chain.objs.size()=1 2024-04-22T20:48:16.678 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yELd0rKzkElCl0Zeo6EON_Qw-qQlYRn_0 2024-04-22T20:48:16.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10993309422939072778', time=2024-04-22T22:29:14.096959+0000, chain.objs.size()=1 2024-04-22T20:48:16.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.VE7gTLc8eJ0bkv1q4bydZXOX1nTtAaz_0 2024-04-22T20:48:16.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14054645089813655583', time=2024-04-22T22:29:14.205885+0000, chain.objs.size()=1 2024-04-22T20:48:16.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.681+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ePunnHmjfLOU1mjH5prWa5wWUXRL2Dq_0 2024-04-22T20:48:16.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.683+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5963559045858291914', time=2024-04-22T22:29:14.213370+0000, chain.objs.size()=1 2024-04-22T20:48:16.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.683+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gT3YG6HVP0wYDeiR4q0DWjHbuGPGVpR_0 2024-04-22T20:48:16.680 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.684+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16404265066490746314', time=2024-04-22T22:29:14.248554+0000, chain.objs.size()=1 2024-04-22T20:48:16.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.684+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.L_6ZdDEzMHWrHl5ltNVFeph_lXhnizZ_0 2024-04-22T20:48:16.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.684+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7356954439083015177', time=2024-04-22T22:29:14.250120+0000, chain.objs.size()=1 2024-04-22T20:48:16.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.684+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.D-MD5-Q1q2tj520Yv4ofUVx7-O8gbe1_0 2024-04-22T20:48:16.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.684+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.598053907209370049', time=2024-04-22T22:29:14.472891+0000, chain.objs.size()=1 2024-04-22T20:48:16.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.684+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hDIAI8viIqyOIPLMv31a9K96Qo6KlcE_0 2024-04-22T20:48:16.687 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.691+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/54094 2024-04-22T20:48:16.690 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=29, truncated=0, next_marker='' 2024-04-22T20:48:16.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10462492825049554050', time=2024-04-22T22:29:14.947630+0000, chain.objs.size()=1 2024-04-22T20:48:16.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.teKq9lbgswuN1pw6U3hP-XavP6rrHMI_0 2024-04-22T20:48:16.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5735727819364960478', time=2024-04-22T22:29:15.481524+0000, chain.objs.size()=1 2024-04-22T20:48:16.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Jc40Nvg28MvcFnQVZfyBnZ_NfeawCzq_0 2024-04-22T20:48:16.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9064910918346446544', time=2024-04-22T22:29:15.482799+0000, chain.objs.size()=1 2024-04-22T20:48:16.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NzTgIgY5_1Uh9nxdvrwvArxO-mZJ0bT_0 2024-04-22T20:48:16.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4644407856496911670', time=2024-04-22T22:29:15.651130+0000, chain.objs.size()=1 2024-04-22T20:48:16.691 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.5bO30KNSbo6phk5pFiTF4Bn6uHp3od7_0 2024-04-22T20:48:16.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13979777919691368635', time=2024-04-22T22:29:16.078301+0000, chain.objs.size()=1 2024-04-22T20:48:16.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.q43abt5531fHX49jbxxzplHC8SpcE6I_0 2024-04-22T20:48:16.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11788266386113296818', time=2024-04-22T22:29:16.349324+0000, chain.objs.size()=1 2024-04-22T20:48:16.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IDTN2lpdjBZHter9FK3mcWjrE7CBlEr_0 2024-04-22T20:48:16.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14221140119187303224', time=2024-04-22T22:29:16.435157+0000, chain.objs.size()=1 2024-04-22T20:48:16.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZEvT_rO-NAnALG0_wAo9CK5ULGnb_7i_0 2024-04-22T20:48:16.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15729225879397171433', time=2024-04-22T22:29:16.717063+0000, chain.objs.size()=1 2024-04-22T20:48:16.692 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7EHxqicNIHjqUB_-oTRDj9W6Pl-NlC3_0 2024-04-22T20:48:16.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12606523211656390195', time=2024-04-22T22:29:16.758314+0000, chain.objs.size()=1 2024-04-22T20:48:16.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BRC7s92_HNLWb4UzW3kMt1RJ_3ldfgC_0 2024-04-22T20:48:16.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17035944490697731182', time=2024-04-22T22:29:17.044942+0000, chain.objs.size()=1 2024-04-22T20:48:16.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4rLTNsyTuGYw6H3Kt-Uh_CmiiitE2H5_0 2024-04-22T20:48:16.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16649908928767619410', time=2024-04-22T22:29:17.212466+0000, chain.objs.size()=1 2024-04-22T20:48:16.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LSi1Kv0AFM53EmeeEmaEQxF9U-ReMn8_0 2024-04-22T20:48:16.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3341751100568390000', time=2024-04-22T22:29:17.489177+0000, chain.objs.size()=1 2024-04-22T20:48:16.693 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.693+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Oo7PNw8IeHXCfnpowpYkkJoyS5LDzmL_0 2024-04-22T20:48:16.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.385087412562555965', time=2024-04-22T22:29:22.409994+0000, chain.objs.size()=1 2024-04-22T20:48:16.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.320__shadow_.mjR9lY7hHQPcbXL1WP03aKmIgA_iMzP_0 2024-04-22T20:48:16.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3943096331397243111', time=2024-04-22T22:30:55.042756+0000, chain.objs.size()=1 2024-04-22T20:48:16.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.449__shadow_.M7N71hGggquQZ2nJpTmiCyNLTvbTJYC_0 2024-04-22T20:48:16.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11690656372289673285', time=2024-04-22T22:31:19.333292+0000, chain.objs.size()=1 2024-04-22T20:48:16.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.480__shadow_.7k8AsZNEwqGXhEo4l4Tbw8goeBMN7WT_0 2024-04-22T20:48:16.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2_f9c2e197f80ffe35540145d8de7f00ea', time=2024-04-22T22:31:53.037902+0000, chain.objs.size()=6 2024-04-22T20:48:16.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.1 2024-04-22T20:48:16.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2 2024-04-22T20:48:16.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.3 2024-04-22T20:48:16.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.4 2024-04-22T20:48:16.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.5 2024-04-22T20:48:16.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.6 2024-04-22T20:48:16.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~dxnRni6wrKi9l2LSRLFxcAZ_ydgo8Ts', time=2024-04-22T22:34:12.833754+0000, chain.objs.size()=1 2024-04-22T20:48:16.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~dxnRni6wrKi9l2LSRLFxcAZ_ydgo8Ts.2 2024-04-22T20:48:16.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~dxnRni6wrKi9l2LSRLFxcAZ_ydgo8Ts', time=2024-04-22T22:34:12.837488+0000, chain.objs.size()=1 2024-04-22T20:48:16.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~dxnRni6wrKi9l2LSRLFxcAZ_ydgo8Ts.3 2024-04-22T20:48:16.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13867053980216145277', time=2024-04-22T22:36:08.774622+0000, chain.objs.size()=1 2024-04-22T20:48:16.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.525__shadow_.XHjjLZL62OlQLFhRZjhvjNihOU7RVb1_0 2024-04-22T20:48:16.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6332112042824504932', time=2024-04-22T22:37:24.627826+0000, chain.objs.size()=1 2024-04-22T20:48:16.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:BV-RDA.LSkSe2pEqL7UWNtNzUX2FiUN_.tEZdGveirH0QgAwGmZNZFSGXtOn7qyo_0 2024-04-22T20:48:16.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.702+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8163317541673989337', time=2024-04-22T22:37:30.663910+0000, chain.objs.size()=1 2024-04-22T20:48:16.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.702+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:HdFNb1p2xd3QmeHX4RU4avjbyDu2m8M_.9_PEqL-RcD3EFFfhVsg-u6U0iSczHtz_0 2024-04-22T20:48:16.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.702+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10482438343250646587', time=2024-04-22T22:38:10.368290+0000, chain.objs.size()=1 2024-04-22T20:48:16.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.702+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:Yj3IF.4b9JX6UXuNHH49TSLOpN6F6qd_.fbSpUHsx_Ib2Esxuj1KSt3q0y5OVouL_0 2024-04-22T20:48:16.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.702+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7606490794108734120', time=2024-04-22T22:38:49.596571+0000, chain.objs.size()=1 2024-04-22T20:48:16.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.702+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:QGHnk7zUQByOlYW9so5lMxtryaFVyN2_.bKAcXE6mFxBzIn02I0bBc-hSPXk6PR6_0 2024-04-22T20:48:16.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6335028661974929936', time=2024-04-22T22:38:59.136604+0000, chain.objs.size()=1 2024-04-22T20:48:16.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:zXW6j7vDXuEHEJ1Fj0fbPFRHyzi9obg_.bbe_tDP6AAGYxfci_f1GzOe0HKyWoZm_0 2024-04-22T20:48:16.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8500791184079833710', time=2024-04-22T22:41:50.512467+0000, chain.objs.size()=1 2024-04-22T20:48:16.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.615__shadow_.vMBvfYiP1C9J-wAkeHVOPErsFikiKmK_0 2024-04-22T20:48:16.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4314536574002684999', time=2024-04-22T22:41:57.159983+0000, chain.objs.size()=1 2024-04-22T20:48:16.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.624__shadow_.edVs9ZhMgoZ7mVQGSBEdWyXJJoPjXsh_0 2024-04-22T20:48:16.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7658456972153807398', time=2024-04-22T22:42:52.526598+0000, chain.objs.size()=1 2024-04-22T20:48:16.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.647__shadow_.6nmX6JeHpf90bBhATSZE0E6wKIAQGD2_0 2024-04-22T20:48:16.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18401599095844034409', time=2024-04-22T22:43:17.809770+0000, chain.objs.size()=1 2024-04-22T20:48:16.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.687__shadow:bUnNGknFU0sH9FnZ5-LuGo-lb32ykGF_.BcwaLYFvuKu8tPRF8B5_ZIfy2E3DMK6_0 2024-04-22T20:48:16.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14360454525238210420', time=2024-04-22T22:43:23.734239+0000, chain.objs.size()=1 2024-04-22T20:48:16.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.696__shadow:TE9D0Y2ld9nc73mRG1y-X1PdjNGdQCS_.TxVnrM_h77ry7o_f6yi0DdBVMFtoJ8x_0 2024-04-22T20:48:16.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.707+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-22T20:48:16.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.712+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.712+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45321' 2024-04-22T20:48:16.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_KME1reRoljEwlO-XtdstCq_OHvzVLn3', time=2024-04-22T22:22:22.442656+0000, chain.objs.size()=1 2024-04-22T20:48:16.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.152__shadow_.SCU6x_zhkiJeWnEuFUdVIKqKiR1SeJF_0 2024-04-22T20:48:16.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10042556213223688941', time=2024-04-22T22:25:11.163884+0000, chain.objs.size()=1 2024-04-22T20:48:16.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.217__shadow_.v-COKXjSYGo-hYLB0g0E0eQ23Kvg0oI_0 2024-04-22T20:48:16.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5418010779545341426', time=2024-04-22T22:25:12.297786+0000, chain.objs.size()=1 2024-04-22T20:48:16.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.219__shadow_.K0ErGMOaGwfXX0qa2q8uXNCzP2ZXcEq_0 2024-04-22T20:48:16.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4393083267106359719', time=2024-04-22T22:25:19.962327+0000, chain.objs.size()=1 2024-04-22T20:48:16.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.226__shadow_.zspc4EAw-u1GhDgwvTGF17hNTJqM8h2_0 2024-04-22T20:48:16.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11481210391100705041', time=2024-04-22T22:25:34.842980+0000, chain.objs.size()=1 2024-04-22T20:48:16.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.239__shadow_.MZTerYKQFypHD2Ss6s4NogHJVx_uSW8_0 2024-04-22T20:48:16.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2061853883009760150', time=2024-04-22T22:26:22.473317+0000, chain.objs.size()=1 2024-04-22T20:48:16.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ahhU5LMw3HjvfxcGwEWmkdCzJfXdtrF_0 2024-04-22T20:48:16.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8415655100585770446', time=2024-04-22T22:26:22.527187+0000, chain.objs.size()=1 2024-04-22T20:48:16.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4Bh8h7E-jGzaeZ0rCMBZgx3FuQTMkLC_0 2024-04-22T20:48:16.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14758436091018986500', time=2024-04-22T22:26:22.916020+0000, chain.objs.size()=1 2024-04-22T20:48:16.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.AIDprIh4vlDjkZoJCmRpIZAud1689Oe_0 2024-04-22T20:48:16.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6259911280132345793', time=2024-04-22T22:26:23.132403+0000, chain.objs.size()=1 2024-04-22T20:48:16.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4PPz-g7MwzlSYPFwkGusqLEwcJGT20o_0 2024-04-22T20:48:16.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15447248860683841303', time=2024-04-22T22:26:23.376971+0000, chain.objs.size()=1 2024-04-22T20:48:16.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FZK2bmd7gqTY2NUiofxG5UeJh3dbVV__0 2024-04-22T20:48:16.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8479882112490435997', time=2024-04-22T22:26:24.443634+0000, chain.objs.size()=1 2024-04-22T20:48:16.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Mjxn9-0R3_q4PMS0Ivx1Z77MpPc9tyh_0 2024-04-22T20:48:16.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14683568104846527412', time=2024-04-22T22:26:24.463407+0000, chain.objs.size()=1 2024-04-22T20:48:16.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rB3EexG_dps8cEvTSk2gAHgCDSqRmYY_0 2024-04-22T20:48:16.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.716+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6728090074435032037', time=2024-04-22T22:26:25.806371+0000, chain.objs.size()=1 2024-04-22T20:48:16.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.716+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WH87KsuKWk0Hm2o0HZsFVj4EsWuGdTE_0 2024-04-22T20:48:16.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.716+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5431840669578515814', time=2024-04-22T22:26:26.261460+0000, chain.objs.size()=1 2024-04-22T20:48:16.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.716+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.djyoeH5x2bpcO3eDemZQ0urzryjQXVB_0 2024-04-22T20:48:16.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.716+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9914885949769736949', time=2024-04-22T22:26:26.823994+0000, chain.objs.size()=1 2024-04-22T20:48:16.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.716+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.C0_rtLGATQrXe2XgqddgQpEMZFb9vQk_0 2024-04-22T20:48:16.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10051264696931219649', time=2024-04-22T22:26:27.018505+0000, chain.objs.size()=1 2024-04-22T20:48:16.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UTOAgdBY7oUmSZLqAriSrAayz8cOgBc_0 2024-04-22T20:48:16.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9989004709541601620', time=2024-04-22T22:26:27.534179+0000, chain.objs.size()=1 2024-04-22T20:48:16.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.u7zSjVT4wkr1SDbXr7ToLSiOMrWPov5_0 2024-04-22T20:48:16.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5833348978715935081', time=2024-04-22T22:26:27.953074+0000, chain.objs.size()=1 2024-04-22T20:48:16.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ISYchtnyzAj4xDNEcb1tG4fzs7LsfEj_0 2024-04-22T20:48:16.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7235768920469877492', time=2024-04-22T22:26:28.026055+0000, chain.objs.size()=1 2024-04-22T20:48:16.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vXAjH7XdS2XSCSD_dgwb2y35Ps15qk4_0 2024-04-22T20:48:16.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7174828718138742797', time=2024-04-22T22:26:28.605321+0000, chain.objs.size()=1 2024-04-22T20:48:16.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JTDlFa-A2HPfWR-mFhkewaViYL6JDEv_0 2024-04-22T20:48:16.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13624212627505592964', time=2024-04-22T22:26:29.195953+0000, chain.objs.size()=1 2024-04-22T20:48:16.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.d5JfM8Kbvl7ELBXlnSHkR4Mu1QPcoDq_0 2024-04-22T20:48:16.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2416590559998178093', time=2024-04-22T22:26:29.792787+0000, chain.objs.size()=1 2024-04-22T20:48:16.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cfOxD4j-hxkNQ_GDMU_dWIg4zCnBoVR_0 2024-04-22T20:48:16.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3293088892772967501', time=2024-04-22T22:26:29.946126+0000, chain.objs.size()=1 2024-04-22T20:48:16.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.34eoTQ42hBj0ypaFPNPsYwArAkaxwnT_0 2024-04-22T20:48:16.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2662578357357618663', time=2024-04-22T22:26:30.728927+0000, chain.objs.size()=1 2024-04-22T20:48:16.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bgZu817XwSrlqC-saqKj804M0oIvR9f_0 2024-04-22T20:48:16.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.325233338650599586', time=2024-04-22T22:26:30.858946+0000, chain.objs.size()=1 2024-04-22T20:48:16.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.51roJTfE38sxQgGWWJZ3bW4znRHFmtF_0 2024-04-22T20:48:16.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10097233662130853651', time=2024-04-22T22:26:31.376460+0000, chain.objs.size()=1 2024-04-22T20:48:16.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.yrKPuGHIn2_r9VC-rILPvTt12Yn_XP7_0 2024-04-22T20:48:16.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1127360288290218409', time=2024-04-22T22:26:31.606589+0000, chain.objs.size()=1 2024-04-22T20:48:16.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2Wram_5Mh_e22bEWeaB28lswqd-6miR_0 2024-04-22T20:48:16.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3848333093566811663', time=2024-04-22T22:26:32.015788+0000, chain.objs.size()=1 2024-04-22T20:48:16.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GDa_wi_F3mmtHmAtU-280LM6j-aQ7x5_0 2024-04-22T20:48:16.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5350301048043302382', time=2024-04-22T22:26:32.385252+0000, chain.objs.size()=1 2024-04-22T20:48:16.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cB_VilJE0QmUvsadRdtQzeFPJF8Ykfo_0 2024-04-22T20:48:16.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1925764837954404862', time=2024-04-22T22:26:32.508925+0000, chain.objs.size()=1 2024-04-22T20:48:16.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.721+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ph-Z1YanJVHO19XPHrqWfYvvO1y3rAl_0 2024-04-22T20:48:16.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.722+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14619729940848903377', time=2024-04-22T22:26:32.574860+0000, chain.objs.size()=1 2024-04-22T20:48:16.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.722+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.X357AO_I6SxED4IKwTEOiYDD4uVxuHf_0 2024-04-22T20:48:16.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.722+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16697539941205468139', time=2024-04-22T22:26:32.680738+0000, chain.objs.size()=1 2024-04-22T20:48:16.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.722+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wW8mcCmYiE-5b6VIt9iEwErRW1EjNuy_0 2024-04-22T20:48:16.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.722+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14103563704868182182', time=2024-04-22T22:26:32.843922+0000, chain.objs.size()=1 2024-04-22T20:48:16.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.722+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uKGm44a2IARUptyN8_MrStL2VYfSi4Z_0 2024-04-22T20:48:16.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.722+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12959964907216556954', time=2024-04-22T22:26:33.798715+0000, chain.objs.size()=1 2024-04-22T20:48:16.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.722+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.miCnGBqKr70yy4LaeSDhievvbMngdeh_0 2024-04-22T20:48:16.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.724+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15390309879049487845', time=2024-04-22T22:26:36.370248+0000, chain.objs.size()=1 2024-04-22T20:48:16.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.724+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.253__shadow_.j2Ro9bmJOgx4Rw4-ZK92AOuqzHJ4L3F_0 2024-04-22T20:48:16.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.724+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.663477882179110505', time=2024-04-22T22:26:40.203077+0000, chain.objs.size()=1 2024-04-22T20:48:16.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.724+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.257__shadow_.TdNnATplFF32j5-lZHBkrBtAyVQqF4C_0 2024-04-22T20:48:16.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.724+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15838026251791994034', time=2024-04-22T22:26:55.838352+0000, chain.objs.size()=1 2024-04-22T20:48:16.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.724+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.273__shadow_.TJFPvYqeXp6XrRwxVJgBCT5GM3Z7ssS_0 2024-04-22T20:48:16.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7325369859271039234', time=2024-04-22T22:27:03.382540+0000, chain.objs.size()=1 2024-04-22T20:48:16.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.280__shadow_.eVeA84ZCnhk3maUoa_qmwyWnSbY7m02_0 2024-04-22T20:48:16.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11579813774208923202', time=2024-04-22T22:27:05.566241+0000, chain.objs.size()=1 2024-04-22T20:48:16.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.PrRFNXNjs3I-fB6iwuxdOw5GR7Jewbt_0 2024-04-22T20:48:16.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7821618365706546518', time=2024-04-22T22:27:05.592024+0000, chain.objs.size()=1 2024-04-22T20:48:16.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.cH94gAQWRzJCCG7WEhgiyzztqAzBJA3_0 2024-04-22T20:48:16.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1162379625424450543', time=2024-04-22T22:27:05.593384+0000, chain.objs.size()=1 2024-04-22T20:48:16.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.e8g_11djD6zI4r1ivT7QcBSVFVLz6UY_0 2024-04-22T20:48:16.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1411739726497956352', time=2024-04-22T22:27:05.684445+0000, chain.objs.size()=1 2024-04-22T20:48:16.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.pokOaEO3all6w9YYCutzJPuGrNKq7Bx_0 2024-04-22T20:48:16.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16744533603426321939', time=2024-04-22T22:27:28.971610+0000, chain.objs.size()=1 2024-04-22T20:48:16.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.302__shadow_.6kuirQE1AY9S_GNENbfG7TsQug78PeP_0 2024-04-22T20:48:16.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17943481117824227875', time=2024-04-22T22:28:17.365188+0000, chain.objs.size()=1 2024-04-22T20:48:16.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.725+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WwJJ_JTns5DFWXES1Gn5dQZY-feu-aV_0 2024-04-22T20:48:16.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.726+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.470692308290545459', time=2024-04-22T22:28:17.509804+0000, chain.objs.size()=1 2024-04-22T20:48:16.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.726+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CJp8zcYbgT-Po1o9NX215YDyzbGBD4J_0 2024-04-22T20:48:16.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.727+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2707870853472022694', time=2024-04-22T22:28:17.735530+0000, chain.objs.size()=1 2024-04-22T20:48:16.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.727+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nNuMydge9sTDGsIHBdKM1fbumyRfwp1_0 2024-04-22T20:48:16.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7404623041368793218', time=2024-04-22T22:28:17.956003+0000, chain.objs.size()=1 2024-04-22T20:48:16.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.03atgeSlYj8YOgUZStZFS96WO5aPGrW_0 2024-04-22T20:48:16.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1747849965963616508', time=2024-04-22T22:28:18.300440+0000, chain.objs.size()=1 2024-04-22T20:48:16.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pN-w3n_sSjz947uBsprJkSTow4iwf-i_0 2024-04-22T20:48:16.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2400748034687834916', time=2024-04-22T22:28:19.758077+0000, chain.objs.size()=1 2024-04-22T20:48:16.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bB9ThYiaMJDNFSBW686dc59PGPXVzks_0 2024-04-22T20:48:16.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3776813953960969695', time=2024-04-22T22:28:20.659335+0000, chain.objs.size()=1 2024-04-22T20:48:16.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.M0Bl59hesjDL7psnu82XBacjp9lvn5P_0 2024-04-22T20:48:16.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13024255669819687069', time=2024-04-22T22:28:20.688065+0000, chain.objs.size()=1 2024-04-22T20:48:16.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SmjgJApA1RiTLwTdo7jyXMrOkMiHi2j_0 2024-04-22T20:48:16.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5727309643358684928', time=2024-04-22T22:28:20.966042+0000, chain.objs.size()=1 2024-04-22T20:48:16.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3ZUPSNlAhNwMiu_wekcQX2mIWOttbp6_0 2024-04-22T20:48:16.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17998655605711273047', time=2024-04-22T22:28:21.743726+0000, chain.objs.size()=1 2024-04-22T20:48:16.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kq4mB_Fhjs0xNesUaNf56nXeLRuZOr2_0 2024-04-22T20:48:16.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9586294523858070984', time=2024-04-22T22:28:23.292724+0000, chain.objs.size()=1 2024-04-22T20:48:16.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dwfXRyjwjuSvvS-I-kuUt2LgH6OBAGU_0 2024-04-22T20:48:16.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2605657365334559986', time=2024-04-22T22:28:25.435398+0000, chain.objs.size()=1 2024-04-22T20:48:16.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5h7OBMRqF_tJJlTTik9TTVebpK7BQgx_0 2024-04-22T20:48:16.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.867910527855248647', time=2024-04-22T22:28:25.788224+0000, chain.objs.size()=1 2024-04-22T20:48:16.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZNFhy2-vAbjkjRUaQN9hjiSekuyunR1_0 2024-04-22T20:48:16.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3019615041691341302', time=2024-04-22T22:28:25.951591+0000, chain.objs.size()=1 2024-04-22T20:48:16.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RRzMQkDUW2uhtr1R88kNQA2l__-KeVX_0 2024-04-22T20:48:16.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.738+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7499623586589299784', time=2024-04-22T22:28:26.140481+0000, chain.objs.size()=1 2024-04-22T20:48:16.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.738+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Nqj_Fppp1zL-RkMSNgR4Lt0o3zUEq6o_0 2024-04-22T20:48:16.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.738+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4610523705282189377', time=2024-04-22T22:28:26.257977+0000, chain.objs.size()=1 2024-04-22T20:48:16.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.738+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.74BFCC16lQwK973lAM11Vf8u3MygXLH_0 2024-04-22T20:48:16.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14036967961308698223', time=2024-04-22T22:28:26.303695+0000, chain.objs.size()=1 2024-04-22T20:48:16.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.92hB220FVRG0TeJcwaFWrEmi4EDWVJn_0 2024-04-22T20:48:16.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11795005077177696087', time=2024-04-22T22:28:26.304294+0000, chain.objs.size()=1 2024-04-22T20:48:16.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.oJdrGGwuY_i8v346LUNqhOPzALQ4U0f_0 2024-04-22T20:48:16.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8981137245647358211', time=2024-04-22T22:28:26.305002+0000, chain.objs.size()=1 2024-04-22T20:48:16.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6EiO32fkovCA6s6Lasctw-Kj_PqOY-F_0 2024-04-22T20:48:16.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17652067462424734715', time=2024-04-22T22:28:26.305851+0000, chain.objs.size()=1 2024-04-22T20:48:16.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kus2I0xp3hAPa7bhH6frRCcIA4U1BRY_0 2024-04-22T20:48:16.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14535484857440340684', time=2024-04-22T22:28:26.560371+0000, chain.objs.size()=1 2024-04-22T20:48:16.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bYhzbZvgc-I_vs16SXarAJGKgI3dLuQ_0 2024-04-22T20:48:16.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14687831470923606359', time=2024-04-22T22:28:27.055838+0000, chain.objs.size()=1 2024-04-22T20:48:16.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pi_atxnrLq-R7JROx0ff5BBaGW6TWbw_0 2024-04-22T20:48:16.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16050357008994670198', time=2024-04-22T22:28:27.150559+0000, chain.objs.size()=1 2024-04-22T20:48:16.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ONFqbYS_pHoueOeoYqNrVZu695Es6NI_0 2024-04-22T20:48:16.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16456538476688144843', time=2024-04-22T22:28:27.249568+0000, chain.objs.size()=1 2024-04-22T20:48:16.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.N2WjUB5Dw9g8bu1joMUJnscdpehZzLp_0 2024-04-22T20:48:16.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16098492617466476690', time=2024-04-22T22:28:27.638340+0000, chain.objs.size()=1 2024-04-22T20:48:16.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.iBtkJk4Dr_Ory4qLDQ6wFVXlaZXxvmy_0 2024-04-22T20:48:16.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13445172987119992271', time=2024-04-22T22:28:28.040699+0000, chain.objs.size()=1 2024-04-22T20:48:16.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.n7_lzmPmStlIvfn6BO3aZ8WiqLuJTz-_0 2024-04-22T20:48:16.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.742+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7538547562734923127', time=2024-04-22T22:28:28.522479+0000, chain.objs.size()=1 2024-04-22T20:48:16.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.742+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.f5nr9s4uAA_NSgn0z7nUCqRLoatxKNh_0 2024-04-22T20:48:16.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.743+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.250379491621123050', time=2024-04-22T22:28:28.947389+0000, chain.objs.size()=1 2024-04-22T20:48:16.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.743+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ErA-Y43kmhgQi-QSBj839NRu3UB0J-u_0 2024-04-22T20:48:16.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.743+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10763677212370718874', time=2024-04-22T22:28:28.948056+0000, chain.objs.size()=1 2024-04-22T20:48:16.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.743+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.AC9hMWvOP09cvDRnyHeb03hOH8Cl3m4_0 2024-04-22T20:48:16.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10299455623595529179', time=2024-04-22T22:28:29.100948+0000, chain.objs.size()=1 2024-04-22T20:48:16.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wWrEyzONi_HZr0HpN3ikmgAwzxH7JRU_0 2024-04-22T20:48:16.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6446348200578820854', time=2024-04-22T22:29:06.179961+0000, chain.objs.size()=1 2024-04-22T20:48:16.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kp6R8gFy2ySWEi8FkF98qbWzAaHWwYg_0 2024-04-22T20:48:16.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9790249462353406570', time=2024-04-22T22:29:06.458565+0000, chain.objs.size()=1 2024-04-22T20:48:16.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.a6hEPMuJ1xv-nKeuAE7bUUs2Oi4tKpo_0 2024-04-22T20:48:16.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16408918551795151759', time=2024-04-22T22:29:06.815726+0000, chain.objs.size()=1 2024-04-22T20:48:16.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Caqa9gLbNMkROGOnJWRcnpCYF4EFovH_0 2024-04-22T20:48:16.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.574725399331958145', time=2024-04-22T22:29:07.041338+0000, chain.objs.size()=1 2024-04-22T20:48:16.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jAj02CCl8XnqNGILMsOs777IBAuIAJf_0 2024-04-22T20:48:16.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1266005685672469356', time=2024-04-22T22:29:07.122956+0000, chain.objs.size()=1 2024-04-22T20:48:16.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PZ1Tpvx4RWKMNua3mOFYYHgKMolx29b_0 2024-04-22T20:48:16.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.108725839096617136', time=2024-04-22T22:29:08.113568+0000, chain.objs.size()=1 2024-04-22T20:48:16.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.744+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dkp5YEIZG69VMi3EKQnn-PimVLlhSEA_0 2024-04-22T20:48:16.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.745+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1571962072405539336', time=2024-04-22T22:29:08.462826+0000, chain.objs.size()=1 2024-04-22T20:48:16.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.745+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WE82Gv5MXKdZWNZP0IrXIMnKAPfdIXH_0 2024-04-22T20:48:16.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.745+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4197290410893244915', time=2024-04-22T22:29:09.627241+0000, chain.objs.size()=1 2024-04-22T20:48:16.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.745+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Z3LwwFfCxypj1PckeywK1bBvSJj8eOb_0 2024-04-22T20:48:16.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.749+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6273217506475023425', time=2024-04-22T22:29:09.706246+0000, chain.objs.size()=1 2024-04-22T20:48:16.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.749+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bJig7tcWLEKOp-qjEFt-RA2_tW-HgcZ_0 2024-04-22T20:48:16.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.749+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14136561848935923516', time=2024-04-22T22:29:10.078878+0000, chain.objs.size()=1 2024-04-22T20:48:16.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.749+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JxOU_ynr7bBERItPCelcdww64-ZGE_5_0 2024-04-22T20:48:16.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16650864587944276164', time=2024-04-22T22:29:10.079723+0000, chain.objs.size()=1 2024-04-22T20:48:16.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6hRstQmz5mxuW6pVuh2C7jzzjPxbzU6_0 2024-04-22T20:48:16.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12849271006362190303', time=2024-04-22T22:29:11.183539+0000, chain.objs.size()=1 2024-04-22T20:48:16.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nJXAEZxSc442JNyDcqvhprunOYM5D12_0 2024-04-22T20:48:16.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3271592406794625274', time=2024-04-22T22:29:11.398639+0000, chain.objs.size()=1 2024-04-22T20:48:16.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6n8EkZvxFvKdmOEGpsJCK2tviii8ty8_0 2024-04-22T20:48:16.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1811537542934475277', time=2024-04-22T22:29:12.473921+0000, chain.objs.size()=1 2024-04-22T20:48:16.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dcYjzbA2VAVgnmOLjgVBBA6n4gYf3Ww_0 2024-04-22T20:48:16.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1235343764397688916', time=2024-04-22T22:29:12.801934+0000, chain.objs.size()=1 2024-04-22T20:48:16.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.i1ZcQaHDa9dr8ClkCaTkj7gfnp_YpN7_0 2024-04-22T20:48:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14890671476576204346', time=2024-04-22T22:29:13.040961+0000, chain.objs.size()=1 2024-04-22T20:48:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.D3Q77XsfAtssev34td12VFoopSwVUxv_0 2024-04-22T20:48:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5907725262447139412', time=2024-04-22T22:29:13.182452+0000, chain.objs.size()=1 2024-04-22T20:48:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pU3AHQ7xWaZYse-1_Vs3NeWR6DitMmB_0 2024-04-22T20:48:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.751+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10263984072258388858', time=2024-04-22T22:29:13.240109+0000, chain.objs.size()=1 2024-04-22T20:48:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.AjClyz4iZnP_i9LCLMyX1Vem2EOCYBN_0 2024-04-22T20:48:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8872996162334141581', time=2024-04-22T22:29:13.338143+0000, chain.objs.size()=1 2024-04-22T20:48:16.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.I1yk73RTUE8jxTJEmLwnb9QreJxecN9_0 2024-04-22T20:48:16.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.755+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13458632476274999607', time=2024-04-22T22:29:13.922207+0000, chain.objs.size()=1 2024-04-22T20:48:16.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.755+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.h3vsJtw4R50C2crgegsFEPm3FRt2-4j_0 2024-04-22T20:48:16.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.755+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2781568142657464326', time=2024-04-22T22:29:15.304312+0000, chain.objs.size()=1 2024-04-22T20:48:16.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.755+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3KmgTm04wwIU6P4lc9FAo8mF9RXFpBZ_0 2024-04-22T20:48:16.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.757+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7338882013727172748', time=2024-04-22T22:29:15.365914+0000, chain.objs.size()=1 2024-04-22T20:48:16.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.757+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.H2rdEUk-KzALp37lreU2iMIeVv-NmnY_0 2024-04-22T20:48:16.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11476916214834610097', time=2024-04-22T22:29:15.558696+0000, chain.objs.size()=1 2024-04-22T20:48:16.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.b-_Z5o2jsROLwDlmy0A7HoJ4DiFm4PN_0 2024-04-22T20:48:16.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2426824241477522491', time=2024-04-22T22:29:15.687591+0000, chain.objs.size()=1 2024-04-22T20:48:16.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.vQbWqghR3gOBWgTCp_A6zFoL2N45k7l_0 2024-04-22T20:48:16.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15446951341676064971', time=2024-04-22T22:29:15.779736+0000, chain.objs.size()=1 2024-04-22T20:48:16.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.FF9WyI0WWM4vN-hb5rdLjsi8bRP-EFO_0 2024-04-22T20:48:16.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8487349100415997939', time=2024-04-22T22:29:15.819330+0000, chain.objs.size()=1 2024-04-22T20:48:16.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HZve9RBHUZfGyZG6djMlfhuS5bgw2hX_0 2024-04-22T20:48:16.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6259566496787099384', time=2024-04-22T22:29:16.893654+0000, chain.objs.size()=1 2024-04-22T20:48:16.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.N6zUStXwUR2rDRUrSTmE5SXraq-LO9__0 2024-04-22T20:48:16.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.763+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/45321 2024-04-22T20:48:16.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-04-22T20:48:16.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13589875847620391862', time=2024-04-22T22:29:17.377660+0000, chain.objs.size()=1 2024-04-22T20:48:16.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8VOkEQH8aXSp9HNAyiXa_bKc7Gk76_r_0 2024-04-22T20:48:16.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5945239986517906988', time=2024-04-22T22:29:17.562824+0000, chain.objs.size()=1 2024-04-22T20:48:16.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MoC27C-us1Xd2-vHuoCLLwtUoYwSP8p_0 2024-04-22T20:48:16.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2455025232460552680', time=2024-04-22T22:31:00.515095+0000, chain.objs.size()=1 2024-04-22T20:48:16.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.454__shadow_.3xabSlaW3c_nSFmtxcZk-euMTBu7mU5_0 2024-04-22T20:48:16.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14621471256623839240', time=2024-04-22T22:37:41.184989+0000, chain.objs.size()=1 2024-04-22T20:48:16.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow:xMxBbc94.HFxKaSJ47MsIp.lux6HWNL_.iyuD0tpjMRq_zDT-7B4FEdbYFl7MTTM_0 2024-04-22T20:48:16.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1934004502985507981', time=2024-04-22T22:38:57.515441+0000, chain.objs.size()=1 2024-04-22T20:48:16.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.574__shadow:Jiy3xvwftQoP0lmllVT9Qvr0.uyQ2Ix_.VATacZtFPvX2hTYWIJkQCzK4lnYX3pt_0 2024-04-22T20:48:16.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10972777070174158037', time=2024-04-22T22:41:48.800808+0000, chain.objs.size()=1 2024-04-22T20:48:16.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.613__shadow_.RFFR96tBhD04pSzIic6Iv3kIzloF4IX_0 2024-04-22T20:48:16.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12601744374603042687', time=2024-04-22T22:42:23.226489+0000, chain.objs.size()=2 2024-04-22T20:48:16.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.638__shadow_.E3WTNsTlTp5ijcI3D5R8Yj14bmpVWZV_0 2024-04-22T20:48:16.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.768+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.638__shadow_.E3WTNsTlTp5ijcI3D5R8Yj14bmpVWZV_1 2024-04-22T20:48:16.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.773+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.781+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-22T20:48:16.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.782+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.783+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44228' 2024-04-22T20:48:16.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_mgd1xKan4sW8qwttKi8R68RCHdboGgw', time=2024-04-22T22:22:22.468338+0000, chain.objs.size()=1 2024-04-22T20:48:16.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.152__shadow_.Nj_OkyiRK8VcxgHVEmHJYiCzTW_0vQK_0 2024-04-22T20:48:16.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10600607819884903533', time=2024-04-22T22:22:46.878445+0000, chain.objs.size()=1 2024-04-22T20:48:16.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.165__shadow_.a-r89y4xKAqGUW4VYZTVAOQzcs_7Ka0_0 2024-04-22T20:48:16.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12504487416296410519', time=2024-04-22T22:22:57.562639+0000, chain.objs.size()=1 2024-04-22T20:48:16.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.177__shadow_.HKpmoAD-SJZhFk6zTzAR5Vl8FEH9P0H_0 2024-04-22T20:48:16.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2464374553925797183', time=2024-04-22T22:25:35.829036+0000, chain.objs.size()=1 2024-04-22T20:48:16.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.240__shadow_.lo9mP-0IC6Onbrcedjsi2qHTtKmSG4s_0 2024-04-22T20:48:16.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13870163642477291357', time=2024-04-22T22:25:41.796236+0000, chain.objs.size()=1 2024-04-22T20:48:16.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.246__shadow_.L8OpXHMsI1wu63G8k25ovZ0IDS0LLIB_0 2024-04-22T20:48:16.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2456721948946063833', time=2024-04-22T22:26:23.297614+0000, chain.objs.size()=1 2024-04-22T20:48:16.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1x3UeMaRgqXib5PxqHeXhbIzfL4yQZq_0 2024-04-22T20:48:16.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5254007159937390271', time=2024-04-22T22:26:24.165423+0000, chain.objs.size()=1 2024-04-22T20:48:16.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BCEHtnbQluJ23YT3jgo4j5elJZE8Xz__0 2024-04-22T20:48:16.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10272030913223800301', time=2024-04-22T22:26:24.268591+0000, chain.objs.size()=1 2024-04-22T20:48:16.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YyEIStsR9nrZoFH1FYMinJQ7Vknwe6D_0 2024-04-22T20:48:16.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4156274101433641047', time=2024-04-22T22:26:24.423277+0000, chain.objs.size()=1 2024-04-22T20:48:16.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OubChHLcikJo-qsn0YcEk5H6C72E0o9_0 2024-04-22T20:48:16.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3551928460281687778', time=2024-04-22T22:26:24.550125+0000, chain.objs.size()=1 2024-04-22T20:48:16.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.K3YIpYLcx5t7S01KizTp5raGomHX08E_0 2024-04-22T20:48:16.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12200382436948999447', time=2024-04-22T22:26:24.798353+0000, chain.objs.size()=1 2024-04-22T20:48:16.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LLmbdvgn0DuRuVqweyTvyqGymqCwVpO_0 2024-04-22T20:48:16.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4684348944692138009', time=2024-04-22T22:26:25.650967+0000, chain.objs.size()=1 2024-04-22T20:48:16.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.784+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.NEV7o_1HFDJ3b7PfawPgKQbubdZ_t30_0 2024-04-22T20:48:16.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.786+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14331871255552024442', time=2024-04-22T22:26:25.686026+0000, chain.objs.size()=1 2024-04-22T20:48:16.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.786+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5nX16hMyQEsJ0NLc_9skloMFwK8Z7b8_0 2024-04-22T20:48:16.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.788+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16570777170640495758', time=2024-04-22T22:26:25.823942+0000, chain.objs.size()=1 2024-04-22T20:48:16.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.788+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qs5vfG4_zbuumYcveBR4goiyOX9Xr0t_0 2024-04-22T20:48:16.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9236461269014297697', time=2024-04-22T22:26:26.057771+0000, chain.objs.size()=1 2024-04-22T20:48:16.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.da-Rt_QMBgo7MZO-tjubi-QjXfQyxvb_0 2024-04-22T20:48:16.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4970920557804950965', time=2024-04-22T22:26:27.663982+0000, chain.objs.size()=1 2024-04-22T20:48:16.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eisC_IfIZuBLEw527NsqSYVycQV7wuu_0 2024-04-22T20:48:16.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4393167919426716526', time=2024-04-22T22:26:27.730468+0000, chain.objs.size()=1 2024-04-22T20:48:16.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8kpOlsxTZ8mUVo638cXeBoOmL0PGHC9_0 2024-04-22T20:48:16.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6433658366996217324', time=2024-04-22T22:26:28.029723+0000, chain.objs.size()=1 2024-04-22T20:48:16.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.iZ9ZHLztNOja9DAzRimZR9-LJucAyCO_0 2024-04-22T20:48:16.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15041910798023327390', time=2024-04-22T22:26:28.313410+0000, chain.objs.size()=1 2024-04-22T20:48:16.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XevGj-I3eklE_zAfIvdxdJEBzLvSEVR_0 2024-04-22T20:48:16.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18156846545040227569', time=2024-04-22T22:26:28.470779+0000, chain.objs.size()=1 2024-04-22T20:48:16.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.nCj0mW1QXeEOdi9LQDWvOxjRyo6bZEw_0 2024-04-22T20:48:16.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9265228542981814880', time=2024-04-22T22:26:29.027435+0000, chain.objs.size()=1 2024-04-22T20:48:16.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rZeZF29kmZUh0SRXfgdaDY0zEx-MwL9_0 2024-04-22T20:48:16.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17130502771084928310', time=2024-04-22T22:26:29.352599+0000, chain.objs.size()=1 2024-04-22T20:48:16.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.J07v3a19b53gAfE2yLUuBZVB-M6Y4xZ_0 2024-04-22T20:48:16.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14766990729941547935', time=2024-04-22T22:26:30.688551+0000, chain.objs.size()=1 2024-04-22T20:48:16.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FnNBMKTTsq3638L5p7Gxd4K-YO7kun2_0 2024-04-22T20:48:16.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14014780521354553734', time=2024-04-22T22:26:30.774873+0000, chain.objs.size()=1 2024-04-22T20:48:16.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.789+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tw7X5TITMwtOEhV5zgbrPK1p7RNMwVJ_0 2024-04-22T20:48:16.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.792+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4524283370612624930', time=2024-04-22T22:26:30.858491+0000, chain.objs.size()=1 2024-04-22T20:48:16.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.792+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.NOEqBXf3z2P_aFfucrA5gdNOtNm_2r2_0 2024-04-22T20:48:16.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9471330238895128176', time=2024-04-22T22:26:31.047400+0000, chain.objs.size()=1 2024-04-22T20:48:16.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Ck9Av66RfqKChuzFODlILLADtxj2lgE_0 2024-04-22T20:48:16.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3887279910527236308', time=2024-04-22T22:26:31.048144+0000, chain.objs.size()=1 2024-04-22T20:48:16.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sd1bizz7-NNGCE5m1lFvOorvIURYwSW_0 2024-04-22T20:48:16.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17323989450183920143', time=2024-04-22T22:26:31.076934+0000, chain.objs.size()=1 2024-04-22T20:48:16.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WeHdcTKIUlJLyqSquiZ4Ew5tmx7oLar_0 2024-04-22T20:48:16.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14519040385605487542', time=2024-04-22T22:26:31.230578+0000, chain.objs.size()=1 2024-04-22T20:48:16.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._5fdthbGw1vvyedX-IauddoASrvWsL3_0 2024-04-22T20:48:16.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3928943657013462846', time=2024-04-22T22:26:31.342818+0000, chain.objs.size()=1 2024-04-22T20:48:16.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MdDPSzq6vOWSrIbzYJak7r7mz3SLNPe_0 2024-04-22T20:48:16.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5547243111206401471', time=2024-04-22T22:26:31.696708+0000, chain.objs.size()=1 2024-04-22T20:48:16.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CEA7be3_uHx9sr8KjVZ43oFcFaO5Ocu_0 2024-04-22T20:48:16.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17090053706689997574', time=2024-04-22T22:26:32.242787+0000, chain.objs.size()=1 2024-04-22T20:48:16.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.nmqlmaLeLTyALBJdFznvmxMGwZ_CNz7_0 2024-04-22T20:48:16.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15285600956730462923', time=2024-04-22T22:26:32.390463+0000, chain.objs.size()=1 2024-04-22T20:48:16.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ggB1qQXLYaaPkaZid1Og7X3afqzCR_s_0 2024-04-22T20:48:16.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5348631329933486345', time=2024-04-22T22:26:32.391049+0000, chain.objs.size()=1 2024-04-22T20:48:16.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sdyulvwI3VZTeOFFWE9e17FTDyqoPln_0 2024-04-22T20:48:16.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10543951947530269971', time=2024-04-22T22:26:32.548754+0000, chain.objs.size()=1 2024-04-22T20:48:16.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.795+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dK6zzQviHdnkJb--cgxHSPSopyEeKuM_0 2024-04-22T20:48:16.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.797+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10624462439603328613', time=2024-04-22T22:26:32.687523+0000, chain.objs.size()=1 2024-04-22T20:48:16.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.797+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HzQ8dJr_nu12kbxVfkxjmRkc5Jm120Y_0 2024-04-22T20:48:16.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.799+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.289433314234136585', time=2024-04-22T22:26:32.860356+0000, chain.objs.size()=1 2024-04-22T20:48:16.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.799+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dP2cm6LoJB-CLklW0qWv-o0P7KVZsQW_0 2024-04-22T20:48:16.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3657505778096444993', time=2024-04-22T22:26:32.900734+0000, chain.objs.size()=1 2024-04-22T20:48:16.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.T5gAS98PRdYWXM-uFtLNo9V-yTditdk_0 2024-04-22T20:48:16.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2690794123143765492', time=2024-04-22T22:26:33.042856+0000, chain.objs.size()=1 2024-04-22T20:48:16.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.k_jTXU2_I3G3qcEO06k_I4B5LaCDu1f_0 2024-04-22T20:48:16.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4383449161311025174', time=2024-04-22T22:26:33.043530+0000, chain.objs.size()=1 2024-04-22T20:48:16.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UlyKwYB2WWklBnJ0DJPOGa9efuuKz0P_0 2024-04-22T20:48:16.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4828174862645002367', time=2024-04-22T22:26:33.932141+0000, chain.objs.size()=1 2024-04-22T20:48:16.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Ys1YXpTzidvWjbmzfiUghOtPz-KCw6n_0 2024-04-22T20:48:16.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8041785616205245969', time=2024-04-22T22:26:33.983088+0000, chain.objs.size()=1 2024-04-22T20:48:16.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GrPpra5CN6H9r5d1mddESvuln0hn70__0 2024-04-22T20:48:16.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11439152908996405959', time=2024-04-22T22:26:34.276727+0000, chain.objs.size()=1 2024-04-22T20:48:16.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6qwSJHY-xPSHBpYyM_V-1ywnbO2l-wd_0 2024-04-22T20:48:16.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16411500742847497914', time=2024-04-22T22:26:48.876627+0000, chain.objs.size()=1 2024-04-22T20:48:16.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.266__shadow_.ykvieRok-7lq2BIjHJdY83FdOrE_wxA_0 2024-04-22T20:48:16.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2881286411363426926', time=2024-04-22T22:26:50.885511+0000, chain.objs.size()=1 2024-04-22T20:48:16.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.268__shadow_.Aj_nzPKqq6vKwl4XcDVJ92n1jsbPVD-_0 2024-04-22T20:48:16.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8379509332891358545', time=2024-04-22T22:26:58.975394+0000, chain.objs.size()=1 2024-04-22T20:48:16.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.800+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.276__shadow_.jFaopqYaBwUDkNnbbzB5RueRuNTYaCc_0 2024-04-22T20:48:16.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14363817558192836864', time=2024-04-22T22:27:19.245940+0000, chain.objs.size()=1 2024-04-22T20:48:16.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.293__shadow_.2bed7b4p199VlE7OQptu2spQlCVCa5p_0 2024-04-22T20:48:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13921724821264416106', time=2024-04-22T22:27:21.474985+0000, chain.objs.size()=1 2024-04-22T20:48:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.295__shadow_.P434S0Aydcg3cOkkNZyTddjdMifxLm3_0 2024-04-22T20:48:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9665122335508562475', time=2024-04-22T22:27:23.044658+0000, chain.objs.size()=1 2024-04-22T20:48:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.296__shadow:BtMiM0WONUHcDSZHLWo6JFM8wYk3Bqo_.ha-LWylg4IN_g5GVMxJ9Ni2-EUX_o5R_0 2024-04-22T20:48:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5823844092317858452', time=2024-04-22T22:28:17.551298+0000, chain.objs.size()=1 2024-04-22T20:48:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TGoD5ZUSzg2U6d1BKYfDvwCUCDEfnVc_0 2024-04-22T20:48:16.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4063355330267387605', time=2024-04-22T22:28:17.605430+0000, chain.objs.size()=1 2024-04-22T20:48:16.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JCxHWFmoA_D9Opu_jrJMwwC6-c6IESj_0 2024-04-22T20:48:16.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1929874039268963845', time=2024-04-22T22:28:17.715755+0000, chain.objs.size()=1 2024-04-22T20:48:16.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SQ_NBNG2oNiRi--OgWx6cVuipeLwlz-_0 2024-04-22T20:48:16.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4460366002503293349', time=2024-04-22T22:28:17.873689+0000, chain.objs.size()=1 2024-04-22T20:48:16.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ziU2JWgEsULPtGjOT1AXsa-RWborMDY_0 2024-04-22T20:48:16.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6738341958128216063', time=2024-04-22T22:28:19.281273+0000, chain.objs.size()=1 2024-04-22T20:48:16.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WE1sZ_Yh1ve4mFTzrvgN9yzauqrbdel_0 2024-04-22T20:48:16.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9719337184900628290', time=2024-04-22T22:28:19.537097+0000, chain.objs.size()=1 2024-04-22T20:48:16.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GfE3BmMG2Z8JBgZP1i6fhH_X56SS0Fz_0 2024-04-22T20:48:16.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7086314534826874841', time=2024-04-22T22:28:20.116648+0000, chain.objs.size()=1 2024-04-22T20:48:16.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.B1OTAM2W2KipIzRXnGBB537phzRMc_3_0 2024-04-22T20:48:16.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16309470376368663422', time=2024-04-22T22:28:20.117978+0000, chain.objs.size()=1 2024-04-22T20:48:16.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yHVriUzGYP0hF295_jpSCJZ_Snwana4_0 2024-04-22T20:48:16.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5280751810603131762', time=2024-04-22T22:28:20.717803+0000, chain.objs.size()=1 2024-04-22T20:48:16.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xwe27HoZZis91cvsSRY-bM9Ve_7KCIK_0 2024-04-22T20:48:16.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1465547555834147365', time=2024-04-22T22:28:20.901255+0000, chain.objs.size()=1 2024-04-22T20:48:16.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OpClfA5ZcsnbtUbDVm1Z8BJTUcJA5pz_0 2024-04-22T20:48:16.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.811+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4231503870564932492', time=2024-04-22T22:28:21.031898+0000, chain.objs.size()=1 2024-04-22T20:48:16.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.811+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OAvkq2ZmEKy3HEgDv5DnEKl9A5-1qMZ_0 2024-04-22T20:48:16.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.811+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8532638192742183965', time=2024-04-22T22:28:21.069697+0000, chain.objs.size()=1 2024-04-22T20:48:16.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.811+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CtWS0FcRK2V_B8KAdBWtrru2vb0IIQW_0 2024-04-22T20:48:16.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7840434244981765700', time=2024-04-22T22:28:21.159253+0000, chain.objs.size()=1 2024-04-22T20:48:16.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mkf5IOxJuHy3VcTMrbb4tNKM46k3z-r_0 2024-04-22T20:48:16.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12750942645047002846', time=2024-04-22T22:28:21.382424+0000, chain.objs.size()=1 2024-04-22T20:48:16.810 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nvOZR6i9G_OKq_xtEokza_Nt3xVx_kU_0 2024-04-22T20:48:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16820586000523691341', time=2024-04-22T22:28:21.555758+0000, chain.objs.size()=1 2024-04-22T20:48:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JAlvjJPTrP_r_CpcExN53OwX3HZo7mz_0 2024-04-22T20:48:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9506760798743621633', time=2024-04-22T22:28:21.704932+0000, chain.objs.size()=1 2024-04-22T20:48:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.r3G76pwDBzWaBjlTihkrZTW2MZ5noxw_0 2024-04-22T20:48:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15134128236344089221', time=2024-04-22T22:28:22.649744+0000, chain.objs.size()=1 2024-04-22T20:48:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.813+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.2Lasch7dG61jN2TI6nMTQus4bZWvl-0_0 2024-04-22T20:48:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.814+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14613509113076294117', time=2024-04-22T22:28:23.011479+0000, chain.objs.size()=1 2024-04-22T20:48:16.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.814+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XSRpqFqqdOJUg7wQco3OepBMziaTImL_0 2024-04-22T20:48:16.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.814+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15448601279159208425', time=2024-04-22T22:28:23.411901+0000, chain.objs.size()=1 2024-04-22T20:48:16.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.814+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8hQrVU_WMg468o7YeWnZTSoR84CgWko_0 2024-04-22T20:48:16.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.814+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13262175462596751881', time=2024-04-22T22:28:24.495828+0000, chain.objs.size()=1 2024-04-22T20:48:16.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.814+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.AlXM9bVawVn4wf8FZVQy9sy91cIYDig_0 2024-04-22T20:48:16.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16574260229542924019', time=2024-04-22T22:28:24.584172+0000, chain.objs.size()=1 2024-04-22T20:48:16.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Kq5_2TCPy7L0pEQ624hULSjiXCFh2Le_0 2024-04-22T20:48:16.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15501396793961774832', time=2024-04-22T22:28:24.774673+0000, chain.objs.size()=1 2024-04-22T20:48:16.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KihcA2-ALq6n-DmbJTbuRKq-xmuckAo_0 2024-04-22T20:48:16.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11819916063106504550', time=2024-04-22T22:28:24.853712+0000, chain.objs.size()=1 2024-04-22T20:48:16.814 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WIX8GhZheiGLQctw2teHw7sg7prOrxa_0 2024-04-22T20:48:16.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18331534481917615160', time=2024-04-22T22:28:25.411972+0000, chain.objs.size()=1 2024-04-22T20:48:16.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pl0yCNCrp37ndcjjBiEuOcSBNvL95H3_0 2024-04-22T20:48:16.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10421680504510497005', time=2024-04-22T22:28:26.097337+0000, chain.objs.size()=1 2024-04-22T20:48:16.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uxXTOgX91tSnuO_NrH95zdTt8-mv6Yi_0 2024-04-22T20:48:16.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8129585126398808278', time=2024-04-22T22:28:26.469874+0000, chain.objs.size()=1 2024-04-22T20:48:16.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.iRavBAOi0nPUqOkGIQ7_xg17lnm4nLz_0 2024-04-22T20:48:16.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16635982317461542355', time=2024-04-22T22:28:26.922763+0000, chain.objs.size()=1 2024-04-22T20:48:16.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.z2BR3wGUObKm47Dv44qDkgBX-EnJHq9_0 2024-04-22T20:48:16.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14892625697693551082', time=2024-04-22T22:28:27.071470+0000, chain.objs.size()=1 2024-04-22T20:48:16.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.at6qGcPv3nWboN7zAZg0RB1zicV4TGs_0 2024-04-22T20:48:16.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5429494422253440316', time=2024-04-22T22:28:27.145658+0000, chain.objs.size()=1 2024-04-22T20:48:16.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.seGwnWbkIx0YWfY6JkWdggk5sxlEykW_0 2024-04-22T20:48:16.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9086430966699759087', time=2024-04-22T22:28:27.642002+0000, chain.objs.size()=1 2024-04-22T20:48:16.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cIyioXYTD4XuGLVK__CMzEMujMYK4wr_0 2024-04-22T20:48:16.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1387858875061497877', time=2024-04-22T22:28:28.493465+0000, chain.objs.size()=1 2024-04-22T20:48:16.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.819+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FCcT9QoC1F7cXn1otGLy5etBVIDoF14_0 2024-04-22T20:48:16.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.822+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10678627346724842654', time=2024-04-22T22:28:28.787300+0000, chain.objs.size()=1 2024-04-22T20:48:16.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.822+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ryIWaF-ANuM6eIpCClrgTSkv-IoVAp4_0 2024-04-22T20:48:16.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.822+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17811421432164089483', time=2024-04-22T22:28:28.939747+0000, chain.objs.size()=1 2024-04-22T20:48:16.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.822+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Pkuea30K8CELyYylzbM7h8_P2cozwqS_0 2024-04-22T20:48:16.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10677636747715815590', time=2024-04-22T22:29:06.155409+0000, chain.objs.size()=1 2024-04-22T20:48:16.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0uRhORG_h8Jg3JHWcxL040EhqI-sB_a_0 2024-04-22T20:48:16.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13785461167242211487', time=2024-04-22T22:29:06.816345+0000, chain.objs.size()=1 2024-04-22T20:48:16.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4AYA4zGkUtgLSISBEJ9ICfApVBy1pAW_0 2024-04-22T20:48:16.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4407017888589126464', time=2024-04-22T22:29:06.983616+0000, chain.objs.size()=1 2024-04-22T20:48:16.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PiN_VwjPah7IPqkwcqTUJIb3SdFSOwk_0 2024-04-22T20:48:16.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18309461205235052604', time=2024-04-22T22:29:07.268524+0000, chain.objs.size()=1 2024-04-22T20:48:16.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MKJhUI2mqjPWOQ3u24EsRT4UFTcKJvX_0 2024-04-22T20:48:16.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8010482423377643422', time=2024-04-22T22:29:07.808573+0000, chain.objs.size()=1 2024-04-22T20:48:16.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.FZA-y0mDbrTxD_87ZJCYmWlnsEsGaes_0 2024-04-22T20:48:16.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2694697782894950133', time=2024-04-22T22:29:07.854266+0000, chain.objs.size()=1 2024-04-22T20:48:16.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QQag8axW9WUcd9n-ZozUrzWfofdCo8E_0 2024-04-22T20:48:16.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3665323998311130509', time=2024-04-22T22:29:07.952103+0000, chain.objs.size()=1 2024-04-22T20:48:16.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zo90nWK3UMAN6P65Fz5AqNwW5AUIPZf_0 2024-04-22T20:48:16.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17015694340967361977', time=2024-04-22T22:29:08.432860+0000, chain.objs.size()=1 2024-04-22T20:48:16.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.E-y82xtobAAkrCdK980emN2YOxpELR3_0 2024-04-22T20:48:16.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8725231122955095387', time=2024-04-22T22:29:08.446954+0000, chain.objs.size()=1 2024-04-22T20:48:16.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.824+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yGcA9NncUptBMRR7OgwI1G4Ok0gSR-o_0 2024-04-22T20:48:16.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9000686318084789712', time=2024-04-22T22:29:08.531877+0000, chain.objs.size()=1 2024-04-22T20:48:16.826 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Yrh46IaHV-G4Pk_DU79qWzHgB_QuOTL_0 2024-04-22T20:48:16.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10959522018767292201', time=2024-04-22T22:29:08.607699+0000, chain.objs.size()=1 2024-04-22T20:48:16.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pMGKTrEPDYJ_pRxUhhbFGYN-aZmQ5NJ_0 2024-04-22T20:48:16.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9349510331468120249', time=2024-04-22T22:29:08.814826+0000, chain.objs.size()=1 2024-04-22T20:48:16.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZUvhtGdM2UcRA4Xs_Wsn5TGvQiQirjZ_0 2024-04-22T20:48:16.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12306808129112765636', time=2024-04-22T22:29:09.412624+0000, chain.objs.size()=1 2024-04-22T20:48:16.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.agQFYqNaPqTmZpfOFHvaDVdPFID05B__0 2024-04-22T20:48:16.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2193668156613113802', time=2024-04-22T22:29:09.519132+0000, chain.objs.size()=1 2024-04-22T20:48:16.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8qDuttFknt-EI4F3gQjqR9KdNwAqnpH_0 2024-04-22T20:48:16.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3299978495136700439', time=2024-04-22T22:29:09.892094+0000, chain.objs.size()=1 2024-04-22T20:48:16.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Bt7xwJXcxZdOO9LN1KBYAjIC82oR1Zy_0 2024-04-22T20:48:16.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3131694902830667288', time=2024-04-22T22:29:10.186120+0000, chain.objs.size()=1 2024-04-22T20:48:16.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oux-JthnmNKHB9iVZYeFbXilTgQWhm3_0 2024-04-22T20:48:16.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1730446667388955243', time=2024-04-22T22:29:10.209253+0000, chain.objs.size()=1 2024-04-22T20:48:16.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Nx8x6xDjo7Rd8XUehV9dUy9yzKiTsbO_0 2024-04-22T20:48:16.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14543371666671994567', time=2024-04-22T22:29:11.055558+0000, chain.objs.size()=1 2024-04-22T20:48:16.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.831+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.74hCLSfEc49zw6sYehvzaFaH-XSECmr_0 2024-04-22T20:48:16.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.838+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/44228 2024-04-22T20:48:16.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=37, truncated=0, next_marker='' 2024-04-22T20:48:16.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4167002977189976780', time=2024-04-22T22:29:11.511372+0000, chain.objs.size()=1 2024-04-22T20:48:16.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kMIT5YJLofueipAdBIVtLwdD_57SuOU_0 2024-04-22T20:48:16.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18205831230093707972', time=2024-04-22T22:29:11.512076+0000, chain.objs.size()=1 2024-04-22T20:48:16.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.I4YdIef7S7Haf1NvUh8Txud1pjUQYsx_0 2024-04-22T20:48:16.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16690982424016803021', time=2024-04-22T22:29:11.643207+0000, chain.objs.size()=1 2024-04-22T20:48:16.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.uSdkHQf_HMNZMyXyjG1U5Bt80--30LT_0 2024-04-22T20:48:16.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12111526687247563676', time=2024-04-22T22:29:11.951061+0000, chain.objs.size()=1 2024-04-22T20:48:16.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.60DezpT-0jfMz-tYPR3_jZPM3yZV69h_0 2024-04-22T20:48:16.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6979913141630734347', time=2024-04-22T22:29:12.410442+0000, chain.objs.size()=1 2024-04-22T20:48:16.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BwVv1EUOWb3TgWVsANtSe2ClEpiFFJT_0 2024-04-22T20:48:16.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6008853186958042943', time=2024-04-22T22:29:12.426637+0000, chain.objs.size()=1 2024-04-22T20:48:16.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pab_A0YzWBb6XznJUq2Wqjf_sZf2RTA_0 2024-04-22T20:48:16.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13797613006863795392', time=2024-04-22T22:29:12.784744+0000, chain.objs.size()=1 2024-04-22T20:48:16.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Xtf_8uZaWmwmf4xlVkEIbJZ4QzNlzdO_0 2024-04-22T20:48:16.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1895980503477278749', time=2024-04-22T22:29:12.937817+0000, chain.objs.size()=1 2024-04-22T20:48:16.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.fw2tY5GnHky8YL35dBm_DRpartEbbs6_0 2024-04-22T20:48:16.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12301790006821425797', time=2024-04-22T22:29:13.041542+0000, chain.objs.size()=1 2024-04-22T20:48:16.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rW-oAmRaA-jcDPuaTvJD5klXCG33JtZ_0 2024-04-22T20:48:16.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10234674584491461905', time=2024-04-22T22:29:13.479877+0000, chain.objs.size()=1 2024-04-22T20:48:16.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hsvSpa3iWfaTaKyKCakBxloYiZFX702_0 2024-04-22T20:48:16.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11521756883296187481', time=2024-04-22T22:29:13.919814+0000, chain.objs.size()=1 2024-04-22T20:48:16.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rOGUr3p_u0kfpazsfxKMcLXUEyOojBY_0 2024-04-22T20:48:16.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13870249989175522387', time=2024-04-22T22:29:14.059527+0000, chain.objs.size()=1 2024-04-22T20:48:16.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xaD5NMP7qJzyIFZUgYTilwS21uFErsu_0 2024-04-22T20:48:16.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11634853433427527222', time=2024-04-22T22:29:15.326394+0000, chain.objs.size()=1 2024-04-22T20:48:16.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.28nUzAkoDxf324hOhCg0lyiTdYq6tkg_0 2024-04-22T20:48:16.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1914006995945948767', time=2024-04-22T22:29:15.475354+0000, chain.objs.size()=1 2024-04-22T20:48:16.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.lUMim3YAMDkHJuCAT21NVnNDEKf8VsP_0 2024-04-22T20:48:16.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5367173512423471130', time=2024-04-22T22:29:15.613646+0000, chain.objs.size()=1 2024-04-22T20:48:16.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RaL5RUOqBVTM3j1edWgHDBRmpRa0as9_0 2024-04-22T20:48:16.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12682506283144135851', time=2024-04-22T22:29:15.668868+0000, chain.objs.size()=1 2024-04-22T20:48:16.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.FNPqom-hstOd1GpFiu6T636JdJPHPE5_0 2024-04-22T20:48:16.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6663552510342324091', time=2024-04-22T22:29:16.184082+0000, chain.objs.size()=1 2024-04-22T20:48:16.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.59E2Lxd-bCgQOx4NWzvUYf8OetkBRfg_0 2024-04-22T20:48:16.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1476468708043542453', time=2024-04-22T22:29:16.341119+0000, chain.objs.size()=1 2024-04-22T20:48:16.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.9p9HngaOH7BFtga4kWzSbXREODFU5j6_0 2024-04-22T20:48:16.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14448391604310808715', time=2024-04-22T22:29:16.341835+0000, chain.objs.size()=1 2024-04-22T20:48:16.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hCQ99unYsPzFPMou0ejmEipeIKjJng4_0 2024-04-22T20:48:16.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15391840924465035054', time=2024-04-22T22:29:17.109744+0000, chain.objs.size()=1 2024-04-22T20:48:16.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.VUEMBjOpmmc6cEUV6ngdrHruBwV3L0F_0 2024-04-22T20:48:16.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14204608067988959030', time=2024-04-22T22:29:17.497990+0000, chain.objs.size()=1 2024-04-22T20:48:16.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tT63eflMsu2B0MoW_w6be3tBSDZ0vZ-_0 2024-04-22T20:48:16.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11222969546718314776', time=2024-04-22T22:29:23.592281+0000, chain.objs.size()=1 2024-04-22T20:48:16.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.322__shadow_.jPW9mV1QbjTDk43iG3GjdkdZDnyIQMY_0 2024-04-22T20:48:16.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14348817546702766424', time=2024-04-22T22:29:26.609789+0000, chain.objs.size()=1 2024-04-22T20:48:16.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.846+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.326__shadow_.rnn9vVPsiUHdXdjXr1onKEt6kosoMkP_0 2024-04-22T20:48:16.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.851+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15450691087012673028', time=2024-04-22T22:30:53.863642+0000, chain.objs.size()=1 2024-04-22T20:48:16.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.851+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.448__shadow_.RNd5BX65QgUFXH4DlOD-i2TriHB58bU_0 2024-04-22T20:48:16.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15514513773974515486', time=2024-04-22T22:30:59.505679+0000, chain.objs.size()=1 2024-04-22T20:48:16.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.453__shadow_.vXgdg6P0iSSfXnPeVulL5IhEPADsN2A_0 2024-04-22T20:48:16.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5866927761808691714', time=2024-04-22T22:31:01.920468+0000, chain.objs.size()=1 2024-04-22T20:48:16.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.455__shadow_.m-q_qzgA7gsGu23nG9AA1co-GPqbERM_0 2024-04-22T20:48:16.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8419828327478835171', time=2024-04-22T22:31:11.757832+0000, chain.objs.size()=1 2024-04-22T20:48:16.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.474__shadow_.JkuIgoE93X_kfwmld0n2rOVgyiOtsQa_0 2024-04-22T20:48:16.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9577822338215402484', time=2024-04-22T22:32:10.178232+0000, chain.objs.size()=1 2024-04-22T20:48:16.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.506__shadow_.VGCd-DHOxgD2gXTnLJYoeg9ueRC0SXV_0 2024-04-22T20:48:16.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~dbRZP9rf-EVpbxPYqF-IFYdv5Py3dyt', time=2024-04-22T22:35:43.994259+0000, chain.objs.size()=1 2024-04-22T20:48:16.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.520__multipart_mymultipart.2~dbRZP9rf-EVpbxPYqF-IFYdv5Py3dyt.1 2024-04-22T20:48:16.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7204394336137198607', time=2024-04-22T22:36:47.058810+0000, chain.objs.size()=1 2024-04-22T20:48:16.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.542__shadow_.Tt0tK7uVlK90RjX1eY0PaNcTyHgO-Vx_0 2024-04-22T20:48:16.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3952677499158910753', time=2024-04-22T22:36:53.248490+0000, chain.objs.size()=1 2024-04-22T20:48:16.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.543__shadow_.t1CEQ7sGxm3NAbLwd1y2lKE2J6W8rMo_0 2024-04-22T20:48:16.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11558850819685659266', time=2024-04-22T22:36:54.451097+0000, chain.objs.size()=1 2024-04-22T20:48:16.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.543__shadow_.sQXtA8O77muSnYQuY_h0WdhdvOf8KKD_0 2024-04-22T20:48:16.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7914676198264633975', time=2024-04-22T22:40:48.000791+0000, chain.objs.size()=12 2024-04-22T20:48:16.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__multipart_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.1 2024-04-22T20:48:16.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__shadow_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.1_1 2024-04-22T20:48:16.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.856+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__multipart_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.2 2024-04-22T20:48:16.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__shadow_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.2_1 2024-04-22T20:48:16.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__multipart_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.3 2024-04-22T20:48:16.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__shadow_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.3_1 2024-04-22T20:48:16.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__multipart_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.4 2024-04-22T20:48:16.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__shadow_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.4_1 2024-04-22T20:48:16.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__multipart_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.5 2024-04-22T20:48:16.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__shadow_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.5_1 2024-04-22T20:48:16.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__multipart_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.6 2024-04-22T20:48:16.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602__shadow_multipart_enc.2~-x7vunr-U9DaHsurSzRH8S4VY5DxeKA.6_1 2024-04-22T20:48:16.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2145662869206297615', time=2024-04-22T22:41:49.456991+0000, chain.objs.size()=1 2024-04-22T20:48:16.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.859+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.614__shadow_.aV0TOqY3FpLRVedyryA-FkFgAsBtIyw_0 2024-04-22T20:48:16.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.861+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12245796887466645701', time=2024-04-22T22:41:56.145681+0000, chain.objs.size()=1 2024-04-22T20:48:16.858 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.861+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.623__shadow_.h6vJxm7O_pPkCV2vlJbhCT1Vlnin9Gn_0 2024-04-22T20:48:16.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.863+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15148357359596495833', time=2024-04-22T22:42:20.305730+0000, chain.objs.size()=1 2024-04-22T20:48:16.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.863+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.636__shadow_.QRP_h9pMiyBhehvrWkDmm8GsQsz7xBr_0 2024-04-22T20:48:16.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.863+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16347691059000898371', time=2024-04-22T22:43:36.166633+0000, chain.objs.size()=1 2024-04-22T20:48:16.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.863+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.700__shadow:9OmyazetxVj0nbM4YKxevy9K1PH9o7x_.l14xKFDH2qf1TZLgKmxOFK_VmjS8JR3_0 2024-04-22T20:48:16.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.868+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.870+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-22T20:48:16.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.872+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.872+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/47724' 2024-04-22T20:48:16.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4999245992144231598', time=2024-04-22T22:21:32.179752+0000, chain.objs.size()=1 2024-04-22T20:48:16.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.147__shadow_.ygt3LlnrM630PbnN7d95ptKqybhHe7N_0 2024-04-22T20:48:16.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_cD9IaYAN73ICveAXdxAlXL18mCmvu9v', time=2024-04-22T22:22:28.102453+0000, chain.objs.size()=1 2024-04-22T20:48:16.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.154__shadow_.eqJA6ZnC6cZFGwZ3pOBlxdaGF39N9C2_0 2024-04-22T20:48:16.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16978867366801720787', time=2024-04-22T22:23:25.846597+0000, chain.objs.size()=1 2024-04-22T20:48:16.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.193__shadow_.M5n0O2y00gFp4Fn_Nr7uIMiWDk63Aee_0 2024-04-22T20:48:16.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4774404288146703610', time=2024-04-22T22:25:12.293391+0000, chain.objs.size()=1 2024-04-22T20:48:16.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.219__shadow_.Y7NyzjOkqL9Y76k64VcasWuUO17aq2y_0 2024-04-22T20:48:16.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3103786556029385121', time=2024-04-22T22:25:21.699315+0000, chain.objs.size()=1 2024-04-22T20:48:16.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.227__shadow_.d9cMZmkddGpSvoo_terHyEj8NRIVp5j_0 2024-04-22T20:48:16.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10448940817280030568', time=2024-04-22T22:25:27.124611+0000, chain.objs.size()=1 2024-04-22T20:48:16.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.232__shadow_._ZOkBYfoK-fQYhp8S2AOLmKJaa-iwxw_0 2024-04-22T20:48:16.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9602067663428777731', time=2024-04-22T22:26:22.757956+0000, chain.objs.size()=1 2024-04-22T20:48:16.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LnuWpp96vy_FkD-EAMTk0s5uMvkdft6_0 2024-04-22T20:48:16.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3220759207038596154', time=2024-04-22T22:26:22.957509+0000, chain.objs.size()=1 2024-04-22T20:48:16.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uvTFEIytI8LiUrDh3dhO5V6ixSwCSu7_0 2024-04-22T20:48:16.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7927646757401471377', time=2024-04-22T22:26:24.534687+0000, chain.objs.size()=1 2024-04-22T20:48:16.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bqps-IyMa7xa9McmtC7mY6GgTPyb6iB_0 2024-04-22T20:48:16.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15813326955720340046', time=2024-04-22T22:26:24.920818+0000, chain.objs.size()=1 2024-04-22T20:48:16.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6JFQqhNxAglPQvNyrcmnCs-c6w3yZpF_0 2024-04-22T20:48:16.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.471804619035246751', time=2024-04-22T22:26:25.017463+0000, chain.objs.size()=1 2024-04-22T20:48:16.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-2WmGgJC1PJovhmnf1v6gpdiy_G-FfV_0 2024-04-22T20:48:16.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5147307665760867488', time=2024-04-22T22:26:25.644706+0000, chain.objs.size()=1 2024-04-22T20:48:16.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0bLWtO-o3qc7N70zUwhSikMXZIVZ0uO_0 2024-04-22T20:48:16.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8840284724609378918', time=2024-04-22T22:26:25.800817+0000, chain.objs.size()=1 2024-04-22T20:48:16.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8AlW8349zc69cW1Adh-ZUasvl1Qhemp_0 2024-04-22T20:48:16.874 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8896494344222345015', time=2024-04-22T22:26:25.973612+0000, chain.objs.size()=1 2024-04-22T20:48:16.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.N1E755ZQ2H4cfOJKJhnPcTfx9fWtaKX_0 2024-04-22T20:48:16.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4836292783727924463', time=2024-04-22T22:26:26.126815+0000, chain.objs.size()=1 2024-04-22T20:48:16.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gSEpu7Rnxn0xeKmtrBDoCu2SyC-cHoP_0 2024-04-22T20:48:16.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5786535801023153213', time=2024-04-22T22:26:26.301092+0000, chain.objs.size()=1 2024-04-22T20:48:16.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YElB2szhtp086WAV0qu5eYnJ0wN_tKb_0 2024-04-22T20:48:16.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13517523035453642158', time=2024-04-22T22:26:26.784063+0000, chain.objs.size()=1 2024-04-22T20:48:16.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6JSksfvs-RaGWTdaDAcfUNWX9U_Rml5_0 2024-04-22T20:48:16.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2632745667612792349', time=2024-04-22T22:26:27.005250+0000, chain.objs.size()=1 2024-04-22T20:48:16.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wN3_vlkNIhhAQhPNlxyr-rB4Nvs1oq2_0 2024-04-22T20:48:16.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3219991838554191599', time=2024-04-22T22:26:27.025481+0000, chain.objs.size()=1 2024-04-22T20:48:16.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.frhIkLIaONOBubkSq2IT7kOD01QV3vI_0 2024-04-22T20:48:16.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4207033049213990433', time=2024-04-22T22:26:28.095616+0000, chain.objs.size()=1 2024-04-22T20:48:16.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xHyaHkJV-tyqAu04QzlhfMFzIrTl7KP_0 2024-04-22T20:48:16.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9577535548119889340', time=2024-04-22T22:26:28.202778+0000, chain.objs.size()=1 2024-04-22T20:48:16.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0hijhUXHDSY51aSfKiDyTA1BxS1uD7z_0 2024-04-22T20:48:16.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4069444378822198973', time=2024-04-22T22:26:28.347664+0000, chain.objs.size()=1 2024-04-22T20:48:16.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3RmbE6UztbWdG1XghMgLwOhTbizbZVo_0 2024-04-22T20:48:16.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2219775620271894792', time=2024-04-22T22:26:28.348335+0000, chain.objs.size()=1 2024-04-22T20:48:16.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.877+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WMJM_nK4i6T-2uqtZrUKcjGPvAzO9EL_0 2024-04-22T20:48:16.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.880+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13573062774512768088', time=2024-04-22T22:26:29.234218+0000, chain.objs.size()=1 2024-04-22T20:48:16.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.880+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VbACuP1lz6TLo_6ra_FN5UPI5xHoGqZ_0 2024-04-22T20:48:16.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.880+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6917298558966300152', time=2024-04-22T22:26:29.324693+0000, chain.objs.size()=1 2024-04-22T20:48:16.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.880+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eKkS1oo58J2TmGpu5ujbqxWm75wPx9s_0 2024-04-22T20:48:16.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.880+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14184585407276593827', time=2024-04-22T22:26:29.833139+0000, chain.objs.size()=1 2024-04-22T20:48:16.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.880+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vLZznuLeGKWdQARim8LRYGS7RnPFvAn_0 2024-04-22T20:48:16.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.880+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10667515513655207984', time=2024-04-22T22:26:30.034828+0000, chain.objs.size()=1 2024-04-22T20:48:16.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.880+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3cZOYV4OKk4CfMdD6w0Erg5VNJSDezl_0 2024-04-22T20:48:16.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.882+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1732925591587017987', time=2024-04-22T22:26:30.116502+0000, chain.objs.size()=1 2024-04-22T20:48:16.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.882+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PdU8bjCmWUyYJzCnSlHmn4D52MqZdq0_0 2024-04-22T20:48:16.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.882+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2523300768054477699', time=2024-04-22T22:26:30.858079+0000, chain.objs.size()=1 2024-04-22T20:48:16.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.882+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sBMvI0UtU9sjxtEI9dfHbd51pHrJ7us_0 2024-04-22T20:48:16.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.882+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10326293767840992478', time=2024-04-22T22:26:31.370189+0000, chain.objs.size()=1 2024-04-22T20:48:16.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.882+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2kuVqXSEAqWMkmfsE_0UDwW0wDTwB_R_0 2024-04-22T20:48:16.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17116991694674022202', time=2024-04-22T22:26:32.365807+0000, chain.objs.size()=1 2024-04-22T20:48:16.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Ren2XrsLMbr_EYMQ3lPftCmvJ6oYzyF_0 2024-04-22T20:48:16.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9835827039152979963', time=2024-04-22T22:26:32.989724+0000, chain.objs.size()=1 2024-04-22T20:48:16.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tY9Fi-tMyeMiXwCre-7XuBc-2tIvOJB_0 2024-04-22T20:48:16.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3142663366622559225', time=2024-04-22T22:26:33.503759+0000, chain.objs.size()=1 2024-04-22T20:48:16.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HpyGSrilMMbKSJ-mZ0zu1KeO1FinBN8_0 2024-04-22T20:48:16.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10137984117476877478', time=2024-04-22T22:26:33.793560+0000, chain.objs.size()=1 2024-04-22T20:48:16.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2Ut1oIYElFatDttrOAoQ7RNR-Vq6OFn_0 2024-04-22T20:48:16.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4516023203394697020', time=2024-04-22T22:26:34.042988+0000, chain.objs.size()=1 2024-04-22T20:48:16.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oI5lj6z_q0spCnmoWhI7ChRBI6mWv7Y_0 2024-04-22T20:48:16.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3905509264478715324', time=2024-04-22T22:26:34.057244+0000, chain.objs.size()=1 2024-04-22T20:48:16.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.v1CiUKLmfX95F0hUeZTeKzqKjZfts7D_0 2024-04-22T20:48:16.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10761834837371965284', time=2024-04-22T22:26:34.165324+0000, chain.objs.size()=1 2024-04-22T20:48:16.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.9JKuhTKBj8b7zhZe2SP_4ND7Ty76onV_0 2024-04-22T20:48:16.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8096035330267166239', time=2024-04-22T22:26:34.329271+0000, chain.objs.size()=1 2024-04-22T20:48:16.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bBxzW7nWUzzDJ-P0ZxsQLEfwTfOoTxR_0 2024-04-22T20:48:16.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11094060497627965704', time=2024-04-22T22:26:34.400257+0000, chain.objs.size()=1 2024-04-22T20:48:16.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.d_xCki1ejyxFcx6Q__bOoYT_kGH_47C_0 2024-04-22T20:48:16.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6572441485730019614', time=2024-04-22T22:26:34.417792+0000, chain.objs.size()=1 2024-04-22T20:48:16.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7RckQmmqcgOKHcve6hSq40p436A6U5O_0 2024-04-22T20:48:16.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.887+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17096169452970440638', time=2024-04-22T22:26:54.857699+0000, chain.objs.size()=1 2024-04-22T20:48:16.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.887+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.272__shadow_.ht4aylbsDmv-acHQ2iP33uVIldjxEWz_0 2024-04-22T20:48:16.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11676179204144943361', time=2024-04-22T22:27:05.567908+0000, chain.objs.size()=1 2024-04-22T20:48:16.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.5BlNC2OBwjz5Wr97hWb5SDAU-gkeiHf_0 2024-04-22T20:48:16.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11491747888059153967', time=2024-04-22T22:27:15.092031+0000, chain.objs.size()=1 2024-04-22T20:48:16.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.289__shadow_.dNpVtBBwZnIHCkpHkW0xkNPovg6yvWz_0 2024-04-22T20:48:16.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1749821757929686496', time=2024-04-22T22:27:19.251547+0000, chain.objs.size()=1 2024-04-22T20:48:16.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.293__shadow_.Nbygwzsp5Dwe45JiMoIfscbmIvTduJR_0 2024-04-22T20:48:16.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8380732695179502179', time=2024-04-22T22:28:17.600844+0000, chain.objs.size()=1 2024-04-22T20:48:16.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GanS3hm1TUTX1fqiB-8JpVGu714cwev_0 2024-04-22T20:48:16.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12397894219097310330', time=2024-04-22T22:28:17.956609+0000, chain.objs.size()=1 2024-04-22T20:48:16.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.EwpLWHnK7WJ6QkN6c3PuTlDsloOzqe8_0 2024-04-22T20:48:16.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14757906002007134368', time=2024-04-22T22:28:17.988193+0000, chain.objs.size()=1 2024-04-22T20:48:16.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6H8Ury2zsryz8OJHPypRP99F-iavxLH_0 2024-04-22T20:48:16.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1482871293812025886', time=2024-04-22T22:28:18.139023+0000, chain.objs.size()=1 2024-04-22T20:48:16.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Ix6KZ8dAAme543fDCSCD1EQtx8LqtVn_0 2024-04-22T20:48:16.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14700759859594048200', time=2024-04-22T22:28:19.242591+0000, chain.objs.size()=1 2024-04-22T20:48:16.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.888+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._0yLJZ4hU3SKt93ERsByU0Y-djWkNmv_0 2024-04-22T20:48:16.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.890+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.90943998469571496', time=2024-04-22T22:28:19.419962+0000, chain.objs.size()=1 2024-04-22T20:48:16.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.890+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dhNF8yJKVmZprzrMoDS93ZYnOfI4iTS_0 2024-04-22T20:48:16.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.890+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12238677706566967992', time=2024-04-22T22:28:19.703530+0000, chain.objs.size()=1 2024-04-22T20:48:16.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.890+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lixe7pwciRtyj1e8QxXJYs-8qrIOFi6_0 2024-04-22T20:48:16.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.890+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7087046836776399165', time=2024-04-22T22:28:21.137755+0000, chain.objs.size()=1 2024-04-22T20:48:16.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.890+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.a43K_m8iM-gNRDYmTqndAdCpANY0a80_0 2024-04-22T20:48:16.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7819265106792491975', time=2024-04-22T22:28:22.727990+0000, chain.objs.size()=1 2024-04-22T20:48:16.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0GvuQuWEQFWVU7xuspctUjSbUSBqouA_0 2024-04-22T20:48:16.889 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15989394283678784439', time=2024-04-22T22:28:22.902315+0000, chain.objs.size()=1 2024-04-22T20:48:16.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.x0QvTKx8uEki2wAC5gQmA0qHFQnusXy_0 2024-04-22T20:48:16.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.891+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2539184263378446404', time=2024-04-22T22:28:22.937850+0000, chain.objs.size()=1 2024-04-22T20:48:16.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.891+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fS60QJwAnEESvA98zphAuJgTz8xaD6l_0 2024-04-22T20:48:16.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.893+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18341582170530208875', time=2024-04-22T22:28:23.103977+0000, chain.objs.size()=1 2024-04-22T20:48:16.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.893+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mr3-ztmSBQeTREfMTieb90w01-_4Zhk_0 2024-04-22T20:48:16.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.893+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9287298287026115445', time=2024-04-22T22:28:23.441938+0000, chain.objs.size()=1 2024-04-22T20:48:16.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.893+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vbqLhTv-engO4qa5fm4SjrajQrB1e3x_0 2024-04-22T20:48:16.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.893+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7999983718867377488', time=2024-04-22T22:28:24.003767+0000, chain.objs.size()=1 2024-04-22T20:48:16.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.893+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1vmHhw7S-73GZ4aUaOSuuxYZq7UBDQJ_0 2024-04-22T20:48:16.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.895+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7305136788611789544', time=2024-04-22T22:28:24.289898+0000, chain.objs.size()=1 2024-04-22T20:48:16.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.895+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zl28WXtLdm9CtuztamEi9spfHvbO4Ey_0 2024-04-22T20:48:16.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.895+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6887894368576896352', time=2024-04-22T22:28:24.655102+0000, chain.objs.size()=1 2024-04-22T20:48:16.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.895+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RaMQQ_N9BfnxzaT3SrI-nDUNiTY3JuY_0 2024-04-22T20:48:16.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.895+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12244296757467603074', time=2024-04-22T22:28:24.758977+0000, chain.objs.size()=1 2024-04-22T20:48:16.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.895+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BSc7wRHOMPPajP76TR9iIwJmyvc4Zcr_0 2024-04-22T20:48:16.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4301176082471060510', time=2024-04-22T22:28:25.503017+0000, chain.objs.size()=1 2024-04-22T20:48:16.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dUSLRnhHiYn47P5KiEKHSeNTejT2BNM_0 2024-04-22T20:48:16.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14183046613679412194', time=2024-04-22T22:28:25.618729+0000, chain.objs.size()=1 2024-04-22T20:48:16.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.AHoLSZZEIsfzqvpygCOkPIWCyr8Of33_0 2024-04-22T20:48:16.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12838296445816512299', time=2024-04-22T22:28:25.784784+0000, chain.objs.size()=1 2024-04-22T20:48:16.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rUAhjp_yExz2mOEGXulailhqKwSiUna_0 2024-04-22T20:48:16.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6802330752102614424', time=2024-04-22T22:28:26.151252+0000, chain.objs.size()=1 2024-04-22T20:48:16.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OdcEl4rytY-9CYMFiCNoYFc1iZD5-8L_0 2024-04-22T20:48:16.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6021284354111082623', time=2024-04-22T22:28:27.638723+0000, chain.objs.size()=1 2024-04-22T20:48:16.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.80HUdKsXAh_zru1E6m4hM4ZZyaXQ1Mm_0 2024-04-22T20:48:16.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12503950126715539478', time=2024-04-22T22:28:27.776453+0000, chain.objs.size()=1 2024-04-22T20:48:16.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.897+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NpfBpN9YkK7qipPJbZx5YDd4ktoYbX__0 2024-04-22T20:48:16.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.898+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10968684668755642873', time=2024-04-22T22:28:27.779590+0000, chain.objs.size()=1 2024-04-22T20:48:16.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.898+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.oy5G4IBI-czd3VXPnlJ3XSNXjDdHATi_0 2024-04-22T20:48:16.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.898+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17791618183720524549', time=2024-04-22T22:28:28.406093+0000, chain.objs.size()=1 2024-04-22T20:48:16.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.898+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-FMEOzqH8uNmcGEZ5xm1TzB959r1S-x_0 2024-04-22T20:48:16.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7099650465395818369', time=2024-04-22T22:28:28.516959+0000, chain.objs.size()=1 2024-04-22T20:48:16.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4BsObsMCfUuKpoDJnpi8hCsNXfAltOz_0 2024-04-22T20:48:16.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11938937590610546006', time=2024-04-22T22:28:28.641644+0000, chain.objs.size()=1 2024-04-22T20:48:16.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.DAKNsjiGJho18pTzimELTaBGkI7OAqS_0 2024-04-22T20:48:16.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9408030693781155589', time=2024-04-22T22:28:28.688816+0000, chain.objs.size()=1 2024-04-22T20:48:16.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.EAW2EsTSafYWjLRWXm_WPbN-pSU7E4I_0 2024-04-22T20:48:16.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15956753402081421897', time=2024-04-22T22:28:29.007201+0000, chain.objs.size()=1 2024-04-22T20:48:16.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.2ZJT-oLBj4qAcI298gBXnBohV_oQT2f_0 2024-04-22T20:48:16.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16367908010521747843', time=2024-04-22T22:28:29.158947+0000, chain.objs.size()=1 2024-04-22T20:48:16.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IyqcoFuveqHzbFKmdqhKAS4NS42zqTS_0 2024-04-22T20:48:16.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17980975569490719131', time=2024-04-22T22:29:06.669841+0000, chain.objs.size()=1 2024-04-22T20:48:16.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WdFm6SBlPFI7ZSyWfWhndAFD6TmeryH_0 2024-04-22T20:48:16.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6277094255898031523', time=2024-04-22T22:29:07.073806+0000, chain.objs.size()=1 2024-04-22T20:48:16.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tWMQ09JTJ7Lm9AE-fJqqHgRBr1KaBkn_0 2024-04-22T20:48:16.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16600609302035187486', time=2024-04-22T22:29:07.158414+0000, chain.objs.size()=1 2024-04-22T20:48:16.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JOJi2NoLXCQvrWrX1m4ySVMn0mP_HPp_0 2024-04-22T20:48:16.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12048765670822223511', time=2024-04-22T22:29:07.279651+0000, chain.objs.size()=1 2024-04-22T20:48:16.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KNJyK85IFJnAqQCdvQ8IyjDv57U1Yw6_0 2024-04-22T20:48:16.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15895476377189220064', time=2024-04-22T22:29:07.810021+0000, chain.objs.size()=1 2024-04-22T20:48:16.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JyRegR8lPMSSxEbo7PlHGyA-6ZiWoQO_0 2024-04-22T20:48:16.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11224167141148506057', time=2024-04-22T22:29:07.811915+0000, chain.objs.size()=1 2024-04-22T20:48:16.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.A1x9TYO5wfK3kWPRe33Wr5f1Q0b3MNf_0 2024-04-22T20:48:16.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1614473725085676094', time=2024-04-22T22:29:07.830735+0000, chain.objs.size()=1 2024-04-22T20:48:16.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xQiyGGa-CjrYtsuVWnUUtJcLT7u-_kh_0 2024-04-22T20:48:16.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.906+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12935717685760298514', time=2024-04-22T22:29:07.963535+0000, chain.objs.size()=1 2024-04-22T20:48:16.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.906+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BrTiWQZci1WK_2fVHed7XtCS2QExwvN_0 2024-04-22T20:48:16.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.906+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18086905569157388889', time=2024-04-22T22:29:08.237648+0000, chain.objs.size()=1 2024-04-22T20:48:16.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.906+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WI2sBwH9meO9Rz2i1B7g6-wxKmgYBXb_0 2024-04-22T20:48:16.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1051662722692715003', time=2024-04-22T22:29:08.282238+0000, chain.objs.size()=1 2024-04-22T20:48:16.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.veRv34hVehMep4pXZ2hFRIpyNHjK5zp_0 2024-04-22T20:48:16.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7390259076562942736', time=2024-04-22T22:29:08.884939+0000, chain.objs.size()=1 2024-04-22T20:48:16.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gwIeGOL8_5YsrmiUAHTsgSEKnmVXGk7_0 2024-04-22T20:48:16.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1872740425373537285', time=2024-04-22T22:29:09.413193+0000, chain.objs.size()=1 2024-04-22T20:48:16.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.D_AyV4hnGToyPbPaB_P7dpeKicImlyV_0 2024-04-22T20:48:16.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6887167108958435672', time=2024-04-22T22:29:09.889686+0000, chain.objs.size()=1 2024-04-22T20:48:16.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Um_a8div_-6mMiOMoXYBZpyqH6h5P0F_0 2024-04-22T20:48:16.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17229563084307066252', time=2024-04-22T22:29:09.890390+0000, chain.objs.size()=1 2024-04-22T20:48:16.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Rrg9AKgl6u4HSFPx8rbP1vXd0FdmZuC_0 2024-04-22T20:48:16.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9014807576684426234', time=2024-04-22T22:29:10.066460+0000, chain.objs.size()=1 2024-04-22T20:48:16.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.uthpLYtn8gKG0QSQIhpA2vLY80yZnE__0 2024-04-22T20:48:16.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.909+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2048105877168875458', time=2024-04-22T22:29:10.292015+0000, chain.objs.size()=1 2024-04-22T20:48:16.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.phI4KvT-ntOsg08f2o7oYLvwD0VdrY9_0 2024-04-22T20:48:16.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3771014190897197857', time=2024-04-22T22:29:10.459338+0000, chain.objs.size()=1 2024-04-22T20:48:16.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.34XGsOrvFUywMVVghTvMvB2EX9ZHnik_0 2024-04-22T20:48:16.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13765434692569041756', time=2024-04-22T22:29:11.951737+0000, chain.objs.size()=1 2024-04-22T20:48:16.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QMpIOQaOv3z25P1TAWhoGeJmbWK1E-G_0 2024-04-22T20:48:16.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11293189141457442935', time=2024-04-22T22:29:12.513581+0000, chain.objs.size()=1 2024-04-22T20:48:16.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HzMe5xkjVilOrbUK43MLtzS4E-Jd7jn_0 2024-04-22T20:48:16.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13682434671940683456', time=2024-04-22T22:29:12.727515+0000, chain.objs.size()=1 2024-04-22T20:48:16.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IPo2_cPSrELQrT9T_0dmKlNtjF_VhQT_0 2024-04-22T20:48:16.910 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7235651299722549423', time=2024-04-22T22:29:12.787249+0000, chain.objs.size()=1 2024-04-22T20:48:16.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RENkISYlenji5Nzkza5JFTFUSA4-obi_0 2024-04-22T20:48:16.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16991768544493922665', time=2024-04-22T22:29:12.788650+0000, chain.objs.size()=1 2024-04-22T20:48:16.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.b8dT8CMxjAUI3jvNgPEOvgE0-Ns8L3L_0 2024-04-22T20:48:16.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9360137304032960226', time=2024-04-22T22:29:12.789301+0000, chain.objs.size()=1 2024-04-22T20:48:16.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Yf1pfyZPJI-eC9oM2qGNwRHbfMR-wyh_0 2024-04-22T20:48:16.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8033276371147260048', time=2024-04-22T22:29:12.872456+0000, chain.objs.size()=1 2024-04-22T20:48:16.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._NGhXVo6Eu20o9FvLdyaSjvtc7vV7rH_0 2024-04-22T20:48:16.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17204426174888067218', time=2024-04-22T22:29:12.873129+0000, chain.objs.size()=1 2024-04-22T20:48:16.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OrCFfD3a6UrvwWQtRO1RQ5-Nj_PpcgV_0 2024-04-22T20:48:16.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8576072218341939171', time=2024-04-22T22:29:12.927651+0000, chain.objs.size()=1 2024-04-22T20:48:16.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NKawWmiMaBYcF5hBs3TilC1UA0CQ8Q2_0 2024-04-22T20:48:16.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.917+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/47724 2024-04-22T20:48:16.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=24, truncated=0, next_marker='' 2024-04-22T20:48:16.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1340267612857050984', time=2024-04-22T22:29:13.443627+0000, chain.objs.size()=1 2024-04-22T20:48:16.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XB85NgXPIwTepGjs58CcYscq2hdY_I6_0 2024-04-22T20:48:16.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10386759142789432090', time=2024-04-22T22:29:14.222928+0000, chain.objs.size()=1 2024-04-22T20:48:16.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iNsxZMmStfnRnuqfxMl58K8wp6wxPei_0 2024-04-22T20:48:16.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14516652515426648804', time=2024-04-22T22:29:15.553101+0000, chain.objs.size()=1 2024-04-22T20:48:16.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DcbzC9l-X0sYdKfn2QJS2EYGw3QeyRp_0 2024-04-22T20:48:16.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4440539251814770783', time=2024-04-22T22:29:15.749147+0000, chain.objs.size()=1 2024-04-22T20:48:16.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MZScu6CCItDXJujYy0ws0Cdd_V-20FW_0 2024-04-22T20:48:16.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12560103262771590769', time=2024-04-22T22:29:16.187890+0000, chain.objs.size()=1 2024-04-22T20:48:16.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.D1bwOWN9hKKo_2sTmLlfIehxmO2TEvN_0 2024-04-22T20:48:16.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9926128619304487450', time=2024-04-22T22:29:16.348332+0000, chain.objs.size()=1 2024-04-22T20:48:16.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._Yo3_Yiur5IxHmzTqTDWtDq-qhMxlWC_0 2024-04-22T20:48:16.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14550855048616014711', time=2024-04-22T22:29:17.598063+0000, chain.objs.size()=1 2024-04-22T20:48:16.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YdQBI_y3PVWmOjfv73m93UpzTIn7nTT_0 2024-04-22T20:48:16.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1530943581085475220', time=2024-04-22T22:29:20.897079+0000, chain.objs.size()=1 2024-04-22T20:48:16.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.318__shadow_.wWRsXMpCFckWy3bzf4q51k9NIDb_F-d_0 2024-04-22T20:48:16.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.488__shadow_.TmHVafHNVDGOusYYgOk7_Sq4e2YLYoU_1_b499c76a1e98a86be843c74532bd972b', time=2024-04-22T22:31:24.765854+0000, chain.objs.size()=1 2024-04-22T20:48:16.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.488__shadow_.TmHVafHNVDGOusYYgOk7_Sq4e2YLYoU_0 2024-04-22T20:48:16.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_1_f2d5a41330dd05803369e3664bd64dc3', time=2024-04-22T22:31:28.273290+0000, chain.objs.size()=1 2024-04-22T20:48:16.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_0 2024-04-22T20:48:16.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2_591487cbdb93e8088c38e09eb576b503', time=2024-04-22T22:31:52.840466+0000, chain.objs.size()=6 2024-04-22T20:48:16.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.1 2024-04-22T20:48:16.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2 2024-04-22T20:48:16.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.3 2024-04-22T20:48:16.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.4 2024-04-22T20:48:16.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.5 2024-04-22T20:48:16.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.6 2024-04-22T20:48:16.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.735286395323493770', time=2024-04-22T22:31:58.357751+0000, chain.objs.size()=1 2024-04-22T20:48:16.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.503__shadow_.jNftU4fmnS--8AcBfX1UIy1nHbeVlrI_0 2024-04-22T20:48:16.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3267662474522881825', time=2024-04-22T22:32:56.227866+0000, chain.objs.size()=1 2024-04-22T20:48:16.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.509__shadow:Li95Dg1EB4Mc4AHDO9EbtrZ39wL61u4_.q3ynDZqri0Z3Y_dc0ilCPt9puNZtgQY_0 2024-04-22T20:48:16.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16510150307952974646', time=2024-04-22T22:34:57.277256+0000, chain.objs.size()=2 2024-04-22T20:48:16.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~EsXEEREngyKjEst5kw0vauSMUBiWyav.1 2024-04-22T20:48:16.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~EsXEEREngyKjEst5kw0vauSMUBiWyav.2 2024-04-22T20:48:16.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11877835574120277761', time=2024-04-22T22:36:46.212169+0000, chain.objs.size()=1 2024-04-22T20:48:16.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.542__shadow_.4Ft2MtbPjm_zivoPkoFTujwkbttrxTa_0 2024-04-22T20:48:16.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~HjLHT30fYhWrOqBUcIkeLRyXiwZkj62', time=2024-04-22T22:37:09.660657+0000, chain.objs.size()=1 2024-04-22T20:48:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.549__multipart_mymultipart.2~HjLHT30fYhWrOqBUcIkeLRyXiwZkj62.1 2024-04-22T20:48:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.810423568530252761', time=2024-04-22T22:37:26.444134+0000, chain.objs.size()=1 2024-04-22T20:48:16.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:iOjcBWU-pcz6kOSSE4CGfSAwWP8o9En_.gcPLYphOY3qvi9IvNlxrhfJJF0gkviy_0 2024-04-22T20:48:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14995944371169212028', time=2024-04-22T22:37:31.543004+0000, chain.objs.size()=1 2024-04-22T20:48:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:FpKJf0Xj1pfyl1hUfdgbde4CyPfzFJE_.mG5r1ZAxnXoCdlxvB-hi7X0Nj4CecW7_0 2024-04-22T20:48:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:mYroPTRUGY7JJ0FlSIYgWmGLZUtmu1u_.uh-IosxowmGuW8QG8mHDszFV7eCd5qI_1_2fc89e865b079c151b3a9391cce27bf3', time=2024-04-22T22:38:51.932915+0000, chain.objs.size()=1 2024-04-22T20:48:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:mYroPTRUGY7JJ0FlSIYgWmGLZUtmu1u_.uh-IosxowmGuW8QG8mHDszFV7eCd5qI_0 2024-04-22T20:48:16.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3933773615315110825', time=2024-04-22T22:39:11.388669+0000, chain.objs.size()=1 2024-04-22T20:48:16.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.594__shadow_.Lx7_alpDym5CJpt1O8umbVCdglXXfb0_0 2024-04-22T20:48:16.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ', time=2024-04-22T22:41:38.950961+0000, chain.objs.size()=12 2024-04-22T20:48:16.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__multipart_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.1 2024-04-22T20:48:16.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__shadow_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.1_1 2024-04-22T20:48:16.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__multipart_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.2 2024-04-22T20:48:16.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__shadow_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.2_1 2024-04-22T20:48:16.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__multipart_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.3 2024-04-22T20:48:16.926 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.930+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__shadow_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.3_1 2024-04-22T20:48:16.926 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.930+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__multipart_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.4 2024-04-22T20:48:16.926 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.930+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__shadow_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.4_1 2024-04-22T20:48:16.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__multipart_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.5 2024-04-22T20:48:16.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__shadow_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.5_1 2024-04-22T20:48:16.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__multipart_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.6 2024-04-22T20:48:16.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.610__shadow_multipart_enc.2~jKZSZs2-v2uKSiCLcRgOGFILyCFkaWQ.6_1 2024-04-22T20:48:16.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3578460345192531035', time=2024-04-22T22:42:01.976937+0000, chain.objs.size()=1 2024-04-22T20:48:16.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.627__shadow_.5MqNG92WaUCrN3Cl665-CikDxGoMbta_0 2024-04-22T20:48:16.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4709343790513652928', time=2024-04-22T22:42:52.526522+0000, chain.objs.size()=1 2024-04-22T20:48:16.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.647__shadow_.zpAFw_PT9EpcNxRnn2Vxxj_RTBGIwJZ_0 2024-04-22T20:48:16.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10707626144863715026', time=2024-04-22T22:43:13.198276+0000, chain.objs.size()=1 2024-04-22T20:48:16.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.931+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.680__shadow:ShSY45gH4Mc-kwAYmC5TmRpxZZHzBHR_.vE6hZT0i3XVsbsxjvmdZblbfADX4gx4_0 2024-04-22T20:48:16.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.938+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.944+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-22T20:48:16.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.946+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45694' 2024-04-22T20:48:16.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17928408009713738221', time=2024-04-22T22:22:13.248712+0000, chain.objs.size()=5 2024-04-22T20:48:16.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-FROZEN.2~kitcn0zlhdPs3X7HHoVLwmV1f77Z7eV.1 2024-04-22T20:48:16.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_foo-FROZEN.2~kitcn0zlhdPs3X7HHoVLwmV1f77Z7eV.1_1 2024-04-22T20:48:16.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-FROZEN.2~kitcn0zlhdPs3X7HHoVLwmV1f77Z7eV.2 2024-04-22T20:48:16.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_foo-FROZEN.2~kitcn0zlhdPs3X7HHoVLwmV1f77Z7eV.2_1 2024-04-22T20:48:16.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-FROZEN.2~kitcn0zlhdPs3X7HHoVLwmV1f77Z7eV.3 2024-04-22T20:48:16.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~uHOKjFni2AkDVLIc4yIz4Tztumv5gHc', time=2024-04-22T22:23:23.683191+0000, chain.objs.size()=1 2024-04-22T20:48:16.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.191__multipart_mymultipart.2~uHOKjFni2AkDVLIc4yIz4Tztumv5gHc.1 2024-04-22T20:48:16.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15098357039973724098', time=2024-04-22T22:25:33.798259+0000, chain.objs.size()=1 2024-04-22T20:48:16.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.238__shadow_.kJZUvIAnoY9hShsVgsW-H9hu9peCnOz_0 2024-04-22T20:48:16.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11708541447558512347', time=2024-04-22T22:25:34.846302+0000, chain.objs.size()=1 2024-04-22T20:48:16.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.239__shadow_.XwXr9oNptp0ofn2jVUlWVODzKEBdGp0_0 2024-04-22T20:48:16.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4136145695651452865', time=2024-04-22T22:25:37.805370+0000, chain.objs.size()=1 2024-04-22T20:48:16.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.242__shadow_.fm7gqaX3qkA91irmxWkn2nO-b28Y2JR_0 2024-04-22T20:48:16.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4389484344111973943', time=2024-04-22T22:25:38.804414+0000, chain.objs.size()=1 2024-04-22T20:48:16.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.243__shadow_.LZyz82SBSOQpKatlHUtOmSdGr7_xzUv_0 2024-04-22T20:48:16.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17944054631442740120', time=2024-04-22T22:26:22.399006+0000, chain.objs.size()=1 2024-04-22T20:48:16.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qaTq6srT8FdcH7A6VHdv-sK_nJ5KzqQ_0 2024-04-22T20:48:16.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2071228207822778932', time=2024-04-22T22:26:22.399649+0000, chain.objs.size()=1 2024-04-22T20:48:16.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.947+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hRf5VmvBctDzgSPtgVNmrnL8lIlJ-gZ_0 2024-04-22T20:48:16.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.949+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4246157131599324769', time=2024-04-22T22:26:22.469877+0000, chain.objs.size()=1 2024-04-22T20:48:16.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.949+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Wm8-FqiONm6Yvttc5PPr7JJYv0AZvsx_0 2024-04-22T20:48:16.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.949+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2825322390639784980', time=2024-04-22T22:26:22.626583+0000, chain.objs.size()=1 2024-04-22T20:48:16.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.949+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.J8efuL65jbssZ_te3ylBFOcQ2WhEQUw_0 2024-04-22T20:48:16.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5527015845240859434', time=2024-04-22T22:26:22.685597+0000, chain.objs.size()=1 2024-04-22T20:48:16.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._jscPXfZDypswZx06RQzbh_eZ10mYRO_0 2024-04-22T20:48:16.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6842523926786050233', time=2024-04-22T22:26:22.784987+0000, chain.objs.size()=1 2024-04-22T20:48:16.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oWmoXoH8-jfxxNxFHcZGi60JDOtIo9L_0 2024-04-22T20:48:16.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12539797451976980257', time=2024-04-22T22:26:23.192803+0000, chain.objs.size()=1 2024-04-22T20:48:16.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HiM4btIJqG5b0_YFMm0SqVjJNZz_O6R_0 2024-04-22T20:48:16.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5656456467265873661', time=2024-04-22T22:26:23.297605+0000, chain.objs.size()=1 2024-04-22T20:48:16.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Nck2_9b9rKeW9-864KrDEmC2VEQlcWb_0 2024-04-22T20:48:16.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6281644579575675754', time=2024-04-22T22:26:23.298363+0000, chain.objs.size()=1 2024-04-22T20:48:16.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.QGukk7QyEMs61tTGB1hM22Y7XbldZC7_0 2024-04-22T20:48:16.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.315179194651627347', time=2024-04-22T22:26:23.473232+0000, chain.objs.size()=1 2024-04-22T20:48:16.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.90ceC44LQxv7bZeNfZhb5FBOBaMt76B_0 2024-04-22T20:48:16.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7982486542310396522', time=2024-04-22T22:26:23.473922+0000, chain.objs.size()=1 2024-04-22T20:48:16.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Bw8kRpkJCAMnEYte8-MjAh8i7ygA83k_0 2024-04-22T20:48:16.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14083980908083636769', time=2024-04-22T22:26:23.497952+0000, chain.objs.size()=1 2024-04-22T20:48:16.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Fr1H9AJn-ucJSsPlu7uE-40hWUqNcuG_0 2024-04-22T20:48:16.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16390873120519986734', time=2024-04-22T22:26:23.983339+0000, chain.objs.size()=1 2024-04-22T20:48:16.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.951+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PuRgLpxYamGHNR7g9LiCtnMS70C5DlA_0 2024-04-22T20:48:16.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.952+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16333302361432313882', time=2024-04-22T22:26:24.054349+0000, chain.objs.size()=1 2024-04-22T20:48:16.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.952+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.E4nOzeYHIBTC3wmkATFqWY2dYUUKLB__0 2024-04-22T20:48:16.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.952+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13335631488696267163', time=2024-04-22T22:26:24.055090+0000, chain.objs.size()=1 2024-04-22T20:48:16.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.952+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.y6IUDtJf6Fnah3o4v1RWzw0PMAN5YW9_0 2024-04-22T20:48:16.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6105183760814193653', time=2024-04-22T22:26:24.550152+0000, chain.objs.size()=1 2024-04-22T20:48:16.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.phZUVqDBvt8CNlAgi6Ny00dKnpLfaUQ_0 2024-04-22T20:48:16.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1892481817214054007', time=2024-04-22T22:26:24.590603+0000, chain.objs.size()=1 2024-04-22T20:48:16.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1ITV5D1FRyDByuHTF9vWWKRihb8QPhN_0 2024-04-22T20:48:16.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7980143267125964782', time=2024-04-22T22:26:25.127796+0000, chain.objs.size()=1 2024-04-22T20:48:16.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.d00K9fPjl6D_hqZLa8i-83oXRJwASSJ_0 2024-04-22T20:48:16.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9851293086158874098', time=2024-04-22T22:26:25.862332+0000, chain.objs.size()=1 2024-04-22T20:48:16.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.fwbSskIlqLUDTFcZY81ZguaJhoT_iDK_0 2024-04-22T20:48:16.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12254480619083938325', time=2024-04-22T22:26:27.438849+0000, chain.objs.size()=1 2024-04-22T20:48:16.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.954+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LIglFQpXym-7Q6HTY7bwqGXPVzOz-hD_0 2024-04-22T20:48:16.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.956+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15737928268498399721', time=2024-04-22T22:26:27.827609+0000, chain.objs.size()=1 2024-04-22T20:48:16.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.956+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Uars3K4Pn5ogcYvfzj3MaUsrrBQR7a0_0 2024-04-22T20:48:16.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17374954600957766589', time=2024-04-22T22:26:28.561213+0000, chain.objs.size()=1 2024-04-22T20:48:16.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.TJCJj9tROXYcZDovrvIBn06JJx1dpNr_0 2024-04-22T20:48:16.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1425628449453541128', time=2024-04-22T22:26:29.184930+0000, chain.objs.size()=1 2024-04-22T20:48:16.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wYE3N2a7QZSA984uTli-84ZyoaFy2Dk_0 2024-04-22T20:48:16.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7522396894732062627', time=2024-04-22T22:26:29.752669+0000, chain.objs.size()=1 2024-04-22T20:48:16.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.QCdLZGMCLlCybXLKdygplGzblTJUPTr_0 2024-04-22T20:48:16.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13236557553878755842', time=2024-04-22T22:26:30.893408+0000, chain.objs.size()=1 2024-04-22T20:48:16.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.t9d5-X3IPwb0ISOV-mg5bpr3qDlI1hp_0 2024-04-22T20:48:16.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12986623829664197879', time=2024-04-22T22:26:31.623202+0000, chain.objs.size()=1 2024-04-22T20:48:16.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.957+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Ux57iMZuINUNDZ9lByVbPNklyh8CWew_0 2024-04-22T20:48:16.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11133709706151622095', time=2024-04-22T22:26:31.696742+0000, chain.objs.size()=1 2024-04-22T20:48:16.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.n_UqkckBgAP4zTDsxcRezqYB_-qFsZ3_0 2024-04-22T20:48:16.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3554258399488323194', time=2024-04-22T22:26:32.900696+0000, chain.objs.size()=1 2024-04-22T20:48:16.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tNq9x2bGr8MlkaBq16UBKlu4i3XhBgG_0 2024-04-22T20:48:16.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6242878528465865578', time=2024-04-22T22:26:32.901444+0000, chain.objs.size()=1 2024-04-22T20:48:16.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uUUCa04NkdSJoZ5DrhsgKI4AGI0YVfz_0 2024-04-22T20:48:16.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17382982425806683523', time=2024-04-22T22:26:33.098832+0000, chain.objs.size()=1 2024-04-22T20:48:16.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ku4Rriaks1qxhdtirPCPzAXKozyeW5o_0 2024-04-22T20:48:16.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6203729472797763305', time=2024-04-22T22:26:33.522364+0000, chain.objs.size()=1 2024-04-22T20:48:16.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.958+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._8sSUVKrxci2HfuThVpND6tqGfPHAhk_0 2024-04-22T20:48:16.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.959+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13314207850228291602', time=2024-04-22T22:26:33.603081+0000, chain.objs.size()=1 2024-04-22T20:48:16.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.959+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hFsM5lFU6nNjxEVExjOCytX8ZUuuNVM_0 2024-04-22T20:48:16.959 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13158731761644700469', time=2024-04-22T22:26:33.755799+0000, chain.objs.size()=1 2024-04-22T20:48:16.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EHKe7shlw9VAycwlkMFAZz7bwqtJiYA_0 2024-04-22T20:48:16.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4093976805518653049', time=2024-04-22T22:26:33.880027+0000, chain.objs.size()=1 2024-04-22T20:48:16.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mWOnNzlugkg1nIL2Eha8oS7KdsxeRyh_0 2024-04-22T20:48:16.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9025260818610102158', time=2024-04-22T22:26:33.983004+0000, chain.objs.size()=1 2024-04-22T20:48:16.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oZk5dsGhaDn0B6rizRGad05UKybArRo_0 2024-04-22T20:48:16.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17853858042588396609', time=2024-04-22T22:26:34.183248+0000, chain.objs.size()=1 2024-04-22T20:48:16.960 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.teuE8-a1nbsaKsAJPLjHdPiZwtSw_1J_0 2024-04-22T20:48:16.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1474980507409533047', time=2024-04-22T22:26:39.252540+0000, chain.objs.size()=1 2024-04-22T20:48:16.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.961+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.256__shadow_.w-U_f82aCq4FVYC4Mzclb16LksyVc6r_0 2024-04-22T20:48:16.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.962+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13108528206874040914', time=2024-04-22T22:26:50.894613+0000, chain.objs.size()=1 2024-04-22T20:48:16.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.962+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.268__shadow_.IY0G_XM8dlG93Cz4r8qysO1AnwUHOys_0 2024-04-22T20:48:16.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.962+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4525482947185734330', time=2024-04-22T22:26:51.889278+0000, chain.objs.size()=1 2024-04-22T20:48:16.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.962+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.269__shadow_.8q--BWvecOExb5Z20MuqXEjwTEZ4eQy_0 2024-04-22T20:48:16.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.962+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16955248458880534246', time=2024-04-22T22:27:00.968681+0000, chain.objs.size()=1 2024-04-22T20:48:16.961 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.962+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.278__shadow_.h9_dPhcdBmV3vXbD1fuckd4NNNUKZ_j_0 2024-04-22T20:48:16.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.962+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8517993751492152159', time=2024-04-22T22:27:07.948903+0000, chain.objs.size()=1 2024-04-22T20:48:16.962 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.962+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.Vqir25Yr_umi09WfBzSw25Q9lqo48Ks_0 2024-04-22T20:48:16.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4243020533239996710', time=2024-04-22T22:27:07.959816+0000, chain.objs.size()=1 2024-04-22T20:48:16.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.73sQW1RjacwYdqrG4X-tGJButpU5qUZ_0 2024-04-22T20:48:16.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4639391325503390698', time=2024-04-22T22:28:17.550807+0000, chain.objs.size()=1 2024-04-22T20:48:16.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FEJ0S3Ig1FFuRYtTP2h3569mU0y36U0_0 2024-04-22T20:48:16.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4934869086855430294', time=2024-04-22T22:28:17.795691+0000, chain.objs.size()=1 2024-04-22T20:48:16.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3uwVu6ElziVwH6qRKc0YMZFPh3v6R3o_0 2024-04-22T20:48:16.963 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9037528185166072457', time=2024-04-22T22:28:18.075856+0000, chain.objs.size()=1 2024-04-22T20:48:16.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cmb0qjiq0rjsJ_rHkHm9MGV2GAqPgcx_0 2024-04-22T20:48:16.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9117563375258835903', time=2024-04-22T22:28:18.227074+0000, chain.objs.size()=1 2024-04-22T20:48:16.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IsnXt5GJKUC_pbFi9qhANxKwJzUH1Jj_0 2024-04-22T20:48:16.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7214690900724824557', time=2024-04-22T22:28:18.227637+0000, chain.objs.size()=1 2024-04-22T20:48:16.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UAWTs0S6Atgd_H7vlygqGaVaLETsoLA_0 2024-04-22T20:48:16.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2902813333865706328', time=2024-04-22T22:28:19.085462+0000, chain.objs.size()=1 2024-04-22T20:48:16.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.QnnHzaBkCxZYObBB5d04-9f8VWcWWYa_0 2024-04-22T20:48:16.964 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2447710783639667781', time=2024-04-22T22:28:19.717723+0000, chain.objs.size()=1 2024-04-22T20:48:16.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7BWglqmapkztuh3h8Mfz8vOW_famvf9_0 2024-04-22T20:48:16.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3253503671172825504', time=2024-04-22T22:28:20.677197+0000, chain.objs.size()=1 2024-04-22T20:48:16.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.965+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LJ2htBE3PqfSxJm7BcPDXV-sbqi0TJh_0 2024-04-22T20:48:16.965 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.967+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3501327559944694186', time=2024-04-22T22:28:20.897674+0000, chain.objs.size()=1 2024-04-22T20:48:16.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.967+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GqkwdG7RifBgS-J0Kxm4MfnbPuNaRi-_0 2024-04-22T20:48:16.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.967+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18134340267841309817', time=2024-04-22T22:28:21.379402+0000, chain.objs.size()=1 2024-04-22T20:48:16.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.967+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.feGJ2jXkx7Viqd9fFTRE9Ob0EGzE-fj_0 2024-04-22T20:48:16.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.969+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9468790437354223788', time=2024-04-22T22:28:21.484417+0000, chain.objs.size()=1 2024-04-22T20:48:16.966 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.969+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dNIeSjb31y8ebh1_B5PSUC6tJ3mK_NV_0 2024-04-22T20:48:16.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14337288621800115576', time=2024-04-22T22:28:21.529682+0000, chain.objs.size()=1 2024-04-22T20:48:16.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gJ_NSxSo_Tww3JmOM-pdOOCrfWBDwKR_0 2024-04-22T20:48:16.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7052406758959702708', time=2024-04-22T22:28:21.645856+0000, chain.objs.size()=1 2024-04-22T20:48:16.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Q3laNQY-g2Yp9A2H7jedA-yF690bXw2_0 2024-04-22T20:48:16.967 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10995293876567796848', time=2024-04-22T22:28:21.806371+0000, chain.objs.size()=1 2024-04-22T20:48:16.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bQyJpqbynzdVk8JOZdXrRKCa6PlDyMs_0 2024-04-22T20:48:16.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15583693074396633708', time=2024-04-22T22:28:21.871930+0000, chain.objs.size()=1 2024-04-22T20:48:16.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rOHQNhPc6vz271o_7LV5EtrbDdA2cKy_0 2024-04-22T20:48:16.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9282975987094104819', time=2024-04-22T22:28:22.737120+0000, chain.objs.size()=1 2024-04-22T20:48:16.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.l9CUGRwVVbnx9f4ClweIC1lhPjfkLzb_0 2024-04-22T20:48:16.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2323008329304438954', time=2024-04-22T22:28:22.813674+0000, chain.objs.size()=1 2024-04-22T20:48:16.968 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.AcYsnn2nTj04A25tlTAdG1Grtx4Za74_0 2024-04-22T20:48:16.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5564808090795185006', time=2024-04-22T22:28:23.083695+0000, chain.objs.size()=1 2024-04-22T20:48:16.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JghJI6jhvocwiQ5-nPwrEkf2OUwXG9C_0 2024-04-22T20:48:16.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8710429514736893517', time=2024-04-22T22:28:23.243180+0000, chain.objs.size()=1 2024-04-22T20:48:16.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.C0wypoWJ4Stiq8vxcLpHc83LS_oC4UT_0 2024-04-22T20:48:16.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11788594674507563734', time=2024-04-22T22:28:24.059694+0000, chain.objs.size()=1 2024-04-22T20:48:16.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KgjEz3AGsTA4kPyDSlVY52r7MaGCPjw_0 2024-04-22T20:48:16.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5151454237878309577', time=2024-04-22T22:28:24.419022+0000, chain.objs.size()=1 2024-04-22T20:48:16.969 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.970+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TBfIaHdLN6V3fLhcV_MkQP1Cz8BTXCG_0 2024-04-22T20:48:16.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.972+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11891712560985870086', time=2024-04-22T22:28:24.612542+0000, chain.objs.size()=1 2024-04-22T20:48:16.970 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.972+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZUIbjCQtoH6wx5SzWLclaDVV5etIADx_0 2024-04-22T20:48:16.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14564231659286039418', time=2024-04-22T22:28:24.699851+0000, chain.objs.size()=1 2024-04-22T20:48:16.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.BiX6HWNIzffDutigFljS4keGSvXXx0Q_0 2024-04-22T20:48:16.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11715629667944142349', time=2024-04-22T22:28:24.899995+0000, chain.objs.size()=1 2024-04-22T20:48:16.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IKlvN4eJoKEqISI7lDmAn8cc5KZT8Yl_0 2024-04-22T20:48:16.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7549919080149369981', time=2024-04-22T22:28:25.729444+0000, chain.objs.size()=1 2024-04-22T20:48:16.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.A1XJT4PrMO_-0VH2xH8Ryh-gt4Tt4vI_0 2024-04-22T20:48:16.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16765190543430017475', time=2024-04-22T22:28:25.794397+0000, chain.objs.size()=1 2024-04-22T20:48:16.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nSDCSQtN0rCG79cGv1dPJ8omzKVnIlp_0 2024-04-22T20:48:16.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16894087068297724651', time=2024-04-22T22:28:25.943005+0000, chain.objs.size()=1 2024-04-22T20:48:16.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pkz-K6X3Ein95o4MTpjydJctSiP7PT2_0 2024-04-22T20:48:16.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1495150102250392963', time=2024-04-22T22:28:26.095933+0000, chain.objs.size()=1 2024-04-22T20:48:16.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.VJ-k4pU8nWhlR4FmPnrzwnRmL27cAR6_0 2024-04-22T20:48:16.972 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2491276792469780720', time=2024-04-22T22:28:26.531602+0000, chain.objs.size()=1 2024-04-22T20:48:16.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.K-0HZZx-UsXksn-YUHdS-oQfVNvptHb_0 2024-04-22T20:48:16.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8205616190226677578', time=2024-04-22T22:28:27.638465+0000, chain.objs.size()=1 2024-04-22T20:48:16.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Pnbp-1buGDxbJUk7wqKy6leTIX79hca_0 2024-04-22T20:48:16.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1916935127846302304', time=2024-04-22T22:29:06.307539+0000, chain.objs.size()=1 2024-04-22T20:48:16.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.9A8Db4FrTZUEGCQMWesmZG1K3_xkzWa_0 2024-04-22T20:48:16.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2879448498870614670', time=2024-04-22T22:29:06.456902+0000, chain.objs.size()=1 2024-04-22T20:48:16.973 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.974+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yiDE2KUd1Zl1xogpMQsuD9uWT0frIEL_0 2024-04-22T20:48:16.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.975+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.956090079056494719', time=2024-04-22T22:29:06.598003+0000, chain.objs.size()=1 2024-04-22T20:48:16.974 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.975+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mqFK8AeNBgWN8rsjWSCPPvfG2qygMiT_0 2024-04-22T20:48:16.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16308035098735817748', time=2024-04-22T22:29:07.808652+0000, chain.objs.size()=1 2024-04-22T20:48:16.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wNfiMzIzNqfQ4U5Z2wV--MFt38Jp-Qg_0 2024-04-22T20:48:16.975 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4171196195164758482', time=2024-04-22T22:29:07.962646+0000, chain.objs.size()=1 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.S2Zr4ZPQT7zcX-nfM8gd8_qNYM1JOoO_0 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8134840493802710098', time=2024-04-22T22:29:08.793040+0000, chain.objs.size()=1 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jT37Z89u7C-kgU_fmw-C2BoDF7lYtf5_0 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16254135221087726602', time=2024-04-22T22:29:08.888587+0000, chain.objs.size()=1 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pXBPiVy3S3UblaSOKeTmO6EekzNeB-Q_0 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15302255157754573757', time=2024-04-22T22:29:09.358195+0000, chain.objs.size()=1 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KqTrCsi_s65hCG9uuG1hVahmf-Rqy3E_0 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3347302762904826748', time=2024-04-22T22:29:10.456047+0000, chain.objs.size()=1 2024-04-22T20:48:16.976 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.978+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.9HV3-0KBON_rV0TpXzUjxtcttcNMJJV_0 2024-04-22T20:48:16.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.979+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3763662564548470622', time=2024-04-22T22:29:11.417423+0000, chain.objs.size()=1 2024-04-22T20:48:16.977 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.979+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.f1pCH4yxluxCwC4GY9v1JRp9QM2Oexl_0 2024-04-22T20:48:16.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13005853431344239288', time=2024-04-22T22:29:11.545808+0000, chain.objs.size()=1 2024-04-22T20:48:16.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.svGD76MfR9GZnqvsjc52CQUIkGfFub3_0 2024-04-22T20:48:16.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6291431533715752574', time=2024-04-22T22:29:11.643207+0000, chain.objs.size()=1 2024-04-22T20:48:16.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.EO1yNppunO1ZI2RUobr635_F71mdw1M_0 2024-04-22T20:48:16.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9079536888945082097', time=2024-04-22T22:29:11.783554+0000, chain.objs.size()=1 2024-04-22T20:48:16.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JrGuLTJThZ5mcVF_3hFVM0Yxw1yJuih_0 2024-04-22T20:48:16.978 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8077645695471877115', time=2024-04-22T22:29:11.825730+0000, chain.objs.size()=1 2024-04-22T20:48:16.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DUQJ_EdMVzooWdwG5cs0pxL4dNpzqYV_0 2024-04-22T20:48:16.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4379954542413641608', time=2024-04-22T22:29:12.718790+0000, chain.objs.size()=1 2024-04-22T20:48:16.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.981+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.crBbPxoJ7aSaJh7x-3ZFaCmUzl14QQb_0 2024-04-22T20:48:16.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13465743303397915146', time=2024-04-22T22:29:12.955438+0000, chain.objs.size()=1 2024-04-22T20:48:16.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hJuWuvZIaTZ5PgEf_ZtTW8nuj7XiAZa_0 2024-04-22T20:48:16.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9210090669977476551', time=2024-04-22T22:29:13.905029+0000, chain.objs.size()=1 2024-04-22T20:48:16.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HbaD7rhyQkJgiREZD7wF6GMBOti6-Us_0 2024-04-22T20:48:16.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10847142400924717857', time=2024-04-22T22:29:14.040738+0000, chain.objs.size()=1 2024-04-22T20:48:16.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6GHD19cH8DQGRbpJwwCyM-BclEyNF1t_0 2024-04-22T20:48:16.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11894623787762657607', time=2024-04-22T22:29:14.562750+0000, chain.objs.size()=1 2024-04-22T20:48:16.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2SrS4tZD5bcbYpoGhS4fkSrAYAOf8DY_0 2024-04-22T20:48:16.980 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15153653140811422660', time=2024-04-22T22:29:15.346111+0000, chain.objs.size()=1 2024-04-22T20:48:16.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zTEl_EySMJSpJjLNlYI-9TqT56Vvlsp_0 2024-04-22T20:48:16.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12911029822981056298', time=2024-04-22T22:29:15.639975+0000, chain.objs.size()=1 2024-04-22T20:48:16.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.982+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wf5UkePOWLSbGhOzVdXN6tCuU1NxGwb_0 2024-04-22T20:48:16.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.984+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6605604639437585368', time=2024-04-22T22:29:15.835068+0000, chain.objs.size()=1 2024-04-22T20:48:16.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.984+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-eOPTWwI683GT6lSJVq59yZEqfbK0Q9_0 2024-04-22T20:48:16.983 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.987+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/45694 2024-04-22T20:48:16.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=17, truncated=0, next_marker='' 2024-04-22T20:48:16.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12160092809430217596', time=2024-04-22T22:29:16.760522+0000, chain.objs.size()=1 2024-04-22T20:48:16.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QtRFyDDSb6ySfhzExOdL_BMNAHddXCv_0 2024-04-22T20:48:16.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10682191808779459139', time=2024-04-22T22:29:16.915782+0000, chain.objs.size()=1 2024-04-22T20:48:16.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wst_sHGM_QteociNKOi72Za5J9e7XYL_0 2024-04-22T20:48:16.986 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1237998416410320197', time=2024-04-22T22:29:16.978616+0000, chain.objs.size()=1 2024-04-22T20:48:16.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zZvOgP9Vk9mVxKpahuCkXRZa5lfQ1-b_0 2024-04-22T20:48:16.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13979788579883617327', time=2024-04-22T22:29:17.109765+0000, chain.objs.size()=1 2024-04-22T20:48:16.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.L21iEPfGQR3HhWfhLgaDoqOq4NlHnw8_0 2024-04-22T20:48:16.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10566735945930287848', time=2024-04-22T22:29:19.208679+0000, chain.objs.size()=1 2024-04-22T20:48:16.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.315__shadow_.B72sMe2gieFD4wAD8A9Wzi9qts634DT_0 2024-04-22T20:48:16.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7565969150640993343', time=2024-04-22T22:29:21.802654+0000, chain.objs.size()=1 2024-04-22T20:48:16.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.319__shadow_.BjqOPqEUC7AjL4CeEEXIyBW48QTaA5V_0 2024-04-22T20:48:16.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12042590636349274023', time=2024-04-22T22:30:46.094517+0000, chain.objs.size()=1 2024-04-22T20:48:16.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.440__shadow_.pd5xI_RTwpIjRfrFyveeBFP0MI5YmlI_0 2024-04-22T20:48:16.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4237156761777722754', time=2024-04-22T22:30:57.285186+0000, chain.objs.size()=1 2024-04-22T20:48:16.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.451__shadow_.bHaMyNzBbqUst2Dy7kNnm4BWKoF2t6w_0 2024-04-22T20:48:16.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15414025216183515445', time=2024-04-22T22:34:12.860406+0000, chain.objs.size()=6 2024-04-22T20:48:16.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~4pUdQTeLgN3G-hD6BiagOzG8RRk464H.1 2024-04-22T20:48:16.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~4pUdQTeLgN3G-hD6BiagOzG8RRk464H.2 2024-04-22T20:48:16.988 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.93d5kd5KfUP3gx0Uy6rz-4ZuzqsfB4mx.3 2024-04-22T20:48:16.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.989+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~4pUdQTeLgN3G-hD6BiagOzG8RRk464H.4 2024-04-22T20:48:16.989 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.991+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~4pUdQTeLgN3G-hD6BiagOzG8RRk464H.5 2024-04-22T20:48:16.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~4pUdQTeLgN3G-hD6BiagOzG8RRk464H.6 2024-04-22T20:48:16.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3744661914334258514', time=2024-04-22T22:36:56.087550+0000, chain.objs.size()=1 2024-04-22T20:48:16.990 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.544__shadow_.PaJseHibixMiMlxydXxOdEojjeZfEYr_0 2024-04-22T20:48:16.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3988927974680410028', time=2024-04-22T22:37:29.254687+0000, chain.objs.size()=1 2024-04-22T20:48:16.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:Q6Ehb76Y6wq4jJ4izqkCqlmK28UPcdv_.NBHJF4vMc79hnbZGCcoT9W5s5wLm4O1_0 2024-04-22T20:48:16.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14309568466906047964', time=2024-04-22T22:37:35.790429+0000, chain.objs.size()=1 2024-04-22T20:48:16.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.560__shadow_.p5d0j6XQkKdFRs2BDqPoNy1IndBB4Y6_0 2024-04-22T20:48:16.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11679977921991474445', time=2024-04-22T22:37:47.888582+0000, chain.objs.size()=1 2024-04-22T20:48:16.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:SzOctklS-6khog8D3ktZCYcbA11oqDn_.ycD8bfJZINfPtskL4r8bGItu3dlHMfy_0 2024-04-22T20:48:16.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3894448385152328985', time=2024-04-22T22:38:05.660595+0000, chain.objs.size()=1 2024-04-22T20:48:16.991 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:BzpLrdn9Cd10Hc0n0-Jx4ncFsXE7ptd_.pxa0Zbcb_rmS9pLXySXi6MdIVnYjXqU_0 2024-04-22T20:48:16.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10079766511925223337', time=2024-04-22T22:41:51.854639+0000, chain.objs.size()=1 2024-04-22T20:48:16.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.617__shadow_.jXczUYirGsd-PKkQQZwarOBi2QEGaMW_0 2024-04-22T20:48:16.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7847025374847521174', time=2024-04-22T22:42:27.674393+0000, chain.objs.size()=1 2024-04-22T20:48:16.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.640__shadow_.wvwsXmBb6_b78er6d4f5RpNpNmFO9Tv_0 2024-04-22T20:48:16.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15691908162722977977', time=2024-04-22T22:43:24.451978+0000, chain.objs.size()=1 2024-04-22T20:48:16.992 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.993+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.697__shadow:pEf-gwBa4Z8tlrsAwq9bLh40VuPzTLv_.NdeqngSVtcVraSAKuew_aOdOIqXT2-__0 2024-04-22T20:48:16.993 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.997+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:16.995 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:16.999+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-22T20:48:16.997 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.000+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:16.997 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:16.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/49288' 2024-04-22T20:48:16.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8612210156249699434', time=2024-04-22T22:23:18.730667+0000, chain.objs.size()=1 2024-04-22T20:48:16.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.188__shadow_.fgqc-MlaGsaRIjLeEkh_znAfeAtQ3Ck_0 2024-04-22T20:48:16.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11152578796193065711', time=2024-04-22T22:25:19.961638+0000, chain.objs.size()=1 2024-04-22T20:48:16.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.226__shadow_.q__SDTS25YMsPEFOQSettEc3DBLhIqG_0 2024-04-22T20:48:16.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2118672885941304951', time=2024-04-22T22:25:21.708088+0000, chain.objs.size()=1 2024-04-22T20:48:16.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.227__shadow_.TBPot1fz4y1NDeHt1TYm-kR73vr3cRq_0 2024-04-22T20:48:16.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15447865910579037217', time=2024-04-22T22:25:24.429500+0000, chain.objs.size()=1 2024-04-22T20:48:17.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.230__shadow_.aMlECBvu6qjRGZ1Zct9EDWq3MOY6RyO_0 2024-04-22T20:48:17.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16227797886832939233', time=2024-04-22T22:26:22.666127+0000, chain.objs.size()=1 2024-04-22T20:48:17.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VhJykUu5ONAblyrgabGax_QrG08DaQ5_0 2024-04-22T20:48:17.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9764635011111156667', time=2024-04-22T22:26:22.765881+0000, chain.objs.size()=1 2024-04-22T20:48:17.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oEDY65Z7DN-yPtlHHJM8I592PGKxdhy_0 2024-04-22T20:48:17.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7337036359495227427', time=2024-04-22T22:26:23.385091+0000, chain.objs.size()=1 2024-04-22T20:48:17.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.yVSlXdReQjNLc_wAinMRYePpi3e3oJo_0 2024-04-22T20:48:17.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2099139558977194846', time=2024-04-22T22:26:23.991205+0000, chain.objs.size()=1 2024-04-22T20:48:17.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JWjfNtVD4rvsUqxZ99fFZUUy1arjNzT_0 2024-04-22T20:48:17.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10622477483459481735', time=2024-04-22T22:26:24.151396+0000, chain.objs.size()=1 2024-04-22T20:48:17.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.iT31q37_M0laujcugrfgTy3SM4MQqDK_0 2024-04-22T20:48:17.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4522419570001851759', time=2024-04-22T22:26:24.209869+0000, chain.objs.size()=1 2024-04-22T20:48:17.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wJE7vtt9Wc4UwwxuubuvNEocYTuXBSv_0 2024-04-22T20:48:17.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14805763970882857949', time=2024-04-22T22:26:24.985126+0000, chain.objs.size()=1 2024-04-22T20:48:17.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.s9zdqzDM2dOys456waYcaciW3co3E0f_0 2024-04-22T20:48:17.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3904992571574689006', time=2024-04-22T22:26:25.023876+0000, chain.objs.size()=1 2024-04-22T20:48:17.002 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.001+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._22aYRvK9ooEEq4nixP7Sob_Gy1rhBb_0 2024-04-22T20:48:17.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10449897032659287843', time=2024-04-22T22:26:25.685266+0000, chain.objs.size()=1 2024-04-22T20:48:17.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3533DCruSr1xjnqygOKRjc4ttw9Uem3_0 2024-04-22T20:48:17.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.005+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.308061933617354620', time=2024-04-22T22:26:25.971098+0000, chain.objs.size()=1 2024-04-22T20:48:17.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.005+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jJfi5f3JwgbppRVr5HtId3LWnO9O5je_0 2024-04-22T20:48:17.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7659641380931234065', time=2024-04-22T22:26:26.989399+0000, chain.objs.size()=1 2024-04-22T20:48:17.003 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rxWMPZ_j0No20bNg_XdQ48F0t5Vmu6A_0 2024-04-22T20:48:17.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16904362658248409344', time=2024-04-22T22:26:27.730718+0000, chain.objs.size()=1 2024-04-22T20:48:17.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.g5wJLSD3iKFvKauqHkEoukTLdPRdoIY_0 2024-04-22T20:48:17.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13126399022739855250', time=2024-04-22T22:26:29.055957+0000, chain.objs.size()=1 2024-04-22T20:48:17.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5hXJWUJInumCDY5jdsdMobZo3JsaTRS_0 2024-04-22T20:48:17.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18212791822379915731', time=2024-04-22T22:26:29.230278+0000, chain.objs.size()=1 2024-04-22T20:48:17.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LoZUEjLbMt1hluWJUYfD0D-RzFcsh8B_0 2024-04-22T20:48:17.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15944068318769820403', time=2024-04-22T22:26:29.271990+0000, chain.objs.size()=1 2024-04-22T20:48:17.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EYUfc7F773nZos9DbGVGRAKsgvCYP5g_0 2024-04-22T20:48:17.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13575253942240113838', time=2024-04-22T22:26:29.527483+0000, chain.objs.size()=1 2024-04-22T20:48:17.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6C2ZCOBZN5dNIayw0mfm4mUptMmsaVE_0 2024-04-22T20:48:17.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17145168431786936209', time=2024-04-22T22:26:29.750485+0000, chain.objs.size()=1 2024-04-22T20:48:17.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LPUQt5z_kq3ZN8UPjN8aS5_jtcuktsS_0 2024-04-22T20:48:17.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2929672972443203704', time=2024-04-22T22:26:30.041144+0000, chain.objs.size()=1 2024-04-22T20:48:17.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.53N2tzAMuX576lKMOJ3bLPFS5tOeR6t_0 2024-04-22T20:48:17.005 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.309741991941851852', time=2024-04-22T22:26:30.755648+0000, chain.objs.size()=1 2024-04-22T20:48:17.006 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.006+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._DUVivS8sQUb5Nwczw3064_XOAjaPBN_0 2024-04-22T20:48:17.006 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.009+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8627751778091552351', time=2024-04-22T22:26:31.201570+0000, chain.objs.size()=1 2024-04-22T20:48:17.006 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.009+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.AN970nBybahSbdsfF1Zyj_mrN2tnUk5_0 2024-04-22T20:48:17.006 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.009+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1461767235623980326', time=2024-04-22T22:26:31.341095+0000, chain.objs.size()=1 2024-04-22T20:48:17.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.009+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Pe9X8yEM6GQQOsymsiMZveOUZHcVPKX_0 2024-04-22T20:48:17.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.009+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8669340042084339586', time=2024-04-22T22:26:31.662311+0000, chain.objs.size()=1 2024-04-22T20:48:17.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.009+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zT7-nXQpvmAGTh1tfRiGhlJ5xC2zEbA_0 2024-04-22T20:48:17.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.009+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6693393240328846437', time=2024-04-22T22:26:32.052736+0000, chain.objs.size()=1 2024-04-22T20:48:17.007 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.009+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HkhH2yxCD-_dciqrZ6QxhoTYuvakwPJ_0 2024-04-22T20:48:17.008 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.010+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11259040443956916479', time=2024-04-22T22:26:32.139215+0000, chain.objs.size()=1 2024-04-22T20:48:17.008 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.010+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VAdv4UsCA9_NsIdz63HF2OwvAsYTWEl_0 2024-04-22T20:48:17.008 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.012+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5983243099436713536', time=2024-04-22T22:26:32.208503+0000, chain.objs.size()=1 2024-04-22T20:48:17.008 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.012+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WQyC2I_uyfyWONZ-60fb5y9e1PFm8yv_0 2024-04-22T20:48:17.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.012+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11686875463664398930', time=2024-04-22T22:26:32.436517+0000, chain.objs.size()=1 2024-04-22T20:48:17.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.012+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.T-lfgaktx_KefUqri6XyW54sFaNHkPR_0 2024-04-22T20:48:17.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12304107138335342715', time=2024-04-22T22:26:34.274465+0000, chain.objs.size()=1 2024-04-22T20:48:17.009 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3CU93AqFQ0X-yONpMsY0Kfe7WbYQ7mK_0 2024-04-22T20:48:17.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14729444250781581848', time=2024-04-22T22:26:34.299702+0000, chain.objs.size()=1 2024-04-22T20:48:17.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-FjWlrvUy0L0uRBgPC86LLumVMoQQon_0 2024-04-22T20:48:17.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6324527949551740447', time=2024-04-22T22:26:34.360526+0000, chain.objs.size()=1 2024-04-22T20:48:17.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZxQOotOc9jaGWlvggLJ-oveB0O9J0S-_0 2024-04-22T20:48:17.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17264489679486718322', time=2024-04-22T22:26:38.309547+0000, chain.objs.size()=1 2024-04-22T20:48:17.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.255__shadow_.-jaulQg7YArXA4ao4NXQDQinIGF2hy7_0 2024-04-22T20:48:17.010 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8740436379653460849', time=2024-04-22T22:26:43.030995+0000, chain.objs.size()=1 2024-04-22T20:48:17.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.260__shadow_.-k1fr5Qr40rQrb3JM1fJGXJUjvbyfcG_0 2024-04-22T20:48:17.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13385944261044982198', time=2024-04-22T22:26:52.871238+0000, chain.objs.size()=1 2024-04-22T20:48:17.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.270__shadow_.sb2Bgs5rso9INM9kktT8WnAImgMBcDW_0 2024-04-22T20:48:17.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5812082148583537127', time=2024-04-22T22:26:58.977891+0000, chain.objs.size()=1 2024-04-22T20:48:17.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.276__shadow_.6MwAQaxAOzmSZc7AIzhdITM7fsIVro2_0 2024-04-22T20:48:17.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1595678292162269925', time=2024-04-22T22:27:07.853524+0000, chain.objs.size()=1 2024-04-22T20:48:17.011 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.013+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.DxnfVmOjefESA58yytWIoD8-LUTzADB_0 2024-04-22T20:48:17.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.015+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8063482524105882586', time=2024-04-22T22:27:07.854271+0000, chain.objs.size()=1 2024-04-22T20:48:17.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.015+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.bFhUrApVVLbvUTKfZDEax6mil5tZCRy_0 2024-04-22T20:48:17.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.015+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.872213852473256565', time=2024-04-22T22:27:14.100208+0000, chain.objs.size()=1 2024-04-22T20:48:17.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.015+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.288__shadow_.8mgxP7fS42mKcfjfm2ZpEaxITfyMnDM_0 2024-04-22T20:48:17.012 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.015+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18287978596404637663', time=2024-04-22T22:27:23.042082+0000, chain.objs.size()=1 2024-04-22T20:48:17.013 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.015+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.296__shadow:CktFPTaK9sfgG.GMTnwq4Cm6Ismhzby_.8Wzir3_VpLRujJbAINKl1ZM_H6hJx1C_0 2024-04-22T20:48:17.013 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.016+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2065621305332206471', time=2024-04-22T22:28:17.646134+0000, chain.objs.size()=1 2024-04-22T20:48:17.013 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.016+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8pZ5XqPeCT5Q-VWTkF2pzD2CKDeebQg_0 2024-04-22T20:48:17.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.017+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1183085491930966778', time=2024-04-22T22:28:18.366041+0000, chain.objs.size()=1 2024-04-22T20:48:17.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.017+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.y465byUUgH3m_7ljc-FaFvcMc-2P4aX_0 2024-04-22T20:48:17.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.017+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3285421450712604374', time=2024-04-22T22:28:18.432769+0000, chain.objs.size()=1 2024-04-22T20:48:17.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.017+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.DqGPNaPzFpjzYMS5bdGqrBpSYqojaN6_0 2024-04-22T20:48:17.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.017+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13830997024550467643', time=2024-04-22T22:28:19.061071+0000, chain.objs.size()=1 2024-04-22T20:48:17.014 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.017+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.f9tSfbj6mM9gEUH15FtKowYZFestmoT_0 2024-04-22T20:48:17.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.017+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6679097621872413133', time=2024-04-22T22:28:19.244796+0000, chain.objs.size()=1 2024-04-22T20:48:17.015 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.017+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CgYzTnrq8TwP7APOvtqAY-2qSMAs2jr_0 2024-04-22T20:48:17.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6525011432089578525', time=2024-04-22T22:28:19.245433+0000, chain.objs.size()=1 2024-04-22T20:48:17.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6cwBIhe__GF9jub7e3IlxwGcQdzd6EA_0 2024-04-22T20:48:17.016 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17218341372229236578', time=2024-04-22T22:28:19.767717+0000, chain.objs.size()=1 2024-04-22T20:48:17.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7SWHWVlvK6Rp7tQ0MKRePzvsdnO1BhP_0 2024-04-22T20:48:17.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2590818304708457796', time=2024-04-22T22:28:19.811356+0000, chain.objs.size()=1 2024-04-22T20:48:17.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mT3hlNp19QQGo6K-7Adwb73E_vykSlE_0 2024-04-22T20:48:17.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10020582930604736230', time=2024-04-22T22:28:19.896400+0000, chain.objs.size()=1 2024-04-22T20:48:17.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.e2tZmSb5fu2VatrtKWIDi_kRYgo06D4_0 2024-04-22T20:48:17.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8646802334068791726', time=2024-04-22T22:28:20.091064+0000, chain.objs.size()=1 2024-04-22T20:48:17.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.019+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SUjFVBMTFdyW-7Iy8RyKllXoGwBOHvj_0 2024-04-22T20:48:17.017 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.020+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9359837566657215945', time=2024-04-22T22:28:20.092160+0000, chain.objs.size()=1 2024-04-22T20:48:17.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.020+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.s2n6MW34iIIajPSpyRQO5T5HxFEHVzy_0 2024-04-22T20:48:17.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.020+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13025618099942166077', time=2024-04-22T22:28:20.209457+0000, chain.objs.size()=1 2024-04-22T20:48:17.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.020+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8dQ5BNkrV5WbaLwlZDf3_SfbH7upbbm_0 2024-04-22T20:48:17.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.020+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14901839599927713052', time=2024-04-22T22:28:20.828899+0000, chain.objs.size()=1 2024-04-22T20:48:17.018 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.020+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.jJ19K2hSmBVE3mLG-FDEA8jtcEqHEfo_0 2024-04-22T20:48:17.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2910248330477019929', time=2024-04-22T22:28:20.829508+0000, chain.objs.size()=1 2024-04-22T20:48:17.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.2VsUGHvYvY0WzM2FOw3Fz0_p_uIR_7A_0 2024-04-22T20:48:17.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16769088735645048778', time=2024-04-22T22:28:22.351091+0000, chain.objs.size()=1 2024-04-22T20:48:17.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.F-ZtJJvFWRg7tH63xbNXzGXomaPSciV_0 2024-04-22T20:48:17.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.021+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7504795167115527356', time=2024-04-22T22:28:22.686763+0000, chain.objs.size()=1 2024-04-22T20:48:17.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.021+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dxPUlD8eULTC-9SwnebMJxgNSw4g3RT_0 2024-04-22T20:48:17.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.022+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5710041452627523705', time=2024-04-22T22:28:22.800610+0000, chain.objs.size()=1 2024-04-22T20:48:17.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.022+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.K0-20AVeC0b3pQoDXL8if9v6sgEpMgJ_0 2024-04-22T20:48:17.020 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.023+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10949697358814244801', time=2024-04-22T22:28:23.076816+0000, chain.objs.size()=1 2024-04-22T20:48:17.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.023+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ID5716f50d1WXp7xk7esHmLQ58XbcqX_0 2024-04-22T20:48:17.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.023+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17694860843541816896', time=2024-04-22T22:28:23.388698+0000, chain.objs.size()=1 2024-04-22T20:48:17.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.023+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kJg2uKAL6oAgezeegvzlLOiWoydsXd8_0 2024-04-22T20:48:17.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.023+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1611860732434332560', time=2024-04-22T22:28:23.440868+0000, chain.objs.size()=1 2024-04-22T20:48:17.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.023+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OicE9vlkveUTCkDhXgBl2-vv2pf74WC_0 2024-04-22T20:48:17.021 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.023+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3864381392370561300', time=2024-04-22T22:28:23.459918+0000, chain.objs.size()=1 2024-04-22T20:48:17.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.023+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TU3t5xmz4IyZgMmDhCqbuRlywKhPvuS_0 2024-04-22T20:48:17.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11261938766403481923', time=2024-04-22T22:28:24.086828+0000, chain.objs.size()=1 2024-04-22T20:48:17.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3NNq7PssTP-lxRNUWN1uZrxLB6UpDsD_0 2024-04-22T20:48:17.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11378848987427023793', time=2024-04-22T22:28:24.474022+0000, chain.objs.size()=1 2024-04-22T20:48:17.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UDX-12Twub7BOIN-z3bHJB693cdwanA_0 2024-04-22T20:48:17.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9884266426596383282', time=2024-04-22T22:28:24.695452+0000, chain.objs.size()=1 2024-04-22T20:48:17.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4YCFrFnFKHHp2mk3d2P3FKKFvNyPUQ0_0 2024-04-22T20:48:17.022 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15360714640577267000', time=2024-04-22T22:28:24.798980+0000, chain.objs.size()=1 2024-04-22T20:48:17.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yv-_rYhvmLgXGEFcAoFjUsRvKH74X7O_0 2024-04-22T20:48:17.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2654743161151868988', time=2024-04-22T22:28:24.799552+0000, chain.objs.size()=1 2024-04-22T20:48:17.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.024+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.a61MQFdgRwfYzKTGP7nth32w9sUl934_0 2024-04-22T20:48:17.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.026+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.54602413093836437', time=2024-04-22T22:28:24.800033+0000, chain.objs.size()=1 2024-04-22T20:48:17.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ePUGCUGmbZkzaKljfbZUA2QGaui7Tr7_0 2024-04-22T20:48:17.023 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10142108821597206668', time=2024-04-22T22:28:25.415926+0000, chain.objs.size()=1 2024-04-22T20:48:17.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rTL_49cHrlTKu6_uCx4SpC0PquKTeI2_0 2024-04-22T20:48:17.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17146612716036681765', time=2024-04-22T22:28:25.960484+0000, chain.objs.size()=1 2024-04-22T20:48:17.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6dOrhImN4_pWYf9nQtQf_-ft3GIgs35_0 2024-04-22T20:48:17.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.027+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16286209128205456284', time=2024-04-22T22:28:26.071223+0000, chain.objs.size()=1 2024-04-22T20:48:17.024 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.027+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.edh9re8IGrJ427CFMEUPywIK3daXmgn_0 2024-04-22T20:48:17.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.028+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15590108639657855073', time=2024-04-22T22:28:27.291548+0000, chain.objs.size()=1 2024-04-22T20:48:17.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.028+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.24PF39O_Y4390pe67MhBji8vpWqpP3L_0 2024-04-22T20:48:17.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.028+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11900825964493832562', time=2024-04-22T22:28:27.292116+0000, chain.objs.size()=1 2024-04-22T20:48:17.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.028+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OrY39onC6ClMMHsc0V3SRd8rJOaytcc_0 2024-04-22T20:48:17.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.028+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11505057455912536467', time=2024-04-22T22:28:27.292703+0000, chain.objs.size()=1 2024-04-22T20:48:17.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.028+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.x7TEdcvB1Y4DvoBN9JEuMGmcwXFpONH_0 2024-04-22T20:48:17.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.028+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15918237554101923910', time=2024-04-22T22:28:27.894855+0000, chain.objs.size()=1 2024-04-22T20:48:17.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.028+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.F1hkBcQQHFs0ZH5Q61-60TQEjxqAw80_0 2024-04-22T20:48:17.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.029+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13808810301861474456', time=2024-04-22T22:28:28.746532+0000, chain.objs.size()=1 2024-04-22T20:48:17.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.029+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Xc_uEGjh93wDguWa4lXCBhp_kvypLmc_0 2024-04-22T20:48:17.026 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251288455547912966', time=2024-04-22T22:28:28.841822+0000, chain.objs.size()=1 2024-04-22T20:48:17.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.jbgbhTCz3zIPv4LNA7TwEu6F2W9UT4n_0 2024-04-22T20:48:17.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14495405339218396706', time=2024-04-22T22:28:28.939910+0000, chain.objs.size()=1 2024-04-22T20:48:17.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.i3pjssJ8BTVwGZdGB6bsfkX8L4xRBgh_0 2024-04-22T20:48:17.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17275084319284721125', time=2024-04-22T22:28:29.065231+0000, chain.objs.size()=1 2024-04-22T20:48:17.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.W3J_X68-1jU3z6O01yx-kOWGjyhAV0y_0 2024-04-22T20:48:17.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13611672539129577124', time=2024-04-22T22:29:06.521202+0000, chain.objs.size()=1 2024-04-22T20:48:17.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MYx3X0mhKY9PNGSG4EzkkS13ibimMRJ_0 2024-04-22T20:48:17.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6037416347449987029', time=2024-04-22T22:29:06.822342+0000, chain.objs.size()=1 2024-04-22T20:48:17.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.030+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.U85oa0oy5wgpyOaTTikMs9QR6-qPlQw_0 2024-04-22T20:48:17.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.031+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2367481388638438186', time=2024-04-22T22:29:07.073281+0000, chain.objs.size()=1 2024-04-22T20:48:17.028 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.031+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mhc-r520cnsSnC5nmzFcSdZsH5CcRsf_0 2024-04-22T20:48:17.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.031+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16570413177903680458', time=2024-04-22T22:29:07.153676+0000, chain.objs.size()=1 2024-04-22T20:48:17.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.031+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IZPBBJZbEOEXLRgH4cu4R4RnwUdhtdT_0 2024-04-22T20:48:17.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.032+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7807794514444662772', time=2024-04-22T22:29:07.806915+0000, chain.objs.size()=1 2024-04-22T20:48:17.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.032+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tV3XfQ5tBMagXZPZGhuD6DYeNQU5POR_0 2024-04-22T20:48:17.029 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.033+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14486011416316243846', time=2024-04-22T22:29:07.959953+0000, chain.objs.size()=1 2024-04-22T20:48:17.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.033+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SmlDm4LehpHf4PogI3njybbTrCcV3MF_0 2024-04-22T20:48:17.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.033+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.115861066119679661', time=2024-04-22T22:29:08.049397+0000, chain.objs.size()=1 2024-04-22T20:48:17.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.033+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JXro0LauJ0hCdKGDXZA4tLT3kDtjXNe_0 2024-04-22T20:48:17.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.033+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17226932284581949636', time=2024-04-22T22:29:08.604206+0000, chain.objs.size()=1 2024-04-22T20:48:17.030 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.033+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mGOO_yeTRZSKtvIH61b64ZyVcVrfr12_0 2024-04-22T20:48:17.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.034+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14675037260228985217', time=2024-04-22T22:29:08.697866+0000, chain.objs.size()=1 2024-04-22T20:48:17.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.034+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.S1SCx_Y3FzTL_Xxi_qJlj3Mblrl7SOJ_0 2024-04-22T20:48:17.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.034+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2113183619230546924', time=2024-04-22T22:29:09.717793+0000, chain.objs.size()=1 2024-04-22T20:48:17.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.034+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.p3QmFEXi841RUz8xl_z8EU_b8KbEuVG_0 2024-04-22T20:48:17.031 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.034+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.499496373235823438', time=2024-04-22T22:29:09.752547+0000, chain.objs.size()=1 2024-04-22T20:48:17.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.034+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rrMZe1UeVhlazGaxAwq_r4akxHkrxh1_0 2024-04-22T20:48:17.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4681681974616908320', time=2024-04-22T22:29:09.796862+0000, chain.objs.size()=1 2024-04-22T20:48:17.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TNyDnI088lUb8Iku615jWfnrxqZEHmh_0 2024-04-22T20:48:17.032 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6844558324832809724', time=2024-04-22T22:29:10.194373+0000, chain.objs.size()=1 2024-04-22T20:48:17.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Bs4lNVgocvzGSx_UWrU1p213vCWhM88_0 2024-04-22T20:48:17.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3425781266426617359', time=2024-04-22T22:29:10.389591+0000, chain.objs.size()=1 2024-04-22T20:48:17.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7N2XJYDMveV2sw11ml7o4B4UY20KDg5_0 2024-04-22T20:48:17.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3247058964330456929', time=2024-04-22T22:29:10.453470+0000, chain.objs.size()=1 2024-04-22T20:48:17.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.g2MQ52XD_ZTaDV9wFjdrhcIHM-3T6HE_0 2024-04-22T20:48:17.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17965927490498009192', time=2024-04-22T22:29:11.048145+0000, chain.objs.size()=1 2024-04-22T20:48:17.033 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.035+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.l64NSiq0mA4AYdEgTlWT1RAje-mq3ED_0 2024-04-22T20:48:17.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.036+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15655622169691469729', time=2024-04-22T22:29:11.306317+0000, chain.objs.size()=1 2024-04-22T20:48:17.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.036+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.9FXkt2yn4QILGW-7ToWg4RwU-dqx_2B_0 2024-04-22T20:48:17.034 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.036+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1562292049665128366', time=2024-04-22T22:29:11.494424+0000, chain.objs.size()=1 2024-04-22T20:48:17.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.036+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-a74pM_a_wJJ6SXtPA0WOfwuY86MA3M_0 2024-04-22T20:48:17.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.037+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17091749280705267731', time=2024-04-22T22:29:11.584513+0000, chain.objs.size()=1 2024-04-22T20:48:17.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.037+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HDsHmCBNxwjwO77YsKylyq2HTDvoN8x_0 2024-04-22T20:48:17.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.037+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17926315463797212849', time=2024-04-22T22:29:11.688080+0000, chain.objs.size()=1 2024-04-22T20:48:17.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.037+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.q1bMEEe-lyQds9fhPr63zFZM0zy5l5M_0 2024-04-22T20:48:17.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.037+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13262554294561680605', time=2024-04-22T22:29:12.472402+0000, chain.objs.size()=1 2024-04-22T20:48:17.035 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.037+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Ey6y3HlT37QGVvm-OQTs87hp0TQ1dJX_0 2024-04-22T20:48:17.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.041+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/49288 2024-04-22T20:48:17.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=29, truncated=0, next_marker='' 2024-04-22T20:48:17.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12971146611360947238', time=2024-04-22T22:29:12.474098+0000, chain.objs.size()=1 2024-04-22T20:48:17.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.9ptAqrJ_WvCLmDxM6UcozP7-IOniljk_0 2024-04-22T20:48:17.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13206507764723133946', time=2024-04-22T22:29:12.580464+0000, chain.objs.size()=1 2024-04-22T20:48:17.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.T28jH6tK0GQR4eGXGv-BWkigX6Gt0WN_0 2024-04-22T20:48:17.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2935864970022231439', time=2024-04-22T22:29:13.407090+0000, chain.objs.size()=1 2024-04-22T20:48:17.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-wIMSM7BNxyAPwl7hghIaWldIvp13nl_0 2024-04-22T20:48:17.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6619516252600498671', time=2024-04-22T22:29:13.512948+0000, chain.objs.size()=1 2024-04-22T20:48:17.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XzjjiOqsPkgen375KI94J0wNjeUP_Nj_0 2024-04-22T20:48:17.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3179519297684308667', time=2024-04-22T22:29:14.070558+0000, chain.objs.size()=1 2024-04-22T20:48:17.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.etiotEZ3zFs8_YU1ALtwW-TRyk03blT_0 2024-04-22T20:48:17.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8780260003868925800', time=2024-04-22T22:29:14.388046+0000, chain.objs.size()=1 2024-04-22T20:48:17.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HvXCKAmIsFK0dPJ1h8W_8_BSrwbsqYR_0 2024-04-22T20:48:17.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7775512411452998840', time=2024-04-22T22:29:14.399859+0000, chain.objs.size()=1 2024-04-22T20:48:17.042 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7Jg_H4wEBJbOPKJtj5U_5rTQryJYgsz_0 2024-04-22T20:48:17.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15244522779930291772', time=2024-04-22T22:29:14.416233+0000, chain.objs.size()=1 2024-04-22T20:48:17.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nLKrUaaVi77kxcfUrElmXLrpfp0nPdg_0 2024-04-22T20:48:17.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1511268602843967976', time=2024-04-22T22:29:14.686429+0000, chain.objs.size()=1 2024-04-22T20:48:17.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7v4t_w7538Fkox82oDOs1y1Z6TfiCYv_0 2024-04-22T20:48:17.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5595699994336012187', time=2024-04-22T22:29:15.351982+0000, chain.objs.size()=1 2024-04-22T20:48:17.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HEqp6NinuR_aq-gmU7IzBYuzTwKpYka_0 2024-04-22T20:48:17.043 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2023685373901047642', time=2024-04-22T22:29:15.352769+0000, chain.objs.size()=1 2024-04-22T20:48:17.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dsPNPHSB0tb04PDw9eJQ-esHLBGyIqu_0 2024-04-22T20:48:17.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6903831010242092707', time=2024-04-22T22:29:15.479003+0000, chain.objs.size()=1 2024-04-22T20:48:17.044 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.043+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rQQYYBZyNrGboomkqw3SoRqmVmSjDYR_0 2024-04-22T20:48:17.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17460829269987679655', time=2024-04-22T22:29:15.676370+0000, chain.objs.size()=1 2024-04-22T20:48:17.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NKg7XQY4wUe1Nq0_qIOD6Owr2dQmaHk_0 2024-04-22T20:48:17.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16710344019664670816', time=2024-04-22T22:29:15.901056+0000, chain.objs.size()=1 2024-04-22T20:48:17.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.cCkg0W4yuOadthdlIOxUe1GgVBXchyc_0 2024-04-22T20:48:17.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3110056664642406631', time=2024-04-22T22:29:16.004319+0000, chain.objs.size()=1 2024-04-22T20:48:17.045 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dCP4-DGSXRyqb1Eb6C0o5-P4DFhYDNj_0 2024-04-22T20:48:17.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13712707062403295132', time=2024-04-22T22:29:16.756308+0000, chain.objs.size()=1 2024-04-22T20:48:17.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NFO0hXKdl9aEnw9ed8r2MBb4vo20mDq_0 2024-04-22T20:48:17.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13948254593034813611', time=2024-04-22T22:29:17.098776+0000, chain.objs.size()=1 2024-04-22T20:48:17.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bbBaD0oXXCZ4kELOZvKuezIhcNfVi8H_0 2024-04-22T20:48:17.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10788487811243351399', time=2024-04-22T22:29:17.490973+0000, chain.objs.size()=1 2024-04-22T20:48:17.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.84lQJWtUtQIyE9Jr4srEnh4Qk0S0IFF_0 2024-04-22T20:48:17.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16386274853894729278', time=2024-04-22T22:30:39.888019+0000, chain.objs.size()=1 2024-04-22T20:48:17.046 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.433__shadow_.qbzkhZDJWMb6JUDif0X0dQ3PbuNBVko_0 2024-04-22T20:48:17.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11024053395911178248', time=2024-04-22T22:30:43.110095+0000, chain.objs.size()=1 2024-04-22T20:48:17.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.436__shadow_.WM8VgxDfMPZ1LxOFxaFGnKrH3JWAts__0 2024-04-22T20:48:17.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9344586613296694290', time=2024-04-22T22:30:58.487106+0000, chain.objs.size()=1 2024-04-22T20:48:17.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.452__shadow_.c8QM-wV7MpOT-j6vAoFRVN9v30IBWbp_0 2024-04-22T20:48:17.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13177682349328352241', time=2024-04-22T22:31:00.514653+0000, chain.objs.size()=1 2024-04-22T20:48:17.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.046+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.454__shadow_.gajWzQ0pLmKJqYFbUWt5fBr_F3uk8j7_0 2024-04-22T20:48:17.047 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.047+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3898413976669077693', time=2024-04-22T22:31:11.423110+0000, chain.objs.size()=1 2024-04-22T20:48:17.048 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.047+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.473__shadow_.PZCj85RJk3sHtH8J3HGNDsYx7rjOuf2_0 2024-04-22T20:48:17.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.050+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~ZJI88HS4RTZSrqj9RIWazU7ZGGulNrD', time=2024-04-22T22:35:44.065398+0000, chain.objs.size()=1 2024-04-22T20:48:17.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.050+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.520__multipart_mymultipart2.2~ZJI88HS4RTZSrqj9RIWazU7ZGGulNrD.1 2024-04-22T20:48:17.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16517020892052695816', time=2024-04-22T22:37:28.259597+0000, chain.objs.size()=1 2024-04-22T20:48:17.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:jrIXY0QvI5NaFeoLuaXYdYCcjMfVx7C_.lgE6qp8wYPxOQnG1HSCpyqj6okx0TQV_0 2024-04-22T20:48:17.049 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18066245152350459946', time=2024-04-22T22:37:49.024689+0000, chain.objs.size()=1 2024-04-22T20:48:17.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.564__shadow:aDYXHIfMzkt-AW7y62TzsNd9uK2OGC2_.ZOsC3wDnYFznXdCwJs89-XXpVvQOGqj_0 2024-04-22T20:48:17.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2635441989724230448', time=2024-04-22T22:39:04.980114+0000, chain.objs.size()=1 2024-04-22T20:48:17.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.584__shadow_.SFN_g8fqPJE8J1TXWydK22QdV-IA7R2_0 2024-04-22T20:48:17.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13043018718436261998', time=2024-04-22T22:44:07.077288+0000, chain.objs.size()=10 2024-04-22T20:48:17.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.1 2024-04-22T20:48:17.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.2 2024-04-22T20:48:17.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.3 2024-04-22T20:48:17.050 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.4 2024-04-22T20:48:17.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.5 2024-04-22T20:48:17.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.6 2024-04-22T20:48:17.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.7 2024-04-22T20:48:17.051 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.052+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.8 2024-04-22T20:48:17.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.055+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.9 2024-04-22T20:48:17.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.055+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.717__multipart_foo.2~6eIVydw8oi7M8AQoEETFyV_vZPFcvke.10 2024-04-22T20:48:17.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.055+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5712302272384578863', time=2024-04-22T22:44:11.488072+0000, chain.objs.size()=1 2024-04-22T20:48:17.052 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.055+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.722__shadow_.fubzbVoVeKziyiCbrPRPifaH1jSRXYh_0 2024-04-22T20:48:17.054 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.058+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.057 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.060+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-22T20:48:17.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.061+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.058 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.062+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.062+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/56069' 2024-04-22T20:48:17.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.062+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3387416110701406835', time=2024-04-22T22:23:20.028237+0000, chain.objs.size()=1 2024-04-22T20:48:17.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.189__shadow_.exx8Q5kLUDx-ySzzlx0XumAunnqJ0mt_0 2024-04-22T20:48:17.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18169488379840206096', time=2024-04-22T22:25:21.711393+0000, chain.objs.size()=1 2024-04-22T20:48:17.060 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.227__shadow_.AqzedMrudWSfSUycjwkJ8yeEJP-MG6p_0 2024-04-22T20:48:17.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7227650114616768760', time=2024-04-22T22:25:28.848122+0000, chain.objs.size()=1 2024-04-22T20:48:17.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.233__shadow_.b-HKlhB0iV_TLuy6TWNyaKIn5qENrs3_0 2024-04-22T20:48:17.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11499197019518466020', time=2024-04-22T22:25:32.811587+0000, chain.objs.size()=1 2024-04-22T20:48:17.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.237__shadow_.K2lW_DsQYUKfjgNA-B9MbvRYUfHy1sl_0 2024-04-22T20:48:17.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10864818232339401605', time=2024-04-22T22:26:22.722570+0000, chain.objs.size()=1 2024-04-22T20:48:17.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.f8gztTiqJJ5nGAtWvjXVDppGsCtHgXl_0 2024-04-22T20:48:17.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15980581603249274171', time=2024-04-22T22:26:24.054349+0000, chain.objs.size()=1 2024-04-22T20:48:17.061 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0jfOOHRuH3pN031QceChzoZWB7B93bd_0 2024-04-22T20:48:17.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6270522825053931418', time=2024-04-22T22:26:24.165354+0000, chain.objs.size()=1 2024-04-22T20:48:17.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jsDryjSnb8Do12Goiqbr2xarFw66mwk_0 2024-04-22T20:48:17.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17668654882627258608', time=2024-04-22T22:26:24.810249+0000, chain.objs.size()=1 2024-04-22T20:48:17.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.clVDeU_n6DFHVvVWyw2aDiOcPix_sJR_0 2024-04-22T20:48:17.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7626355063777671049', time=2024-04-22T22:26:24.867490+0000, chain.objs.size()=1 2024-04-22T20:48:17.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VOVky3tvwNldB4rLNHWBWLensSDy5MQ_0 2024-04-22T20:48:17.062 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9458825176251517572', time=2024-04-22T22:26:24.943044+0000, chain.objs.size()=1 2024-04-22T20:48:17.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YQ-1rvMgtGRf_m-uCKiwQ4BmpqEDK42_0 2024-04-22T20:48:17.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2840180233304849108', time=2024-04-22T22:26:24.980063+0000, chain.objs.size()=1 2024-04-22T20:48:17.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.brVXgGSEjgoLGK7lcOrZuDHPaTY5OqC_0 2024-04-22T20:48:17.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12049491440075369809', time=2024-04-22T22:26:26.813517+0000, chain.objs.size()=1 2024-04-22T20:48:17.063 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.063+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5lItGUyp_M0dvEdHcjsHTe4NFuUxZM7_0 2024-04-22T20:48:17.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10152327144153311210', time=2024-04-22T22:26:26.973232+0000, chain.objs.size()=1 2024-04-22T20:48:17.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2e37rsg-NQaxHdvqXNm2u6oq3ZHXHRW_0 2024-04-22T20:48:17.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12771274275239559777', time=2024-04-22T22:26:27.524086+0000, chain.objs.size()=1 2024-04-22T20:48:17.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lqWqmTHYFP0zRJ2Zuczfk6MWyrfJk8y_0 2024-04-22T20:48:17.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8900789967903745281', time=2024-04-22T22:26:28.029697+0000, chain.objs.size()=1 2024-04-22T20:48:17.064 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eMoQjCsm-rs36I9IIflUNNr1PKWDmte_0 2024-04-22T20:48:17.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15754084309547393627', time=2024-04-22T22:26:28.104716+0000, chain.objs.size()=1 2024-04-22T20:48:17.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VX-opQv2HjRa4TbgT6E1YNyG4KpJeHF_0 2024-04-22T20:48:17.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6963803963336642613', time=2024-04-22T22:26:28.105414+0000, chain.objs.size()=1 2024-04-22T20:48:17.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rsa7XCk-Ydd93WJ8Wja0pBivvuKFCUu_0 2024-04-22T20:48:17.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12546296840623383571', time=2024-04-22T22:26:28.192966+0000, chain.objs.size()=1 2024-04-22T20:48:17.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4GZ02_qWC4Ut9LHFeuCNG_KRE6iVjOx_0 2024-04-22T20:48:17.065 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15683472181790446604', time=2024-04-22T22:26:29.027434+0000, chain.objs.size()=1 2024-04-22T20:48:17.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.M52iJ13jpMYtI8MKjNwNlnfkOmKF2xc_0 2024-04-22T20:48:17.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2508487788821377194', time=2024-04-22T22:26:29.352653+0000, chain.objs.size()=1 2024-04-22T20:48:17.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.T7S6YbV--Y3VnLCox3I7gMeGv8nj_Lg_0 2024-04-22T20:48:17.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1410383603584324611', time=2024-04-22T22:26:29.542386+0000, chain.objs.size()=1 2024-04-22T20:48:17.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dXadq1_uC56sxpVRXrzJWmEreQb7BxV_0 2024-04-22T20:48:17.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1839253514497660445', time=2024-04-22T22:26:29.630932+0000, chain.objs.size()=1 2024-04-22T20:48:17.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.boOW0b6VPOwFEeETv0BWEHDEW2BYOPM_0 2024-04-22T20:48:17.066 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8501486345009023399', time=2024-04-22T22:26:29.631520+0000, chain.objs.size()=1 2024-04-22T20:48:17.067 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.066+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uU89Zlu3aEAgg1bAkCgVXmHSZVAyMvA_0 2024-04-22T20:48:17.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.069+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9165378961065619076', time=2024-04-22T22:26:29.657570+0000, chain.objs.size()=1 2024-04-22T20:48:17.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.069+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.u93LYgGHM73MArZflQ8TgS4ZweSJwtV_0 2024-04-22T20:48:17.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5618312257124786848', time=2024-04-22T22:26:29.684995+0000, chain.objs.size()=1 2024-04-22T20:48:17.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7J2b50hRtXY74pq58DR6vd2V0z562sc_0 2024-04-22T20:48:17.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6691220008525284143', time=2024-04-22T22:26:29.900147+0000, chain.objs.size()=1 2024-04-22T20:48:17.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1JSuHrUaCm_0bzD2J9I9GYgG9lB4YWI_0 2024-04-22T20:48:17.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4882519981562064421', time=2024-04-22T22:26:30.138642+0000, chain.objs.size()=1 2024-04-22T20:48:17.068 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GJZ6akHe3QuIi9O1VKTdBmnmVuOJ2cd_0 2024-04-22T20:48:17.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7084917345262063575', time=2024-04-22T22:26:30.552399+0000, chain.objs.size()=1 2024-04-22T20:48:17.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.OipnHC7Pk_0sHLW8HKNU6LjPUqJX9M2_0 2024-04-22T20:48:17.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.763232755030294490', time=2024-04-22T22:26:30.663117+0000, chain.objs.size()=1 2024-04-22T20:48:17.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.huHR0k7Spz5rVB6wH_UjoQfISUm8HjV_0 2024-04-22T20:48:17.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.878146646747607200', time=2024-04-22T22:26:31.230593+0000, chain.objs.size()=1 2024-04-22T20:48:17.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sB5SPFyEEszKPFdjOV3Cg4fljh8mIU3_0 2024-04-22T20:48:17.069 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14334182468554709698', time=2024-04-22T22:26:31.420477+0000, chain.objs.size()=1 2024-04-22T20:48:17.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.070+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8_ewNkiECdR4nzX7LYQ5cSe9Yg-o443_0 2024-04-22T20:48:17.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17251965817624125350', time=2024-04-22T22:26:32.525362+0000, chain.objs.size()=1 2024-04-22T20:48:17.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lA5gTYk78XOWToxyCkz68IEqM7Eg-9L_0 2024-04-22T20:48:17.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10799469634635561449', time=2024-04-22T22:26:32.815518+0000, chain.objs.size()=1 2024-04-22T20:48:17.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.M-V7CysdreJ2zpY61CmrGUgE-2cIYZy_0 2024-04-22T20:48:17.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.071+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17215007124370698952', time=2024-04-22T22:26:33.098831+0000, chain.objs.size()=1 2024-04-22T20:48:17.070 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.071+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FBi6hV3fZgG2eH8MkTrih8y0_LKne7N_0 2024-04-22T20:48:17.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16785325045965991724', time=2024-04-22T22:26:33.522310+0000, chain.objs.size()=1 2024-04-22T20:48:17.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pxOXw-1PfOZgdcPo81U4jIeLNFUVKUw_0 2024-04-22T20:48:17.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12760027854021517885', time=2024-04-22T22:26:34.058821+0000, chain.objs.size()=1 2024-04-22T20:48:17.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gmUO-SCYQvMKV6zBPrqZL_4dPv0dLPM_0 2024-04-22T20:48:17.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12062475919310143557', time=2024-04-22T22:26:34.192091+0000, chain.objs.size()=1 2024-04-22T20:48:17.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zKtjzmUihlGi6z6Esn62Go7Q8wukeND_0 2024-04-22T20:48:17.072 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12139457625753611403', time=2024-04-22T22:26:34.451132+0000, chain.objs.size()=1 2024-04-22T20:48:17.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eSZm3LTYWsIW9mgtnSd-mwRZFuEL_nW_0 2024-04-22T20:48:17.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4604768818535383880', time=2024-04-22T22:26:39.250338+0000, chain.objs.size()=1 2024-04-22T20:48:17.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.256__shadow_.KXZ9gbdcbaFpv7ZaWxpIJq4F7baD2c7_0 2024-04-22T20:48:17.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4299650244744095020', time=2024-04-22T22:26:47.889857+0000, chain.objs.size()=1 2024-04-22T20:48:17.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.265__shadow_.Tksyy4cp5llu3xV1pmPEdM2YIRYR_PA_0 2024-04-22T20:48:17.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13130226313079930305', time=2024-04-22T22:27:07.853080+0000, chain.objs.size()=1 2024-04-22T20:48:17.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.7jIl-YwgrD70TC81mRN_KZ6ypSorFd4_0 2024-04-22T20:48:17.073 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17417196055130890081', time=2024-04-22T22:27:09.959617+0000, chain.objs.size()=1 2024-04-22T20:48:17.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.284__shadow_.F_Y0wJVzaM9EdgZ5i0L7jpZK0P4hG3f_0 2024-04-22T20:48:17.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9412173853520394667', time=2024-04-22T22:27:09.961394+0000, chain.objs.size()=1 2024-04-22T20:48:17.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.284__shadow_.tP2q0Eg4NOEGCuAFzAbCkZvZk0yypMs_0 2024-04-22T20:48:17.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12237521243212936214', time=2024-04-22T22:27:14.087860+0000, chain.objs.size()=1 2024-04-22T20:48:17.074 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.075+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.288__shadow_.Fb-7TUrK_ovhXTaog05le9trU5xMqVg_0 2024-04-22T20:48:17.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.076+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.226534651619760932', time=2024-04-22T22:28:17.550650+0000, chain.objs.size()=1 2024-04-22T20:48:17.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.076+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kz7GrRrHUqRNH2A56RN61n4zWnjgBdB_0 2024-04-22T20:48:17.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.079+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11443822794761770965', time=2024-04-22T22:28:17.648044+0000, chain.objs.size()=1 2024-04-22T20:48:17.075 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.079+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.sk66ea6MbvPV7uXOAAUcQBZ9KSJlcQz_0 2024-04-22T20:48:17.076 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6944099375358889405', time=2024-04-22T22:28:17.778679+0000, chain.objs.size()=1 2024-04-22T20:48:17.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RcJ-pnJrom449_36_7ZTA64btbqyRVx_0 2024-04-22T20:48:17.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8077780854855388029', time=2024-04-22T22:28:18.272303+0000, chain.objs.size()=1 2024-04-22T20:48:17.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bwIFV07nUlo1iLrNk15NPnI3iWhk-SJ_0 2024-04-22T20:48:17.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12247467277744103527', time=2024-04-22T22:28:18.432561+0000, chain.objs.size()=1 2024-04-22T20:48:17.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.QhT8qLJnpZ8jqltYKmgXvoVmgmbSahA_0 2024-04-22T20:48:17.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2750158231942333591', time=2024-04-22T22:28:19.247029+0000, chain.objs.size()=1 2024-04-22T20:48:17.077 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4B8m5pDZSinO5LTCuV1JthEG_j47LbN_0 2024-04-22T20:48:17.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9158435761385580164', time=2024-04-22T22:28:19.423012+0000, chain.objs.size()=1 2024-04-22T20:48:17.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5yv6wzXY3Xy5mG6O-Kby8naRnndrmp9_0 2024-04-22T20:48:17.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15445564389245913156', time=2024-04-22T22:28:19.686539+0000, chain.objs.size()=1 2024-04-22T20:48:17.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.I9PrgllKC50212x1saF_VuMNHlXodbw_0 2024-04-22T20:48:17.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16630116121471256962', time=2024-04-22T22:28:19.687567+0000, chain.objs.size()=1 2024-04-22T20:48:17.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Sbf3D7uNuhN-YZoM8iFdZTpUbyRa7w1_0 2024-04-22T20:48:17.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13564467214969057932', time=2024-04-22T22:28:20.018527+0000, chain.objs.size()=1 2024-04-22T20:48:17.078 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.829fDKSa-ujtEqC_5oo4o1ruPX2kcyB_0 2024-04-22T20:48:17.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10239023101109019561', time=2024-04-22T22:28:20.225209+0000, chain.objs.size()=1 2024-04-22T20:48:17.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Nfa-GLyNAEYVc_EZC5cdRSIbdKn56QC_0 2024-04-22T20:48:17.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13375760723701215350', time=2024-04-22T22:28:21.101363+0000, chain.objs.size()=1 2024-04-22T20:48:17.079 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.080+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.deKjUCi7EIsNSG5w1tjcUseRipJWjwV_0 2024-04-22T20:48:17.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.082+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14743967914458636481', time=2024-04-22T22:28:21.261626+0000, chain.objs.size()=1 2024-04-22T20:48:17.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.082+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.eEk54_Q4wsAmyRKXipiXAQluI9bSEuq_0 2024-04-22T20:48:17.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17905380802592417188', time=2024-04-22T22:28:21.329228+0000, chain.objs.size()=1 2024-04-22T20:48:17.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PrX7uMVAzKFl_d6ouhkhtotuF9gXgt0_0 2024-04-22T20:48:17.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.748950732399229129', time=2024-04-22T22:28:21.743787+0000, chain.objs.size()=1 2024-04-22T20:48:17.080 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-rxL-4xYZuA5iVIiTfq-RxZN4dBdDOM_0 2024-04-22T20:48:17.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14959091551835618983', time=2024-04-22T22:28:22.350870+0000, chain.objs.size()=1 2024-04-22T20:48:17.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SiIxduVa6QFp2RdjPjqpe2eUH35H44n_0 2024-04-22T20:48:17.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5758638786581761527', time=2024-04-22T22:28:22.552091+0000, chain.objs.size()=1 2024-04-22T20:48:17.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Q35Jbkb8aPSo9GdReR8ZkTghSkItIhz_0 2024-04-22T20:48:17.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1671327571212087626', time=2024-04-22T22:28:22.645881+0000, chain.objs.size()=1 2024-04-22T20:48:17.081 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.083+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.YX0uhWllcj6sFVmxX_0-mmbSZ1VmIgf_0 2024-04-22T20:48:17.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2441345635246598303', time=2024-04-22T22:28:22.942594+0000, chain.objs.size()=1 2024-04-22T20:48:17.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.P4CnZ-BfPQUi1KlyBOUi_ieQMl__Zfe_0 2024-04-22T20:48:17.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5812538612689858607', time=2024-04-22T22:28:24.176254+0000, chain.objs.size()=1 2024-04-22T20:48:17.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.L5qPMI_eosjTrUevZOWvuzfE3e9Ewhj_0 2024-04-22T20:48:17.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9170809562359002049', time=2024-04-22T22:28:24.486893+0000, chain.objs.size()=1 2024-04-22T20:48:17.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LUFVit9RSH70zQXUnfBA75w-EJCRFt5_0 2024-04-22T20:48:17.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12603085578831044243', time=2024-04-22T22:28:25.405070+0000, chain.objs.size()=1 2024-04-22T20:48:17.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6LIVjeH6jbi61PzR1GzMHelmJHNBbRc_0 2024-04-22T20:48:17.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11690844770356931328', time=2024-04-22T22:28:25.942966+0000, chain.objs.size()=1 2024-04-22T20:48:17.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.084+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9EJbwRBxbjFq8Jgjikb27vBmL6oyadw_0 2024-04-22T20:48:17.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.086+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11008389119580952236', time=2024-04-22T22:28:26.411937+0000, chain.objs.size()=1 2024-04-22T20:48:17.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.086+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4bo52sqpSS-YcLoYwQsi5bSM_YSqNde_0 2024-04-22T20:48:17.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.086+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3180308974403435792', time=2024-04-22T22:28:26.468166+0000, chain.objs.size()=1 2024-04-22T20:48:17.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.086+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.niiCIB6XkRdw4na5I0Y5xyAbeLLyMlD_0 2024-04-22T20:48:17.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1805811211501193321', time=2024-04-22T22:28:27.306493+0000, chain.objs.size()=1 2024-04-22T20:48:17.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZsPWlJHHR7s1fejqUl_2NsEium2bFXe_0 2024-04-22T20:48:17.084 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4979767860236764499', time=2024-04-22T22:28:27.477620+0000, chain.objs.size()=1 2024-04-22T20:48:17.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.iel07oaQT7aTXv6iScxe77QAjFbS4M__0 2024-04-22T20:48:17.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7708884545581738907', time=2024-04-22T22:28:27.682448+0000, chain.objs.size()=1 2024-04-22T20:48:17.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.adcF_DoYeCozecUnEE4EDimUi4XsigQ_0 2024-04-22T20:48:17.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18238818589414174340', time=2024-04-22T22:28:27.986438+0000, chain.objs.size()=1 2024-04-22T20:48:17.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gVgIMlqygW6Cdl73_9vOEHvjs6T7wF3_0 2024-04-22T20:48:17.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8728235991489135973', time=2024-04-22T22:28:28.610586+0000, chain.objs.size()=1 2024-04-22T20:48:17.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5dTL0usinD_Rw2LIV_92LMHz2bSP1Zd_0 2024-04-22T20:48:17.085 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11532598681068843313', time=2024-04-22T22:28:28.701366+0000, chain.objs.size()=1 2024-04-22T20:48:17.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-dE2q6YO41UzX9ZKZe5cZZlVLVVT8bO_0 2024-04-22T20:48:17.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5611267837158379428', time=2024-04-22T22:28:28.917249+0000, chain.objs.size()=1 2024-04-22T20:48:17.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.2TCYtQ3Kumy_lPqmW2vcCRK2azoIoB__0 2024-04-22T20:48:17.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14048100219255158182', time=2024-04-22T22:28:28.939309+0000, chain.objs.size()=1 2024-04-22T20:48:17.086 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.087+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.iLxl13p3cH4IjeYR2Of282PfexwT_-a_0 2024-04-22T20:48:17.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.090+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10594095988620875537', time=2024-04-22T22:28:29.174896+0000, chain.objs.size()=1 2024-04-22T20:48:17.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.090+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Ggqm7Ygu3SaMqX9pLymH1Wuq6XPZ37F_0 2024-04-22T20:48:17.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.090+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.602871747165961749', time=2024-04-22T22:29:06.177436+0000, chain.objs.size()=1 2024-04-22T20:48:17.087 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.090+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.u0JcmSfCyl2jJdoQOC4q7X9zKDWkLcJ_0 2024-04-22T20:48:17.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.090+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8845017113375445295', time=2024-04-22T22:29:06.368624+0000, chain.objs.size()=1 2024-04-22T20:48:17.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.090+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DnLw3bxfQOozR4KBZlOppX6KCfLATDn_0 2024-04-22T20:48:17.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.090+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18030487223237876651', time=2024-04-22T22:29:07.854205+0000, chain.objs.size()=1 2024-04-22T20:48:17.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.090+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ituczbF-rN2y8WhM26kYyWl61ZRAPaj_0 2024-04-22T20:48:17.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.091+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8813860599123783554', time=2024-04-22T22:29:07.934583+0000, chain.objs.size()=1 2024-04-22T20:48:17.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.091+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.aREjwCQqajQPCn4HbnsG9eDReW_lEn-_0 2024-04-22T20:48:17.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.091+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16022828875001124840', time=2024-04-22T22:29:07.952111+0000, chain.objs.size()=1 2024-04-22T20:48:17.088 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.091+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oiMAr77HvdgSYdB_6-Mb5_Sidflvh7Q_0 2024-04-22T20:48:17.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.091+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7730872757623840874', time=2024-04-22T22:29:08.373692+0000, chain.objs.size()=1 2024-04-22T20:48:17.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.091+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ELt_f0h59QVNfL3BTVI9aaXejpoFAfp_0 2024-04-22T20:48:17.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.092+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15251547320485005771', time=2024-04-22T22:29:08.602343+0000, chain.objs.size()=1 2024-04-22T20:48:17.089 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.092+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.VRnsAAecj0cTk8ffGlFD_tYpbpoud1U_0 2024-04-22T20:48:17.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.093+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10202194168422965235', time=2024-04-22T22:29:09.412624+0000, chain.objs.size()=1 2024-04-22T20:48:17.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.093+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.32u4PI8bnfH_hPgkBrDL1VB_yaT7zg-_0 2024-04-22T20:48:17.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.093+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17694033746627296246', time=2024-04-22T22:29:09.541459+0000, chain.objs.size()=1 2024-04-22T20:48:17.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.093+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8QFkQv9nOfqpVOav9cbDNy5ksQ1G9n9_0 2024-04-22T20:48:17.090 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.093+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9774969582161799811', time=2024-04-22T22:29:09.656403+0000, chain.objs.size()=1 2024-04-22T20:48:17.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.093+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ayRue7U0QtdKs-mTt6Ny26ogDzjpvwX_0 2024-04-22T20:48:17.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4950667135283891388', time=2024-04-22T22:29:10.238482+0000, chain.objs.size()=1 2024-04-22T20:48:17.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.UzbqBtu6tpt8s2G8HqQoYAYE5ujSgXL_0 2024-04-22T20:48:17.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15860164202598199138', time=2024-04-22T22:29:10.356064+0000, chain.objs.size()=1 2024-04-22T20:48:17.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YWD5cEzVcAJm-u5qKwUrra5NGLi4HnK_0 2024-04-22T20:48:17.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15670744827079928258', time=2024-04-22T22:29:11.783504+0000, chain.objs.size()=1 2024-04-22T20:48:17.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RjdC4dGHGJYe08KA9Mq9MJjmosv_mWd_0 2024-04-22T20:48:17.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12237133542071136600', time=2024-04-22T22:29:11.951061+0000, chain.objs.size()=1 2024-04-22T20:48:17.092 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PKQUmDpFuXW7Kb6ST0WUGG7Gk2ufO4j_0 2024-04-22T20:48:17.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3457444172022615571', time=2024-04-22T22:29:12.011287+0000, chain.objs.size()=1 2024-04-22T20:48:17.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RpK7Y0Miq46VNedYFf0EthZCoT5v6o5_0 2024-04-22T20:48:17.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15538348963108841822', time=2024-04-22T22:29:12.426638+0000, chain.objs.size()=1 2024-04-22T20:48:17.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-NkLr8qdQGu4DiXWtAlQgN11ccHchoH_0 2024-04-22T20:48:17.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12214088783428596392', time=2024-04-22T22:29:12.651999+0000, chain.objs.size()=1 2024-04-22T20:48:17.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tJrTtUAwtzaCFHHheMndE3s2U_qDYTr_0 2024-04-22T20:48:17.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11136934425044538556', time=2024-04-22T22:29:12.863255+0000, chain.objs.size()=1 2024-04-22T20:48:17.093 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.095+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0q0Ghq42cP4QTHmWGPDyZmZ5ve6bjQL_0 2024-04-22T20:48:17.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5803842938822530044', time=2024-04-22T22:29:13.905029+0000, chain.objs.size()=1 2024-04-22T20:48:17.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oxLEVDI-3bydkvE6T5Wnh_DTID0wxRl_0 2024-04-22T20:48:17.094 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15952053955972321346', time=2024-04-22T22:29:14.059550+0000, chain.objs.size()=1 2024-04-22T20:48:17.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LszTtr9lwhpXC_M1NyNKSkFoFg0RMA1_0 2024-04-22T20:48:17.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13282888113057199143', time=2024-04-22T22:29:16.083216+0000, chain.objs.size()=1 2024-04-22T20:48:17.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xoiretVYK0eDReUL7ZqI8ScQbkIr_Fi_0 2024-04-22T20:48:17.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.097+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16893250638346235263', time=2024-04-22T22:29:16.759858+0000, chain.objs.size()=1 2024-04-22T20:48:17.095 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.097+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RPjPRZLjtsMNJMbzw4YlUzBDhVBaAwB_0 2024-04-22T20:48:17.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.101+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/56069 2024-04-22T20:48:17.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=19, truncated=0, next_marker='' 2024-04-22T20:48:17.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8152093398625646867', time=2024-04-22T22:29:17.305530+0000, chain.objs.size()=1 2024-04-22T20:48:17.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ldOT7tHHF-ELC80Xa2Js_BX7xamKxPa_0 2024-04-22T20:48:17.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8108470208300971897', time=2024-04-22T22:29:17.334338+0000, chain.objs.size()=1 2024-04-22T20:48:17.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Kw9BzcHijpX2sRGPMDjIiozDbYivCbX_0 2024-04-22T20:48:17.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12894918452088985245', time=2024-04-22T22:29:17.371622+0000, chain.objs.size()=1 2024-04-22T20:48:17.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.aKfol9zvnoY4am0drg8HMUPEy_AEbf8_0 2024-04-22T20:48:17.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15338693135835625368', time=2024-04-22T22:29:17.372233+0000, chain.objs.size()=1 2024-04-22T20:48:17.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kQzIvYfCvlSxLb5qBAKaKVazRS5cSqw_0 2024-04-22T20:48:17.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14858207508213660397', time=2024-04-22T22:29:17.635107+0000, chain.objs.size()=1 2024-04-22T20:48:17.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sY4geIdOFWTAC8BGukjDi4GdUESbmo__0 2024-04-22T20:48:17.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4383687598286338570', time=2024-04-22T22:31:10.489151+0000, chain.objs.size()=1 2024-04-22T20:48:17.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.cVNlSrU3RwlAJXMRRWuzq7HnCrK0dco_0 2024-04-22T20:48:17.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16660577255748682808', time=2024-04-22T22:31:11.426641+0000, chain.objs.size()=1 2024-04-22T20:48:17.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.473__shadow_.rpDzWqT9zCFpyaqmwHCWDXWkEXz3DnY_0 2024-04-22T20:48:17.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_1_ca4e15d4ca8668e5c3d481945a5676af', time=2024-04-22T22:31:27.904508+0000, chain.objs.size()=1 2024-04-22T20:48:17.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.492__shadow:XT4Bpt-NneZKBvxIEeWhVJzgJiXQONo_.raNY5A0Tx5D9pV_jbpbYuZd1JtlZYIQ_0 2024-04-22T20:48:17.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2_d54112b24761e1e4ae3f84e0cca1d5af', time=2024-04-22T22:31:53.211429+0000, chain.objs.size()=6 2024-04-22T20:48:17.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.1 2024-04-22T20:48:17.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.2 2024-04-22T20:48:17.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.3 2024-04-22T20:48:17.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.103+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.4 2024-04-22T20:48:17.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.5 2024-04-22T20:48:17.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.496__multipart_srcmultipart.2~IYNNIohdZVvIfXz3zdwwtNrSZOCvQOg.6 2024-04-22T20:48:17.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4094718204280079475', time=2024-04-22T22:31:59.824868+0000, chain.objs.size()=1 2024-04-22T20:48:17.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.505__multipart_mymultipartcopy.2~vu4oDfAtZzpicwb2t63ajjPlDNKeOgU.1 2024-04-22T20:48:17.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13344036425362249968', time=2024-04-22T22:32:04.118743+0000, chain.objs.size()=1 2024-04-22T20:48:17.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.507__multipart_mymultipart.2~w6QXMizqOpzYwZlb1_XKiLFNmu6TnjJ.1 2024-04-22T20:48:17.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~C51974niHIaqTTopyxetkZv6tIMOkd8', time=2024-04-22T22:35:44.029296+0000, chain.objs.size()=2 2024-04-22T20:48:17.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.520__multipart_mymultipart.2~C51974niHIaqTTopyxetkZv6tIMOkd8.1 2024-04-22T20:48:17.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.106+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.520__multipart_mymultipart.2~C51974niHIaqTTopyxetkZv6tIMOkd8.2 2024-04-22T20:48:17.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.107+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7599334553793738990', time=2024-04-22T22:36:59.412577+0000, chain.objs.size()=1 2024-04-22T20:48:17.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.545__shadow_.ob9KFMIiBifNj6iEYgbDydoh9baPzaW_0 2024-04-22T20:48:17.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5865616049204411207', time=2024-04-22T22:37:10.155953+0000, chain.objs.size()=1 2024-04-22T20:48:17.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.549__multipart_mymultipart.AqLnMmrSMUZPXyT_1Esw42WpLoCjvoYG.1 2024-04-22T20:48:17.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14328314266798984397', time=2024-04-22T22:37:30.074330+0000, chain.objs.size()=1 2024-04-22T20:48:17.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:QvJgMZ8qKrFxcx1SNhvlbYuGU1CFZDN_.F8rdY43NpqWi27T6uaamcHlXjr4PvAk_0 2024-04-22T20:48:17.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9346614843722169571', time=2024-04-22T22:37:32.472457+0000, chain.objs.size()=1 2024-04-22T20:48:17.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:x0dir1rHNJM0sbSyXUZmbem0.P-LCBy_.CjDd6fT69J9AYlzVFHCNJRgbFoD_IIw_0 2024-04-22T20:48:17.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13963022564953420142', time=2024-04-22T22:38:04.254047+0000, chain.objs.size()=1 2024-04-22T20:48:17.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:eFcrrKqduAwoJq5kh.ffw8rhp1-BHGB_.A5lbK1tfIasSf9tmPW_I5JztZ4M5Ptl_0 2024-04-22T20:48:17.107 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1790282998652168936', time=2024-04-22T22:38:37.087082+0000, chain.objs.size()=7 2024-04-22T20:48:17.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.108+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~GxfFs_KAy6667knioIdcDHa35Ljfk2b.1 2024-04-22T20:48:17.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.109+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__shadow_testobj.2~GxfFs_KAy6667knioIdcDHa35Ljfk2b.1_1 2024-04-22T20:48:17.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.109+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~GxfFs_KAy6667knioIdcDHa35Ljfk2b.2 2024-04-22T20:48:17.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.109+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__shadow_testobj.2~GxfFs_KAy6667knioIdcDHa35Ljfk2b.2_1 2024-04-22T20:48:17.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.110+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~GxfFs_KAy6667knioIdcDHa35Ljfk2b.3 2024-04-22T20:48:17.108 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.110+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__shadow_testobj.2~GxfFs_KAy6667knioIdcDHa35Ljfk2b.3_1 2024-04-22T20:48:17.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.112+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~GxfFs_KAy6667knioIdcDHa35Ljfk2b.4 2024-04-22T20:48:17.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.112+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5101032531421615664', time=2024-04-22T22:38:43.100675+0000, chain.objs.size()=4 2024-04-22T20:48:17.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.112+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~Fx6sh0RKlPZ2ah9nZXNl6dRG-pCNutN.1 2024-04-22T20:48:17.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.112+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~Fx6sh0RKlPZ2ah9nZXNl6dRG-pCNutN.2 2024-04-22T20:48:17.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.112+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~Fx6sh0RKlPZ2ah9nZXNl6dRG-pCNutN.3 2024-04-22T20:48:17.109 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.112+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~Fx6sh0RKlPZ2ah9nZXNl6dRG-pCNutN.4 2024-04-22T20:48:17.113 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.116+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.115 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.119+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-22T20:48:17.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.120+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.117 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.121+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/49488' 2024-04-22T20:48:17.119 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1884165535809272401', time=2024-04-22T22:21:16.747133+0000, chain.objs.size()=1 2024-04-22T20:48:17.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.136__shadow_.urmbGtOOjvbVoCCjP3ATUjQZTbGx3Oe_0 2024-04-22T20:48:17.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_wF0UWk-rXhlJmWbrCaYlJf6aVR3OoZM', time=2024-04-22T22:21:54.090183+0000, chain.objs.size()=3 2024-04-22T20:48:17.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.wWdpi_GmiU3sjBZ5ti11jsFbt4jb5eX_0 2024-04-22T20:48:17.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.wWdpi_GmiU3sjBZ5ti11jsFbt4jb5eX_1 2024-04-22T20:48:17.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.wWdpi_GmiU3sjBZ5ti11jsFbt4jb5eX_2 2024-04-22T20:48:17.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14005645619219556275', time=2024-04-22T22:22:06.528152+0000, chain.objs.size()=5 2024-04-22T20:48:17.120 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-LUKEWARM.2~QwyJwHJ_u_gGtXkQbObxqmUr3niaJwa.1 2024-04-22T20:48:17.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_foo-LUKEWARM.2~QwyJwHJ_u_gGtXkQbObxqmUr3niaJwa.1_1 2024-04-22T20:48:17.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-LUKEWARM.2~QwyJwHJ_u_gGtXkQbObxqmUr3niaJwa.2 2024-04-22T20:48:17.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_foo-LUKEWARM.2~QwyJwHJ_u_gGtXkQbObxqmUr3niaJwa.2_1 2024-04-22T20:48:17.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-LUKEWARM.2~QwyJwHJ_u_gGtXkQbObxqmUr3niaJwa.3 2024-04-22T20:48:17.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.925087029482222070', time=2024-04-22T22:22:29.608609+0000, chain.objs.size()=1 2024-04-22T20:48:17.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.155__shadow_.gKGDOrvm26zD9tkCEpSxBdjhQJHHn5u_0 2024-04-22T20:48:17.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11218770910599890155', time=2024-04-22T22:25:15.295702+0000, chain.objs.size()=1 2024-04-22T20:48:17.121 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.222__shadow_.7dRr4vxPXHyxJVzHdix9YdpdMGa-KK2_0 2024-04-22T20:48:17.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12047084908117577695', time=2024-04-22T22:25:16.302948+0000, chain.objs.size()=1 2024-04-22T20:48:17.122 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.122+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.223__shadow_.vbEuKRW_SHr1PcVIwMia1kgA3FrYJUZ_0 2024-04-22T20:48:17.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.125+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.104524419817588621', time=2024-04-22T22:25:35.821650+0000, chain.objs.size()=1 2024-04-22T20:48:17.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.125+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.240__shadow_.e9nhG2Cg15eeOPariILzkVPalYN_lDo_0 2024-04-22T20:48:17.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14956678536573959262', time=2024-04-22T22:25:37.815419+0000, chain.objs.size()=1 2024-04-22T20:48:17.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.242__shadow_.8NaZfSJhaW7cm6GfrWEaTqHHNx2AR9T_0 2024-04-22T20:48:17.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15282632636506631944', time=2024-04-22T22:25:42.732783+0000, chain.objs.size()=1 2024-04-22T20:48:17.123 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.247__shadow_.m3SvU5WSoyKm6D-QAE3kkj8koWAHVzx_0 2024-04-22T20:48:17.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.617028181823018411', time=2024-04-22T22:26:22.323666+0000, chain.objs.size()=1 2024-04-22T20:48:17.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5QyM9VPaNapfOdWZnHdJgyG8YAT3NBS_0 2024-04-22T20:48:17.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18119098920250314431', time=2024-04-22T22:26:22.506662+0000, chain.objs.size()=1 2024-04-22T20:48:17.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qBDFK0WjH27_H_B7of5JE3AmKBeGcb6_0 2024-04-22T20:48:17.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6125527687594361268', time=2024-04-22T22:26:22.590328+0000, chain.objs.size()=1 2024-04-22T20:48:17.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.agtR3YW6lUm8emCg_1lfRSlUEntLb_b_0 2024-04-22T20:48:17.124 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7301933484593373593', time=2024-04-22T22:26:23.081264+0000, chain.objs.size()=1 2024-04-22T20:48:17.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HI2uqNq4I-StZRqF3qZWG8BL58pu0OX_0 2024-04-22T20:48:17.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2955000277104865265', time=2024-04-22T22:26:23.380926+0000, chain.objs.size()=1 2024-04-22T20:48:17.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._E3erLM3MBBY1on9Blv0SRSS9H4CNzC_0 2024-04-22T20:48:17.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14078314414602977205', time=2024-04-22T22:26:24.004283+0000, chain.objs.size()=1 2024-04-22T20:48:17.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.67TpFSCuRcHM4vYvtSOg4leuLyZsPoV_0 2024-04-22T20:48:17.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3087880402428120007', time=2024-04-22T22:26:24.439569+0000, chain.objs.size()=1 2024-04-22T20:48:17.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aMQSUtOqVKiA5g6hpSichkfPwoGLV-s_0 2024-04-22T20:48:17.125 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2129600562257093059', time=2024-04-22T22:26:24.719455+0000, chain.objs.size()=1 2024-04-22T20:48:17.126 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.126+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CHtM9cNI94rG5KV5SHIjdZL9OW8geJh_0 2024-04-22T20:48:17.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.127+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15952557284981028645', time=2024-04-22T22:26:25.016858+0000, chain.objs.size()=1 2024-04-22T20:48:17.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.127+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wceCqmC7F33S4LWTko_yMjxPV1K7UwP_0 2024-04-22T20:48:17.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11222154742096387098', time=2024-04-22T22:26:25.660052+0000, chain.objs.size()=1 2024-04-22T20:48:17.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.RetW1HS7KI6y3QuXw9-WXYzgYLAfu5z_0 2024-04-22T20:48:17.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2913905893118775323', time=2024-04-22T22:26:25.970399+0000, chain.objs.size()=1 2024-04-22T20:48:17.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2iCtHa7AqlZV3nUuxajyojiHSIuASJn_0 2024-04-22T20:48:17.127 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18331414778094847681', time=2024-04-22T22:26:26.137870+0000, chain.objs.size()=1 2024-04-22T20:48:17.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.YhPa_cFrTQ-2uNt-U0z3jRn_B0HbalA_0 2024-04-22T20:48:17.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13289867515299891535', time=2024-04-22T22:26:26.185556+0000, chain.objs.size()=1 2024-04-22T20:48:17.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mLI5bsq_EU0MQtVnwO5KccR8W_6OOJ6_0 2024-04-22T20:48:17.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18283644691380162219', time=2024-04-22T22:26:26.813232+0000, chain.objs.size()=1 2024-04-22T20:48:17.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5p2VQ1BQiPrtHZceBvm65xAYvF5KnGP_0 2024-04-22T20:48:17.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6988682977278706812', time=2024-04-22T22:26:28.025765+0000, chain.objs.size()=1 2024-04-22T20:48:17.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cVpvWGU_Nl2OkplA7DI2uCF7WykznHF_0 2024-04-22T20:48:17.128 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2770473760176493928', time=2024-04-22T22:26:28.154567+0000, chain.objs.size()=1 2024-04-22T20:48:17.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jG7n09my4Cc8r6JY5WIC-UJYItreyuG_0 2024-04-22T20:48:17.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14947807016779537247', time=2024-04-22T22:26:28.339067+0000, chain.objs.size()=1 2024-04-22T20:48:17.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.RKBjpWAb6TKQpM7peLn6vEWbolPmp93_0 2024-04-22T20:48:17.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7111058077997862441', time=2024-04-22T22:26:29.438599+0000, chain.objs.size()=1 2024-04-22T20:48:17.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.129+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.iXKJJV6i0Odj5GW9ILAEUN_4KCB1IG1_0 2024-04-22T20:48:17.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.130+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6851131613264801674', time=2024-04-22T22:26:29.439295+0000, chain.objs.size()=1 2024-04-22T20:48:17.129 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.130+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ftGSZq2n4pL7O9u23U9RXdVaCOni5oM_0 2024-04-22T20:48:17.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.131+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11983163655179926637', time=2024-04-22T22:26:29.965747+0000, chain.objs.size()=1 2024-04-22T20:48:17.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.131+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zHcXUZ9I0sB_aIKKT4fDwTwKIuST6h3_0 2024-04-22T20:48:17.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.133+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.839176604569766554', time=2024-04-22T22:26:30.155065+0000, chain.objs.size()=1 2024-04-22T20:48:17.130 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.133+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FroY7b_ziOpIRiFmxexk4NZanrs7XUj_0 2024-04-22T20:48:17.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16502609815381890614', time=2024-04-22T22:26:30.184055+0000, chain.objs.size()=1 2024-04-22T20:48:17.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pfYebW5MKLBrayD04B_g5HPLvhIuCwd_0 2024-04-22T20:48:17.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1429261837682697101', time=2024-04-22T22:26:31.163803+0000, chain.objs.size()=1 2024-04-22T20:48:17.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xinh9DaIms-Fgc451rhrxFVagfJRdJt_0 2024-04-22T20:48:17.131 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6506356267453277763', time=2024-04-22T22:26:31.493212+0000, chain.objs.size()=1 2024-04-22T20:48:17.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ww7XLy274IoDNXOJMg4ujUhr3CqHC4m_0 2024-04-22T20:48:17.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15984354868920436299', time=2024-04-22T22:26:31.607103+0000, chain.objs.size()=1 2024-04-22T20:48:17.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XCxGdK-fpnYyIdtrwtS88MNGDzmHCLc_0 2024-04-22T20:48:17.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10028498177037388739', time=2024-04-22T22:26:31.687478+0000, chain.objs.size()=1 2024-04-22T20:48:17.132 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.134+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qXQHPSQ7RGO2H6_WEuUHVD7CV4FWs6F_0 2024-04-22T20:48:17.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7874299822465053189', time=2024-04-22T22:26:32.015303+0000, chain.objs.size()=1 2024-04-22T20:48:17.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Yrr6i4Y06QSqACy4vCGJHkJtekusjYy_0 2024-04-22T20:48:17.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4666899852817816972', time=2024-04-22T22:26:32.016670+0000, chain.objs.size()=1 2024-04-22T20:48:17.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CWiXNAt__us68KoVgm-MgIExjMztPOo_0 2024-04-22T20:48:17.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14718080348403758138', time=2024-04-22T22:26:32.201728+0000, chain.objs.size()=1 2024-04-22T20:48:17.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hZDyviFWWMcDuPLKIA4KO5f6n6iYqrY_0 2024-04-22T20:48:17.133 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12932656290719334062', time=2024-04-22T22:26:32.501247+0000, chain.objs.size()=1 2024-04-22T20:48:17.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.o-xVCC-nRbIswlSpMHsD4KgWpGkv7OE_0 2024-04-22T20:48:17.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14741073369465367864', time=2024-04-22T22:26:32.501947+0000, chain.objs.size()=1 2024-04-22T20:48:17.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.135+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EFWhirzn4xfEYgrX4ZjwkgjP1jW1P0u_0 2024-04-22T20:48:17.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.136+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6830841829266071901', time=2024-04-22T22:26:32.746717+0000, chain.objs.size()=1 2024-04-22T20:48:17.134 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.136+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hpTX1E2bhm41V9HY0k4h6sj7viZblOh_0 2024-04-22T20:48:17.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.137+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10883875091230470535', time=2024-04-22T22:26:32.895758+0000, chain.objs.size()=1 2024-04-22T20:48:17.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.137+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rc39ZlEzPJOS0mR1q37M8oo4hplYN9D_0 2024-04-22T20:48:17.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.138+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15186266755595533461', time=2024-04-22T22:26:34.263358+0000, chain.objs.size()=1 2024-04-22T20:48:17.135 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.138+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EW-lKb_PT4xV2m5DLR_cXM9bHCwdqck_0 2024-04-22T20:48:17.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.138+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15888525255379993229', time=2024-04-22T22:27:00.968211+0000, chain.objs.size()=1 2024-04-22T20:48:17.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.138+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.278__shadow_.GwEvFBnKkKMEtH9zTjAVsZI3Di_ECU2_0 2024-04-22T20:48:17.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.138+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2875279917021029784', time=2024-04-22T22:27:05.709319+0000, chain.objs.size()=1 2024-04-22T20:48:17.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.138+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.4jthaGhhDoosJwg2dHERSFZ8HBBj-zt_0 2024-04-22T20:48:17.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.138+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6644447577506114303', time=2024-04-22T22:27:05.710011+0000, chain.objs.size()=1 2024-04-22T20:48:17.136 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.138+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.5NdfWwVc7-lxtQHkWH9VBbAOAQb5gGO_0 2024-04-22T20:48:17.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.140+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10799551237505291802', time=2024-04-22T22:27:07.886445+0000, chain.objs.size()=1 2024-04-22T20:48:17.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.140+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.h09P8TjDBjmaymUtajjjEYvvs6HBUEk_0 2024-04-22T20:48:17.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.140+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14504662994626804904', time=2024-04-22T22:27:19.246742+0000, chain.objs.size()=1 2024-04-22T20:48:17.137 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.140+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.293__shadow_.f44VtEHPNVwtR7_D67jcvYIbzcXpWXj_0 2024-04-22T20:48:17.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17458972311313366548', time=2024-04-22T22:28:17.712738+0000, chain.objs.size()=1 2024-04-22T20:48:17.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MG9j7e87XpBHFMxfXNikGOgY-W-h5ob_0 2024-04-22T20:48:17.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14352782932059471644', time=2024-04-22T22:28:17.952726+0000, chain.objs.size()=1 2024-04-22T20:48:17.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.J4_8CnDkcDpuF5a_PmtJFi-lGHT_3pO_0 2024-04-22T20:48:17.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9964812556439459459', time=2024-04-22T22:28:18.132997+0000, chain.objs.size()=1 2024-04-22T20:48:17.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.--gP0lrleagVSp9AAxceBtbpxgnlmTG_0 2024-04-22T20:48:17.139 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2559371390413021418', time=2024-04-22T22:28:18.139333+0000, chain.objs.size()=1 2024-04-22T20:48:17.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xsepi9_I2CGXjxW6sfMJZ2IOmKsD_UT_0 2024-04-22T20:48:17.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15753504299740011057', time=2024-04-22T22:28:19.364954+0000, chain.objs.size()=1 2024-04-22T20:48:17.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.h4bDL8tMBU7Qg8wCvbykJZ1TP3GvSNB_0 2024-04-22T20:48:17.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3726831319004428415', time=2024-04-22T22:28:20.195523+0000, chain.objs.size()=1 2024-04-22T20:48:17.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.86_GbsIyog869Mgksxrl5-rUoRUpulE_0 2024-04-22T20:48:17.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9857924260465576663', time=2024-04-22T22:28:20.213058+0000, chain.objs.size()=1 2024-04-22T20:48:17.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Qk34i7w5JUz0Ij3Wz9Pe4jowvL5zD7u_0 2024-04-22T20:48:17.140 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7649731658485226529', time=2024-04-22T22:28:20.709959+0000, chain.objs.size()=1 2024-04-22T20:48:17.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FVJB1YXrRE1FLM2K6xKipqwGS-tI7vW_0 2024-04-22T20:48:17.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1012749488344357566', time=2024-04-22T22:28:21.353846+0000, chain.objs.size()=1 2024-04-22T20:48:17.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.H4gB6kayDzXxqhpTa3SFD26IMVFc7cO_0 2024-04-22T20:48:17.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8448971652398194443', time=2024-04-22T22:28:21.444503+0000, chain.objs.size()=1 2024-04-22T20:48:17.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.142+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zTaBTvgCS55UKrQOjJa_VNOzqYY3YM0_0 2024-04-22T20:48:17.141 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.143+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8959624810183134747', time=2024-04-22T22:28:22.728594+0000, chain.objs.size()=1 2024-04-22T20:48:17.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.143+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.aLTgs1My31IQHH2gegB5xNaWpZ_eBW8_0 2024-04-22T20:48:17.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6357011700688440626', time=2024-04-22T22:28:22.766697+0000, chain.objs.size()=1 2024-04-22T20:48:17.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kzVXFChQl0O_cAk64s0MxectX2yZvSq_0 2024-04-22T20:48:17.142 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14441596547802019555', time=2024-04-22T22:28:24.062486+0000, chain.objs.size()=1 2024-04-22T20:48:17.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bplA6FRKtu2Z_Wcfmry6CJwlzHpBtpD_0 2024-04-22T20:48:17.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4400157502969056591', time=2024-04-22T22:28:24.251848+0000, chain.objs.size()=1 2024-04-22T20:48:17.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LxtQDYdmV9_gh20WhsgJlK747JUkXco_0 2024-04-22T20:48:17.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5694369627833205398', time=2024-04-22T22:28:24.254072+0000, chain.objs.size()=1 2024-04-22T20:48:17.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lMyGzwYgwaPRaZY3HYgUpHtLTUtIXAH_0 2024-04-22T20:48:17.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12974535697791052672', time=2024-04-22T22:28:24.795179+0000, chain.objs.size()=1 2024-04-22T20:48:17.143 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vDBVuWEw7HZo_yJHLAXHVKH9NFh_PxI_0 2024-04-22T20:48:17.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12229575345506318036', time=2024-04-22T22:28:24.796496+0000, chain.objs.size()=1 2024-04-22T20:48:17.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OiPBcUcF7FWNvewGKi0pYJMzBZALELR_0 2024-04-22T20:48:17.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15774180481347782279', time=2024-04-22T22:28:24.860012+0000, chain.objs.size()=1 2024-04-22T20:48:17.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4uCIwFJqGvSEIEkaNHV8rnLeyzje2mp_0 2024-04-22T20:48:17.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10293332096920944896', time=2024-04-22T22:28:25.401702+0000, chain.objs.size()=1 2024-04-22T20:48:17.144 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.145+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1GHjzV8IBTELxtW7-07Ebt3D2_H_b7J_0 2024-04-22T20:48:17.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.147+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7432243487816828040', time=2024-04-22T22:28:25.451120+0000, chain.objs.size()=1 2024-04-22T20:48:17.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.147+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.P1yZe2IgrGsi-95acCSls-HUK3Ns7TR_0 2024-04-22T20:48:17.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.147+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14906477032666232322', time=2024-04-22T22:28:27.562535+0000, chain.objs.size()=1 2024-04-22T20:48:17.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.147+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.i_q5gJ6jMAJCQllSODmYcCpAvKx_D4m_0 2024-04-22T20:48:17.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.147+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15021234206242969645', time=2024-04-22T22:28:28.747963+0000, chain.objs.size()=1 2024-04-22T20:48:17.145 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.147+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._qy0GF5w9rvgW3zL69Q9ZFMLX1ufVKr_0 2024-04-22T20:48:17.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.149+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15311095888835001111', time=2024-04-22T22:29:06.396685+0000, chain.objs.size()=1 2024-04-22T20:48:17.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.149+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SVvohQ1TPAj1U6WBpNGC6CPCi495arn_0 2024-04-22T20:48:17.146 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11151886886579578670', time=2024-04-22T22:29:06.397169+0000, chain.objs.size()=1 2024-04-22T20:48:17.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DZbdEP-MMBrtyfIU-G82Mh0XewNY965_0 2024-04-22T20:48:17.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16865122868538650562', time=2024-04-22T22:29:06.773267+0000, chain.objs.size()=1 2024-04-22T20:48:17.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1kLJP4MYmYEniXXLtMpJEQcUL1DvWka_0 2024-04-22T20:48:17.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10436422727898655260', time=2024-04-22T22:29:07.159709+0000, chain.objs.size()=1 2024-04-22T20:48:17.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.AcumQ8iRoIYutHw8B5_mPRrdx-brmoU_0 2024-04-22T20:48:17.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4252128862204388747', time=2024-04-22T22:29:07.284832+0000, chain.objs.size()=1 2024-04-22T20:48:17.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.naY2Uf2aM9obWlEXOH9EzV9Lsu9jfai_0 2024-04-22T20:48:17.147 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14519347143160212680', time=2024-04-22T22:29:07.814138+0000, chain.objs.size()=1 2024-04-22T20:48:17.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ePUy3U0L91pnyysOwDaak0wQxVNWdGv_0 2024-04-22T20:48:17.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14748551979663398248', time=2024-04-22T22:29:08.146507+0000, chain.objs.size()=1 2024-04-22T20:48:17.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HerwxqCjSc0OmVx4W7NkMCUxr3iNWAt_0 2024-04-22T20:48:17.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1898727517336909913', time=2024-04-22T22:29:08.882317+0000, chain.objs.size()=1 2024-04-22T20:48:17.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.kCtK4a7bivb9miYVojIePKDU2mZux9e_0 2024-04-22T20:48:17.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3426380406168035253', time=2024-04-22T22:29:09.568990+0000, chain.objs.size()=1 2024-04-22T20:48:17.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RNpEAVtVr9QKwX4t_3r4Y8Tj6djr7Ma_0 2024-04-22T20:48:17.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1369310413280205326', time=2024-04-22T22:29:09.751660+0000, chain.objs.size()=1 2024-04-22T20:48:17.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZBoqygRCTPtqEm8UjctDsEnfjlETiFd_0 2024-04-22T20:48:17.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3066450667206358379', time=2024-04-22T22:29:10.094147+0000, chain.objs.size()=1 2024-04-22T20:48:17.149 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.150+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GayGjSBG7n-bllPJfcqMPIFtT_z9229_0 2024-04-22T20:48:17.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.153+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.570150035799031934', time=2024-04-22T22:29:10.241317+0000, chain.objs.size()=1 2024-04-22T20:48:17.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.fwMsXro2PT15lzFR-qRKCIFuC1nAss2_0 2024-04-22T20:48:17.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16291221907515669059', time=2024-04-22T22:29:10.354570+0000, chain.objs.size()=1 2024-04-22T20:48:17.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XDj1a7LFi6JLOPuXKklh8QN4LmQnvty_0 2024-04-22T20:48:17.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8370750711490751646', time=2024-04-22T22:29:11.653642+0000, chain.objs.size()=1 2024-04-22T20:48:17.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RgGZnqJxwLX2c6drrdy0FRs9X6ECdiZ_0 2024-04-22T20:48:17.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3262096179118375141', time=2024-04-22T22:29:11.692435+0000, chain.objs.size()=1 2024-04-22T20:48:17.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qV91iFXgGJtFGHFu9WTC6cYUYq8B8EU_0 2024-04-22T20:48:17.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16266291031437648104', time=2024-04-22T22:29:11.693139+0000, chain.objs.size()=1 2024-04-22T20:48:17.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2ofQlOSguFxnSU_RzgDaULwxRoL7Tzi_0 2024-04-22T20:48:17.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18010237215242554150', time=2024-04-22T22:29:11.964553+0000, chain.objs.size()=1 2024-04-22T20:48:17.152 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.154+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6urLDzLqZfZl5XKXYpln_yjlJ1v3LUG_0 2024-04-22T20:48:17.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18259640850445259981', time=2024-04-22T22:29:11.993750+0000, chain.objs.size()=1 2024-04-22T20:48:17.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LDwfla-03nom9HCQu-UGZfv07QtoKcL_0 2024-04-22T20:48:17.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2730261315971510156', time=2024-04-22T22:29:12.960436+0000, chain.objs.size()=1 2024-04-22T20:48:17.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.O9WIYYnX0hF4_DwwVUBwkX2a49uqmYR_0 2024-04-22T20:48:17.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12838484463776190161', time=2024-04-22T22:29:13.138731+0000, chain.objs.size()=1 2024-04-22T20:48:17.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jQ4WHTE1paG2TmIe7SR9H8ZaLBfbNf3_0 2024-04-22T20:48:17.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7321735075166588811', time=2024-04-22T22:29:13.364014+0000, chain.objs.size()=1 2024-04-22T20:48:17.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-B_qQE13vB1UhfuQl5XNJ_gTIB7Hmem_0 2024-04-22T20:48:17.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4122005337182787361', time=2024-04-22T22:29:13.488340+0000, chain.objs.size()=1 2024-04-22T20:48:17.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0AOc1Xd7VCLLPMwYnAgcgoN-wUDB4hU_0 2024-04-22T20:48:17.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2535483875282657284', time=2024-04-22T22:29:13.488942+0000, chain.objs.size()=1 2024-04-22T20:48:17.154 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.156+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.N81z-CDQ8GAStLq1r4OCJjMx2qJT-ED_0 2024-04-22T20:48:17.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.157+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4114478322324546357', time=2024-04-22T22:29:14.382257+0000, chain.objs.size()=1 2024-04-22T20:48:17.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.157+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.craW9nk0fS1Sj8FQbpRp0myMHBGXlB-_0 2024-04-22T20:48:17.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.157+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11336659951592457578', time=2024-04-22T22:29:14.411072+0000, chain.objs.size()=1 2024-04-22T20:48:17.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.157+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gIXfrSPsrs2bQkDUVvWCXWkvMudBRza_0 2024-04-22T20:48:17.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.158+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9515443349235681913', time=2024-04-22T22:29:14.504971+0000, chain.objs.size()=1 2024-04-22T20:48:17.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.158+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.fRugyfjW_LfVNkzPTOjfy8Re-XV90LO_0 2024-04-22T20:48:17.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.158+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1609269897692646274', time=2024-04-22T22:29:14.512169+0000, chain.objs.size()=1 2024-04-22T20:48:17.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.158+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GQ-kiVvT0c7mAUUWCnBJNhGh9lzAw5S_0 2024-04-22T20:48:17.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.158+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15736489189789752237', time=2024-04-22T22:29:14.702639+0000, chain.objs.size()=1 2024-04-22T20:48:17.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.158+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rdXY8zzkWTkTNE3ovBaSHm3HhChC3iH_0 2024-04-22T20:48:17.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.161+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18010421176573859489', time=2024-04-22T22:29:15.309400+0000, chain.objs.size()=1 2024-04-22T20:48:17.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.161+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OofaoJ8M--_7wUlfasSnSyKi1rXOC-j_0 2024-04-22T20:48:17.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.161+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8758742594793653188', time=2024-04-22T22:29:15.365366+0000, chain.objs.size()=1 2024-04-22T20:48:17.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.161+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.5od5qf1rB8M7yB8skc4dB65xGysK8uM_0 2024-04-22T20:48:17.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.164+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/49488 2024-04-22T20:48:17.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=25, truncated=0, next_marker='' 2024-04-22T20:48:17.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4311373290802711156', time=2024-04-22T22:29:15.649484+0000, chain.objs.size()=1 2024-04-22T20:48:17.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.AJ-djnw6_Z_0gzNOWg8ZYiVJVRTj3fH_0 2024-04-22T20:48:17.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14065048967146418396', time=2024-04-22T22:29:15.775425+0000, chain.objs.size()=1 2024-04-22T20:48:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Gcp4my9mVb-GEnCoVw6mEIGJ7A1S3X-_0 2024-04-22T20:48:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18420224173999005447', time=2024-04-22T22:29:15.868797+0000, chain.objs.size()=1 2024-04-22T20:48:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZwGXesC60fYYOPZrgMhFAQbMQDeAYS1_0 2024-04-22T20:48:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3073153486423173930', time=2024-04-22T22:29:15.993553+0000, chain.objs.size()=1 2024-04-22T20:48:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LLVu_hjLuM9mIKEPccAuF99ZfAfPvQO_0 2024-04-22T20:48:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8424928052639905363', time=2024-04-22T22:29:16.198450+0000, chain.objs.size()=1 2024-04-22T20:48:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jBLafULDUoNVrpHrO9-yHMDs4Npv7qm_0 2024-04-22T20:48:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1670475054334688009', time=2024-04-22T22:29:16.821022+0000, chain.objs.size()=1 2024-04-22T20:48:17.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PcluVG-JRt_gv4WQW5ah3ZcukkGYJpj_0 2024-04-22T20:48:17.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10769612437615582305', time=2024-04-22T22:29:16.899907+0000, chain.objs.size()=1 2024-04-22T20:48:17.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.eqjzwPE7WVN5bHtKndS-OHKfIYP3Iz8_0 2024-04-22T20:48:17.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7647949821524259815', time=2024-04-22T22:29:54.598277+0000, chain.objs.size()=1 2024-04-22T20:48:17.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.368__shadow_.WgSLmGLPrao6x77J_GD7lBgF64nor4E_0 2024-04-22T20:48:17.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10742736486624643941', time=2024-04-22T22:29:58.565463+0000, chain.objs.size()=1 2024-04-22T20:48:17.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.373__shadow_.S1oPW2eVjool96TqXOeyaaPchKrcMWV_0 2024-04-22T20:48:17.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14019027304433151421', time=2024-04-22T22:31:10.500232+0000, chain.objs.size()=1 2024-04-22T20:48:17.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.Rzpf68pZQGbii7HSdtMd0Lai-hZI3LU_0 2024-04-22T20:48:17.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16663432548728654685', time=2024-04-22T22:31:10.501059+0000, chain.objs.size()=1 2024-04-22T20:48:17.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.JhnEXlxQjdoz_pVuFMQsdoJRCZqKlNg_0 2024-04-22T20:48:17.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15528173249109225253', time=2024-04-22T22:31:11.423052+0000, chain.objs.size()=1 2024-04-22T20:48:17.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.166+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.473__shadow_.ae9IgqhZn3dAG9PIdoc4Y_UPWy0V8Ao_0 2024-04-22T20:48:17.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.169+0000 7f2f5ce44640 0 -- 172.21.15.136:0/416170015 >> [v2:172.21.15.164:6824/100873265,v1:172.21.15.164:6825/100873265] conn(0x55df04f6d710 msgr2=0x55df04f6fb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:48:17.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.489__shadow_.Gw_ON63wph6ghE8hz-zYs4BgNCPKD0G_1_5c3cb9add46abd5d244beb6206cd1c33', time=2024-04-22T22:31:24.937180+0000, chain.objs.size()=1 2024-04-22T20:48:17.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.489__shadow_.Gw_ON63wph6ghE8hz-zYs4BgNCPKD0G_0 2024-04-22T20:48:17.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5091719552106537280', time=2024-04-22T22:36:08.055671+0000, chain.objs.size()=4 2024-04-22T20:48:17.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.524__multipart_mymultipart.2~3m4Xy8p5EEo0corrHLt5mzu65Rqt1oL.1 2024-04-22T20:48:17.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.524__multipart_mymultipart.2~3m4Xy8p5EEo0corrHLt5mzu65Rqt1oL.2 2024-04-22T20:48:17.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.524__multipart_mymultipart.6O1TBlIxChRR9VswR1VN3IbQUMD9BI0q.3 2024-04-22T20:48:17.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.524__multipart_mymultipart.2~3m4Xy8p5EEo0corrHLt5mzu65Rqt1oL.4 2024-04-22T20:48:17.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12730758784360616845', time=2024-04-22T22:36:34.519252+0000, chain.objs.size()=1 2024-04-22T20:48:17.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.539__shadow_.iiIcItVcFUJPOUoLKscw_s3qQpMQnEe_0 2024-04-22T20:48:17.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10587290828122455012', time=2024-04-22T22:37:24.249492+0000, chain.objs.size()=1 2024-04-22T20:48:17.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:3hjxCgZleqmoat6RaSRrzg6wvVJPJ6Y_.ZQPj3jH6OKjhxVVt2AzGNZoO0if66e7_0 2024-04-22T20:48:17.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3147205508323460720', time=2024-04-22T22:37:33.734034+0000, chain.objs.size()=1 2024-04-22T20:48:17.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.558__shadow:RJFnI3OO8FTGXHXXuWcy6URoktKbhmi_.7S0lIijbVPbi_JyjkMGLOwIOQhs1pyH_0 2024-04-22T20:48:17.168 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10755088825478538544', time=2024-04-22T22:38:00.162928+0000, chain.objs.size()=1 2024-04-22T20:48:17.169 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:pln6AWYV6U23mvJt5xbhlcIeAVfTbtr_.bPORWyKnoMEnWtrwDOLJF08kgdr6xGN_0 2024-04-22T20:48:17.169 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12402775966519882857', time=2024-04-22T22:42:18.926313+0000, chain.objs.size()=2 2024-04-22T20:48:17.169 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.634__shadow_.-vD2zDA7vf19mZlfKX1G1YKg_LQUXV5_0 2024-04-22T20:48:17.169 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.170+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.634__shadow_.-vD2zDA7vf19mZlfKX1G1YKg_LQUXV5_1 2024-04-22T20:48:17.170 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.173+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16933621722704699857', time=2024-04-22T22:42:55.906849+0000, chain.objs.size()=1 2024-04-22T20:48:17.170 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.173+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.650__shadow_.GWYpGRUyD7x0QxvrolN1oBqTr5PmiFg_0 2024-04-22T20:48:17.170 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.173+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.650__shadow_.GWYpGRUyD7x0QxvrolN1oBqTr5PmiFg_1_1212b832065835b7955e8eb5a5249386', time=2024-04-22T22:42:56.081857+0000, chain.objs.size()=1 2024-04-22T20:48:17.170 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.173+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.650__shadow_.GWYpGRUyD7x0QxvrolN1oBqTr5PmiFg_0 2024-04-22T20:48:17.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.173+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9217294148859936322', time=2024-04-22T22:43:09.948735+0000, chain.objs.size()=1 2024-04-22T20:48:17.171 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.173+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.675__shadow_.w_Gp3CorEotq90iij6VtIOQzF5OOpdV_0 2024-04-22T20:48:17.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.175+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12383319962457083470', time=2024-04-22T22:43:15.672549+0000, chain.objs.size()=1 2024-04-22T20:48:17.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.175+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.684__shadow:HApinXobtKBIzGYIBUWCEhbx5mcktJT_.RZvILPPf4z44uJh2WDHtSji7tGsvI1h_0 2024-04-22T20:48:17.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.175+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13069578112827908185', time=2024-04-22T22:43:18.559691+0000, chain.objs.size()=1 2024-04-22T20:48:17.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.175+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.688__shadow:StYO7eL3uXxq.84aGa3KnLOwHu.XgWf_.RlfMd9hGA2CDPUVAc3qb6qNEbvnRor3_0 2024-04-22T20:48:17.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.175+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10758592489647830146', time=2024-04-22T22:43:46.091130+0000, chain.objs.size()=1 2024-04-22T20:48:17.172 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.175+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.701__shadow:jdbIuyTe51OHclUHhaK4kEP.1zweYrz_.o68nDu7MzGa4tYQNEf6s0IH48bGyFc5_0 2024-04-22T20:48:17.174 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.178+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.176 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.180+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-22T20:48:17.178 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.181+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.178 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.182+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5de46640 0 -- 172.21.15.136:0/416170015 >> [v2:172.21.15.136:6816/3821859466,v1:172.21.15.136:6817/3821859466] conn(0x55df04f69460 msgr2=0x55df04f6b8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:48:17.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/48638' 2024-04-22T20:48:17.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_6wvLMuvPAg5ylnaUKubGSJgqiP3KlIe', time=2024-04-22T22:21:55.309559+0000, chain.objs.size()=3 2024-04-22T20:48:17.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.XNB74i53XMCusL5r3-bIK5PrAKKzAh8_0 2024-04-22T20:48:17.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.XNB74i53XMCusL5r3-bIK5PrAKKzAh8_1 2024-04-22T20:48:17.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.XNB74i53XMCusL5r3-bIK5PrAKKzAh8_2 2024-04-22T20:48:17.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5221000737255402622', time=2024-04-22T22:22:28.787170+0000, chain.objs.size()=1 2024-04-22T20:48:17.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.155__shadow_.PRDO8R3YapZtFIPTm32Zp_Asrk4Lhnz_0 2024-04-22T20:48:17.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.315390682643756046', time=2024-04-22T22:25:23.435662+0000, chain.objs.size()=1 2024-04-22T20:48:17.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.229__shadow_.A-zTyWwULq--hRLRuTBhm6oaU811Fmb_0 2024-04-22T20:48:17.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17074677386548030685', time=2024-04-22T22:25:39.792405+0000, chain.objs.size()=1 2024-04-22T20:48:17.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.244__shadow_.CA7PIhPO0ltAXFF9VVsXAJ8eod1ZlSl_0 2024-04-22T20:48:17.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6398612929141479428', time=2024-04-22T22:26:22.919547+0000, chain.objs.size()=1 2024-04-22T20:48:17.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.fc_uB595Ro-plYKIuCbldNyWhv5MHEE_0 2024-04-22T20:48:17.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5163218275425572577', time=2024-04-22T22:26:24.007778+0000, chain.objs.size()=1 2024-04-22T20:48:17.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hP8iOBXRizxXr7igeT96SQlB2I5xZQi_0 2024-04-22T20:48:17.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17376120382316263934', time=2024-04-22T22:26:24.262814+0000, chain.objs.size()=1 2024-04-22T20:48:17.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-ennqLS8x5D06DJdiKPqRbIZsAJW8VG_0 2024-04-22T20:48:17.182 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2317371327447500439', time=2024-04-22T22:26:24.539717+0000, chain.objs.size()=1 2024-04-22T20:48:17.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-_9f8ssvt3H5zjeTiK99q3KyIYl9vg7_0 2024-04-22T20:48:17.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16242227262355087642', time=2024-04-22T22:26:25.127596+0000, chain.objs.size()=1 2024-04-22T20:48:17.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aQ1ol5DQ5W1GHak6O-hXzMFzjJdQyyx_0 2024-04-22T20:48:17.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.183731982563219526', time=2024-04-22T22:26:25.819730+0000, chain.objs.size()=1 2024-04-22T20:48:17.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.183+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vKuzl8hK5d-fKYAtrEcMBKp7fWOOSFo_0 2024-04-22T20:48:17.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14508487267728743494', time=2024-04-22T22:26:25.946340+0000, chain.objs.size()=1 2024-04-22T20:48:17.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uBV-z7MlezE-Jqntj79rkmw9bQhL6Yl_0 2024-04-22T20:48:17.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5d645640 0 -- 172.21.15.136:0/416170015 >> [v2:172.21.15.136:6824/3414382223,v1:172.21.15.136:6825/3414382223] conn(0x7f2e2807f6b0 msgr2=0x7f2e2807eb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:48:17.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2085497912391245885', time=2024-04-22T22:26:26.328855+0000, chain.objs.size()=1 2024-04-22T20:48:17.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bACD818jm28naJ04w72LUiaUGYOtPiI_0 2024-04-22T20:48:17.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10023044399919116990', time=2024-04-22T22:26:26.459027+0000, chain.objs.size()=1 2024-04-22T20:48:17.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LS9FeLZG08UaBrZEvrzOTu_15q2CMjT_0 2024-04-22T20:48:17.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5093220339240701719', time=2024-04-22T22:26:26.756289+0000, chain.objs.size()=1 2024-04-22T20:48:17.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vkMqv2BtnzW-eU2tuRcUfkKLQO6BUcf_0 2024-04-22T20:48:17.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10695418695717883880', time=2024-04-22T22:26:26.883430+0000, chain.objs.size()=1 2024-04-22T20:48:17.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._bTiFE_SJIYZ1cmAkn8o63JookWSt5L_0 2024-04-22T20:48:17.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3368185334338634441', time=2024-04-22T22:26:26.936321+0000, chain.objs.size()=1 2024-04-22T20:48:17.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.trdgA7NeHZEpXRKe9HNY3hmCrGITot2_0 2024-04-22T20:48:17.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7749421492137707762', time=2024-04-22T22:26:27.728597+0000, chain.objs.size()=1 2024-04-22T20:48:17.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2Fp5aqFdjapjbKY1fS2cKFA9klBPySs_0 2024-04-22T20:48:17.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3550023891522314855', time=2024-04-22T22:26:28.347919+0000, chain.objs.size()=1 2024-04-22T20:48:17.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DtbCpQPGwZ10xaTaBy0Gi3iTTTRBLEt_0 2024-04-22T20:48:17.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14988142270092943536', time=2024-04-22T22:26:28.537778+0000, chain.objs.size()=1 2024-04-22T20:48:17.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.RjeYobfoqjrn9WRYt3E98S9QEDjTbyR_0 2024-04-22T20:48:17.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3112069269854569078', time=2024-04-22T22:26:28.609928+0000, chain.objs.size()=1 2024-04-22T20:48:17.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.S-BQDhPOJNlx_SYhIaPcLh7yDig_1HF_0 2024-04-22T20:48:17.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3925878776579263175', time=2024-04-22T22:26:29.221369+0000, chain.objs.size()=1 2024-04-22T20:48:17.187 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.187+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oubc9Dd4z7rEqHbcUfQVw1-HPE-Q7yt_0 2024-04-22T20:48:17.189 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.743038378034190148', time=2024-04-22T22:26:29.274884+0000, chain.objs.size()=1 2024-04-22T20:48:17.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lx4Sy6a72dNrnMLUp8Grhd3VgRR0Vvs_0 2024-04-22T20:48:17.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16327937797925377918', time=2024-04-22T22:26:29.612710+0000, chain.objs.size()=1 2024-04-22T20:48:17.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zDjEuqnN0hlSg2G4C1-flFzuyeT0Ap0_0 2024-04-22T20:48:17.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6806010410376547541', time=2024-04-22T22:26:29.931589+0000, chain.objs.size()=1 2024-04-22T20:48:17.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.4FAx4GXxSpa8t3uiD4WzgfJnOjU_XjM_0 2024-04-22T20:48:17.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4615455706362879364', time=2024-04-22T22:26:30.551160+0000, chain.objs.size()=1 2024-04-22T20:48:17.190 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.QPfo1D2_sdvsEdq5jKtWWTJz5xkn5P2_0 2024-04-22T20:48:17.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16459009957649637489', time=2024-04-22T22:26:30.764310+0000, chain.objs.size()=1 2024-04-22T20:48:17.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.j5SsGxzbD30JmAaGAEWrxogs8e6CIBk_0 2024-04-22T20:48:17.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17223576319051330821', time=2024-04-22T22:26:30.835140+0000, chain.objs.size()=1 2024-04-22T20:48:17.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.192+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.fhK9FPYwTNkkXtrbpnlqVQfuDqWy1dB_0 2024-04-22T20:48:17.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4460005061364638568', time=2024-04-22T22:26:31.150972+0000, chain.objs.size()=1 2024-04-22T20:48:17.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8RuuqxlOHlap7UkWZn2mXBzmp9L3gzI_0 2024-04-22T20:48:17.191 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2258582121700237676', time=2024-04-22T22:26:31.539655+0000, chain.objs.size()=1 2024-04-22T20:48:17.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.9joVCLKhETixVQr31oy3uEVCRUaLTe-_0 2024-04-22T20:48:17.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15583352530217281521', time=2024-04-22T22:26:31.621436+0000, chain.objs.size()=1 2024-04-22T20:48:17.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BRX9Qd-ynJ2-vBJxg5KPeBt0lwvq8R__0 2024-04-22T20:48:17.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3639882987795466698', time=2024-04-22T22:26:31.692951+0000, chain.objs.size()=1 2024-04-22T20:48:17.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.219jCBpk0Q82qQLWExXNYAcYam-TWAP_0 2024-04-22T20:48:17.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12219073499202889363', time=2024-04-22T22:26:32.240129+0000, chain.objs.size()=1 2024-04-22T20:48:17.192 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.193+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vOgl0bbVrLNo214NJSEVPdUgjdbOEqB_0 2024-04-22T20:48:17.193 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12117450786292580114', time=2024-04-22T22:26:32.438279+0000, chain.objs.size()=1 2024-04-22T20:48:17.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.G14fflgEEzyzUOPxQGTaFdIarZT3Qpm_0 2024-04-22T20:48:17.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7550195360377666143', time=2024-04-22T22:26:32.601630+0000, chain.objs.size()=1 2024-04-22T20:48:17.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.QY-eNecmVVnelcJnWlm4PLlLdYqia9O_0 2024-04-22T20:48:17.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17826473071712550496', time=2024-04-22T22:26:32.780827+0000, chain.objs.size()=1 2024-04-22T20:48:17.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5TZXzunwgsawKgtTrTyR3YHk849s6eA_0 2024-04-22T20:48:17.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7110907789473660142', time=2024-04-22T22:26:33.156655+0000, chain.objs.size()=1 2024-04-22T20:48:17.194 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.j2mxhu2RNuVo42OgszP33Zqx8ARcC7t_0 2024-04-22T20:48:17.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10920746679718022800', time=2024-04-22T22:26:33.544495+0000, chain.objs.size()=1 2024-04-22T20:48:17.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2Uaax5IhGDu3I5EO-QCHVs1Kdyross9_0 2024-04-22T20:48:17.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2363713281263347358', time=2024-04-22T22:26:33.920605+0000, chain.objs.size()=1 2024-04-22T20:48:17.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LApZCToT4M10VMy40AGHBkz-VZxJbbB_0 2024-04-22T20:48:17.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.859114688124273999', time=2024-04-22T22:26:34.317870+0000, chain.objs.size()=1 2024-04-22T20:48:17.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zP1OmZq732LE9Tk-CKTHAcJ81jaXqKn_0 2024-04-22T20:48:17.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9855264810936916574', time=2024-04-22T22:26:49.861103+0000, chain.objs.size()=1 2024-04-22T20:48:17.195 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.267__shadow_.xQSO-quJQMluc1n5NiDAC314h54FgMI_0 2024-04-22T20:48:17.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9708486743438632327', time=2024-04-22T22:26:52.876550+0000, chain.objs.size()=1 2024-04-22T20:48:17.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.270__shadow_.a7NkwpvIaWnGwKvccb06X65GW1nM6qK_0 2024-04-22T20:48:17.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3506129948635509745', time=2024-04-22T22:26:56.937468+0000, chain.objs.size()=1 2024-04-22T20:48:17.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.274__shadow_.nGh4SNfj3NV12LyIlOWgQGX304aOqHK_0 2024-04-22T20:48:17.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16332717178296340035', time=2024-04-22T22:26:58.968077+0000, chain.objs.size()=1 2024-04-22T20:48:17.196 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.196+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.276__shadow_.lFBjmikoRvWv2SGUiHnYOgrjaGp5wdK_0 2024-04-22T20:48:17.197 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.199+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6097157422035607001', time=2024-04-22T22:26:58.975173+0000, chain.objs.size()=1 2024-04-22T20:48:17.197 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.199+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.276__shadow_.1LORO3p7jS3jHnebpqcjQVm2Dsv2Avn_0 2024-04-22T20:48:17.197 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.200+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15851874000438935570', time=2024-04-22T22:27:05.566753+0000, chain.objs.size()=1 2024-04-22T20:48:17.197 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.200+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.NdFNTgg2RISQe196OIBh8w9n9QqO2ho_0 2024-04-22T20:48:17.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2848266985566196362', time=2024-04-22T22:27:38.417586+0000, chain.objs.size()=1 2024-04-22T20:48:17.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.310__shadow_.KvFVG8mTGw-YP_kMGVchXm0yY_xrBq0_0 2024-04-22T20:48:17.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17500391300730446511', time=2024-04-22T22:28:17.612803+0000, chain.objs.size()=1 2024-04-22T20:48:17.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uiQRnkiHkCa-Q58QzmQD2GHuPI7IqnJ_0 2024-04-22T20:48:17.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5165116526238595968', time=2024-04-22T22:28:17.815882+0000, chain.objs.size()=1 2024-04-22T20:48:17.199 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.re26WQsyxz8o2EQKuKNMHkBvjw7od0C_0 2024-04-22T20:48:17.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10786429389109024197', time=2024-04-22T22:28:17.890292+0000, chain.objs.size()=1 2024-04-22T20:48:17.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.2ONMy1Q4kpVAyUxaAg0UIIfykHsErBd_0 2024-04-22T20:48:17.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16148503146306418793', time=2024-04-22T22:28:17.930659+0000, chain.objs.size()=1 2024-04-22T20:48:17.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.o6Bie69FpJF71uO_vDQNsTpOzdHn9KV_0 2024-04-22T20:48:17.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14479316392526055879', time=2024-04-22T22:28:17.975488+0000, chain.objs.size()=1 2024-04-22T20:48:17.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.B-0nNYFkXgMVOMVGt4KtXRkMws4iYu7_0 2024-04-22T20:48:17.200 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1122701311827154643', time=2024-04-22T22:28:18.405200+0000, chain.objs.size()=1 2024-04-22T20:48:17.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.201+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PB94mJDiRJwSLzNtm4EkLRAxQb6TURp_0 2024-04-22T20:48:17.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.202+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3975913108314209524', time=2024-04-22T22:28:19.092987+0000, chain.objs.size()=1 2024-04-22T20:48:17.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.202+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lAbGw3EELoFGvG1iqjZPBzdt8ET7e0u_0 2024-04-22T20:48:17.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.202+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1307704391318570281', time=2024-04-22T22:28:20.029098+0000, chain.objs.size()=1 2024-04-22T20:48:17.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.202+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.baApWP6ECEWcMgPUY1Ft7HuCTGV35Rd_0 2024-04-22T20:48:17.201 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14157668170589446538', time=2024-04-22T22:28:22.476107+0000, chain.objs.size()=1 2024-04-22T20:48:17.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9MXQQoncZyTLr0-YfhhVdkt5Y3NGoaU_0 2024-04-22T20:48:17.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.204+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11505332091155485631', time=2024-04-22T22:28:22.824284+0000, chain.objs.size()=1 2024-04-22T20:48:17.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.204+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vz2taOAL7_1XQdu9F_6KDXu4vILFY9E_0 2024-04-22T20:48:17.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4125938009665092032', time=2024-04-22T22:28:23.104154+0000, chain.objs.size()=1 2024-04-22T20:48:17.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1jqjv44w2yz0re47OLKRWtYA0uzAvJM_0 2024-04-22T20:48:17.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13362552447001699501', time=2024-04-22T22:28:23.104743+0000, chain.objs.size()=1 2024-04-22T20:48:17.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CNPUfVjt1Jaz5See9cdJWD19zg843tG_0 2024-04-22T20:48:17.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5448032460067550811', time=2024-04-22T22:28:23.181688+0000, chain.objs.size()=1 2024-04-22T20:48:17.203 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kt5yDWUJY6XYV-8dGjDcaz-1jhaaNgf_0 2024-04-22T20:48:17.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5580029453746964774', time=2024-04-22T22:28:23.254102+0000, chain.objs.size()=1 2024-04-22T20:48:17.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kVZXMjHdFgRs99bsihVvo5vzo4GLJps_0 2024-04-22T20:48:17.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1925164538562742348', time=2024-04-22T22:28:23.867553+0000, chain.objs.size()=1 2024-04-22T20:48:17.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Vfek3uFUR8PsYUNWiOl9ALQl4J-WmZP_0 2024-04-22T20:48:17.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10864981897643925641', time=2024-04-22T22:28:23.907438+0000, chain.objs.size()=1 2024-04-22T20:48:17.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.M9F1WAHkxlUshNtz3yX6uAqo3N9o61h_0 2024-04-22T20:48:17.204 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3727050003270558311', time=2024-04-22T22:28:25.029880+0000, chain.objs.size()=1 2024-04-22T20:48:17.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Lp2Jiq9JP6D_wyOXXBPrQ2snCCllsMy_0 2024-04-22T20:48:17.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18225255399609775281', time=2024-04-22T22:28:25.031390+0000, chain.objs.size()=1 2024-04-22T20:48:17.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CANi2KiZvzfxSC0tDNiASa-J6jKiD8Y_0 2024-04-22T20:48:17.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3641120115561497256', time=2024-04-22T22:28:25.532687+0000, chain.objs.size()=1 2024-04-22T20:48:17.205 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.206+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Py39wfqfdrwp1XGrNC5x7Y8JVFGIvaY_0 2024-04-22T20:48:17.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.208+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9048290022906861287', time=2024-04-22T22:28:25.639108+0000, chain.objs.size()=1 2024-04-22T20:48:17.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.208+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.YV-z2PWje8L9mETh3liFn8rX1jlHxUF_0 2024-04-22T20:48:17.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.208+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6092986963270626193', time=2024-04-22T22:28:25.909784+0000, chain.objs.size()=1 2024-04-22T20:48:17.206 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.208+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.08D_SH_jKk_nmXECnnOMZZVv99JniZZ_0 2024-04-22T20:48:17.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18329276211243597713', time=2024-04-22T22:28:26.101120+0000, chain.objs.size()=1 2024-04-22T20:48:17.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0ILc82pYPZ8UNySvK-xeayobFpHPke5_0 2024-04-22T20:48:17.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12581255364839770845', time=2024-04-22T22:28:26.562330+0000, chain.objs.size()=1 2024-04-22T20:48:17.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LOK6B5iFdzrkNiaaiasj084K276f63J_0 2024-04-22T20:48:17.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6425111435452764900', time=2024-04-22T22:28:26.987145+0000, chain.objs.size()=1 2024-04-22T20:48:17.207 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LbUho9ELGMGS63EwyfmC8gtjgE6s0nR_0 2024-04-22T20:48:17.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.210+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14688127276226021197', time=2024-04-22T22:28:27.460995+0000, chain.objs.size()=1 2024-04-22T20:48:17.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.210+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ddqyJkDzd935q0XoSeocIti5NVL0Iar_0 2024-04-22T20:48:17.208 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11428873425627383171', time=2024-04-22T22:28:27.491274+0000, chain.objs.size()=1 2024-04-22T20:48:17.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LEKJawdND2tfrXyN-6LldBsoReEAu9U_0 2024-04-22T20:48:17.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17008993398665942348', time=2024-04-22T22:28:27.510269+0000, chain.objs.size()=1 2024-04-22T20:48:17.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3WfRPilkkpLtH0gompUeEXQtQIcTiXC_0 2024-04-22T20:48:17.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15156898396251187966', time=2024-04-22T22:28:27.617908+0000, chain.objs.size()=1 2024-04-22T20:48:17.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.t_iJdAryLWAx_HY2qCUAqiTcmr4vkWq_0 2024-04-22T20:48:17.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6368227895837606543', time=2024-04-22T22:28:27.992593+0000, chain.objs.size()=1 2024-04-22T20:48:17.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.QwQ52NNhwfsMRWPU0NkKsewGTbstUwL_0 2024-04-22T20:48:17.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15170697555552863118', time=2024-04-22T22:28:27.993375+0000, chain.objs.size()=1 2024-04-22T20:48:17.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Iz7xoD88Bq_EFn7gAOOC6QxAR9hlhUB_0 2024-04-22T20:48:17.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8068764196242850009', time=2024-04-22T22:28:28.050946+0000, chain.objs.size()=1 2024-04-22T20:48:17.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.211+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5xLGA7Nh5E5Lv7i_Ba-H0SLC25DeIsa_0 2024-04-22T20:48:17.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.212+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7132923343491529705', time=2024-04-22T22:28:28.581743+0000, chain.objs.size()=1 2024-04-22T20:48:17.210 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.212+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nsk-mRoFqe45d91KNDtbin6qz0woPsa_0 2024-04-22T20:48:17.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6446714272096057852', time=2024-04-22T22:28:28.791146+0000, chain.objs.size()=1 2024-04-22T20:48:17.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.okHIKgsWEcLAU8ZETPfcbu7HTfNj-fp_0 2024-04-22T20:48:17.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6030880178563341990', time=2024-04-22T22:28:28.971056+0000, chain.objs.size()=1 2024-04-22T20:48:17.212 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7_NAJyS2IuCCHUYOegNjIhetEIcjBr0_0 2024-04-22T20:48:17.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18109300807697883823', time=2024-04-22T22:28:29.296455+0000, chain.objs.size()=1 2024-04-22T20:48:17.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lnjlsHIMMUDDLa8HPcO2oHTvqh9Pljq_0 2024-04-22T20:48:17.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10639490742264350069', time=2024-04-22T22:29:06.166327+0000, chain.objs.size()=1 2024-04-22T20:48:17.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WkPTpNxOzE8emaM9NhMVrv5AyRch4nj_0 2024-04-22T20:48:17.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6099060689409705406', time=2024-04-22T22:29:06.365827+0000, chain.objs.size()=1 2024-04-22T20:48:17.213 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4xYl-7spSA9nfQ_YcZRja9PUuExd6ZY_0 2024-04-22T20:48:17.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16773362547164347923', time=2024-04-22T22:29:06.495676+0000, chain.objs.size()=1 2024-04-22T20:48:17.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mzbYIIvl3hFRvp63OYRrjdK_kDo6944_0 2024-04-22T20:48:17.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5624805100189809418', time=2024-04-22T22:29:06.772782+0000, chain.objs.size()=1 2024-04-22T20:48:17.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RGBv2U05J4MOfGTR7PHK9RiwuPJtvQ0_0 2024-04-22T20:48:17.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6389735327625757711', time=2024-04-22T22:29:07.152687+0000, chain.objs.size()=1 2024-04-22T20:48:17.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QdT5CqXqyIXoZpDHZwEVoXfrXRCUBzp_0 2024-04-22T20:48:17.214 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16469436014367078886', time=2024-04-22T22:29:07.850098+0000, chain.objs.size()=1 2024-04-22T20:48:17.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.H2wV-Ry1VFnFO6AbrBjZeh4c0NyWxE3_0 2024-04-22T20:48:17.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9703026455099003888', time=2024-04-22T22:29:08.051170+0000, chain.objs.size()=1 2024-04-22T20:48:17.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PIud7KNV97Y0BpG3s7eiyAWBF7oPxog_0 2024-04-22T20:48:17.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13017978167362605880', time=2024-04-22T22:29:08.324433+0000, chain.objs.size()=1 2024-04-22T20:48:17.215 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.215+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LPGFjB0ZMfgDQtcdMydGVSpPJFiyewI_0 2024-04-22T20:48:17.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.217+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15478370892520712417', time=2024-04-22T22:29:08.449144+0000, chain.objs.size()=1 2024-04-22T20:48:17.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.217+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GNm9UexkNDQNhr_3MthdM9ZyhLkKjJD_0 2024-04-22T20:48:17.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.218+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5221365848839937129', time=2024-04-22T22:29:09.519303+0000, chain.objs.size()=1 2024-04-22T20:48:17.216 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.218+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HheY_faeTT5UT-1iRxT1oPUyE9Hev-5_0 2024-04-22T20:48:17.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9892179772491134886', time=2024-04-22T22:29:09.555871+0000, chain.objs.size()=1 2024-04-22T20:48:17.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.vVPIFHeFsBRa1guVfG7oDgm7SP24BYc_0 2024-04-22T20:48:17.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4938364841780553497', time=2024-04-22T22:29:09.915163+0000, chain.objs.size()=1 2024-04-22T20:48:17.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QaMj8ZCE96krrrGojM5r1VtzTlzc2yV_0 2024-04-22T20:48:17.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16665083888529593948', time=2024-04-22T22:29:10.475818+0000, chain.objs.size()=1 2024-04-22T20:48:17.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.c8fRqWkjlqPT3KUoMyvAJBbkIKnwwL9_0 2024-04-22T20:48:17.217 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14806317274374607011', time=2024-04-22T22:29:11.221939+0000, chain.objs.size()=1 2024-04-22T20:48:17.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.G8zOjB84nSbbAtG0MynxRJi741stHeC_0 2024-04-22T20:48:17.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7185773170728069741', time=2024-04-22T22:29:11.335587+0000, chain.objs.size()=1 2024-04-22T20:48:17.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wIv-4rwni3oGNhI12JEBPSerYFWcEpx_0 2024-04-22T20:48:17.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13109439279646275676', time=2024-04-22T22:29:11.394988+0000, chain.objs.size()=1 2024-04-22T20:48:17.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Z-mFd0V69aQOZDHElCetCxzqBaU0gvt_0 2024-04-22T20:48:17.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11240055983748619538', time=2024-04-22T22:29:11.645672+0000, chain.objs.size()=1 2024-04-22T20:48:17.218 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1nj3qMfWNI1kewFPpZb7LbWb39eZ9ol_0 2024-04-22T20:48:17.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11314259992845983756', time=2024-04-22T22:29:12.009880+0000, chain.objs.size()=1 2024-04-22T20:48:17.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pPKVjZxd9XE_wWVgh_vEqi9Kc7v4ezL_0 2024-04-22T20:48:17.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2151473734847411463', time=2024-04-22T22:29:12.427638+0000, chain.objs.size()=1 2024-04-22T20:48:17.219 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.219+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NTT2KnnebBCYQbjcMViD6x2xjNS7Mbf_0 2024-04-22T20:48:17.220 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.224+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/48638 2024-04-22T20:48:17.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=30, truncated=0, next_marker='' 2024-04-22T20:48:17.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17522782469757566204', time=2024-04-22T22:29:12.611263+0000, chain.objs.size()=1 2024-04-22T20:48:17.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qdsF-Pa7reUDkCNM7IWNJemsWw2COxG_0 2024-04-22T20:48:17.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8550995125400574805', time=2024-04-22T22:29:12.748537+0000, chain.objs.size()=1 2024-04-22T20:48:17.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Ighwlx8Aj7HJ3nkCu9Kdn3Dya5-AEol_0 2024-04-22T20:48:17.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13484457691801725548', time=2024-04-22T22:29:13.164387+0000, chain.objs.size()=1 2024-04-22T20:48:17.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZisK00CE4CYFJ29y_blSV3NVYKyumaX_0 2024-04-22T20:48:17.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17398186915847387264', time=2024-04-22T22:29:13.345058+0000, chain.objs.size()=1 2024-04-22T20:48:17.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3suBkq06gI2L47FUY1jMq055ZuocwbE_0 2024-04-22T20:48:17.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16641349795126609846', time=2024-04-22T22:29:13.480966+0000, chain.objs.size()=1 2024-04-22T20:48:17.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sQ4Xsyna43oUAArgOSDDRFR3bqI3qEF_0 2024-04-22T20:48:17.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9393218923590810123', time=2024-04-22T22:29:13.907205+0000, chain.objs.size()=1 2024-04-22T20:48:17.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Ow2uP4SzqerTZ_rN8Bz_1g1s2l6M0QQ_0 2024-04-22T20:48:17.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11716883855453788345', time=2024-04-22T22:29:14.232913+0000, chain.objs.size()=1 2024-04-22T20:48:17.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SaUYtaNo08XsHbEc1y5S2FgMEROQOyL_0 2024-04-22T20:48:17.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.683091261517607091', time=2024-04-22T22:29:14.337743+0000, chain.objs.size()=1 2024-04-22T20:48:17.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HQxpOhQIBL64zZbW-8hAKuV_71ClEiI_0 2024-04-22T20:48:17.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5960138567266564526', time=2024-04-22T22:29:15.773520+0000, chain.objs.size()=1 2024-04-22T20:48:17.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2-PAqsRL9j4qcQn8-OT0fhm8Obd-Kxr_0 2024-04-22T20:48:17.226 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10311978800405477644', time=2024-04-22T22:29:16.370243+0000, chain.objs.size()=1 2024-04-22T20:48:17.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0cjJ9VpH_YWpS9_hvlXHOn9O3aZSUXt_0 2024-04-22T20:48:17.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18352943652675038860', time=2024-04-22T22:29:16.768554+0000, chain.objs.size()=1 2024-04-22T20:48:17.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tgr9ZCd3a8_86XZbIx0EgViHEJ3WuhX_0 2024-04-22T20:48:17.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5465422618086940169', time=2024-04-22T22:29:16.769710+0000, chain.objs.size()=1 2024-04-22T20:48:17.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.226+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IO84S2zYOjgCPYNP3QFffB2_HnaOoba_0 2024-04-22T20:48:17.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.229+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14039943194336487409', time=2024-04-22T22:29:16.915583+0000, chain.objs.size()=1 2024-04-22T20:48:17.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.229+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MR1sL89aw1ujpW_E2c7KP9XIsDgU5aP_0 2024-04-22T20:48:17.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10782846112802938979', time=2024-04-22T22:29:49.519143+0000, chain.objs.size()=1 2024-04-22T20:48:17.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.362__shadow_.XsRwy6cvE4R0mPImqDda9vJkipIvcqm_0 2024-04-22T20:48:17.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17205357019696972751', time=2024-04-22T22:30:35.613277+0000, chain.objs.size()=1 2024-04-22T20:48:17.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.428__shadow_.TOuiYK2fGIpQZze5MIBxIBQMtwre7Mp_0 2024-04-22T20:48:17.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15687929793578517113', time=2024-04-22T22:30:45.426302+0000, chain.objs.size()=1 2024-04-22T20:48:17.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.439__shadow_.KZWO_Ov0RmuY-Bt7nQG0oDFKH4lEYUu_0 2024-04-22T20:48:17.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3800547050923743605', time=2024-04-22T22:30:51.609689+0000, chain.objs.size()=1 2024-04-22T20:48:17.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.446__shadow_.FsZ8u0GZdKGJ4eEtaqnVqdwJd3Bl8Sq_0 2024-04-22T20:48:17.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17058790060083237022', time=2024-04-22T22:31:00.147245+0000, chain.objs.size()=1 2024-04-22T20:48:17.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.454__shadow_.TT3bhPYxiVVsiIftSK_vUUkkcHE9Mv1_0 2024-04-22T20:48:17.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11878892893820862276', time=2024-04-22T22:31:00.510846+0000, chain.objs.size()=1 2024-04-22T20:48:17.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.454__shadow_.i6N2GEJte7iJdmmdxMZ-ZROi92qqxgx_0 2024-04-22T20:48:17.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11440114093691711476', time=2024-04-22T22:32:07.077617+0000, chain.objs.size()=1 2024-04-22T20:48:17.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.507__multipart_mymultipart.2~yHaiF9aY1-IOWkff5-TdfmsOhd1wURJ.1 2024-04-22T20:48:17.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18199661813481926726', time=2024-04-22T22:34:59.561243+0000, chain.objs.size()=3 2024-04-22T20:48:17.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~4YTn8N8IPuT6dpfj7EEdtRTCJTPMQNo.1 2024-04-22T20:48:17.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~4YTn8N8IPuT6dpfj7EEdtRTCJTPMQNo.2 2024-04-22T20:48:17.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.230+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~4YTn8N8IPuT6dpfj7EEdtRTCJTPMQNo.3 2024-04-22T20:48:17.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.232+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3777057826259929634', time=2024-04-22T22:36:42.253094+0000, chain.objs.size()=1 2024-04-22T20:48:17.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.232+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.541__shadow_.lAIbRikC6ax1ZyMoYGiB10YY-S4F9JU_0 2024-04-22T20:48:17.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.233+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5029262681875371405', time=2024-04-22T22:37:32.228215+0000, chain.objs.size()=1 2024-04-22T20:48:17.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.233+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:RHuVL2seCeWI29T797Obo5ZNC8oURyC_.aq_r5BbUkg_kFCbC6hJKaMgRtZ-FRQV_0 2024-04-22T20:48:17.232 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.233+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5532153465194923264', time=2024-04-22T22:37:38.398958+0000, chain.objs.size()=1 2024-04-22T20:48:17.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.234+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow_.Gddie1T-Qvjh1S9kXHNXf-FQmaKuwtj_0 2024-04-22T20:48:17.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.234+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10622995342456254032', time=2024-04-22T22:37:59.032488+0000, chain.objs.size()=1 2024-04-22T20:48:17.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.234+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:wBjRzBbEw2QIR9Ds9pHwMQjUwLLLWLN_.ajqyDSqg7ZPZ8VL4oLhVT98oV6_PjNX_0 2024-04-22T20:48:17.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7761098448541584806', time=2024-04-22T22:38:49.594470+0000, chain.objs.size()=1 2024-04-22T20:48:17.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:-Ez53lMVN5nfC8xg4w4SkVencdaJfXp_.GQcjrIWjXxr1DcuQX7kPVabBbagXB44_0 2024-04-22T20:48:17.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6727095865477743480', time=2024-04-22T22:38:51.934536+0000, chain.objs.size()=1 2024-04-22T20:48:17.233 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:mYroPTRUGY7JJ0FlSIYgWmGLZUtmu1u_.uh-IosxowmGuW8QG8mHDszFV7eCd5qI_0 2024-04-22T20:48:17.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.78478724714354076', time=2024-04-22T22:38:53.912611+0000, chain.objs.size()=1 2024-04-22T20:48:17.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.571__shadow:x.mfJAgP76kPfLw-5f2BkoITJ.ehPxk_.ctiDIIqQyAty2YR5oOm9TGSabmx-7rX_0 2024-04-22T20:48:17.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11656325339115684411', time=2024-04-22T22:41:59.941313+0000, chain.objs.size()=1 2024-04-22T20:48:17.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.626__shadow_.PvK5SmtvL6PZKSBKzSVnO09kDbGl9oQ_0 2024-04-22T20:48:17.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.722291399779001681', time=2024-04-22T22:42:19.633369+0000, chain.objs.size()=1 2024-04-22T20:48:17.234 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.235+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.635__shadow_.ialh8GCSkVApk_qGi-OGdHfIExi97Co_0 2024-04-22T20:48:17.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.238+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.237 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.241+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-22T20:48:17.238 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.242+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.239 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.242+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42819' 2024-04-22T20:48:17.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8515634589099352319', time=2024-04-22T22:21:52.826983+0000, chain.objs.size()=3 2024-04-22T20:48:17.240 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.v57J8wARD9FZ53I6LPMpiDAHTHR8Uyo_0 2024-04-22T20:48:17.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.v57J8wARD9FZ53I6LPMpiDAHTHR8Uyo_1 2024-04-22T20:48:17.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.v57J8wARD9FZ53I6LPMpiDAHTHR8Uyo_2 2024-04-22T20:48:17.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3541477187613927255', time=2024-04-22T22:25:17.294381+0000, chain.objs.size()=1 2024-04-22T20:48:17.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.224__shadow_.sRG9Rtdi1h8pfISMIZ9JH0kH4OpQZ7X_0 2024-04-22T20:48:17.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.823600609835662919', time=2024-04-22T22:25:17.299086+0000, chain.objs.size()=1 2024-04-22T20:48:17.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.224__shadow_.0Wot67S1t-uWCa5-Ev2asAulMWesPiP_0 2024-04-22T20:48:17.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11651916655977935054', time=2024-04-22T22:25:18.284274+0000, chain.objs.size()=1 2024-04-22T20:48:17.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.225__shadow_.zvcLrUQDx_PSUSRtVe7PwtjF9GWmx7J_0 2024-04-22T20:48:17.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5705461779061205129', time=2024-04-22T22:25:28.866582+0000, chain.objs.size()=1 2024-04-22T20:48:17.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.233__shadow_._-gXDDz9wJIo0ujWD5qNrqnnFBo238u_0 2024-04-22T20:48:17.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4156847765377185002', time=2024-04-22T22:26:22.834075+0000, chain.objs.size()=1 2024-04-22T20:48:17.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.KnvrpxtgdMNLPG3YQz63bBeSOA7Uwqj_0 2024-04-22T20:48:17.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17060401336115725117', time=2024-04-22T22:26:22.874164+0000, chain.objs.size()=1 2024-04-22T20:48:17.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.C2mdRQI-wGidbruulhCc5G1wfz9IRj-_0 2024-04-22T20:48:17.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12017023230399447991', time=2024-04-22T22:26:23.438370+0000, chain.objs.size()=1 2024-04-22T20:48:17.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3MbmjfooRsk9lPn_fLEN3Y9KjAzKiwP_0 2024-04-22T20:48:17.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17707073285626201267', time=2024-04-22T22:26:24.180659+0000, chain.objs.size()=1 2024-04-22T20:48:17.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2A_yPqQj5FAO_ZzrmO0L1tEnEZw5dWR_0 2024-04-22T20:48:17.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2636018838410946202', time=2024-04-22T22:26:24.629780+0000, chain.objs.size()=1 2024-04-22T20:48:17.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.243+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vPS-x4ryFZh6YxDec-Xyn8wl5FNKbnD_0 2024-04-22T20:48:17.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4045466718065319258', time=2024-04-22T22:26:24.716113+0000, chain.objs.size()=1 2024-04-22T20:48:17.244 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MbTTHkgucRyhOHJ6XPETC6gJ3wahPS__0 2024-04-22T20:48:17.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7067762450212318079', time=2024-04-22T22:26:25.645436+0000, chain.objs.size()=1 2024-04-22T20:48:17.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.10a0zkl03WLYKHjoFlD2rs4T1WqGYr4_0 2024-04-22T20:48:17.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10382719314329523625', time=2024-04-22T22:26:25.974663+0000, chain.objs.size()=1 2024-04-22T20:48:17.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hSZfEEMDUBEtAtE80uHHxXUQb3H-fN8_0 2024-04-22T20:48:17.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7177746977979497087', time=2024-04-22T22:26:26.123451+0000, chain.objs.size()=1 2024-04-22T20:48:17.245 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.in-j0Hu32aH1GA8u7UqhQ72s0BICptK_0 2024-04-22T20:48:17.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9388081812594370460', time=2024-04-22T22:26:26.207182+0000, chain.objs.size()=1 2024-04-22T20:48:17.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cjI88kmaJurEVkwbgNyJDvBN0JXFVcR_0 2024-04-22T20:48:17.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15696063960502366154', time=2024-04-22T22:26:26.280634+0000, chain.objs.size()=1 2024-04-22T20:48:17.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hUqgEkhq9OyRUTWqhvWCazeLYonWPyY_0 2024-04-22T20:48:17.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4644298288876282351', time=2024-04-22T22:26:26.420010+0000, chain.objs.size()=1 2024-04-22T20:48:17.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HNtVZFgFihloiToPRSmRB7ZVjWBEUn0_0 2024-04-22T20:48:17.246 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6416729573679166386', time=2024-04-22T22:26:26.793341+0000, chain.objs.size()=1 2024-04-22T20:48:17.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oRxUB1QdhkULcRw5Nd_9y1n14U_Jtng_0 2024-04-22T20:48:17.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6973878551539910606', time=2024-04-22T22:26:26.980573+0000, chain.objs.size()=1 2024-04-22T20:48:17.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.o77hIAnjUOtlRotyf_QrzNDH29rUhM__0 2024-04-22T20:48:17.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4397271685417628966', time=2024-04-22T22:26:26.985878+0000, chain.objs.size()=1 2024-04-22T20:48:17.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6IEAJXjZk9nlCo6siX63AIrO8BJfkpS_0 2024-04-22T20:48:17.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10353280323695262154', time=2024-04-22T22:26:27.653618+0000, chain.objs.size()=1 2024-04-22T20:48:17.247 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.247+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0osZaQmrnOUAQ2cQcR_ogUwyHjV227M_0 2024-04-22T20:48:17.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.250+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14704755320694614491', time=2024-04-22T22:26:28.492510+0000, chain.objs.size()=1 2024-04-22T20:48:17.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.250+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.fTDf8tD5Z2dmYYl_HKVCUgQBWiJqZQf_0 2024-04-22T20:48:17.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10822485910979593518', time=2024-04-22T22:26:29.057784+0000, chain.objs.size()=1 2024-04-22T20:48:17.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UAve9V7vYoCaosATxJ1C4R3oIv_Gi0G_0 2024-04-22T20:48:17.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13625474775799418231', time=2024-04-22T22:26:29.316694+0000, chain.objs.size()=1 2024-04-22T20:48:17.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.URqBjgvihegBlg4BVKJeulM1Ie9Hmh1_0 2024-04-22T20:48:17.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6027216845587852579', time=2024-04-22T22:26:29.706712+0000, chain.objs.size()=1 2024-04-22T20:48:17.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.RSrnxuGeQrkGlkdsMvuVESVT0W7rADK_0 2024-04-22T20:48:17.249 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13133205722996100576', time=2024-04-22T22:26:29.874589+0000, chain.objs.size()=1 2024-04-22T20:48:17.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.thFd7kBeQAom3v8oZYfoc1UHZT0XbkP_0 2024-04-22T20:48:17.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4176323547919156519', time=2024-04-22T22:26:30.549861+0000, chain.objs.size()=1 2024-04-22T20:48:17.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.26qPNEQdst1Tm7fzPktGbqxqCsUUdsY_0 2024-04-22T20:48:17.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.231040865638100543', time=2024-04-22T22:26:30.550353+0000, chain.objs.size()=1 2024-04-22T20:48:17.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vE8y_zQ8506_xUFR3h-Os0f8jG_kEoY_0 2024-04-22T20:48:17.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7752317634871296605', time=2024-04-22T22:26:31.328536+0000, chain.objs.size()=1 2024-04-22T20:48:17.250 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EuDI8_wnrPZWfu8Cm2crJdsmCcjBKsq_0 2024-04-22T20:48:17.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9472780302068592755', time=2024-04-22T22:26:32.357984+0000, chain.objs.size()=1 2024-04-22T20:48:17.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.psXrNB2x7e42iu18CJ25Tqup95fpQcb_0 2024-04-22T20:48:17.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16005922398508781478', time=2024-04-22T22:26:32.881087+0000, chain.objs.size()=1 2024-04-22T20:48:17.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Ea5byD3aySm8Nfs9n4fc3fooma1_ENd_0 2024-04-22T20:48:17.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5498867725718998911', time=2024-04-22T22:26:32.895661+0000, chain.objs.size()=1 2024-04-22T20:48:17.251 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.251+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vgXXZOBWFhJy6_w6pzf_-pNKjA4Ntx3_0 2024-04-22T20:48:17.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.253+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.137594354825490551', time=2024-04-22T22:26:32.988667+0000, chain.objs.size()=1 2024-04-22T20:48:17.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.253+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BiD-zaxGXm6ZysuVTYbaVMJ32DAh38Z_0 2024-04-22T20:48:17.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11297309337974321278', time=2024-04-22T22:26:33.066643+0000, chain.objs.size()=1 2024-04-22T20:48:17.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.n1eS9q8BftE2mGAjIRcaY5LFZRP4LN3_0 2024-04-22T20:48:17.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9170355548550697442', time=2024-04-22T22:26:33.152043+0000, chain.objs.size()=1 2024-04-22T20:48:17.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5H1TU2eOqFIRmD_rEYGp2kIGBcuidle_0 2024-04-22T20:48:17.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.995158850420161587', time=2024-04-22T22:26:33.173926+0000, chain.objs.size()=1 2024-04-22T20:48:17.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gc3bhq9mlh4VHtqt0VORdRHY7RZ2qFZ_0 2024-04-22T20:48:17.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14163083921917197016', time=2024-04-22T22:26:33.502737+0000, chain.objs.size()=1 2024-04-22T20:48:17.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jPjJr41LSG540Ib6k4zZYs_0D6kHC5r_0 2024-04-22T20:48:17.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9519355150161083877', time=2024-04-22T22:26:33.503365+0000, chain.objs.size()=1 2024-04-22T20:48:17.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tQcoNBIINHJed7kT8dJ0UOL28iaXpI5_0 2024-04-22T20:48:17.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6844306697909714002', time=2024-04-22T22:26:33.512394+0000, chain.objs.size()=1 2024-04-22T20:48:17.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.44Bh5Kpa7d0ItipSgvgdPLfBdh2G8Bb_0 2024-04-22T20:48:17.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7322980056341792796', time=2024-04-22T22:26:33.840054+0000, chain.objs.size()=1 2024-04-22T20:48:17.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DKlEodp70AqKpFgG-U-WKpR39QAd2Uf_0 2024-04-22T20:48:17.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15117996425542706514', time=2024-04-22T22:26:34.194433+0000, chain.objs.size()=1 2024-04-22T20:48:17.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.255+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sargXsq7yUIJbc21OViz-Fej0VvA8tu_0 2024-04-22T20:48:17.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4622930990082691245', time=2024-04-22T22:26:34.364918+0000, chain.objs.size()=1 2024-04-22T20:48:17.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.RE5zUX41ytdp67nBZ6x8i0xaWFRfxV0_0 2024-04-22T20:48:17.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5729997051195742624', time=2024-04-22T22:26:34.386507+0000, chain.objs.size()=1 2024-04-22T20:48:17.256 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LPTPO1ffNIkmOkqbE48OoWq89wHk1Qw_0 2024-04-22T20:48:17.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.257+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12479568574810944982', time=2024-04-22T22:26:57.996058+0000, chain.objs.size()=1 2024-04-22T20:48:17.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.257+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.275__shadow_.yV5egXsW00b4ttykR23YoUCycLVxK7I_0 2024-04-22T20:48:17.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.258+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2770834125785044111', time=2024-04-22T22:27:14.095006+0000, chain.objs.size()=1 2024-04-22T20:48:17.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.258+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.288__shadow_.i8TpMB-MLJFjPC9nNwaOWVio5J7gIT9_0 2024-04-22T20:48:17.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.258+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5094249193822798525', time=2024-04-22T22:27:20.239516+0000, chain.objs.size()=1 2024-04-22T20:48:17.257 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.258+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.294__shadow_.Gju8nStzg6ad5IReLk4ONWZCaPbbSww_0 2024-04-22T20:48:17.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7204287826557231166', time=2024-04-22T22:27:34.448952+0000, chain.objs.size()=1 2024-04-22T20:48:17.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.304__shadow_.4qbMP4R9Y4nienAV47giuN1v3jxUHJ2_0 2024-04-22T20:48:17.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3242714860826330188', time=2024-04-22T22:28:17.691118+0000, chain.objs.size()=1 2024-04-22T20:48:17.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qt0OiYg66GCNuLpqKwykptFOW23GCn6_0 2024-04-22T20:48:17.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18406692088591184446', time=2024-04-22T22:28:17.806780+0000, chain.objs.size()=1 2024-04-22T20:48:17.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.W_LODE5TJ7FvLvwk3r4wENIxFf9oKn7_0 2024-04-22T20:48:17.258 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10741771165843449140', time=2024-04-22T22:28:17.953165+0000, chain.objs.size()=1 2024-04-22T20:48:17.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.VotX9-arLGkkh8yCBytgl-DDIpMHvD8_0 2024-04-22T20:48:17.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.156006018680426549', time=2024-04-22T22:28:19.421530+0000, chain.objs.size()=1 2024-04-22T20:48:17.259 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.259+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WsVoM_083mEV_AZqLaCVbc7RWyW9M5M_0 2024-04-22T20:48:17.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.260+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2933932180959405643', time=2024-04-22T22:28:19.882617+0000, chain.objs.size()=1 2024-04-22T20:48:17.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.260+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.eZbyVqeB5vuTdtsLR1dpENglgcJsYmH_0 2024-04-22T20:48:17.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.260+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3092579629492174352', time=2024-04-22T22:28:19.913612+0000, chain.objs.size()=1 2024-04-22T20:48:17.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.260+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kRIbkrP_MMBg6PPOD1ogv2hcQo6dO-B_0 2024-04-22T20:48:17.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6663615747788180388', time=2024-04-22T22:28:20.009155+0000, chain.objs.size()=1 2024-04-22T20:48:17.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wj39HT8sM5rXrTWFDa8hOgPqLWDnH00_0 2024-04-22T20:48:17.260 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.261+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7577599194897491408', time=2024-04-22T22:28:21.106466+0000, chain.objs.size()=1 2024-04-22T20:48:17.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.261+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bWGxE8fgi1l7Zi4vFbKLghYWTLkdxVs_0 2024-04-22T20:48:17.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.262+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10740068597419203562', time=2024-04-22T22:28:21.137098+0000, chain.objs.size()=1 2024-04-22T20:48:17.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.262+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.EewuycfH7Rk_JUXNCyx-TlxIT8aM2uG_0 2024-04-22T20:48:17.261 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.263+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1574290435346842250', time=2024-04-22T22:28:21.137642+0000, chain.objs.size()=1 2024-04-22T20:48:17.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.263+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.v5w813RJ5PAbBd92kWUv4TKsGm_4HJs_0 2024-04-22T20:48:17.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.263+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.704397601363339339', time=2024-04-22T22:28:21.155421+0000, chain.objs.size()=1 2024-04-22T20:48:17.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.263+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MRLYGwlSQlvy-q8pfxjLaIvXJleDpBB_0 2024-04-22T20:48:17.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.263+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2122879774903970451', time=2024-04-22T22:28:21.297893+0000, chain.objs.size()=1 2024-04-22T20:48:17.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.263+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Hk9FCIbbc0DLqSfTz-oBj9bIzpNVb6V_0 2024-04-22T20:48:17.262 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.119557414767116688', time=2024-04-22T22:28:21.504622+0000, chain.objs.size()=1 2024-04-22T20:48:17.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.sohgEFlZ0k0St3EAeyoPdMdeLgr1oKF_0 2024-04-22T20:48:17.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7169915739617082291', time=2024-04-22T22:28:21.770910+0000, chain.objs.size()=1 2024-04-22T20:48:17.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.YBtQD7zFnSHEK-ucX1IlRdyzxhlQruj_0 2024-04-22T20:48:17.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14911126940234883069', time=2024-04-22T22:28:22.355645+0000, chain.objs.size()=1 2024-04-22T20:48:17.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6KgpTJvrkp3mj5sVLpLOvGc6C2_VJUW_0 2024-04-22T20:48:17.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14485472730860078355', time=2024-04-22T22:28:22.490191+0000, chain.objs.size()=1 2024-04-22T20:48:17.263 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gS7J1N9cNIj6mp1p8M4xDgtMQsVEtMb_0 2024-04-22T20:48:17.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.227762085568763454', time=2024-04-22T22:28:22.505822+0000, chain.objs.size()=1 2024-04-22T20:48:17.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.264+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Da3tUQU0zcbJ3O3bkDW0YRAG7zLuwLG_0 2024-04-22T20:48:17.264 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.265+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10890670115526752900', time=2024-04-22T22:28:22.545309+0000, chain.objs.size()=1 2024-04-22T20:48:17.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.265+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yeC0lk-297hIEmkpRg5idSr9poAK_55_0 2024-04-22T20:48:17.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.266+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5807889995768980113', time=2024-04-22T22:28:23.079880+0000, chain.objs.size()=1 2024-04-22T20:48:17.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.266+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dZbqyYXzwTvtxhjdckjfL97zzWCxYce_0 2024-04-22T20:48:17.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.266+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.688636655300204726', time=2024-04-22T22:28:23.221134+0000, chain.objs.size()=1 2024-04-22T20:48:17.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.266+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.aart7ndqDDLkTxrgtJFkdWBOJqpKMu6_0 2024-04-22T20:48:17.265 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4974531691486912499', time=2024-04-22T22:28:23.237843+0000, chain.objs.size()=1 2024-04-22T20:48:17.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UY_HxcaAKjCbhCRToeY0GVpdYJ6T-Kw_0 2024-04-22T20:48:17.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13573952228397942499', time=2024-04-22T22:28:23.239159+0000, chain.objs.size()=1 2024-04-22T20:48:17.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LFu_OGzXu20sIXhrLBg2kfQ9gBbHtqR_0 2024-04-22T20:48:17.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3879849116574309228', time=2024-04-22T22:28:23.465050+0000, chain.objs.size()=1 2024-04-22T20:48:17.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TeoZVsjZ7jml4VSvy9W56tPwGV0eHJa_0 2024-04-22T20:48:17.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4038762709133252745', time=2024-04-22T22:28:24.254381+0000, chain.objs.size()=1 2024-04-22T20:48:17.266 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.p8pig2WvZmSU57Jxkm5AymXIAbTWeG9_0 2024-04-22T20:48:17.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12176479494175380961', time=2024-04-22T22:28:24.313144+0000, chain.objs.size()=1 2024-04-22T20:48:17.267 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yi2E1nfvix3sncVzBEuJMD8wVyRTeOG_0 2024-04-22T20:48:17.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5001417224624872149', time=2024-04-22T22:28:24.324658+0000, chain.objs.size()=1 2024-04-22T20:48:17.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gdTMGz43WSq8MCjRdDSZVFPo2IRnBjv_0 2024-04-22T20:48:17.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16194125670290456670', time=2024-04-22T22:28:24.918503+0000, chain.objs.size()=1 2024-04-22T20:48:17.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LE0pqdIyWcgWkazil3h4wB6u-XZ6Clw_0 2024-04-22T20:48:17.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8994973402086837647', time=2024-04-22T22:28:25.025233+0000, chain.objs.size()=1 2024-04-22T20:48:17.268 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.267+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZIgYZd_pTwNsnu4LYE-gGRyNuyL5UqD_0 2024-04-22T20:48:17.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.268+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3811967290828468123', time=2024-04-22T22:28:25.041775+0000, chain.objs.size()=1 2024-04-22T20:48:17.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.268+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3qWgj4fL8367iZd3dIfWIpBWUeLw0zb_0 2024-04-22T20:48:17.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.269+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11621502199770355616', time=2024-04-22T22:28:25.677212+0000, chain.objs.size()=1 2024-04-22T20:48:17.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.269+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PNynOKdQ4UKGxHZFvGII9ECp__HtABB_0 2024-04-22T20:48:17.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.269+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2588987579899661731', time=2024-04-22T22:28:25.788224+0000, chain.objs.size()=1 2024-04-22T20:48:17.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.269+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Ve3B7siNJ_LHUS0XqNQokHhJFuZsYs5_0 2024-04-22T20:48:17.269 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.271+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6202520391387632121', time=2024-04-22T22:28:26.304115+0000, chain.objs.size()=1 2024-04-22T20:48:17.270 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.271+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.A3yHjWv6h43feVc4jXvzY2qqZ04KHve_0 2024-04-22T20:48:17.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.273+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14389493037102960121', time=2024-04-22T22:28:26.547053+0000, chain.objs.size()=1 2024-04-22T20:48:17.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.273+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nFiK1K50PyOpYeyAkadnIlgMTJ7riLr_0 2024-04-22T20:48:17.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.273+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3292424273595237184', time=2024-04-22T22:28:27.389122+0000, chain.objs.size()=1 2024-04-22T20:48:17.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.273+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.poCtbDq4Y0kEvSyvShWlOdRmGEM2TVs_0 2024-04-22T20:48:17.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.273+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4360419439466656872', time=2024-04-22T22:28:27.638286+0000, chain.objs.size()=1 2024-04-22T20:48:17.271 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.273+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0qdCHxhtLCla-dkzKDK9SGqpYDMllYe_0 2024-04-22T20:48:17.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13418182026742276686', time=2024-04-22T22:28:27.884695+0000, chain.objs.size()=1 2024-04-22T20:48:17.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GkPR3qn3UyjEGVkg4i252XBJ8FShBKN_0 2024-04-22T20:48:17.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14879078416572507847', time=2024-04-22T22:28:28.061448+0000, chain.objs.size()=1 2024-04-22T20:48:17.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UDkV5MwXpir7VN5-pyIbeq6Zn6Yr7Ch_0 2024-04-22T20:48:17.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2800887033111323763', time=2024-04-22T22:28:28.820590+0000, chain.objs.size()=1 2024-04-22T20:48:17.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Hx0mY-zkbbAsUXDLBbDWgHfc0wUjno4_0 2024-04-22T20:48:17.272 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4625717567343184834', time=2024-04-22T22:28:29.101004+0000, chain.objs.size()=1 2024-04-22T20:48:17.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NgSi_kvTRSoGTdI4KsfwE59vyHeG9bk_0 2024-04-22T20:48:17.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4873585198551780673', time=2024-04-22T22:29:06.322584+0000, chain.objs.size()=1 2024-04-22T20:48:17.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7RWKDrUeEehfPhS5hIFFcN-wDd6kZq4_0 2024-04-22T20:48:17.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8786576498491732110', time=2024-04-22T22:29:06.496139+0000, chain.objs.size()=1 2024-04-22T20:48:17.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dTq70k422LItnmdET1wlJxRpzoROMF8_0 2024-04-22T20:48:17.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7699170829797111199', time=2024-04-22T22:29:06.673781+0000, chain.objs.size()=1 2024-04-22T20:48:17.273 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NmR9w_WdU-wfj9GHX7C5kVL_6yysIFM_0 2024-04-22T20:48:17.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.36657579329908550', time=2024-04-22T22:29:06.675789+0000, chain.objs.size()=1 2024-04-22T20:48:17.274 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.274+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.F_sCm-MqBiIiD3-aPvXSh8g0AA5iubD_0 2024-04-22T20:48:17.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.276+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12753406697804411759', time=2024-04-22T22:29:08.143755+0000, chain.objs.size()=1 2024-04-22T20:48:17.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.276+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Mv4KlJvN2ye0spTyCaVb8k0PhvRtSs3_0 2024-04-22T20:48:17.275 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7367566613222233531', time=2024-04-22T22:29:08.147203+0000, chain.objs.size()=1 2024-04-22T20:48:17.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.P2zAeI1YTtq6Y6ai7qdSBAGAG7zk7V6_0 2024-04-22T20:48:17.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.965182368340003299', time=2024-04-22T22:29:08.147856+0000, chain.objs.size()=1 2024-04-22T20:48:17.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SMw9LPHUeCY_XctXf7PS_1YL0aejB9z_0 2024-04-22T20:48:17.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10699354007478262094', time=2024-04-22T22:29:08.278132+0000, chain.objs.size()=1 2024-04-22T20:48:17.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.f8b9Ear0-jE5BcphJoUMTJRohVfumt7_0 2024-04-22T20:48:17.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11837784301971792092', time=2024-04-22T22:29:08.279411+0000, chain.objs.size()=1 2024-04-22T20:48:17.276 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.UG-8xaGr-wyzrZuCp6nM-nB4bjEVWCA_0 2024-04-22T20:48:17.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7602762531457048254', time=2024-04-22T22:29:08.543349+0000, chain.objs.size()=1 2024-04-22T20:48:17.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.fGX__4g7bLMR6g7Qux3Z7lgYd5DH7b7_0 2024-04-22T20:48:17.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6095317525252392979', time=2024-04-22T22:29:08.804601+0000, chain.objs.size()=1 2024-04-22T20:48:17.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sjEvXFEZMfblheWyR3aloWL3Lw9v0th_0 2024-04-22T20:48:17.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.220787001949482045', time=2024-04-22T22:29:09.552477+0000, chain.objs.size()=1 2024-04-22T20:48:17.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.FMlnB3_l7Q0Y-Sj3ZBDyow0b4V-qie__0 2024-04-22T20:48:17.277 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11205184270090544336', time=2024-04-22T22:29:09.891074+0000, chain.objs.size()=1 2024-04-22T20:48:17.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZTkvlY0rOf_5-UoW11b80w6yeWyU8gx_0 2024-04-22T20:48:17.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2803484619594434281', time=2024-04-22T22:29:10.252962+0000, chain.objs.size()=1 2024-04-22T20:48:17.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.278+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.j3To0Y1WxPcPMNvefcAECChW_f65KAt_0 2024-04-22T20:48:17.278 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.282+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/42819 2024-04-22T20:48:17.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=41, truncated=0, next_marker='' 2024-04-22T20:48:17.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1848356015145823040', time=2024-04-22T22:29:10.348082+0000, chain.objs.size()=1 2024-04-22T20:48:17.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.22CGoEwuYnOSiGwDSFn6q2Ob0f3fk68_0 2024-04-22T20:48:17.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10256554500233219535', time=2024-04-22T22:29:10.392128+0000, chain.objs.size()=1 2024-04-22T20:48:17.282 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pViXX3eaPccBc8eN4n4bvSr-4kDiVYH_0 2024-04-22T20:48:17.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16453197283813142215', time=2024-04-22T22:29:10.466749+0000, chain.objs.size()=1 2024-04-22T20:48:17.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._3JEfUkNdUlUsAXj2cHX4CJMCy1xAFc_0 2024-04-22T20:48:17.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12973056404988922310', time=2024-04-22T22:29:11.322488+0000, chain.objs.size()=1 2024-04-22T20:48:17.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.d_vYwsVX0YmrknPv2d_wusWoeNSHeik_0 2024-04-22T20:48:17.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11118734766729340430', time=2024-04-22T22:29:11.534609+0000, chain.objs.size()=1 2024-04-22T20:48:17.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GGlBJbQTxtFPZhqi6BiTF2IQ5RdX7jH_0 2024-04-22T20:48:17.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10648993341574444253', time=2024-04-22T22:29:11.633889+0000, chain.objs.size()=1 2024-04-22T20:48:17.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dChw7OGwKTn9rtmGbVzT0rVtlHgPRWd_0 2024-04-22T20:48:17.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16589126342232249456', time=2024-04-22T22:29:11.851326+0000, chain.objs.size()=1 2024-04-22T20:48:17.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.284+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.E6nUIY2ge0grRpRoqnld-4ykB06seP2_0 2024-04-22T20:48:17.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18000796921363102855', time=2024-04-22T22:29:12.727039+0000, chain.objs.size()=1 2024-04-22T20:48:17.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Tm4irY5Gaokod4NrUHkMjnK_JhAudRZ_0 2024-04-22T20:48:17.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11452971013547077477', time=2024-04-22T22:29:13.920909+0000, chain.objs.size()=1 2024-04-22T20:48:17.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IyMlMX5rZMGZKP29kHRsxPj_D8b_9yz_0 2024-04-22T20:48:17.285 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17237358946894518779', time=2024-04-22T22:29:13.922261+0000, chain.objs.size()=1 2024-04-22T20:48:17.285 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hlktNPV2Bjfh_qhT3A5k0o6Ax9YFoFX_0 2024-04-22T20:48:17.285 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13939798260240489157', time=2024-04-22T22:29:15.486508+0000, chain.objs.size()=1 2024-04-22T20:48:17.285 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iemilq4GMP5Kf72YFAlLM57isW7PpAu_0 2024-04-22T20:48:17.285 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3968769684281917898', time=2024-04-22T22:29:15.601150+0000, chain.objs.size()=1 2024-04-22T20:48:17.285 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.285+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.blu3GOKoerWDTHimj2W2D9HPFksOzWX_0 2024-04-22T20:48:17.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.286+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11351159801636988894', time=2024-04-22T22:29:15.946314+0000, chain.objs.size()=1 2024-04-22T20:48:17.286 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.286+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mJxjOJgCZ2XLcn3g_B_GYqDQDdoM3is_0 2024-04-22T20:48:17.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.288+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17569184471981079909', time=2024-04-22T22:29:15.956249+0000, chain.objs.size()=1 2024-04-22T20:48:17.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.288+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.lonDA2EWxwi8t7kP1SOJ36zojpcDtKg_0 2024-04-22T20:48:17.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.288+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8742338635554845891', time=2024-04-22T22:29:16.184094+0000, chain.objs.size()=1 2024-04-22T20:48:17.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.288+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8b5tPDRkzceeq5E-i1KEAePMLoZ59E9_0 2024-04-22T20:48:17.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16071886738280024865', time=2024-04-22T22:29:16.235140+0000, chain.objs.size()=1 2024-04-22T20:48:17.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8YAuxOhxjr-ak6VaoQfT_SAW1_1XyET_0 2024-04-22T20:48:17.287 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.431375281579494079', time=2024-04-22T22:29:16.373405+0000, chain.objs.size()=1 2024-04-22T20:48:17.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GddbTiZSCRVFX6QGeNho6eDXbAz2mCE_0 2024-04-22T20:48:17.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3460809176382641063', time=2024-04-22T22:29:17.105953+0000, chain.objs.size()=1 2024-04-22T20:48:17.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IYQ3JWEtwG9Bzqwpalt39Lmea45AtQS_0 2024-04-22T20:48:17.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10566469353076535169', time=2024-04-22T22:29:17.213230+0000, chain.objs.size()=1 2024-04-22T20:48:17.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.dvR9Un6gQ-gjlUkIhnIds9aJdY4f-Aw_0 2024-04-22T20:48:17.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13390935086458063954', time=2024-04-22T22:29:17.213816+0000, chain.objs.size()=1 2024-04-22T20:48:17.288 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TeWtKrEXy0rwKLWlnqJ8Zqrdj9q2bgS_0 2024-04-22T20:48:17.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13452857048686361165', time=2024-04-22T22:29:17.323301+0000, chain.objs.size()=1 2024-04-22T20:48:17.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HahwEfOm8bYkCfysv0B1UZUvP-QtQ7c_0 2024-04-22T20:48:17.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16095234792706576781', time=2024-04-22T22:29:17.324621+0000, chain.objs.size()=1 2024-04-22T20:48:17.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.CcY6YDltnkhP6pAcav3xP6lF6Kk0wt-_0 2024-04-22T20:48:17.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7323422514450143004', time=2024-04-22T22:29:17.421247+0000, chain.objs.size()=1 2024-04-22T20:48:17.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.fK5vj4W7Cegfvg2D11Guj2YIeWACVAU_0 2024-04-22T20:48:17.289 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5738751875815548619', time=2024-04-22T22:29:17.621130+0000, chain.objs.size()=1 2024-04-22T20:48:17.290 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.289+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.VaEdyAAJOwcuETa16FYPJNx0rj075Ho_0 2024-04-22T20:48:17.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17603032184408098891', time=2024-04-22T22:29:27.723926+0000, chain.objs.size()=1 2024-04-22T20:48:17.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.328__shadow_.77UFnX6mZFy-0eZ1KfFIP-ZHS4SI1X5_0 2024-04-22T20:48:17.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16851654477994265541', time=2024-04-22T22:30:31.834196+0000, chain.objs.size()=1 2024-04-22T20:48:17.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.423__shadow_.4y83vKzma7R7SA-fiubbrQDykRnyGcQ_0 2024-04-22T20:48:17.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.292+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10117459843608666923', time=2024-04-22T22:30:36.404145+0000, chain.objs.size()=1 2024-04-22T20:48:17.291 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.292+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.429__shadow_.--MQJ8jDnIRipML53iBF9vGU3dMs5s9_0 2024-04-22T20:48:17.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16320235111087580155', time=2024-04-22T22:30:51.606163+0000, chain.objs.size()=1 2024-04-22T20:48:17.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.446__shadow_.MGsyIaX7-TdaoKPsVdLUtKImCyn3pbg_0 2024-04-22T20:48:17.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3027022247499292630', time=2024-04-22T22:31:01.921113+0000, chain.objs.size()=1 2024-04-22T20:48:17.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.455__shadow_.F_N28Ra8Btrq7ys5HS_ao6I5Q4yFGGd_0 2024-04-22T20:48:17.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.476__shadow_.ZGAd0XLAkcZi9s6EFonT6Hhv6FfNYeY_1_b979322bafe22989138a9ead84732329', time=2024-04-22T22:31:16.309018+0000, chain.objs.size()=1 2024-04-22T20:48:17.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.476__shadow_.ZGAd0XLAkcZi9s6EFonT6Hhv6FfNYeY_0 2024-04-22T20:48:17.292 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.480__shadow_.7k8AsZNEwqGXhEo4l4Tbw8goeBMN7WT_1_ff92c84b53c986049b459a00b4bebac5', time=2024-04-22T22:31:19.509982+0000, chain.objs.size()=1 2024-04-22T20:48:17.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.480__shadow_.7k8AsZNEwqGXhEo4l4Tbw8goeBMN7WT_0 2024-04-22T20:48:17.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.485__shadow_.w2l_3sOFmMOJVmjO1JZYrPIBbJRaO_4_1_bf9d8918ad27d756db357112aabb79d9', time=2024-04-22T22:31:22.115793+0000, chain.objs.size()=1 2024-04-22T20:48:17.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.485__shadow_.w2l_3sOFmMOJVmjO1JZYrPIBbJRaO_4_0 2024-04-22T20:48:17.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17764158439026405429', time=2024-04-22T22:31:24.939524+0000, chain.objs.size()=1 2024-04-22T20:48:17.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.489__shadow_.Gw_ON63wph6ghE8hz-zYs4BgNCPKD0G_0 2024-04-22T20:48:17.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4796931018139935734', time=2024-04-22T22:37:06.447941+0000, chain.objs.size()=1 2024-04-22T20:48:17.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.546__shadow_.iTydNAqn1fcVSvqph3GrCRjAYrKpqCp_0 2024-04-22T20:48:17.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11820044973263714040', time=2024-04-22T22:37:38.978599+0000, chain.objs.size()=1 2024-04-22T20:48:17.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.293+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow:bBmwhm0NeSgVohEd5jH15wMX4eW3WlG_.PhlaT5_9YHHejluO3MJJMqhJ6EElS2L_0 2024-04-22T20:48:17.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10753741767164302110', time=2024-04-22T22:37:54.601790+0000, chain.objs.size()=1 2024-04-22T20:48:17.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:eS9psQ5zMV1K1oqpqWMTa98Nao0bkuD_.GjNwcQaPEalF0WI2gtZKKSrk_Fe9lTk_0 2024-04-22T20:48:17.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5389739247467431093', time=2024-04-22T22:37:55.822676+0000, chain.objs.size()=1 2024-04-22T20:48:17.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:pxyLMXnyuidXkByjGGEgeb.s-j5Ba9f_.b0i3wxt8g5RRqE4luVfV_1J0w31PvXo_0 2024-04-22T20:48:17.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18077486642907461750', time=2024-04-22T22:38:08.423952+0000, chain.objs.size()=1 2024-04-22T20:48:17.295 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:o21RQ.l3u-XiPk0cLAzQcnZrYUnNzd1_.Wb33l56w_vZuSud9dM2oH6NzskdBKt0_0 2024-04-22T20:48:17.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3438621319813408692', time=2024-04-22T22:38:09.518859+0000, chain.objs.size()=1 2024-04-22T20:48:17.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:R4sOAM25OO-9WgjKk.XVYiYU0S55dyx_.0EabLZPGh8xJlq5Jt36KiEHB9IIRwj5_0 2024-04-22T20:48:17.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12270646452734997911', time=2024-04-22T22:38:59.192716+0000, chain.objs.size()=1 2024-04-22T20:48:17.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.296+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:AiGTg1A5ktkjSTlgMrdSO7S7vy7AGiX_.jy5BCoHlsnUTKut1-tt43SKHVQsFWgY_0 2024-04-22T20:48:17.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.297+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6772383755203675826', time=2024-04-22T22:40:58.100643+0000, chain.objs.size()=1 2024-04-22T20:48:17.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.297+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.604__shadow_.v3WC7grUUM9_JGm-Lf5sl6XyzuMsHS4_0 2024-04-22T20:48:17.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.300+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.299 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.302+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-22T20:48:17.300 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.304+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.300 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.304+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.302 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42904' 2024-04-22T20:48:17.302 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16019348579931762784', time=2024-04-22T22:25:13.300511+0000, chain.objs.size()=1 2024-04-22T20:48:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.220__shadow_.An6_7JSHPVznowXubdBY9Lr9k2_tETL_0 2024-04-22T20:48:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10975650445107960980', time=2024-04-22T22:25:25.439016+0000, chain.objs.size()=1 2024-04-22T20:48:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.231__shadow_.g9iHWZTveGvsPZkAVP02WGj8IgJnc7F_0 2024-04-22T20:48:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18393741866283620148', time=2024-04-22T22:25:28.856222+0000, chain.objs.size()=1 2024-04-22T20:48:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.233__shadow_.bJUcKi3HrTNzojafgm7V12UuKVpk30j_0 2024-04-22T20:48:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5459322063224701627', time=2024-04-22T22:25:35.827983+0000, chain.objs.size()=1 2024-04-22T20:48:17.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.240__shadow_.7W0TLxDtF8p_JDl6fMRuhf6Dxh3BLw8_0 2024-04-22T20:48:17.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6631947802008602179', time=2024-04-22T22:26:22.326354+0000, chain.objs.size()=1 2024-04-22T20:48:17.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mABOzCb2wEamy3fXMFgMb8ANYpiqu3y_0 2024-04-22T20:48:17.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6480423279616301211', time=2024-04-22T22:26:23.429426+0000, chain.objs.size()=1 2024-04-22T20:48:17.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.O0sg_tVESz3qsR0flRlbPVDl0dCEWzc_0 2024-04-22T20:48:17.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15437837046955414500', time=2024-04-22T22:26:24.238662+0000, chain.objs.size()=1 2024-04-22T20:48:17.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WwAva0H3g_CciCfMg_keDYSN1zbv2ot_0 2024-04-22T20:48:17.304 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8329165176971040174', time=2024-04-22T22:26:25.660977+0000, chain.objs.size()=1 2024-04-22T20:48:17.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bluz0AQlUWMQnvH4XbW4GM8OdWDrART_0 2024-04-22T20:48:17.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13105278194720772213', time=2024-04-22T22:26:25.971098+0000, chain.objs.size()=1 2024-04-22T20:48:17.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1xUf_EOO7b5gPctMYZ8bseVGrEdsXuh_0 2024-04-22T20:48:17.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18134698739354443648', time=2024-04-22T22:26:26.224168+0000, chain.objs.size()=1 2024-04-22T20:48:17.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.3-giyrWnzZ_MrgC5e3vm4RK9J5DJPs2_0 2024-04-22T20:48:17.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16642909363643190607', time=2024-04-22T22:26:27.475627+0000, chain.objs.size()=1 2024-04-22T20:48:17.305 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.G-EuKvLoyxov0mx1aEK5ywqtFfWtqxF_0 2024-04-22T20:48:17.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2001416921639814181', time=2024-04-22T22:26:27.505803+0000, chain.objs.size()=1 2024-04-22T20:48:17.306 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.305+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.NEcA1KGwhwrJNQnxpXWhCSGNM_rByEu_0 2024-04-22T20:48:17.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.307+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8193563328900972443', time=2024-04-22T22:26:27.506470+0000, chain.objs.size()=1 2024-04-22T20:48:17.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.307+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.J3GNeVY7n3uSIR3cECB4pxtTNMRIxyf_0 2024-04-22T20:48:17.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.308+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8155908340838117172', time=2024-04-22T22:26:27.821082+0000, chain.objs.size()=1 2024-04-22T20:48:17.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.308+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jWf1tQPw7iZs-LwJb93wwIBrZjkV1Mi_0 2024-04-22T20:48:17.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3909670890574332943', time=2024-04-22T22:26:27.925129+0000, chain.objs.size()=1 2024-04-22T20:48:17.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xnTtYP8DjBSkHZfmzotVyjLDEXkE1Zc_0 2024-04-22T20:48:17.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13131675341158176538', time=2024-04-22T22:26:28.232666+0000, chain.objs.size()=1 2024-04-22T20:48:17.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6FkWIIVYNh0RvzYOpoEKCzk_0L4g9y1_0 2024-04-22T20:48:17.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18160331908210658705', time=2024-04-22T22:26:28.516477+0000, chain.objs.size()=1 2024-04-22T20:48:17.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.94m7yPRol7A2LyKYLoCVouxQIpF7jQY_0 2024-04-22T20:48:17.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7956462317916674265', time=2024-04-22T22:26:29.345928+0000, chain.objs.size()=1 2024-04-22T20:48:17.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.I4g07mGc4UImdRS-cpyUh2vMiyCFNmx_0 2024-04-22T20:48:17.308 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2836351390321946534', time=2024-04-22T22:26:29.346636+0000, chain.objs.size()=1 2024-04-22T20:48:17.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Hd4CcYUu4s6_z2enPpptfOKyfwnynMh_0 2024-04-22T20:48:17.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7187410465876046384', time=2024-04-22T22:26:29.460507+0000, chain.objs.size()=1 2024-04-22T20:48:17.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ry6ggdQT_JTEj3oQdaeMXxJtAdRcrdc_0 2024-04-22T20:48:17.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16335764852956910270', time=2024-04-22T22:26:30.031070+0000, chain.objs.size()=1 2024-04-22T20:48:17.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CT3-ymSxgwUf2GzmkKgOF_48Uoe2Nje_0 2024-04-22T20:48:17.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.647224293069522721', time=2024-04-22T22:26:32.053568+0000, chain.objs.size()=1 2024-04-22T20:48:17.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0e0bUd7KNGMkMHM-IGKlOjneLmjH1kN_0 2024-04-22T20:48:17.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18243039385589773468', time=2024-04-22T22:26:32.201283+0000, chain.objs.size()=1 2024-04-22T20:48:17.310 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.309+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qkZ-Mapr0zVv039HtkIcchX4oi7weLb_0 2024-04-22T20:48:17.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.311+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8637141281437772937', time=2024-04-22T22:26:32.657890+0000, chain.objs.size()=1 2024-04-22T20:48:17.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.311+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HV7kyOoRHB9mod0Iqwl-jh7tDLNECns_0 2024-04-22T20:48:17.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10639041021309467815', time=2024-04-22T22:26:33.707070+0000, chain.objs.size()=1 2024-04-22T20:48:17.311 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rBwYmbsuhzJbk-stZgTX5odRys43otX_0 2024-04-22T20:48:17.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7757893684222877621', time=2024-04-22T22:26:33.748934+0000, chain.objs.size()=1 2024-04-22T20:48:17.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VLviVFfySbKvieNSXj54bJmZN5IC8LY_0 2024-04-22T20:48:17.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14568871349047534791', time=2024-04-22T22:26:33.973701+0000, chain.objs.size()=1 2024-04-22T20:48:17.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-hyzuRVIW31A_P8Ir7gyK6MZG0WWEc6_0 2024-04-22T20:48:17.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12577321103207130954', time=2024-04-22T22:26:33.974365+0000, chain.objs.size()=1 2024-04-22T20:48:17.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZBe_R_RaHl9DZC8fCkdB79kdPjdNc6Q_0 2024-04-22T20:48:17.312 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6160569829035887003', time=2024-04-22T22:26:33.975135+0000, chain.objs.size()=1 2024-04-22T20:48:17.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EaoIwAk4_byJc_vUol79zpDbgFkNPcY_0 2024-04-22T20:48:17.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14684909446514557960', time=2024-04-22T22:26:34.177308+0000, chain.objs.size()=1 2024-04-22T20:48:17.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.313+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.i98VeLeBRZ-5RGn7mPiL7HbJlnGXqR4_0 2024-04-22T20:48:17.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.314+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.982865501256210441', time=2024-04-22T22:26:35.415184+0000, chain.objs.size()=1 2024-04-22T20:48:17.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.314+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.252__shadow_.CvN9GkJyllliPe2D3hljQ-mu6Y119T1_0 2024-04-22T20:48:17.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.314+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4911973130666795715', time=2024-04-22T22:26:37.351893+0000, chain.objs.size()=1 2024-04-22T20:48:17.313 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.314+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.254__shadow_.CNL9ze9rnAP2L8njejHMS2oOkrC8aqq_0 2024-04-22T20:48:17.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.314+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4599292187717291305', time=2024-04-22T22:26:48.878605+0000, chain.objs.size()=1 2024-04-22T20:48:17.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.314+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.266__shadow_.6e5e7giIgK9BR9i_RsEfCBQyeDwMqRW_0 2024-04-22T20:48:17.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.314+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16829569175309182991', time=2024-04-22T22:26:55.838819+0000, chain.objs.size()=1 2024-04-22T20:48:17.314 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.314+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.273__shadow_.88JDBRg0yF4kvPDDBX4pXJ1JKks-WjY_0 2024-04-22T20:48:17.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.315+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9467720415188442972', time=2024-04-22T22:26:57.996137+0000, chain.objs.size()=1 2024-04-22T20:48:17.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.315+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.275__shadow_.Mf6rgGZbfrand3-T66zBccznMTwJAmS_0 2024-04-22T20:48:17.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16744260694773736987', time=2024-04-22T22:27:00.959503+0000, chain.objs.size()=1 2024-04-22T20:48:17.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.278__shadow_.NtJ-QlG5fZ70hBxlKDSUWvlWPu8Bg4F_0 2024-04-22T20:48:17.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17295281801566480505', time=2024-04-22T22:27:15.087444+0000, chain.objs.size()=1 2024-04-22T20:48:17.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.289__shadow_.r4j0mH85rbYPnRDgBuah61GrQWdxBwB_0 2024-04-22T20:48:17.315 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1033175232744284028', time=2024-04-22T22:27:17.067359+0000, chain.objs.size()=1 2024-04-22T20:48:17.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.291__shadow_.6jKPRzGJJlHVhyl6oqnJ8Lg9kR41zpc_0 2024-04-22T20:48:17.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14491330087738499200', time=2024-04-22T22:27:18.098370+0000, chain.objs.size()=1 2024-04-22T20:48:17.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.292__shadow_.ZBGdI-mSHC_5y-3uBwzoVfhaUQDjzIN_0 2024-04-22T20:48:17.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6404119163253316608', time=2024-04-22T22:28:17.354028+0000, chain.objs.size()=1 2024-04-22T20:48:17.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.hJxltRcw0LqoLyxbn5C9sOStwe4Ak5n_0 2024-04-22T20:48:17.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4068772391570414413', time=2024-04-22T22:28:17.354781+0000, chain.objs.size()=1 2024-04-22T20:48:17.316 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Vo3BtGgTS3lzLkArEAD-R80A4NPiioC_0 2024-04-22T20:48:17.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4681200028270211542', time=2024-04-22T22:28:17.389901+0000, chain.objs.size()=1 2024-04-22T20:48:17.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.I-3Qp7CbApALYDddcx4mpNJ_v6PRxek_0 2024-04-22T20:48:17.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16686704429166621432', time=2024-04-22T22:28:17.793185+0000, chain.objs.size()=1 2024-04-22T20:48:17.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nMcmheeQue99RsN9TT31qtCVa-QHiJC_0 2024-04-22T20:48:17.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9909038207250827379', time=2024-04-22T22:28:18.119454+0000, chain.objs.size()=1 2024-04-22T20:48:17.317 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.317+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KfeB2Iuhgi1Vao5GfNNf2iAZe9OTqLn_0 2024-04-22T20:48:17.318 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.318+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11704413036852384473', time=2024-04-22T22:28:18.310397+0000, chain.objs.size()=1 2024-04-22T20:48:17.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.318+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.D5wMl31QtnfBwksNhYbGbVXGtJLl0fI_0 2024-04-22T20:48:17.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.319+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11649284252069996853', time=2024-04-22T22:28:18.432886+0000, chain.objs.size()=1 2024-04-22T20:48:17.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.319+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-ATuyLcemjR5ALCB1CedML00NWQ0jBC_0 2024-04-22T20:48:17.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.320+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14185825401955745946', time=2024-04-22T22:28:18.528015+0000, chain.objs.size()=1 2024-04-22T20:48:17.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.320+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8sdQSA0Xwr-v4aR0XxaD8hJ4WzzfCRg_0 2024-04-22T20:48:17.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8235484917176605685', time=2024-04-22T22:28:19.061043+0000, chain.objs.size()=1 2024-04-22T20:48:17.319 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.01U9poFEX3y12rAgCf9K5O7Q9X5_Jhj_0 2024-04-22T20:48:17.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5de46640 0 -- 172.21.15.136:0/416170015 >> [v2:172.21.15.136:6816/3821859466,v1:172.21.15.136:6817/3821859466] conn(0x7f2e2807fb00 msgr2=0x7f2e280891b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-04-22T20:48:17.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17106827070366744925', time=2024-04-22T22:28:19.084761+0000, chain.objs.size()=1 2024-04-22T20:48:17.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7Ppba7bAvt4naNxFOHB-QL65wK4qO8z_0 2024-04-22T20:48:17.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18235358062086482149', time=2024-04-22T22:28:19.085533+0000, chain.objs.size()=1 2024-04-22T20:48:17.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gWHHL97yZ7orRbMG0Fv45VwkfJN_XMg_0 2024-04-22T20:48:17.320 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17739552174821875854', time=2024-04-22T22:28:19.244888+0000, chain.objs.size()=1 2024-04-22T20:48:17.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.321+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GAHNja41jD89cppWK4fVzSpdc7bl8Yc_0 2024-04-22T20:48:17.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1046831035653219165', time=2024-04-22T22:28:19.945371+0000, chain.objs.size()=1 2024-04-22T20:48:17.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.vQ8gE_WP6XqO2G6abHXs1uGLW7mqOmz_0 2024-04-22T20:48:17.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14660738905856974791', time=2024-04-22T22:28:20.179155+0000, chain.objs.size()=1 2024-04-22T20:48:17.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.W_iimAJi2mc1mHbKgKRVezadUAB3Ftk_0 2024-04-22T20:48:17.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4010115826188631195', time=2024-04-22T22:28:20.196263+0000, chain.objs.size()=1 2024-04-22T20:48:17.321 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LYuwQgQCTddy3zJPI_KIbwTYmfQpXdj_0 2024-04-22T20:48:17.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.322+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17585623656669817765', time=2024-04-22T22:28:20.217284+0000, chain.objs.size()=1 2024-04-22T20:48:17.322 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.322+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.eKGDmKbE1htREugdPkdZXnmX7fha3fd_0 2024-04-22T20:48:17.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.323+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13951202797621901339', time=2024-04-22T22:28:21.256441+0000, chain.objs.size()=1 2024-04-22T20:48:17.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.323+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GhYuRcPzBwasXTigRXFNpvWgk8YTQCM_0 2024-04-22T20:48:17.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.323+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12614108101651839549', time=2024-04-22T22:28:22.351090+0000, chain.objs.size()=1 2024-04-22T20:48:17.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.323+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.HbjWhXh5sCgpML88ZprRaVtgV1OGUWe_0 2024-04-22T20:48:17.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.323+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7325015938311365310', time=2024-04-22T22:28:23.076844+0000, chain.objs.size()=1 2024-04-22T20:48:17.323 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.323+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yq3AMkaadXSLOnRZkidyR6k6XJS6548_0 2024-04-22T20:48:17.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.325+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7834088445567777752', time=2024-04-22T22:28:23.102797+0000, chain.objs.size()=1 2024-04-22T20:48:17.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.325+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PhoyaKPPa0EgRQsVfNh7CUJWd3cWnDR_0 2024-04-22T20:48:17.324 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.327+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16569731702994894544', time=2024-04-22T22:28:23.300825+0000, chain.objs.size()=1 2024-04-22T20:48:17.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.327+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.I962RRXsumWvDXJBogSS0f21bI7vBlM_0 2024-04-22T20:48:17.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16704540292315062740', time=2024-04-22T22:28:23.890596+0000, chain.objs.size()=1 2024-04-22T20:48:17.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OeLwIDw9D7qIc89mehdXNpQeh4CgpN7_0 2024-04-22T20:48:17.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3690974168928415161', time=2024-04-22T22:28:24.129775+0000, chain.objs.size()=1 2024-04-22T20:48:17.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.jQJM3j8FkxZ7oe7jvwCiCSR0TUuTRfE_0 2024-04-22T20:48:17.325 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8256780996265173499', time=2024-04-22T22:28:24.379421+0000, chain.objs.size()=1 2024-04-22T20:48:17.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wSpFQhwVUd2hb3zVUQHkya7ia9Vp_5s_0 2024-04-22T20:48:17.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14323833197863465769', time=2024-04-22T22:28:24.539624+0000, chain.objs.size()=1 2024-04-22T20:48:17.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uA6y4CYpMPNgo_YbEK1ptEpxxgwNsKV_0 2024-04-22T20:48:17.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14249386600672412506', time=2024-04-22T22:28:24.555362+0000, chain.objs.size()=1 2024-04-22T20:48:17.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.e76QyG4QVALvyjqb4AFUp6YW01hMJnf_0 2024-04-22T20:48:17.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15084428364919711710', time=2024-04-22T22:28:24.695452+0000, chain.objs.size()=1 2024-04-22T20:48:17.326 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XrtfjT394AL5pARmrdedItsLn-YWW3w_0 2024-04-22T20:48:17.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17644382685740714972', time=2024-04-22T22:28:26.211099+0000, chain.objs.size()=1 2024-04-22T20:48:17.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RVgQUx7kgbEhLVPmXsTacD-1dRme5jR_0 2024-04-22T20:48:17.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15111584418689752576', time=2024-04-22T22:28:26.527001+0000, chain.objs.size()=1 2024-04-22T20:48:17.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7Pxxly_GOEdTqgKhp254KLOV8uddECC_0 2024-04-22T20:48:17.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11148000288719493114', time=2024-04-22T22:28:27.728518+0000, chain.objs.size()=1 2024-04-22T20:48:17.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.LmC401q8rg975LYtFMjjRLgUDu8ZsJM_0 2024-04-22T20:48:17.327 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.129481826116205938', time=2024-04-22T22:28:28.402798+0000, chain.objs.size()=1 2024-04-22T20:48:17.328 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.328+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.crbKKyF2VvNaDHOpFjR-9OOVTqfjzZk_0 2024-04-22T20:48:17.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.330+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15961677856845503235', time=2024-04-22T22:29:06.183854+0000, chain.objs.size()=1 2024-04-22T20:48:17.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.330+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sKepJJU3aCDwWIOGEY3XcRzJ1FADRem_0 2024-04-22T20:48:17.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8057689549291042681', time=2024-04-22T22:29:06.190198+0000, chain.objs.size()=1 2024-04-22T20:48:17.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SKbSsoJ3ieqCy15DlJQxKo4bxpJ8-_i_0 2024-04-22T20:48:17.329 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15710831919535969967', time=2024-04-22T22:29:07.959164+0000, chain.objs.size()=1 2024-04-22T20:48:17.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TkOPR2ejfwL-Dq14t4PGwkr7UoRJGpl_0 2024-04-22T20:48:17.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17069974457193195368', time=2024-04-22T22:29:08.119727+0000, chain.objs.size()=1 2024-04-22T20:48:17.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iFlvKNWLXNZSKVIg6fAiPoStInMv5z3_0 2024-04-22T20:48:17.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10412063443426257921', time=2024-04-22T22:29:09.859890+0000, chain.objs.size()=1 2024-04-22T20:48:17.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KEWsR3Q0jHFsIA8JYXpVC2MqmJ_-9AR_0 2024-04-22T20:48:17.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.800957745165433008', time=2024-04-22T22:29:09.919368+0000, chain.objs.size()=1 2024-04-22T20:48:17.330 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LH3TBgBhNazT6AtRXeFCnMiM0qeo5DB_0 2024-04-22T20:48:17.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11352647164745838436', time=2024-04-22T22:29:10.183680+0000, chain.objs.size()=1 2024-04-22T20:48:17.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.z02rLZNymf5kRArMg9cQq8Li1UC6pzT_0 2024-04-22T20:48:17.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7391124343150515008', time=2024-04-22T22:29:10.389610+0000, chain.objs.size()=1 2024-04-22T20:48:17.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.331+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MH6fOd1PnuhngVSY96gJ3fxOtkUGrUf_0 2024-04-22T20:48:17.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.332+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2398515649984294737', time=2024-04-22T22:29:10.458576+0000, chain.objs.size()=1 2024-04-22T20:48:17.331 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.332+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.EpMpXokJ_PftcChqQaGH6q3If7ReWq6_0 2024-04-22T20:48:17.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.332+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13816431508237084493', time=2024-04-22T22:29:10.894874+0000, chain.objs.size()=1 2024-04-22T20:48:17.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.332+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.s2JqV0qkyrXx0wmGFxN9CWX1WXfSH-s_0 2024-04-22T20:48:17.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.332+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7345725674613714657', time=2024-04-22T22:29:11.223840+0000, chain.objs.size()=1 2024-04-22T20:48:17.332 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.332+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nSXvnC9mqutbUdBLxIu2iqwCP-YdxUT_0 2024-04-22T20:48:17.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.334+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8365448466299461289', time=2024-04-22T22:29:11.688052+0000, chain.objs.size()=1 2024-04-22T20:48:17.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.334+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.D7RlSaC7Il6_4LxuF-idScR1C6dCPHf_0 2024-04-22T20:48:17.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.334+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15952038272776513087', time=2024-04-22T22:29:12.008368+0000, chain.objs.size()=1 2024-04-22T20:48:17.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.334+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2ipzWMqvsczJovY-JhWmIrkhyf4GyKZ_0 2024-04-22T20:48:17.333 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15134241914280568776', time=2024-04-22T22:29:12.473219+0000, chain.objs.size()=1 2024-04-22T20:48:17.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.O_yrN89hW3hF4IlhvjGNsuruGZ6rJpw_0 2024-04-22T20:48:17.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11856356171508022343', time=2024-04-22T22:29:12.473981+0000, chain.objs.size()=1 2024-04-22T20:48:17.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.5Vt10iejlb3p33WYIWeaKRWxcT-nTUL_0 2024-04-22T20:48:17.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15215031841260089037', time=2024-04-22T22:29:13.185478+0000, chain.objs.size()=1 2024-04-22T20:48:17.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.pFjPN9Pu9gcaENKTFXqXy5vaLyFspb__0 2024-04-22T20:48:17.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9397328442735436150', time=2024-04-22T22:29:13.475191+0000, chain.objs.size()=1 2024-04-22T20:48:17.334 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Q3nkDBVAAqQXdXROp9odMJV2vEZImEP_0 2024-04-22T20:48:17.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3908570747567198349', time=2024-04-22T22:29:14.199732+0000, chain.objs.size()=1 2024-04-22T20:48:17.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.335+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YEZ8QGXn17oWNgXMQ95KHrKDmTQALf__0 2024-04-22T20:48:17.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.336+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5086697622296894272', time=2024-04-22T22:29:14.907607+0000, chain.objs.size()=1 2024-04-22T20:48:17.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.336+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6eJUwzSITK94a2qwzEFxz26bqQYunpO_0 2024-04-22T20:48:17.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.336+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7683656131168437543', time=2024-04-22T22:29:15.302939+0000, chain.objs.size()=1 2024-04-22T20:48:17.335 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.336+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.C29my4g10khntZ5USw2XFQ3cPkmVaKg_0 2024-04-22T20:48:17.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.338+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17824821850024136148', time=2024-04-22T22:29:15.478928+0000, chain.objs.size()=1 2024-04-22T20:48:17.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.338+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BBoLD_DUtoHqfi4cXnegKyAHRozOKSt_0 2024-04-22T20:48:17.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.338+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18434866907986177118', time=2024-04-22T22:29:15.511983+0000, chain.objs.size()=1 2024-04-22T20:48:17.336 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.338+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.47XKoP6xds3ws9MJLJgb_AogaXQdVJ-_0 2024-04-22T20:48:17.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.338+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3253570776266980848', time=2024-04-22T22:29:16.100041+0000, chain.objs.size()=1 2024-04-22T20:48:17.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.338+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2AGrOqL2vbRci-7c1EAd4tQpKX8J9oO_0 2024-04-22T20:48:17.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.338+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13020608707240150877', time=2024-04-22T22:29:16.272912+0000, chain.objs.size()=1 2024-04-22T20:48:17.337 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.338+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BH-F4Aa-0YENdz-13cvEMMTB59-pXzX_0 2024-04-22T20:48:17.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5055506591270451462', time=2024-04-22T22:29:17.202286+0000, chain.objs.size()=1 2024-04-22T20:48:17.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.E-zzkB0Q4JjXxBZx2yFyAjrhT1GQCkL_0 2024-04-22T20:48:17.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18399826916192604836', time=2024-04-22T22:29:17.370915+0000, chain.objs.size()=1 2024-04-22T20:48:17.338 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OMt7ryxed2IhmKkGllUFIJRiDGqC0p1_0 2024-04-22T20:48:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4080787957255752297', time=2024-04-22T22:30:49.318133+0000, chain.objs.size()=1 2024-04-22T20:48:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.444__shadow_.Is_QHow7r5oSwK8nB4TjtaBxA3ShGCG_0 2024-04-22T20:48:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7689772689204051841', time=2024-04-22T22:30:52.751482+0000, chain.objs.size()=1 2024-04-22T20:48:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.447__shadow_.k4RRK6A2spTxYWNJLzAvB6I9iphxGXm_0 2024-04-22T20:48:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12303407388197684965', time=2024-04-22T22:31:10.486413+0000, chain.objs.size()=1 2024-04-22T20:48:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.BMd7LNyP61fu8Oa8CEaMqa2GKpaCdXl_0 2024-04-22T20:48:17.339 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2200946138728806308', time=2024-04-22T22:32:10.177585+0000, chain.objs.size()=1 2024-04-22T20:48:17.340 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.341+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.506__shadow_.f7Uwedcj4-X_eGseVAMNBPN2VnQTdk2_0 2024-04-22T20:48:17.341 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.344+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/42904 2024-04-22T20:48:17.344 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-04-22T20:48:17.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~4pUdQTeLgN3G-hD6BiagOzG8RRk464H', time=2024-04-22T22:33:48.292439+0000, chain.objs.size()=1 2024-04-22T20:48:17.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~4pUdQTeLgN3G-hD6BiagOzG8RRk464H.3 2024-04-22T20:48:17.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~3m4Xy8p5EEo0corrHLt5mzu65Rqt1oL', time=2024-04-22T22:35:52.566937+0000, chain.objs.size()=1 2024-04-22T20:48:17.345 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.524__multipart_mymultipart.2~3m4Xy8p5EEo0corrHLt5mzu65Rqt1oL.3 2024-04-22T20:48:17.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18392492587960026516', time=2024-04-22T22:37:25.625827+0000, chain.objs.size()=1 2024-04-22T20:48:17.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:10DI3Cx9vv5OP0S0JBeaygQX-DIGrVa_.mSLIQ0vuA-qmPVdT-yWeszqyLl-ytcl_0 2024-04-22T20:48:17.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7157334290382341671', time=2024-04-22T22:37:34.580490+0000, chain.objs.size()=1 2024-04-22T20:48:17.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.559__shadow_.evZTebMCWFKPMIkuFzbyryEpmzQpp22_0 2024-04-22T20:48:17.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='ukbvc2e5b5j8h0x2sz8tj38lsk4wgt0n', time=2024-04-22T22:37:43.310377+0000, chain.objs.size()=1 2024-04-22T20:48:17.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.563__shadow_.uXfK6eZrss11FPud7BnQMsQ5t9xXxH__0 2024-04-22T20:48:17.346 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13213531852155235556', time=2024-04-22T22:38:06.287181+0000, chain.objs.size()=1 2024-04-22T20:48:17.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:9o0Ipkc42GZmBnTI0znnRB6WgCoV8g1_.7tPG66zX_VESxBYKqY59yFDX0nUKgIH_0 2024-04-22T20:48:17.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12858284944091671178', time=2024-04-22T22:38:49.549076+0000, chain.objs.size()=1 2024-04-22T20:48:17.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:Q82v70ZD1GlzB3ifTC1ChZvlhumCZeV_.DaoXYhJQYqPzrrEvcf3OUBbe84kXiZ2_0 2024-04-22T20:48:17.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6330915290214491227', time=2024-04-22T22:38:49.775482+0000, chain.objs.size()=1 2024-04-22T20:48:17.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:nj6PUA4a6Ogrv7JM0VzbfSCRa2GEWL2_.FpY1EaeDMDhSLBSa2qEvAxsecz7J-3k_0 2024-04-22T20:48:17.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12942951193141592584', time=2024-04-22T22:38:53.893929+0000, chain.objs.size()=1 2024-04-22T20:48:17.347 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.571__shadow:gvcq32NIJR6cA8tWn95z3ucho7B29vG_.c2cH-CepelylPsBsvIOW-9pdLbTlfbV_0 2024-04-22T20:48:17.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12018469329287076332', time=2024-04-22T22:42:54.698460+0000, chain.objs.size()=1 2024-04-22T20:48:17.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.649__shadow_.-WdDMkS53jN6oqU3BxACUeCdnleT4mk_0 2024-04-22T20:48:17.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6282310601071201880', time=2024-04-22T22:43:03.688215+0000, chain.objs.size()=1 2024-04-22T20:48:17.348 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.346+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.663__shadow_.rfGPmQVYmb2JD_JEoVfBt3Zp4xvGANW_0 2024-04-22T20:48:17.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.350+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.352+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-22T20:48:17.349 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.353+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.350 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.353+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.351 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/52364' 2024-04-22T20:48:17.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_aAT1h6zcl5Y2sytDANL_U53PwgyA4uB', time=2024-04-22T22:22:00.953704+0000, chain.objs.size()=3 2024-04-22T20:48:17.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.fvV4e6OZFw_wtDaVZ40sa6TkF352h5g_0 2024-04-22T20:48:17.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.fvV4e6OZFw_wtDaVZ40sa6TkF352h5g_1 2024-04-22T20:48:17.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.fvV4e6OZFw_wtDaVZ40sa6TkF352h5g_2 2024-04-22T20:48:17.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1413325170247495310', time=2024-04-22T22:25:29.855538+0000, chain.objs.size()=1 2024-04-22T20:48:17.352 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.234__shadow_.SPdGlkjMB8sjGWrDIXvnVC5Z9Ev4vml_0 2024-04-22T20:48:17.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17249383768306809093', time=2024-04-22T22:25:36.820794+0000, chain.objs.size()=1 2024-04-22T20:48:17.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.241__shadow_.SNHtEX4jUwh_RM36V0eosaHpQM9jH1__0 2024-04-22T20:48:17.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13354834618138207635', time=2024-04-22T22:25:45.663239+0000, chain.objs.size()=1 2024-04-22T20:48:17.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.250__shadow_.XiZimsf75QMcBGhVuvgY2UaDFMAzo9E_0 2024-04-22T20:48:17.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8876041343746685883', time=2024-04-22T22:26:22.322911+0000, chain.objs.size()=1 2024-04-22T20:48:17.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eMq1UFAahR6Z8XruFY-XrH3TA3YvTBT_0 2024-04-22T20:48:17.353 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6725763539016614626', time=2024-04-22T22:26:22.464007+0000, chain.objs.size()=1 2024-04-22T20:48:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.D8iTAJECV_wVZUEcT8ihgKJJZco9_KU_0 2024-04-22T20:48:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5402296578124584070', time=2024-04-22T22:26:22.464658+0000, chain.objs.size()=1 2024-04-22T20:48:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.flA1xc5W_D-EFhEqV4Mk7zWLKIN9Gm3_0 2024-04-22T20:48:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11639614261542075872', time=2024-04-22T22:26:22.662297+0000, chain.objs.size()=1 2024-04-22T20:48:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ebC8eRGz_GobOmJHcxuy5yuHdM5TGIs_0 2024-04-22T20:48:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18207280120731925213', time=2024-04-22T22:26:23.262525+0000, chain.objs.size()=1 2024-04-22T20:48:17.354 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SLWa4Z2VB70GxalksmIfdOewn4cPlIz_0 2024-04-22T20:48:17.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9668440678978761989', time=2024-04-22T22:26:23.313724+0000, chain.objs.size()=1 2024-04-22T20:48:17.355 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.354+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8y30S7v6LJ1pGfgVXOKFiduyYw49MC3_0 2024-04-22T20:48:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17232939908509585056', time=2024-04-22T22:26:23.423190+0000, chain.objs.size()=1 2024-04-22T20:48:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xGqgCqsrowOeUh7r-O80z2tBOypI28-_0 2024-04-22T20:48:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17800488337704658456', time=2024-04-22T22:26:24.394622+0000, chain.objs.size()=1 2024-04-22T20:48:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DCMUil-KE8YujZJbZi17g2rA01yfzGC_0 2024-04-22T20:48:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6264839381352196325', time=2024-04-22T22:26:24.536676+0000, chain.objs.size()=1 2024-04-22T20:48:17.356 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.30ONVCH-PfpNLKB3QIrtaknTf-kNZmA_0 2024-04-22T20:48:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17081283776552005177', time=2024-04-22T22:26:24.862818+0000, chain.objs.size()=1 2024-04-22T20:48:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tf5dQbayS6C4vMtWiyWTfLuNI4_lh9i_0 2024-04-22T20:48:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2619401537887136073', time=2024-04-22T22:26:24.872486+0000, chain.objs.size()=1 2024-04-22T20:48:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.KVJ6H8oDZTG5_L6LeLr-ZVqypLor-hC_0 2024-04-22T20:48:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2909971899923692057', time=2024-04-22T22:26:25.025477+0000, chain.objs.size()=1 2024-04-22T20:48:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6VxxSslPx_TJZO42lUe90p6biqg9niN_0 2024-04-22T20:48:17.357 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1352546275265404063', time=2024-04-22T22:26:25.100419+0000, chain.objs.size()=1 2024-04-22T20:48:17.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GsAoAIxrYRJHPa26EwFTwCpQXHkQtes_0 2024-04-22T20:48:17.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1158338439744238976', time=2024-04-22T22:26:25.101284+0000, chain.objs.size()=1 2024-04-22T20:48:17.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Haph47sXz61Nv0lnqQvx5kpqBpc2ocR_0 2024-04-22T20:48:17.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17091856915575489350', time=2024-04-22T22:26:25.815093+0000, chain.objs.size()=1 2024-04-22T20:48:17.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XAfhyAxBBwwT1JhCkIOJmG0GAZFLLDj_0 2024-04-22T20:48:17.358 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6714606881170007681', time=2024-04-22T22:26:26.067238+0000, chain.objs.size()=1 2024-04-22T20:48:17.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Mh1dcpYAVZT4PbT-Snaen-3muAumSbs_0 2024-04-22T20:48:17.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13993643380922029709', time=2024-04-22T22:26:26.299454+0000, chain.objs.size()=1 2024-04-22T20:48:17.359 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.358+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bbbDUqseEyC6Selym9uEDaKcgyUpOBn_0 2024-04-22T20:48:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.361+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.835326286582333980', time=2024-04-22T22:26:27.982854+0000, chain.objs.size()=1 2024-04-22T20:48:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.361+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wWpPyz0k_gemijk-S6qviO1N1zhNKLB_0 2024-04-22T20:48:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.361+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16226959494066035803', time=2024-04-22T22:26:28.025049+0000, chain.objs.size()=1 2024-04-22T20:48:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.361+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZBlUSUJMjhFwgj65KYw23QbkisfQSAf_0 2024-04-22T20:48:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.361+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.99354825729296880', time=2024-04-22T22:26:28.161768+0000, chain.objs.size()=1 2024-04-22T20:48:17.360 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xU4kFoX1aKugXTMPO89aFzKrHrYvSPR_0 2024-04-22T20:48:17.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9115898583515059048', time=2024-04-22T22:26:28.189360+0000, chain.objs.size()=1 2024-04-22T20:48:17.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hgWypxE4s8Ttdv452KLpqeJUrrHg7O3_0 2024-04-22T20:48:17.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11722623838547450022', time=2024-04-22T22:26:28.342266+0000, chain.objs.size()=1 2024-04-22T20:48:17.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7Q2s88FNfou2k-gbNakxVtMTyVuhy_v_0 2024-04-22T20:48:17.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9866609873293282921', time=2024-04-22T22:26:28.631554+0000, chain.objs.size()=1 2024-04-22T20:48:17.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.I6qFuTzE6CIvDD9uZ5OfI0VtngDkzYx_0 2024-04-22T20:48:17.361 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5743339220253502117', time=2024-04-22T22:26:29.439404+0000, chain.objs.size()=1 2024-04-22T20:48:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8CH47fbI6813qteSm88k7c77l7U3sMg_0 2024-04-22T20:48:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1581889312012741993', time=2024-04-22T22:26:29.573741+0000, chain.objs.size()=1 2024-04-22T20:48:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Fv8Ib5qbYrs3BMLCi92jMfxI4aN_ZOk_0 2024-04-22T20:48:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9596400682465515336', time=2024-04-22T22:26:29.735990+0000, chain.objs.size()=1 2024-04-22T20:48:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.X9Bnag6uGTq-fKGtQsBMWsp5X0MoSYq_0 2024-04-22T20:48:17.362 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17272422125250498908', time=2024-04-22T22:26:30.518766+0000, chain.objs.size()=1 2024-04-22T20:48:17.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SUm-hhCY_w313xqRD3N9lMONzfyadHA_0 2024-04-22T20:48:17.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13775499443072734556', time=2024-04-22T22:26:30.593883+0000, chain.objs.size()=1 2024-04-22T20:48:17.363 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.362+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.i7NeZddOgxDtcU2CdLQKhnaG6Qf6loA_0 2024-04-22T20:48:17.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8497575450328795795', time=2024-04-22T22:26:30.999652+0000, chain.objs.size()=1 2024-04-22T20:48:17.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.NbRF_FE-XjOWmK5DTfaB1T_fdXqRAYu_0 2024-04-22T20:48:17.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2701669119125941057', time=2024-04-22T22:26:31.687543+0000, chain.objs.size()=1 2024-04-22T20:48:17.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dXHcUCvI0zDloQS6dErDrc-kxWNCoEq_0 2024-04-22T20:48:17.364 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13439599407393932205', time=2024-04-22T22:26:32.966623+0000, chain.objs.size()=1 2024-04-22T20:48:17.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7ReniykzRMfDCHWC_BWWpeRc9-zt-4d_0 2024-04-22T20:48:17.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17209143685770978405', time=2024-04-22T22:26:34.440464+0000, chain.objs.size()=1 2024-04-22T20:48:17.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.T8mSdTyZZG4Cm9a62vJRnosxR1lw8du_0 2024-04-22T20:48:17.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15754441458200265216', time=2024-04-22T22:26:40.203682+0000, chain.objs.size()=1 2024-04-22T20:48:17.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.257__shadow_.E9BXbIQ7ceJqET_a2ichT2Xl69bO23G_0 2024-04-22T20:48:17.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3842289578464088448', time=2024-04-22T22:26:42.104781+0000, chain.objs.size()=1 2024-04-22T20:48:17.365 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.259__shadow_.q_4ZdZeyvKgO2XckmXR1OTfxMD3fquc_0 2024-04-22T20:48:17.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12548445556561774417', time=2024-04-22T22:26:47.889233+0000, chain.objs.size()=1 2024-04-22T20:48:17.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.265__shadow_.0uGjkWJVLDzIMtPN1_pZ2-TdLsU_1Ut_0 2024-04-22T20:48:17.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.624599768053161430', time=2024-04-22T22:26:49.866227+0000, chain.objs.size()=1 2024-04-22T20:48:17.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.267__shadow_.2Cbq9MLKDpDYA6Bw0l3mZsYXprojNJA_0 2024-04-22T20:48:17.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16761892133059039513', time=2024-04-22T22:26:54.848159+0000, chain.objs.size()=1 2024-04-22T20:48:17.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.272__shadow_.GUcAgjPVjtsOUoBDfinRxAW1hVXZ8Or_0 2024-04-22T20:48:17.366 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4091085844703640489', time=2024-04-22T22:27:00.958664+0000, chain.objs.size()=1 2024-04-22T20:48:17.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.278__shadow_.KRuZIX5a3xUMRCalk0Dc0g-9N4qGJGD_0 2024-04-22T20:48:17.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11413988011204002976', time=2024-04-22T22:27:05.709527+0000, chain.objs.size()=1 2024-04-22T20:48:17.367 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.366+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.C1uRWobXERxdEBMgVyvM_0G8uL36zH8_0 2024-04-22T20:48:17.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.370+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4953385076282389838', time=2024-04-22T22:27:07.893493+0000, chain.objs.size()=1 2024-04-22T20:48:17.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.370+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.KR6tF8cBDAOyLyDV4YAq1pb8cHJIOWD_0 2024-04-22T20:48:17.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.371+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12452299762234601639', time=2024-04-22T22:27:08.964360+0000, chain.objs.size()=1 2024-04-22T20:48:17.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.371+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.283__shadow_.JXMoHsa2cT1wrIXphbzxyokNnoyZAfR_0 2024-04-22T20:48:17.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14832575864082783776', time=2024-04-22T22:27:11.986979+0000, chain.objs.size()=1 2024-04-22T20:48:17.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.286__shadow_.JFZ5u_EhOkFJjCcEU8PQ8-3J9rK17CZ_0 2024-04-22T20:48:17.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9151017515277799963', time=2024-04-22T22:27:11.987801+0000, chain.objs.size()=1 2024-04-22T20:48:17.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.286__shadow_.t_GWatebLkTG5uDvuebbqoTkAyZNju3_0 2024-04-22T20:48:17.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6712902959820476713', time=2024-04-22T22:27:13.047855+0000, chain.objs.size()=1 2024-04-22T20:48:17.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.287__shadow_.XqIBjWfqlBee7MDkDyDDO9uQUStt2Ce_0 2024-04-22T20:48:17.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1560485621676474349', time=2024-04-22T22:27:16.067262+0000, chain.objs.size()=1 2024-04-22T20:48:17.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.290__shadow_.30PeojPVODazGY6CKIvoqZFkT-xe-oV_0 2024-04-22T20:48:17.370 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3738483914163166627', time=2024-04-22T22:27:21.479828+0000, chain.objs.size()=1 2024-04-22T20:48:17.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.295__shadow_.2M2hdtI-E3GpgjaZHz_8NxwNBbPWBGq_0 2024-04-22T20:48:17.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3918874624393866764', time=2024-04-22T22:27:39.279644+0000, chain.objs.size()=1 2024-04-22T20:48:17.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.311__shadow_.eDcFbwzZelJt6O6z4zXtUrXWyOqpV1K_0 2024-04-22T20:48:17.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3580819078085240275', time=2024-04-22T22:28:17.332247+0000, chain.objs.size()=1 2024-04-22T20:48:17.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0hMsnp9UTTu-kwPX2g7jmBX_4zdZ-eq_0 2024-04-22T20:48:17.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18051499235261505725', time=2024-04-22T22:28:17.351544+0000, chain.objs.size()=1 2024-04-22T20:48:17.371 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Oos6d8E1sAH5rAjWtHXwbArLC0X1oUP_0 2024-04-22T20:48:17.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17096899710499533706', time=2024-04-22T22:28:18.213902+0000, chain.objs.size()=1 2024-04-22T20:48:17.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.372+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MLbK91ZY0S3lhZhHhhR9MPPp8H3cS2p_0 2024-04-22T20:48:17.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.374+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5927234739271218499', time=2024-04-22T22:28:18.362223+0000, chain.objs.size()=1 2024-04-22T20:48:17.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.374+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Qp94fS9Chu6UlXgv9MI_1p1mZT3m_5g_0 2024-04-22T20:48:17.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.374+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17266929998069137485', time=2024-04-22T22:28:19.032528+0000, chain.objs.size()=1 2024-04-22T20:48:17.373 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.374+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ZkrKqQirXGO5t2qHkKf2EJaZtPCU_xh_0 2024-04-22T20:48:17.373 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.376+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5934520091608448895', time=2024-04-22T22:28:19.228242+0000, chain.objs.size()=1 2024-04-22T20:48:17.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.376+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lzN7hWF2KHPCCv_XgSguErXKXWj2cGM_0 2024-04-22T20:48:17.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.376+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12395398924503294367', time=2024-04-22T22:28:19.419787+0000, chain.objs.size()=1 2024-04-22T20:48:17.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.376+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.s57DD-7b_zpaD4a4YYQPyxCzI6Jcc36_0 2024-04-22T20:48:17.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.376+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17039258665869534617', time=2024-04-22T22:28:20.676660+0000, chain.objs.size()=1 2024-04-22T20:48:17.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.376+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.M1cdjd5ddRmdTE9Kemz2vrifSHexcN1_0 2024-04-22T20:48:17.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.376+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17714740003978911733', time=2024-04-22T22:28:20.827638+0000, chain.objs.size()=1 2024-04-22T20:48:17.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.376+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gSOMjSK5Rj8YPGmA6trlGIxDigPUEpN_0 2024-04-22T20:48:17.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7065100644875661049', time=2024-04-22T22:28:22.323662+0000, chain.objs.size()=1 2024-04-22T20:48:17.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9IqpquaaENpZAv7K9txY2hP7yr6J-wG_0 2024-04-22T20:48:17.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15861488544652643738', time=2024-04-22T22:28:22.360927+0000, chain.objs.size()=1 2024-04-22T20:48:17.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.jhc6E4HCNhuR9gaaXve6OydMe-eLUQi_0 2024-04-22T20:48:17.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14918098263580871275', time=2024-04-22T22:28:22.773082+0000, chain.objs.size()=1 2024-04-22T20:48:17.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qLnQiHe5HrWIvkvzAaGNkwbPO0gjFrw_0 2024-04-22T20:48:17.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8375802006221605782', time=2024-04-22T22:28:23.383524+0000, chain.objs.size()=1 2024-04-22T20:48:17.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GeDkBpprQsufa_XULtOmnb50FeCHw1L_0 2024-04-22T20:48:17.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15996235094832225217', time=2024-04-22T22:28:24.030945+0000, chain.objs.size()=1 2024-04-22T20:48:17.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.F7F5-RkwCMELy0wN-9LiGB5uc7qSMCB_0 2024-04-22T20:48:17.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12615868961828381359', time=2024-04-22T22:28:24.108820+0000, chain.objs.size()=1 2024-04-22T20:48:17.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3JPpUBOI7NQpq65B_m4nZfuPTCTWi6C_0 2024-04-22T20:48:17.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4686332180688122034', time=2024-04-22T22:28:25.371334+0000, chain.objs.size()=1 2024-04-22T20:48:17.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.377+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8Xx1TClGj-QPgfOuKqljKqEOs0PBljw_0 2024-04-22T20:48:17.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17435926854733179115', time=2024-04-22T22:28:26.133561+0000, chain.objs.size()=1 2024-04-22T20:48:17.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ru41Wr5Isv9GvcDYTLfIguk-MVsg8Tr_0 2024-04-22T20:48:17.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6092596032288145267', time=2024-04-22T22:28:26.870657+0000, chain.objs.size()=1 2024-04-22T20:48:17.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wIZFwcmVLmTEpPTaUi4afwslO20Cbgc_0 2024-04-22T20:48:17.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13736715728771715261', time=2024-04-22T22:28:26.942977+0000, chain.objs.size()=1 2024-04-22T20:48:17.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PAH54Dd3UFvQGkPIFFNKoczs7FoW52-_0 2024-04-22T20:48:17.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3480368833849805401', time=2024-04-22T22:28:27.391169+0000, chain.objs.size()=1 2024-04-22T20:48:17.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dfBcWmAtg1egEM22W_LTWbdtuNMNU88_0 2024-04-22T20:48:17.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13208540728239590833', time=2024-04-22T22:28:27.598652+0000, chain.objs.size()=1 2024-04-22T20:48:17.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TSSWvTHl5kju8QWCHVvSw8o-ziOlo11_0 2024-04-22T20:48:17.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11174925516255826016', time=2024-04-22T22:28:27.830196+0000, chain.objs.size()=1 2024-04-22T20:48:17.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.381+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0a_OchD7-19Txtav5eEteFxWLHEU-jS_0 2024-04-22T20:48:17.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3097836942531486590', time=2024-04-22T22:28:27.830853+0000, chain.objs.size()=1 2024-04-22T20:48:17.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lQ_1svT_1mEbAYtH5rZ9sQCt2ZVbpwQ_0 2024-04-22T20:48:17.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9181100661557958308', time=2024-04-22T22:28:28.513219+0000, chain.objs.size()=1 2024-04-22T20:48:17.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.AEn5V9dAMx-tNp2zKEhp6s44tl3VNTk_0 2024-04-22T20:48:17.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15808667012882072430', time=2024-04-22T22:28:28.671217+0000, chain.objs.size()=1 2024-04-22T20:48:17.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tQe_18MgDmcbdxx_d5Gn_IRooqbdvVm_0 2024-04-22T20:48:17.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5701413942572090621', time=2024-04-22T22:28:29.141108+0000, chain.objs.size()=1 2024-04-22T20:48:17.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TypuBXSCeK-xEjdTHrtg-MVxiEWpzi1_0 2024-04-22T20:48:17.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14225042557010730108', time=2024-04-22T22:28:29.143319+0000, chain.objs.size()=1 2024-04-22T20:48:17.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.382+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FB28E8w48GZ8T9rLYEdOzLH9vSbCxHB_0 2024-04-22T20:48:17.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.385+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17709558231415534223', time=2024-04-22T22:28:29.301294+0000, chain.objs.size()=1 2024-04-22T20:48:17.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.385+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NKqd0VDixEVXZJ4AMrQBmdtAOuiI4tC_0 2024-04-22T20:48:17.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.385+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9660917903286415207', time=2024-04-22T22:29:06.326203+0000, chain.objs.size()=1 2024-04-22T20:48:17.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.385+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8tm1yqveXke1gBHhu6Ru2PAhWVyWEkt_0 2024-04-22T20:48:17.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7918456312314757526', time=2024-04-22T22:29:06.613321+0000, chain.objs.size()=1 2024-04-22T20:48:17.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YAW2RlrjkRAnmjYRUJ3IC-9pCH8V5kw_0 2024-04-22T20:48:17.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6699527661273245595', time=2024-04-22T22:29:06.740543+0000, chain.objs.size()=1 2024-04-22T20:48:17.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yvG4jt033MVSSkxWWZv7a0Qdrc27OnI_0 2024-04-22T20:48:17.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2302068509803538381', time=2024-04-22T22:29:08.012362+0000, chain.objs.size()=1 2024-04-22T20:48:17.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hZ_Vs37D-_4t2kGDPP74y54OYQsXLjb_0 2024-04-22T20:48:17.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11486252107979521991', time=2024-04-22T22:29:08.464414+0000, chain.objs.size()=1 2024-04-22T20:48:17.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NFm-6ee5LFlWkqGevJ5RLMqQT3PxLOw_0 2024-04-22T20:48:17.384 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16692434636838221053', time=2024-04-22T22:29:08.631793+0000, chain.objs.size()=1 2024-04-22T20:48:17.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.C0xZhMFkO4lmO-P57lLJLQ91-EZ4KBZ_0 2024-04-22T20:48:17.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12440115476028673357', time=2024-04-22T22:29:09.571544+0000, chain.objs.size()=1 2024-04-22T20:48:17.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.plT2rlZhRoOC4SWV-d44e0EM9iSdb3B_0 2024-04-22T20:48:17.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13734194634107256956', time=2024-04-22T22:29:09.572165+0000, chain.objs.size()=1 2024-04-22T20:48:17.385 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0527VSWw2DcBl4IQ-nBpiXigS14jIpl_0 2024-04-22T20:48:17.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4551671932109228459', time=2024-04-22T22:29:09.893577+0000, chain.objs.size()=1 2024-04-22T20:48:17.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-LZfZmNtz1j7g9ygh2LEQfJYSNkQw9L_0 2024-04-22T20:48:17.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8465218013021240623', time=2024-04-22T22:29:09.894099+0000, chain.objs.size()=1 2024-04-22T20:48:17.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.386+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bN0xZdA3Ng5_adNybQClnSL8gmf718u_0 2024-04-22T20:48:17.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.389+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6060978441099189002', time=2024-04-22T22:29:09.896497+0000, chain.objs.size()=1 2024-04-22T20:48:17.386 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.389+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KEJc0-EO0SHr0sTZ_UrzjiB0UVqn1Ig_0 2024-04-22T20:48:17.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.389+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1966564252015756183', time=2024-04-22T22:29:10.084194+0000, chain.objs.size()=1 2024-04-22T20:48:17.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.389+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.C4L8fwHkP9F2vAPG5BJK_lB1lTIMRu-_0 2024-04-22T20:48:17.387 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.390+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13656437424447668591', time=2024-04-22T22:29:10.375570+0000, chain.objs.size()=1 2024-04-22T20:48:17.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.390+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Ck2u-VZicqPfCdURpjivWAgZCOKfqEO_0 2024-04-22T20:48:17.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11652934558307721560', time=2024-04-22T22:29:10.468801+0000, chain.objs.size()=1 2024-04-22T20:48:17.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BqSccDKGom4z6badTVuL-Vxr83Vop9n_0 2024-04-22T20:48:17.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12081245075418023624', time=2024-04-22T22:29:10.939665+0000, chain.objs.size()=1 2024-04-22T20:48:17.388 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.eL4-zkga4kDbc_bgiR5pHmRl5USqY3p_0 2024-04-22T20:48:17.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3240815179777988293', time=2024-04-22T22:29:11.576055+0000, chain.objs.size()=1 2024-04-22T20:48:17.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.9rLvt6o-mGMjziAhKMwvv14uSfUoSrI_0 2024-04-22T20:48:17.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5784032814082514604', time=2024-04-22T22:29:11.957226+0000, chain.objs.size()=1 2024-04-22T20:48:17.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.yUw-2OLyhKtMQ2Ax3cMo4VXnoFx4apB_0 2024-04-22T20:48:17.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13418954093957689211', time=2024-04-22T22:29:13.130902+0000, chain.objs.size()=1 2024-04-22T20:48:17.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.L15BSyTiHQ-u5iWJa-QnxHFYuFkUoPa_0 2024-04-22T20:48:17.389 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5346463383128538884', time=2024-04-22T22:29:13.395354+0000, chain.objs.size()=1 2024-04-22T20:48:17.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-NLXP8DZpSNAlxmtkB5W5IEZIats89G_0 2024-04-22T20:48:17.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6425221874958578137', time=2024-04-22T22:29:13.510756+0000, chain.objs.size()=1 2024-04-22T20:48:17.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.m92v4JOSr7LSGX7hNcAPoHvkVkHnJyH_0 2024-04-22T20:48:17.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7060184000570727272', time=2024-04-22T22:29:14.223200+0000, chain.objs.size()=1 2024-04-22T20:48:17.390 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.391+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._cAaG9yP8VWo31NTIustL912qpBtBlX_0 2024-04-22T20:48:17.392 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.395+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/52364 2024-04-22T20:48:17.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=22, truncated=0, next_marker='' 2024-04-22T20:48:17.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6905578710535858533', time=2024-04-22T22:29:14.711194+0000, chain.objs.size()=1 2024-04-22T20:48:17.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mttAqPgt5Q5pA0FOgJPiMmprtDU1_Dt_0 2024-04-22T20:48:17.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11238720489724322509', time=2024-04-22T22:29:14.973288+0000, chain.objs.size()=1 2024-04-22T20:48:17.395 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IB78Y5iJ8LDSOE1SZSBAh1DkD2M9-Gs_0 2024-04-22T20:48:17.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9628703008468945169', time=2024-04-22T22:29:14.996577+0000, chain.objs.size()=1 2024-04-22T20:48:17.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LiOZGjQ9rwSLoUHdYRKANqMuqfb6hX0_0 2024-04-22T20:48:17.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9944298474608398842', time=2024-04-22T22:29:14.997399+0000, chain.objs.size()=1 2024-04-22T20:48:17.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.AyFLJRWVfGtTZQXa7Bb1rULieGlct28_0 2024-04-22T20:48:17.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15929173723781986827', time=2024-04-22T22:29:15.644394+0000, chain.objs.size()=1 2024-04-22T20:48:17.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Kg29FE_CIm29V5gp2A1zVYXqZiYrd3a_0 2024-04-22T20:48:17.396 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17581493061955745996', time=2024-04-22T22:29:15.866036+0000, chain.objs.size()=1 2024-04-22T20:48:17.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BXp5V8xmqiHT6q6AZCz6FHNFaCPHnrM_0 2024-04-22T20:48:17.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8763188861336830498', time=2024-04-22T22:29:16.352066+0000, chain.objs.size()=1 2024-04-22T20:48:17.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.knLlTuQnMF94dyWGD09fPbb_fZWJlD__0 2024-04-22T20:48:17.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12007061126122390879', time=2024-04-22T22:29:16.926062+0000, chain.objs.size()=1 2024-04-22T20:48:17.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.aiF6JpIZbgyI4ddURPS8zzuHLxfD4aE_0 2024-04-22T20:48:17.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13943435967093056113', time=2024-04-22T22:29:16.978375+0000, chain.objs.size()=1 2024-04-22T20:48:17.397 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xbH_oQonY5UaBdzGwgVKKt1kuEk5OIM_0 2024-04-22T20:48:17.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5380508678482236584', time=2024-04-22T22:31:08.209271+0000, chain.objs.size()=1 2024-04-22T20:48:17.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.470__shadow_.6rg-Ik7DcS2lRMsgFiDkquNVMM1_Tpl_0 2024-04-22T20:48:17.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2195594949854674822', time=2024-04-22T22:35:01.650983+0000, chain.objs.size()=3 2024-04-22T20:48:17.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~V9TF1iKexy4PgfGLr3PC8j0_H8utMgq.1 2024-04-22T20:48:17.398 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.398+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~V9TF1iKexy4PgfGLr3PC8j0_H8utMgq.2 2024-04-22T20:48:17.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~V9TF1iKexy4PgfGLr3PC8j0_H8utMgq.3 2024-04-22T20:48:17.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.726379162596538365', time=2024-04-22T22:35:07.262476+0000, chain.objs.size()=1 2024-04-22T20:48:17.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~3RP0PNU2LO_wI_xBxW6AeCPzJdSkBO_.1 2024-04-22T20:48:17.399 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16126864839398143640', time=2024-04-22T22:36:33.721025+0000, chain.objs.size()=1 2024-04-22T20:48:17.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.539__shadow_.d4fkRxU70CLnLvasRY5AL9AXq4KeLS2_0 2024-04-22T20:48:17.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6605166787717706845', time=2024-04-22T22:37:29.698365+0000, chain.objs.size()=1 2024-04-22T20:48:17.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:QzsXOx16tV0fAzwu2iS7WWZBSjY21lg_.NFrF3NFBfXqy5zk-P7-SGa986BrdgHa_0 2024-04-22T20:48:17.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14860937099402116372', time=2024-04-22T22:37:33.584726+0000, chain.objs.size()=1 2024-04-22T20:48:17.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.558__shadow:rqaKpyauCAGt8Shh30uHPzzXP0WUFk3_.W_lu4NUHnUiMAlGgAP7B6VlHNh2Ilg7_0 2024-04-22T20:48:17.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8291550220464129276', time=2024-04-22T22:37:38.563522+0000, chain.objs.size()=1 2024-04-22T20:48:17.400 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow_.LwMHZCIfw1OxEegX_r4PT7SPAQgwpU__0 2024-04-22T20:48:17.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17708762539547638245', time=2024-04-22T22:37:39.640553+0000, chain.objs.size()=1 2024-04-22T20:48:17.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow:pBB3BMA.OmfMMdNFib6J-SsLakyKcFd_.2eVe2rzzTBZysNKs-H-7kWIGFWMa2IM_0 2024-04-22T20:48:17.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14810821929307617744', time=2024-04-22T22:37:43.402636+0000, chain.objs.size()=1 2024-04-22T20:48:17.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.563__shadow:mPlRF9aMrZNRVK2APiAo0EZwfHCUd2C_.7vyNFStrGTzsJ2InMilDjLAXWWLK6n2_0 2024-04-22T20:48:17.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:-dVByXoDlV-EFnizGQqWv8SzVp2FAWe_.kWtQjnq6VIeWmMQahTWNPn-ZtLhy_ww_1_1dd173e9012639b2aec9c109f271b8b0', time=2024-04-22T22:38:51.933730+0000, chain.objs.size()=1 2024-04-22T20:48:17.401 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:-dVByXoDlV-EFnizGQqWv8SzVp2FAWe_.kWtQjnq6VIeWmMQahTWNPn-ZtLhy_ww_0 2024-04-22T20:48:17.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9755519301187707669', time=2024-04-22T22:41:30.367123+0000, chain.objs.size()=12 2024-04-22T20:48:17.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__multipart_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.1 2024-04-22T20:48:17.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.401+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__shadow_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.1_1 2024-04-22T20:48:17.402 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.405+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__multipart_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.2 2024-04-22T20:48:17.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.406+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__shadow_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.2_1 2024-04-22T20:48:17.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.406+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__multipart_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.3 2024-04-22T20:48:17.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.406+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__shadow_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.3_1 2024-04-22T20:48:17.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.406+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__multipart_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.4 2024-04-22T20:48:17.403 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.406+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__shadow_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.4_1 2024-04-22T20:48:17.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.407+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__multipart_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.5 2024-04-22T20:48:17.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.407+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__shadow_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.5_1 2024-04-22T20:48:17.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.407+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__multipart_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.6 2024-04-22T20:48:17.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.407+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.609__shadow_multipart_enc.2~FKemysgUA26lUwmKU_ZlzChn72vc9BJ.6_1 2024-04-22T20:48:17.404 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.407+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15448053237043569213', time=2024-04-22T22:41:54.935781+0000, chain.objs.size()=1 2024-04-22T20:48:17.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.407+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.621__shadow_.s-tptxtoIsnQlSHkomn8954gGPA1ZqD_0 2024-04-22T20:48:17.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.407+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4601231515918996486', time=2024-04-22T22:42:21.018327+0000, chain.objs.size()=1 2024-04-22T20:48:17.405 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.407+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.637__shadow_.VqqyxxuuWPtLEXhNX-JhH0E-INoHDDx_0 2024-04-22T20:48:17.410 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.413+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.412 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.416+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-22T20:48:17.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.417+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.414 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/48433' 2024-04-22T20:48:17.416 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4750971922852366914', time=2024-04-22T22:21:39.847239+0000, chain.objs.size()=3 2024-04-22T20:48:17.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-STANDARD.2~9OUPU6-lbgqVBh3WMkwxjWuBIwixOD4.1 2024-04-22T20:48:17.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-STANDARD.2~9OUPU6-lbgqVBh3WMkwxjWuBIwixOD4.2 2024-04-22T20:48:17.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.148__multipart_mymultipart-STANDARD.2~9OUPU6-lbgqVBh3WMkwxjWuBIwixOD4.3 2024-04-22T20:48:17.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4837656740723225554', time=2024-04-22T22:21:59.395452+0000, chain.objs.size()=5 2024-04-22T20:48:17.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-STANDARD.2~aZCUCatKKAA_yPT6XOtf3m7SatYSceU.1 2024-04-22T20:48:17.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.418+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_foo-STANDARD.2~aZCUCatKKAA_yPT6XOtf3m7SatYSceU.1_1 2024-04-22T20:48:17.417 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-STANDARD.2~aZCUCatKKAA_yPT6XOtf3m7SatYSceU.2 2024-04-22T20:48:17.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_foo-STANDARD.2~aZCUCatKKAA_yPT6XOtf3m7SatYSceU.2_1 2024-04-22T20:48:17.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__multipart_foo-STANDARD.2~aZCUCatKKAA_yPT6XOtf3m7SatYSceU.3 2024-04-22T20:48:17.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17853090397092080', time=2024-04-22T22:23:16.904156+0000, chain.objs.size()=1 2024-04-22T20:48:17.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.186__shadow_.NkFuGmp29PjNpPpNr7xK_lrZDaC5L2N_0 2024-04-22T20:48:17.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~QaXaarLP6dPtEoflggSCxrHfaHltEPK', time=2024-04-22T22:23:21.760449+0000, chain.objs.size()=1 2024-04-22T20:48:17.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.190__multipart_mymultipart.2~QaXaarLP6dPtEoflggSCxrHfaHltEPK.1 2024-04-22T20:48:17.418 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13086565932342410281', time=2024-04-22T22:25:17.294061+0000, chain.objs.size()=1 2024-04-22T20:48:17.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.224__shadow_.Y_J9k6aDa8FuSm3Nfu-EXAp50pp4WVV_0 2024-04-22T20:48:17.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13420501892024287079', time=2024-04-22T22:25:30.839466+0000, chain.objs.size()=1 2024-04-22T20:48:17.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.419+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.235__shadow_.fqoQA-67WQ8d2Q7Kpv47N6459UMKqn3_0 2024-04-22T20:48:17.419 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10521451603875103858', time=2024-04-22T22:25:33.800231+0000, chain.objs.size()=1 2024-04-22T20:48:17.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.238__shadow_.RGcJnm-ZxyVkhtCvkYwyfd9us9KqHGd_0 2024-04-22T20:48:17.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7475162209945350019', time=2024-04-22T22:25:39.797453+0000, chain.objs.size()=1 2024-04-22T20:48:17.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.244__shadow_.L8xOnu2UJw7peIbPBekbK0Mc4FaTUaL_0 2024-04-22T20:48:17.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4647791325065832056', time=2024-04-22T22:25:43.664084+0000, chain.objs.size()=1 2024-04-22T20:48:17.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.248__shadow_.OblKMVTaR3zN5ckXHVXjWOtgYzh2U0h_0 2024-04-22T20:48:17.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5991074715759224866', time=2024-04-22T22:25:45.672867+0000, chain.objs.size()=1 2024-04-22T20:48:17.420 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.250__shadow_.Q-kq8U6IGQ9KMUbQ-BAgkZccP_0Sky-_0 2024-04-22T20:48:17.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.786144667147311665', time=2024-04-22T22:26:22.345615+0000, chain.objs.size()=1 2024-04-22T20:48:17.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._mvGoVf1EtwV0j8ZejTxIT3xRPHJ_qn_0 2024-04-22T20:48:17.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10311638062951084687', time=2024-04-22T22:26:22.473232+0000, chain.objs.size()=1 2024-04-22T20:48:17.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.kBRcftaStRFcA1__5iEBN_Uqz3wdnSO_0 2024-04-22T20:48:17.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14979555216798544519', time=2024-04-22T22:26:22.553713+0000, chain.objs.size()=1 2024-04-22T20:48:17.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cprL2vWdDeaPLlUphlh020OuWkBDbcG_0 2024-04-22T20:48:17.421 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14426053523642370829', time=2024-04-22T22:26:22.851908+0000, chain.objs.size()=1 2024-04-22T20:48:17.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DD0afK_jJczSvTvJf0uobblz5_w5Ams_0 2024-04-22T20:48:17.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6385959680376593206', time=2024-04-22T22:26:23.131951+0000, chain.objs.size()=1 2024-04-22T20:48:17.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.By0wMIuK34VDqAiMZTL63KLMCwwZOO-_0 2024-04-22T20:48:17.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4713878087395416420', time=2024-04-22T22:26:23.245492+0000, chain.objs.size()=1 2024-04-22T20:48:17.422 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.AD5c-eESJaf0pOHkzfgg9_2XSUdo2bC_0 2024-04-22T20:48:17.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4870710857242773880', time=2024-04-22T22:26:23.943004+0000, chain.objs.size()=1 2024-04-22T20:48:17.423 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.423+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uGGyFHcrID_DnXPhCVgxEaN1h-AJqif_0 2024-04-22T20:48:17.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.426+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7688371135918320079', time=2024-04-22T22:26:24.009365+0000, chain.objs.size()=1 2024-04-22T20:48:17.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.426+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DRw2DX_vKInZuIO-JdfUxWVSO0Qlpkh_0 2024-04-22T20:48:17.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.426+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12379260157947944393', time=2024-04-22T22:26:24.140000+0000, chain.objs.size()=1 2024-04-22T20:48:17.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.426+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.EsaUjsZSgSGthVmGT3z8obw0bdsh12E_0 2024-04-22T20:48:17.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.426+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.369608212116387205', time=2024-04-22T22:26:24.339256+0000, chain.objs.size()=1 2024-04-22T20:48:17.424 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.426+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.T_gMCwabfvQ_B0T-gaoIS58xEH5XF4P_0 2024-04-22T20:48:17.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17748947575204340378', time=2024-04-22T22:26:24.339771+0000, chain.objs.size()=1 2024-04-22T20:48:17.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.l58sWWs_Pm9yeLUTwIgW9jhqfNr3L7B_0 2024-04-22T20:48:17.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3043616891893340415', time=2024-04-22T22:26:24.444555+0000, chain.objs.size()=1 2024-04-22T20:48:17.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._iu3CN9MEinDStWM5whHUKd0ixgRySj_0 2024-04-22T20:48:17.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10734095133781274013', time=2024-04-22T22:26:24.861464+0000, chain.objs.size()=1 2024-04-22T20:48:17.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0PIIULLLtwecX9DefZNNt8qDG4VT9Xe_0 2024-04-22T20:48:17.425 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11293937581872871554', time=2024-04-22T22:26:25.634916+0000, chain.objs.size()=1 2024-04-22T20:48:17.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lSkH1XEOkADDryr_7WNHzLXlwFMrCke_0 2024-04-22T20:48:17.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16065263823260064538', time=2024-04-22T22:26:25.979668+0000, chain.objs.size()=1 2024-04-22T20:48:17.426 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.427+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.kIf24_qzA3_3BY1zXlt9oo4X118sdmV_0 2024-04-22T20:48:17.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.428+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2756486595800143632', time=2024-04-22T22:26:26.304440+0000, chain.objs.size()=1 2024-04-22T20:48:17.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.428+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.9Uj78AQv_3Ub_Mr6rHkTjHYKtaqKTUP_0 2024-04-22T20:48:17.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.428+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10382023541362704831', time=2024-04-22T22:26:26.457581+0000, chain.objs.size()=1 2024-04-22T20:48:17.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.428+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1JtCn2Lu0hugada5iCuRC3vrnCwlxTF_0 2024-04-22T20:48:17.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.428+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5752774199069315431', time=2024-04-22T22:26:26.822895+0000, chain.objs.size()=1 2024-04-22T20:48:17.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.428+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8RyD_DrT87YvnY4mZkE5Dqw5lBGo1eY_0 2024-04-22T20:48:17.427 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.430+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2404968699865906411', time=2024-04-22T22:26:27.636319+0000, chain.objs.size()=1 2024-04-22T20:48:17.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.430+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zPEkGX3Rk8WiYQfpIO1AHzdFtE3YX11_0 2024-04-22T20:48:17.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.430+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9504177484548538559', time=2024-04-22T22:26:27.728254+0000, chain.objs.size()=1 2024-04-22T20:48:17.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.430+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Ma74uy5EkPiZDurPVj_PPMrLin32NYo_0 2024-04-22T20:48:17.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.430+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13009939413883883629', time=2024-04-22T22:26:28.209568+0000, chain.objs.size()=1 2024-04-22T20:48:17.428 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.430+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UPt08jXMnAo8NCbgpE-9P1zlxibe7Mx_0 2024-04-22T20:48:17.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.431+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2960607733650554600', time=2024-04-22T22:26:28.344882+0000, chain.objs.size()=1 2024-04-22T20:48:17.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.431+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._JdrQKF283J2QHpGSpbnsZBzmg4n2Jo_0 2024-04-22T20:48:17.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.431+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.407420171724854511', time=2024-04-22T22:26:28.466336+0000, chain.objs.size()=1 2024-04-22T20:48:17.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.431+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.98hpodPctKaI1FIXDIF94876FMktQUp_0 2024-04-22T20:48:17.429 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.431+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7412210952897379943', time=2024-04-22T22:26:29.163705+0000, chain.objs.size()=1 2024-04-22T20:48:17.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.431+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VhfmBjXVIi8Ut6ZvMnhcJeUPzvhq5CW_0 2024-04-22T20:48:17.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.432+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16242260338157064748', time=2024-04-22T22:26:29.181633+0000, chain.objs.size()=1 2024-04-22T20:48:17.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.432+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DQTLIycsQhAZv2EgxJNy1qaDHVmKt4g_0 2024-04-22T20:48:17.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.432+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8880064896300661061', time=2024-04-22T22:26:29.493627+0000, chain.objs.size()=1 2024-04-22T20:48:17.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.432+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.FRqpTu2I9Gh5Xtydiaa_unIe0vb0-Ys_0 2024-04-22T20:48:17.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.432+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3718693518509812898', time=2024-04-22T22:26:29.530759+0000, chain.objs.size()=1 2024-04-22T20:48:17.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.432+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.C0BsfrhbNzOXSMRjRxT8YMANjo450ZC_0 2024-04-22T20:48:17.430 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.432+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1027926391594258875', time=2024-04-22T22:26:29.604025+0000, chain.objs.size()=1 2024-04-22T20:48:17.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.432+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-uTD89fUGdobjt3Zvvi8ZYhhdypvmjr_0 2024-04-22T20:48:17.431 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.433+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8201498087522740910', time=2024-04-22T22:26:29.671430+0000, chain.objs.size()=1 2024-04-22T20:48:17.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.433+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8znBSVX5Miuh6ua13-Z5c6gS2tjV80G_0 2024-04-22T20:48:17.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7096578446670112926', time=2024-04-22T22:26:29.715737+0000, chain.objs.size()=1 2024-04-22T20:48:17.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.fSUHLLEOio1OW_Rktvw0I0-MAZvCky1_0 2024-04-22T20:48:17.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12841693048498203233', time=2024-04-22T22:26:29.755950+0000, chain.objs.size()=1 2024-04-22T20:48:17.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.80kY4Zt2SA2BYPrDgIqObv9VZxNvjNX_0 2024-04-22T20:48:17.432 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.434+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5780702786062367123', time=2024-04-22T22:26:29.792614+0000, chain.objs.size()=1 2024-04-22T20:48:17.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.434+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tmgXVGTW81TtPuD1JjFfKKcWOK94MDA_0 2024-04-22T20:48:17.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.435+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3270140831780824977', time=2024-04-22T22:26:30.544591+0000, chain.objs.size()=1 2024-04-22T20:48:17.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.435+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_._WuC6kIs_YPxtHT6iEXt7b3-K_mIEli_0 2024-04-22T20:48:17.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.435+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2933274113805557429', time=2024-04-22T22:26:31.407352+0000, chain.objs.size()=1 2024-04-22T20:48:17.433 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.435+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.COhJbdeQcsWBdipluNIxxveZaCw9akq_0 2024-04-22T20:48:17.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14474375137865930722', time=2024-04-22T22:26:32.210365+0000, chain.objs.size()=1 2024-04-22T20:48:17.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Gm1uTzedM8apYgB-iCnseAs3ZXzZidT_0 2024-04-22T20:48:17.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12768532597918554318', time=2024-04-22T22:26:32.680193+0000, chain.objs.size()=1 2024-04-22T20:48:17.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BIwvJ5fJjAT_OwxB6bkECJXndslTONj_0 2024-04-22T20:48:17.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11007278211154046753', time=2024-04-22T22:26:33.104392+0000, chain.objs.size()=1 2024-04-22T20:48:17.434 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.IB4LR3Pl8szdu3HOMbpxSXCHlpMbZB6_0 2024-04-22T20:48:17.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5598766139647978675', time=2024-04-22T22:26:34.222030+0000, chain.objs.size()=1 2024-04-22T20:48:17.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-UBMH6sbYtyKeSPWb-S7bGgdRRSHIq8_0 2024-04-22T20:48:17.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17821458434814445876', time=2024-04-22T22:26:34.450002+0000, chain.objs.size()=1 2024-04-22T20:48:17.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6e5N5tLXzw_l6qlFLDDWmwTkwJ8X5dC_0 2024-04-22T20:48:17.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1497778084454915243', time=2024-04-22T22:26:56.945081+0000, chain.objs.size()=1 2024-04-22T20:48:17.435 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.436+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.274__shadow_.iW_hTh4X_nE4qzfxM7G49SEscn4dw8o_0 2024-04-22T20:48:17.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3497688942551935158', time=2024-04-22T22:26:59.970890+0000, chain.objs.size()=1 2024-04-22T20:48:17.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.277__shadow_.auy4byQnFypksu_FyHuZNentor96IQh_0 2024-04-22T20:48:17.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2841509899012916695', time=2024-04-22T22:26:59.976682+0000, chain.objs.size()=1 2024-04-22T20:48:17.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.277__shadow_.yXFQW0BakgtAySiBGwmht1ch7mpTCJU_0 2024-04-22T20:48:17.436 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17824607394186381403', time=2024-04-22T22:27:05.605194+0000, chain.objs.size()=1 2024-04-22T20:48:17.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_._5Cx0c95mthOCdzynjihieV6rfp_Fo7_0 2024-04-22T20:48:17.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11988096319838239794', time=2024-04-22T22:27:05.710429+0000, chain.objs.size()=1 2024-04-22T20:48:17.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_._D9fm770J5DeeQnWlA2q3xPg_4tgFw5_0 2024-04-22T20:48:17.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16689001878838903411', time=2024-04-22T22:27:11.987969+0000, chain.objs.size()=1 2024-04-22T20:48:17.437 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.439+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.286__shadow_.9SL1RBHc1785GjZafLu27nuwRGvAL7w_0 2024-04-22T20:48:17.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.440+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16127634357530518454', time=2024-04-22T22:27:20.227929+0000, chain.objs.size()=1 2024-04-22T20:48:17.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.440+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.294__shadow_.t9Mhie2Qlm3jQBO-BGAUL368LubZ8kX_0 2024-04-22T20:48:17.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.440+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7779784947357012097', time=2024-04-22T22:27:20.239467+0000, chain.objs.size()=1 2024-04-22T20:48:17.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.440+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.294__shadow_.n4Uu9ow9pLO0gj4ojJWbI5tSqfvmqPu_0 2024-04-22T20:48:17.438 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.441+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17120841969711445716', time=2024-04-22T22:28:17.794226+0000, chain.objs.size()=1 2024-04-22T20:48:17.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.441+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wCDcE227mN_RXS5JiR-FVuv0_NEfaGJ_0 2024-04-22T20:48:17.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.441+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14996229804044438876', time=2024-04-22T22:28:18.055802+0000, chain.objs.size()=1 2024-04-22T20:48:17.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.441+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.43zb4BEVS9K8ZsMXS_-uAq8yd9_LdPz_0 2024-04-22T20:48:17.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.441+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4535853499606467489', time=2024-04-22T22:28:18.435857+0000, chain.objs.size()=1 2024-04-22T20:48:17.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.441+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0k35VOp35Lwi6vHIXj-vDuynMN4fllu_0 2024-04-22T20:48:17.439 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.441+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12035553159251831152', time=2024-04-22T22:28:19.278888+0000, chain.objs.size()=1 2024-04-22T20:48:17.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.441+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.5USSrRC5y2Nof5QUrSG8a5j6ApPzwaU_0 2024-04-22T20:48:17.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3321481838083989095', time=2024-04-22T22:28:19.568318+0000, chain.objs.size()=1 2024-04-22T20:48:17.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7KAZMw7sqS9_LCCDofvaHbIuxfjZYAS_0 2024-04-22T20:48:17.440 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1490395407397138081', time=2024-04-22T22:28:19.759515+0000, chain.objs.size()=1 2024-04-22T20:48:17.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SIqFrHzQZyQreUcfk9qXn6CAuJlu4Av_0 2024-04-22T20:48:17.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17572111512951203322', time=2024-04-22T22:28:19.760246+0000, chain.objs.size()=1 2024-04-22T20:48:17.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.V12W61AlRy4hJ0LacZ3HmJ_r-9e-H_A_0 2024-04-22T20:48:17.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4253254725824448922', time=2024-04-22T22:28:20.815499+0000, chain.objs.size()=1 2024-04-22T20:48:17.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NkWxFCSPDe2CjnUJvI70LGbHv5bM1Op_0 2024-04-22T20:48:17.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10990845995104278882', time=2024-04-22T22:28:20.865572+0000, chain.objs.size()=1 2024-04-22T20:48:17.441 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mCfoUBGHMqr0meMeszPuWk3mFTd2zGm_0 2024-04-22T20:48:17.442 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14967653191609657198', time=2024-04-22T22:28:21.047356+0000, chain.objs.size()=1 2024-04-22T20:48:17.442 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.443+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fMP3XAalTn4_CMxpMk3ABeCcrW9c3YD_0 2024-04-22T20:48:17.442 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.444+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15439148143227064996', time=2024-04-22T22:28:21.136976+0000, chain.objs.size()=1 2024-04-22T20:48:17.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.444+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Q3oQcnSmOU9svh9ShSDWM92z1RYF9Jb_0 2024-04-22T20:48:17.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.445+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1084201022488438641', time=2024-04-22T22:28:21.250162+0000, chain.objs.size()=1 2024-04-22T20:48:17.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.445+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.G08hWxBXnB1_WZ7xbNMohV63YTyW87H_0 2024-04-22T20:48:17.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.445+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6525692662183329603', time=2024-04-22T22:28:21.790135+0000, chain.objs.size()=1 2024-04-22T20:48:17.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.445+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dZaZdVQF8c63sXOhjFtrByW1WIkt1Dh_0 2024-04-22T20:48:17.443 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.445+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3967144747999065076', time=2024-04-22T22:28:21.850445+0000, chain.objs.size()=1 2024-04-22T20:48:17.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.445+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NF9q5xuo8uao8KKvjhUNZtJsmii8N2T_0 2024-04-22T20:48:17.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.445+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8326588466339833269', time=2024-04-22T22:28:22.610873+0000, chain.objs.size()=1 2024-04-22T20:48:17.444 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.445+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kEnEjMIwxrqwawVNfp6qzV820Hpl9UP_0 2024-04-22T20:48:17.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.447+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2936375566172471690', time=2024-04-22T22:28:23.292737+0000, chain.objs.size()=1 2024-04-22T20:48:17.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.447+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.QzGSbMAx5XenCz-EdLSXUnoNbRU0FNB_0 2024-04-22T20:48:17.445 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18352214666466213968', time=2024-04-22T22:28:24.017659+0000, chain.objs.size()=1 2024-04-22T20:48:17.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.nNjrGg95PsXkKtK4s5tTcRu5TGJ_OaO_0 2024-04-22T20:48:17.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3354142012825421389', time=2024-04-22T22:28:24.254331+0000, chain.objs.size()=1 2024-04-22T20:48:17.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bmxn8PdNNUqU2VJvpV5v7BOKz5pNmCB_0 2024-04-22T20:48:17.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8498948757541069248', time=2024-04-22T22:28:24.381881+0000, chain.objs.size()=1 2024-04-22T20:48:17.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.24PNPa0fWrzjgApAZ-M6O5tch76rnlx_0 2024-04-22T20:48:17.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14758628088296986965', time=2024-04-22T22:28:25.788124+0000, chain.objs.size()=1 2024-04-22T20:48:17.446 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CxKMRUfaXh9l1Mw1rfEXD4CYMg-6Boy_0 2024-04-22T20:48:17.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17735735473806534542', time=2024-04-22T22:28:25.865928+0000, chain.objs.size()=1 2024-04-22T20:48:17.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.QUSw-DZnn99gYuDUH8lmJ1lE7xdqbGk_0 2024-04-22T20:48:17.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9327693255884635111', time=2024-04-22T22:28:26.411087+0000, chain.objs.size()=1 2024-04-22T20:48:17.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.448+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ExVeAYACr4vtOUYKFQrDBc-jooFlGic_0 2024-04-22T20:48:17.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.450+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8266575578326876181', time=2024-04-22T22:28:26.486825+0000, chain.objs.size()=1 2024-04-22T20:48:17.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.450+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.y2mD3ZNKqueFrroYX3B5VKmuU6vAaVw_0 2024-04-22T20:48:17.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.450+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5446883826500170671', time=2024-04-22T22:28:26.922905+0000, chain.objs.size()=1 2024-04-22T20:48:17.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.450+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.CyCTekuVzhey8Zr1rm0yN1tUPQ5amJZ_0 2024-04-22T20:48:17.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.450+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17641252033810141270', time=2024-04-22T22:28:26.986471+0000, chain.objs.size()=1 2024-04-22T20:48:17.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.450+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lEuZRcl7toBAa_PCxb3hyrnEF5A1Xak_0 2024-04-22T20:48:17.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.450+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.649205723017608967', time=2024-04-22T22:28:27.139291+0000, chain.objs.size()=1 2024-04-22T20:48:17.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.450+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Drwc3YjNZnl0g5ROq0snQLBf35iP5Hp_0 2024-04-22T20:48:17.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.451+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10826198215555413141', time=2024-04-22T22:28:28.525394+0000, chain.objs.size()=1 2024-04-22T20:48:17.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.451+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bKBNbIzGmP_WnNw87E0BRCNyGk81PYF_0 2024-04-22T20:48:17.449 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.451+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16680611049103396363', time=2024-04-22T22:29:06.456896+0000, chain.objs.size()=1 2024-04-22T20:48:17.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.451+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.UgRUcEL76frlISCBvhQgSO3RIAxHqBG_0 2024-04-22T20:48:17.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.452+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4666037042970762932', time=2024-04-22T22:29:06.494793+0000, chain.objs.size()=1 2024-04-22T20:48:17.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.452+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HPniMyuaUTLFJpBtvYqdB2_RHz9DTTq_0 2024-04-22T20:48:17.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.452+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13863629567403587993', time=2024-04-22T22:29:06.527129+0000, chain.objs.size()=1 2024-04-22T20:48:17.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.452+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1NqAVewSH3koAwDzfWiruZS8DiSddyI_0 2024-04-22T20:48:17.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.454+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5895139288428944321', time=2024-04-22T22:29:08.278318+0000, chain.objs.size()=1 2024-04-22T20:48:17.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.454+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MJoPSXm-gtQXeVJkU4skKD8UCxTKjdd_0 2024-04-22T20:48:17.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.454+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8725817259581270366', time=2024-04-22T22:29:08.436044+0000, chain.objs.size()=1 2024-04-22T20:48:17.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.454+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.YegIo9vOdJqmZjVtc9FRHKil8Ll53JV_0 2024-04-22T20:48:17.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.454+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14565382713086413283', time=2024-04-22T22:29:09.410600+0000, chain.objs.size()=1 2024-04-22T20:48:17.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.454+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.t02cnD7NnKAwdAlkPrQRJiVz6-oSy3x_0 2024-04-22T20:48:17.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7270877997353039590', time=2024-04-22T22:29:10.348083+0000, chain.objs.size()=1 2024-04-22T20:48:17.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TY3rgSISOkpDsYun7r657Q-iI3Za8Y5_0 2024-04-22T20:48:17.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16676576651672116729', time=2024-04-22T22:29:10.891958+0000, chain.objs.size()=1 2024-04-22T20:48:17.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._RjPMTPXzF1CRyd78hOou2n-WTHrTMz_0 2024-04-22T20:48:17.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2045990118919936803', time=2024-04-22T22:29:11.052291+0000, chain.objs.size()=1 2024-04-22T20:48:17.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1hSyRKMaeKQdLmFScqwUGqZNF-sONAp_0 2024-04-22T20:48:17.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12150366277362886384', time=2024-04-22T22:29:11.482745+0000, chain.objs.size()=1 2024-04-22T20:48:17.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.E0J8IFb5WVzdTFSA7aGYQYeADcB10YX_0 2024-04-22T20:48:17.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4934935829022025887', time=2024-04-22T22:29:12.611528+0000, chain.objs.size()=1 2024-04-22T20:48:17.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.o1tZOblN-X-MHeMqpQ7wnPTeHt55D0M_0 2024-04-22T20:48:17.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11666260915319628808', time=2024-04-22T22:29:14.029603+0000, chain.objs.size()=1 2024-04-22T20:48:17.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.455+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ekty0mVa2Dyb3q3-veNFLkbUKjioiZ0_0 2024-04-22T20:48:17.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.456+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6000740082718653732', time=2024-04-22T22:29:16.004124+0000, chain.objs.size()=1 2024-04-22T20:48:17.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.456+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.anmL7tD5l-jcVIhAMk_hJLAQLtXpAQc_0 2024-04-22T20:48:17.455 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.456+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8816634660751620253', time=2024-04-22T22:29:16.758362+0000, chain.objs.size()=1 2024-04-22T20:48:17.455 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.456+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.FSpLB0WWmcnjxbxGuotvcOpwNTiTHaF_0 2024-04-22T20:48:17.455 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.457+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3715926077625872798', time=2024-04-22T22:29:17.031086+0000, chain.objs.size()=1 2024-04-22T20:48:17.455 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.457+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Aqb5nfJ1s5Pu1tkbC8vRYKd-DpzNUr1_0 2024-04-22T20:48:17.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.458+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12220531752943081684', time=2024-04-22T22:29:17.072997+0000, chain.objs.size()=1 2024-04-22T20:48:17.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.458+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XJ89JaCAKQaAc4iVvchml75JTTZ5RHl_0 2024-04-22T20:48:17.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.461+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/48433 2024-04-22T20:48:17.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=20, truncated=0, next_marker='' 2024-04-22T20:48:17.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4696835386359337456', time=2024-04-22T22:29:17.326180+0000, chain.objs.size()=1 2024-04-22T20:48:17.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zVyCBJ3EoES5zqq42SK-1Ql7jek3Nc9_0 2024-04-22T20:48:17.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11777575549032485074', time=2024-04-22T22:29:17.421230+0000, chain.objs.size()=1 2024-04-22T20:48:17.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zqacpaIPiPOvyrCkpmNA1xDAmE75VN__0 2024-04-22T20:48:17.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1624430686099148271', time=2024-04-22T22:29:20.416056+0000, chain.objs.size()=1 2024-04-22T20:48:17.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.317__shadow_.cTXOPylrw3a_ilj7vfdYIzjEOTUXsMc_0 2024-04-22T20:48:17.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17218786037968052470', time=2024-04-22T22:29:31.742392+0000, chain.objs.size()=1 2024-04-22T20:48:17.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.333__shadow_.Biah9mSorsVkjyBvP7VXL2XvyI-INOU_0 2024-04-22T20:48:17.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12003263567926104415', time=2024-04-22T22:31:10.502710+0000, chain.objs.size()=1 2024-04-22T20:48:17.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.472__shadow_.KjS8YZuEeEvJWbSh9iFduK8CV9wEdxv_0 2024-04-22T20:48:17.462 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.463+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6393540438581154182', time=2024-04-22T22:32:09.336322+0000, chain.objs.size()=1 2024-04-22T20:48:17.463 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.507__multipart_mymultipart.2~akAJEkJpyXOEWO9cCxbtgMY3VvHO3VA.1 2024-04-22T20:48:17.463 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8037865785775650238', time=2024-04-22T22:32:52.044290+0000, chain.objs.size()=3 2024-04-22T20:48:17.463 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~p0UZm1MgB84tmOsFZPaF8ACIKGclby-.1 2024-04-22T20:48:17.463 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~p0UZm1MgB84tmOsFZPaF8ACIKGclby-.2 2024-04-22T20:48:17.463 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~p0UZm1MgB84tmOsFZPaF8ACIKGclby-.3 2024-04-22T20:48:17.463 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~sT2ZJgR2b-RyAXhQK2XBFx84aKM79sZ', time=2024-04-22T22:33:06.257627+0000, chain.objs.size()=1 2024-04-22T20:48:17.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~sT2ZJgR2b-RyAXhQK2XBFx84aKM79sZ.1 2024-04-22T20:48:17.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2747074245196603579', time=2024-04-22T22:37:09.414508+0000, chain.objs.size()=1 2024-04-22T20:48:17.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.549__multipart_mymultipart.2~kcKZ-c8NGui0MRK-CyZzJ_3H9Oq1rIt.1 2024-04-22T20:48:17.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13495254252477338935', time=2024-04-22T22:37:40.228291+0000, chain.objs.size()=1 2024-04-22T20:48:17.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.464+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow:gV2krPCYlQXLv4fBHPVMWlrR-pIA3C8_.HRMzrrAU3nHxJwYaYAKs6Dbs5vxb2aJ_0 2024-04-22T20:48:17.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17447313005215695905', time=2024-04-22T22:37:41.563641+0000, chain.objs.size()=1 2024-04-22T20:48:17.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow:c1m1nJn18fLEofD1c12fUQwxuNDj3a5_.KKcTOAdM5w8ZNWzLcgn7uunceBSSYeN_0 2024-04-22T20:48:17.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9034853456171689273', time=2024-04-22T22:37:53.275286+0000, chain.objs.size()=1 2024-04-22T20:48:17.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:Ei-rXqsha2aPQMvwG0Gapfh7sl3rDYT_.Rxcm23KiJC_KbE2W7Gzze3P9RskQCSn_0 2024-04-22T20:48:17.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14493611560007410859', time=2024-04-22T22:38:47.270577+0000, chain.objs.size()=7 2024-04-22T20:48:17.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~irtqeEDYyLVhBx5HVAI48PUM0oIKlA2.1 2024-04-22T20:48:17.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__shadow_testobj.2~irtqeEDYyLVhBx5HVAI48PUM0oIKlA2.1_1 2024-04-22T20:48:17.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~irtqeEDYyLVhBx5HVAI48PUM0oIKlA2.2 2024-04-22T20:48:17.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__shadow_testobj.2~irtqeEDYyLVhBx5HVAI48PUM0oIKlA2.2_1 2024-04-22T20:48:17.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~irtqeEDYyLVhBx5HVAI48PUM0oIKlA2.3 2024-04-22T20:48:17.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__shadow_testobj.2~irtqeEDYyLVhBx5HVAI48PUM0oIKlA2.3_1 2024-04-22T20:48:17.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.467+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.566__multipart_testobj.2~irtqeEDYyLVhBx5HVAI48PUM0oIKlA2.4 2024-04-22T20:48:17.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.468+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15724914959070044448', time=2024-04-22T22:38:57.537452+0000, chain.objs.size()=1 2024-04-22T20:48:17.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.468+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.574__shadow:jis.WN2z4smB9-ax1iJGq8yCXUIWYAd_.DSOolDji_WGh6dsR_VthxSl3dOLtb2D_0 2024-04-22T20:48:17.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.468+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV', time=2024-04-22T22:41:47.449693+0000, chain.objs.size()=12 2024-04-22T20:48:17.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.468+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__multipart_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.1 2024-04-22T20:48:17.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.471+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__shadow_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.1_1 2024-04-22T20:48:17.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.471+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__multipart_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.2 2024-04-22T20:48:17.469 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__shadow_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.2_1 2024-04-22T20:48:17.469 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__multipart_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.3 2024-04-22T20:48:17.470 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__shadow_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.3_1 2024-04-22T20:48:17.470 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__multipart_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.4 2024-04-22T20:48:17.470 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__shadow_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.4_1 2024-04-22T20:48:17.470 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__multipart_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.5 2024-04-22T20:48:17.470 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__shadow_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.5_1 2024-04-22T20:48:17.470 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__multipart_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.6 2024-04-22T20:48:17.471 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.473+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.611__shadow_multipart_enc.2~-7Fz6JMqdXQmOuga2UcVKeZJ1CmHZdV.6_1 2024-04-22T20:48:17.471 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.474+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3159834294910686819', time=2024-04-22T22:41:57.343855+0000, chain.objs.size()=1 2024-04-22T20:48:17.471 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.474+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.625__shadow_.YjSyDzuDfDLXOXn7nh0drR5s-aQfw5a_0 2024-04-22T20:48:17.471 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.474+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14326108831078628915', time=2024-04-22T22:42:51.135753+0000, chain.objs.size()=6 2024-04-22T20:48:17.472 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.474+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.644__multipart_bar.2~BMl3kzkIv5NcvIZdegrNelVAKLUMAaj.1 2024-04-22T20:48:17.472 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.475+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.644__multipart_bar.2~BMl3kzkIv5NcvIZdegrNelVAKLUMAaj.2 2024-04-22T20:48:17.474 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.477+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.644__multipart_bar.2~BMl3kzkIv5NcvIZdegrNelVAKLUMAaj.3 2024-04-22T20:48:17.474 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.477+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.644__multipart_bar.2~BMl3kzkIv5NcvIZdegrNelVAKLUMAaj.4 2024-04-22T20:48:17.474 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.477+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.644__multipart_bar.2~BMl3kzkIv5NcvIZdegrNelVAKLUMAaj.5 2024-04-22T20:48:17.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.477+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.644__multipart_bar.2~BMl3kzkIv5NcvIZdegrNelVAKLUMAaj.6 2024-04-22T20:48:17.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.477+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5003260572441330948', time=2024-04-22T22:43:11.039269+0000, chain.objs.size()=1 2024-04-22T20:48:17.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.477+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.677__shadow:jyCEEkFeKh7f.rIqJHK9nmym6-HS3Bf_.F25b1wqamBUKfCscBfUio667AEnumS-_0 2024-04-22T20:48:17.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.478+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11696775919645934965', time=2024-04-22T22:43:17.051297+0000, chain.objs.size()=1 2024-04-22T20:48:17.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.478+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.686__multipart_file1.2~c0E1k0QVES9wzVs5zt_FF3OCZ8DEi9Q.1 2024-04-22T20:48:17.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.478+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12662913650807515698', time=2024-04-22T22:43:46.821367+0000, chain.objs.size()=1 2024-04-22T20:48:17.475 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.478+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.702__shadow_.Y124PckKJuyRelgHUVM0UmYeFkBRUy6_0 2024-04-22T20:48:17.478 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.481+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.481 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.484+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-22T20:48:17.482 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.485+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.482 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.483 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=0, next_marker='' 2024-04-22T20:48:17.483 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_8uXQK-aGKbfgd8k0C_xhKj_R8XvWKVF', time=2024-04-22T22:21:48.871127+0000, chain.objs.size()=3 2024-04-22T20:48:17.484 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.qhkSgNWoTkhW2wnFk-WHs56k_8thG6f_0 2024-04-22T20:48:17.484 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.qhkSgNWoTkhW2wnFk-WHs56k_8thG6f_1 2024-04-22T20:48:17.484 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.qhkSgNWoTkhW2wnFk-WHs56k_8thG6f_2 2024-04-22T20:48:17.484 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='___gf0Bjx4wBDX3aAeBtjSYBJ-urTbeX', time=2024-04-22T22:21:55.282622+0000, chain.objs.size()=3 2024-04-22T20:48:17.484 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.6-sQWdkAqt_45wu626SkIcz7l_QmFKr_0 2024-04-22T20:48:17.484 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.6-sQWdkAqt_45wu626SkIcz7l_QmFKr_1 2024-04-22T20:48:17.485 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.486+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.6-sQWdkAqt_45wu626SkIcz7l_QmFKr_2 2024-04-22T20:48:17.485 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_T721-EBwj1CTMlAWhiYdNEnzCc_bKZC', time=2024-04-22T22:22:16.211182+0000, chain.objs.size()=3 2024-04-22T20:48:17.485 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.Dct1qha9ZEKch2nhaNaajE64A0rEqqD_0 2024-04-22T20:48:17.485 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.Dct1qha9ZEKch2nhaNaajE64A0rEqqD_1 2024-04-22T20:48:17.485 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.150__shadow_.Dct1qha9ZEKch2nhaNaajE64A0rEqqD_2 2024-04-22T20:48:17.485 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15018797138976253130', time=2024-04-22T22:22:47.832271+0000, chain.objs.size()=1 2024-04-22T20:48:17.486 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.166__shadow_.QrUu9vsyeMozoObfHLbKudOcWY3IYCB_0 2024-04-22T20:48:17.486 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12317382479326488495', time=2024-04-22T22:25:16.304365+0000, chain.objs.size()=1 2024-04-22T20:48:17.486 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.223__shadow_.5C0IbTYzBeCoGOmrn7PDueZVL1Pjo3Z_0 2024-04-22T20:48:17.486 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9952207931379830063', time=2024-04-22T22:26:22.689422+0000, chain.objs.size()=1 2024-04-22T20:48:17.486 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.487+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.LmgWVEFxybiK8lmjhE1GfjNRv9tHwU4_0 2024-04-22T20:48:17.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7799881673212517623', time=2024-04-22T22:26:23.239124+0000, chain.objs.size()=1 2024-04-22T20:48:17.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MiZaHeQUFjZO8yJsOLJko9pnFGFzfpm_0 2024-04-22T20:48:17.487 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12452037507331764020', time=2024-04-22T22:26:23.275597+0000, chain.objs.size()=1 2024-04-22T20:48:17.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Is8m899h7Vr46Utey8qvhzVANwhTsqE_0 2024-04-22T20:48:17.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7510573723612325819', time=2024-04-22T22:26:23.429425+0000, chain.objs.size()=1 2024-04-22T20:48:17.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CNcoqAjMyDQ7o_bbQOYjHB0T5Jd2ZWv_0 2024-04-22T20:48:17.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17139318642700111736', time=2024-04-22T22:26:23.463860+0000, chain.objs.size()=1 2024-04-22T20:48:17.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.nqZVk1eOcS9uKOWGg8GaauU_d25StI0_0 2024-04-22T20:48:17.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16931156767997083878', time=2024-04-22T22:26:24.563716+0000, chain.objs.size()=1 2024-04-22T20:48:17.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.qJE_b9ifafOdFG1hFydAAgT_bv4RKeY_0 2024-04-22T20:48:17.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15672520380454569494', time=2024-04-22T22:26:25.971999+0000, chain.objs.size()=1 2024-04-22T20:48:17.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cWLO0-9cX3pH3etkwOHSNA8e2vCM55q_0 2024-04-22T20:48:17.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3884562117706285921', time=2024-04-22T22:26:26.120722+0000, chain.objs.size()=1 2024-04-22T20:48:17.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.68p5UNZyk0_46SX4wYD0d3-V98-mADX_0 2024-04-22T20:48:17.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9097539200558651335', time=2024-04-22T22:26:26.543874+0000, chain.objs.size()=1 2024-04-22T20:48:17.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.490+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.mre2OcTpIYrllDefzevCFoAVAlAq0YF_0 2024-04-22T20:48:17.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2612621698566556871', time=2024-04-22T22:26:27.925150+0000, chain.objs.size()=1 2024-04-22T20:48:17.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.wdDsfU2crUgmh-E3XYuUo0R6dLqKsNX_0 2024-04-22T20:48:17.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10408698409001209281', time=2024-04-22T22:26:27.950646+0000, chain.objs.size()=1 2024-04-22T20:48:17.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ZKiV9_dOmkr9vBHF-C_8ZHo1qVTJ8fO_0 2024-04-22T20:48:17.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9026278023002241439', time=2024-04-22T22:26:28.408787+0000, chain.objs.size()=1 2024-04-22T20:48:17.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.T2SyP-zbHwtHQnsNw8KXKpvrijrYZjR_0 2024-04-22T20:48:17.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.922467993764586988', time=2024-04-22T22:26:28.608266+0000, chain.objs.size()=1 2024-04-22T20:48:17.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.p0tMeTedDRhilI_Su-1BsEjyKuf3-6y_0 2024-04-22T20:48:17.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16146814422817593706', time=2024-04-22T22:26:29.177754+0000, chain.objs.size()=1 2024-04-22T20:48:17.491 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.n_LoZO7cO7bk1O2w9cIqtvYt5epURI1_0 2024-04-22T20:48:17.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8093596076844971764', time=2024-04-22T22:26:29.904260+0000, chain.objs.size()=1 2024-04-22T20:48:17.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.NZxzceoGCbm1mzRPrJa8Z3e7ORSbCKT_0 2024-04-22T20:48:17.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8764275414961098508', time=2024-04-22T22:26:30.127005+0000, chain.objs.size()=1 2024-04-22T20:48:17.492 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.493+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.khHjXsBa6Y1F3blt7W02DUG6oM9LkDT_0 2024-04-22T20:48:17.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.494+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17835376488331200920', time=2024-04-22T22:26:30.963413+0000, chain.objs.size()=1 2024-04-22T20:48:17.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.494+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.u2xqlhUpX67uMWjYf18P6oLpNo7KbVa_0 2024-04-22T20:48:17.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.494+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1818973499478491369', time=2024-04-22T22:26:32.201274+0000, chain.objs.size()=1 2024-04-22T20:48:17.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.494+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.8e26Ma4lD_DFEoAjFpnFIbVsLrY0j3J_0 2024-04-22T20:48:17.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.494+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8161604126224955656', time=2024-04-22T22:26:32.208581+0000, chain.objs.size()=1 2024-04-22T20:48:17.493 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.494+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Xtfrvh4eCZ4aFRPVnbmLUHcMuyIpgzT_0 2024-04-22T20:48:17.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.494+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11498564544127499401', time=2024-04-22T22:26:32.593231+0000, chain.objs.size()=1 2024-04-22T20:48:17.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.494+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.i0-oK0SW1a-q6bfOkGkd1YC5eoAjdEB_0 2024-04-22T20:48:17.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.496+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13939814480311410666', time=2024-04-22T22:26:32.678365+0000, chain.objs.size()=1 2024-04-22T20:48:17.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.496+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hhUMwe3n4tpOEAn_lWzQQ_nLB2GUcCJ_0 2024-04-22T20:48:17.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.496+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16692236682393947160', time=2024-04-22T22:26:33.057693+0000, chain.objs.size()=1 2024-04-22T20:48:17.494 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.496+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1szzaGngconYcMIi7wVsoq8EVQxz6Qo_0 2024-04-22T20:48:17.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.497+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12709426195616847578', time=2024-04-22T22:26:33.068485+0000, chain.objs.size()=1 2024-04-22T20:48:17.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.497+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Soty-512y-HdUDO6WVPN2w57--POMw4_0 2024-04-22T20:48:17.495 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.497+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12934003329706066419', time=2024-04-22T22:26:33.504001+0000, chain.objs.size()=1 2024-04-22T20:48:17.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.497+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Yfoueb60E2M1DaqrRseKt_8GatcXkdt_0 2024-04-22T20:48:17.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.498+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10031946482939986101', time=2024-04-22T22:26:34.326079+0000, chain.objs.size()=1 2024-04-22T20:48:17.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.498+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pCt-9N6GrGYV8ZBM4chf2P1fn86D8yy_0 2024-04-22T20:48:17.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.498+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2537111528153599957', time=2024-04-22T22:26:41.149780+0000, chain.objs.size()=1 2024-04-22T20:48:17.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.498+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.258__shadow_.hSN_ulA8AnrAWkpCeQ_ERwk1fizEAMl_0 2024-04-22T20:48:17.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5352901327491980962', time=2024-04-22T22:26:44.030889+0000, chain.objs.size()=1 2024-04-22T20:48:17.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.261__shadow_.5Ode7NrZTEuQL9l-GeYHtNKF7Da5adB_0 2024-04-22T20:48:17.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13377111054806142156', time=2024-04-22T22:26:46.911723+0000, chain.objs.size()=1 2024-04-22T20:48:17.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.264__shadow_.4ujBFiUnNqohBdB_K6gN9CH3wVJmPPy_0 2024-04-22T20:48:17.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11501748716064379575', time=2024-04-22T22:26:48.882647+0000, chain.objs.size()=1 2024-04-22T20:48:17.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.266__shadow_.U1S1jJGr8Pw9bk18G6k0PbvV6AxrIuX_0 2024-04-22T20:48:17.696 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18330229888277336348', time=2024-04-22T22:26:53.870690+0000, chain.objs.size()=1 2024-04-22T20:48:17.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.271__shadow_.8coOT7q_JKt88nG8c2CD_exPTAVBl-L_0 2024-04-22T20:48:17.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.830844745196698886', time=2024-04-22T22:27:01.957807+0000, chain.objs.size()=1 2024-04-22T20:48:17.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.279__shadow_.Uh6x9y1U1FSfbpcKparJPKvoY1tw75N_0 2024-04-22T20:48:17.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4765440455897792131', time=2024-04-22T22:27:05.609030+0000, chain.objs.size()=1 2024-04-22T20:48:17.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.281__shadow_.v5BypZe-I2yhezqlq3PwgchbY-agov8_0 2024-04-22T20:48:17.697 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.44734240486627157', time=2024-04-22T22:27:07.853603+0000, chain.objs.size()=1 2024-04-22T20:48:17.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.2lcqAa1njiVjAC-EZAGWqjOhpGo-7-l_0 2024-04-22T20:48:17.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4329575324253410300', time=2024-04-22T22:27:08.961163+0000, chain.objs.size()=1 2024-04-22T20:48:17.698 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.698+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.283__shadow_.lvIEUyHWX4YmSRP7GzilyXGlWGRlpcF_0 2024-04-22T20:48:17.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.699+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13342975871525966429', time=2024-04-22T22:27:20.239612+0000, chain.objs.size()=1 2024-04-22T20:48:17.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.699+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.294__shadow_.EIdy3Cu8VOCk1tnvmuf4se_SXMqQA63_0 2024-04-22T20:48:17.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.699+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10956588064993210581', time=2024-04-22T22:27:37.289290+0000, chain.objs.size()=1 2024-04-22T20:48:17.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.699+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.308__shadow:wE6SmCegchLxEfyruwrx3eIFdUt3-KF_.HdGGy3F5vxpt1eeeeSDvYJi59I7i9Nq_0 2024-04-22T20:48:17.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.699+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16145923837199663758', time=2024-04-22T22:28:18.057286+0000, chain.objs.size()=1 2024-04-22T20:48:17.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.699+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JYVJ3T4VStId2DLyUVCex0kZTc704fe_0 2024-04-22T20:48:17.699 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12324427635885551294', time=2024-04-22T22:28:18.058010+0000, chain.objs.size()=1 2024-04-22T20:48:17.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tHw4GKhi98xH7gVhFkZ5iPgXgv7OPRn_0 2024-04-22T20:48:17.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.701+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2540638164135237300', time=2024-04-22T22:28:19.061042+0000, chain.objs.size()=1 2024-04-22T20:48:17.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.701+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.z1FTN9zXbXbeAF6PgGl43z4jBt8uKR__0 2024-04-22T20:48:17.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18128007297595028091', time=2024-04-22T22:28:19.061832+0000, chain.objs.size()=1 2024-04-22T20:48:17.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KxzishYrnNS_IJnqOXtXGltMbFNukpr_0 2024-04-22T20:48:17.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18328996370459550477', time=2024-04-22T22:28:19.767697+0000, chain.objs.size()=1 2024-04-22T20:48:17.701 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.02Qhlyru1CtevN9w8QIL8qTcgWolmKC_0 2024-04-22T20:48:17.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11086074186110322519', time=2024-04-22T22:28:19.997364+0000, chain.objs.size()=1 2024-04-22T20:48:17.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bp6kyv0FWASQkeyLAcUC1cmSGTeYqvq_0 2024-04-22T20:48:17.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8197524118606725049', time=2024-04-22T22:28:20.091063+0000, chain.objs.size()=1 2024-04-22T20:48:17.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.c7fsk5p7LSIRswyQqC1fizmcWrKQm0O_0 2024-04-22T20:48:17.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14430469364399130035', time=2024-04-22T22:28:20.709086+0000, chain.objs.size()=1 2024-04-22T20:48:17.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IvQcDfiLc09awR1YHAlLuwJWzloXDJR_0 2024-04-22T20:48:17.702 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.981037224597947602', time=2024-04-22T22:28:21.476591+0000, chain.objs.size()=1 2024-04-22T20:48:17.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MMi67n4ceg_r1CNZ24HGz_vfJLIZ4Av_0 2024-04-22T20:48:17.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14331342012655350792', time=2024-04-22T22:28:21.808896+0000, chain.objs.size()=1 2024-04-22T20:48:17.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.W0YFVtPG-5P-8Bf7V49fgpljMw8KYb2_0 2024-04-22T20:48:17.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15062226644087554771', time=2024-04-22T22:28:21.809596+0000, chain.objs.size()=1 2024-04-22T20:48:17.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.mtUWYbjAq3OMA6vLxWAz3NkL0R2BTRk_0 2024-04-22T20:48:17.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17804775090061687379', time=2024-04-22T22:28:21.810292+0000, chain.objs.size()=1 2024-04-22T20:48:17.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.703+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IL-Mg8z1DWA9ydf2mUD6GlYiqnoWGuZ_0 2024-04-22T20:48:17.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.704+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.119221414698723049', time=2024-04-22T22:28:22.327429+0000, chain.objs.size()=1 2024-04-22T20:48:17.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.704+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.M0KnuYtfcq51pYFEfRLWs6MeSDSXczW_0 2024-04-22T20:48:17.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.704+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14596543790776537649', time=2024-04-22T22:28:22.328216+0000, chain.objs.size()=1 2024-04-22T20:48:17.704 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.704+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FwyfO0_t4H8HaCx119SZBj9LQ_14K5V_0 2024-04-22T20:48:17.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.705+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3631429066644150963', time=2024-04-22T22:28:22.813337+0000, chain.objs.size()=1 2024-04-22T20:48:17.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.705+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.A3qT9ycc4K3UlVuSeE7Ms0opSBp1O0W_0 2024-04-22T20:48:17.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.708+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3903212287256549811', time=2024-04-22T22:28:23.341532+0000, chain.objs.size()=1 2024-04-22T20:48:17.705 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.708+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.npS1DEXZb6MP4SV0f4UzQlJURwhZScg_0 2024-04-22T20:48:17.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.708+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8332133381229829885', time=2024-04-22T22:28:23.960812+0000, chain.objs.size()=1 2024-04-22T20:48:17.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.708+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KKYl_pfMZs91rhHhSFnWJhHZdqcgzyL_0 2024-04-22T20:48:17.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.708+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.497324088479960917', time=2024-04-22T22:28:24.215427+0000, chain.objs.size()=1 2024-04-22T20:48:17.706 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.708+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.zFoQcF1B0xjcbi7djTZa1SJrFYVHvJQ_0 2024-04-22T20:48:17.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4846181288869439553', time=2024-04-22T22:28:25.021296+0000, chain.objs.size()=1 2024-04-22T20:48:17.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wYHF5oOgkdraqTcxiYWVc2Cg5GSYzv-_0 2024-04-22T20:48:17.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7652132018526716721', time=2024-04-22T22:28:25.415859+0000, chain.objs.size()=1 2024-04-22T20:48:17.707 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.DzFxhnL_iuTSNb8XaDHXmdF1jUFeMrc_0 2024-04-22T20:48:17.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8288686906569948638', time=2024-04-22T22:28:25.416620+0000, chain.objs.size()=1 2024-04-22T20:48:17.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rYsxWMZDd4OsZLbVsKNGpk4sse_l5yE_0 2024-04-22T20:48:17.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15538682132771638407', time=2024-04-22T22:28:25.794852+0000, chain.objs.size()=1 2024-04-22T20:48:17.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.qyqcmheUciLttl86-qanPPhZDWqQWNw_0 2024-04-22T20:48:17.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10994850875141832031', time=2024-04-22T22:28:26.137147+0000, chain.objs.size()=1 2024-04-22T20:48:17.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PVy9sQdLwLW6xgEOIn6_M_nEXebQayD_0 2024-04-22T20:48:17.708 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3814429739403055451', time=2024-04-22T22:28:26.962706+0000, chain.objs.size()=1 2024-04-22T20:48:17.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.HkllqZ_TLmVGi06Z24y0sBVftrh9Cp2_0 2024-04-22T20:48:17.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6373768870064271904', time=2024-04-22T22:28:27.142480+0000, chain.objs.size()=1 2024-04-22T20:48:17.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-NChFm8CqAx1UXtpI5mO2d040CkL6JG_0 2024-04-22T20:48:17.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13759612357128008435', time=2024-04-22T22:28:27.783830+0000, chain.objs.size()=1 2024-04-22T20:48:17.709 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.709+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FiN88XBpJIyLierm2P9ntYmAbV2YRu3_0 2024-04-22T20:48:17.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.711+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4817407295373004933', time=2024-04-22T22:29:06.307740+0000, chain.objs.size()=1 2024-04-22T20:48:17.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.711+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Wz9LWgsXpZv5RRNLIVCWS0uUPPq9vR-_0 2024-04-22T20:48:17.710 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17695293670110652205', time=2024-04-22T22:29:06.773354+0000, chain.objs.size()=1 2024-04-22T20:48:17.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.E2nzf_Jhdk_aAjBuLD7Y9f6VkfIWsgT_0 2024-04-22T20:48:17.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8470059073900230612', time=2024-04-22T22:29:07.806978+0000, chain.objs.size()=1 2024-04-22T20:48:17.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ReOFYWazqvDHDvk9AJ1Q0x_E36wGK53_0 2024-04-22T20:48:17.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4257199200969260087', time=2024-04-22T22:29:08.284187+0000, chain.objs.size()=1 2024-04-22T20:48:17.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QK4kLhUWXzzX89sdsFYrNmym7m5WrEk_0 2024-04-22T20:48:17.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16053998755295589247', time=2024-04-22T22:29:08.604204+0000, chain.objs.size()=1 2024-04-22T20:48:17.711 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NY2ere5M39Gxk6n7xvmbv1TtK6zTQRD_0 2024-04-22T20:48:17.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14042362050799379290', time=2024-04-22T22:29:08.631941+0000, chain.objs.size()=1 2024-04-22T20:48:17.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.712+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.U65VnSn23TdNYktFrcaOi1_2ED2a4za_0 2024-04-22T20:48:17.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17577947942437709682', time=2024-04-22T22:29:09.412242+0000, chain.objs.size()=1 2024-04-22T20:48:17.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.FK_w15WMnyNt3XdP1x7DLKOkQcloRxP_0 2024-04-22T20:48:17.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5376253578134401234', time=2024-04-22T22:29:09.717793+0000, chain.objs.size()=1 2024-04-22T20:48:17.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bnlbDZRd_R_wJxvsXh3r-vFoA5NPsm7_0 2024-04-22T20:48:17.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7680361498250756408', time=2024-04-22T22:29:10.215030+0000, chain.objs.size()=1 2024-04-22T20:48:17.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KoYk3nv9B5PaxkzgznWFwsAAmmjn-at_0 2024-04-22T20:48:17.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10590771310024356479', time=2024-04-22T22:29:10.869759+0000, chain.objs.size()=1 2024-04-22T20:48:17.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oubI47y3V-dCVRLmUrezCWfNLo8w7TN_0 2024-04-22T20:48:17.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5772233332040274839', time=2024-04-22T22:29:10.895711+0000, chain.objs.size()=1 2024-04-22T20:48:17.713 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.713+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0vx0ZhaDrQyUEHNnqkuWhkYK52h2s9a_0 2024-04-22T20:48:17.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.715+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13647007498700404380', time=2024-04-22T22:29:11.048144+0000, chain.objs.size()=1 2024-04-22T20:48:17.714 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.715+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.w8nHCw8nDYGtD9YRho79clWAyzndBEK_0 2024-04-22T20:48:17.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.715+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5467673853549456782', time=2024-04-22T22:29:11.223750+0000, chain.objs.size()=1 2024-04-22T20:48:17.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.715+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Vdqr3dMuBBannh2EwL16eytmWyCqFCr_0 2024-04-22T20:48:17.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9688087402556792306', time=2024-04-22T22:29:11.224336+0000, chain.objs.size()=1 2024-04-22T20:48:17.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nz6VNuazlt4zx6UT7bDB1Z1_lKlmXVg_0 2024-04-22T20:48:17.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6606159551407173727', time=2024-04-22T22:29:11.952247+0000, chain.objs.size()=1 2024-04-22T20:48:17.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-eXoJxHyCPUzFDV5cFt-SdIKBiWzLCJ_0 2024-04-22T20:48:17.715 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4936235867813827080', time=2024-04-22T22:29:12.472371+0000, chain.objs.size()=1 2024-04-22T20:48:17.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sncARc0lzsNCmCzrplr9vi_DSNkpt82_0 2024-04-22T20:48:17.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18018550973334286675', time=2024-04-22T22:29:12.473230+0000, chain.objs.size()=1 2024-04-22T20:48:17.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OUuahK6xfbRJ5kTwygqmwJFTQVfVcrw_0 2024-04-22T20:48:17.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12101028070784816315', time=2024-04-22T22:29:13.015196+0000, chain.objs.size()=1 2024-04-22T20:48:17.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bGxWX1NwzJwB73cOIw4zELICOW6UWSU_0 2024-04-22T20:48:17.716 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16767919286355794515', time=2024-04-22T22:29:13.185552+0000, chain.objs.size()=1 2024-04-22T20:48:17.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.717+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8ipUtLMRsOhq5BXHMj6M241WdeXI4L4_0 2024-04-22T20:48:17.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10298306758098116921', time=2024-04-22T22:29:13.499601+0000, chain.objs.size()=1 2024-04-22T20:48:17.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.WrbV0QEHSfILZGC3wchm-kpok4DfdrD_0 2024-04-22T20:48:17.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13621603581960082044', time=2024-04-22T22:29:14.070505+0000, chain.objs.size()=1 2024-04-22T20:48:17.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sLCnovoUrMLzqbff-A84ozJgEnATw_0_0 2024-04-22T20:48:17.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5567417379008263921', time=2024-04-22T22:29:14.481931+0000, chain.objs.size()=1 2024-04-22T20:48:17.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xTCW4WKN51etLnB4IefS7mSdoA-3gHR_0 2024-04-22T20:48:17.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8596490188790225773', time=2024-04-22T22:29:14.556836+0000, chain.objs.size()=1 2024-04-22T20:48:17.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XoxTtqTJoadv5vyCk6ykeLXAtPA8XJV_0 2024-04-22T20:48:17.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16626115529992108454', time=2024-04-22T22:29:15.676439+0000, chain.objs.size()=1 2024-04-22T20:48:17.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.718+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.iTuFuizcPTz9w8zjKKRPCe4O8TIA4VA_0 2024-04-22T20:48:17.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9395163435858625652', time=2024-04-22T22:29:16.756308+0000, chain.objs.size()=1 2024-04-22T20:48:17.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ipnfhYrJHklCjlWQ-WKRNN5iD0rcmpb_0 2024-04-22T20:48:17.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4151698033520566252', time=2024-04-22T22:29:16.934815+0000, chain.objs.size()=1 2024-04-22T20:48:17.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IisiW_Ogi4xECZsbmwiJzp4xa0ACj6S_0 2024-04-22T20:48:17.720 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5018256958950619918', time=2024-04-22T22:29:16.962356+0000, chain.objs.size()=1 2024-04-22T20:48:17.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.S0FSY6TxJiNTcw6AG6Teq1KyEPPnNep_0 2024-04-22T20:48:17.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6002222124321363934', time=2024-04-22T22:35:12.963962+0000, chain.objs.size()=2 2024-04-22T20:48:17.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~MAboAolE2aa38bv1ZZroPx3yAbWNb7q.1 2024-04-22T20:48:17.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.514__multipart_mymultipart.2~MAboAolE2aa38bv1ZZroPx3yAbWNb7q.2 2024-04-22T20:48:17.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9021714232349273251', time=2024-04-22T22:37:07.604982+0000, chain.objs.size()=1 2024-04-22T20:48:17.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.547__shadow_.65rlAOARv6_tWPHHEcEQQC-91Hgi5QB_0 2024-04-22T20:48:17.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14303183832093867964', time=2024-04-22T22:37:38.457459+0000, chain.objs.size()=1 2024-04-22T20:48:17.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.562__shadow_.KQ8KiFuK4nvs6-sX24C2CyNpMdD7xTS_0 2024-04-22T20:48:17.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9697889221695021598', time=2024-04-22T22:38:02.525192+0000, chain.objs.size()=1 2024-04-22T20:48:17.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:3qjx2hjmWvJhP4aeNXLQyHLJ3yLo3e2_.VHlqqMzVgoM4qgJ2WcsJk_rADSx8I5t_0 2024-04-22T20:48:17.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12833032524918306180', time=2024-04-22T22:42:06.840507+0000, chain.objs.size()=1 2024-04-22T20:48:17.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.629__shadow_.Riy_NK1onRqB2plpFsrHOuSB_IPqDxb_0 2024-04-22T20:48:17.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8850703406810264713', time=2024-04-22T22:42:54.698437+0000, chain.objs.size()=1 2024-04-22T20:48:17.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.723+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.649__shadow_.IT7vmI-G-lCXcdlz2eLbgxVRD6NLe4P_0 2024-04-22T20:48:17.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.728+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.730+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-22T20:48:17.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.732+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.732+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43364' 2024-04-22T20:48:17.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17245186712748701074', time=2024-04-22T22:22:22.313591+0000, chain.objs.size()=1 2024-04-22T20:48:17.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.151__shadow_.f7WZ67gynQ3PdUt9NEcCqnbIGipkVqx_0 2024-04-22T20:48:17.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13897336620698402316', time=2024-04-22T22:23:24.784067+0000, chain.objs.size()=1 2024-04-22T20:48:17.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.192__shadow_.qEfFOVu4dsvDK6VtVY0tfp9s_Hmmz1w_0 2024-04-22T20:48:17.731 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12730731612037385036', time=2024-04-22T22:25:27.139573+0000, chain.objs.size()=1 2024-04-22T20:48:17.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.232__shadow_.ED--8ve_sNyaja6xPEW-nHO4AaoW6GJ_0 2024-04-22T20:48:17.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4644125279797287312', time=2024-04-22T22:25:33.793312+0000, chain.objs.size()=1 2024-04-22T20:48:17.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.238__shadow_.S6mIMokUafk6Y-vAtrCWsggeNo0_KgR_0 2024-04-22T20:48:17.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18304221553239465838', time=2024-04-22T22:25:40.805417+0000, chain.objs.size()=1 2024-04-22T20:48:17.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.245__shadow_.6f2Df8BmqOIO0m1RNf-ppSX-f1QdbSZ_0 2024-04-22T20:48:17.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17297971621046546920', time=2024-04-22T22:26:22.589987+0000, chain.objs.size()=1 2024-04-22T20:48:17.732 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.D8ym3YPBZlf9Sh0C4nm0QnUI5lQ1Zo5_0 2024-04-22T20:48:17.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5603628138656385414', time=2024-04-22T22:26:22.590677+0000, chain.objs.size()=1 2024-04-22T20:48:17.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.kxFstAGpxQgbno_vqQu_jwH9tws9ql__0 2024-04-22T20:48:17.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17641755916873965443', time=2024-04-22T22:26:22.685072+0000, chain.objs.size()=1 2024-04-22T20:48:17.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2J7qOVJhAX3jwFSm0jwj8z__tpDPhPE_0 2024-04-22T20:48:17.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15648135147734181750', time=2024-04-22T22:26:22.785836+0000, chain.objs.size()=1 2024-04-22T20:48:17.733 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WdBEbXRG6Fi93RyEZ36_vohWD4ThZl6_0 2024-04-22T20:48:17.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2642637346759976973', time=2024-04-22T22:26:22.985542+0000, chain.objs.size()=1 2024-04-22T20:48:17.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.9MbuE_yVvsRXSqWhhBXgQSKOVsK9p_c_0 2024-04-22T20:48:17.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.867651781968151230', time=2024-04-22T22:26:24.054430+0000, chain.objs.size()=1 2024-04-22T20:48:17.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aQcySdeVDTPw3U5V0ESNlGKY1vO4Khs_0 2024-04-22T20:48:17.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8601481518865913984', time=2024-04-22T22:26:24.165443+0000, chain.objs.size()=1 2024-04-22T20:48:17.734 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.733+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5q58qq-3QfdXR66YQdXlZoauzScWFPI_0 2024-04-22T20:48:17.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.736+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11768395600927985487', time=2024-04-22T22:26:24.268536+0000, chain.objs.size()=1 2024-04-22T20:48:17.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.736+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vfQhyNYaEScS_3SiCFUOrBkyccbaevr_0 2024-04-22T20:48:17.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.736+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11050820419236639093', time=2024-04-22T22:26:24.423326+0000, chain.objs.size()=1 2024-04-22T20:48:17.735 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.736+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tayfNVxkBO0cGFFnIs1JQ8qVcpbhlyu_0 2024-04-22T20:48:17.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.736+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10541286779207920349', time=2024-04-22T22:26:24.737623+0000, chain.objs.size()=1 2024-04-22T20:48:17.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.736+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PfUb0UNKXpTlU3tjJZgAGszYyLh2jvQ_0 2024-04-22T20:48:17.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.736+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10678954675186407079', time=2024-04-22T22:26:24.738259+0000, chain.objs.size()=1 2024-04-22T20:48:17.736 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.736+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.h_KLfQgVKFxxwAPULcD177EsGhXaaSL_0 2024-04-22T20:48:17.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1393122164594587620', time=2024-04-22T22:26:25.664860+0000, chain.objs.size()=1 2024-04-22T20:48:17.737 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2X-s6zhq7orWWZL2HtFE_OWYyroiGD__0 2024-04-22T20:48:17.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4116902951268655902', time=2024-04-22T22:26:25.685962+0000, chain.objs.size()=1 2024-04-22T20:48:17.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.n5mtGxQUdbS8a9rV2BeKeByNMGZomN6_0 2024-04-22T20:48:17.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12111891616294576484', time=2024-04-22T22:26:25.686667+0000, chain.objs.size()=1 2024-04-22T20:48:17.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.zRs6cMqNd-jfP7I1riG7AV1-Cad0Qfo_0 2024-04-22T20:48:17.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18074207489173045839', time=2024-04-22T22:26:26.301087+0000, chain.objs.size()=1 2024-04-22T20:48:17.738 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MF2fLeuwpU6CiUigWJbS8DoCdMoULMa_0 2024-04-22T20:48:17.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13507696187197633151', time=2024-04-22T22:26:27.477691+0000, chain.objs.size()=1 2024-04-22T20:48:17.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Qo2PeNvn1Zud6rq1tzPw8GRAcHjttsg_0 2024-04-22T20:48:17.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17926122109298909282', time=2024-04-22T22:26:28.154606+0000, chain.objs.size()=1 2024-04-22T20:48:17.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.X3GY7ZL80QnmPG_kcA4IzYHSUPd6i3g_0 2024-04-22T20:48:17.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13426610433440710328', time=2024-04-22T22:26:28.192894+0000, chain.objs.size()=1 2024-04-22T20:48:17.739 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HntgZhR8nf6JOjGlfTIZrM7YUQkpUUQ_0 2024-04-22T20:48:17.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1069374473199975629', time=2024-04-22T22:26:28.640469+0000, chain.objs.size()=1 2024-04-22T20:48:17.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.uQkiq7viyRi2_bUs4jA9yTjpJ5nuq0M_0 2024-04-22T20:48:17.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6426106965122832052', time=2024-04-22T22:26:31.540196+0000, chain.objs.size()=1 2024-04-22T20:48:17.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.739+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Rgtv6fg2AlfT0HxQLwEqM7Eq3o66S3K_0 2024-04-22T20:48:17.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.740+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4414642026302838040', time=2024-04-22T22:26:32.525385+0000, chain.objs.size()=1 2024-04-22T20:48:17.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.740+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.U_FA2ZBrAKmek9irlE-uSE4qMBylcaj_0 2024-04-22T20:48:17.740 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.740+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5618665797976880591', time=2024-04-22T22:26:34.058856+0000, chain.objs.size()=1 2024-04-22T20:48:17.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.740+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.yWrCe6EY9EGxt8TKrMXwuTqNnGPrVAf_0 2024-04-22T20:48:17.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.741+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11898579180799755018', time=2024-04-22T22:26:44.966008+0000, chain.objs.size()=1 2024-04-22T20:48:17.741 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.741+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.262__shadow_.-IwO27arkgGSiCx0FtKI9RjJyjeIpzj_0 2024-04-22T20:48:17.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.743+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9555509969569905173', time=2024-04-22T22:27:07.974298+0000, chain.objs.size()=1 2024-04-22T20:48:17.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.743+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.JBjA255ei9C0Ms5VCDmtLf6IE_VUZ4o_0 2024-04-22T20:48:17.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.743+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16248228246211322386', time=2024-04-22T22:27:10.975575+0000, chain.objs.size()=1 2024-04-22T20:48:17.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.743+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.285__shadow_.jyQBOry4Ue25mo0e05y5FhxeXo57rDl_0 2024-04-22T20:48:17.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.743+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11508447165117773271', time=2024-04-22T22:27:17.069059+0000, chain.objs.size()=1 2024-04-22T20:48:17.742 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.743+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.291__shadow_.a3bNmC2b4vt6x2qk18Yoq9xI50chNOT_0 2024-04-22T20:48:17.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.743+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14504915811265151588', time=2024-04-22T22:27:38.409250+0000, chain.objs.size()=1 2024-04-22T20:48:17.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.743+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.310__shadow_.QLCsHuws7QNKUkFtGf10DamgYrHmCUR_0 2024-04-22T20:48:17.743 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9927622006152090851', time=2024-04-22T22:27:38.417032+0000, chain.objs.size()=1 2024-04-22T20:48:17.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.310__shadow_._yVMljBPXQOSOyB_btoG9-o1YGef4UW_0 2024-04-22T20:48:17.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2611030132293226410', time=2024-04-22T22:27:39.276495+0000, chain.objs.size()=1 2024-04-22T20:48:17.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.311__shadow_.ZYn9pXJo0ggp5GbwXE5OOMwmHQjg_HJ_0 2024-04-22T20:48:17.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18095906709931670337', time=2024-04-22T22:28:17.361684+0000, chain.objs.size()=1 2024-04-22T20:48:17.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.H4nz_khaoht7j9xLo640fSAiKIplvrF_0 2024-04-22T20:48:17.744 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8841106248760224033', time=2024-04-22T22:28:17.552915+0000, chain.objs.size()=1 2024-04-22T20:48:17.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.e-mMBjL-WFHQKj4C5M40bKQ3qPeU7Ri_0 2024-04-22T20:48:17.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3975453575429757513', time=2024-04-22T22:28:19.085386+0000, chain.objs.size()=1 2024-04-22T20:48:17.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XbJK7vbU5oso5OW3qHDHcGo9Bz7oZlh_0 2024-04-22T20:48:17.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5196375497736870339', time=2024-04-22T22:28:19.423102+0000, chain.objs.size()=1 2024-04-22T20:48:17.745 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.746+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0RhNB_zSV-T07GaI3GZCnbjjqfHMoo2_0 2024-04-22T20:48:17.746 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15783215529006699338', time=2024-04-22T22:28:19.594332+0000, chain.objs.size()=1 2024-04-22T20:48:17.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.88THgkBFZb_Ja2sCA4zEamezFehBjur_0 2024-04-22T20:48:17.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14988017959881704372', time=2024-04-22T22:28:19.754882+0000, chain.objs.size()=1 2024-04-22T20:48:17.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.0bfUgWum3FD_G3N1gPZJLo20nA3VqvM_0 2024-04-22T20:48:17.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18057468751793602660', time=2024-04-22T22:28:19.755543+0000, chain.objs.size()=1 2024-04-22T20:48:17.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4XiwHqAJt4iASlPfUvdLvuGodISZ5Ug_0 2024-04-22T20:48:17.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17030375102892572178', time=2024-04-22T22:28:19.960351+0000, chain.objs.size()=1 2024-04-22T20:48:17.747 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.US4Q0RF5E3xnbpfMXrEYIPq7A7Ug8d8_0 2024-04-22T20:48:17.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8116028142870735195', time=2024-04-22T22:28:20.092954+0000, chain.objs.size()=1 2024-04-22T20:48:17.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.J9hdLUIo0fhYv6-6ysGEl1zBxpOYO7d_0 2024-04-22T20:48:17.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15116295265878758104', time=2024-04-22T22:28:20.717938+0000, chain.objs.size()=1 2024-04-22T20:48:17.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Nz4Q7JTrHNa8SGUd0lyDzImd4sS4zNN_0 2024-04-22T20:48:17.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14179956004042268324', time=2024-04-22T22:28:20.896902+0000, chain.objs.size()=1 2024-04-22T20:48:17.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.enqtfiqVr1QSPXEV31C3fHA-CoTH7-C_0 2024-04-22T20:48:17.748 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8652031439689653809', time=2024-04-22T22:28:20.897658+0000, chain.objs.size()=1 2024-04-22T20:48:17.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.748+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.sTCMUH4Kz7p7OyMnMqQ3pvULTzp8NvM_0 2024-04-22T20:48:17.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.750+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5592972722448045452', time=2024-04-22T22:28:21.556254+0000, chain.objs.size()=1 2024-04-22T20:48:17.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.750+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UKGZvOzxKJkG3ooDEwczBaByHPtjwm9_0 2024-04-22T20:48:17.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.750+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16047284092613775401', time=2024-04-22T22:28:21.645808+0000, chain.objs.size()=1 2024-04-22T20:48:17.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.750+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.N9ujJEWPQLj_mkVeAzklHH7SePtsxYb_0 2024-04-22T20:48:17.749 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.750+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4124104183865658641', time=2024-04-22T22:28:22.350832+0000, chain.objs.size()=1 2024-04-22T20:48:17.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.750+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.50fO96fZUgJ_aGnhw7nVk4QYlkxZf-o_0 2024-04-22T20:48:17.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.750+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11849473164118180568', time=2024-04-22T22:28:22.890514+0000, chain.objs.size()=1 2024-04-22T20:48:17.750 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.750+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Wp8AUOqqsxfzbrQ0eJrSLfWM-0JWF28_0 2024-04-22T20:48:17.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11555304022969769159', time=2024-04-22T22:28:23.028608+0000, chain.objs.size()=1 2024-04-22T20:48:17.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RVlY-HIO98M26SO0E69KytoBu8bNcfK_0 2024-04-22T20:48:17.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8629889883230204924', time=2024-04-22T22:28:23.354734+0000, chain.objs.size()=1 2024-04-22T20:48:17.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Z6QSDdiutFdXOwXaEZcZ91b8Jyk5XLC_0 2024-04-22T20:48:17.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3350960450101608416', time=2024-04-22T22:28:23.408328+0000, chain.objs.size()=1 2024-04-22T20:48:17.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.IbBbAdrV1pFKNomRjFjF1l1HEmwSa1I_0 2024-04-22T20:48:17.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2767411068045255864', time=2024-04-22T22:28:23.883917+0000, chain.objs.size()=1 2024-04-22T20:48:17.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XHry8vbBCJGcfuaVYOh0ZSRiRLXyhEz_0 2024-04-22T20:48:17.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15869843064450474656', time=2024-04-22T22:28:24.292279+0000, chain.objs.size()=1 2024-04-22T20:48:17.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.752+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gJAljlUIiyLldUrA5eWQvm0nCyiB-zk_0 2024-04-22T20:48:17.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.753+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.441904768361253315', time=2024-04-22T22:28:24.486827+0000, chain.objs.size()=1 2024-04-22T20:48:17.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.753+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.jxd0jgeKT-Nw6XM1_dPBba3FU8A4Hcw_0 2024-04-22T20:48:17.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.753+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8111910632876923555', time=2024-04-22T22:28:24.541993+0000, chain.objs.size()=1 2024-04-22T20:48:17.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.753+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OAeULlObtxEC-F4AjX43V6PEQGv351i_0 2024-04-22T20:48:17.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.753+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3280545993318858684', time=2024-04-22T22:28:24.809079+0000, chain.objs.size()=1 2024-04-22T20:48:17.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.753+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6sVt1q0YqtzFZSKcZZ6nnZIZQmN58oj_0 2024-04-22T20:48:17.753 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.754+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15349801101256960991', time=2024-04-22T22:28:24.951752+0000, chain.objs.size()=1 2024-04-22T20:48:17.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.754+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.VuP3Ism0uJFUrkSNVuP_Ufs9-2-vCHH_0 2024-04-22T20:48:17.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.754+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16164671666480545082', time=2024-04-22T22:28:25.404922+0000, chain.objs.size()=1 2024-04-22T20:48:17.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.754+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.hFOmcLtnFUhSkaLDr17H4_I-KpdIko-_0 2024-04-22T20:48:17.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.754+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.791716653428018799', time=2024-04-22T22:28:25.867559+0000, chain.objs.size()=1 2024-04-22T20:48:17.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.754+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KKY5a7ClaYh443ATQK4xPBw0kchxq8y_0 2024-04-22T20:48:17.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.755+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7377243250264039741', time=2024-04-22T22:28:26.369689+0000, chain.objs.size()=1 2024-04-22T20:48:17.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.755+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.tGpwZilAfikBt6-dbpb03NKvGrvP533_0 2024-04-22T20:48:17.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4563694348321011413', time=2024-04-22T22:28:26.505528+0000, chain.objs.size()=1 2024-04-22T20:48:17.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.dVXJ68Rm1GVOaWbQtp0yEBUthHPnBDO_0 2024-04-22T20:48:17.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16124088353242295504', time=2024-04-22T22:28:26.560509+0000, chain.objs.size()=1 2024-04-22T20:48:17.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.67cv7vao3VHSMu6S4-a2KaxamMNOEq-_0 2024-04-22T20:48:17.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10113819542575552743', time=2024-04-22T22:28:27.333451+0000, chain.objs.size()=1 2024-04-22T20:48:17.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.f3p9SCRwrjWW9Zz0tzMzujmvwdF-Xm7_0 2024-04-22T20:48:17.756 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11612999736518800269', time=2024-04-22T22:28:27.638466+0000, chain.objs.size()=1 2024-04-22T20:48:17.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xcc04LBYmREQB5Mrn_RH0QrWmfbKNjR_0 2024-04-22T20:48:17.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4588185137131165466', time=2024-04-22T22:28:28.062862+0000, chain.objs.size()=1 2024-04-22T20:48:17.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.756+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._7cNTlIRXH6nIQw4ioVeVH_RcfdQ95M_0 2024-04-22T20:48:17.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5199457025594598472', time=2024-04-22T22:28:28.342923+0000, chain.objs.size()=1 2024-04-22T20:48:17.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.B5DZHppjezGP3okY0Jw8KJQ5SouiQJb_0 2024-04-22T20:48:17.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.142190832582870658', time=2024-04-22T22:28:28.343749+0000, chain.objs.size()=1 2024-04-22T20:48:17.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wsASbJ4cT1hhk5VGtBbTYVVGXyYeh_q_0 2024-04-22T20:48:17.757 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15341029021471015141', time=2024-04-22T22:28:28.746883+0000, chain.objs.size()=1 2024-04-22T20:48:17.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.G5-QfGyySL8OG8BeC0LdkECRtZSf0YW_0 2024-04-22T20:48:17.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5939140400178355943', time=2024-04-22T22:28:28.939264+0000, chain.objs.size()=1 2024-04-22T20:48:17.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XNjLnH9Qkjs3FHk683wKbIL-WnB3rCT_0 2024-04-22T20:48:17.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7595316153093333753', time=2024-04-22T22:28:29.006741+0000, chain.objs.size()=1 2024-04-22T20:48:17.758 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.758+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UclukER9W4ze8-P_ZChwjGweHcv9Bpy_0 2024-04-22T20:48:17.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10442126922897267692', time=2024-04-22T22:28:29.258072+0000, chain.objs.size()=1 2024-04-22T20:48:17.759 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.g16WuQ1scAkV6XEwgyOSUz805qPukzl_0 2024-04-22T20:48:17.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6087573224150800176', time=2024-04-22T22:28:29.258810+0000, chain.objs.size()=1 2024-04-22T20:48:17.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._X40xErzUDmcPvQY9yJoIVmTuaKUZBc_0 2024-04-22T20:48:17.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17470211970052481943', time=2024-04-22T22:29:06.668830+0000, chain.objs.size()=1 2024-04-22T20:48:17.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hPP-DxL4OAr3K8Zsr8RBQQ86m_i7IJ5_0 2024-04-22T20:48:17.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10301073258376858607', time=2024-04-22T22:29:06.855433+0000, chain.objs.size()=1 2024-04-22T20:48:17.760 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.j3ey2MfibTz52FBSNSym-vssGGvDxQG_0 2024-04-22T20:48:17.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5644806908672775991', time=2024-04-22T22:29:06.950610+0000, chain.objs.size()=1 2024-04-22T20:48:17.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.SW2goG9O0MrFTO8jmroJpEPt5IQaTLn_0 2024-04-22T20:48:17.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.186343925086858349', time=2024-04-22T22:29:07.053176+0000, chain.objs.size()=1 2024-04-22T20:48:17.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.760+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.tr-Rwy1CUg5lGtcJ_pnZh8oDkI9UIpd_0 2024-04-22T20:48:17.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2710581220236423413', time=2024-04-22T22:29:07.200504+0000, chain.objs.size()=1 2024-04-22T20:48:17.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.B3AQzMMDnK-CFB15BozLQkwwEejVz99_0 2024-04-22T20:48:17.761 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5614209676554124464', time=2024-04-22T22:29:08.391504+0000, chain.objs.size()=1 2024-04-22T20:48:17.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.eLbxKkU7Kzylv1bJTbu_JLuPyuHL-___0 2024-04-22T20:48:17.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13993221611833043703', time=2024-04-22T22:29:08.602430+0000, chain.objs.size()=1 2024-04-22T20:48:17.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-ZKCdTggED6xwemqbZqv-C_i-NQJFan_0 2024-04-22T20:48:17.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5393545556728080338', time=2024-04-22T22:29:08.728038+0000, chain.objs.size()=1 2024-04-22T20:48:17.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.n8TBzVAdJn0H0UB-GqXpoQeD3rLgf64_0 2024-04-22T20:48:17.762 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10516977915898642191', time=2024-04-22T22:29:09.412963+0000, chain.objs.size()=1 2024-04-22T20:48:17.763 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.762+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.H1_pgPSwPw4zFTKx56OIM5i82p7B8zl_0 2024-04-22T20:48:17.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2491171352345022067', time=2024-04-22T22:29:10.005653+0000, chain.objs.size()=1 2024-04-22T20:48:17.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8rTNgddacWgIufekVi9OVMM-5psmZEe_0 2024-04-22T20:48:17.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8918251051420612676', time=2024-04-22T22:29:10.006501+0000, chain.objs.size()=1 2024-04-22T20:48:17.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ByIH8IFdOXydVJVGfb5EfAV2AMjbqAW_0 2024-04-22T20:48:17.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3811199574448973376', time=2024-04-22T22:29:10.209240+0000, chain.objs.size()=1 2024-04-22T20:48:17.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RPtQCjY-Y6smdKQg_pe4uzWYYoYtb_c_0 2024-04-22T20:48:17.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15508599173324019473', time=2024-04-22T22:29:10.375060+0000, chain.objs.size()=1 2024-04-22T20:48:17.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.g0HuR2dggJ_pwM8r8Q1V9vdQ4yV3qZl_0 2024-04-22T20:48:17.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8631228871237722178', time=2024-04-22T22:29:10.969099+0000, chain.objs.size()=1 2024-04-22T20:48:17.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.xBkcvQvX3WZT_ccGHBrUnPMBCnRCTi4_0 2024-04-22T20:48:17.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6152108876968088146', time=2024-04-22T22:29:11.072719+0000, chain.objs.size()=1 2024-04-22T20:48:17.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.u2tqvHyO-Z_GrreDa43y7TPB6P6ZZcA_0 2024-04-22T20:48:17.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8354939445197019143', time=2024-04-22T22:29:11.922271+0000, chain.objs.size()=1 2024-04-22T20:48:17.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4YnLYKQmFWCxMAR62n5xtY80oX6Bmfr_0 2024-04-22T20:48:17.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11058080325841001892', time=2024-04-22T22:29:12.457451+0000, chain.objs.size()=1 2024-04-22T20:48:17.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.unBWzx4JWwA_3J23YkstVCxdOQLd0-y_0 2024-04-22T20:48:17.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11906347059308804502', time=2024-04-22T22:29:12.610076+0000, chain.objs.size()=1 2024-04-22T20:48:17.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mCY3mfdAPmEagWuI0-XISsbzIVI6p1k_0 2024-04-22T20:48:17.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16007490593534681940', time=2024-04-22T22:29:13.041541+0000, chain.objs.size()=1 2024-04-22T20:48:17.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.765+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hsE_TPG2k9Tb36IAARRssp6CFTr63BS_0 2024-04-22T20:48:17.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.766+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3367914351993955830', time=2024-04-22T22:29:13.238812+0000, chain.objs.size()=1 2024-04-22T20:48:17.767 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.766+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._Dx0yjQGBT2wDmzt_QTrZ919r9wZWpq_0 2024-04-22T20:48:17.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.769+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13485650044183311056', time=2024-04-22T22:29:13.520677+0000, chain.objs.size()=1 2024-04-22T20:48:17.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.769+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.FRxV_8FXe0_eNSZXV2hJDWD_NmSmm9w_0 2024-04-22T20:48:17.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.769+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2359156238056785690', time=2024-04-22T22:29:14.035110+0000, chain.objs.size()=1 2024-04-22T20:48:17.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.769+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.3It92QouTZJKy-T1drVX4oXtR56TV_9_0 2024-04-22T20:48:17.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.769+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3561853189460844072', time=2024-04-22T22:29:14.232697+0000, chain.objs.size()=1 2024-04-22T20:48:17.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.769+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.O8HxbD9-kkZFW-t5yF8uBf-ihuZhQ7l_0 2024-04-22T20:48:17.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.769+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17728065544940646841', time=2024-04-22T22:29:14.314120+0000, chain.objs.size()=1 2024-04-22T20:48:17.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.769+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.B4uqvdY5i-3oT1L2KFydJIMLLYv9Kuc_0 2024-04-22T20:48:17.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.770+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14486251006499300503', time=2024-04-22T22:29:14.663710+0000, chain.objs.size()=1 2024-04-22T20:48:17.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.770+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.X6kGbUfkfgEwgRXnP80bFkpOcbXbEte_0 2024-04-22T20:48:17.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.770+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10765981141051850748', time=2024-04-22T22:29:14.687644+0000, chain.objs.size()=1 2024-04-22T20:48:17.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.770+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IFSfA4QLg0_gFyXB9HPJW53jq-vOxNh_0 2024-04-22T20:48:17.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.773+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/43364 2024-04-22T20:48:17.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=23, truncated=0, next_marker='' 2024-04-22T20:48:17.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.733483918735673273', time=2024-04-22T22:29:14.992593+0000, chain.objs.size()=1 2024-04-22T20:48:17.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mGuO9kubVskvB4qQKxY0QzrjbksKiHn_0 2024-04-22T20:48:17.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18059364818598682156', time=2024-04-22T22:29:15.992244+0000, chain.objs.size()=1 2024-04-22T20:48:17.773 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hcJjA0MAfynKRYw6RgQEkh9dOO10EYy_0 2024-04-22T20:48:17.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11233780456991955109', time=2024-04-22T22:29:16.271109+0000, chain.objs.size()=1 2024-04-22T20:48:17.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.n80sP-eyFz0Ndl9nMjWi6VORS2PaRXt_0 2024-04-22T20:48:17.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14127186134690111643', time=2024-04-22T22:29:16.759888+0000, chain.objs.size()=1 2024-04-22T20:48:17.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.1Q4Wvk31szqWs_9Cr6axdmE3Ayfbk2a_0 2024-04-22T20:48:17.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9409817807150752130', time=2024-04-22T22:29:16.902523+0000, chain.objs.size()=1 2024-04-22T20:48:17.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.jjoV4hhJxoV9q20CWLebAW5gfFcVflC_0 2024-04-22T20:48:17.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.775+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2039933473999460752', time=2024-04-22T22:29:17.334361+0000, chain.objs.size()=1 2024-04-22T20:48:17.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.bn-pq3vxErxoBs1JY0FCO-ILT8HvjqG_0 2024-04-22T20:48:17.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8678510382271829257', time=2024-04-22T22:30:57.288958+0000, chain.objs.size()=1 2024-04-22T20:48:17.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.451__shadow_.UfX1in2CE0dr9B7tzLnWtuR3eBxBMdp_0 2024-04-22T20:48:17.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7618260617022217502', time=2024-04-22T22:30:59.036337+0000, chain.objs.size()=1 2024-04-22T20:48:17.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.453__shadow_.OUfd764mpp0NkjH0pOq3dZCnIhlsvGg_0 2024-04-22T20:48:17.775 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B', time=2024-04-22T22:34:39.438935+0000, chain.objs.size()=1 2024-04-22T20:48:17.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B.1 2024-04-22T20:48:17.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B', time=2024-04-22T22:34:39.442300+0000, chain.objs.size()=1 2024-04-22T20:48:17.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B.2 2024-04-22T20:48:17.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B', time=2024-04-22T22:34:39.445963+0000, chain.objs.size()=1 2024-04-22T20:48:17.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B.3 2024-04-22T20:48:17.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B', time=2024-04-22T22:34:39.449517+0000, chain.objs.size()=1 2024-04-22T20:48:17.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.776+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B.4 2024-04-22T20:48:17.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.778+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B', time=2024-04-22T22:34:39.452964+0000, chain.objs.size()=1 2024-04-22T20:48:17.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.778+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B.5 2024-04-22T20:48:17.779 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.778+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B', time=2024-04-22T22:34:39.456464+0000, chain.objs.size()=1 2024-04-22T20:48:17.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.778+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~04wbBQkFMqca1jd0bvqKeLWMyjGvv8B.6 2024-04-22T20:48:17.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2456078798330362157', time=2024-04-22T22:37:08.124846+0000, chain.objs.size()=1 2024-04-22T20:48:17.780 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.547__shadow_.LyDmKGwrQX30uJW0ESUNaKOFPjcxYCf_0 2024-04-22T20:48:17.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10247254025813278970', time=2024-04-22T22:37:17.600893+0000, chain.objs.size()=3 2024-04-22T20:48:17.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.551__shadow_.BcR6mz_Yt0EmgGyA9yxK3zDYs1dYIaB_0 2024-04-22T20:48:17.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.551__shadow_.BcR6mz_Yt0EmgGyA9yxK3zDYs1dYIaB_1 2024-04-22T20:48:17.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.551__shadow_.BcR6mz_Yt0EmgGyA9yxK3zDYs1dYIaB_2 2024-04-22T20:48:17.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13675444364690707536', time=2024-04-22T22:37:51.350138+0000, chain.objs.size()=1 2024-04-22T20:48:17.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:VIi-4MRQCxYNrTp0VzzUhXXZpPwCdUM_.7hLm_2vpY-atpgbU5PtG205yiu5dF_e_0 2024-04-22T20:48:17.781 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.859380265210312409', time=2024-04-22T22:38:59.109854+0000, chain.objs.size()=1 2024-04-22T20:48:17.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:c53agXwrhnbXfW0cWu27vMrtx84FKiq_.RskwHs1qJxdVHKIbrPAVdTg7myykPdQ_0 2024-04-22T20:48:17.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2734030209331766782', time=2024-04-22T22:38:59.421968+0000, chain.objs.size()=1 2024-04-22T20:48:17.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:ekvTgO1iP2w9DTOnfK0M0NncDS0h8ZL_._N-nXC1AVpJtEXhqMRUg7cPqTuPXJOV_0 2024-04-22T20:48:17.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5519345604987406048', time=2024-04-22T22:39:12.693193+0000, chain.objs.size()=1 2024-04-22T20:48:17.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.596__shadow_.beemEXjtcCj4V8hQEoWEn9Kbq0LajcU_0 2024-04-22T20:48:17.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17876409546146854541', time=2024-04-22T22:43:10.637132+0000, chain.objs.size()=1 2024-04-22T20:48:17.782 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.779+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.676__shadow:5htvMh9ps.4ovPw9wmZ4DH0hsRSosdJ_.npzymK_9gqipnylcKqXyvhr7YMd1ZLp_0 2024-04-22T20:48:17.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.782+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12651937980004103493', time=2024-04-22T22:43:16.298041+0000, chain.objs.size()=1 2024-04-22T20:48:17.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.782+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.685__shadow:wYsQUj4XB2fDUol7voZ-Drm8QN70eKi_.VDBbvT-ga1TBFXffLrEhS1ANILNtIyU_0 2024-04-22T20:48:17.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.782+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1643499089952652529', time=2024-04-22T22:43:21.925536+0000, chain.objs.size()=1 2024-04-22T20:48:17.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.782+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.693__shadow_.2uVdsWk-tlknVO2Zn7hu-XISF3Y5EtV_0 2024-04-22T20:48:17.783 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.785+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.784 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.787+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-22T20:48:17.785 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.786 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.789+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/49111' 2024-04-22T20:48:17.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='_ggx9jKWTVmmYfLLUB_OhApBIW7ZKQkD', time=2024-04-22T22:21:43.657618+0000, chain.objs.size()=1 2024-04-22T20:48:17.787 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.149__shadow_.kjvAmCPyJEn9KSFoyN2QgLUoEg21CcF_0 2024-04-22T20:48:17.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17959057748218465598', time=2024-04-22T22:25:15.295986+0000, chain.objs.size()=1 2024-04-22T20:48:17.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.222__shadow_.jX9RMIDaaaCqsqlAN1Iu2E2SLt2GVIV_0 2024-04-22T20:48:17.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18017068601016506711', time=2024-04-22T22:25:16.306192+0000, chain.objs.size()=1 2024-04-22T20:48:17.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.223__shadow_.IVtvxik5-XDgCNvQmWAkG3XkAZKlauB_0 2024-04-22T20:48:17.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17270043270278545058', time=2024-04-22T22:25:29.863608+0000, chain.objs.size()=1 2024-04-22T20:48:17.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.234__shadow_.D1hG0MLy5w_andCLLCdaLVAETdiHsnv_0 2024-04-22T20:48:17.788 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11859152232130499574', time=2024-04-22T22:25:32.813289+0000, chain.objs.size()=1 2024-04-22T20:48:17.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.237__shadow_.-kOFDGDEFdb62374fDYSgx004GlBpps_0 2024-04-22T20:48:17.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3898201236684184895', time=2024-04-22T22:26:22.527621+0000, chain.objs.size()=1 2024-04-22T20:48:17.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.AWp19XY084QxGFlSmCGEWxP2Ayu-Uee_0 2024-04-22T20:48:17.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2954839188800868023', time=2024-04-22T22:26:22.698143+0000, chain.objs.size()=1 2024-04-22T20:48:17.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pIfbGW32_e6TSw51Hx0czsbuRYzI-1t_0 2024-04-22T20:48:17.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13135834411728255087', time=2024-04-22T22:26:23.229458+0000, chain.objs.size()=1 2024-04-22T20:48:17.789 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.vxY2IcUe20w0GQ_6L6yG_auyNkT_sK0_0 2024-04-22T20:48:17.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2247013775131771107', time=2024-04-22T22:26:24.084804+0000, chain.objs.size()=1 2024-04-22T20:48:17.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.aR6vbdKq18C4v9zxi1HsjKp1KlJP7pW_0 2024-04-22T20:48:17.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8261699622713025395', time=2024-04-22T22:26:24.157976+0000, chain.objs.size()=1 2024-04-22T20:48:17.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xI-CwKcXRGTRZC8a3La4OqmAmjJjqwq_0 2024-04-22T20:48:17.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1583343758234631145', time=2024-04-22T22:26:24.319841+0000, chain.objs.size()=1 2024-04-22T20:48:17.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xSNMDFUYo2qu_CWiLYmwiurqCVwq540_0 2024-04-22T20:48:17.790 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10518957868366416773', time=2024-04-22T22:26:24.665389+0000, chain.objs.size()=1 2024-04-22T20:48:17.791 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.790+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.JmLeiwtXsD9EKx27Z6CpuAvor8ZG7sd_0 2024-04-22T20:48:17.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.792+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1139271706646087152', time=2024-04-22T22:26:24.907161+0000, chain.objs.size()=1 2024-04-22T20:48:17.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.792+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.l-EMaAypbg_cvJf3XuMm8HyQrCDB0lZ_0 2024-04-22T20:48:17.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8983278569889173025', time=2024-04-22T22:26:24.907907+0000, chain.objs.size()=1 2024-04-22T20:48:17.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2zBO8HZpB-MF6Oj808q9ZG-wtfjjK74_0 2024-04-22T20:48:17.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5748311575015187620', time=2024-04-22T22:26:25.643463+0000, chain.objs.size()=1 2024-04-22T20:48:17.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MJtTGMaE7PiT2ZqU7jYsFf64EvBmWKV_0 2024-04-22T20:48:17.792 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14873937921115354256', time=2024-04-22T22:26:26.162941+0000, chain.objs.size()=1 2024-04-22T20:48:17.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2Q5eX1so8SuFYJs8nKpHB5OFJx0CJKR_0 2024-04-22T20:48:17.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4843888688253197254', time=2024-04-22T22:26:26.460353+0000, chain.objs.size()=1 2024-04-22T20:48:17.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.37Shan9RTsMWZDf8OQDpw-9gEZ8Uyte_0 2024-04-22T20:48:17.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4770300728315834658', time=2024-04-22T22:26:26.993058+0000, chain.objs.size()=1 2024-04-22T20:48:17.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.hhyt3O4tpCGIq2Co0o4iFAV4yOZHZ-K_0 2024-04-22T20:48:17.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17507490460483384095', time=2024-04-22T22:26:26.995231+0000, chain.objs.size()=1 2024-04-22T20:48:17.793 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.7NBiyMzsOvAG0gsccgVCccc3af_zNW6_0 2024-04-22T20:48:17.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5425129387062398184', time=2024-04-22T22:26:27.486130+0000, chain.objs.size()=1 2024-04-22T20:48:17.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.793+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.0ddnR-73IJ_7tGhYOuJkuYjcr3ezBGK_0 2024-04-22T20:48:17.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.794+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10274688505164568545', time=2024-04-22T22:26:27.830721+0000, chain.objs.size()=1 2024-04-22T20:48:17.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.794+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.P_rpkILOcYZ6B2tK9ArfL16fNqhxRDJ_0 2024-04-22T20:48:17.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.794+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16141826679067645199', time=2024-04-22T22:26:27.854391+0000, chain.objs.size()=1 2024-04-22T20:48:17.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.794+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.WQ9G2JqzWEMo1685Kn485dLTrWsgRo-_0 2024-04-22T20:48:17.794 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.794+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5522944867286880204', time=2024-04-22T22:26:28.214904+0000, chain.objs.size()=1 2024-04-22T20:48:17.795 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.794+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.xwfcFRSbwUJTUDeIKuoSHYO87YVmq84_0 2024-04-22T20:48:17.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.796+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11040340318522780692', time=2024-04-22T22:26:28.275357+0000, chain.objs.size()=1 2024-04-22T20:48:17.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.796+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.BOGa6go6Utsl_Bao7PB5EiSkPps4SuR_0 2024-04-22T20:48:17.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4249660422103537549', time=2024-04-22T22:26:29.176704+0000, chain.objs.size()=1 2024-04-22T20:48:17.796 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Aj-vxAuT2MbGLGwLSw1SsR6XO-KWQa0_0 2024-04-22T20:48:17.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3423155472525056686', time=2024-04-22T22:26:29.750125+0000, chain.objs.size()=1 2024-04-22T20:48:17.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dkP2-3bDx-9oFv-K3unBMgzRAUa5iDw_0 2024-04-22T20:48:17.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17044074212295642513', time=2024-04-22T22:26:30.766855+0000, chain.objs.size()=1 2024-04-22T20:48:17.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MGkarxgxJNCfOxl0hgpvvtxonY09I1Y_0 2024-04-22T20:48:17.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17560076417896745856', time=2024-04-22T22:26:30.839585+0000, chain.objs.size()=1 2024-04-22T20:48:17.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.5UeCsJm-m-LjZswYCjZeGOYhG9EXfSi_0 2024-04-22T20:48:17.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11101448685171905264', time=2024-04-22T22:26:30.965363+0000, chain.objs.size()=1 2024-04-22T20:48:17.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.797+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ezNMP_SmWgVkpeju-uQHlT4o98ECCCo_0 2024-04-22T20:48:17.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12852568014415372456', time=2024-04-22T22:26:31.644063+0000, chain.objs.size()=1 2024-04-22T20:48:17.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ELPGEJ-qTpfjM2z4t61-gFbSLMzdoEO_0 2024-04-22T20:48:17.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.126017748970281744', time=2024-04-22T22:26:32.248716+0000, chain.objs.size()=1 2024-04-22T20:48:17.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.t-DU4-JpttWh0aRQz88lYrEoFRoiRCg_0 2024-04-22T20:48:17.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9156161749533259033', time=2024-04-22T22:26:32.385689+0000, chain.objs.size()=1 2024-04-22T20:48:17.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.kBsGcGN2YPH9dDl_JVDAcB1RQ2A7YxT_0 2024-04-22T20:48:17.798 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14141298290035344704', time=2024-04-22T22:26:32.388055+0000, chain.objs.size()=1 2024-04-22T20:48:17.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-oWsXAw2hAYVf023Awqkd5jCiBvpgMJ_0 2024-04-22T20:48:17.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2688752159452893276', time=2024-04-22T22:26:32.521221+0000, chain.objs.size()=1 2024-04-22T20:48:17.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.lCO04dqt5xD83WIrStEyaFp7K_i3StG_0 2024-04-22T20:48:17.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8618346378946978659', time=2024-04-22T22:26:32.827627+0000, chain.objs.size()=1 2024-04-22T20:48:17.799 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.798+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Zs8nlbihYFNbhWhNLdqChPBikg6Unwu_0 2024-04-22T20:48:17.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.801+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4962269524029538913', time=2024-04-22T22:26:32.828149+0000, chain.objs.size()=1 2024-04-22T20:48:17.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.801+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.pwVwIB3tbNe5HbBXH8GFUoBXHZXAqg6_0 2024-04-22T20:48:17.800 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.801+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5833568502654758906', time=2024-04-22T22:26:33.545313+0000, chain.objs.size()=1 2024-04-22T20:48:17.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.801+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MXHYVye3ncZN7t40NFYa80vlBC9E_Dt_0 2024-04-22T20:48:17.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7088259360629739702', time=2024-04-22T22:26:33.593864+0000, chain.objs.size()=1 2024-04-22T20:48:17.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oppGecLaVKm_5Ek8umqukQNMv7W0Uje_0 2024-04-22T20:48:17.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.80828450990146628', time=2024-04-22T22:26:33.628613+0000, chain.objs.size()=1 2024-04-22T20:48:17.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.2mb_rzmKULKIB9a2ZThj8BCwcUpaKvS_0 2024-04-22T20:48:17.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4934684526371498568', time=2024-04-22T22:26:33.684958+0000, chain.objs.size()=1 2024-04-22T20:48:17.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.-mT9s9vtYn-PTGQY0XWRGpz01uGxeul_0 2024-04-22T20:48:17.801 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11647253247073936296', time=2024-04-22T22:26:33.733883+0000, chain.objs.size()=1 2024-04-22T20:48:17.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.6RZB3TLZKfbp3vm9XnmVYe-GolG94xr_0 2024-04-22T20:48:17.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15049356914391986625', time=2024-04-22T22:26:34.390156+0000, chain.objs.size()=1 2024-04-22T20:48:17.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.O1D0tizIAjFDcOpU2erJmN0_ai-CxGy_0 2024-04-22T20:48:17.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11511287713359651656', time=2024-04-22T22:27:08.969692+0000, chain.objs.size()=1 2024-04-22T20:48:17.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.283__shadow_.eM6zdJUAZXCl0lUPicdVnASHdzA7cpS_0 2024-04-22T20:48:17.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16118815960454802300', time=2024-04-22T22:27:13.050342+0000, chain.objs.size()=1 2024-04-22T20:48:17.802 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.287__shadow_.EK6w8HWP2M6hujqu8xVF4Bb0VqJdzLI_0 2024-04-22T20:48:17.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2212929534857341446', time=2024-04-22T22:27:16.068540+0000, chain.objs.size()=1 2024-04-22T20:48:17.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.802+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.290__shadow_.mgPntk1Z5VA6WazIK9m2maQ_VMJiWn__0 2024-04-22T20:48:17.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11996807421348822709', time=2024-04-22T22:28:17.368950+0000, chain.objs.size()=1 2024-04-22T20:48:17.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bsY4iFJr9MIG8nVj5O_DfgCG3pTTkTC_0 2024-04-22T20:48:17.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12153601939918128139', time=2024-04-22T22:28:17.415037+0000, chain.objs.size()=1 2024-04-22T20:48:17.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OvPVHjYex09ZcQr4-_9d5Yovk_-qKLN_0 2024-04-22T20:48:17.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.805+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11076034977977000895', time=2024-04-22T22:28:18.078823+0000, chain.objs.size()=1 2024-04-22T20:48:17.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.805+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bSs-l8Azh-wgNyupjJgMmo8vTrzM2od_0 2024-04-22T20:48:17.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.806+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2377821379980207896', time=2024-04-22T22:28:18.330281+0000, chain.objs.size()=1 2024-04-22T20:48:17.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.806+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.puGEygy_Hg9YQpkRdGbrzWYMLrdfE5a_0 2024-04-22T20:48:17.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.806+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3568135081408851193', time=2024-04-22T22:28:19.093867+0000, chain.objs.size()=1 2024-04-22T20:48:17.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.806+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.UdshxuwN1Jc9nuDN8dj4Ry7AD2rrAua_0 2024-04-22T20:48:17.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.806+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16948760409036522772', time=2024-04-22T22:28:19.470214+0000, chain.objs.size()=1 2024-04-22T20:48:17.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.806+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.hWLAp6FO7G39H1kWqdS43YQQj-1HcvQ_0 2024-04-22T20:48:17.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2892476151454512336', time=2024-04-22T22:28:19.594779+0000, chain.objs.size()=1 2024-04-22T20:48:17.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XSkFsckTNl8ablx0mqUarQfxnpLbuLQ_0 2024-04-22T20:48:17.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16983334964466897746', time=2024-04-22T22:28:19.834150+0000, chain.objs.size()=1 2024-04-22T20:48:17.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.q_xUDClLHsTWAks6w33LWJ2eLdZnJ27_0 2024-04-22T20:48:17.806 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9228038140871994335', time=2024-04-22T22:28:19.901293+0000, chain.objs.size()=1 2024-04-22T20:48:17.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4_G6n3WkX1X7xEwHCJar8bGX-cUodXI_0 2024-04-22T20:48:17.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.807+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11202671103997758507', time=2024-04-22T22:28:20.097141+0000, chain.objs.size()=1 2024-04-22T20:48:17.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.807+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.xtWAN2xTOZQG-1zSOifRtMnTPnLxdNz_0 2024-04-22T20:48:17.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8703784083174683485', time=2024-04-22T22:28:20.143078+0000, chain.objs.size()=1 2024-04-22T20:48:17.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._8YEg1plxdiqYeTyclPbXxmeQ2iZqyt_0 2024-04-22T20:48:17.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13623727077653136052', time=2024-04-22T22:28:21.677786+0000, chain.objs.size()=1 2024-04-22T20:48:17.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.R41e5FDcKVpKURSztG4Tm5rt8fRcoAm_0 2024-04-22T20:48:17.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.808+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3311623844628927035', time=2024-04-22T22:28:21.768772+0000, chain.objs.size()=1 2024-04-22T20:48:17.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.808+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6KhlEYxmEXyysPfmxSpQvOiVjCGgZUn_0 2024-04-22T20:48:17.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17682835186751850365', time=2024-04-22T22:28:22.351606+0000, chain.objs.size()=1 2024-04-22T20:48:17.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.c2BDKoEmHZPJT6yfabM_2KHKYUa_uIP_0 2024-04-22T20:48:17.808 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6032694187228595097', time=2024-04-22T22:28:23.254125+0000, chain.objs.size()=1 2024-04-22T20:48:17.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ycQzw8xVGRd2zXFTewI0JZqm-r68xgH_0 2024-04-22T20:48:17.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5859992323195289944', time=2024-04-22T22:28:24.129547+0000, chain.objs.size()=1 2024-04-22T20:48:17.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.oU_14QaQOKGME3h7sP7JeAkRqmjo9yV_0 2024-04-22T20:48:17.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.810+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12092202213292526439', time=2024-04-22T22:28:24.379987+0000, chain.objs.size()=1 2024-04-22T20:48:17.809 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.810+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.msfxoC7VakRLlFxMgJtF-mT6fc6zrAc_0 2024-04-22T20:48:17.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9042351585001336974', time=2024-04-22T22:28:24.619820+0000, chain.objs.size()=1 2024-04-22T20:48:17.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.6bYUe-ZZOq9C2kkz3tac7Iqedpk2MrI_0 2024-04-22T20:48:17.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8899036527341251780', time=2024-04-22T22:28:25.029923+0000, chain.objs.size()=1 2024-04-22T20:48:17.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.1I4p6dBb-xvgw7HlMY64QhKV6tB0sMf_0 2024-04-22T20:48:17.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11875460303326579968', time=2024-04-22T22:28:25.410183+0000, chain.objs.size()=1 2024-04-22T20:48:17.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.EjxGVn_MRILZIQtL4j06VLblzaHkWt5_0 2024-04-22T20:48:17.811 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3265429922314306312', time=2024-04-22T22:28:25.549988+0000, chain.objs.size()=1 2024-04-22T20:48:17.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XSM-zoMfnZI_86kYeGUxlboS8OxdoSv_0 2024-04-22T20:48:17.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1375758264063220287', time=2024-04-22T22:28:25.639210+0000, chain.objs.size()=1 2024-04-22T20:48:17.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._ZwsQaA1VWMih-OhxAAzXTRtN9QKqu2_0 2024-04-22T20:48:17.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2129260531330332333', time=2024-04-22T22:28:25.720298+0000, chain.objs.size()=1 2024-04-22T20:48:17.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.RJ_dnhj5wP1qY2_JymqfuGbSWO2_6Nb_0 2024-04-22T20:48:17.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6547717166983087270', time=2024-04-22T22:28:27.181504+0000, chain.objs.size()=1 2024-04-22T20:48:17.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MX0OQ5y4JIqQ47-lv_mKAsnWFWFDZ99_0 2024-04-22T20:48:17.812 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11116142488867366534', time=2024-04-22T22:28:27.423557+0000, chain.objs.size()=1 2024-04-22T20:48:17.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.812+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.SkmRuDrvqX72b-UC_7QDdkKTWaqqUkC_0 2024-04-22T20:48:17.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.814+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3518426102435052509', time=2024-04-22T22:28:27.569485+0000, chain.objs.size()=1 2024-04-22T20:48:17.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.814+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9gDaP5JFAyf04kyaTKTtGjL3d0Lbd_I_0 2024-04-22T20:48:17.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.814+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17785191726337022374', time=2024-04-22T22:28:27.651603+0000, chain.objs.size()=1 2024-04-22T20:48:17.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.814+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.WNVwlXhwP8BZpShjPkvYKnK2nRxf2WT_0 2024-04-22T20:48:17.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.814+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9697655262258152707', time=2024-04-22T22:28:27.993384+0000, chain.objs.size()=1 2024-04-22T20:48:17.813 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.814+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.G79uyPlb4o5sMAibR2cl5NeKZP2QSb1_0 2024-04-22T20:48:17.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.816+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17392391827892101607', time=2024-04-22T22:28:28.748908+0000, chain.objs.size()=1 2024-04-22T20:48:17.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.816+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-6F9lGHVuPitnhQYV4i872YLGmt5JCS_0 2024-04-22T20:48:17.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.816+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14339080356590670167', time=2024-04-22T22:28:29.192149+0000, chain.objs.size()=1 2024-04-22T20:48:17.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.816+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.v7KgJMPMOgODJ6FX0PfYX8gmBa4FbsK_0 2024-04-22T20:48:17.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.816+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6772815059194689958', time=2024-04-22T22:29:06.331950+0000, chain.objs.size()=1 2024-04-22T20:48:17.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.816+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KcmE34fGXw8qTFLXeiQyAC6DLMvEQhF_0 2024-04-22T20:48:17.815 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.816+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8047355113532944882', time=2024-04-22T22:29:06.365867+0000, chain.objs.size()=1 2024-04-22T20:48:17.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.816+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.2zqRjLaxFf3UoSTXRKTJPgcyvWz1kEE_0 2024-04-22T20:48:17.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6577911545196726009', time=2024-04-22T22:29:06.495692+0000, chain.objs.size()=1 2024-04-22T20:48:17.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6jWwW_OhDoX7Khvy2ywaFrc-iI92LF2_0 2024-04-22T20:48:17.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10260598071786434385', time=2024-04-22T22:29:08.141981+0000, chain.objs.size()=1 2024-04-22T20:48:17.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.eOid5AQBPLn7C_i-54cOZlp7E9OeMNq_0 2024-04-22T20:48:17.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7694818149176175176', time=2024-04-22T22:29:08.142823+0000, chain.objs.size()=1 2024-04-22T20:48:17.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.BS0NJY-qcG0dzElrv7MmNJGgWjGxCVn_0 2024-04-22T20:48:17.816 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15574532501041221087', time=2024-04-22T22:29:09.555868+0000, chain.objs.size()=1 2024-04-22T20:48:17.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IFYgMUtP1lbFNqjtHB745IRzK6AemUu_0 2024-04-22T20:48:17.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3457594683886864982', time=2024-04-22T22:29:10.216302+0000, chain.objs.size()=1 2024-04-22T20:48:17.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.817+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Mw6YG12qFO3WhwE8axESHdCBtiNQgnA_0 2024-04-22T20:48:17.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.818+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15802433538479278647', time=2024-04-22T22:29:10.386148+0000, chain.objs.size()=1 2024-04-22T20:48:17.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.818+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QJZlUPm2g2rPeyQgyywEaERrzktYqDO_0 2024-04-22T20:48:17.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.818+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2149322686723545465', time=2024-04-22T22:29:11.491419+0000, chain.objs.size()=1 2024-04-22T20:48:17.817 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.818+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.CCeqPUXrrcpOGzYJNcH-4ZcOrjejX_u_0 2024-04-22T20:48:17.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9151585056697552415', time=2024-04-22T22:29:11.537309+0000, chain.objs.size()=1 2024-04-22T20:48:17.818 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.VELOJ8xGB-F0-oQ9KMuOwJLjFt4LJmD_0 2024-04-22T20:48:17.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.945628889231642242', time=2024-04-22T22:29:11.824275+0000, chain.objs.size()=1 2024-04-22T20:48:17.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MaUF4UFa8G4jZIxK-cJodi8cXH9UyzQ_0 2024-04-22T20:48:17.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8888243707611427519', time=2024-04-22T22:29:12.001277+0000, chain.objs.size()=1 2024-04-22T20:48:17.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7WM40RqUQwO88PRM1Yeb4x4kcU9w-Jl_0 2024-04-22T20:48:17.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14950345759602718306', time=2024-04-22T22:29:12.427655+0000, chain.objs.size()=1 2024-04-22T20:48:17.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.sTsRMvlDZ7ie9VvQbaHoKnX2Q8nwFuZ_0 2024-04-22T20:48:17.819 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18382496110790718686', time=2024-04-22T22:29:12.784206+0000, chain.objs.size()=1 2024-04-22T20:48:17.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.FB4Uj95Jt_QhnLSS4nlmyhJPLWbUAwF_0 2024-04-22T20:48:17.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13105961222272385646', time=2024-04-22T22:29:13.019363+0000, chain.objs.size()=1 2024-04-22T20:48:17.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.820+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.E4uL8VJ-Ni00LWQ5vTx6zbxkEwBF87X_0 2024-04-22T20:48:17.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.821+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7654765917094987016', time=2024-04-22T22:29:13.419014+0000, chain.objs.size()=1 2024-04-22T20:48:17.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.821+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.EuhtHWXB0to7eaIuepBKJXUN-ehxr5__0 2024-04-22T20:48:17.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.821+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17455608636256104657', time=2024-04-22T22:29:13.907305+0000, chain.objs.size()=1 2024-04-22T20:48:17.820 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.821+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XLWMYJdcC10sPfcBx7aVSay6hEvBHN6_0 2024-04-22T20:48:17.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.821+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8632026427136011360', time=2024-04-22T22:29:14.170915+0000, chain.objs.size()=1 2024-04-22T20:48:17.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.821+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hGq28SIIWZ_mOL-d2PNG_OU8EEZbqJf_0 2024-04-22T20:48:17.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.821+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8354659411137931137', time=2024-04-22T22:29:14.558341+0000, chain.objs.size()=1 2024-04-22T20:48:17.821 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.821+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qI8R83XIs0wTorU8B_qmneB2ADyh-Xl_0 2024-04-22T20:48:17.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10578038446975159382', time=2024-04-22T22:29:14.737012+0000, chain.objs.size()=1 2024-04-22T20:48:17.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.oyU0TQV2SAtmLMT8Bkqh1fmp2STj3nD_0 2024-04-22T20:48:17.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6220041729472209790', time=2024-04-22T22:29:14.901067+0000, chain.objs.size()=1 2024-04-22T20:48:17.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.P0kfmrRwmlMB1GUDg6avULf1PqYOVVn_0 2024-04-22T20:48:17.822 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15846406640380298596', time=2024-04-22T22:29:14.960902+0000, chain.objs.size()=1 2024-04-22T20:48:17.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.urGmkivm2VAXYhin9pbQ5UXtZVKvpmc_0 2024-04-22T20:48:17.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.823+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9226916531450314747', time=2024-04-22T22:29:14.999424+0000, chain.objs.size()=1 2024-04-22T20:48:17.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.823+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.U2mP5HQ8FTqCrDXC-ml14K52cSH2mPb_0 2024-04-22T20:48:17.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.824+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8520427369873157962', time=2024-04-22T22:29:15.507435+0000, chain.objs.size()=1 2024-04-22T20:48:17.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.824+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.q03OKkJKo7hvKVSHwRePX1roUiFaB4H_0 2024-04-22T20:48:17.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.825+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15853668260690022889', time=2024-04-22T22:29:16.978794+0000, chain.objs.size()=1 2024-04-22T20:48:17.823 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.825+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OL6v52NaIvOVMT-y1DPsRBU2sn-1-u9_0 2024-04-22T20:48:17.824 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.827+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/49111 2024-04-22T20:48:17.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=14, truncated=0, next_marker='' 2024-04-22T20:48:17.827 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13124160940717613262', time=2024-04-22T22:29:17.231226+0000, chain.objs.size()=1 2024-04-22T20:48:17.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6-kGmj9x-PeB4WhqFH2bJZaA0X3OXYn_0 2024-04-22T20:48:17.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6791925216899501895', time=2024-04-22T22:29:17.372775+0000, chain.objs.size()=1 2024-04-22T20:48:17.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-lCo7lmQ7wyGy9u4f21ZIyoFsAd7pk5_0 2024-04-22T20:48:17.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18442404370434121967', time=2024-04-22T22:29:17.502990+0000, chain.objs.size()=1 2024-04-22T20:48:17.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.JzKzaDJgNYnij2gZvXZpSOddyNfvte8_0 2024-04-22T20:48:17.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14310038644950262302', time=2024-04-22T22:29:17.620060+0000, chain.objs.size()=1 2024-04-22T20:48:17.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TkQvI5u5gSFxrsI67kwaOsMYDdx5TGG_0 2024-04-22T20:48:17.828 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16403404475011660223', time=2024-04-22T22:29:37.125104+0000, chain.objs.size()=1 2024-04-22T20:48:17.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.342__shadow_.5IbYR4E1hVK_F6_UImLb54EO0Ea0rXt_0 2024-04-22T20:48:17.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.989794224635712486', time=2024-04-22T22:30:29.362137+0000, chain.objs.size()=1 2024-04-22T20:48:17.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.419__shadow_.YLxUm0O4FMzrLeuobqfz6xOCImCdFhk_0 2024-04-22T20:48:17.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12827955316299485693', time=2024-04-22T22:30:32.549622+0000, chain.objs.size()=1 2024-04-22T20:48:17.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.424__shadow_.Qk6MsEcybP-DoOTTo4zYRq1U4Ch7DZP_0 2024-04-22T20:48:17.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15012591423164314825', time=2024-04-22T22:36:55.604616+0000, chain.objs.size()=1 2024-04-22T20:48:17.829 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.544__shadow_.WrV2DRSV61hepoYIU-4_xlZZtT7jvFo_0 2024-04-22T20:48:17.830 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5552315215216645177', time=2024-04-22T22:37:10.804218+0000, chain.objs.size()=1 2024-04-22T20:48:17.830 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.550__shadow_.QWi3_39lENBQXJBXQ_aUMP6L_HVhdz3_0 2024-04-22T20:48:17.830 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4560983047393913639', time=2024-04-22T22:37:21.999517+0000, chain.objs.size()=1 2024-04-22T20:48:17.830 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:UaTQTLj2RH.nD9iwxiKcVjzV1gpBO7y_.NzC8jyA10ciSZ-CVjwjQibM3M_DrUEO_0 2024-04-22T20:48:17.830 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5300845491302978128', time=2024-04-22T22:37:31.917422+0000, chain.objs.size()=1 2024-04-22T20:48:17.830 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.557__shadow:n6.fPG9ipVy-NATyDxmKfD9Tu7h8oY9_.HYF30g1byCg03VywjdjKuBe8ztP9iEf_0 2024-04-22T20:48:17.830 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.650__shadow_.IpZmspcWjK9KvmU6Q4Cr0LQd0LrIB6y_1_af5ae20d862ae1ef89aaf9b306d97f16', time=2024-04-22T22:42:56.081911+0000, chain.objs.size()=1 2024-04-22T20:48:17.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.830+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.650__shadow_.IpZmspcWjK9KvmU6Q4Cr0LQd0LrIB6y_0 2024-04-22T20:48:17.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.832+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5406765105657457178', time=2024-04-22T22:43:22.601518+0000, chain.objs.size()=1 2024-04-22T20:48:17.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.832+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.694__shadow:vUTXW55g7UozmUQBEtBgHmhU.A23PAy_.mNII9zbC75CsODTzYQMZi5KC5w4Qetn_0 2024-04-22T20:48:17.831 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.833+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18399257444169928331', time=2024-04-22T22:44:33.793075+0000, chain.objs.size()=1 2024-04-22T20:48:17.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.833+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.723__shadow_.At_IdvO9PkZ_bxk8bHAadS4HrHzIUh5_0 2024-04-22T20:48:17.832 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.835+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.834 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.837+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-22T20:48:17.835 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.839+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.836 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.839+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43082' 2024-04-22T20:48:17.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5391658155087328697', time=2024-04-22T22:22:33.695890+0000, chain.objs.size()=1 2024-04-22T20:48:17.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.160__shadow_.h5V2fpdBtpK4oAXFJ9sy2MrwicCFiDN_0 2024-04-22T20:48:17.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7865375785547036253', time=2024-04-22T22:25:21.709622+0000, chain.objs.size()=1 2024-04-22T20:48:17.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.227__shadow_.xTQwzBXtyuqjPHaTvrJ_qKJiPVS0Veu_0 2024-04-22T20:48:17.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3707923215322238342', time=2024-04-22T22:25:25.445425+0000, chain.objs.size()=1 2024-04-22T20:48:17.838 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.231__shadow_.YnGv61sg2Svo2P1L6nTESL8XVrDKzEk_0 2024-04-22T20:48:17.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17371172620729766187', time=2024-04-22T22:25:27.139206+0000, chain.objs.size()=1 2024-04-22T20:48:17.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.232__shadow_._auf0gtmg9cIE1wXFpA6mz97g4IDEnR_0 2024-04-22T20:48:17.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13858646973359725780', time=2024-04-22T22:25:31.823509+0000, chain.objs.size()=1 2024-04-22T20:48:17.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.236__shadow_.p2-GbZnxclGRWlBKJhtJRAparLJYMD3_0 2024-04-22T20:48:17.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14935388745366734638', time=2024-04-22T22:25:37.812563+0000, chain.objs.size()=1 2024-04-22T20:48:17.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.242__shadow_.EVEv1gQ_YnFWbQDhWwpitdGNAXj3nsj_0 2024-04-22T20:48:17.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3576068031835238234', time=2024-04-22T22:26:22.331311+0000, chain.objs.size()=1 2024-04-22T20:48:17.839 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.1IU6VxsvdPQXJDsU27GNpHnMXTu1Hyc_0 2024-04-22T20:48:17.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18436073414543265155', time=2024-04-22T22:26:22.335268+0000, chain.objs.size()=1 2024-04-22T20:48:17.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.o68qztfvDYZYZkks6CPfTdC9Yoo8h0o_0 2024-04-22T20:48:17.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4068437619872279890', time=2024-04-22T22:26:22.618274+0000, chain.objs.size()=1 2024-04-22T20:48:17.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.sD0LwvZ9MEQ6yMB-luOlHgO-7UqcO10_0 2024-04-22T20:48:17.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1931987098109833079', time=2024-04-22T22:26:23.168312+0000, chain.objs.size()=1 2024-04-22T20:48:17.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.iaSHQjOeu6thMGSNHSgVh8qWhQNl0ty_0 2024-04-22T20:48:17.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11053593559586781455', time=2024-04-22T22:26:23.247862+0000, chain.objs.size()=1 2024-04-22T20:48:17.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.HicNo3CtAJ-LxQytwpTfOOTn7eCtviX_0 2024-04-22T20:48:17.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.840+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6544568188283647666', time=2024-04-22T22:26:23.261959+0000, chain.objs.size()=1 2024-04-22T20:48:17.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.841+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.9wESHzOnpRIvUi5LMWIUUIhV9WWKcE9_0 2024-04-22T20:48:17.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.843+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5986146882976930959', time=2024-04-22T22:26:23.943681+0000, chain.objs.size()=1 2024-04-22T20:48:17.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.843+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bEBM-ADQhLX1gCNCPnduv0ZHZRSr-qf_0 2024-04-22T20:48:17.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.843+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10762759445839572111', time=2024-04-22T22:26:26.442270+0000, chain.objs.size()=1 2024-04-22T20:48:17.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.843+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.NJ_gpfsSYsuZrXEvH75IIxIt4m7dmMd_0 2024-04-22T20:48:17.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.844+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1005930923148126221', time=2024-04-22T22:26:26.446654+0000, chain.objs.size()=1 2024-04-22T20:48:17.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.844+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.yPt02IGz0o_SviwE7pzcJ3YU24AA_FC_0 2024-04-22T20:48:17.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.844+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11008906798187686481', time=2024-04-22T22:26:26.463263+0000, chain.objs.size()=1 2024-04-22T20:48:17.842 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.844+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.QSHfNJRbeIoQgfe3wdXhPQ9fKb95K9x_0 2024-04-22T20:48:17.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4862376440196683495', time=2024-04-22T22:26:26.978440+0000, chain.objs.size()=1 2024-04-22T20:48:17.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.MPZYHQWyib-g_Y-rgOn6cnk548kLMvV_0 2024-04-22T20:48:17.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12417537107239531589', time=2024-04-22T22:26:28.407889+0000, chain.objs.size()=1 2024-04-22T20:48:17.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.TaxZD4OTy3aJx-v2m5pLjfgs2hnVW5f_0 2024-04-22T20:48:17.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3144455788613258776', time=2024-04-22T22:26:29.050661+0000, chain.objs.size()=1 2024-04-22T20:48:17.843 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oZwTY8b-H-NwsedXuffaczYx1maGxUJ_0 2024-04-22T20:48:17.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6762642673820775539', time=2024-04-22T22:26:29.051215+0000, chain.objs.size()=1 2024-04-22T20:48:17.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PfusFQ7xD7xu0EanjXwLGhD7uw_m7av_0 2024-04-22T20:48:17.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1275094158847689761', time=2024-04-22T22:26:29.438103+0000, chain.objs.size()=1 2024-04-22T20:48:17.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.PvTUD05rGWceY_msJR1Xd8CMIyAo9M6_0 2024-04-22T20:48:17.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14296345638743479393', time=2024-04-22T22:26:29.456070+0000, chain.objs.size()=1 2024-04-22T20:48:17.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.o0FucD4sTABvGXTbF7nnPfrQUh3ej7G_0 2024-04-22T20:48:17.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1308158018576793348', time=2024-04-22T22:26:29.493570+0000, chain.objs.size()=1 2024-04-22T20:48:17.844 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.845+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.UTiFdrFZrBLm2vChtL2BRVbMfexPkDM_0 2024-04-22T20:48:17.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.847+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16476199360884179048', time=2024-04-22T22:26:29.850281+0000, chain.objs.size()=1 2024-04-22T20:48:17.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.847+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.VnTWYNJCVZ4A9VskPzyd8gg4rJ3lHNl_0 2024-04-22T20:48:17.845 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.847+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7102471965308661838', time=2024-04-22T22:26:29.877332+0000, chain.objs.size()=1 2024-04-22T20:48:17.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.847+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.f-XAmNvp5HBAgFWgsuz1hI08SzQCB-d_0 2024-04-22T20:48:17.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.848+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.271662535381941154', time=2024-04-22T22:26:30.117856+0000, chain.objs.size()=1 2024-04-22T20:48:17.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.848+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oojDipnjbABsozAWdJV0HHpwKMfCHcA_0 2024-04-22T20:48:17.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.848+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3629884695988276083', time=2024-04-22T22:26:30.124559+0000, chain.objs.size()=1 2024-04-22T20:48:17.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.848+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.h10t_q6LJFPrqOWixaP3yC7qe8Z20N3_0 2024-04-22T20:48:17.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.848+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7708601419121998581', time=2024-04-22T22:26:30.171745+0000, chain.objs.size()=1 2024-04-22T20:48:17.846 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.848+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.K4cG8QJ1gBC-zQYpET5SFiquc_YMpXd_0 2024-04-22T20:48:17.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.849+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16053289304898402102', time=2024-04-22T22:26:33.729397+0000, chain.objs.size()=1 2024-04-22T20:48:17.847 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.849+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CkRA6wnHLaS4dF4Fp6Kh-_w5ViDD3Xu_0 2024-04-22T20:48:17.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.849+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.137647945699295166', time=2024-04-22T22:26:34.089688+0000, chain.objs.size()=1 2024-04-22T20:48:17.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.849+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.XVt45tRFVHyugzsLQZqVkfMsPQOz03h_0 2024-04-22T20:48:17.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.849+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7160827981559529541', time=2024-04-22T22:26:55.832217+0000, chain.objs.size()=1 2024-04-22T20:48:17.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.849+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.273__shadow_.D4OiFFL9rPrdPn2WTcFrO9ZOGb9OSYc_0 2024-04-22T20:48:17.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11369781682163785415', time=2024-04-22T22:27:07.944136+0000, chain.objs.size()=1 2024-04-22T20:48:17.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.hqFtPcYNP-FdzRo-QYRuz1MS-X2Z2Rr_0 2024-04-22T20:48:17.848 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16341063625639968126', time=2024-04-22T22:27:07.975092+0000, chain.objs.size()=1 2024-04-22T20:48:17.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.282__shadow_.p7OtvlY1ist0vgLS5_C8_s6zFE0b8Il_0 2024-04-22T20:48:17.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.850+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17077271595012928634', time=2024-04-22T22:27:13.048581+0000, chain.objs.size()=1 2024-04-22T20:48:17.849 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.850+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.287__shadow_.JYBVLjpWaJk7XwXaMICSDa-SgZt1YZh_0 2024-04-22T20:48:17.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.851+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14105592461431251642', time=2024-04-22T22:28:17.352789+0000, chain.objs.size()=1 2024-04-22T20:48:17.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.851+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4hZHEzNJQMst8ci2vt_tN52gxELs1FY_0 2024-04-22T20:48:17.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.852+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6128322732449124230', time=2024-04-22T22:28:18.068197+0000, chain.objs.size()=1 2024-04-22T20:48:17.850 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.852+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wptiXmo1cLXv9zAaixllOQfOPo-v-45_0 2024-04-22T20:48:17.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.852+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17200499824647096179', time=2024-04-22T22:28:18.390796+0000, chain.objs.size()=1 2024-04-22T20:48:17.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.852+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ngLmmdCK_PaHZuV5OS16MYY8KEIMYlj_0 2024-04-22T20:48:17.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.852+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8614821787385772572', time=2024-04-22T22:28:19.239846+0000, chain.objs.size()=1 2024-04-22T20:48:17.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.852+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.DUUACZ5cNCAQ5MmyIXXFAjo9iW2ysc4_0 2024-04-22T20:48:17.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.852+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16840279642691635608', time=2024-04-22T22:28:19.598276+0000, chain.objs.size()=1 2024-04-22T20:48:17.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.852+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kowX8vlGlrBclM-JKwXtbr6cAZ111Xf_0 2024-04-22T20:48:17.851 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.853+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7012755037860516307', time=2024-04-22T22:28:19.797512+0000, chain.objs.size()=1 2024-04-22T20:48:17.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.853+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Z6FRVEYSV5c04mDKd9NOEuFhXRGDgY5_0 2024-04-22T20:48:17.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.853+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18123768619183140026', time=2024-04-22T22:28:19.941972+0000, chain.objs.size()=1 2024-04-22T20:48:17.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.853+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rEbJZWGXslzc5pn7K6WmNju61Js2zgh_0 2024-04-22T20:48:17.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.853+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4692306013696112693', time=2024-04-22T22:28:20.686999+0000, chain.objs.size()=1 2024-04-22T20:48:17.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.853+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bfmAwn8cvcmzQ0eWW6yS1mUE__lTxCg_0 2024-04-22T20:48:17.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.854+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2211479639188912201', time=2024-04-22T22:28:20.688407+0000, chain.objs.size()=1 2024-04-22T20:48:17.852 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.854+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-467COyqu17DYEM-E750VEotlDVIsNJ_0 2024-04-22T20:48:17.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.855+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10476337114073265803', time=2024-04-22T22:28:20.847122+0000, chain.objs.size()=1 2024-04-22T20:48:17.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.855+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.plNS-EVNKXELvnTd6uxvR1Giyzq3qWr_0 2024-04-22T20:48:17.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.855+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5675560418694393234', time=2024-04-22T22:28:21.696914+0000, chain.objs.size()=1 2024-04-22T20:48:17.853 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.855+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ND8dX4BnPsIoqYke6jH5QeHhrXviOKz_0 2024-04-22T20:48:17.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15124714646632787787', time=2024-04-22T22:28:21.803901+0000, chain.objs.size()=1 2024-04-22T20:48:17.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.etna0J1pCsG6-LWxRCZTXUzNT3uuOKO_0 2024-04-22T20:48:17.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13291345624829272294', time=2024-04-22T22:28:23.865958+0000, chain.objs.size()=1 2024-04-22T20:48:17.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.b6pWAoiOMgc7M4idLKUWPIhuHrBXlKs_0 2024-04-22T20:48:17.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13708153747257519872', time=2024-04-22T22:28:24.246748+0000, chain.objs.size()=1 2024-04-22T20:48:17.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.VCS39QvhOiBu1zpgVojd0JewoA24i-6_0 2024-04-22T20:48:17.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11281101058976808147', time=2024-04-22T22:28:24.800497+0000, chain.objs.size()=1 2024-04-22T20:48:17.854 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yqpiRli0g0ojsUkyd2dw6C5Wcg869T9_0 2024-04-22T20:48:17.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3608792255777698059', time=2024-04-22T22:28:25.940070+0000, chain.objs.size()=1 2024-04-22T20:48:17.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.856+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.7nAyhspBGyq4YGpd7YkgAgUm-F8-zI3_0 2024-04-22T20:48:17.855 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12142226189780288422', time=2024-04-22T22:28:26.315995+0000, chain.objs.size()=1 2024-04-22T20:48:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.rZZzNOf8yZYAu93B_XJN65RWMqSx0JX_0 2024-04-22T20:48:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.239881809321072744', time=2024-04-22T22:28:26.410469+0000, chain.objs.size()=1 2024-04-22T20:48:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cOQKPGDTLmC_SnYren7ZblIOKAf0kRH_0 2024-04-22T20:48:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14404487419994343248', time=2024-04-22T22:28:26.411015+0000, chain.objs.size()=1 2024-04-22T20:48:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.b5WH11M0U3wrKpcx0YNRpyWcYVhLCEk_0 2024-04-22T20:48:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.857+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4038310664582863539', time=2024-04-22T22:28:26.559737+0000, chain.objs.size()=1 2024-04-22T20:48:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.857+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.uhHydDCNksZBFBa3jwoZal8f_1K7iYZ_0 2024-04-22T20:48:17.856 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.858+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2111591249792047041', time=2024-04-22T22:28:26.928980+0000, chain.objs.size()=1 2024-04-22T20:48:17.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.858+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.ePUai_HKT5r2s9lGe5dIjVMW8n5dZ2U_0 2024-04-22T20:48:17.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.858+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4365900713180728833', time=2024-04-22T22:28:27.247185+0000, chain.objs.size()=1 2024-04-22T20:48:17.857 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.858+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.yVkvcaM8Kn7R8qHEYDZ-pSW9tuOgkLB_0 2024-04-22T20:48:17.860 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9815270892083045982', time=2024-04-22T22:28:27.248022+0000, chain.objs.size()=1 2024-04-22T20:48:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.VvINi5dEoKo046vTOEswtLQUhMncX2s_0 2024-04-22T20:48:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12426971684472776998', time=2024-04-22T22:28:27.281308+0000, chain.objs.size()=1 2024-04-22T20:48:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.NmxNAiWTxfWgIern3aHJzGvV_8y1bpb_0 2024-04-22T20:48:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14059226054027746412', time=2024-04-22T22:28:27.722460+0000, chain.objs.size()=1 2024-04-22T20:48:17.861 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FXaOD0D6Lj2sNdR8EwSU9R0y2Y_SsJ3_0 2024-04-22T20:48:17.862 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15392635400847674634', time=2024-04-22T22:28:27.722986+0000, chain.objs.size()=1 2024-04-22T20:48:17.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Y6r4YYXfTvWNfwbcSl7LB-BfF2Q_pCt_0 2024-04-22T20:48:17.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13805248981600020710', time=2024-04-22T22:28:27.835120+0000, chain.objs.size()=1 2024-04-22T20:48:17.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MikeZIaCkbvtwwqErcdLpYtRLCVOEW8_0 2024-04-22T20:48:17.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12592818057039544665', time=2024-04-22T22:28:27.882128+0000, chain.objs.size()=1 2024-04-22T20:48:17.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.iW8FhCWEN0FmKFyADRskkRPC96Aw8N4_0 2024-04-22T20:48:17.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11396714103826635867', time=2024-04-22T22:28:27.955884+0000, chain.objs.size()=1 2024-04-22T20:48:17.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.864+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_._EnV4duWuQz00bcyhObbJcJtR_a3ffx_0 2024-04-22T20:48:17.863 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.865+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6765456900149584697', time=2024-04-22T22:28:28.384400+0000, chain.objs.size()=1 2024-04-22T20:48:17.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.865+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.lWdKpei_Npnk49XRZTHUFv1gB26XawZ_0 2024-04-22T20:48:17.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.866+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5175207388200313601', time=2024-04-22T22:28:28.672417+0000, chain.objs.size()=1 2024-04-22T20:48:17.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.866+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.AreFWGOMv7S_fy48L2I_VnCunOeem6X_0 2024-04-22T20:48:17.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.866+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9063237710548394382', time=2024-04-22T22:28:28.793722+0000, chain.objs.size()=1 2024-04-22T20:48:17.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.866+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.3lN7bD3Tzxql29x5giOAhm9trYFayfb_0 2024-04-22T20:48:17.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.866+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.295199627830852495', time=2024-04-22T22:28:28.821738+0000, chain.objs.size()=1 2024-04-22T20:48:17.864 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.866+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4Xhi0mdopdOpx0aFHaYV_-Of3xLyH-a_0 2024-04-22T20:48:17.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.868+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9188000735133916880', time=2024-04-22T22:28:29.252188+0000, chain.objs.size()=1 2024-04-22T20:48:17.865 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.868+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.C7b6me-Nu1R4UbnCwNMTVuyKv1OOBEy_0 2024-04-22T20:48:17.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16143913768185536304', time=2024-04-22T22:29:06.312036+0000, chain.objs.size()=1 2024-04-22T20:48:17.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.7-cYzR1hILWn77f_dWJHPPho3_2TFcr_0 2024-04-22T20:48:17.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16814195455790264685', time=2024-04-22T22:29:06.366109+0000, chain.objs.size()=1 2024-04-22T20:48:17.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gjnpnXCtjz8e_yiM18uO2uM1sU7f5N6_0 2024-04-22T20:48:17.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13740955792321395511', time=2024-04-22T22:29:08.283236+0000, chain.objs.size()=1 2024-04-22T20:48:17.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ywbkmucUQcT94FnSdiKDv7VO8Aq7nkK_0 2024-04-22T20:48:17.866 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5058778533698346199', time=2024-04-22T22:29:08.283731+0000, chain.objs.size()=1 2024-04-22T20:48:17.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ujp7OVSUKOw1UAyVpntJuah42w-1COI_0 2024-04-22T20:48:17.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11942897600773634173', time=2024-04-22T22:29:08.382049+0000, chain.objs.size()=1 2024-04-22T20:48:17.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LgQdcZbi8We7BU_nMQeCn1OLk5khcTD_0 2024-04-22T20:48:17.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18219763983251063801', time=2024-04-22T22:29:08.579450+0000, chain.objs.size()=1 2024-04-22T20:48:17.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QSxrorcJ8sdkkic-QADDUIcImczsaSj_0 2024-04-22T20:48:17.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11134989704469195892', time=2024-04-22T22:29:08.789674+0000, chain.objs.size()=1 2024-04-22T20:48:17.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.QBOu43jBgJzcdNc46MCYt554QzjbjW6_0 2024-04-22T20:48:17.867 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11051269772968943884', time=2024-04-22T22:29:09.521202+0000, chain.objs.size()=1 2024-04-22T20:48:17.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MLLNebqFPqYklrbpvfU_nM-pzzKEg80_0 2024-04-22T20:48:17.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4439415183505089316', time=2024-04-22T22:29:09.651360+0000, chain.objs.size()=1 2024-04-22T20:48:17.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ahaRY-wKp8eCT0IBCW06A2A2K9SWSOH_0 2024-04-22T20:48:17.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8382744763516551376', time=2024-04-22T22:29:09.717114+0000, chain.objs.size()=1 2024-04-22T20:48:17.868 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.869+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.wg2qGhkuABoTkkBT3ck3kAdZb_Q8uru_0 2024-04-22T20:48:17.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.871+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5494123051102840921', time=2024-04-22T22:29:10.093802+0000, chain.objs.size()=1 2024-04-22T20:48:17.869 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.871+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nnfkZeKzCDeE6T0IZ-n1YyaSGqwEv7N_0 2024-04-22T20:48:17.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2465192307065846830', time=2024-04-22T22:29:10.214668+0000, chain.objs.size()=1 2024-04-22T20:48:17.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.CqYwyN-rq2sZLIEH51ZykxaSuZWuCns_0 2024-04-22T20:48:17.870 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17860336469482861384', time=2024-04-22T22:29:11.268531+0000, chain.objs.size()=1 2024-04-22T20:48:17.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.4OmT15MQp-yZcH54MbiV-jQonaciRhm_0 2024-04-22T20:48:17.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14674487734190532982', time=2024-04-22T22:29:11.393887+0000, chain.objs.size()=1 2024-04-22T20:48:17.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.asoeZf4OgRRV0Te1Gs6hNZc0xjpmwj2_0 2024-04-22T20:48:17.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6097202455120106411', time=2024-04-22T22:29:11.527523+0000, chain.objs.size()=1 2024-04-22T20:48:17.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.AxtJ4hMUUZMf5eIk7MxoDjd73Z8X292_0 2024-04-22T20:48:17.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11138855544589902750', time=2024-04-22T22:29:12.572690+0000, chain.objs.size()=1 2024-04-22T20:48:17.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hIV44NK48Dm1aczFfDHpSQmB_gbdkLY_0 2024-04-22T20:48:17.871 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.142633608521287910', time=2024-04-22T22:29:13.193695+0000, chain.objs.size()=1 2024-04-22T20:48:17.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.8pSEi1PYPbJSEAma5ZpeANiLOfm1w3J_0 2024-04-22T20:48:17.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8740302392805197273', time=2024-04-22T22:29:13.518989+0000, chain.objs.size()=1 2024-04-22T20:48:17.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.Uvbh-8A54BB55bS1IoCfF8cSxv57v47_0 2024-04-22T20:48:17.872 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14487404746822741071', time=2024-04-22T22:29:13.918234+0000, chain.objs.size()=1 2024-04-22T20:48:17.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zaZBlGIrQeMcLTrU8No-r_RPQ0kYhqI_0 2024-04-22T20:48:17.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12399884108049680943', time=2024-04-22T22:29:14.415940+0000, chain.objs.size()=1 2024-04-22T20:48:17.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.873+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.-oe06mqiaqlOklXtJz5L5gRe-t-zSo3_0 2024-04-22T20:48:17.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.874+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16818011302100425703', time=2024-04-22T22:29:14.511535+0000, chain.objs.size()=1 2024-04-22T20:48:17.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.874+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.IE2e0Ba6-mcqzzIE2wdzZ-9vra8mY7__0 2024-04-22T20:48:17.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.875+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7165880126700546259', time=2024-04-22T22:29:14.665212+0000, chain.objs.size()=1 2024-04-22T20:48:17.873 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.875+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.E_Vke35by3a-eFDapfr-jwx9TfWQ2RO_0 2024-04-22T20:48:17.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16703600215401709760', time=2024-04-22T22:29:14.666601+0000, chain.objs.size()=1 2024-04-22T20:48:17.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.n-AA51ocicejCusc0r_AmA0mG7FS7US_0 2024-04-22T20:48:17.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17516066952370572064', time=2024-04-22T22:29:14.789853+0000, chain.objs.size()=1 2024-04-22T20:48:17.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.o_P1fS32A1cAst1Bbgjkv6fOvsS6juv_0 2024-04-22T20:48:17.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4035402480814217919', time=2024-04-22T22:29:15.511697+0000, chain.objs.size()=1 2024-04-22T20:48:17.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HN_bMgI8bq_veGLlZRlQY6V3hFCyd-p_0 2024-04-22T20:48:17.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15934001362783145566', time=2024-04-22T22:29:15.965624+0000, chain.objs.size()=1 2024-04-22T20:48:17.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.audNtXDSvilxrsNOurt00jPdsxQcO8M_0 2024-04-22T20:48:17.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8950676504362515264', time=2024-04-22T22:29:15.995240+0000, chain.objs.size()=1 2024-04-22T20:48:17.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.UADjb6Idp7vyzO3ivq7Bx1EcdiIu6Da_0 2024-04-22T20:48:17.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5141191790630380438', time=2024-04-22T22:29:16.187867+0000, chain.objs.size()=1 2024-04-22T20:48:17.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GoQpg4IxifGPJ4D4BRy-h4RICIqu51L_0 2024-04-22T20:48:17.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16867687359892279897', time=2024-04-22T22:29:16.320678+0000, chain.objs.size()=1 2024-04-22T20:48:17.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.NL4JOwQbPyP9OlmR1-8Z3kCr-86n_mM_0 2024-04-22T20:48:17.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15103880696756736467', time=2024-04-22T22:29:16.427269+0000, chain.objs.size()=1 2024-04-22T20:48:17.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.H_SdGeLliROVr2dnthlUUeaE5QTRva7_0 2024-04-22T20:48:17.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2456538374164823228', time=2024-04-22T22:29:16.936457+0000, chain.objs.size()=1 2024-04-22T20:48:17.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.o4WdQzP7vQwiZmL4T4EwiMwP8nEtBDl_0 2024-04-22T20:48:17.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6572695792733680680', time=2024-04-22T22:29:17.326864+0000, chain.objs.size()=1 2024-04-22T20:48:17.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.879+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.GGwTXniC7YFYtsAfGvTt5ISjLAaIJy-_0 2024-04-22T20:48:17.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.883+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/43082 2024-04-22T20:48:17.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=14, truncated=0, next_marker='' 2024-04-22T20:48:17.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12597353631684077692', time=2024-04-22T22:29:17.507331+0000, chain.objs.size()=1 2024-04-22T20:48:17.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_._-xrRjvKJ98sjnNC_uOqSp1YLOuk8Vw_0 2024-04-22T20:48:17.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10061246141597339673', time=2024-04-22T22:30:02.590467+0000, chain.objs.size()=1 2024-04-22T20:48:17.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.379__shadow_.sxSWitSAU9CvfcfOIo2_m3aY598JLih_0 2024-04-22T20:48:17.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.53899370838001044', time=2024-04-22T22:30:37.898091+0000, chain.objs.size()=1 2024-04-22T20:48:17.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.431__shadow_.We1YXSHkPLKkk_MlGYBiHibx8n0D8nM_0 2024-04-22T20:48:17.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7426155835137311122', time=2024-04-22T22:31:15.562951+0000, chain.objs.size()=1 2024-04-22T20:48:17.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.475__shadow_.Cw43RM8kCmoHPhj_R79zmKaFTLJQkNI_0 2024-04-22T20:48:17.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.487__shadow_.myWPrazPOs9LTSWZOKSSgogKObIntxr_1_65160a0dfcaec10112708277b0b316d1', time=2024-04-22T22:31:23.627028+0000, chain.objs.size()=1 2024-04-22T20:48:17.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.487__shadow_.myWPrazPOs9LTSWZOKSSgogKObIntxr_0 2024-04-22T20:48:17.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri', time=2024-04-22T22:33:28.120312+0000, chain.objs.size()=2 2024-04-22T20:48:17.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__multipart_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.2 2024-04-22T20:48:17.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.511__shadow_mymultipart.2~rj0_JoIBNhlOmNlEWGQ-neVXkYh1Bri.2_1 2024-04-22T20:48:17.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2531427985666844026', time=2024-04-22T22:34:53.716826+0000, chain.objs.size()=1 2024-04-22T20:48:17.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.512__multipart_mymultipart.2~625clVVdlvJ5xliT6BuMathbhQGNClZ.1 2024-04-22T20:48:17.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6040527959641561239', time=2024-04-22T22:37:18.886178+0000, chain.objs.size()=1 2024-04-22T20:48:17.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.553__shadow_.YjIf6WDOX2LrnydRiK1kSaMM-tRPnq2_0 2024-04-22T20:48:17.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1977613193387124295', time=2024-04-22T22:38:59.495234+0000, chain.objs.size()=1 2024-04-22T20:48:17.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.575__shadow:vkUq0fzd6cwCq5-utoNrXn0H.VAjAyb_.NGYotXZNPcvf_ujhKBKK2FomEHJeSYd_0 2024-04-22T20:48:17.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.946229014675768063', time=2024-04-22T22:39:08.944488+0000, chain.objs.size()=1 2024-04-22T20:48:17.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.591__shadow_.X9DMLGBA7z8iy37tEwt-6yPJ8zrzK74_0 2024-04-22T20:48:17.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5244823870322185816', time=2024-04-22T22:43:03.676505+0000, chain.objs.size()=1 2024-04-22T20:48:17.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.886+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.663__shadow_.WbJ0P0ZFaRsvhNbvKRm1ocL-Up-CP6n_0 2024-04-22T20:48:17.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.889+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15691082622746285752', time=2024-04-22T22:43:09.123361+0000, chain.objs.size()=1 2024-04-22T20:48:17.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.889+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.674__shadow:u3gpTlzs7e0-aIehQ0P4XMdM6GZrJPY_.rj_yVtJlmx1Q2XtU-nXZsIFS5wl2rmp_0 2024-04-22T20:48:17.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.889+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.702__shadow_.Y124PckKJuyRelgHUVM0UmYeFkBRUy6_1_dbfa75999dbe92fc69a8b7fdbf5eb889', time=2024-04-22T22:43:46.817344+0000, chain.objs.size()=1 2024-04-22T20:48:17.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.889+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.702__shadow_.Y124PckKJuyRelgHUVM0UmYeFkBRUy6_0 2024-04-22T20:48:17.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.889+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17078964302115094036', time=2024-04-22T22:43:47.547365+0000, chain.objs.size()=1 2024-04-22T20:48:17.887 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.889+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.703__shadow_.jfSzKLFSNJUO2rjPebH1vRTtIzB_CX2_0 2024-04-22T20:48:17.888 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.892+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.890 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.894+0000 7f2f5fb46800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-22T20:48:17.891 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.895+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-22T20:48:17.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.895+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-22T20:48:17.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/50808' 2024-04-22T20:48:17.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5733297188157084798', time=2024-04-22T22:25:18.279190+0000, chain.objs.size()=1 2024-04-22T20:48:17.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.225__shadow_.7nBlbxRd6h8BfMLOtytaOkxiUMfIPGL_0 2024-04-22T20:48:17.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18136043396763173628', time=2024-04-22T22:25:19.951198+0000, chain.objs.size()=1 2024-04-22T20:48:17.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.226__shadow_.FKtojIeHWOOjBkW1ArioQS7iMNhPU7e_0 2024-04-22T20:48:17.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8244983925865942453', time=2024-04-22T22:25:28.861258+0000, chain.objs.size()=1 2024-04-22T20:48:17.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.233__shadow_.VaY0pCwLawpVgbZjx7RzAqEbEnHOrnJ_0 2024-04-22T20:48:17.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3121532696608900089', time=2024-04-22T22:25:38.808626+0000, chain.objs.size()=1 2024-04-22T20:48:17.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.243__shadow_.XjtWHo-28i0fbTuBlQVPhTQxrtc9W6v_0 2024-04-22T20:48:17.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14997661948474140429', time=2024-04-22T22:25:40.809919+0000, chain.objs.size()=1 2024-04-22T20:48:17.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.245__shadow_.QpxnVZJlfXwYlu2-J-8uwgXCsFNP2Qf_0 2024-04-22T20:48:17.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12589924794104063295', time=2024-04-22T22:25:41.798523+0000, chain.objs.size()=1 2024-04-22T20:48:17.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.246__shadow_.hjN6Q4w_EZZzqRuz61RMInFkiHlGIR3_0 2024-04-22T20:48:17.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16790347581370762478', time=2024-04-22T22:25:44.667882+0000, chain.objs.size()=1 2024-04-22T20:48:17.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.249__shadow_.efVzoycNl4qifHw5GnQ-WSE1XFAl-Zh_0 2024-04-22T20:48:17.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.178944592015567719', time=2024-04-22T22:26:22.326324+0000, chain.objs.size()=1 2024-04-22T20:48:17.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cPNB0EAFHWyHY9ompH9KL24bN4cU8tp_0 2024-04-22T20:48:17.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14365970309284562091', time=2024-04-22T22:26:22.326954+0000, chain.objs.size()=1 2024-04-22T20:48:17.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.boW2s_F6U6tsCrTDezvx3hWuPDrB15k_0 2024-04-22T20:48:17.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.239070090657296027', time=2024-04-22T22:26:22.763070+0000, chain.objs.size()=1 2024-04-22T20:48:17.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.iUiJ0O51BK9LOO1wo5DfjzGUJVJOlll_0 2024-04-22T20:48:17.896 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14613661577018455305', time=2024-04-22T22:26:23.161817+0000, chain.objs.size()=1 2024-04-22T20:48:17.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oWOZ7T9gy6eDFTGf52wDNG-xAlULnYn_0 2024-04-22T20:48:17.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9620092536682089381', time=2024-04-22T22:26:23.381076+0000, chain.objs.size()=1 2024-04-22T20:48:17.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.896+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.h2N0j_QWyf27HwM6qgGWefURdEiWo1I_0 2024-04-22T20:48:17.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.898+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3791757211309051445', time=2024-04-22T22:26:24.339307+0000, chain.objs.size()=1 2024-04-22T20:48:17.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.899+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.nSA9BWcbkN6XammSFf6kIDJs1Yl1yWM_0 2024-04-22T20:48:17.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15777813276866813224', time=2024-04-22T22:26:24.402670+0000, chain.objs.size()=1 2024-04-22T20:48:17.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.be758D9nh_V5kkJqE8BnrjhA_Ldm2oI_0 2024-04-22T20:48:17.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16506922412906871392', time=2024-04-22T22:26:24.659573+0000, chain.objs.size()=1 2024-04-22T20:48:17.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.CDROdvKTx9D8Qu9HifrNjNHKwUapTWy_0 2024-04-22T20:48:17.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7698582442590461476', time=2024-04-22T22:26:25.972108+0000, chain.objs.size()=1 2024-04-22T20:48:17.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Me_5NTQxbeYFpP6Ys2nO_SSln7Cbu7G_0 2024-04-22T20:48:17.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15830509014503332345', time=2024-04-22T22:26:27.017799+0000, chain.objs.size()=1 2024-04-22T20:48:17.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jd98z7aHs_5Zmvy29PCXk-CqtJ38XXu_0 2024-04-22T20:48:17.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3141066017130508835', time=2024-04-22T22:26:27.475037+0000, chain.objs.size()=1 2024-04-22T20:48:17.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.v2aYS0F6cjv_wTRO0r1kSpWU-TT5so0_0 2024-04-22T20:48:17.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9522909551176078057', time=2024-04-22T22:26:27.505789+0000, chain.objs.size()=1 2024-04-22T20:48:17.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.rBj54KwkqfR5aGn3mJW8rgxFMI0mBRD_0 2024-04-22T20:48:17.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12191867425852297221', time=2024-04-22T22:26:27.606721+0000, chain.objs.size()=1 2024-04-22T20:48:17.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SFs3rna6Dufww-H8jnuSxNIUH9YJ8CW_0 2024-04-22T20:48:17.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9550103617851637476', time=2024-04-22T22:26:28.018671+0000, chain.objs.size()=1 2024-04-22T20:48:17.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jwRRr8MnX3SbgKIVefaeHzCZ1HUfFEq_0 2024-04-22T20:48:17.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10001396644008848637', time=2024-04-22T22:26:28.210863+0000, chain.objs.size()=1 2024-04-22T20:48:17.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.ycg_lWh3qZAOUYqlRUltG7172LOOzxP_0 2024-04-22T20:48:17.900 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6919060445648075302', time=2024-04-22T22:26:28.343285+0000, chain.objs.size()=1 2024-04-22T20:48:17.901 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.900+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.DQzZ3OGf8fgLPOOs5ttD42X8YHcmi4v_0 2024-04-22T20:48:17.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.902+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3962506971660473326', time=2024-04-22T22:26:28.469956+0000, chain.objs.size()=1 2024-04-22T20:48:17.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.902+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SX8jWQNfhcM-aNqHkNgx5rUEi6x2Oc6_0 2024-04-22T20:48:17.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.903+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1266235152538390571', time=2024-04-22T22:26:29.015957+0000, chain.objs.size()=1 2024-04-22T20:48:17.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.903+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.eC9ziuES-I8XJEhLZAScLYAjz-YMRKi_0 2024-04-22T20:48:17.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.903+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16232786317783015894', time=2024-04-22T22:26:29.393850+0000, chain.objs.size()=1 2024-04-22T20:48:17.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.903+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.GkOKrknhEwXzvNCcFp8tn86SkrR1X8a_0 2024-04-22T20:48:17.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.903+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1468365871003819035', time=2024-04-22T22:26:30.141635+0000, chain.objs.size()=1 2024-04-22T20:48:17.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.903+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.oc9ysIZ7V0s9N-YmXRybqlgsafE8w75_0 2024-04-22T20:48:17.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.903+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14894623498037590547', time=2024-04-22T22:26:30.526014+0000, chain.objs.size()=1 2024-04-22T20:48:17.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.903+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.R_N0J9ZDS9Uh_TmYMbvGWDA7BgjWCA5_0 2024-04-22T20:48:17.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16419529093834009027', time=2024-04-22T22:26:30.756403+0000, chain.objs.size()=1 2024-04-22T20:48:17.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.57wLl-WbeEdf0Gf2Mh2rc54btTA2MEU_0 2024-04-22T20:48:17.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11846331216335224086', time=2024-04-22T22:26:30.834843+0000, chain.objs.size()=1 2024-04-22T20:48:17.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SQZakNRezFa7jdNfFnh0ktw7MI1duc__0 2024-04-22T20:48:17.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15001707424860240284', time=2024-04-22T22:26:31.229686+0000, chain.objs.size()=1 2024-04-22T20:48:17.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.tgeUkJ_mYhMDQQIdAcGqVyC2S23mPIS_0 2024-04-22T20:48:17.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11771548320573214217', time=2024-04-22T22:26:31.230498+0000, chain.objs.size()=1 2024-04-22T20:48:17.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.dQoofW6IoBhC_LB8hSd09gd8MhO4tFG_0 2024-04-22T20:48:17.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5694454752603678671', time=2024-04-22T22:26:31.410555+0000, chain.objs.size()=1 2024-04-22T20:48:17.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.L20_6WtS2HmU8q8f6lleDzcJTxZswpf_0 2024-04-22T20:48:17.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4188165619190380180', time=2024-04-22T22:26:31.412652+0000, chain.objs.size()=1 2024-04-22T20:48:17.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.904+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.Lqe8njcLcuh5FXEnkbI3b9EP9vQpRqF_0 2024-04-22T20:48:17.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.906+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8668493046430621438', time=2024-04-22T22:26:31.682945+0000, chain.objs.size()=1 2024-04-22T20:48:17.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.906+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.gX9Z5cOC3yuVQ2IpkC5RZZT7EHGUGFh_0 2024-04-22T20:48:17.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.906+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.905928084862102073', time=2024-04-22T22:26:32.201285+0000, chain.objs.size()=1 2024-04-22T20:48:17.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.906+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.jRoKnoyBl-EpHunGaDbmDg69h99YdsA_0 2024-04-22T20:48:17.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2322687653799261117', time=2024-04-22T22:26:32.678322+0000, chain.objs.size()=1 2024-04-22T20:48:17.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.cO0w-LzJPFkBdtLC9hZ2vcW-4GTa34g_0 2024-04-22T20:48:17.906 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9927606840503891765', time=2024-04-22T22:26:32.750639+0000, chain.objs.size()=1 2024-04-22T20:48:17.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.X8McRY2Sq6_EWBiHvB_Jwwp8EM4XrDB_0 2024-04-22T20:48:17.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9258773032064684068', time=2024-04-22T22:26:33.057717+0000, chain.objs.size()=1 2024-04-22T20:48:17.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.SFx25xCsRFWDEfJrKkZ-hY56nfbPozG_0 2024-04-22T20:48:17.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1431520750459349995', time=2024-04-22T22:26:33.881102+0000, chain.objs.size()=1 2024-04-22T20:48:17.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bEOubTLSrJ1i9xGvs9CdLHR9qwM-gxI_0 2024-04-22T20:48:17.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3078755748165284264', time=2024-04-22T22:26:34.359774+0000, chain.objs.size()=1 2024-04-22T20:48:17.907 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.907+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.q_alDvzqICqj9IaN4Pp8n4fPGqF4yOG_0 2024-04-22T20:48:17.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.908+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12164576893456389042', time=2024-04-22T22:26:34.424573+0000, chain.objs.size()=1 2024-04-22T20:48:17.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.908+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.251__shadow_.bnpHdoct59ofHKqHj8Tt1u9ZTta7rKC_0 2024-04-22T20:48:17.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2826626492502460055', time=2024-04-22T22:26:50.891285+0000, chain.objs.size()=1 2024-04-22T20:48:17.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.268__shadow_.0SBhg9I8qrxNIfJ-UuwM-RhDv1kjwij_0 2024-04-22T20:48:17.908 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5581713040414706285', time=2024-04-22T22:26:53.870654+0000, chain.objs.size()=1 2024-04-22T20:48:17.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.271__shadow_.RtWrs2wCBBy3mnEziQM6G_VqUBPE_Ka_0 2024-04-22T20:48:17.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.932887891450729181', time=2024-04-22T22:26:56.942982+0000, chain.objs.size()=1 2024-04-22T20:48:17.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.274__shadow_.pQkVFEkvVzYR25V2JVZmo4WXGMooXxx_0 2024-04-22T20:48:17.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10752821110097528803', time=2024-04-22T22:27:01.957825+0000, chain.objs.size()=1 2024-04-22T20:48:17.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.279__shadow_.rAGhcQOwDkMopBCGRSW0MV2ZU_V9uvh_0 2024-04-22T20:48:17.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.336086563273165831', time=2024-04-22T22:27:08.967894+0000, chain.objs.size()=1 2024-04-22T20:48:17.909 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.910+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.283__shadow_.bsk0deyKH2M--ngc9XLrhuYN_2vlZga_0 2024-04-22T20:48:17.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17324256324360232165', time=2024-04-22T22:27:18.109582+0000, chain.objs.size()=1 2024-04-22T20:48:17.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.292__shadow_.nPy6o5wYcYQL8nG8a5eEFV1nl9jJLIp_0 2024-04-22T20:48:17.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.417827489353343030', time=2024-04-22T22:28:17.354038+0000, chain.objs.size()=1 2024-04-22T20:48:17.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.kxJTSAXG53N6VorgjHw0A5yZKriPgVx_0 2024-04-22T20:48:17.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10413365152973411548', time=2024-04-22T22:28:17.544828+0000, chain.objs.size()=1 2024-04-22T20:48:17.911 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Etw2jvhFZOJ4jyLFTE7CmZMn_MHXqQW_0 2024-04-22T20:48:17.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2714084517183137552', time=2024-04-22T22:28:17.645482+0000, chain.objs.size()=1 2024-04-22T20:48:17.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.pu4tkLfcKYx4Ppwbpe_BysJdf0TKC4Z_0 2024-04-22T20:48:17.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8238932510109069457', time=2024-04-22T22:28:17.793212+0000, chain.objs.size()=1 2024-04-22T20:48:17.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.S6vC4A1uVt7mUt_bNcNIohstYhRKu2D_0 2024-04-22T20:48:17.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3252822902791671821', time=2024-04-22T22:28:18.528012+0000, chain.objs.size()=1 2024-04-22T20:48:17.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.912+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9-vT_3V8Q-tIubfbiY7mL47CSJAzVP1_0 2024-04-22T20:48:17.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7121345348203527807', time=2024-04-22T22:28:19.084766+0000, chain.objs.size()=1 2024-04-22T20:48:17.912 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.MYG2M2RMXpJjfbCFMIVDdOC7O5hyARO_0 2024-04-22T20:48:17.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7055707918131260510', time=2024-04-22T22:28:19.246140+0000, chain.objs.size()=1 2024-04-22T20:48:17.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.fvN2XHNZvNoQZ3xdiM94PSWT3ZojFTh_0 2024-04-22T20:48:17.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17892210474373423108', time=2024-04-22T22:28:19.945351+0000, chain.objs.size()=1 2024-04-22T20:48:17.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.gSBMrKFzEn6Hjm8k9_qp6iRTMuVm6Xx_0 2024-04-22T20:48:17.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17302479059140127935', time=2024-04-22T22:28:20.224196+0000, chain.objs.size()=1 2024-04-22T20:48:17.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.wHjYa6MicIp8Pen6KDbCxcdb9r9wPCg_0 2024-04-22T20:48:17.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.227524346531097283', time=2024-04-22T22:28:20.790370+0000, chain.objs.size()=1 2024-04-22T20:48:17.913 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.914+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.GyEUQSvNhqWgPY_AfbNhD9BRNtfFOIm_0 2024-04-22T20:48:17.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.916+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12689278026388007243', time=2024-04-22T22:28:20.890089+0000, chain.objs.size()=1 2024-04-22T20:48:17.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.916+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.TcKEjhjaPYnjqGQktr1nNnqaH0JXEFH_0 2024-04-22T20:48:17.914 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.916+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17295833378467865355', time=2024-04-22T22:28:21.377544+0000, chain.objs.size()=1 2024-04-22T20:48:17.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.916+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Ik2afXz0r06Yx10AkNcV-ySfmNZzGU0_0 2024-04-22T20:48:17.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.916+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.2551814110830896285', time=2024-04-22T22:28:22.638617+0000, chain.objs.size()=1 2024-04-22T20:48:17.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.916+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.9rfUJCPbzNK7U8cETQEVzE6hwKqa4rG_0 2024-04-22T20:48:17.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.916+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8345050409622892162', time=2024-04-22T22:28:22.639349+0000, chain.objs.size()=1 2024-04-22T20:48:17.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.916+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KAX1BK8saUtMHnIeWlxbmzw5StHxk1D_0 2024-04-22T20:48:17.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.917+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12904977715729037436', time=2024-04-22T22:28:23.341461+0000, chain.objs.size()=1 2024-04-22T20:48:17.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.917+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XJq8GeHLysqhyXQ3W1h37ax0Qnmy1rs_0 2024-04-22T20:48:17.915 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.917+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12361160156538768185', time=2024-04-22T22:28:23.489484+0000, chain.objs.size()=1 2024-04-22T20:48:17.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.917+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.OifIxY4Emp92Z0nSaAAjNtZFrgDC8Ak_0 2024-04-22T20:48:17.916 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.918+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4988963659042107679', time=2024-04-22T22:28:24.379421+0000, chain.objs.size()=1 2024-04-22T20:48:17.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.918+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.PGSo1EmMfvGOeA00AR_wX1ahg5pb3oI_0 2024-04-22T20:48:17.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.918+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1157018386248199017', time=2024-04-22T22:28:24.380273+0000, chain.objs.size()=1 2024-04-22T20:48:17.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.918+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.eCAiXpuMc6GW7x5A0ya4yRf06TzZN6c_0 2024-04-22T20:48:17.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9504764400717377977', time=2024-04-22T22:28:24.566989+0000, chain.objs.size()=1 2024-04-22T20:48:17.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.bqU0EVpr2r0bb4ffCBlRFGS0paq5tb1_0 2024-04-22T20:48:17.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.178447559478188616', time=2024-04-22T22:28:24.696112+0000, chain.objs.size()=1 2024-04-22T20:48:17.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.JkFSZboFXOQqGApC2XioSm9M8x1EBcZ_0 2024-04-22T20:48:17.917 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7514980469704666570', time=2024-04-22T22:28:24.759183+0000, chain.objs.size()=1 2024-04-22T20:48:17.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.4OHKDoYI1xfyD4y_L51Q3Fk4dBLQUXy_0 2024-04-22T20:48:17.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.919+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.604384863100226861', time=2024-04-22T22:28:25.021302+0000, chain.objs.size()=1 2024-04-22T20:48:17.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.919+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cV-BK6Jyj9rfQ1IDA0Tiiqs6nyayc6y_0 2024-04-22T20:48:17.918 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4483378812781594276', time=2024-04-22T22:28:25.982711+0000, chain.objs.size()=1 2024-04-22T20:48:17.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.XZ1cU7BKQis9PNSLLOYWhJNIBMO8ZVl_0 2024-04-22T20:48:17.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9217286443628310958', time=2024-04-22T22:28:26.085114+0000, chain.objs.size()=1 2024-04-22T20:48:17.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.Psmxd50j5ZhdfQAsm7ScU-ojmLnstLX_0 2024-04-22T20:48:17.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.920+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14255150749588688598', time=2024-04-22T22:28:27.476241+0000, chain.objs.size()=1 2024-04-22T20:48:17.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.920+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.KsmzVnWvtBPkJzF6S5eZZrEJXtU8tv2_0 2024-04-22T20:48:17.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.922+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.450871723075823456', time=2024-04-22T22:28:27.955283+0000, chain.objs.size()=1 2024-04-22T20:48:17.919 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.922+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.-cMsgnNvbvlJsoRGaApKgme3th_WqS5_0 2024-04-22T20:48:17.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14645813990310245308', time=2024-04-22T22:28:28.345943+0000, chain.objs.size()=1 2024-04-22T20:48:17.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.avV8amt4L7sWHCjQnptSmOe49J5u9GL_0 2024-04-22T20:48:17.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1429245860279326781', time=2024-04-22T22:28:28.402102+0000, chain.objs.size()=1 2024-04-22T20:48:17.920 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.cXMgFPqcB1iXbqx-EA4ky3vpPjd0jlB_0 2024-04-22T20:48:17.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11159115450414829934', time=2024-04-22T22:28:28.402800+0000, chain.objs.size()=1 2024-04-22T20:48:17.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.FUrLvtRvLt5xe8Rv4Y5RHfStecVO2iI_0 2024-04-22T20:48:17.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.923+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1112294715758818332', time=2024-04-22T22:28:28.841188+0000, chain.objs.size()=1 2024-04-22T20:48:17.921 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.923+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.8d9ZHMnDplbGf7STB2CPSJm8mL-k0te_0 2024-04-22T20:48:17.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18279481046634118780', time=2024-04-22T22:28:29.295586+0000, chain.objs.size()=1 2024-04-22T20:48:17.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.312__shadow_.w-1xFHstwIJb3CZ06BNYU2y8I3gGZyX_0 2024-04-22T20:48:17.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1472102843917325463', time=2024-04-22T22:29:06.332313+0000, chain.objs.size()=1 2024-04-22T20:48:17.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.RvHVPLaEmLVxdgag3FPm9o70MG8a-fu_0 2024-04-22T20:48:17.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10519316172374515270', time=2024-04-22T22:29:06.581756+0000, chain.objs.size()=1 2024-04-22T20:48:17.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.TlArojr0-if1U_QGwhsx0-AjPv1MmVe_0 2024-04-22T20:48:17.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16187623706791010209', time=2024-04-22T22:29:06.743104+0000, chain.objs.size()=1 2024-04-22T20:48:17.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.cdVDwAMGQ8vaHlDrQmIccxpalq5yy-D_0 2024-04-22T20:48:17.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3923702832482628742', time=2024-04-22T22:29:06.791543+0000, chain.objs.size()=1 2024-04-22T20:48:17.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.qWo4CL2k44GBU8opYfRforoAVBhecMz_0 2024-04-22T20:48:17.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1802502097987957607', time=2024-04-22T22:29:07.201105+0000, chain.objs.size()=1 2024-04-22T20:48:17.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.924+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PJrug77Rd4b8uo5Qw-f5MQilBXr3j8e_0 2024-04-22T20:48:17.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.925+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9878179809012084710', time=2024-04-22T22:29:07.249595+0000, chain.objs.size()=1 2024-04-22T20:48:17.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.925+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.EF7mNhCTCsM6qhpkRIX3RK_Yp6NRmyC_0 2024-04-22T20:48:17.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.927+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16218970377912801000', time=2024-04-22T22:29:07.959193+0000, chain.objs.size()=1 2024-04-22T20:48:17.924 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.927+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.z9oEvaXNNnZxJE0KmlUzrje4QQR8akk_0 2024-04-22T20:48:17.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14288563324736368037', time=2024-04-22T22:29:08.119703+0000, chain.objs.size()=1 2024-04-22T20:48:17.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.DT4jz3P4FW-IYLhnkZjsqGMXV37lcpT_0 2024-04-22T20:48:17.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7225791543169598644', time=2024-04-22T22:29:08.697191+0000, chain.objs.size()=1 2024-04-22T20:48:17.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.x-2HZBm-yLMOMn11SvyiZLShX3-H3kg_0 2024-04-22T20:48:17.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8874117082408158950', time=2024-04-22T22:29:09.367326+0000, chain.objs.size()=1 2024-04-22T20:48:17.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.nb9SNTYQfsvMfydksh1uRq4I8fygWKA_0 2024-04-22T20:48:17.925 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17612742925904599194', time=2024-04-22T22:29:09.412142+0000, chain.objs.size()=1 2024-04-22T20:48:17.926 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0O2iwpcspk9M-ugHsIZuHfSCjZlGTAS_0 2024-04-22T20:48:17.926 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.928+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.405578410911247285', time=2024-04-22T22:29:09.751834+0000, chain.objs.size()=1 2024-04-22T20:48:17.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.OGzFG2wruqbesPJU3OStcR7BPo-K4lI_0 2024-04-22T20:48:17.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3726879785942152172', time=2024-04-22T22:29:10.345261+0000, chain.objs.size()=1 2024-04-22T20:48:17.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.KOijpWBp87enuyBQQKAAumqwFUaxZ1T_0 2024-04-22T20:48:17.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15604339607613150812', time=2024-04-22T22:29:10.390327+0000, chain.objs.size()=1 2024-04-22T20:48:17.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.MZmYiJMiRRV4R-ZGDdEt3sLaQ3KUDnY_0 2024-04-22T20:48:17.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.18194309155717502194', time=2024-04-22T22:29:10.966210+0000, chain.objs.size()=1 2024-04-22T20:48:17.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.uyQATWk6E1n1eKeLG2BTXKC-Bf7giVQ_0 2024-04-22T20:48:17.927 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7466434151771190216', time=2024-04-22T22:29:11.223831+0000, chain.objs.size()=1 2024-04-22T20:48:17.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.M_elWK5BH0LlxlSXHZ9w93LmM3oFzx6_0 2024-04-22T20:48:17.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.318427392167013263', time=2024-04-22T22:29:12.473301+0000, chain.objs.size()=1 2024-04-22T20:48:17.928 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.929+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.hoaQaqPugQRmGUcA92N-afnXWfUSaPX_0 2024-04-22T20:48:17.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.932+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1356373965040563376', time=2024-04-22T22:29:12.570359+0000, chain.objs.size()=1 2024-04-22T20:48:17.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.932+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.gM26vxtWgX2F3UM_vjQ_5ZXI-R2bKjO_0 2024-04-22T20:48:17.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.932+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13111441473776241979', time=2024-04-22T22:29:13.105833+0000, chain.objs.size()=1 2024-04-22T20:48:17.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.932+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.0N5-AMWCAY2-BPLPCLKbS_gDoyS6195_0 2024-04-22T20:48:17.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.932+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14883959584169958064', time=2024-04-22T22:29:13.326297+0000, chain.objs.size()=1 2024-04-22T20:48:17.929 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.932+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.zhj_YpIu-BpR1jq1j0OaEcgaa7MFmoa_0 2024-04-22T20:48:17.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.932+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17145366966056332412', time=2024-04-22T22:29:13.326986+0000, chain.objs.size()=1 2024-04-22T20:48:17.930 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.932+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.PjQvi-u6U8myyVjyIRcqGGmhfeY_Are_0 2024-04-22T20:48:17.931 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.935+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 0/50808 2024-04-22T20:48:17.934 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=32, truncated=0, next_marker='' 2024-04-22T20:48:17.934 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17885400363886458418', time=2024-04-22T22:29:13.907436+0000, chain.objs.size()=1 2024-04-22T20:48:17.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.CUVPxrssUmJZAM6A4pb1KQVabsrWpAA_0 2024-04-22T20:48:17.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.11344795881614178532', time=2024-04-22T22:29:14.112982+0000, chain.objs.size()=1 2024-04-22T20:48:17.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.LcDPBDbQwZ15H6WPSUZOl8_OaAF0SM__0 2024-04-22T20:48:17.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6536365451670334335', time=2024-04-22T22:29:14.839133+0000, chain.objs.size()=1 2024-04-22T20:48:17.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.mqCtvJde-sZ14wlRwBHZqUvDZgEEQBf_0 2024-04-22T20:48:17.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.615027108688288574', time=2024-04-22T22:29:14.935835+0000, chain.objs.size()=1 2024-04-22T20:48:17.935 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.XtaedM6LYIewde4ZNoJ8qCQ0LarwjYs_0 2024-04-22T20:48:17.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10591690118093222947', time=2024-04-22T22:29:15.353264+0000, chain.objs.size()=1 2024-04-22T20:48:17.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.uHknXn7g_BIahuCYu4fU-nCKGPM3oiN_0 2024-04-22T20:48:17.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.4025468525973115537', time=2024-04-22T22:29:15.353742+0000, chain.objs.size()=1 2024-04-22T20:48:17.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ralJ5EwIFVkcuMGkbEjH38lsCCJ64WZ_0 2024-04-22T20:48:17.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3973862208873969181', time=2024-04-22T22:29:15.677068+0000, chain.objs.size()=1 2024-04-22T20:48:17.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.6tegASpw2RmAAhDIhFa0bghe3myhZZs_0 2024-04-22T20:48:17.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.7331837705430788884', time=2024-04-22T22:29:15.746013+0000, chain.objs.size()=1 2024-04-22T20:48:17.936 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.rE3PMBseJTKq7VNYCZ5k0QBUpmFaXRl_0 2024-04-22T20:48:17.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17096304597321465187', time=2024-04-22T22:29:17.452900+0000, chain.objs.size()=1 2024-04-22T20:48:17.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.HO4nKCkkvl6ewDAFfm_tEZqlvX8n4pu_0 2024-04-22T20:48:17.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6659675249814208179', time=2024-04-22T22:29:17.490279+0000, chain.objs.size()=1 2024-04-22T20:48:17.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ChHn5Cr6pD696HBwSZGAXWE2LuQLxzX_0 2024-04-22T20:48:17.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9123417216679015187', time=2024-04-22T22:29:17.557819+0000, chain.objs.size()=1 2024-04-22T20:48:17.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.313__shadow_.ZVuaH8iNvZjHBt80r5wHMuYRAzoPs-6_0 2024-04-22T20:48:17.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15648551958244638585', time=2024-04-22T22:29:19.812396+0000, chain.objs.size()=1 2024-04-22T20:48:17.937 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.937+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.316__shadow_.6eXbxATqnnfOmEM4k5PzEVEotgPn5MR_0 2024-04-22T20:48:17.938 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.5702431084686716765', time=2024-04-22T22:29:48.862698+0000, chain.objs.size()=1 2024-04-22T20:48:17.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.361__shadow_.EGlegdc8X1lcUsvgkgiWi5Jjg13aJGx_0 2024-04-22T20:48:17.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3375395970840104061', time=2024-04-22T22:29:53.333574+0000, chain.objs.size()=1 2024-04-22T20:48:17.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.366__shadow_.5O_VzHJB0F6ChywomGr7uXCBQfGBj9P_0 2024-04-22T20:48:17.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10396181221570041039', time=2024-04-22T22:30:31.449326+0000, chain.objs.size()=1 2024-04-22T20:48:17.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.422__shadow_.QuiOdSgu4FCwr1VQmxtlZve-D9k1wGV_0 2024-04-22T20:48:17.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.10856737239674269297', time=2024-04-22T22:30:38.638436+0000, chain.objs.size()=1 2024-04-22T20:48:17.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.432__shadow_.CM9r21OE0mheDiTAOhCf2m1u1gXJG7G_0 2024-04-22T20:48:17.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.14375919328841359074', time=2024-04-22T22:30:50.436647+0000, chain.objs.size()=1 2024-04-22T20:48:17.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.445__shadow_.KJCZ4zcCEc6EvZQVLZwtX9k93Tt44GR_0 2024-04-22T20:48:17.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8722800292029677578', time=2024-04-22T22:31:08.210794+0000, chain.objs.size()=1 2024-04-22T20:48:17.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.470__shadow_.axjM7xHZ91CXVzc-87mOrwjpEv4sJA5_0 2024-04-22T20:48:17.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1564689617956029658', time=2024-04-22T22:31:23.623216+0000, chain.objs.size()=1 2024-04-22T20:48:17.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.487__shadow_.myWPrazPOs9LTSWZOKSSgogKObIntxr_0 2024-04-22T20:48:17.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.1977297596720534609', time=2024-04-22T22:32:10.177590+0000, chain.objs.size()=1 2024-04-22T20:48:17.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.506__shadow_.raf8jJfnaUp1L1G2nWTBQzoqn0pKTpw_0 2024-04-22T20:48:17.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.16676724625190847500', time=2024-04-22T22:32:46.706159+0000, chain.objs.size()=3 2024-04-22T20:48:17.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~SN3aJVsFvZk-29gHFSJnp6e0V-L627s.1 2024-04-22T20:48:17.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~SN3aJVsFvZk-29gHFSJnp6e0V-L627s.2 2024-04-22T20:48:17.941 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.941+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.510__multipart_mymultipart.2~SN3aJVsFvZk-29gHFSJnp6e0V-L627s.3 2024-04-22T20:48:17.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15098222544167836355', time=2024-04-22T22:37:53.783670+0000, chain.objs.size()=1 2024-04-22T20:48:17.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:FgsWx6aC2JOUSdF4y5cCjEZHXnWT5cI_.Kwfh4C_APu_CknIUpEPjS87NI2IrX2P_0 2024-04-22T20:48:17.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.15623425902292929013', time=2024-04-22T22:38:01.810691+0000, chain.objs.size()=1 2024-04-22T20:48:17.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.565__shadow:4hx0x.BkAZQNeBKIQnZlxuf8Wfd309Q_.SFYYNm3k0M3qlDNnCVo857ancAw3JLC_0 2024-04-22T20:48:17.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.8857915660379592248', time=2024-04-22T22:38:49.655296+0000, chain.objs.size()=1 2024-04-22T20:48:17.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:NEgs3EuYUaNUe0W0HQHdZSYzbvFBMmx_.7wyXh48WmT57WCPuukC-X5EDVE26Ih-_0 2024-04-22T20:48:17.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.6115618214258406280', time=2024-04-22T22:38:49.655937+0000, chain.objs.size()=1 2024-04-22T20:48:17.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.567__shadow:VPLIm1VhEL4PtcI2RVYTVF0gwxYefKd_.9LB7UO-tztYbADnsNitqwzacaFIpk6s_0 2024-04-22T20:48:17.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.13225617503665357548', time=2024-04-22T22:38:51.966327+0000, chain.objs.size()=1 2024-04-22T20:48:17.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:-dVByXoDlV-EFnizGQqWv8SzVp2FAWe_.kWtQjnq6VIeWmMQahTWNPn-ZtLhy_ww_0 2024-04-22T20:48:17.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:mYroPTRUGY7JJ0FlSIYgWmGLZUtmu1u_.uh-IosxowmGuW8QG8mHDszFV7eCd5qI_1_98917842c8fee929986b4ebbd4754a44', time=2024-04-22T22:38:52.201500+0000, chain.objs.size()=1 2024-04-22T20:48:17.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.568__shadow:mYroPTRUGY7JJ0FlSIYgWmGLZUtmu1u_.uh-IosxowmGuW8QG8mHDszFV7eCd5qI_0 2024-04-22T20:48:17.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.12041191602081573630', time=2024-04-22T22:41:54.101275+0000, chain.objs.size()=1 2024-04-22T20:48:17.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.620__shadow_.ZXsi2plHCx_n5oS8IFOEx2ItRytpXzf_0 2024-04-22T20:48:17.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9582078858061054073', time=2024-04-22T22:42:25.451595+0000, chain.objs.size()=1 2024-04-22T20:48:17.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.639__shadow_.Zf4AzbIwXVtwJKgDZGedgrghLi4kdif_0 2024-04-22T20:48:17.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.3072997644462800173', time=2024-04-22T22:42:52.530984+0000, chain.objs.size()=1 2024-04-22T20:48:17.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.945+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.647__shadow_.j0Y7sHtyhvlRUqi8cgCjLTq7LspWBrU_0 2024-04-22T20:48:17.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.946+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.9268809244434856951', time=2024-04-22T22:42:59.182284+0000, chain.objs.size()=1 2024-04-22T20:48:17.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.946+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.656__shadow_.fw8OB1Zr9yOSwR53pg12xwcYwfBcFQt_0 2024-04-22T20:48:17.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.946+0000 7f2f5fb46800 20 garbage collection: RGWGC::process iterating over entry tag='19263407-9cd1-4c73-8de4-71c0779d4a94.4665.17063210737011016384', time=2024-04-22T22:43:20.578476+0000, chain.objs.size()=1 2024-04-22T20:48:17.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.946+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:19263407-9cd1-4c73-8de4-71c0779d4a94.4665.691__shadow:CRXKb-07cEvg72DPtmGJQQGLcsZHoO6_.yODKsAoQg_6-cFnGXWiLMJqWNg_19I0_0 2024-04-22T20:48:17.947 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.950+0000 7f2f5fb46800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-22T20:48:17.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.953+0000 7f2e137e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-22T20:48:17.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.955+0000 7f2f5fb46800 20 remove_watcher() i=0 2024-04-22T20:48:17.952 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.955+0000 7f2f5fb46800 2 removed watcher, disabling cache 2024-04-22T20:48:17.953 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.956+0000 7f2f5fb46800 20 remove_watcher() i=1 2024-04-22T20:48:17.954 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.957+0000 7f2f5fb46800 20 remove_watcher() i=2 2024-04-22T20:48:17.955 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.958+0000 7f2f5fb46800 20 remove_watcher() i=3 2024-04-22T20:48:17.956 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.959+0000 7f2f5fb46800 20 remove_watcher() i=4 2024-04-22T20:48:17.957 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.960+0000 7f2f5fb46800 20 remove_watcher() i=5 2024-04-22T20:48:17.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.961+0000 7f2f5fb46800 20 remove_watcher() i=6 2024-04-22T20:48:17.958 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-22T20:48:17.962+0000 7f2f5fb46800 20 remove_watcher() i=7 2024-04-22T20:48:17.969 DEBUG:teuthology.run_tasks:Unwinding manager openssl_keys 2024-04-22T20:48:17.980 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/ca/root.crt /home/ubuntu/cephtest/ca/root.key 2024-04-22T20:48:18.036 DEBUG:teuthology.orchestra.run.smithi136:> rmdir --ignore-fail-on-non-empty /home/ubuntu/cephtest/ca 2024-04-22T20:48:18.103 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/ca/rgw.client.0.crt /home/ubuntu/cephtest/ca/rgw.client.0.key 2024-04-22T20:48:18.172 DEBUG:teuthology.orchestra.run.smithi136:> rmdir --ignore-fail-on-non-empty /home/ubuntu/cephtest/ca 2024-04-22T20:48:18.240 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm /usr/share/pki/ca-trust-source/anchors/root.crt 2024-04-22T20:48:18.317 DEBUG:teuthology.orchestra.run.smithi136:> sudo update-ca-trust 2024-04-22T20:48:19.124 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-22T20:48:19.134 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-22T20:48:19.135 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T20:48:19.471 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:19.486 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T20:48:19.491 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":1968,"stamp":"2024-04-22T20:48:18.327450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":869472021,"num_objects":4919,"num_object_clones":0,"num_object_copies":14345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4919,"num_whiteouts":0,"num_read":154246,"num_read_kb":2718939,"num_write":140739,"num_write_kb":2429877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13467,"ondisk_log_size":13467,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2919188,"kb_used_data":1444180,"kb_used_omap":12,"kb_used_meta":1474931,"kb_avail":752055532,"statfs":{"total":773094113280,"available":770104864768,"internally_reserved":0,"allocated":1478840320,"data_stored":1463003556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1510329906},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":34,"num_read_kb":39,"num_write":27,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005878"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.177805+0000","last_change":"2024-04-22T20:19:16.796805+0000","last_active":"2024-04-22T20:44:20.177805+0000","last_peered":"2024-04-22T20:44:20.177805+0000","last_clean":"2024-04-22T20:44:20.177805+0000","last_became_active":"2024-04-22T20:19:16.796520+0000","last_became_peered":"2024-04-22T20:19:16.796520+0000","last_unstale":"2024-04-22T20:44:20.177805+0000","last_undegraded":"2024-04-22T20:44:20.177805+0000","last_fullsized":"2024-04-22T20:44:20.177805+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:42.980562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"261'4","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186412+0000","last_change":"2024-04-22T20:05:17.900640+0000","last_active":"2024-04-22T20:45:20.186412+0000","last_peered":"2024-04-22T20:45:20.186412+0000","last_clean":"2024-04-22T20:45:20.186412+0000","last_became_active":"2024-04-22T20:05:17.900325+0000","last_became_peered":"2024-04-22T20:05:17.900325+0000","last_unstale":"2024-04-22T20:45:20.186412+0000","last_undegraded":"2024-04-22T20:45:20.186412+0000","last_fullsized":"2024-04-22T20:45:20.186412+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:55:20.209199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154687+0000","last_change":"2024-04-22T20:05:13.813157+0000","last_active":"2024-04-22T20:45:15.154687+0000","last_peered":"2024-04-22T20:45:15.154687+0000","last_clean":"2024-04-22T20:45:15.154687+0000","last_became_active":"2024-04-22T20:05:13.812933+0000","last_became_peered":"2024-04-22T20:05:13.812933+0000","last_unstale":"2024-04-22T20:45:15.154687+0000","last_undegraded":"2024-04-22T20:45:15.154687+0000","last_fullsized":"2024-04-22T20:45:15.154687+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:47:15.012252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"261'20","reported_seq":100,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.176985+0000","last_change":"2024-04-22T20:01:04.876250+0000","last_active":"2024-04-22T20:44:50.176985+0000","last_peered":"2024-04-22T20:44:50.176985+0000","last_clean":"2024-04-22T20:44:50.176985+0000","last_became_active":"2024-04-22T20:01:04.875981+0000","last_became_peered":"2024-04-22T20:01:04.875981+0000","last_unstale":"2024-04-22T20:44:50.176985+0000","last_undegraded":"2024-04-22T20:44:50.176985+0000","last_fullsized":"2024-04-22T20:44:50.176985+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:47:50.228681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.060560+0000","last_change":"2024-04-22T20:19:16.790684+0000","last_active":"2024-04-22T20:44:20.060560+0000","last_peered":"2024-04-22T20:44:20.060560+0000","last_clean":"2024-04-22T20:44:20.060560+0000","last_became_active":"2024-04-22T20:19:16.790377+0000","last_became_peered":"2024-04-22T20:19:16.790377+0000","last_unstale":"2024-04-22T20:44:20.060560+0000","last_undegraded":"2024-04-22T20:44:20.060560+0000","last_fullsized":"2024-04-22T20:44:20.060560+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:16:51.462209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.178227+0000","last_change":"2024-04-22T20:05:17.899675+0000","last_active":"2024-04-22T20:45:20.178227+0000","last_peered":"2024-04-22T20:45:20.178227+0000","last_clean":"2024-04-22T20:45:20.178227+0000","last_became_active":"2024-04-22T20:05:17.899338+0000","last_became_peered":"2024-04-22T20:05:17.899338+0000","last_unstale":"2024-04-22T20:45:20.178227+0000","last_undegraded":"2024-04-22T20:45:20.178227+0000","last_fullsized":"2024-04-22T20:45:20.178227+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:09:36.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"258'4","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.043137+0000","last_change":"2024-04-22T20:05:13.808152+0000","last_active":"2024-04-22T20:45:15.043137+0000","last_peered":"2024-04-22T20:45:15.043137+0000","last_clean":"2024-04-22T20:45:15.043137+0000","last_became_active":"2024-04-22T20:05:13.807231+0000","last_became_peered":"2024-04-22T20:05:13.807231+0000","last_unstale":"2024-04-22T20:45:15.043137+0000","last_undegraded":"2024-04-22T20:45:15.043137+0000","last_fullsized":"2024-04-22T20:45:15.043137+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:18:02.300257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"261'22","reported_seq":101,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.066043+0000","last_change":"2024-04-22T20:01:04.874850+0000","last_active":"2024-04-22T20:45:15.066043+0000","last_peered":"2024-04-22T20:45:15.066043+0000","last_clean":"2024-04-22T20:45:15.066043+0000","last_became_active":"2024-04-22T20:01:04.874575+0000","last_became_peered":"2024-04-22T20:01:04.874575+0000","last_unstale":"2024-04-22T20:45:15.066043+0000","last_undegraded":"2024-04-22T20:45:15.066043+0000","last_fullsized":"2024-04-22T20:45:15.066043+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:00:34.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":48,"num_write":47,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.298533+0000","last_change":"2024-04-22T20:19:16.794970+0000","last_active":"2024-04-22T20:43:30.298533+0000","last_peered":"2024-04-22T20:43:30.298533+0000","last_clean":"2024-04-22T20:43:30.298533+0000","last_became_active":"2024-04-22T20:19:16.794700+0000","last_became_peered":"2024-04-22T20:19:16.794700+0000","last_unstale":"2024-04-22T20:43:30.298533+0000","last_undegraded":"2024-04-22T20:43:30.298533+0000","last_fullsized":"2024-04-22T20:43:30.298533+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:59.399131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.177554+0000","last_change":"2024-04-22T20:05:18.211663+0000","last_active":"2024-04-22T20:44:55.177554+0000","last_peered":"2024-04-22T20:44:55.177554+0000","last_clean":"2024-04-22T20:44:55.177554+0000","last_became_active":"2024-04-22T20:05:18.211400+0000","last_became_peered":"2024-04-22T20:05:18.211400+0000","last_unstale":"2024-04-22T20:44:55.177554+0000","last_undegraded":"2024-04-22T20:44:55.177554+0000","last_fullsized":"2024-04-22T20:44:55.177554+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:07:00.571570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.067028+0000","last_change":"2024-04-22T20:05:13.809630+0000","last_active":"2024-04-22T20:45:25.067028+0000","last_peered":"2024-04-22T20:45:25.067028+0000","last_clean":"2024-04-22T20:45:25.067028+0000","last_became_active":"2024-04-22T20:05:13.809349+0000","last_became_peered":"2024-04-22T20:05:13.809349+0000","last_unstale":"2024-04-22T20:45:25.067028+0000","last_undegraded":"2024-04-22T20:45:25.067028+0000","last_fullsized":"2024-04-22T20:45:25.067028+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:04:26.130081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"261'24","reported_seq":114,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.167723+0000","last_change":"2024-04-22T20:01:04.870729+0000","last_active":"2024-04-22T20:43:35.167723+0000","last_peered":"2024-04-22T20:43:35.167723+0000","last_clean":"2024-04-22T20:43:35.167723+0000","last_became_active":"2024-04-22T20:01:04.869785+0000","last_became_peered":"2024-04-22T20:01:04.869785+0000","last_unstale":"2024-04-22T20:43:35.167723+0000","last_undegraded":"2024-04-22T20:43:35.167723+0000","last_fullsized":"2024-04-22T20:43:35.167723+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:24:59.438391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":82,"num_write":36,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.141064+0000","last_change":"2024-04-22T20:19:16.795441+0000","last_active":"2024-04-22T20:43:30.141064+0000","last_peered":"2024-04-22T20:43:30.141064+0000","last_clean":"2024-04-22T20:43:30.141064+0000","last_became_active":"2024-04-22T20:19:16.793667+0000","last_became_peered":"2024-04-22T20:19:16.793667+0000","last_unstale":"2024-04-22T20:43:30.141064+0000","last_undegraded":"2024-04-22T20:43:30.141064+0000","last_fullsized":"2024-04-22T20:43:30.141064+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:09.885678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"261'7","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:15.187088+0000","last_change":"2024-04-22T20:06:05.349958+0000","last_active":"2024-04-22T20:46:15.187088+0000","last_peered":"2024-04-22T20:46:15.187088+0000","last_clean":"2024-04-22T20:46:15.187088+0000","last_became_active":"2024-04-22T20:06:05.345554+0000","last_became_peered":"2024-04-22T20:06:05.345554+0000","last_unstale":"2024-04-22T20:46:15.187088+0000","last_undegraded":"2024-04-22T20:46:15.187088+0000","last_fullsized":"2024-04-22T20:46:15.187088+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:50:22.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":17,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1f","version":"258'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.173992+0000","last_change":"2024-04-22T20:05:13.806565+0000","last_active":"2024-04-22T20:44:50.173992+0000","last_peered":"2024-04-22T20:44:50.173992+0000","last_clean":"2024-04-22T20:44:50.173992+0000","last_became_active":"2024-04-22T20:05:13.806246+0000","last_became_peered":"2024-04-22T20:05:13.806246+0000","last_unstale":"2024-04-22T20:44:50.173992+0000","last_undegraded":"2024-04-22T20:44:50.173992+0000","last_fullsized":"2024-04-22T20:44:50.173992+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:04:13.772282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":16720,"num_write":13,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"261'59","reported_seq":198,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.141957+0000","last_change":"2024-04-22T20:01:04.874395+0000","last_active":"2024-04-22T20:43:35.141957+0000","last_peered":"2024-04-22T20:43:35.141957+0000","last_clean":"2024-04-22T20:43:35.141957+0000","last_became_active":"2024-04-22T20:01:04.873899+0000","last_became_peered":"2024-04-22T20:01:04.873899+0000","last_unstale":"2024-04-22T20:43:35.141957+0000","last_undegraded":"2024-04-22T20:43:35.141957+0000","last_fullsized":"2024-04-22T20:43:35.141957+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:23:27.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":183,"num_write":99,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.155406+0000","last_change":"2024-04-22T20:19:16.794646+0000","last_active":"2024-04-22T20:45:20.155406+0000","last_peered":"2024-04-22T20:45:20.155406+0000","last_clean":"2024-04-22T20:45:20.155406+0000","last_became_active":"2024-04-22T20:19:16.794110+0000","last_became_peered":"2024-04-22T20:19:16.794110+0000","last_unstale":"2024-04-22T20:45:20.155406+0000","last_undegraded":"2024-04-22T20:45:20.155406+0000","last_fullsized":"2024-04-22T20:45:20.155406+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:42.561415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"261'1","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:10.176246+0000","last_change":"2024-04-22T20:05:17.898277+0000","last_active":"2024-04-22T20:44:10.176246+0000","last_peered":"2024-04-22T20:44:10.176246+0000","last_clean":"2024-04-22T20:44:10.176246+0000","last_became_active":"2024-04-22T20:05:17.897771+0000","last_became_peered":"2024-04-22T20:05:17.897771+0000","last_unstale":"2024-04-22T20:44:10.176246+0000","last_undegraded":"2024-04-22T20:44:10.176246+0000","last_fullsized":"2024-04-22T20:44:10.176246+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:22:07.487507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"258'5","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.170541+0000","last_change":"2024-04-22T20:05:13.812772+0000","last_active":"2024-04-22T20:43:55.170541+0000","last_peered":"2024-04-22T20:43:55.170541+0000","last_clean":"2024-04-22T20:43:55.170541+0000","last_became_active":"2024-04-22T20:05:13.812308+0000","last_became_peered":"2024-04-22T20:05:13.812308+0000","last_unstale":"2024-04-22T20:43:55.170541+0000","last_undegraded":"2024-04-22T20:43:55.170541+0000","last_fullsized":"2024-04-22T20:43:55.170541+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:52.451966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":9,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"261'8","reported_seq":72,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.141914+0000","last_change":"2024-04-22T20:01:04.877587+0000","last_active":"2024-04-22T20:43:35.141914+0000","last_peered":"2024-04-22T20:43:35.141914+0000","last_clean":"2024-04-22T20:43:35.141914+0000","last_became_active":"2024-04-22T20:01:04.877209+0000","last_became_peered":"2024-04-22T20:01:04.877209+0000","last_unstale":"2024-04-22T20:43:35.141914+0000","last_undegraded":"2024-04-22T20:43:35.141914+0000","last_fullsized":"2024-04-22T20:43:35.141914+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:57.901050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.037056+0000","last_change":"2024-04-22T20:19:16.789917+0000","last_active":"2024-04-22T20:44:20.037056+0000","last_peered":"2024-04-22T20:44:20.037056+0000","last_clean":"2024-04-22T20:44:20.037056+0000","last_became_active":"2024-04-22T20:19:16.789599+0000","last_became_peered":"2024-04-22T20:19:16.789599+0000","last_unstale":"2024-04-22T20:44:20.037056+0000","last_undegraded":"2024-04-22T20:44:20.037056+0000","last_fullsized":"2024-04-22T20:44:20.037056+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:15.659980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"261'1","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.303074+0000","last_change":"2024-04-22T20:05:17.891180+0000","last_active":"2024-04-22T20:44:00.303074+0000","last_peered":"2024-04-22T20:44:00.303074+0000","last_clean":"2024-04-22T20:44:00.303074+0000","last_became_active":"2024-04-22T20:05:17.890841+0000","last_became_peered":"2024-04-22T20:05:17.890841+0000","last_unstale":"2024-04-22T20:44:00.303074+0000","last_undegraded":"2024-04-22T20:44:00.303074+0000","last_fullsized":"2024-04-22T20:44:00.303074+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:53:40.164764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":1,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.167430+0000","last_change":"2024-04-22T20:05:13.807783+0000","last_active":"2024-04-22T20:44:05.167430+0000","last_peered":"2024-04-22T20:44:05.167430+0000","last_clean":"2024-04-22T20:44:05.167430+0000","last_became_active":"2024-04-22T20:05:13.807516+0000","last_became_peered":"2024-04-22T20:05:13.807516+0000","last_unstale":"2024-04-22T20:44:05.167430+0000","last_undegraded":"2024-04-22T20:44:05.167430+0000","last_fullsized":"2024-04-22T20:44:05.167430+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:58:08.969201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"261'77","reported_seq":239,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.299254+0000","last_change":"2024-04-22T20:01:04.873339+0000","last_active":"2024-04-22T20:43:35.299254+0000","last_peered":"2024-04-22T20:43:35.299254+0000","last_clean":"2024-04-22T20:43:35.299254+0000","last_became_active":"2024-04-22T20:01:04.873015+0000","last_became_peered":"2024-04-22T20:01:04.873015+0000","last_unstale":"2024-04-22T20:43:35.299254+0000","last_undegraded":"2024-04-22T20:43:35.299254+0000","last_fullsized":"2024-04-22T20:43:35.299254+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:49:05.613359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":236,"num_write":125,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"261'1","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053060+0000","last_change":"2024-04-22T20:19:16.794883+0000","last_active":"2024-04-22T20:44:20.053060+0000","last_peered":"2024-04-22T20:44:20.053060+0000","last_clean":"2024-04-22T20:44:20.053060+0000","last_became_active":"2024-04-22T20:19:16.794564+0000","last_became_peered":"2024-04-22T20:19:16.794564+0000","last_unstale":"2024-04-22T20:44:20.053060+0000","last_undegraded":"2024-04-22T20:44:20.053060+0000","last_fullsized":"2024-04-22T20:44:20.053060+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:15.121442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"261'3","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314023+0000","last_change":"2024-04-22T20:05:17.901743+0000","last_active":"2024-04-22T20:45:20.314023+0000","last_peered":"2024-04-22T20:45:20.314023+0000","last_clean":"2024-04-22T20:45:20.314023+0000","last_became_active":"2024-04-22T20:05:17.901446+0000","last_became_peered":"2024-04-22T20:05:17.901446+0000","last_unstale":"2024-04-22T20:45:20.314023+0000","last_undegraded":"2024-04-22T20:45:20.314023+0000","last_fullsized":"2024-04-22T20:45:20.314023+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:13:01.401575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":11264,"num_write":8,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"261'1","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.177454+0000","last_change":"2024-04-22T20:05:13.798628+0000","last_active":"2024-04-22T20:45:15.177454+0000","last_peered":"2024-04-22T20:45:15.177454+0000","last_clean":"2024-04-22T20:45:15.177454+0000","last_became_active":"2024-04-22T20:05:13.798311+0000","last_became_peered":"2024-04-22T20:05:13.798311+0000","last_unstale":"2024-04-22T20:45:15.177454+0000","last_undegraded":"2024-04-22T20:45:15.177454+0000","last_fullsized":"2024-04-22T20:45:15.177454+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:21:50.192609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'27","reported_seq":116,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.176374+0000","last_change":"2024-04-22T20:01:04.873715+0000","last_active":"2024-04-22T20:44:45.176374+0000","last_peered":"2024-04-22T20:44:45.176374+0000","last_clean":"2024-04-22T20:44:45.176374+0000","last_became_active":"2024-04-22T20:01:04.873426+0000","last_became_peered":"2024-04-22T20:01:04.873426+0000","last_unstale":"2024-04-22T20:44:45.176374+0000","last_undegraded":"2024-04-22T20:44:45.176374+0000","last_fullsized":"2024-04-22T20:44:45.176374+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:29:40.987253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":80,"num_write":54,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053106+0000","last_change":"2024-04-22T20:19:16.800874+0000","last_active":"2024-04-22T20:44:20.053106+0000","last_peered":"2024-04-22T20:44:20.053106+0000","last_clean":"2024-04-22T20:44:20.053106+0000","last_became_active":"2024-04-22T20:19:16.800611+0000","last_became_peered":"2024-04-22T20:19:16.800611+0000","last_unstale":"2024-04-22T20:44:20.053106+0000","last_undegraded":"2024-04-22T20:44:20.053106+0000","last_fullsized":"2024-04-22T20:44:20.053106+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:27:17.425924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"258'4","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.155286+0000","last_change":"2024-04-22T20:05:17.892900+0000","last_active":"2024-04-22T20:45:20.155286+0000","last_peered":"2024-04-22T20:45:20.155286+0000","last_clean":"2024-04-22T20:45:20.155286+0000","last_became_active":"2024-04-22T20:05:17.892624+0000","last_became_peered":"2024-04-22T20:05:17.892624+0000","last_unstale":"2024-04-22T20:45:20.155286+0000","last_undegraded":"2024-04-22T20:45:20.155286+0000","last_fullsized":"2024-04-22T20:45:20.155286+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:02.296574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3072,"num_write":4,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'1","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179926+0000","last_change":"2024-04-22T20:05:13.816306+0000","last_active":"2024-04-22T20:45:15.179926+0000","last_peered":"2024-04-22T20:45:15.179926+0000","last_clean":"2024-04-22T20:45:15.179926+0000","last_became_active":"2024-04-22T20:05:13.816011+0000","last_became_peered":"2024-04-22T20:05:13.816011+0000","last_unstale":"2024-04-22T20:45:15.179926+0000","last_undegraded":"2024-04-22T20:45:15.179926+0000","last_fullsized":"2024-04-22T20:45:15.179926+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:44.934646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"261'43","reported_seq":150,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.038576+0000","last_change":"2024-04-22T20:01:04.872669+0000","last_active":"2024-04-22T20:44:35.038576+0000","last_peered":"2024-04-22T20:44:35.038576+0000","last_clean":"2024-04-22T20:44:35.038576+0000","last_became_active":"2024-04-22T20:01:04.872356+0000","last_became_peered":"2024-04-22T20:01:04.872356+0000","last_unstale":"2024-04-22T20:44:35.038576+0000","last_undegraded":"2024-04-22T20:44:35.038576+0000","last_fullsized":"2024-04-22T20:44:35.038576+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:11.955524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":119,"num_write":72,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041329+0000","last_change":"2024-04-22T20:19:16.796888+0000","last_active":"2024-04-22T20:45:00.041329+0000","last_peered":"2024-04-22T20:45:00.041329+0000","last_clean":"2024-04-22T20:45:00.041329+0000","last_became_active":"2024-04-22T20:19:16.796155+0000","last_became_peered":"2024-04-22T20:19:16.796155+0000","last_unstale":"2024-04-22T20:45:00.041329+0000","last_undegraded":"2024-04-22T20:45:00.041329+0000","last_fullsized":"2024-04-22T20:45:00.041329+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:25:57.681625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145610+0000","last_change":"2024-04-22T20:05:18.209303+0000","last_active":"2024-04-22T20:44:00.145610+0000","last_peered":"2024-04-22T20:44:00.145610+0000","last_clean":"2024-04-22T20:44:00.145610+0000","last_became_active":"2024-04-22T20:05:18.209023+0000","last_became_peered":"2024-04-22T20:05:18.209023+0000","last_unstale":"2024-04-22T20:44:00.145610+0000","last_undegraded":"2024-04-22T20:44:00.145610+0000","last_fullsized":"2024-04-22T20:44:00.145610+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:14:17.745357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"258'6","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.184202+0000","last_change":"2024-04-22T20:05:13.809878+0000","last_active":"2024-04-22T20:45:05.184202+0000","last_peered":"2024-04-22T20:45:05.184202+0000","last_clean":"2024-04-22T20:45:05.184202+0000","last_became_active":"2024-04-22T20:05:13.809548+0000","last_became_peered":"2024-04-22T20:05:13.809548+0000","last_unstale":"2024-04-22T20:45:05.184202+0000","last_undegraded":"2024-04-22T20:45:05.184202+0000","last_fullsized":"2024-04-22T20:45:05.184202+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:10.962028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":6146,"num_write":5,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'63","reported_seq":241,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.175344+0000","last_change":"2024-04-22T20:01:04.873570+0000","last_active":"2024-04-22T20:44:35.175344+0000","last_peered":"2024-04-22T20:44:35.175344+0000","last_clean":"2024-04-22T20:44:35.175344+0000","last_became_active":"2024-04-22T20:01:04.873236+0000","last_became_peered":"2024-04-22T20:01:04.873236+0000","last_unstale":"2024-04-22T20:44:35.175344+0000","last_undegraded":"2024-04-22T20:44:35.175344+0000","last_fullsized":"2024-04-22T20:44:35.175344+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:38.761169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":250,"num_write":111,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.064494+0000","last_change":"2024-04-22T20:19:16.791895+0000","last_active":"2024-04-22T20:45:00.064494+0000","last_peered":"2024-04-22T20:45:00.064494+0000","last_clean":"2024-04-22T20:45:00.064494+0000","last_became_active":"2024-04-22T20:19:16.791597+0000","last_became_peered":"2024-04-22T20:19:16.791597+0000","last_unstale":"2024-04-22T20:45:00.064494+0000","last_undegraded":"2024-04-22T20:45:00.064494+0000","last_fullsized":"2024-04-22T20:45:00.064494+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:51.409610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":40,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.153541+0000","last_change":"2024-04-22T20:05:18.211042+0000","last_active":"2024-04-22T20:45:05.153541+0000","last_peered":"2024-04-22T20:45:05.153541+0000","last_clean":"2024-04-22T20:45:05.153541+0000","last_became_active":"2024-04-22T20:05:18.210704+0000","last_became_peered":"2024-04-22T20:05:18.210704+0000","last_unstale":"2024-04-22T20:45:05.153541+0000","last_undegraded":"2024-04-22T20:45:05.153541+0000","last_fullsized":"2024-04-22T20:45:05.153541+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:53:47.345090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.048999+0000","last_change":"2024-04-22T20:05:13.811174+0000","last_active":"2024-04-22T20:43:55.048999+0000","last_peered":"2024-04-22T20:43:55.048999+0000","last_clean":"2024-04-22T20:43:55.048999+0000","last_became_active":"2024-04-22T20:05:13.810902+0000","last_became_peered":"2024-04-22T20:05:13.810902+0000","last_unstale":"2024-04-22T20:43:55.048999+0000","last_undegraded":"2024-04-22T20:43:55.048999+0000","last_fullsized":"2024-04-22T20:43:55.048999+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:05:52.847563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"250'9","reported_seq":76,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.149931+0000","last_change":"2024-04-22T20:01:04.874512+0000","last_active":"2024-04-22T20:44:35.149931+0000","last_peered":"2024-04-22T20:44:35.149931+0000","last_clean":"2024-04-22T20:44:35.149931+0000","last_became_active":"2024-04-22T20:01:04.874230+0000","last_became_peered":"2024-04-22T20:01:04.874230+0000","last_unstale":"2024-04-22T20:44:35.149931+0000","last_undegraded":"2024-04-22T20:44:35.149931+0000","last_fullsized":"2024-04-22T20:44:35.149931+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:17:12.953078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.059314+0000","last_change":"2024-04-22T20:19:16.795626+0000","last_active":"2024-04-22T20:45:00.059314+0000","last_peered":"2024-04-22T20:45:00.059314+0000","last_clean":"2024-04-22T20:45:00.059314+0000","last_became_active":"2024-04-22T20:19:16.795139+0000","last_became_peered":"2024-04-22T20:19:16.795139+0000","last_unstale":"2024-04-22T20:45:00.059314+0000","last_undegraded":"2024-04-22T20:45:00.059314+0000","last_fullsized":"2024-04-22T20:45:00.059314+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:17:47.054792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"261'15","reported_seq":74,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145637+0000","last_change":"2024-04-22T20:05:18.212823+0000","last_active":"2024-04-22T20:44:00.145637+0000","last_peered":"2024-04-22T20:44:00.145637+0000","last_clean":"2024-04-22T20:44:00.145637+0000","last_became_active":"2024-04-22T20:05:18.212336+0000","last_became_peered":"2024-04-22T20:05:18.212336+0000","last_unstale":"2024-04-22T20:44:00.145637+0000","last_undegraded":"2024-04-22T20:44:00.145637+0000","last_fullsized":"2024-04-22T20:44:00.145637+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:32.041183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":38918,"num_write":22,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"261'1","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.058251+0000","last_change":"2024-04-22T20:05:13.816425+0000","last_active":"2024-04-22T20:43:55.058251+0000","last_peered":"2024-04-22T20:43:55.058251+0000","last_clean":"2024-04-22T20:43:55.058251+0000","last_became_active":"2024-04-22T20:05:13.816036+0000","last_became_peered":"2024-04-22T20:05:13.816036+0000","last_unstale":"2024-04-22T20:43:55.058251+0000","last_undegraded":"2024-04-22T20:43:55.058251+0000","last_fullsized":"2024-04-22T20:43:55.058251+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:36.892934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'53","reported_seq":176,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.062011+0000","last_change":"2024-04-22T20:01:04.878348+0000","last_active":"2024-04-22T20:44:35.062011+0000","last_peered":"2024-04-22T20:44:35.062011+0000","last_clean":"2024-04-22T20:44:35.062011+0000","last_became_active":"2024-04-22T20:01:04.878067+0000","last_became_peered":"2024-04-22T20:01:04.878067+0000","last_unstale":"2024-04-22T20:44:35.062011+0000","last_undegraded":"2024-04-22T20:44:35.062011+0000","last_fullsized":"2024-04-22T20:44:35.062011+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:52:49.300812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":150,"num_write":99,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.183432+0000","last_change":"2024-04-22T20:19:16.796141+0000","last_active":"2024-04-22T20:45:00.183432+0000","last_peered":"2024-04-22T20:45:00.183432+0000","last_clean":"2024-04-22T20:45:00.183432+0000","last_became_active":"2024-04-22T20:19:16.795855+0000","last_became_peered":"2024-04-22T20:19:16.795855+0000","last_unstale":"2024-04-22T20:45:00.183432+0000","last_undegraded":"2024-04-22T20:45:00.183432+0000","last_fullsized":"2024-04-22T20:45:00.183432+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:02.646044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166753+0000","last_change":"2024-04-22T20:05:17.902267+0000","last_active":"2024-04-22T20:44:00.166753+0000","last_peered":"2024-04-22T20:44:00.166753+0000","last_clean":"2024-04-22T20:44:00.166753+0000","last_became_active":"2024-04-22T20:05:17.901954+0000","last_became_peered":"2024-04-22T20:05:17.901954+0000","last_unstale":"2024-04-22T20:44:00.166753+0000","last_undegraded":"2024-04-22T20:44:00.166753+0000","last_fullsized":"2024-04-22T20:44:00.166753+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:44:04.704303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'2","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.170642+0000","last_change":"2024-04-22T20:05:13.813528+0000","last_active":"2024-04-22T20:43:55.170642+0000","last_peered":"2024-04-22T20:43:55.170642+0000","last_clean":"2024-04-22T20:43:55.170642+0000","last_became_active":"2024-04-22T20:05:13.813256+0000","last_became_peered":"2024-04-22T20:05:13.813256+0000","last_unstale":"2024-04-22T20:43:55.170642+0000","last_undegraded":"2024-04-22T20:43:55.170642+0000","last_fullsized":"2024-04-22T20:43:55.170642+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:19.323647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":44,"num_write":2,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"261'46","reported_seq":164,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.064807+0000","last_change":"2024-04-22T20:01:04.874056+0000","last_active":"2024-04-22T20:45:35.064807+0000","last_peered":"2024-04-22T20:45:35.064807+0000","last_clean":"2024-04-22T20:45:35.064807+0000","last_became_active":"2024-04-22T20:01:04.873533+0000","last_became_peered":"2024-04-22T20:01:04.873533+0000","last_unstale":"2024-04-22T20:45:35.064807+0000","last_undegraded":"2024-04-22T20:45:35.064807+0000","last_fullsized":"2024-04-22T20:45:35.064807+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:43.868445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":192,"num_read_kb":137,"num_write":82,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158918+0000","last_change":"2024-04-22T20:19:16.793148+0000","last_active":"2024-04-22T20:45:50.158918+0000","last_peered":"2024-04-22T20:45:50.158918+0000","last_clean":"2024-04-22T20:45:50.158918+0000","last_became_active":"2024-04-22T20:19:16.792856+0000","last_became_peered":"2024-04-22T20:19:16.792856+0000","last_unstale":"2024-04-22T20:45:50.158918+0000","last_undegraded":"2024-04-22T20:45:50.158918+0000","last_fullsized":"2024-04-22T20:45:50.158918+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:34.522183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'7","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041319+0000","last_change":"2024-04-22T20:05:17.897292+0000","last_active":"2024-04-22T20:45:00.041319+0000","last_peered":"2024-04-22T20:45:00.041319+0000","last_clean":"2024-04-22T20:45:00.041319+0000","last_became_active":"2024-04-22T20:05:17.897040+0000","last_became_peered":"2024-04-22T20:05:17.897040+0000","last_unstale":"2024-04-22T20:45:00.041319+0000","last_undegraded":"2024-04-22T20:45:00.041319+0000","last_fullsized":"2024-04-22T20:45:00.041319+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:13:37.748724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":12290,"num_write":6,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"261'3","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.174581+0000","last_change":"2024-04-22T20:05:13.814475+0000","last_active":"2024-04-22T20:44:55.174581+0000","last_peered":"2024-04-22T20:44:55.174581+0000","last_clean":"2024-04-22T20:44:55.174581+0000","last_became_active":"2024-04-22T20:05:13.814222+0000","last_became_peered":"2024-04-22T20:05:13.814222+0000","last_unstale":"2024-04-22T20:44:55.174581+0000","last_undegraded":"2024-04-22T20:44:55.174581+0000","last_fullsized":"2024-04-22T20:44:55.174581+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:29:09.438132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"261'54","reported_seq":190,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063228+0000","last_change":"2024-04-22T20:01:04.873752+0000","last_active":"2024-04-22T20:45:25.063228+0000","last_peered":"2024-04-22T20:45:25.063228+0000","last_clean":"2024-04-22T20:45:25.063228+0000","last_became_active":"2024-04-22T20:01:04.873422+0000","last_became_peered":"2024-04-22T20:01:04.873422+0000","last_unstale":"2024-04-22T20:45:25.063228+0000","last_undegraded":"2024-04-22T20:45:25.063228+0000","last_fullsized":"2024-04-22T20:45:25.063228+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:49.464347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":184,"num_write":82,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.167040+0000","last_change":"2024-04-22T20:19:16.795990+0000","last_active":"2024-04-22T20:43:30.167040+0000","last_peered":"2024-04-22T20:43:30.167040+0000","last_clean":"2024-04-22T20:43:30.167040+0000","last_became_active":"2024-04-22T20:19:16.795633+0000","last_became_peered":"2024-04-22T20:19:16.795633+0000","last_unstale":"2024-04-22T20:43:30.167040+0000","last_undegraded":"2024-04-22T20:43:30.167040+0000","last_fullsized":"2024-04-22T20:43:30.167040+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:55.584364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"261'8","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041286+0000","last_change":"2024-04-22T20:05:17.900013+0000","last_active":"2024-04-22T20:45:00.041286+0000","last_peered":"2024-04-22T20:45:00.041286+0000","last_clean":"2024-04-22T20:45:00.041286+0000","last_became_active":"2024-04-22T20:05:17.899682+0000","last_became_peered":"2024-04-22T20:05:17.899682+0000","last_unstale":"2024-04-22T20:45:00.041286+0000","last_undegraded":"2024-04-22T20:45:00.041286+0000","last_fullsized":"2024-04-22T20:45:00.041286+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:57:08.679442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":8210,"num_write":12,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"261'5","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.152411+0000","last_change":"2024-04-22T20:05:13.813246+0000","last_active":"2024-04-22T20:44:55.152411+0000","last_peered":"2024-04-22T20:44:55.152411+0000","last_clean":"2024-04-22T20:44:55.152411+0000","last_became_active":"2024-04-22T20:05:13.812893+0000","last_became_peered":"2024-04-22T20:05:13.812893+0000","last_unstale":"2024-04-22T20:44:55.152411+0000","last_undegraded":"2024-04-22T20:44:55.152411+0000","last_fullsized":"2024-04-22T20:44:55.152411+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:26:36.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":15,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"261'47","reported_seq":163,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.181140+0000","last_change":"2024-04-22T20:01:04.878065+0000","last_active":"2024-04-22T20:45:25.181140+0000","last_peered":"2024-04-22T20:45:25.181140+0000","last_clean":"2024-04-22T20:45:25.181140+0000","last_became_active":"2024-04-22T20:01:04.877755+0000","last_became_peered":"2024-04-22T20:01:04.877755+0000","last_unstale":"2024-04-22T20:45:25.181140+0000","last_undegraded":"2024-04-22T20:45:25.181140+0000","last_fullsized":"2024-04-22T20:45:25.181140+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:41:43.369530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":135,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"260'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.178798+0000","last_change":"2024-04-22T20:19:16.793874+0000","last_active":"2024-04-22T20:45:05.178798+0000","last_peered":"2024-04-22T20:45:05.178798+0000","last_clean":"2024-04-22T20:45:05.178798+0000","last_became_active":"2024-04-22T20:19:16.793555+0000","last_became_peered":"2024-04-22T20:19:16.793555+0000","last_unstale":"2024-04-22T20:45:05.178798+0000","last_undegraded":"2024-04-22T20:45:05.178798+0000","last_fullsized":"2024-04-22T20:45:05.178798+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:33:04.108646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"258'2","reported_seq":44,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.152971+0000","last_change":"2024-04-22T20:05:17.890318+0000","last_active":"2024-04-22T20:45:00.152971+0000","last_peered":"2024-04-22T20:45:00.152971+0000","last_clean":"2024-04-22T20:45:00.152971+0000","last_became_active":"2024-04-22T20:05:17.890070+0000","last_became_peered":"2024-04-22T20:05:17.890070+0000","last_unstale":"2024-04-22T20:45:00.152971+0000","last_undegraded":"2024-04-22T20:45:00.152971+0000","last_fullsized":"2024-04-22T20:45:00.152971+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:14:41.324774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"261'3","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.310559+0000","last_change":"2024-04-22T20:05:13.807133+0000","last_active":"2024-04-22T20:44:55.310559+0000","last_peered":"2024-04-22T20:44:55.310559+0000","last_clean":"2024-04-22T20:44:55.310559+0000","last_became_active":"2024-04-22T20:05:13.806882+0000","last_became_peered":"2024-04-22T20:05:13.806882+0000","last_unstale":"2024-04-22T20:44:55.310559+0000","last_undegraded":"2024-04-22T20:44:55.310559+0000","last_fullsized":"2024-04-22T20:44:55.310559+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:52:50.911604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"261'11","reported_seq":80,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.178877+0000","last_change":"2024-04-22T20:01:04.861370+0000","last_active":"2024-04-22T20:45:25.178877+0000","last_peered":"2024-04-22T20:45:25.178877+0000","last_clean":"2024-04-22T20:45:25.178877+0000","last_became_active":"2024-04-22T20:01:04.860991+0000","last_became_peered":"2024-04-22T20:01:04.860991+0000","last_unstale":"2024-04-22T20:45:25.178877+0000","last_undegraded":"2024-04-22T20:45:25.178877+0000","last_fullsized":"2024-04-22T20:45:25.178877+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:26:49.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":31,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158885+0000","last_change":"2024-04-22T20:19:16.794789+0000","last_active":"2024-04-22T20:45:50.158885+0000","last_peered":"2024-04-22T20:45:50.158885+0000","last_clean":"2024-04-22T20:45:50.158885+0000","last_became_active":"2024-04-22T20:19:16.791995+0000","last_became_peered":"2024-04-22T20:19:16.791995+0000","last_unstale":"2024-04-22T20:45:50.158885+0000","last_undegraded":"2024-04-22T20:45:50.158885+0000","last_fullsized":"2024-04-22T20:45:50.158885+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:04:16.246591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"258'4","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.311157+0000","last_change":"2024-04-22T20:05:17.890780+0000","last_active":"2024-04-22T20:45:00.311157+0000","last_peered":"2024-04-22T20:45:00.311157+0000","last_clean":"2024-04-22T20:45:00.311157+0000","last_became_active":"2024-04-22T20:05:17.890408+0000","last_became_peered":"2024-04-22T20:05:17.890408+0000","last_unstale":"2024-04-22T20:45:00.311157+0000","last_undegraded":"2024-04-22T20:45:00.311157+0000","last_fullsized":"2024-04-22T20:45:00.311157+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:30:52.042989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1342,"num_write":4,"num_write_kb":1342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"258'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.182683+0000","last_change":"2024-04-22T20:05:13.808782+0000","last_active":"2024-04-22T20:44:55.182683+0000","last_peered":"2024-04-22T20:44:55.182683+0000","last_clean":"2024-04-22T20:44:55.182683+0000","last_became_active":"2024-04-22T20:05:13.808473+0000","last_became_peered":"2024-04-22T20:05:13.808473+0000","last_unstale":"2024-04-22T20:44:55.182683+0000","last_undegraded":"2024-04-22T20:44:55.182683+0000","last_fullsized":"2024-04-22T20:44:55.182683+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:39:30.056682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"261'46","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.155908+0000","last_change":"2024-04-22T20:01:04.875088+0000","last_active":"2024-04-22T20:45:25.155908+0000","last_peered":"2024-04-22T20:45:25.155908+0000","last_clean":"2024-04-22T20:45:25.155908+0000","last_became_active":"2024-04-22T20:01:04.874816+0000","last_became_peered":"2024-04-22T20:01:04.874816+0000","last_unstale":"2024-04-22T20:45:25.155908+0000","last_undegraded":"2024-04-22T20:45:25.155908+0000","last_fullsized":"2024-04-22T20:45:25.155908+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:59:09.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":291,"num_read_kb":200,"num_write":62,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.066966+0000","last_change":"2024-04-22T20:19:16.795282+0000","last_active":"2024-04-22T20:45:25.066966+0000","last_peered":"2024-04-22T20:45:25.066966+0000","last_clean":"2024-04-22T20:45:25.066966+0000","last_became_active":"2024-04-22T20:19:16.794954+0000","last_became_peered":"2024-04-22T20:19:16.794954+0000","last_unstale":"2024-04-22T20:45:25.066966+0000","last_undegraded":"2024-04-22T20:45:25.066966+0000","last_fullsized":"2024-04-22T20:45:25.066966+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:53:00.610478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"261'3","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158894+0000","last_change":"2024-04-22T20:05:17.889647+0000","last_active":"2024-04-22T20:45:50.158894+0000","last_peered":"2024-04-22T20:45:50.158894+0000","last_clean":"2024-04-22T20:45:50.158894+0000","last_became_active":"2024-04-22T20:05:17.889370+0000","last_became_peered":"2024-04-22T20:05:17.889370+0000","last_unstale":"2024-04-22T20:45:50.158894+0000","last_undegraded":"2024-04-22T20:45:50.158894+0000","last_fullsized":"2024-04-22T20:45:50.158894+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:26:55.859900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5246080,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":4100,"num_write":3,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"261'1","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.066445+0000","last_change":"2024-04-22T20:05:13.815613+0000","last_active":"2024-04-22T20:45:45.066445+0000","last_peered":"2024-04-22T20:45:45.066445+0000","last_clean":"2024-04-22T20:45:45.066445+0000","last_became_active":"2024-04-22T20:05:13.815317+0000","last_became_peered":"2024-04-22T20:05:13.815317+0000","last_unstale":"2024-04-22T20:45:45.066445+0000","last_undegraded":"2024-04-22T20:45:45.066445+0000","last_fullsized":"2024-04-22T20:45:45.066445+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:25:12.295293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"261'28","reported_seq":118,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.145012+0000","last_change":"2024-04-22T20:01:04.878886+0000","last_active":"2024-04-22T20:43:55.145012+0000","last_peered":"2024-04-22T20:43:55.145012+0000","last_clean":"2024-04-22T20:43:55.145012+0000","last_became_active":"2024-04-22T20:01:04.878599+0000","last_became_peered":"2024-04-22T20:01:04.878599+0000","last_unstale":"2024-04-22T20:43:55.145012+0000","last_undegraded":"2024-04-22T20:43:55.145012+0000","last_fullsized":"2024-04-22T20:43:55.145012+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:57:43.301162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":77,"num_write":54,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.050575+0000","last_change":"2024-04-22T20:19:16.794996+0000","last_active":"2024-04-22T20:44:05.050575+0000","last_peered":"2024-04-22T20:44:05.050575+0000","last_clean":"2024-04-22T20:44:05.050575+0000","last_became_active":"2024-04-22T20:19:16.793678+0000","last_became_peered":"2024-04-22T20:19:16.793678+0000","last_unstale":"2024-04-22T20:44:05.050575+0000","last_undegraded":"2024-04-22T20:44:05.050575+0000","last_fullsized":"2024-04-22T20:44:05.050575+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:48:13.172887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158865+0000","last_change":"2024-04-22T20:05:17.892875+0000","last_active":"2024-04-22T20:45:50.158865+0000","last_peered":"2024-04-22T20:45:50.158865+0000","last_clean":"2024-04-22T20:45:50.158865+0000","last_became_active":"2024-04-22T20:05:17.892604+0000","last_became_peered":"2024-04-22T20:05:17.892604+0000","last_unstale":"2024-04-22T20:45:50.158865+0000","last_undegraded":"2024-04-22T20:45:50.158865+0000","last_fullsized":"2024-04-22T20:45:50.158865+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:20.757720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"261'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.068955+0000","last_change":"2024-04-22T20:05:13.814193+0000","last_active":"2024-04-22T20:45:45.068955+0000","last_peered":"2024-04-22T20:45:45.068955+0000","last_clean":"2024-04-22T20:45:45.068955+0000","last_became_active":"2024-04-22T20:05:13.813946+0000","last_became_peered":"2024-04-22T20:05:13.813946+0000","last_unstale":"2024-04-22T20:45:45.068955+0000","last_undegraded":"2024-04-22T20:45:45.068955+0000","last_fullsized":"2024-04-22T20:45:45.068955+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:04:48.745503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":15362,"num_write":8,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"261'28","reported_seq":126,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.173638+0000","last_change":"2024-04-22T20:01:04.873158+0000","last_active":"2024-04-22T20:43:50.173638+0000","last_peered":"2024-04-22T20:43:50.173638+0000","last_clean":"2024-04-22T20:43:50.173638+0000","last_became_active":"2024-04-22T20:01:04.872834+0000","last_became_peered":"2024-04-22T20:01:04.872834+0000","last_unstale":"2024-04-22T20:43:50.173638+0000","last_undegraded":"2024-04-22T20:43:50.173638+0000","last_fullsized":"2024-04-22T20:43:50.173638+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:25.675744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":90,"num_write":50,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.313962+0000","last_change":"2024-04-22T20:19:16.791183+0000","last_active":"2024-04-22T20:45:20.313962+0000","last_peered":"2024-04-22T20:45:20.313962+0000","last_clean":"2024-04-22T20:45:20.313962+0000","last_became_active":"2024-04-22T20:19:16.790917+0000","last_became_peered":"2024-04-22T20:19:16.790917+0000","last_unstale":"2024-04-22T20:45:20.313962+0000","last_undegraded":"2024-04-22T20:45:20.313962+0000","last_fullsized":"2024-04-22T20:45:20.313962+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:12:37.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:00.048158+0000","last_change":"2024-04-22T20:05:18.211747+0000","last_active":"2024-04-22T20:46:00.048158+0000","last_peered":"2024-04-22T20:46:00.048158+0000","last_clean":"2024-04-22T20:46:00.048158+0000","last_became_active":"2024-04-22T20:05:18.211445+0000","last_became_peered":"2024-04-22T20:05:18.211445+0000","last_unstale":"2024-04-22T20:46:00.048158+0000","last_undegraded":"2024-04-22T20:46:00.048158+0000","last_fullsized":"2024-04-22T20:46:00.048158+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:14:07.558996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"258'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183750+0000","last_change":"2024-04-22T20:05:13.809343+0000","last_active":"2024-04-22T20:45:45.183750+0000","last_peered":"2024-04-22T20:45:45.183750+0000","last_clean":"2024-04-22T20:45:45.183750+0000","last_became_active":"2024-04-22T20:05:13.806928+0000","last_became_peered":"2024-04-22T20:05:13.806928+0000","last_unstale":"2024-04-22T20:45:45.183750+0000","last_undegraded":"2024-04-22T20:45:45.183750+0000","last_fullsized":"2024-04-22T20:45:45.183750+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:36:20.049785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"261'28","reported_seq":117,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.301681+0000","last_change":"2024-04-22T20:01:04.869517+0000","last_active":"2024-04-22T20:43:50.301681+0000","last_peered":"2024-04-22T20:43:50.301681+0000","last_clean":"2024-04-22T20:43:50.301681+0000","last_became_active":"2024-04-22T20:01:04.869226+0000","last_became_peered":"2024-04-22T20:01:04.869226+0000","last_unstale":"2024-04-22T20:43:50.301681+0000","last_undegraded":"2024-04-22T20:43:50.301681+0000","last_fullsized":"2024-04-22T20:43:50.301681+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:02:16.025206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":80,"num_write":56,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062354+0000","last_change":"2024-04-22T20:19:16.796398+0000","last_active":"2024-04-22T20:45:20.062354+0000","last_peered":"2024-04-22T20:45:20.062354+0000","last_clean":"2024-04-22T20:45:20.062354+0000","last_became_active":"2024-04-22T20:19:16.795524+0000","last_became_peered":"2024-04-22T20:19:16.795524+0000","last_unstale":"2024-04-22T20:45:20.062354+0000","last_undegraded":"2024-04-22T20:45:20.062354+0000","last_fullsized":"2024-04-22T20:45:20.062354+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:24:50.852300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158909+0000","last_change":"2024-04-22T20:05:17.899183+0000","last_active":"2024-04-22T20:45:50.158909+0000","last_peered":"2024-04-22T20:45:50.158909+0000","last_clean":"2024-04-22T20:45:50.158909+0000","last_became_active":"2024-04-22T20:05:17.898858+0000","last_became_peered":"2024-04-22T20:05:17.898858+0000","last_unstale":"2024-04-22T20:45:50.158909+0000","last_undegraded":"2024-04-22T20:45:50.158909+0000","last_fullsized":"2024-04-22T20:45:50.158909+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:21.640059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"258'3","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.047547+0000","last_change":"2024-04-22T20:05:13.808403+0000","last_active":"2024-04-22T20:45:55.047547+0000","last_peered":"2024-04-22T20:45:55.047547+0000","last_clean":"2024-04-22T20:45:55.047547+0000","last_became_active":"2024-04-22T20:05:13.807945+0000","last_became_peered":"2024-04-22T20:05:13.807945+0000","last_unstale":"2024-04-22T20:45:55.047547+0000","last_undegraded":"2024-04-22T20:45:55.047547+0000","last_fullsized":"2024-04-22T20:45:55.047547+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:02:06.578487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":8,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"261'24","reported_seq":105,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.057658+0000","last_change":"2024-04-22T20:01:04.872099+0000","last_active":"2024-04-22T20:43:50.057658+0000","last_peered":"2024-04-22T20:43:50.057658+0000","last_clean":"2024-04-22T20:43:50.057658+0000","last_became_active":"2024-04-22T20:01:04.871829+0000","last_became_peered":"2024-04-22T20:01:04.871829+0000","last_unstale":"2024-04-22T20:43:50.057658+0000","last_undegraded":"2024-04-22T20:43:50.057658+0000","last_fullsized":"2024-04-22T20:43:50.057658+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:30:28.808258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":61,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145629+0000","last_change":"2024-04-22T20:19:16.794419+0000","last_active":"2024-04-22T20:44:00.145629+0000","last_peered":"2024-04-22T20:44:00.145629+0000","last_clean":"2024-04-22T20:44:00.145629+0000","last_became_active":"2024-04-22T20:19:16.793727+0000","last_became_peered":"2024-04-22T20:19:16.793727+0000","last_unstale":"2024-04-22T20:44:00.145629+0000","last_undegraded":"2024-04-22T20:44:00.145629+0000","last_fullsized":"2024-04-22T20:44:00.145629+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:50.871340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"261'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169594+0000","last_change":"2024-04-22T20:05:17.899691+0000","last_active":"2024-04-22T20:44:20.169594+0000","last_peered":"2024-04-22T20:44:20.169594+0000","last_clean":"2024-04-22T20:44:20.169594+0000","last_became_active":"2024-04-22T20:05:17.899328+0000","last_became_peered":"2024-04-22T20:05:17.899328+0000","last_unstale":"2024-04-22T20:44:20.169594+0000","last_undegraded":"2024-04-22T20:44:20.169594+0000","last_fullsized":"2024-04-22T20:44:20.169594+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:57:36.265424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":9,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.172707+0000","last_change":"2024-04-22T20:05:13.812566+0000","last_active":"2024-04-22T20:44:15.172707+0000","last_peered":"2024-04-22T20:44:15.172707+0000","last_clean":"2024-04-22T20:44:15.172707+0000","last_became_active":"2024-04-22T20:05:13.811986+0000","last_became_peered":"2024-04-22T20:05:13.811986+0000","last_unstale":"2024-04-22T20:44:15.172707+0000","last_undegraded":"2024-04-22T20:44:15.172707+0000","last_fullsized":"2024-04-22T20:44:15.172707+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:29.867735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5291296,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":8240,"num_write":9,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"261'66","reported_seq":211,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:06.776789+0000","last_change":"2024-04-22T20:01:04.875092+0000","last_active":"2024-04-22T20:44:06.776789+0000","last_peered":"2024-04-22T20:44:06.776789+0000","last_clean":"2024-04-22T20:44:06.776789+0000","last_became_active":"2024-04-22T20:01:04.874827+0000","last_became_peered":"2024-04-22T20:01:04.874827+0000","last_unstale":"2024-04-22T20:44:06.776789+0000","last_undegraded":"2024-04-22T20:44:06.776789+0000","last_fullsized":"2024-04-22T20:44:06.776789+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:49.143833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":281,"num_read_kb":200,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.052345+0000","last_change":"2024-04-22T20:19:16.794759+0000","last_active":"2024-04-22T20:44:15.052345+0000","last_peered":"2024-04-22T20:44:15.052345+0000","last_clean":"2024-04-22T20:44:15.052345+0000","last_became_active":"2024-04-22T20:19:16.794434+0000","last_became_peered":"2024-04-22T20:19:16.794434+0000","last_unstale":"2024-04-22T20:44:15.052345+0000","last_undegraded":"2024-04-22T20:44:15.052345+0000","last_fullsized":"2024-04-22T20:44:15.052345+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:47.954843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"261'6","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.052938+0000","last_change":"2024-04-22T20:05:18.209353+0000","last_active":"2024-04-22T20:44:20.052938+0000","last_peered":"2024-04-22T20:44:20.052938+0000","last_clean":"2024-04-22T20:44:20.052938+0000","last_became_active":"2024-04-22T20:05:18.209025+0000","last_became_peered":"2024-04-22T20:05:18.209025+0000","last_unstale":"2024-04-22T20:44:20.052938+0000","last_undegraded":"2024-04-22T20:44:20.052938+0000","last_fullsized":"2024-04-22T20:44:20.052938+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:14:29.836329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":13312,"num_write":11,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"261'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.036411+0000","last_change":"2024-04-22T20:05:13.807877+0000","last_active":"2024-04-22T20:44:15.036411+0000","last_peered":"2024-04-22T20:44:15.036411+0000","last_clean":"2024-04-22T20:44:15.036411+0000","last_became_active":"2024-04-22T20:05:13.807603+0000","last_became_peered":"2024-04-22T20:05:13.807603+0000","last_unstale":"2024-04-22T20:44:15.036411+0000","last_undegraded":"2024-04-22T20:44:15.036411+0000","last_fullsized":"2024-04-22T20:44:15.036411+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:05:08.359131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"258'23","reported_seq":112,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.055220+0000","last_change":"2024-04-22T20:01:04.869170+0000","last_active":"2024-04-22T20:44:35.055220+0000","last_peered":"2024-04-22T20:44:35.055220+0000","last_clean":"2024-04-22T20:44:35.055220+0000","last_became_active":"2024-04-22T20:01:04.868847+0000","last_became_peered":"2024-04-22T20:01:04.868847+0000","last_unstale":"2024-04-22T20:44:35.055220+0000","last_undegraded":"2024-04-22T20:44:35.055220+0000","last_fullsized":"2024-04-22T20:44:35.055220+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:17:28.313341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":76,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.050611+0000","last_change":"2024-04-22T20:19:16.796622+0000","last_active":"2024-04-22T20:44:05.050611+0000","last_peered":"2024-04-22T20:44:05.050611+0000","last_clean":"2024-04-22T20:44:05.050611+0000","last_became_active":"2024-04-22T20:19:16.795613+0000","last_became_peered":"2024-04-22T20:19:16.795613+0000","last_unstale":"2024-04-22T20:44:05.050611+0000","last_undegraded":"2024-04-22T20:44:05.050611+0000","last_fullsized":"2024-04-22T20:44:05.050611+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:22:45.476024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"261'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.047029+0000","last_change":"2024-04-22T20:05:17.891585+0000","last_active":"2024-04-22T20:45:50.047029+0000","last_peered":"2024-04-22T20:45:50.047029+0000","last_clean":"2024-04-22T20:45:50.047029+0000","last_became_active":"2024-04-22T20:05:17.891331+0000","last_became_peered":"2024-04-22T20:05:17.891331+0000","last_unstale":"2024-04-22T20:45:50.047029+0000","last_undegraded":"2024-04-22T20:45:50.047029+0000","last_fullsized":"2024-04-22T20:45:50.047029+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:10.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":14,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.173996+0000","last_change":"2024-04-22T20:05:13.814072+0000","last_active":"2024-04-22T20:44:25.173996+0000","last_peered":"2024-04-22T20:44:25.173996+0000","last_clean":"2024-04-22T20:44:25.173996+0000","last_became_active":"2024-04-22T20:05:13.813811+0000","last_became_peered":"2024-04-22T20:05:13.813811+0000","last_unstale":"2024-04-22T20:44:25.173996+0000","last_undegraded":"2024-04-22T20:44:25.173996+0000","last_fullsized":"2024-04-22T20:44:25.173996+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:57.039895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"258'49","reported_seq":170,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.148654+0000","last_change":"2024-04-22T20:01:04.874422+0000","last_active":"2024-04-22T20:44:25.148654+0000","last_peered":"2024-04-22T20:44:25.148654+0000","last_clean":"2024-04-22T20:44:25.148654+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:44:25.148654+0000","last_undegraded":"2024-04-22T20:44:25.148654+0000","last_fullsized":"2024-04-22T20:44:25.148654+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:56:32.298685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":145,"num_write":78,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.171678+0000","last_change":"2024-04-22T20:19:16.793813+0000","last_active":"2024-04-22T20:44:05.171678+0000","last_peered":"2024-04-22T20:44:05.171678+0000","last_clean":"2024-04-22T20:44:05.171678+0000","last_became_active":"2024-04-22T20:19:16.793486+0000","last_became_peered":"2024-04-22T20:19:16.793486+0000","last_unstale":"2024-04-22T20:44:05.171678+0000","last_undegraded":"2024-04-22T20:44:05.171678+0000","last_fullsized":"2024-04-22T20:44:05.171678+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:47:12.878940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:30.054561+0000","last_change":"2024-04-22T20:05:17.897286+0000","last_active":"2024-04-22T20:44:30.054561+0000","last_peered":"2024-04-22T20:44:30.054561+0000","last_clean":"2024-04-22T20:44:30.054561+0000","last_became_active":"2024-04-22T20:05:17.897008+0000","last_became_peered":"2024-04-22T20:05:17.897008+0000","last_unstale":"2024-04-22T20:44:30.054561+0000","last_undegraded":"2024-04-22T20:44:30.054561+0000","last_fullsized":"2024-04-22T20:44:30.054561+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:56:58.848648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"261'11","reported_seq":72,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182061+0000","last_change":"2024-04-22T20:05:13.798707+0000","last_active":"2024-04-22T20:45:45.182061+0000","last_peered":"2024-04-22T20:45:45.182061+0000","last_clean":"2024-04-22T20:45:45.182061+0000","last_became_active":"2024-04-22T20:05:13.798319+0000","last_became_peered":"2024-04-22T20:05:13.798319+0000","last_unstale":"2024-04-22T20:45:45.182061+0000","last_undegraded":"2024-04-22T20:45:45.182061+0000","last_fullsized":"2024-04-22T20:45:45.182061+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:02.215898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":20,"num_read_kb":16388,"num_write":14,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"261'37","reported_seq":138,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.306289+0000","last_change":"2024-04-22T20:01:04.869368+0000","last_active":"2024-04-22T20:44:25.306289+0000","last_peered":"2024-04-22T20:44:25.306289+0000","last_clean":"2024-04-22T20:44:25.306289+0000","last_became_active":"2024-04-22T20:01:04.869079+0000","last_became_peered":"2024-04-22T20:01:04.869079+0000","last_unstale":"2024-04-22T20:44:25.306289+0000","last_undegraded":"2024-04-22T20:44:25.306289+0000","last_fullsized":"2024-04-22T20:44:25.306289+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:36:35.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":61,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053026+0000","last_change":"2024-04-22T20:19:16.796317+0000","last_active":"2024-04-22T20:44:20.053026+0000","last_peered":"2024-04-22T20:44:20.053026+0000","last_clean":"2024-04-22T20:44:20.053026+0000","last_became_active":"2024-04-22T20:19:16.795462+0000","last_became_peered":"2024-04-22T20:19:16.795462+0000","last_unstale":"2024-04-22T20:44:20.053026+0000","last_undegraded":"2024-04-22T20:44:20.053026+0000","last_fullsized":"2024-04-22T20:44:20.053026+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:38:48.277835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"258'6","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.178822+0000","last_change":"2024-04-22T20:05:17.892351+0000","last_active":"2024-04-22T20:45:05.178822+0000","last_peered":"2024-04-22T20:45:05.178822+0000","last_clean":"2024-04-22T20:45:05.178822+0000","last_became_active":"2024-04-22T20:05:17.892044+0000","last_became_peered":"2024-04-22T20:05:17.892044+0000","last_unstale":"2024-04-22T20:45:05.178822+0000","last_undegraded":"2024-04-22T20:45:05.178822+0000","last_fullsized":"2024-04-22T20:45:05.178822+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:47:37.172131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"258'4","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057775+0000","last_change":"2024-04-22T20:05:13.809668+0000","last_active":"2024-04-22T20:44:50.057775+0000","last_peered":"2024-04-22T20:44:50.057775+0000","last_clean":"2024-04-22T20:44:50.057775+0000","last_became_active":"2024-04-22T20:05:13.809368+0000","last_became_peered":"2024-04-22T20:05:13.809368+0000","last_unstale":"2024-04-22T20:44:50.057775+0000","last_undegraded":"2024-04-22T20:44:50.057775+0000","last_fullsized":"2024-04-22T20:44:50.057775+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:35.986772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"261'35","reported_seq":138,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.174074+0000","last_change":"2024-04-22T20:01:04.874032+0000","last_active":"2024-04-22T20:44:25.174074+0000","last_peered":"2024-04-22T20:44:25.174074+0000","last_clean":"2024-04-22T20:44:25.174074+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:44:25.174074+0000","last_undegraded":"2024-04-22T20:44:25.174074+0000","last_fullsized":"2024-04-22T20:44:25.174074+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:44.638764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":59,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.177834+0000","last_change":"2024-04-22T20:19:16.791940+0000","last_active":"2024-04-22T20:44:20.177834+0000","last_peered":"2024-04-22T20:44:20.177834+0000","last_clean":"2024-04-22T20:44:20.177834+0000","last_became_active":"2024-04-22T20:19:16.791663+0000","last_became_peered":"2024-04-22T20:19:16.791663+0000","last_unstale":"2024-04-22T20:44:20.177834+0000","last_undegraded":"2024-04-22T20:44:20.177834+0000","last_fullsized":"2024-04-22T20:44:20.177834+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:35:00.807867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"261'5","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.177498+0000","last_change":"2024-04-22T20:05:18.211014+0000","last_active":"2024-04-22T20:44:55.177498+0000","last_peered":"2024-04-22T20:44:55.177498+0000","last_clean":"2024-04-22T20:44:55.177498+0000","last_became_active":"2024-04-22T20:05:18.210736+0000","last_became_peered":"2024-04-22T20:05:18.210736+0000","last_unstale":"2024-04-22T20:44:55.177498+0000","last_undegraded":"2024-04-22T20:44:55.177498+0000","last_fullsized":"2024-04-22T20:44:55.177498+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:55:21.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":20482,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"261'5","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.064529+0000","last_change":"2024-04-22T20:05:13.813554+0000","last_active":"2024-04-22T20:45:00.064529+0000","last_peered":"2024-04-22T20:45:00.064529+0000","last_clean":"2024-04-22T20:45:00.064529+0000","last_became_active":"2024-04-22T20:05:13.813277+0000","last_became_peered":"2024-04-22T20:05:13.813277+0000","last_unstale":"2024-04-22T20:45:00.064529+0000","last_undegraded":"2024-04-22T20:45:00.064529+0000","last_fullsized":"2024-04-22T20:45:00.064529+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:43:53.225799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":15,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"261'61","reported_seq":194,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.306363+0000","last_change":"2024-04-22T20:01:04.872160+0000","last_active":"2024-04-22T20:44:25.306363+0000","last_peered":"2024-04-22T20:44:25.306363+0000","last_clean":"2024-04-22T20:44:25.306363+0000","last_became_active":"2024-04-22T20:01:04.871859+0000","last_became_peered":"2024-04-22T20:01:04.871859+0000","last_unstale":"2024-04-22T20:44:25.306363+0000","last_undegraded":"2024-04-22T20:44:25.306363+0000","last_fullsized":"2024-04-22T20:44:25.306363+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:17.093967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":172,"num_write":109,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.044253+0000","last_change":"2024-04-22T20:19:16.790599+0000","last_active":"2024-04-22T20:45:25.044253+0000","last_peered":"2024-04-22T20:45:25.044253+0000","last_clean":"2024-04-22T20:45:25.044253+0000","last_became_active":"2024-04-22T20:19:16.789688+0000","last_became_peered":"2024-04-22T20:19:16.789688+0000","last_unstale":"2024-04-22T20:45:25.044253+0000","last_undegraded":"2024-04-22T20:45:25.044253+0000","last_fullsized":"2024-04-22T20:45:25.044253+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:11.661623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"258'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053135+0000","last_change":"2024-04-22T20:05:18.211736+0000","last_active":"2024-04-22T20:44:20.053135+0000","last_peered":"2024-04-22T20:44:20.053135+0000","last_clean":"2024-04-22T20:44:20.053135+0000","last_became_active":"2024-04-22T20:05:18.211478+0000","last_became_peered":"2024-04-22T20:05:18.211478+0000","last_unstale":"2024-04-22T20:44:20.053135+0000","last_undegraded":"2024-04-22T20:44:20.053135+0000","last_fullsized":"2024-04-22T20:44:20.053135+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:06:10.866642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.177134+0000","last_change":"2024-04-22T20:05:13.811784+0000","last_active":"2024-04-22T20:44:15.177134+0000","last_peered":"2024-04-22T20:44:15.177134+0000","last_clean":"2024-04-22T20:44:15.177134+0000","last_became_active":"2024-04-22T20:05:13.811561+0000","last_became_peered":"2024-04-22T20:05:13.811561+0000","last_unstale":"2024-04-22T20:44:15.177134+0000","last_undegraded":"2024-04-22T20:44:15.177134+0000","last_fullsized":"2024-04-22T20:44:15.177134+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:35:04.990469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":9,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"261'33","reported_seq":135,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.059415+0000","last_change":"2024-04-22T20:01:04.875229+0000","last_active":"2024-04-22T20:45:00.059415+0000","last_peered":"2024-04-22T20:45:00.059415+0000","last_clean":"2024-04-22T20:45:00.059415+0000","last_became_active":"2024-04-22T20:01:04.874790+0000","last_became_peered":"2024-04-22T20:01:04.874790+0000","last_unstale":"2024-04-22T20:45:00.059415+0000","last_undegraded":"2024-04-22T20:45:00.059415+0000","last_fullsized":"2024-04-22T20:45:00.059415+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:48.031491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":100,"num_write":57,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169578+0000","last_change":"2024-04-22T20:19:16.794065+0000","last_active":"2024-04-22T20:44:20.169578+0000","last_peered":"2024-04-22T20:44:20.169578+0000","last_clean":"2024-04-22T20:44:20.169578+0000","last_became_active":"2024-04-22T20:19:16.793188+0000","last_became_peered":"2024-04-22T20:19:16.793188+0000","last_unstale":"2024-04-22T20:44:20.169578+0000","last_undegraded":"2024-04-22T20:44:20.169578+0000","last_fullsized":"2024-04-22T20:44:20.169578+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:39.203315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"261'3","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169675+0000","last_change":"2024-04-22T20:05:17.901392+0000","last_active":"2024-04-22T20:44:20.169675+0000","last_peered":"2024-04-22T20:44:20.169675+0000","last_clean":"2024-04-22T20:44:20.169675+0000","last_became_active":"2024-04-22T20:05:17.901128+0000","last_became_peered":"2024-04-22T20:05:17.901128+0000","last_unstale":"2024-04-22T20:44:20.169675+0000","last_undegraded":"2024-04-22T20:44:20.169675+0000","last_fullsized":"2024-04-22T20:44:20.169675+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:50:08.370833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":8,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"261'4","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.052367+0000","last_change":"2024-04-22T20:05:13.815731+0000","last_active":"2024-04-22T20:44:15.052367+0000","last_peered":"2024-04-22T20:44:15.052367+0000","last_clean":"2024-04-22T20:44:15.052367+0000","last_became_active":"2024-04-22T20:05:13.815332+0000","last_became_peered":"2024-04-22T20:05:13.815332+0000","last_unstale":"2024-04-22T20:44:15.052367+0000","last_undegraded":"2024-04-22T20:44:15.052367+0000","last_fullsized":"2024-04-22T20:44:15.052367+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:07:16.209328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":9216,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'28","reported_seq":124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040298+0000","last_change":"2024-04-22T20:01:04.872148+0000","last_active":"2024-04-22T20:44:50.040298+0000","last_peered":"2024-04-22T20:44:50.040298+0000","last_clean":"2024-04-22T20:44:50.040298+0000","last_became_active":"2024-04-22T20:01:04.871840+0000","last_became_peered":"2024-04-22T20:01:04.871840+0000","last_unstale":"2024-04-22T20:44:50.040298+0000","last_undegraded":"2024-04-22T20:44:50.040298+0000","last_fullsized":"2024-04-22T20:44:50.040298+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:23.933273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":89,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"261'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.066551+0000","last_change":"2024-04-22T20:19:16.794411+0000","last_active":"2024-04-22T20:45:20.066551+0000","last_peered":"2024-04-22T20:45:20.066551+0000","last_clean":"2024-04-22T20:45:20.066551+0000","last_became_active":"2024-04-22T20:19:16.794123+0000","last_became_peered":"2024-04-22T20:19:16.794123+0000","last_unstale":"2024-04-22T20:45:20.066551+0000","last_undegraded":"2024-04-22T20:45:20.066551+0000","last_fullsized":"2024-04-22T20:45:20.066551+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:39.176302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"261'2","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.040820+0000","last_change":"2024-04-22T20:05:17.895539+0000","last_active":"2024-04-22T20:44:55.040820+0000","last_peered":"2024-04-22T20:44:55.040820+0000","last_clean":"2024-04-22T20:44:55.040820+0000","last_became_active":"2024-04-22T20:05:17.895177+0000","last_became_peered":"2024-04-22T20:05:17.895177+0000","last_unstale":"2024-04-22T20:44:55.040820+0000","last_undegraded":"2024-04-22T20:44:55.040820+0000","last_fullsized":"2024-04-22T20:44:55.040820+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:46:34.641098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1024,"num_write":2,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"261'2","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063150+0000","last_change":"2024-04-22T20:05:13.809357+0000","last_active":"2024-04-22T20:45:25.063150+0000","last_peered":"2024-04-22T20:45:25.063150+0000","last_clean":"2024-04-22T20:45:25.063150+0000","last_became_active":"2024-04-22T20:05:13.809047+0000","last_became_peered":"2024-04-22T20:05:13.809047+0000","last_unstale":"2024-04-22T20:45:25.063150+0000","last_undegraded":"2024-04-22T20:45:25.063150+0000","last_fullsized":"2024-04-22T20:45:25.063150+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:17.206006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":2,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"261'33","reported_seq":130,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.181926+0000","last_change":"2024-04-22T20:01:04.872953+0000","last_active":"2024-04-22T20:44:50.181926+0000","last_peered":"2024-04-22T20:44:50.181926+0000","last_clean":"2024-04-22T20:44:50.181926+0000","last_became_active":"2024-04-22T20:01:04.872370+0000","last_became_peered":"2024-04-22T20:01:04.872370+0000","last_unstale":"2024-04-22T20:44:50.181926+0000","last_undegraded":"2024-04-22T20:44:50.181926+0000","last_fullsized":"2024-04-22T20:44:50.181926+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:22:37.771203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":57,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.180655+0000","last_change":"2024-04-22T20:19:16.793728+0000","last_active":"2024-04-22T20:45:20.180655+0000","last_peered":"2024-04-22T20:45:20.180655+0000","last_clean":"2024-04-22T20:45:20.180655+0000","last_became_active":"2024-04-22T20:19:16.793429+0000","last_became_peered":"2024-04-22T20:19:16.793429+0000","last_unstale":"2024-04-22T20:45:20.180655+0000","last_undegraded":"2024-04-22T20:45:20.180655+0000","last_fullsized":"2024-04-22T20:45:20.180655+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:27:12.156563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"261'1","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.187790+0000","last_change":"2024-04-22T20:05:17.901476+0000","last_active":"2024-04-22T20:45:30.187790+0000","last_peered":"2024-04-22T20:45:30.187790+0000","last_clean":"2024-04-22T20:45:30.187790+0000","last_became_active":"2024-04-22T20:05:17.900874+0000","last_became_peered":"2024-04-22T20:05:17.900874+0000","last_unstale":"2024-04-22T20:45:30.187790+0000","last_undegraded":"2024-04-22T20:45:30.187790+0000","last_fullsized":"2024-04-22T20:45:30.187790+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:20:07.700066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057705+0000","last_change":"2024-04-22T20:05:13.814214+0000","last_active":"2024-04-22T20:44:50.057705+0000","last_peered":"2024-04-22T20:44:50.057705+0000","last_clean":"2024-04-22T20:44:50.057705+0000","last_became_active":"2024-04-22T20:05:13.813934+0000","last_became_peered":"2024-04-22T20:05:13.813934+0000","last_unstale":"2024-04-22T20:44:50.057705+0000","last_undegraded":"2024-04-22T20:44:50.057705+0000","last_fullsized":"2024-04-22T20:44:50.057705+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:15.355174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1024,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"261'37","reported_seq":146,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.173934+0000","last_change":"2024-04-22T20:01:04.869971+0000","last_active":"2024-04-22T20:44:50.173934+0000","last_peered":"2024-04-22T20:44:50.173934+0000","last_clean":"2024-04-22T20:44:50.173934+0000","last_became_active":"2024-04-22T20:01:04.869660+0000","last_became_peered":"2024-04-22T20:01:04.869660+0000","last_unstale":"2024-04-22T20:44:50.173934+0000","last_undegraded":"2024-04-22T20:44:50.173934+0000","last_fullsized":"2024-04-22T20:44:50.173934+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:08:02.708029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":113,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.173415+0000","last_change":"2024-04-22T20:19:16.795977+0000","last_active":"2024-04-22T20:44:20.173415+0000","last_peered":"2024-04-22T20:44:20.173415+0000","last_clean":"2024-04-22T20:44:20.173415+0000","last_became_active":"2024-04-22T20:19:16.795509+0000","last_became_peered":"2024-04-22T20:19:16.795509+0000","last_unstale":"2024-04-22T20:44:20.173415+0000","last_undegraded":"2024-04-22T20:44:20.173415+0000","last_fullsized":"2024-04-22T20:44:20.173415+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:07.456021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.310587+0000","last_change":"2024-04-22T20:05:17.891942+0000","last_active":"2024-04-22T20:44:55.310587+0000","last_peered":"2024-04-22T20:44:55.310587+0000","last_clean":"2024-04-22T20:44:55.310587+0000","last_became_active":"2024-04-22T20:05:17.891523+0000","last_became_peered":"2024-04-22T20:05:17.891523+0000","last_unstale":"2024-04-22T20:44:55.310587+0000","last_undegraded":"2024-04-22T20:44:55.310587+0000","last_fullsized":"2024-04-22T20:44:55.310587+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:38.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040268+0000","last_change":"2024-04-22T20:05:13.805972+0000","last_active":"2024-04-22T20:44:50.040268+0000","last_peered":"2024-04-22T20:44:50.040268+0000","last_clean":"2024-04-22T20:44:50.040268+0000","last_became_active":"2024-04-22T20:05:13.805704+0000","last_became_peered":"2024-04-22T20:05:13.805704+0000","last_unstale":"2024-04-22T20:44:50.040268+0000","last_undegraded":"2024-04-22T20:44:50.040268+0000","last_fullsized":"2024-04-22T20:44:50.040268+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:03.816391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"261'65","reported_seq":224,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309884+0000","last_change":"2024-04-22T20:01:04.866275+0000","last_active":"2024-04-22T20:44:50.309884+0000","last_peered":"2024-04-22T20:44:50.309884+0000","last_clean":"2024-04-22T20:44:50.309884+0000","last_became_active":"2024-04-22T20:01:04.865981+0000","last_became_peered":"2024-04-22T20:01:04.865981+0000","last_unstale":"2024-04-22T20:44:50.309884+0000","last_undegraded":"2024-04-22T20:44:50.309884+0000","last_fullsized":"2024-04-22T20:44:50.309884+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:25.969487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":226,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:30.054611+0000","last_change":"2024-04-22T20:19:16.799325+0000","last_active":"2024-04-22T20:44:30.054611+0000","last_peered":"2024-04-22T20:44:30.054611+0000","last_clean":"2024-04-22T20:44:30.054611+0000","last_became_active":"2024-04-22T20:19:16.799063+0000","last_became_peered":"2024-04-22T20:19:16.799063+0000","last_unstale":"2024-04-22T20:44:30.054611+0000","last_undegraded":"2024-04-22T20:44:30.054611+0000","last_fullsized":"2024-04-22T20:44:30.054611+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:20:02.254828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"261'1","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.174634+0000","last_change":"2024-04-22T20:05:17.901995+0000","last_active":"2024-04-22T20:44:55.174634+0000","last_peered":"2024-04-22T20:44:55.174634+0000","last_clean":"2024-04-22T20:44:55.174634+0000","last_became_active":"2024-04-22T20:05:17.901680+0000","last_became_peered":"2024-04-22T20:05:17.901680+0000","last_unstale":"2024-04-22T20:44:55.174634+0000","last_undegraded":"2024-04-22T20:44:55.174634+0000","last_fullsized":"2024-04-22T20:44:55.174634+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:26.628494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40145,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":80,"num_write":1,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309933+0000","last_change":"2024-04-22T20:05:13.811164+0000","last_active":"2024-04-22T20:44:50.309933+0000","last_peered":"2024-04-22T20:44:50.309933+0000","last_clean":"2024-04-22T20:44:50.309933+0000","last_became_active":"2024-04-22T20:05:13.810865+0000","last_became_peered":"2024-04-22T20:05:13.810865+0000","last_unstale":"2024-04-22T20:44:50.309933+0000","last_undegraded":"2024-04-22T20:44:50.309933+0000","last_fullsized":"2024-04-22T20:44:50.309933+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:47:15.297255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"261'57","reported_seq":180,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.061413+0000","last_change":"2024-04-22T20:01:04.870803+0000","last_active":"2024-04-22T20:45:15.061413+0000","last_peered":"2024-04-22T20:45:15.061413+0000","last_clean":"2024-04-22T20:45:15.061413+0000","last_became_active":"2024-04-22T20:01:04.870253+0000","last_became_peered":"2024-04-22T20:01:04.870253+0000","last_unstale":"2024-04-22T20:45:15.061413+0000","last_undegraded":"2024-04-22T20:45:15.061413+0000","last_fullsized":"2024-04-22T20:45:15.061413+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:32:34.069500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":145,"num_write":102,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"261'281","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.065532+0000","last_change":"2024-04-22T19:53:04.823797+0000","last_active":"2024-04-22T20:45:10.065532+0000","last_peered":"2024-04-22T20:45:10.065532+0000","last_clean":"2024-04-22T20:45:10.065532+0000","last_became_active":"2024-04-22T19:53:04.823351+0000","last_became_peered":"2024-04-22T19:53:04.823351+0000","last_unstale":"2024-04-22T20:45:10.065532+0000","last_undegraded":"2024-04-22T20:45:10.065532+0000","last_fullsized":"2024-04-22T20:45:10.065532+0000","mapping_epoch":35,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:24.940347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9460561,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":447,"num_read_kb":495,"num_write":1235,"num_write_kb":39497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"261'169","reported_seq":763,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174873+0000","last_change":"2024-04-22T19:51:28.511862+0000","last_active":"2024-04-22T20:44:00.174873+0000","last_peered":"2024-04-22T20:44:00.174873+0000","last_clean":"2024-04-22T20:44:00.174873+0000","last_became_active":"2024-04-22T19:51:28.511595+0000","last_became_peered":"2024-04-22T19:51:28.511595+0000","last_unstale":"2024-04-22T20:44:00.174873+0000","last_undegraded":"2024-04-22T20:44:00.174873+0000","last_fullsized":"2024-04-22T20:44:00.174873+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:36.200495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":537,"num_write":262,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"261'248","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:48.785097+0000","last_change":"2024-04-22T19:51:26.496852+0000","last_active":"2024-04-22T20:46:48.785097+0000","last_peered":"2024-04-22T20:46:48.785097+0000","last_clean":"2024-04-22T20:46:48.785097+0000","last_became_active":"2024-04-22T19:51:26.496188+0000","last_became_peered":"2024-04-22T19:51:26.496188+0000","last_unstale":"2024-04-22T20:46:48.785097+0000","last_undegraded":"2024-04-22T20:46:48.785097+0000","last_fullsized":"2024-04-22T20:46:48.785097+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:51:56.333450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10265266,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":429,"num_read_kb":9758,"num_write":1044,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"261'314","reported_seq":886,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:28.101937+0000","last_change":"2024-04-22T19:51:28.519482+0000","last_active":"2024-04-22T20:47:28.101937+0000","last_peered":"2024-04-22T20:47:28.101937+0000","last_clean":"2024-04-22T20:47:28.101937+0000","last_became_active":"2024-04-22T19:51:28.519111+0000","last_became_peered":"2024-04-22T19:51:28.519111+0000","last_unstale":"2024-04-22T20:47:28.101937+0000","last_undegraded":"2024-04-22T20:47:28.101937+0000","last_fullsized":"2024-04-22T20:47:28.101937+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:06:08.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1190,"num_read_kb":913,"num_write":510,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"261'247","reported_seq":868,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:36.787572+0000","last_change":"2024-04-22T19:51:26.503143+0000","last_active":"2024-04-22T20:46:36.787572+0000","last_peered":"2024-04-22T20:46:36.787572+0000","last_clean":"2024-04-22T20:46:36.787572+0000","last_became_active":"2024-04-22T19:51:26.502744+0000","last_became_peered":"2024-04-22T19:51:26.502744+0000","last_unstale":"2024-04-22T20:46:36.787572+0000","last_undegraded":"2024-04-22T20:46:36.787572+0000","last_fullsized":"2024-04-22T20:46:36.787572+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:30:34.457597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9763357,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":370,"num_read_kb":7116,"num_write":962,"num_write_kb":21496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"261'150","reported_seq":726,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166689+0000","last_change":"2024-04-22T19:51:28.518949+0000","last_active":"2024-04-22T20:44:00.166689+0000","last_peered":"2024-04-22T20:44:00.166689+0000","last_clean":"2024-04-22T20:44:00.166689+0000","last_became_active":"2024-04-22T19:51:28.518650+0000","last_became_peered":"2024-04-22T19:51:28.518650+0000","last_unstale":"2024-04-22T20:44:00.166689+0000","last_undegraded":"2024-04-22T20:44:00.166689+0000","last_fullsized":"2024-04-22T20:44:00.166689+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:25:03.391586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":572,"num_read_kb":453,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"261'259","reported_seq":874,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:14.145384+0000","last_change":"2024-04-22T19:51:26.504934+0000","last_active":"2024-04-22T20:47:14.145384+0000","last_peered":"2024-04-22T20:47:14.145384+0000","last_clean":"2024-04-22T20:47:14.145384+0000","last_became_active":"2024-04-22T19:51:26.504450+0000","last_became_peered":"2024-04-22T19:51:26.504450+0000","last_unstale":"2024-04-22T20:47:14.145384+0000","last_undegraded":"2024-04-22T20:47:14.145384+0000","last_fullsized":"2024-04-22T20:47:14.145384+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:47:54.670392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20493175,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":359,"num_read_kb":25508,"num_write":1125,"num_write_kb":37655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"261'81","reported_seq":676,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.171137+0000","last_change":"2024-04-22T19:52:04.620094+0000","last_active":"2024-04-22T20:44:00.171137+0000","last_peered":"2024-04-22T20:44:00.171137+0000","last_clean":"2024-04-22T20:44:00.171137+0000","last_became_active":"2024-04-22T19:52:04.619688+0000","last_became_peered":"2024-04-22T19:52:04.619688+0000","last_unstale":"2024-04-22T20:44:00.171137+0000","last_undegraded":"2024-04-22T20:44:00.171137+0000","last_fullsized":"2024-04-22T20:44:00.171137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:50:37.537623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":353,"num_read_kb":299,"num_write":120,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"261'222","reported_seq":910,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:03.977215+0000","last_change":"2024-04-22T19:51:26.509141+0000","last_active":"2024-04-22T20:47:03.977215+0000","last_peered":"2024-04-22T20:47:03.977215+0000","last_clean":"2024-04-22T20:47:03.977215+0000","last_became_active":"2024-04-22T19:51:26.507185+0000","last_became_peered":"2024-04-22T19:51:26.507185+0000","last_unstale":"2024-04-22T20:47:03.977215+0000","last_undegraded":"2024-04-22T20:47:03.977215+0000","last_fullsized":"2024-04-22T20:47:03.977215+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:14.928877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563190,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":383,"num_read_kb":10200,"num_write":866,"num_write_kb":30630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"261'40","reported_seq":594,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.316183+0000","last_change":"2024-04-22T19:51:28.513981+0000","last_active":"2024-04-22T20:45:35.316183+0000","last_peered":"2024-04-22T20:45:35.316183+0000","last_clean":"2024-04-22T20:45:35.316183+0000","last_became_active":"2024-04-22T19:51:28.513693+0000","last_became_peered":"2024-04-22T19:51:28.513693+0000","last_unstale":"2024-04-22T20:45:35.316183+0000","last_undegraded":"2024-04-22T20:45:35.316183+0000","last_fullsized":"2024-04-22T20:45:35.316183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:54:32.701108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"261'389","reported_seq":1131,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:54.327300+0000","last_change":"2024-04-22T19:51:26.509287+0000","last_active":"2024-04-22T20:46:54.327300+0000","last_peered":"2024-04-22T20:46:54.327300+0000","last_clean":"2024-04-22T20:46:54.327300+0000","last_became_active":"2024-04-22T19:51:26.508716+0000","last_became_peered":"2024-04-22T19:51:26.508716+0000","last_unstale":"2024-04-22T20:46:54.327300+0000","last_undegraded":"2024-04-22T20:46:54.327300+0000","last_fullsized":"2024-04-22T20:46:54.327300+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:42:06.130701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11174792,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":980,"num_read_kb":17244,"num_write":1468,"num_write_kb":25883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"261'144","reported_seq":742,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181403+0000","last_change":"2024-04-22T19:51:28.519053+0000","last_active":"2024-04-22T20:45:40.181403+0000","last_peered":"2024-04-22T20:45:40.181403+0000","last_clean":"2024-04-22T20:45:40.181403+0000","last_became_active":"2024-04-22T19:51:28.518501+0000","last_became_peered":"2024-04-22T19:51:28.518501+0000","last_unstale":"2024-04-22T20:45:40.181403+0000","last_undegraded":"2024-04-22T20:45:40.181403+0000","last_fullsized":"2024-04-22T20:45:40.181403+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:36.369584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":597,"num_read_kb":478,"num_write":226,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"261'300","reported_seq":961,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.049692+0000","last_change":"2024-04-22T19:51:26.508109+0000","last_active":"2024-04-22T20:44:00.049692+0000","last_peered":"2024-04-22T20:44:00.049692+0000","last_clean":"2024-04-22T20:44:00.049692+0000","last_became_active":"2024-04-22T19:51:26.507141+0000","last_became_peered":"2024-04-22T19:51:26.507141+0000","last_unstale":"2024-04-22T20:44:00.049692+0000","last_undegraded":"2024-04-22T20:44:00.049692+0000","last_fullsized":"2024-04-22T20:44:00.049692+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:27.201168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11136278,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":473,"num_read_kb":43743,"num_write":1312,"num_write_kb":32515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"261'151","reported_seq":736,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.067532+0000","last_change":"2024-04-22T19:51:28.516437+0000","last_active":"2024-04-22T20:47:38.067532+0000","last_peered":"2024-04-22T20:47:38.067532+0000","last_clean":"2024-04-22T20:47:38.067532+0000","last_became_active":"2024-04-22T19:51:28.516012+0000","last_became_peered":"2024-04-22T19:51:28.516012+0000","last_unstale":"2024-04-22T20:47:38.067532+0000","last_undegraded":"2024-04-22T20:47:38.067532+0000","last_fullsized":"2024-04-22T20:47:38.067532+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:38.182307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":470,"num_write":219,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"261'268","reported_seq":912,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:28.486641+0000","last_change":"2024-04-22T19:51:26.505832+0000","last_active":"2024-04-22T20:46:28.486641+0000","last_peered":"2024-04-22T20:46:28.486641+0000","last_clean":"2024-04-22T20:46:28.486641+0000","last_became_active":"2024-04-22T19:51:26.502928+0000","last_became_peered":"2024-04-22T19:51:26.502928+0000","last_unstale":"2024-04-22T20:46:28.486641+0000","last_undegraded":"2024-04-22T20:46:28.486641+0000","last_fullsized":"2024-04-22T20:46:28.486641+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:47:30.201226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9686578,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":378,"num_read_kb":51847,"num_write":1138,"num_write_kb":39313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"261'60","reported_seq":629,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:57.506260+0000","last_change":"2024-04-22T19:51:28.510044+0000","last_active":"2024-04-22T20:46:57.506260+0000","last_peered":"2024-04-22T20:46:57.506260+0000","last_clean":"2024-04-22T20:46:57.506260+0000","last_became_active":"2024-04-22T19:51:28.509774+0000","last_became_peered":"2024-04-22T19:51:28.509774+0000","last_unstale":"2024-04-22T20:46:57.506260+0000","last_undegraded":"2024-04-22T20:46:57.506260+0000","last_fullsized":"2024-04-22T20:46:57.506260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:55:37.172957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":211,"num_write":81,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"261'125","reported_seq":734,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176694+0000","last_change":"2024-04-22T19:51:28.505929+0000","last_active":"2024-04-22T20:45:10.176694+0000","last_peered":"2024-04-22T20:45:10.176694+0000","last_clean":"2024-04-22T20:45:10.176694+0000","last_became_active":"2024-04-22T19:51:28.505562+0000","last_became_peered":"2024-04-22T19:51:28.505562+0000","last_unstale":"2024-04-22T20:45:10.176694+0000","last_undegraded":"2024-04-22T20:45:10.176694+0000","last_fullsized":"2024-04-22T20:45:10.176694+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:33:32.464020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":540,"num_read_kb":443,"num_write":193,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"261'253","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057867+0000","last_change":"2024-04-22T19:51:26.507324+0000","last_active":"2024-04-22T20:44:50.057867+0000","last_peered":"2024-04-22T20:44:50.057867+0000","last_clean":"2024-04-22T20:44:50.057867+0000","last_became_active":"2024-04-22T19:51:26.504717+0000","last_became_peered":"2024-04-22T19:51:26.504717+0000","last_unstale":"2024-04-22T20:44:50.057867+0000","last_undegraded":"2024-04-22T20:44:50.057867+0000","last_fullsized":"2024-04-22T20:44:50.057867+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:31:10.322998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6322974,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":420,"num_read_kb":28705,"num_write":1056,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"261'105","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060776+0000","last_change":"2024-04-22T19:51:28.519503+0000","last_active":"2024-04-22T20:45:10.060776+0000","last_peered":"2024-04-22T20:45:10.060776+0000","last_clean":"2024-04-22T20:45:10.060776+0000","last_became_active":"2024-04-22T19:51:28.519230+0000","last_became_peered":"2024-04-22T19:51:28.519230+0000","last_unstale":"2024-04-22T20:45:10.060776+0000","last_undegraded":"2024-04-22T20:45:10.060776+0000","last_fullsized":"2024-04-22T20:45:10.060776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:23:00.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":465,"num_read_kb":383,"num_write":180,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"261'321","reported_seq":978,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.308459+0000","last_change":"2024-04-22T19:51:26.496564+0000","last_active":"2024-04-22T20:44:40.308459+0000","last_peered":"2024-04-22T20:44:40.308459+0000","last_clean":"2024-04-22T20:44:40.308459+0000","last_became_active":"2024-04-22T19:51:26.496156+0000","last_became_peered":"2024-04-22T19:51:26.496156+0000","last_unstale":"2024-04-22T20:44:40.308459+0000","last_undegraded":"2024-04-22T20:44:40.308459+0000","last_fullsized":"2024-04-22T20:44:40.308459+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:25.947249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77420,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":551,"num_read_kb":792,"num_write":1456,"num_write_kb":25865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"261'172","reported_seq":764,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186369+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:45:20.186369+0000","last_peered":"2024-04-22T20:45:20.186369+0000","last_clean":"2024-04-22T20:45:20.186369+0000","last_became_active":"2024-04-22T19:51:28.515673+0000","last_became_peered":"2024-04-22T19:51:28.515673+0000","last_unstale":"2024-04-22T20:45:20.186369+0000","last_undegraded":"2024-04-22T20:45:20.186369+0000","last_fullsized":"2024-04-22T20:45:20.186369+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:06.001104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":666,"num_read_kb":529,"num_write":258,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"261'291","reported_seq":1001,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.308420+0000","last_change":"2024-04-22T19:51:26.502589+0000","last_active":"2024-04-22T20:44:40.308420+0000","last_peered":"2024-04-22T20:44:40.308420+0000","last_clean":"2024-04-22T20:44:40.308420+0000","last_became_active":"2024-04-22T19:51:26.502156+0000","last_became_peered":"2024-04-22T19:51:26.502156+0000","last_unstale":"2024-04-22T20:44:40.308420+0000","last_undegraded":"2024-04-22T20:44:40.308420+0000","last_fullsized":"2024-04-22T20:44:40.308420+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:22:39.535038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23449640,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":550,"num_read_kb":68133,"num_write":1243,"num_write_kb":41095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"261'290","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:09.419441+0000","last_change":"2024-04-22T19:51:28.510666+0000","last_active":"2024-04-22T20:47:09.419441+0000","last_peered":"2024-04-22T20:47:09.419441+0000","last_clean":"2024-04-22T20:47:09.419441+0000","last_became_active":"2024-04-22T19:51:28.510415+0000","last_became_peered":"2024-04-22T19:51:28.510415+0000","last_unstale":"2024-04-22T20:47:09.419441+0000","last_undegraded":"2024-04-22T20:47:09.419441+0000","last_fullsized":"2024-04-22T20:47:09.419441+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:43.003473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1119,"num_read_kb":869,"num_write":475,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"261'251","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172460+0000","last_change":"2024-04-22T19:51:26.507272+0000","last_active":"2024-04-22T20:44:40.172460+0000","last_peered":"2024-04-22T20:44:40.172460+0000","last_clean":"2024-04-22T20:44:40.172460+0000","last_became_active":"2024-04-22T19:51:26.506671+0000","last_became_peered":"2024-04-22T19:51:26.506671+0000","last_unstale":"2024-04-22T20:44:40.172460+0000","last_undegraded":"2024-04-22T20:44:40.172460+0000","last_fullsized":"2024-04-22T20:44:40.172460+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:54:08.993777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8504557,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":414,"num_read_kb":6535,"num_write":1034,"num_write_kb":36003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"261'155","reported_seq":775,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.062504+0000","last_change":"2024-04-22T19:51:28.512453+0000","last_active":"2024-04-22T20:44:40.062504+0000","last_peered":"2024-04-22T20:44:40.062504+0000","last_clean":"2024-04-22T20:44:40.062504+0000","last_became_active":"2024-04-22T19:51:28.511889+0000","last_became_peered":"2024-04-22T19:51:28.511889+0000","last_unstale":"2024-04-22T20:44:40.062504+0000","last_undegraded":"2024-04-22T20:44:40.062504+0000","last_fullsized":"2024-04-22T20:44:40.062504+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:45.195439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":679,"num_read_kb":547,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"261'265","reported_seq":912,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.157609+0000","last_change":"2024-04-22T19:51:26.498087+0000","last_active":"2024-04-22T20:47:38.157609+0000","last_peered":"2024-04-22T20:47:38.157609+0000","last_clean":"2024-04-22T20:47:38.157609+0000","last_became_active":"2024-04-22T19:51:26.497718+0000","last_became_peered":"2024-04-22T19:51:26.497718+0000","last_unstale":"2024-04-22T20:47:38.157609+0000","last_undegraded":"2024-04-22T20:47:38.157609+0000","last_fullsized":"2024-04-22T20:47:38.157609+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:50:38.709806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6162293,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":402,"num_read_kb":30110,"num_write":1095,"num_write_kb":27210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"261'93","reported_seq":699,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:44.069351+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:46:44.069351+0000","last_peered":"2024-04-22T20:46:44.069351+0000","last_clean":"2024-04-22T20:46:44.069351+0000","last_became_active":"2024-04-22T19:51:28.507673+0000","last_became_peered":"2024-04-22T19:51:28.507673+0000","last_unstale":"2024-04-22T20:46:44.069351+0000","last_undegraded":"2024-04-22T20:46:44.069351+0000","last_fullsized":"2024-04-22T20:46:44.069351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:18:40.338876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":404,"num_read_kb":350,"num_write":133,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"261'347","reported_seq":1058,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.243867+0000","last_change":"2024-04-22T19:51:26.511696+0000","last_active":"2024-04-22T20:46:30.243867+0000","last_peered":"2024-04-22T20:46:30.243867+0000","last_clean":"2024-04-22T20:46:30.243867+0000","last_became_active":"2024-04-22T19:51:26.511213+0000","last_became_peered":"2024-04-22T19:51:26.511213+0000","last_unstale":"2024-04-22T20:46:30.243867+0000","last_undegraded":"2024-04-22T20:46:30.243867+0000","last_fullsized":"2024-04-22T20:46:30.243867+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:51:44.984733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13939483,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":664,"num_read_kb":36415,"num_write":1732,"num_write_kb":40524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"261'126","reported_seq":647,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.145449+0000","last_change":"2024-04-22T19:51:36.567880+0000","last_active":"2024-04-22T20:48:04.145449+0000","last_peered":"2024-04-22T20:48:04.145449+0000","last_clean":"2024-04-22T20:48:04.145449+0000","last_became_active":"2024-04-22T19:51:36.567561+0000","last_became_peered":"2024-04-22T19:51:36.567561+0000","last_unstale":"2024-04-22T20:48:04.145449+0000","last_undegraded":"2024-04-22T20:48:04.145449+0000","last_fullsized":"2024-04-22T20:48:04.145449+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:10:43.936463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":168,"num_write":307,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"261'38","reported_seq":584,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154700+0000","last_change":"2024-04-22T19:51:28.520238+0000","last_active":"2024-04-22T20:45:15.154700+0000","last_peered":"2024-04-22T20:45:15.154700+0000","last_clean":"2024-04-22T20:45:15.154700+0000","last_became_active":"2024-04-22T19:51:28.519994+0000","last_became_peered":"2024-04-22T19:51:28.519994+0000","last_unstale":"2024-04-22T20:45:15.154700+0000","last_undegraded":"2024-04-22T20:45:15.154700+0000","last_fullsized":"2024-04-22T20:45:15.154700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:48:21.210710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":138,"num_write":52,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"261'264","reported_seq":997,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060800+0000","last_change":"2024-04-22T19:53:05.051779+0000","last_active":"2024-04-22T20:45:10.060800+0000","last_peered":"2024-04-22T20:45:10.060800+0000","last_clean":"2024-04-22T20:45:10.060800+0000","last_became_active":"2024-04-22T19:53:05.051272+0000","last_became_peered":"2024-04-22T19:53:05.051272+0000","last_unstale":"2024-04-22T20:45:10.060800+0000","last_undegraded":"2024-04-22T20:45:10.060800+0000","last_fullsized":"2024-04-22T20:45:10.060800+0000","mapping_epoch":35,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:14:57.868131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6579999,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":609,"num_read_kb":11069,"num_write":1217,"num_write_kb":33730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180678+0000","last_change":"2024-04-22T19:51:30.521121+0000","last_active":"2024-04-22T20:44:40.180678+0000","last_peered":"2024-04-22T20:44:40.180678+0000","last_clean":"2024-04-22T20:44:40.180678+0000","last_became_active":"2024-04-22T19:51:30.520848+0000","last_became_peered":"2024-04-22T19:51:30.520848+0000","last_unstale":"2024-04-22T20:44:40.180678+0000","last_undegraded":"2024-04-22T20:44:40.180678+0000","last_fullsized":"2024-04-22T20:44:40.180678+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:30.383458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"261'324","reported_seq":923,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.193392+0000","last_change":"2024-04-22T19:51:32.537967+0000","last_active":"2024-04-22T20:47:10.193392+0000","last_peered":"2024-04-22T20:47:10.193392+0000","last_clean":"2024-04-22T20:47:10.193392+0000","last_became_active":"2024-04-22T19:51:32.537695+0000","last_became_peered":"2024-04-22T19:51:32.537695+0000","last_unstale":"2024-04-22T20:47:10.193392+0000","last_undegraded":"2024-04-22T20:47:10.193392+0000","last_fullsized":"2024-04-22T20:47:10.193392+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:12:32.127700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58993,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":537,"num_read_kb":1015,"num_write":432,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.057042+0000","last_change":"2024-04-22T19:51:34.557621+0000","last_active":"2024-04-22T20:44:45.057042+0000","last_peered":"2024-04-22T20:44:45.057042+0000","last_clean":"2024-04-22T20:44:45.057042+0000","last_became_active":"2024-04-22T19:51:34.557285+0000","last_became_peered":"2024-04-22T19:51:34.557285+0000","last_unstale":"2024-04-22T20:44:45.057042+0000","last_undegraded":"2024-04-22T20:44:45.057042+0000","last_fullsized":"2024-04-22T20:44:45.057042+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:14.949781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"261'2399","reported_seq":4258,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.711091+0000","last_change":"2024-04-22T19:51:36.567463+0000","last_active":"2024-04-22T20:48:03.711091+0000","last_peered":"2024-04-22T20:48:03.711091+0000","last_clean":"2024-04-22T20:48:03.711091+0000","last_became_active":"2024-04-22T19:51:36.567195+0000","last_became_peered":"2024-04-22T19:51:36.567195+0000","last_unstale":"2024-04-22T20:48:03.711091+0000","last_undegraded":"2024-04-22T20:48:03.711091+0000","last_fullsized":"2024-04-22T20:48:03.711091+0000","mapping_epoch":30,"log_start":"261'2300","ondisk_log_start":"261'2300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":2300,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:46:00.619550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5986,"num_read_kb":5990,"num_write":4781,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"261'130","reported_seq":720,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:53.028609+0000","last_change":"2024-04-22T19:51:28.513076+0000","last_active":"2024-04-22T20:46:53.028609+0000","last_peered":"2024-04-22T20:46:53.028609+0000","last_clean":"2024-04-22T20:46:53.028609+0000","last_became_active":"2024-04-22T19:51:28.512786+0000","last_became_peered":"2024-04-22T19:51:28.512786+0000","last_unstale":"2024-04-22T20:46:53.028609+0000","last_undegraded":"2024-04-22T20:46:53.028609+0000","last_fullsized":"2024-04-22T20:46:53.028609+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:20:09.250321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":408,"num_write":191,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"261'229","reported_seq":832,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:55.640581+0000","last_change":"2024-04-22T19:51:26.500209+0000","last_active":"2024-04-22T20:46:55.640581+0000","last_peered":"2024-04-22T20:46:55.640581+0000","last_clean":"2024-04-22T20:46:55.640581+0000","last_became_active":"2024-04-22T19:51:26.499885+0000","last_became_peered":"2024-04-22T19:51:26.499885+0000","last_unstale":"2024-04-22T20:46:55.640581+0000","last_undegraded":"2024-04-22T20:46:55.640581+0000","last_fullsized":"2024-04-22T20:46:55.640581+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:06:40.641478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8391210,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":339,"num_read_kb":16305,"num_write":901,"num_write_kb":23193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154709+0000","last_change":"2024-04-22T19:51:30.527682+0000","last_active":"2024-04-22T20:45:15.154709+0000","last_peered":"2024-04-22T20:45:15.154709+0000","last_clean":"2024-04-22T20:45:15.154709+0000","last_became_active":"2024-04-22T19:51:30.526622+0000","last_became_peered":"2024-04-22T19:51:30.526622+0000","last_unstale":"2024-04-22T20:45:15.154709+0000","last_undegraded":"2024-04-22T20:45:15.154709+0000","last_fullsized":"2024-04-22T20:45:15.154709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:20:42.245730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"261'86","reported_seq":611,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.173330+0000","last_change":"2024-04-22T19:51:32.536487+0000","last_active":"2024-04-22T20:44:45.173330+0000","last_peered":"2024-04-22T20:44:45.173330+0000","last_clean":"2024-04-22T20:44:45.173330+0000","last_became_active":"2024-04-22T19:51:32.536180+0000","last_became_peered":"2024-04-22T19:51:32.536180+0000","last_unstale":"2024-04-22T20:44:45.173330+0000","last_undegraded":"2024-04-22T20:44:45.173330+0000","last_fullsized":"2024-04-22T20:44:45.173330+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:21.299001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":95,"num_write":64,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.173306+0000","last_change":"2024-04-22T19:51:34.549833+0000","last_active":"2024-04-22T20:44:45.173306+0000","last_peered":"2024-04-22T20:44:45.173306+0000","last_clean":"2024-04-22T20:44:45.173306+0000","last_became_active":"2024-04-22T19:51:34.549551+0000","last_became_peered":"2024-04-22T19:51:34.549551+0000","last_unstale":"2024-04-22T20:44:45.173306+0000","last_undegraded":"2024-04-22T20:44:45.173306+0000","last_fullsized":"2024-04-22T20:44:45.173306+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:14:36.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"261'146","reported_seq":645,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.823329+0000","last_change":"2024-04-22T19:51:36.573966+0000","last_active":"2024-04-22T20:48:00.823329+0000","last_peered":"2024-04-22T20:48:00.823329+0000","last_clean":"2024-04-22T20:48:00.823329+0000","last_became_active":"2024-04-22T19:51:36.573189+0000","last_became_peered":"2024-04-22T19:51:36.573189+0000","last_unstale":"2024-04-22T20:48:00.823329+0000","last_undegraded":"2024-04-22T20:48:00.823329+0000","last_fullsized":"2024-04-22T20:48:00.823329+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:07:48.890371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":328,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"261'212","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.175878+0000","last_change":"2024-04-22T19:51:28.518334+0000","last_active":"2024-04-22T20:44:40.175878+0000","last_peered":"2024-04-22T20:44:40.175878+0000","last_clean":"2024-04-22T20:44:40.175878+0000","last_became_active":"2024-04-22T19:51:28.517737+0000","last_became_peered":"2024-04-22T19:51:28.517737+0000","last_unstale":"2024-04-22T20:44:40.175878+0000","last_undegraded":"2024-04-22T20:44:40.175878+0000","last_fullsized":"2024-04-22T20:44:40.175878+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:47.145426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":871,"num_read_kb":733,"num_write":338,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"261'241","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:59.594399+0000","last_change":"2024-04-22T19:51:26.509257+0000","last_active":"2024-04-22T20:46:59.594399+0000","last_peered":"2024-04-22T20:46:59.594399+0000","last_clean":"2024-04-22T20:46:59.594399+0000","last_became_active":"2024-04-22T19:51:26.508933+0000","last_became_peered":"2024-04-22T19:51:26.508933+0000","last_unstale":"2024-04-22T20:46:59.594399+0000","last_undegraded":"2024-04-22T20:46:59.594399+0000","last_fullsized":"2024-04-22T20:46:59.594399+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:00:26.499126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8849792,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":456,"num_read_kb":15260,"num_write":1066,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":502,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.150710+0000","last_change":"2024-04-22T19:51:30.528152+0000","last_active":"2024-04-22T20:44:40.150710+0000","last_peered":"2024-04-22T20:44:40.150710+0000","last_clean":"2024-04-22T20:44:40.150710+0000","last_became_active":"2024-04-22T19:51:30.527712+0000","last_became_peered":"2024-04-22T19:51:30.527712+0000","last_unstale":"2024-04-22T20:44:40.150710+0000","last_undegraded":"2024-04-22T20:44:40.150710+0000","last_fullsized":"2024-04-22T20:44:40.150710+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:03:58.918129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"261'112","reported_seq":656,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.066993+0000","last_change":"2024-04-22T19:51:32.536484+0000","last_active":"2024-04-22T20:45:25.066993+0000","last_peered":"2024-04-22T20:45:25.066993+0000","last_clean":"2024-04-22T20:45:25.066993+0000","last_became_active":"2024-04-22T19:51:32.536177+0000","last_became_peered":"2024-04-22T19:51:32.536177+0000","last_unstale":"2024-04-22T20:45:25.066993+0000","last_undegraded":"2024-04-22T20:45:25.066993+0000","last_fullsized":"2024-04-22T20:45:25.066993+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:39:22.960954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"261'519","reported_seq":2079,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.552754+0000","last_change":"2024-04-22T19:51:34.562120+0000","last_active":"2024-04-22T20:48:08.552754+0000","last_peered":"2024-04-22T20:48:08.552754+0000","last_clean":"2024-04-22T20:48:08.552754+0000","last_became_active":"2024-04-22T19:51:34.561828+0000","last_became_peered":"2024-04-22T19:51:34.561828+0000","last_unstale":"2024-04-22T20:48:08.552754+0000","last_undegraded":"2024-04-22T20:48:08.552754+0000","last_fullsized":"2024-04-22T20:48:08.552754+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:46.946439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"261'174","reported_seq":986,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.139062+0000","last_change":"2024-04-22T19:51:36.565908+0000","last_active":"2024-04-22T20:48:03.139062+0000","last_peered":"2024-04-22T20:48:03.139062+0000","last_clean":"2024-04-22T20:48:03.139062+0000","last_became_active":"2024-04-22T19:51:36.565673+0000","last_became_peered":"2024-04-22T19:51:36.565673+0000","last_unstale":"2024-04-22T20:48:03.139062+0000","last_undegraded":"2024-04-22T20:48:03.139062+0000","last_fullsized":"2024-04-22T20:48:03.139062+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:45:46.054789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":579,"num_read_kb":567,"num_write":371,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"261'176","reported_seq":814,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.150685+0000","last_change":"2024-04-22T19:51:28.516658+0000","last_active":"2024-04-22T20:44:40.150685+0000","last_peered":"2024-04-22T20:44:40.150685+0000","last_clean":"2024-04-22T20:44:40.150685+0000","last_became_active":"2024-04-22T19:51:28.515670+0000","last_became_peered":"2024-04-22T19:51:28.515670+0000","last_unstale":"2024-04-22T20:44:40.150685+0000","last_undegraded":"2024-04-22T20:44:40.150685+0000","last_fullsized":"2024-04-22T20:44:40.150685+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:38:08.341570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":729,"num_read_kb":590,"num_write":271,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"261'261","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.305295+0000","last_change":"2024-04-22T19:51:26.503775+0000","last_active":"2024-04-22T20:47:38.305295+0000","last_peered":"2024-04-22T20:47:38.305295+0000","last_clean":"2024-04-22T20:47:38.305295+0000","last_became_active":"2024-04-22T19:51:26.503438+0000","last_became_peered":"2024-04-22T19:51:26.503438+0000","last_unstale":"2024-04-22T20:47:38.305295+0000","last_undegraded":"2024-04-22T20:47:38.305295+0000","last_fullsized":"2024-04-22T20:47:38.305295+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:39:05.281766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13078725,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":525,"num_read_kb":32833,"num_write":1247,"num_write_kb":25399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180715+0000","last_change":"2024-04-22T19:51:30.523415+0000","last_active":"2024-04-22T20:44:40.180715+0000","last_peered":"2024-04-22T20:44:40.180715+0000","last_clean":"2024-04-22T20:44:40.180715+0000","last_became_active":"2024-04-22T19:51:30.523125+0000","last_became_peered":"2024-04-22T19:51:30.523125+0000","last_unstale":"2024-04-22T20:44:40.180715+0000","last_undegraded":"2024-04-22T20:44:40.180715+0000","last_fullsized":"2024-04-22T20:44:40.180715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:33.007742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"261'247","reported_seq":812,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.078728+0000","last_change":"2024-04-22T19:51:33.248486+0000","last_active":"2024-04-22T20:47:10.078728+0000","last_peered":"2024-04-22T20:47:10.078728+0000","last_clean":"2024-04-22T20:47:10.078728+0000","last_became_active":"2024-04-22T19:51:33.248160+0000","last_became_peered":"2024-04-22T19:51:33.248160+0000","last_unstale":"2024-04-22T20:47:10.078728+0000","last_undegraded":"2024-04-22T20:47:10.078728+0000","last_fullsized":"2024-04-22T20:47:10.078728+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:35.418273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51814,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":412,"num_read_kb":843,"num_write":338,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"261'531","reported_seq":4224,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.556421+0000","last_change":"2024-04-22T19:51:34.556255+0000","last_active":"2024-04-22T20:48:08.556421+0000","last_peered":"2024-04-22T20:48:08.556421+0000","last_clean":"2024-04-22T20:48:08.556421+0000","last_became_active":"2024-04-22T19:51:34.555720+0000","last_became_peered":"2024-04-22T19:51:34.555720+0000","last_unstale":"2024-04-22T20:48:08.556421+0000","last_undegraded":"2024-04-22T20:48:08.556421+0000","last_fullsized":"2024-04-22T20:48:08.556421+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":500,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:58.605732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"261'385","reported_seq":1057,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189325+0000","last_change":"2024-04-22T19:51:28.513088+0000","last_active":"2024-04-22T20:45:40.189325+0000","last_peered":"2024-04-22T20:45:40.189325+0000","last_clean":"2024-04-22T20:45:40.189325+0000","last_became_active":"2024-04-22T19:51:28.512125+0000","last_became_peered":"2024-04-22T19:51:28.512125+0000","last_unstale":"2024-04-22T20:45:40.189325+0000","last_undegraded":"2024-04-22T20:45:40.189325+0000","last_fullsized":"2024-04-22T20:45:40.189325+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:22:31.839887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1728,"num_read_kb":1430,"num_write":663,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"261'238","reported_seq":880,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.318260+0000","last_change":"2024-04-22T19:51:26.508318+0000","last_active":"2024-04-22T20:47:44.318260+0000","last_peered":"2024-04-22T20:47:44.318260+0000","last_clean":"2024-04-22T20:47:44.318260+0000","last_became_active":"2024-04-22T19:51:26.504654+0000","last_became_peered":"2024-04-22T19:51:26.504654+0000","last_unstale":"2024-04-22T20:47:44.318260+0000","last_undegraded":"2024-04-22T20:47:44.318260+0000","last_fullsized":"2024-04-22T20:47:44.318260+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:29:46.972228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12959802,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":378,"num_read_kb":80525,"num_write":948,"num_write_kb":26179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.065667+0000","last_change":"2024-04-22T19:51:30.526634+0000","last_active":"2024-04-22T20:45:40.065667+0000","last_peered":"2024-04-22T20:45:40.065667+0000","last_clean":"2024-04-22T20:45:40.065667+0000","last_became_active":"2024-04-22T19:51:30.526306+0000","last_became_peered":"2024-04-22T19:51:30.526306+0000","last_unstale":"2024-04-22T20:45:40.065667+0000","last_undegraded":"2024-04-22T20:45:40.065667+0000","last_fullsized":"2024-04-22T20:45:40.065667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:46.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"261'329","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:05.210994+0000","last_change":"2024-04-22T19:51:33.248375+0000","last_active":"2024-04-22T20:48:05.210994+0000","last_peered":"2024-04-22T20:48:05.210994+0000","last_clean":"2024-04-22T20:48:05.210994+0000","last_became_active":"2024-04-22T19:51:33.248059+0000","last_became_peered":"2024-04-22T19:51:33.248059+0000","last_unstale":"2024-04-22T20:48:05.210994+0000","last_undegraded":"2024-04-22T20:48:05.210994+0000","last_fullsized":"2024-04-22T20:48:05.210994+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:25:25.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48009,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":524,"num_read_kb":923,"num_write":402,"num_write_kb":246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.045374+0000","last_change":"2024-04-22T19:51:34.557926+0000","last_active":"2024-04-22T20:45:35.045374+0000","last_peered":"2024-04-22T20:45:35.045374+0000","last_clean":"2024-04-22T20:45:35.045374+0000","last_became_active":"2024-04-22T19:51:34.557662+0000","last_became_peered":"2024-04-22T19:51:34.557662+0000","last_unstale":"2024-04-22T20:45:35.045374+0000","last_undegraded":"2024-04-22T20:45:35.045374+0000","last_fullsized":"2024-04-22T20:45:35.045374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:25.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"261'562","reported_seq":1370,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.537178+0000","last_change":"2024-04-22T19:51:36.571168+0000","last_active":"2024-04-22T20:48:08.537178+0000","last_peered":"2024-04-22T20:48:08.537178+0000","last_clean":"2024-04-22T20:48:08.537178+0000","last_became_active":"2024-04-22T19:51:36.570520+0000","last_became_peered":"2024-04-22T19:51:36.570520+0000","last_unstale":"2024-04-22T20:48:08.537178+0000","last_undegraded":"2024-04-22T20:48:08.537178+0000","last_fullsized":"2024-04-22T20:48:08.537178+0000","mapping_epoch":30,"log_start":"261'500","ondisk_log_start":"261'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":500,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:56.456085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1283,"num_read_kb":1280,"num_write":1166,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"261'311","reported_seq":959,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181329+0000","last_change":"2024-04-22T19:51:28.512328+0000","last_active":"2024-04-22T20:45:40.181329+0000","last_peered":"2024-04-22T20:45:40.181329+0000","last_clean":"2024-04-22T20:45:40.181329+0000","last_became_active":"2024-04-22T19:51:28.511527+0000","last_became_peered":"2024-04-22T19:51:28.511527+0000","last_unstale":"2024-04-22T20:45:40.181329+0000","last_undegraded":"2024-04-22T20:45:40.181329+0000","last_fullsized":"2024-04-22T20:45:40.181329+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:22.483205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1221,"num_read_kb":977,"num_write":455,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"261'258","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.064025+0000","last_change":"2024-04-22T19:51:26.505351+0000","last_active":"2024-04-22T20:45:30.064025+0000","last_peered":"2024-04-22T20:45:30.064025+0000","last_clean":"2024-04-22T20:45:30.064025+0000","last_became_active":"2024-04-22T19:51:26.504472+0000","last_became_peered":"2024-04-22T19:51:26.504472+0000","last_unstale":"2024-04-22T20:45:30.064025+0000","last_undegraded":"2024-04-22T20:45:30.064025+0000","last_fullsized":"2024-04-22T20:45:30.064025+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:25.100075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11703329,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":411,"num_read_kb":33343,"num_write":1065,"num_write_kb":49829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"32'5","reported_seq":826,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:07.851401+0000","last_change":"2024-04-22T19:51:30.524309+0000","last_active":"2024-04-22T20:48:07.851401+0000","last_peered":"2024-04-22T20:48:07.851401+0000","last_clean":"2024-04-22T20:48:07.851401+0000","last_became_active":"2024-04-22T19:51:30.524043+0000","last_became_peered":"2024-04-22T19:51:30.524043+0000","last_unstale":"2024-04-22T20:48:07.851401+0000","last_undegraded":"2024-04-22T20:48:07.851401+0000","last_fullsized":"2024-04-22T20:48:07.851401+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:51.651333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":489,"num_read_kb":794,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"261'261","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.172994+0000","last_change":"2024-04-22T19:51:32.539254+0000","last_active":"2024-04-22T20:47:50.172994+0000","last_peered":"2024-04-22T20:47:50.172994+0000","last_clean":"2024-04-22T20:47:50.172994+0000","last_became_active":"2024-04-22T19:51:32.537968+0000","last_became_peered":"2024-04-22T19:51:32.537968+0000","last_unstale":"2024-04-22T20:47:50.172994+0000","last_undegraded":"2024-04-22T20:47:50.172994+0000","last_fullsized":"2024-04-22T20:47:50.172994+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:38.345231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51807,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":422,"num_read_kb":877,"num_write":344,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158827+0000","last_change":"2024-04-22T19:51:34.556181+0000","last_active":"2024-04-22T20:45:50.158827+0000","last_peered":"2024-04-22T20:45:50.158827+0000","last_clean":"2024-04-22T20:45:50.158827+0000","last_became_active":"2024-04-22T19:51:34.555889+0000","last_became_peered":"2024-04-22T19:51:34.555889+0000","last_unstale":"2024-04-22T20:45:50.158827+0000","last_undegraded":"2024-04-22T20:45:50.158827+0000","last_fullsized":"2024-04-22T20:45:50.158827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:03:03.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"261'79","reported_seq":577,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.319292+0000","last_change":"2024-04-22T19:51:36.574838+0000","last_active":"2024-04-22T20:45:55.319292+0000","last_peered":"2024-04-22T20:45:55.319292+0000","last_clean":"2024-04-22T20:45:55.319292+0000","last_became_active":"2024-04-22T19:51:36.574555+0000","last_became_peered":"2024-04-22T19:51:36.574555+0000","last_unstale":"2024-04-22T20:45:55.319292+0000","last_undegraded":"2024-04-22T20:45:55.319292+0000","last_fullsized":"2024-04-22T20:45:55.319292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:07.667952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":91,"num_write":194,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"261'188","reported_seq":783,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.115015+0000","last_change":"2024-04-22T19:51:28.519762+0000","last_active":"2024-04-22T20:46:41.115015+0000","last_peered":"2024-04-22T20:46:41.115015+0000","last_clean":"2024-04-22T20:46:41.115015+0000","last_became_active":"2024-04-22T19:51:28.519424+0000","last_became_peered":"2024-04-22T19:51:28.519424+0000","last_unstale":"2024-04-22T20:46:41.115015+0000","last_undegraded":"2024-04-22T20:46:41.115015+0000","last_fullsized":"2024-04-22T20:46:41.115015+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:13.130467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":758,"num_read_kb":601,"num_write":297,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"261'219","reported_seq":841,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.063988+0000","last_change":"2024-04-22T19:51:26.511055+0000","last_active":"2024-04-22T20:45:30.063988+0000","last_peered":"2024-04-22T20:45:30.063988+0000","last_clean":"2024-04-22T20:45:30.063988+0000","last_became_active":"2024-04-22T19:51:26.510273+0000","last_became_peered":"2024-04-22T19:51:26.510273+0000","last_unstale":"2024-04-22T20:45:30.063988+0000","last_undegraded":"2024-04-22T20:45:30.063988+0000","last_fullsized":"2024-04-22T20:45:30.063988+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:53.305766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15039158,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":310,"num_read_kb":64999,"num_write":940,"num_write_kb":29343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.067468+0000","last_change":"2024-04-22T19:51:30.525335+0000","last_active":"2024-04-22T20:45:30.067468+0000","last_peered":"2024-04-22T20:45:30.067468+0000","last_clean":"2024-04-22T20:45:30.067468+0000","last_became_active":"2024-04-22T19:51:30.525018+0000","last_became_peered":"2024-04-22T19:51:30.525018+0000","last_unstale":"2024-04-22T20:45:30.067468+0000","last_undegraded":"2024-04-22T20:45:30.067468+0000","last_fullsized":"2024-04-22T20:45:30.067468+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:49:24.770975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"261'304","reported_seq":892,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.085578+0000","last_change":"2024-04-22T19:51:32.538989+0000","last_active":"2024-04-22T20:47:55.085578+0000","last_peered":"2024-04-22T20:47:55.085578+0000","last_clean":"2024-04-22T20:47:55.085578+0000","last_became_active":"2024-04-22T19:51:32.538656+0000","last_became_peered":"2024-04-22T19:51:32.538656+0000","last_unstale":"2024-04-22T20:47:55.085578+0000","last_undegraded":"2024-04-22T20:47:55.085578+0000","last_fullsized":"2024-04-22T20:47:55.085578+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:36:49.025357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61009,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":485,"num_read_kb":947,"num_write":390,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"261'504","reported_seq":2193,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.557320+0000","last_change":"2024-04-22T19:51:34.561771+0000","last_active":"2024-04-22T20:48:08.557320+0000","last_peered":"2024-04-22T20:48:08.557320+0000","last_clean":"2024-04-22T20:48:08.557320+0000","last_became_active":"2024-04-22T19:51:34.561488+0000","last_became_peered":"2024-04-22T19:51:34.561488+0000","last_unstale":"2024-04-22T20:48:08.557320+0000","last_undegraded":"2024-04-22T20:48:08.557320+0000","last_fullsized":"2024-04-22T20:48:08.557320+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":400,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:08:35.813443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"261'134","reported_seq":623,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.184424+0000","last_change":"2024-04-22T19:51:36.573554+0000","last_active":"2024-04-22T20:45:50.184424+0000","last_peered":"2024-04-22T20:45:50.184424+0000","last_clean":"2024-04-22T20:45:50.184424+0000","last_became_active":"2024-04-22T19:51:36.573167+0000","last_became_peered":"2024-04-22T19:51:36.573167+0000","last_unstale":"2024-04-22T20:45:50.184424+0000","last_undegraded":"2024-04-22T20:45:50.184424+0000","last_fullsized":"2024-04-22T20:45:50.184424+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:24.336268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"261'150","reported_seq":715,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181727+0000","last_change":"2024-04-22T19:51:28.518692+0000","last_active":"2024-04-22T20:45:30.181727+0000","last_peered":"2024-04-22T20:45:30.181727+0000","last_clean":"2024-04-22T20:45:30.181727+0000","last_became_active":"2024-04-22T19:51:28.518398+0000","last_became_peered":"2024-04-22T19:51:28.518398+0000","last_unstale":"2024-04-22T20:45:30.181727+0000","last_undegraded":"2024-04-22T20:45:30.181727+0000","last_fullsized":"2024-04-22T20:45:30.181727+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:33.700338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":596,"num_read_kb":465,"num_write":243,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"261'246","reported_seq":915,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.890178+0000","last_change":"2024-04-22T19:52:05.113375+0000","last_active":"2024-04-22T20:46:41.890178+0000","last_peered":"2024-04-22T20:46:41.890178+0000","last_clean":"2024-04-22T20:46:41.890178+0000","last_became_active":"2024-04-22T19:52:05.112910+0000","last_became_peered":"2024-04-22T19:52:05.112910+0000","last_unstale":"2024-04-22T20:46:41.890178+0000","last_undegraded":"2024-04-22T20:46:41.890178+0000","last_fullsized":"2024-04-22T20:46:41.890178+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:44:17.477228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4708959,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":450,"num_read_kb":39016,"num_write":1025,"num_write_kb":34073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.069399+0000","last_change":"2024-04-22T19:51:30.529388+0000","last_active":"2024-04-22T20:45:50.069399+0000","last_peered":"2024-04-22T20:45:50.069399+0000","last_clean":"2024-04-22T20:45:50.069399+0000","last_became_active":"2024-04-22T19:51:30.529011+0000","last_became_peered":"2024-04-22T19:51:30.529011+0000","last_unstale":"2024-04-22T20:45:50.069399+0000","last_undegraded":"2024-04-22T20:45:50.069399+0000","last_fullsized":"2024-04-22T20:45:50.069399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:02:44.821662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"261'478","reported_seq":1102,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.503866+0000","last_change":"2024-04-22T19:51:32.537571+0000","last_active":"2024-04-22T20:48:08.503866+0000","last_peered":"2024-04-22T20:48:08.503866+0000","last_clean":"2024-04-22T20:48:08.503866+0000","last_became_active":"2024-04-22T19:51:32.537267+0000","last_became_peered":"2024-04-22T19:51:32.537267+0000","last_unstale":"2024-04-22T20:48:08.503866+0000","last_undegraded":"2024-04-22T20:48:08.503866+0000","last_fullsized":"2024-04-22T20:48:08.503866+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":400,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:04.937868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123858,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":830,"num_read_kb":1829,"num_write":692,"num_write_kb":607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.068905+0000","last_change":"2024-04-22T19:51:34.552745+0000","last_active":"2024-04-22T20:45:45.068905+0000","last_peered":"2024-04-22T20:45:45.068905+0000","last_clean":"2024-04-22T20:45:45.068905+0000","last_became_active":"2024-04-22T19:51:34.552446+0000","last_became_peered":"2024-04-22T19:51:34.552446+0000","last_unstale":"2024-04-22T20:45:45.068905+0000","last_undegraded":"2024-04-22T20:45:45.068905+0000","last_fullsized":"2024-04-22T20:45:45.068905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:59.206061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"261'106","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.047062+0000","last_change":"2024-04-22T19:51:36.570982+0000","last_active":"2024-04-22T20:45:50.047062+0000","last_peered":"2024-04-22T20:45:50.047062+0000","last_clean":"2024-04-22T20:45:50.047062+0000","last_became_active":"2024-04-22T19:51:36.570666+0000","last_became_peered":"2024-04-22T19:51:36.570666+0000","last_unstale":"2024-04-22T20:45:50.047062+0000","last_undegraded":"2024-04-22T20:45:50.047062+0000","last_fullsized":"2024-04-22T20:45:50.047062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:18.966614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":242,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"261'168","reported_seq":795,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:16.403532+0000","last_change":"2024-04-22T19:51:28.508639+0000","last_active":"2024-04-22T20:47:16.403532+0000","last_peered":"2024-04-22T20:47:16.403532+0000","last_clean":"2024-04-22T20:47:16.403532+0000","last_became_active":"2024-04-22T19:51:28.508288+0000","last_became_peered":"2024-04-22T19:51:28.508288+0000","last_unstale":"2024-04-22T20:47:16.403532+0000","last_undegraded":"2024-04-22T20:47:16.403532+0000","last_fullsized":"2024-04-22T20:47:16.403532+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:22:30.411824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":576,"num_write":244,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"261'258","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:48.776114+0000","last_change":"2024-04-22T19:51:26.506251+0000","last_active":"2024-04-22T20:46:48.776114+0000","last_peered":"2024-04-22T20:46:48.776114+0000","last_clean":"2024-04-22T20:46:48.776114+0000","last_became_active":"2024-04-22T19:51:26.503669+0000","last_became_peered":"2024-04-22T19:51:26.503669+0000","last_unstale":"2024-04-22T20:46:48.776114+0000","last_undegraded":"2024-04-22T20:46:48.776114+0000","last_fullsized":"2024-04-22T20:46:48.776114+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:52.491938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8753181,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":397,"num_read_kb":4498,"num_write":1038,"num_write_kb":33252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.067499+0000","last_change":"2024-04-22T19:51:30.526698+0000","last_active":"2024-04-22T20:45:30.067499+0000","last_peered":"2024-04-22T20:45:30.067499+0000","last_clean":"2024-04-22T20:45:30.067499+0000","last_became_active":"2024-04-22T19:51:30.526420+0000","last_became_peered":"2024-04-22T19:51:30.526420+0000","last_unstale":"2024-04-22T20:45:30.067499+0000","last_undegraded":"2024-04-22T20:45:30.067499+0000","last_fullsized":"2024-04-22T20:45:30.067499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:14.342753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"261'264","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.199269+0000","last_change":"2024-04-22T19:51:32.536764+0000","last_active":"2024-04-22T20:47:50.199269+0000","last_peered":"2024-04-22T20:47:50.199269+0000","last_clean":"2024-04-22T20:47:50.199269+0000","last_became_active":"2024-04-22T19:51:32.536463+0000","last_became_peered":"2024-04-22T19:51:32.536463+0000","last_unstale":"2024-04-22T20:47:50.199269+0000","last_undegraded":"2024-04-22T20:47:50.199269+0000","last_fullsized":"2024-04-22T20:47:50.199269+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:29:00.474073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48720,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":444,"num_read_kb":848,"num_write":396,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183789+0000","last_change":"2024-04-22T19:51:34.555066+0000","last_active":"2024-04-22T20:45:45.183789+0000","last_peered":"2024-04-22T20:45:45.183789+0000","last_clean":"2024-04-22T20:45:45.183789+0000","last_became_active":"2024-04-22T19:51:34.554737+0000","last_became_peered":"2024-04-22T19:51:34.554737+0000","last_unstale":"2024-04-22T20:45:45.183789+0000","last_undegraded":"2024-04-22T20:45:45.183789+0000","last_fullsized":"2024-04-22T20:45:45.183789+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:09:00.070765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"261'93","reported_seq":590,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:42.565656+0000","last_change":"2024-04-22T19:51:36.571558+0000","last_active":"2024-04-22T20:46:42.565656+0000","last_peered":"2024-04-22T20:46:42.565656+0000","last_clean":"2024-04-22T20:46:42.565656+0000","last_became_active":"2024-04-22T19:51:36.571086+0000","last_became_peered":"2024-04-22T19:51:36.571086+0000","last_unstale":"2024-04-22T20:46:42.565656+0000","last_undegraded":"2024-04-22T20:46:42.565656+0000","last_fullsized":"2024-04-22T20:46:42.565656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:37:15.182216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":72,"num_write":227,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"261'253","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:07.364455+0000","last_change":"2024-04-22T19:51:28.514713+0000","last_active":"2024-04-22T20:46:07.364455+0000","last_peered":"2024-04-22T20:46:07.364455+0000","last_clean":"2024-04-22T20:46:07.364455+0000","last_became_active":"2024-04-22T19:51:28.514445+0000","last_became_peered":"2024-04-22T19:51:28.514445+0000","last_unstale":"2024-04-22T20:46:07.364455+0000","last_undegraded":"2024-04-22T20:46:07.364455+0000","last_fullsized":"2024-04-22T20:46:07.364455+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:39.638389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":723,"num_write":394,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'331","reported_seq":1077,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:59.585410+0000","last_change":"2024-04-22T19:51:26.496789+0000","last_active":"2024-04-22T20:46:59.585410+0000","last_peered":"2024-04-22T20:46:59.585410+0000","last_clean":"2024-04-22T20:46:59.585410+0000","last_became_active":"2024-04-22T19:51:26.496408+0000","last_became_peered":"2024-04-22T19:51:26.496408+0000","last_unstale":"2024-04-22T20:46:59.585410+0000","last_undegraded":"2024-04-22T20:46:59.585410+0000","last_fullsized":"2024-04-22T20:46:59.585410+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:19:23.202029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1053522,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":776,"num_read_kb":8021,"num_write":1416,"num_write_kb":27015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.044898+0000","last_change":"2024-04-22T19:51:30.527588+0000","last_active":"2024-04-22T20:45:30.044898+0000","last_peered":"2024-04-22T20:45:30.044898+0000","last_clean":"2024-04-22T20:45:30.044898+0000","last_became_active":"2024-04-22T19:51:30.527309+0000","last_became_peered":"2024-04-22T19:51:30.527309+0000","last_unstale":"2024-04-22T20:45:30.044898+0000","last_undegraded":"2024-04-22T20:45:30.044898+0000","last_fullsized":"2024-04-22T20:45:30.044898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:22:07.894332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"261'303","reported_seq":894,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.082212+0000","last_change":"2024-04-22T19:51:32.540003+0000","last_active":"2024-04-22T20:48:00.082212+0000","last_peered":"2024-04-22T20:48:00.082212+0000","last_clean":"2024-04-22T20:48:00.082212+0000","last_became_active":"2024-04-22T19:51:32.539732+0000","last_became_peered":"2024-04-22T19:51:32.539732+0000","last_unstale":"2024-04-22T20:48:00.082212+0000","last_undegraded":"2024-04-22T20:48:00.082212+0000","last_fullsized":"2024-04-22T20:48:00.082212+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:19.413189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63653,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":499,"num_read_kb":1000,"num_write":402,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"261'485","reported_seq":2198,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.553773+0000","last_change":"2024-04-22T19:51:34.556537+0000","last_active":"2024-04-22T20:48:08.553773+0000","last_peered":"2024-04-22T20:48:08.553773+0000","last_clean":"2024-04-22T20:48:08.553773+0000","last_became_active":"2024-04-22T19:51:34.556124+0000","last_became_peered":"2024-04-22T19:51:34.556124+0000","last_unstale":"2024-04-22T20:48:08.553773+0000","last_undegraded":"2024-04-22T20:48:08.553773+0000","last_fullsized":"2024-04-22T20:48:08.553773+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":400,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:14:07.299855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"261'116","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.060760+0000","last_change":"2024-04-22T19:51:36.573163+0000","last_active":"2024-04-22T20:47:55.060760+0000","last_peered":"2024-04-22T20:47:55.060760+0000","last_clean":"2024-04-22T20:47:55.060760+0000","last_became_active":"2024-04-22T19:51:36.572913+0000","last_became_peered":"2024-04-22T19:51:36.572913+0000","last_unstale":"2024-04-22T20:47:55.060760+0000","last_undegraded":"2024-04-22T20:47:55.060760+0000","last_fullsized":"2024-04-22T20:47:55.060760+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:40:20.320547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":102,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"261'338","reported_seq":1080,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.179830+0000","last_change":"2024-04-22T19:51:28.516099+0000","last_active":"2024-04-22T20:45:30.179830+0000","last_peered":"2024-04-22T20:45:30.179830+0000","last_clean":"2024-04-22T20:45:30.179830+0000","last_became_active":"2024-04-22T19:51:28.515809+0000","last_became_peered":"2024-04-22T19:51:28.515809+0000","last_unstale":"2024-04-22T20:45:30.179830+0000","last_undegraded":"2024-04-22T20:45:30.179830+0000","last_fullsized":"2024-04-22T20:45:30.179830+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:19.069001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1504,"num_read_kb":1227,"num_write":616,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"261'280","reported_seq":915,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181307+0000","last_change":"2024-04-22T19:51:26.499730+0000","last_active":"2024-04-22T20:45:40.181307+0000","last_peered":"2024-04-22T20:45:40.181307+0000","last_clean":"2024-04-22T20:45:40.181307+0000","last_became_active":"2024-04-22T19:51:26.499362+0000","last_became_peered":"2024-04-22T19:51:26.499362+0000","last_unstale":"2024-04-22T20:45:40.181307+0000","last_undegraded":"2024-04-22T20:45:40.181307+0000","last_fullsized":"2024-04-22T20:45:40.181307+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:28:07.429252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9762556,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":451,"num_read_kb":26712,"num_write":1196,"num_write_kb":36357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"25'2","reported_seq":1134,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.108639+0000","last_change":"2024-04-22T19:51:30.527770+0000","last_active":"2024-04-22T20:48:15.108639+0000","last_peered":"2024-04-22T20:48:15.108639+0000","last_clean":"2024-04-22T20:48:15.108639+0000","last_became_active":"2024-04-22T19:51:30.527490+0000","last_became_peered":"2024-04-22T19:51:30.527490+0000","last_unstale":"2024-04-22T20:48:15.108639+0000","last_undegraded":"2024-04-22T20:48:15.108639+0000","last_fullsized":"2024-04-22T20:48:15.108639+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:39:09.682743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":639,"num_read_kb":639,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"261'90","reported_seq":620,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157133+0000","last_change":"2024-04-22T19:51:33.248919+0000","last_active":"2024-04-22T20:45:35.157133+0000","last_peered":"2024-04-22T20:45:35.157133+0000","last_clean":"2024-04-22T20:45:35.157133+0000","last_became_active":"2024-04-22T19:51:33.248349+0000","last_became_peered":"2024-04-22T19:51:33.248349+0000","last_unstale":"2024-04-22T20:45:35.157133+0000","last_undegraded":"2024-04-22T20:45:35.157133+0000","last_fullsized":"2024-04-22T20:45:35.157133+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:03:31.126924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":60,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"261'515","reported_seq":2069,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.554704+0000","last_change":"2024-04-22T19:51:34.562158+0000","last_active":"2024-04-22T20:48:08.554704+0000","last_peered":"2024-04-22T20:48:08.554704+0000","last_clean":"2024-04-22T20:48:08.554704+0000","last_became_active":"2024-04-22T19:51:34.561825+0000","last_became_peered":"2024-04-22T19:51:34.561825+0000","last_unstale":"2024-04-22T20:48:08.554704+0000","last_undegraded":"2024-04-22T20:48:08.554704+0000","last_fullsized":"2024-04-22T20:48:08.554704+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:22:52.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"261'130","reported_seq":625,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.182787+0000","last_change":"2024-04-22T19:51:36.573308+0000","last_active":"2024-04-22T20:45:50.182787+0000","last_peered":"2024-04-22T20:45:50.182787+0000","last_clean":"2024-04-22T20:45:50.182787+0000","last_became_active":"2024-04-22T19:51:36.573026+0000","last_became_peered":"2024-04-22T19:51:36.573026+0000","last_unstale":"2024-04-22T20:45:50.182787+0000","last_undegraded":"2024-04-22T20:45:50.182787+0000","last_fullsized":"2024-04-22T20:45:50.182787+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:19.108938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":119,"num_write":309,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"261'207","reported_seq":818,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045988+0000","last_change":"2024-04-22T19:51:28.512959+0000","last_active":"2024-04-22T20:45:40.045988+0000","last_peered":"2024-04-22T20:45:40.045988+0000","last_clean":"2024-04-22T20:45:40.045988+0000","last_became_active":"2024-04-22T19:51:28.512275+0000","last_became_peered":"2024-04-22T19:51:28.512275+0000","last_unstale":"2024-04-22T20:45:40.045988+0000","last_undegraded":"2024-04-22T20:45:40.045988+0000","last_fullsized":"2024-04-22T20:45:40.045988+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:24:20.294556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":832,"num_read_kb":677,"num_write":302,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"261'258","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:31.723467+0000","last_change":"2024-04-22T19:51:26.498630+0000","last_active":"2024-04-22T20:46:31.723467+0000","last_peered":"2024-04-22T20:46:31.723467+0000","last_clean":"2024-04-22T20:46:31.723467+0000","last_became_active":"2024-04-22T19:51:26.498276+0000","last_became_peered":"2024-04-22T19:51:26.498276+0000","last_unstale":"2024-04-22T20:46:31.723467+0000","last_undegraded":"2024-04-22T20:46:31.723467+0000","last_fullsized":"2024-04-22T20:46:31.723467+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:43:15.698499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9975117,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":398,"num_read_kb":30145,"num_write":1092,"num_write_kb":23135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"32'3","reported_seq":820,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.478858+0000","last_change":"2024-04-22T19:51:30.530726+0000","last_active":"2024-04-22T20:48:08.478858+0000","last_peered":"2024-04-22T20:48:08.478858+0000","last_clean":"2024-04-22T20:48:08.478858+0000","last_became_active":"2024-04-22T19:51:30.530363+0000","last_became_peered":"2024-04-22T19:51:30.530363+0000","last_unstale":"2024-04-22T20:48:08.478858+0000","last_undegraded":"2024-04-22T20:48:08.478858+0000","last_fullsized":"2024-04-22T20:48:08.478858+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:58:21.578312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":489,"num_read_kb":487,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"261'396","reported_seq":999,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:06.461115+0000","last_change":"2024-04-22T19:51:32.540935+0000","last_active":"2024-04-22T20:48:06.461115+0000","last_peered":"2024-04-22T20:48:06.461115+0000","last_clean":"2024-04-22T20:48:06.461115+0000","last_became_active":"2024-04-22T19:51:32.540622+0000","last_became_peered":"2024-04-22T19:51:32.540622+0000","last_unstale":"2024-04-22T20:48:06.461115+0000","last_undegraded":"2024-04-22T20:48:06.461115+0000","last_fullsized":"2024-04-22T20:48:06.461115+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":300,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:57:03.444994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1216,"num_read_kb":1715,"num_write":1756,"num_write_kb":1624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.316223+0000","last_change":"2024-04-22T19:51:34.551287+0000","last_active":"2024-04-22T20:45:35.316223+0000","last_peered":"2024-04-22T20:45:35.316223+0000","last_clean":"2024-04-22T20:45:35.316223+0000","last_became_active":"2024-04-22T19:51:34.551009+0000","last_became_peered":"2024-04-22T19:51:34.551009+0000","last_unstale":"2024-04-22T20:45:35.316223+0000","last_undegraded":"2024-04-22T20:45:35.316223+0000","last_fullsized":"2024-04-22T20:45:35.316223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:24:58.026337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"261'121","reported_seq":620,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.018220+0000","last_change":"2024-04-22T19:51:36.571807+0000","last_active":"2024-04-22T20:48:04.018220+0000","last_peered":"2024-04-22T20:48:04.018220+0000","last_clean":"2024-04-22T20:48:04.018220+0000","last_became_active":"2024-04-22T19:51:36.571517+0000","last_became_peered":"2024-04-22T19:51:36.571517+0000","last_unstale":"2024-04-22T20:48:04.018220+0000","last_undegraded":"2024-04-22T20:48:04.018220+0000","last_fullsized":"2024-04-22T20:48:04.018220+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:15:52.550381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":272,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"261'4186","reported_seq":4781,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.317812+0000","last_change":"2024-04-22T19:51:28.515172+0000","last_active":"2024-04-22T20:45:45.317812+0000","last_peered":"2024-04-22T20:45:45.317812+0000","last_clean":"2024-04-22T20:45:45.317812+0000","last_became_active":"2024-04-22T19:51:28.514869+0000","last_became_peered":"2024-04-22T19:51:28.514869+0000","last_unstale":"2024-04-22T20:45:45.317812+0000","last_undegraded":"2024-04-22T20:45:45.317812+0000","last_fullsized":"2024-04-22T20:45:45.317812+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":3000,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:44.084365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14767,"num_read_kb":11981,"num_write":6295,"num_write_kb":3142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":520,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063174+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T20:45:25.063174+0000","last_peered":"2024-04-22T20:45:25.063174+0000","last_clean":"2024-04-22T20:45:25.063174+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T20:45:25.063174+0000","last_undegraded":"2024-04-22T20:45:25.063174+0000","last_fullsized":"2024-04-22T20:45:25.063174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"261'278","reported_seq":973,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.326360+0000","last_change":"2024-04-22T19:51:26.506322+0000","last_active":"2024-04-22T20:47:44.326360+0000","last_peered":"2024-04-22T20:47:44.326360+0000","last_clean":"2024-04-22T20:47:44.326360+0000","last_became_active":"2024-04-22T19:51:26.505982+0000","last_became_peered":"2024-04-22T19:51:26.505982+0000","last_unstale":"2024-04-22T20:47:44.326360+0000","last_undegraded":"2024-04-22T20:47:44.326360+0000","last_fullsized":"2024-04-22T20:47:44.326360+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:46.483074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691084,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":484,"num_read_kb":107291,"num_write":1302,"num_write_kb":45616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.065725+0000","last_change":"2024-04-22T19:51:30.527554+0000","last_active":"2024-04-22T20:45:40.065725+0000","last_peered":"2024-04-22T20:45:40.065725+0000","last_clean":"2024-04-22T20:45:40.065725+0000","last_became_active":"2024-04-22T19:51:30.526939+0000","last_became_peered":"2024-04-22T19:51:30.526939+0000","last_unstale":"2024-04-22T20:45:40.065725+0000","last_undegraded":"2024-04-22T20:45:40.065725+0000","last_fullsized":"2024-04-22T20:45:40.065725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:34:00.754603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"261'700","reported_seq":1386,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.522859+0000","last_change":"2024-04-22T19:51:32.543191+0000","last_active":"2024-04-22T20:48:08.522859+0000","last_peered":"2024-04-22T20:48:08.522859+0000","last_clean":"2024-04-22T20:48:08.522859+0000","last_became_active":"2024-04-22T19:51:32.542970+0000","last_became_peered":"2024-04-22T19:51:32.542970+0000","last_unstale":"2024-04-22T20:48:08.522859+0000","last_undegraded":"2024-04-22T20:48:08.522859+0000","last_fullsized":"2024-04-22T20:48:08.522859+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":600,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:42.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171781,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1364,"num_read_kb":2811,"num_write":1300,"num_write_kb":1164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183762+0000","last_change":"2024-04-22T19:51:34.554048+0000","last_active":"2024-04-22T20:45:45.183762+0000","last_peered":"2024-04-22T20:45:45.183762+0000","last_clean":"2024-04-22T20:45:45.183762+0000","last_became_active":"2024-04-22T19:51:34.553704+0000","last_became_peered":"2024-04-22T19:51:34.553704+0000","last_unstale":"2024-04-22T20:45:45.183762+0000","last_undegraded":"2024-04-22T20:45:45.183762+0000","last_fullsized":"2024-04-22T20:45:45.183762+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:19:14.613388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"261'266","reported_seq":1001,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.066411+0000","last_change":"2024-04-22T19:51:36.571003+0000","last_active":"2024-04-22T20:48:04.066411+0000","last_peered":"2024-04-22T20:48:04.066411+0000","last_clean":"2024-04-22T20:48:04.066411+0000","last_became_active":"2024-04-22T19:51:36.570717+0000","last_became_peered":"2024-04-22T19:51:36.570717+0000","last_unstale":"2024-04-22T20:48:04.066411+0000","last_undegraded":"2024-04-22T20:48:04.066411+0000","last_fullsized":"2024-04-22T20:48:04.066411+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:16.336092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":568,"num_write":495,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"261'50","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045937+0000","last_change":"2024-04-22T19:51:28.506462+0000","last_active":"2024-04-22T20:45:40.045937+0000","last_peered":"2024-04-22T20:45:40.045937+0000","last_clean":"2024-04-22T20:45:40.045937+0000","last_became_active":"2024-04-22T19:51:28.505719+0000","last_became_peered":"2024-04-22T19:51:28.505719+0000","last_unstale":"2024-04-22T20:45:40.045937+0000","last_undegraded":"2024-04-22T20:45:40.045937+0000","last_fullsized":"2024-04-22T20:45:40.045937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:02.609039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":69,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":514,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186325+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T20:45:20.186325+0000","last_peered":"2024-04-22T20:45:20.186325+0000","last_clean":"2024-04-22T20:45:20.186325+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T20:45:20.186325+0000","last_undegraded":"2024-04-22T20:45:20.186325+0000","last_fullsized":"2024-04-22T20:45:20.186325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:20:10.880037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"261'278","reported_seq":954,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:18.964925+0000","last_change":"2024-04-22T19:51:26.501306+0000","last_active":"2024-04-22T20:46:18.964925+0000","last_peered":"2024-04-22T20:46:18.964925+0000","last_clean":"2024-04-22T20:46:18.964925+0000","last_became_active":"2024-04-22T19:51:26.500968+0000","last_became_peered":"2024-04-22T19:51:26.500968+0000","last_unstale":"2024-04-22T20:46:18.964925+0000","last_undegraded":"2024-04-22T20:46:18.964925+0000","last_fullsized":"2024-04-22T20:46:18.964925+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:20:53.110538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10522090,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":517,"num_read_kb":30331,"num_write":1214,"num_write_kb":32709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.184374+0000","last_change":"2024-04-22T19:51:30.526621+0000","last_active":"2024-04-22T20:45:50.184374+0000","last_peered":"2024-04-22T20:45:50.184374+0000","last_clean":"2024-04-22T20:45:50.184374+0000","last_became_active":"2024-04-22T19:51:30.526328+0000","last_became_peered":"2024-04-22T19:51:30.526328+0000","last_unstale":"2024-04-22T20:45:50.184374+0000","last_undegraded":"2024-04-22T20:45:50.184374+0000","last_fullsized":"2024-04-22T20:45:50.184374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:59:48.789109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"261'124","reported_seq":677,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.158336+0000","last_change":"2024-04-22T19:51:32.541073+0000","last_active":"2024-04-22T20:45:45.158336+0000","last_peered":"2024-04-22T20:45:45.158336+0000","last_clean":"2024-04-22T20:45:45.158336+0000","last_became_active":"2024-04-22T19:51:32.540805+0000","last_became_peered":"2024-04-22T19:51:32.540805+0000","last_unstale":"2024-04-22T20:45:45.158336+0000","last_undegraded":"2024-04-22T20:45:45.158336+0000","last_fullsized":"2024-04-22T20:45:45.158336+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:55.727535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":112,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.317842+0000","last_change":"2024-04-22T19:51:34.554535+0000","last_active":"2024-04-22T20:45:45.317842+0000","last_peered":"2024-04-22T20:45:45.317842+0000","last_clean":"2024-04-22T20:45:45.317842+0000","last_became_active":"2024-04-22T19:51:34.554307+0000","last_became_peered":"2024-04-22T19:51:34.554307+0000","last_unstale":"2024-04-22T20:45:45.317842+0000","last_undegraded":"2024-04-22T20:45:45.317842+0000","last_fullsized":"2024-04-22T20:45:45.317842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:13.279562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"261'123","reported_seq":625,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.823391+0000","last_change":"2024-04-22T19:51:36.571438+0000","last_active":"2024-04-22T20:48:03.823391+0000","last_peered":"2024-04-22T20:48:03.823391+0000","last_clean":"2024-04-22T20:48:03.823391+0000","last_became_active":"2024-04-22T19:51:36.571130+0000","last_became_peered":"2024-04-22T19:51:36.571130+0000","last_unstale":"2024-04-22T20:48:03.823391+0000","last_undegraded":"2024-04-22T20:48:03.823391+0000","last_fullsized":"2024-04-22T20:48:03.823391+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:10:47.000615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":131,"num_write":305,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"261'90","reported_seq":658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045996+0000","last_change":"2024-04-22T19:51:28.514065+0000","last_active":"2024-04-22T20:45:40.045996+0000","last_peered":"2024-04-22T20:45:40.045996+0000","last_clean":"2024-04-22T20:45:40.045996+0000","last_became_active":"2024-04-22T19:51:28.513451+0000","last_became_peered":"2024-04-22T19:51:28.513451+0000","last_unstale":"2024-04-22T20:45:40.045996+0000","last_undegraded":"2024-04-22T20:45:40.045996+0000","last_fullsized":"2024-04-22T20:45:40.045996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:52:37.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":305,"num_write":130,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":519,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062391+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T20:45:20.062391+0000","last_peered":"2024-04-22T20:45:20.062391+0000","last_clean":"2024-04-22T20:45:20.062391+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T20:45:20.062391+0000","last_undegraded":"2024-04-22T20:45:20.062391+0000","last_fullsized":"2024-04-22T20:45:20.062391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"261'262","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045858+0000","last_change":"2024-04-22T19:51:26.495730+0000","last_active":"2024-04-22T20:45:40.045858+0000","last_peered":"2024-04-22T20:45:40.045858+0000","last_clean":"2024-04-22T20:45:40.045858+0000","last_became_active":"2024-04-22T19:51:26.495405+0000","last_became_peered":"2024-04-22T19:51:26.495405+0000","last_unstale":"2024-04-22T20:45:40.045858+0000","last_undegraded":"2024-04-22T20:45:40.045858+0000","last_fullsized":"2024-04-22T20:45:40.045858+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:00.884012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23787988,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":57834,"num_write":1209,"num_write_kb":50104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"25'1","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.068477+0000","last_change":"2024-04-22T19:51:30.521583+0000","last_active":"2024-04-22T20:45:40.068477+0000","last_peered":"2024-04-22T20:45:40.068477+0000","last_clean":"2024-04-22T20:45:40.068477+0000","last_became_active":"2024-04-22T19:51:30.521134+0000","last_became_peered":"2024-04-22T19:51:30.521134+0000","last_unstale":"2024-04-22T20:45:40.068477+0000","last_undegraded":"2024-04-22T20:45:40.068477+0000","last_fullsized":"2024-04-22T20:45:40.068477+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:49:17.105976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"261'221","reported_seq":1448,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.449853+0000","last_change":"2024-04-22T19:51:33.250383+0000","last_active":"2024-04-22T20:48:08.449853+0000","last_peered":"2024-04-22T20:48:08.449853+0000","last_clean":"2024-04-22T20:48:08.449853+0000","last_became_active":"2024-04-22T19:51:33.250082+0000","last_became_peered":"2024-04-22T19:51:33.250082+0000","last_unstale":"2024-04-22T20:48:08.449853+0000","last_undegraded":"2024-04-22T20:48:08.449853+0000","last_fullsized":"2024-04-22T20:48:08.449853+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:43:34.357226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":188,"num_read_kb":180,"num_write":143,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183725+0000","last_change":"2024-04-22T19:51:34.548541+0000","last_active":"2024-04-22T20:45:45.183725+0000","last_peered":"2024-04-22T20:45:45.183725+0000","last_clean":"2024-04-22T20:45:45.183725+0000","last_became_active":"2024-04-22T19:51:34.546652+0000","last_became_peered":"2024-04-22T19:51:34.546652+0000","last_unstale":"2024-04-22T20:45:45.183725+0000","last_undegraded":"2024-04-22T20:45:45.183725+0000","last_fullsized":"2024-04-22T20:45:45.183725+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:09:15.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"261'254","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.218285+0000","last_change":"2024-04-22T19:51:36.569306+0000","last_active":"2024-04-22T20:48:03.218285+0000","last_peered":"2024-04-22T20:48:03.218285+0000","last_clean":"2024-04-22T20:48:03.218285+0000","last_became_active":"2024-04-22T19:51:36.568767+0000","last_became_peered":"2024-04-22T19:51:36.568767+0000","last_unstale":"2024-04-22T20:48:03.218285+0000","last_undegraded":"2024-04-22T20:48:03.218285+0000","last_fullsized":"2024-04-22T20:48:03.218285+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:39.794900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":625,"num_write":569,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"261'124","reported_seq":751,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183078+0000","last_change":"2024-04-22T19:51:28.518594+0000","last_active":"2024-04-22T20:45:40.183078+0000","last_peered":"2024-04-22T20:45:40.183078+0000","last_clean":"2024-04-22T20:45:40.183078+0000","last_became_active":"2024-04-22T19:51:28.518084+0000","last_became_peered":"2024-04-22T19:51:28.518084+0000","last_unstale":"2024-04-22T20:45:40.183078+0000","last_undegraded":"2024-04-22T20:45:40.183078+0000","last_fullsized":"2024-04-22T20:45:40.183078+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:38:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":533,"num_read_kb":444,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.044325+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T20:45:25.044325+0000","last_peered":"2024-04-22T20:45:25.044325+0000","last_clean":"2024-04-22T20:45:25.044325+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T20:45:25.044325+0000","last_undegraded":"2024-04-22T20:45:25.044325+0000","last_fullsized":"2024-04-22T20:45:25.044325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:22.264706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":668,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063186+0000","last_change":"2024-04-22T19:51:12.892628+0000","last_active":"2024-04-22T20:45:25.063186+0000","last_peered":"2024-04-22T20:45:25.063186+0000","last_clean":"2024-04-22T20:45:25.063186+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T20:45:25.063186+0000","last_undegraded":"2024-04-22T20:45:25.063186+0000","last_fullsized":"2024-04-22T20:45:25.063186+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:34.018008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"261'271","reported_seq":914,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.324539+0000","last_change":"2024-04-22T19:51:26.497957+0000","last_active":"2024-04-22T20:47:44.324539+0000","last_peered":"2024-04-22T20:47:44.324539+0000","last_clean":"2024-04-22T20:47:44.324539+0000","last_became_active":"2024-04-22T19:51:26.497608+0000","last_became_peered":"2024-04-22T19:51:26.497608+0000","last_unstale":"2024-04-22T20:47:44.324539+0000","last_undegraded":"2024-04-22T20:47:44.324539+0000","last_fullsized":"2024-04-22T20:47:44.324539+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:02:40.473180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16410029,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":454,"num_read_kb":42686,"num_write":1180,"num_write_kb":37697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189509+0000","last_change":"2024-04-22T19:51:30.524749+0000","last_active":"2024-04-22T20:45:40.189509+0000","last_peered":"2024-04-22T20:45:40.189509+0000","last_clean":"2024-04-22T20:45:40.189509+0000","last_became_active":"2024-04-22T19:51:30.524437+0000","last_became_peered":"2024-04-22T19:51:30.524437+0000","last_unstale":"2024-04-22T20:45:40.189509+0000","last_undegraded":"2024-04-22T20:45:40.189509+0000","last_fullsized":"2024-04-22T20:45:40.189509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:54.055370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"261'1086","reported_seq":2608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.549870+0000","last_change":"2024-04-22T19:51:32.538859+0000","last_active":"2024-04-22T20:48:08.549870+0000","last_peered":"2024-04-22T20:48:08.549870+0000","last_clean":"2024-04-22T20:48:08.549870+0000","last_became_active":"2024-04-22T19:51:32.538560+0000","last_became_peered":"2024-04-22T19:51:32.538560+0000","last_unstale":"2024-04-22T20:48:08.549870+0000","last_undegraded":"2024-04-22T20:48:08.549870+0000","last_fullsized":"2024-04-22T20:48:08.549870+0000","mapping_epoch":26,"log_start":"261'1000","ondisk_log_start":"261'1000","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":1000,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:19.961941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174953,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1739,"num_read_kb":3173,"num_write":1082,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.067953+0000","last_change":"2024-04-22T19:51:34.557341+0000","last_active":"2024-04-22T20:45:55.067953+0000","last_peered":"2024-04-22T20:45:55.067953+0000","last_clean":"2024-04-22T20:45:55.067953+0000","last_became_active":"2024-04-22T19:51:34.557022+0000","last_became_peered":"2024-04-22T19:51:34.557022+0000","last_unstale":"2024-04-22T20:45:55.067953+0000","last_undegraded":"2024-04-22T20:45:55.067953+0000","last_fullsized":"2024-04-22T20:45:55.067953+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:59:49.539365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"261'163","reported_seq":671,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:00.048103+0000","last_change":"2024-04-22T19:51:36.567874+0000","last_active":"2024-04-22T20:46:00.048103+0000","last_peered":"2024-04-22T20:46:00.048103+0000","last_clean":"2024-04-22T20:46:00.048103+0000","last_became_active":"2024-04-22T19:51:36.567603+0000","last_became_peered":"2024-04-22T19:51:36.567603+0000","last_unstale":"2024-04-22T20:46:00.048103+0000","last_undegraded":"2024-04-22T20:46:00.048103+0000","last_fullsized":"2024-04-22T20:46:00.048103+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:34:18.236728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":641,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":222,"num_read_kb":216,"num_write":373,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"261'364","reported_seq":998,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:32.355984+0000","last_change":"2024-04-22T19:51:28.512889+0000","last_active":"2024-04-22T20:46:32.355984+0000","last_peered":"2024-04-22T20:46:32.355984+0000","last_clean":"2024-04-22T20:46:32.355984+0000","last_became_active":"2024-04-22T19:51:28.512561+0000","last_became_peered":"2024-04-22T19:51:28.512561+0000","last_unstale":"2024-04-22T20:46:32.355984+0000","last_undegraded":"2024-04-22T20:46:32.355984+0000","last_fullsized":"2024-04-22T20:46:32.355984+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:13.130467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1455,"num_read_kb":1128,"num_write":627,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"261'324","reported_seq":1041,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183045+0000","last_change":"2024-04-22T19:51:26.500958+0000","last_active":"2024-04-22T20:45:40.183045+0000","last_peered":"2024-04-22T20:45:40.183045+0000","last_clean":"2024-04-22T20:45:40.183045+0000","last_became_active":"2024-04-22T19:51:26.498178+0000","last_became_peered":"2024-04-22T19:51:26.498178+0000","last_unstale":"2024-04-22T20:45:40.183045+0000","last_undegraded":"2024-04-22T20:45:40.183045+0000","last_fullsized":"2024-04-22T20:45:40.183045+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:54.358145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14073286,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":653,"num_read_kb":47025,"num_write":1473,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.155991+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T20:45:25.155991+0000","last_peered":"2024-04-22T20:45:25.155991+0000","last_clean":"2024-04-22T20:45:25.155991+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T20:45:25.155991+0000","last_undegraded":"2024-04-22T20:45:25.155991+0000","last_fullsized":"2024-04-22T20:45:25.155991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157032+0000","last_change":"2024-04-22T19:51:30.523537+0000","last_active":"2024-04-22T20:45:35.157032+0000","last_peered":"2024-04-22T20:45:35.157032+0000","last_clean":"2024-04-22T20:45:35.157032+0000","last_became_active":"2024-04-22T19:51:30.523242+0000","last_became_peered":"2024-04-22T19:51:30.523242+0000","last_unstale":"2024-04-22T20:45:35.157032+0000","last_undegraded":"2024-04-22T20:45:35.157032+0000","last_fullsized":"2024-04-22T20:45:35.157032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:21.847987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"261'692","reported_seq":1402,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.510075+0000","last_change":"2024-04-22T19:51:33.248507+0000","last_active":"2024-04-22T20:48:08.510075+0000","last_peered":"2024-04-22T20:48:08.510075+0000","last_clean":"2024-04-22T20:48:08.510075+0000","last_became_active":"2024-04-22T19:51:33.248144+0000","last_became_peered":"2024-04-22T19:51:33.248144+0000","last_unstale":"2024-04-22T20:48:08.510075+0000","last_undegraded":"2024-04-22T20:48:08.510075+0000","last_fullsized":"2024-04-22T20:48:08.510075+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":600,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:02:24.314016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":153222,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1224,"num_read_kb":2558,"num_write":1000,"num_write_kb":811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.158258+0000","last_change":"2024-04-22T19:51:34.562290+0000","last_active":"2024-04-22T20:45:45.158258+0000","last_peered":"2024-04-22T20:45:45.158258+0000","last_clean":"2024-04-22T20:45:45.158258+0000","last_became_active":"2024-04-22T19:51:34.562001+0000","last_became_peered":"2024-04-22T19:51:34.562001+0000","last_unstale":"2024-04-22T20:45:45.158258+0000","last_undegraded":"2024-04-22T20:45:45.158258+0000","last_fullsized":"2024-04-22T20:45:45.158258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:30:17.324055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"261'105","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:02.644513+0000","last_change":"2024-04-22T19:51:36.570607+0000","last_active":"2024-04-22T20:48:02.644513+0000","last_peered":"2024-04-22T20:48:02.644513+0000","last_clean":"2024-04-22T20:48:02.644513+0000","last_became_active":"2024-04-22T19:51:36.570134+0000","last_became_peered":"2024-04-22T19:51:36.570134+0000","last_unstale":"2024-04-22T20:48:02.644513+0000","last_undegraded":"2024-04-22T20:48:02.644513+0000","last_fullsized":"2024-04-22T20:48:02.644513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:44.632753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":256,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"261'101","reported_seq":688,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.244225+0000","last_change":"2024-04-22T19:51:28.516624+0000","last_active":"2024-04-22T20:47:05.244225+0000","last_peered":"2024-04-22T20:47:05.244225+0000","last_clean":"2024-04-22T20:47:05.244225+0000","last_became_active":"2024-04-22T19:51:28.516178+0000","last_became_peered":"2024-04-22T19:51:28.516178+0000","last_unstale":"2024-04-22T20:47:05.244225+0000","last_undegraded":"2024-04-22T20:47:05.244225+0000","last_fullsized":"2024-04-22T20:47:05.244225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:27:27.066102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":337,"num_write":140,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"261'295","reported_seq":990,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.157770+0000","last_change":"2024-04-22T19:51:26.503699+0000","last_active":"2024-04-22T20:45:40.157770+0000","last_peered":"2024-04-22T20:45:40.157770+0000","last_clean":"2024-04-22T20:45:40.157770+0000","last_became_active":"2024-04-22T19:51:26.502765+0000","last_became_peered":"2024-04-22T19:51:26.502765+0000","last_unstale":"2024-04-22T20:45:40.157770+0000","last_undegraded":"2024-04-22T20:45:40.157770+0000","last_fullsized":"2024-04-22T20:45:40.157770+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:12:35.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14551057,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":509,"num_read_kb":127053,"num_write":1319,"num_write_kb":34012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":518,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.180533+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T20:45:35.180533+0000","last_peered":"2024-04-22T20:45:35.180533+0000","last_clean":"2024-04-22T20:45:35.180533+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T20:45:35.180533+0000","last_undegraded":"2024-04-22T20:45:35.180533+0000","last_fullsized":"2024-04-22T20:45:35.180533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181286+0000","last_change":"2024-04-22T19:51:30.529790+0000","last_active":"2024-04-22T20:45:40.181286+0000","last_peered":"2024-04-22T20:45:40.181286+0000","last_clean":"2024-04-22T20:45:40.181286+0000","last_became_active":"2024-04-22T19:51:30.529437+0000","last_became_peered":"2024-04-22T19:51:30.529437+0000","last_unstale":"2024-04-22T20:45:40.181286+0000","last_undegraded":"2024-04-22T20:45:40.181286+0000","last_fullsized":"2024-04-22T20:45:40.181286+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:46:59.053426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"261'110","reported_seq":651,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182039+0000","last_change":"2024-04-22T19:51:32.534640+0000","last_active":"2024-04-22T20:45:45.182039+0000","last_peered":"2024-04-22T20:45:45.182039+0000","last_clean":"2024-04-22T20:45:45.182039+0000","last_became_active":"2024-04-22T19:51:32.534280+0000","last_became_peered":"2024-04-22T19:51:32.534280+0000","last_unstale":"2024-04-22T20:45:45.182039+0000","last_undegraded":"2024-04-22T20:45:45.182039+0000","last_fullsized":"2024-04-22T20:45:45.182039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:52.620330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"261'531","reported_seq":2151,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.555610+0000","last_change":"2024-04-22T19:51:34.557012+0000","last_active":"2024-04-22T20:48:08.555610+0000","last_peered":"2024-04-22T20:48:08.555610+0000","last_clean":"2024-04-22T20:48:08.555610+0000","last_became_active":"2024-04-22T19:51:34.556673+0000","last_became_peered":"2024-04-22T19:51:34.556673+0000","last_unstale":"2024-04-22T20:48:08.555610+0000","last_undegraded":"2024-04-22T20:48:08.555610+0000","last_fullsized":"2024-04-22T20:48:08.555610+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":500,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:18.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"261'149","reported_seq":639,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182998+0000","last_change":"2024-04-22T19:51:36.575637+0000","last_active":"2024-04-22T20:45:40.182998+0000","last_peered":"2024-04-22T20:45:40.182998+0000","last_clean":"2024-04-22T20:45:40.182998+0000","last_became_active":"2024-04-22T19:51:36.575351+0000","last_became_peered":"2024-04-22T19:51:36.575351+0000","last_unstale":"2024-04-22T20:45:40.182998+0000","last_undegraded":"2024-04-22T20:45:40.182998+0000","last_fullsized":"2024-04-22T20:45:40.182998+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:38:03.020774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":80,"num_write":319,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"261'146","reported_seq":771,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182955+0000","last_change":"2024-04-22T19:51:28.514202+0000","last_active":"2024-04-22T20:45:40.182955+0000","last_peered":"2024-04-22T20:45:40.182955+0000","last_clean":"2024-04-22T20:45:40.182955+0000","last_became_active":"2024-04-22T19:51:28.513940+0000","last_became_peered":"2024-04-22T19:51:28.513940+0000","last_unstale":"2024-04-22T20:45:40.182955+0000","last_undegraded":"2024-04-22T20:45:40.182955+0000","last_fullsized":"2024-04-22T20:45:40.182955+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:14:36.426996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":550,"num_write":219,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"261'276","reported_seq":914,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:28.475475+0000","last_change":"2024-04-22T19:51:26.506454+0000","last_active":"2024-04-22T20:46:28.475475+0000","last_peered":"2024-04-22T20:46:28.475475+0000","last_clean":"2024-04-22T20:46:28.475475+0000","last_became_active":"2024-04-22T19:51:26.505842+0000","last_became_peered":"2024-04-22T19:51:26.505842+0000","last_unstale":"2024-04-22T20:46:28.475475+0000","last_undegraded":"2024-04-22T20:46:28.475475+0000","last_fullsized":"2024-04-22T20:46:28.475475+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:24:40.890650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10581571,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":407,"num_read_kb":22189,"num_write":1035,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314053+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T20:45:20.314053+0000","last_peered":"2024-04-22T20:45:20.314053+0000","last_clean":"2024-04-22T20:45:20.314053+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T20:45:20.314053+0000","last_undegraded":"2024-04-22T20:45:20.314053+0000","last_fullsized":"2024-04-22T20:45:20.314053+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:01.430732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317090+0000","last_change":"2024-04-22T19:51:30.526076+0000","last_active":"2024-04-22T20:45:40.317090+0000","last_peered":"2024-04-22T20:45:40.317090+0000","last_clean":"2024-04-22T20:45:40.317090+0000","last_became_active":"2024-04-22T19:51:30.525777+0000","last_became_peered":"2024-04-22T19:51:30.525777+0000","last_unstale":"2024-04-22T20:45:40.317090+0000","last_undegraded":"2024-04-22T20:45:40.317090+0000","last_fullsized":"2024-04-22T20:45:40.317090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:18.779258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"261'179","reported_seq":746,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.046459+0000","last_change":"2024-04-22T19:51:33.248819+0000","last_active":"2024-04-22T20:45:45.046459+0000","last_peered":"2024-04-22T20:45:45.046459+0000","last_clean":"2024-04-22T20:45:45.046459+0000","last_became_active":"2024-04-22T19:51:33.248354+0000","last_became_peered":"2024-04-22T19:51:33.248354+0000","last_unstale":"2024-04-22T20:45:45.046459+0000","last_undegraded":"2024-04-22T20:45:45.046459+0000","last_fullsized":"2024-04-22T20:45:45.046459+0000","mapping_epoch":26,"log_start":"258'100","ondisk_log_start":"258'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:02:04.601587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":222,"num_read_kb":211,"num_write":148,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"261'544","reported_seq":4419,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.551803+0000","last_change":"2024-04-22T19:51:34.561431+0000","last_active":"2024-04-22T20:48:08.551803+0000","last_peered":"2024-04-22T20:48:08.551803+0000","last_clean":"2024-04-22T20:48:08.551803+0000","last_became_active":"2024-04-22T19:51:34.561206+0000","last_became_peered":"2024-04-22T19:51:34.561206+0000","last_unstale":"2024-04-22T20:48:08.551803+0000","last_undegraded":"2024-04-22T20:48:08.551803+0000","last_fullsized":"2024-04-22T20:48:08.551803+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":500,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:11:06.654339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"261'182","reported_seq":679,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.174164+0000","last_change":"2024-04-22T19:51:36.569612+0000","last_active":"2024-04-22T20:48:00.174164+0000","last_peered":"2024-04-22T20:48:00.174164+0000","last_clean":"2024-04-22T20:48:00.174164+0000","last_became_active":"2024-04-22T19:51:36.569338+0000","last_became_peered":"2024-04-22T19:51:36.569338+0000","last_unstale":"2024-04-22T20:48:00.174164+0000","last_undegraded":"2024-04-22T20:48:00.174164+0000","last_fullsized":"2024-04-22T20:48:00.174164+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:23:38.495469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":109,"num_write":422,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"261'109","reported_seq":704,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045900+0000","last_change":"2024-04-22T19:51:28.512614+0000","last_active":"2024-04-22T20:45:40.045900+0000","last_peered":"2024-04-22T20:45:40.045900+0000","last_clean":"2024-04-22T20:45:40.045900+0000","last_became_active":"2024-04-22T19:51:28.512337+0000","last_became_peered":"2024-04-22T19:51:28.512337+0000","last_unstale":"2024-04-22T20:45:40.045900+0000","last_undegraded":"2024-04-22T20:45:40.045900+0000","last_fullsized":"2024-04-22T20:45:40.045900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:06:34.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":434,"num_read_kb":362,"num_write":158,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'260","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.190983+0000","last_change":"2024-04-22T19:51:26.508311+0000","last_active":"2024-04-22T20:45:50.190983+0000","last_peered":"2024-04-22T20:45:50.190983+0000","last_clean":"2024-04-22T20:45:50.190983+0000","last_became_active":"2024-04-22T19:51:26.507804+0000","last_became_peered":"2024-04-22T19:51:26.507804+0000","last_unstale":"2024-04-22T20:45:50.190983+0000","last_undegraded":"2024-04-22T20:45:50.190983+0000","last_fullsized":"2024-04-22T20:45:50.190983+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:56:54.837990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111886,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":452,"num_read_kb":11012,"num_write":1175,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314068+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T20:45:20.314068+0000","last_peered":"2024-04-22T20:45:20.314068+0000","last_clean":"2024-04-22T20:45:20.314068+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T20:45:20.314068+0000","last_undegraded":"2024-04-22T20:45:20.314068+0000","last_fullsized":"2024-04-22T20:45:20.314068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:23.800559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182974+0000","last_change":"2024-04-22T19:51:30.527109+0000","last_active":"2024-04-22T20:45:40.182974+0000","last_peered":"2024-04-22T20:45:40.182974+0000","last_clean":"2024-04-22T20:45:40.182974+0000","last_became_active":"2024-04-22T19:51:30.526837+0000","last_became_peered":"2024-04-22T19:51:30.526837+0000","last_unstale":"2024-04-22T20:45:40.182974+0000","last_undegraded":"2024-04-22T20:45:40.182974+0000","last_fullsized":"2024-04-22T20:45:40.182974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:13.442240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"261'532","reported_seq":1192,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.513945+0000","last_change":"2024-04-22T19:51:32.542607+0000","last_active":"2024-04-22T20:48:08.513945+0000","last_peered":"2024-04-22T20:48:08.513945+0000","last_clean":"2024-04-22T20:48:08.513945+0000","last_became_active":"2024-04-22T19:51:32.542344+0000","last_became_peered":"2024-04-22T19:51:32.542344+0000","last_unstale":"2024-04-22T20:48:08.513945+0000","last_undegraded":"2024-04-22T20:48:08.513945+0000","last_fullsized":"2024-04-22T20:48:08.513945+0000","mapping_epoch":26,"log_start":"261'500","ondisk_log_start":"261'500","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":500,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:56:47.214088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115148,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":920,"num_read_kb":1851,"num_write":780,"num_write_kb":630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.181987+0000","last_change":"2024-04-22T19:51:34.551264+0000","last_active":"2024-04-22T20:45:45.181987+0000","last_peered":"2024-04-22T20:45:45.181987+0000","last_clean":"2024-04-22T20:45:45.181987+0000","last_became_active":"2024-04-22T19:51:34.550951+0000","last_became_peered":"2024-04-22T19:51:34.550951+0000","last_unstale":"2024-04-22T20:45:45.181987+0000","last_undegraded":"2024-04-22T20:45:45.181987+0000","last_fullsized":"2024-04-22T20:45:45.181987+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:34:20.504558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"261'112","reported_seq":617,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.198696+0000","last_change":"2024-04-22T19:51:36.573230+0000","last_active":"2024-04-22T20:47:55.198696+0000","last_peered":"2024-04-22T20:47:55.198696+0000","last_clean":"2024-04-22T20:47:55.198696+0000","last_became_active":"2024-04-22T19:51:36.572976+0000","last_became_peered":"2024-04-22T19:51:36.572976+0000","last_unstale":"2024-04-22T20:47:55.198696+0000","last_undegraded":"2024-04-22T20:47:55.198696+0000","last_fullsized":"2024-04-22T20:47:55.198696+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:14.048628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":120,"num_write":270,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"261'130","reported_seq":681,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.160770+0000","last_change":"2024-04-22T19:51:36.574914+0000","last_active":"2024-04-22T20:48:03.160770+0000","last_peered":"2024-04-22T20:48:03.160770+0000","last_clean":"2024-04-22T20:48:03.160770+0000","last_became_active":"2024-04-22T19:51:36.574238+0000","last_became_peered":"2024-04-22T19:51:36.574238+0000","last_unstale":"2024-04-22T20:48:03.160770+0000","last_undegraded":"2024-04-22T20:48:03.160770+0000","last_fullsized":"2024-04-22T20:48:03.160770+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:09.269509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":193,"num_read_kb":192,"num_write":291,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"261'123","reported_seq":739,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176757+0000","last_change":"2024-04-22T19:51:28.515646+0000","last_active":"2024-04-22T20:45:10.176757+0000","last_peered":"2024-04-22T20:45:10.176757+0000","last_clean":"2024-04-22T20:45:10.176757+0000","last_became_active":"2024-04-22T19:51:28.515334+0000","last_became_peered":"2024-04-22T19:51:28.515334+0000","last_unstale":"2024-04-22T20:45:10.176757+0000","last_undegraded":"2024-04-22T20:45:10.176757+0000","last_fullsized":"2024-04-22T20:45:10.176757+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:47:44.805680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":424,"num_write":184,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"261'258","reported_seq":971,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.178176+0000","last_change":"2024-04-22T19:51:26.506527+0000","last_active":"2024-04-22T20:45:20.178176+0000","last_peered":"2024-04-22T20:45:20.178176+0000","last_clean":"2024-04-22T20:45:20.178176+0000","last_became_active":"2024-04-22T19:51:26.506150+0000","last_became_peered":"2024-04-22T19:51:26.506150+0000","last_unstale":"2024-04-22T20:45:20.178176+0000","last_undegraded":"2024-04-22T20:45:20.178176+0000","last_fullsized":"2024-04-22T20:45:20.178176+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:28.117664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7665617,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":650,"num_read_kb":35400,"num_write":1047,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.042550+0000","last_change":"2024-04-22T19:51:30.523906+0000","last_active":"2024-04-22T20:45:10.042550+0000","last_peered":"2024-04-22T20:45:10.042550+0000","last_clean":"2024-04-22T20:45:10.042550+0000","last_became_active":"2024-04-22T19:51:30.522903+0000","last_became_peered":"2024-04-22T19:51:30.522903+0000","last_unstale":"2024-04-22T20:45:10.042550+0000","last_undegraded":"2024-04-22T20:45:10.042550+0000","last_fullsized":"2024-04-22T20:45:10.042550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:12.331782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"261'291","reported_seq":870,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171073+0000","last_change":"2024-04-22T19:51:32.539884+0000","last_active":"2024-04-22T20:47:35.171073+0000","last_peered":"2024-04-22T20:47:35.171073+0000","last_clean":"2024-04-22T20:47:35.171073+0000","last_became_active":"2024-04-22T19:51:32.539631+0000","last_became_peered":"2024-04-22T19:51:32.539631+0000","last_unstale":"2024-04-22T20:47:35.171073+0000","last_undegraded":"2024-04-22T20:47:35.171073+0000","last_fullsized":"2024-04-22T20:47:35.171073+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:00:28.566624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59572,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":482,"num_read_kb":991,"num_write":398,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154587+0000","last_change":"2024-04-22T19:51:34.556940+0000","last_active":"2024-04-22T20:45:15.154587+0000","last_peered":"2024-04-22T20:45:15.154587+0000","last_clean":"2024-04-22T20:45:15.154587+0000","last_became_active":"2024-04-22T19:51:34.556650+0000","last_became_peered":"2024-04-22T19:51:34.556650+0000","last_unstale":"2024-04-22T20:45:15.154587+0000","last_undegraded":"2024-04-22T20:45:15.154587+0000","last_fullsized":"2024-04-22T20:45:15.154587+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:47:48.478954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"261'99","reported_seq":599,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.172452+0000","last_change":"2024-04-22T19:51:36.567251+0000","last_active":"2024-04-22T20:47:45.172452+0000","last_peered":"2024-04-22T20:47:45.172452+0000","last_clean":"2024-04-22T20:47:45.172452+0000","last_became_active":"2024-04-22T19:51:36.566980+0000","last_became_peered":"2024-04-22T19:51:36.566980+0000","last_unstale":"2024-04-22T20:47:45.172452+0000","last_undegraded":"2024-04-22T20:47:45.172452+0000","last_fullsized":"2024-04-22T20:47:45.172452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:01:00.638641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":235,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"261'130","reported_seq":734,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.042558+0000","last_change":"2024-04-22T19:51:28.517914+0000","last_active":"2024-04-22T20:45:10.042558+0000","last_peered":"2024-04-22T20:45:10.042558+0000","last_clean":"2024-04-22T20:45:10.042558+0000","last_became_active":"2024-04-22T19:51:28.517675+0000","last_became_peered":"2024-04-22T19:51:28.517675+0000","last_unstale":"2024-04-22T20:45:10.042558+0000","last_undegraded":"2024-04-22T20:45:10.042558+0000","last_fullsized":"2024-04-22T20:45:10.042558+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:18:10.255589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":543,"num_read_kb":446,"num_write":195,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"261'276","reported_seq":976,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:16.477341+0000","last_change":"2024-04-22T19:51:26.502622+0000","last_active":"2024-04-22T20:45:16.477341+0000","last_peered":"2024-04-22T20:45:16.477341+0000","last_clean":"2024-04-22T20:45:16.477341+0000","last_became_active":"2024-04-22T19:51:26.501897+0000","last_became_peered":"2024-04-22T19:51:26.501897+0000","last_unstale":"2024-04-22T20:45:16.477341+0000","last_undegraded":"2024-04-22T20:45:16.477341+0000","last_fullsized":"2024-04-22T20:45:16.477341+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:33.272475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4946722,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":548,"num_read_kb":41436,"num_write":1081,"num_write_kb":29344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.154110+0000","last_change":"2024-04-22T19:51:30.524665+0000","last_active":"2024-04-22T20:45:10.154110+0000","last_peered":"2024-04-22T20:45:10.154110+0000","last_clean":"2024-04-22T20:45:10.154110+0000","last_became_active":"2024-04-22T19:51:30.524388+0000","last_became_peered":"2024-04-22T19:51:30.524388+0000","last_unstale":"2024-04-22T20:45:10.154110+0000","last_undegraded":"2024-04-22T20:45:10.154110+0000","last_fullsized":"2024-04-22T20:45:10.154110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:58.393800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"261'49","reported_seq":553,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179874+0000","last_change":"2024-04-22T19:51:32.539546+0000","last_active":"2024-04-22T20:45:15.179874+0000","last_peered":"2024-04-22T20:45:15.179874+0000","last_clean":"2024-04-22T20:45:15.179874+0000","last_became_active":"2024-04-22T19:51:32.539260+0000","last_became_peered":"2024-04-22T19:51:32.539260+0000","last_unstale":"2024-04-22T20:45:15.179874+0000","last_undegraded":"2024-04-22T20:45:15.179874+0000","last_fullsized":"2024-04-22T20:45:15.179874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:13:32.092655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":36,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.061461+0000","last_change":"2024-04-22T19:51:34.551584+0000","last_active":"2024-04-22T20:45:15.061461+0000","last_peered":"2024-04-22T20:45:15.061461+0000","last_clean":"2024-04-22T20:45:15.061461+0000","last_became_active":"2024-04-22T19:51:34.551294+0000","last_became_peered":"2024-04-22T19:51:34.551294+0000","last_unstale":"2024-04-22T20:45:15.061461+0000","last_undegraded":"2024-04-22T20:45:15.061461+0000","last_fullsized":"2024-04-22T20:45:15.061461+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:53:58.560478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"261'201","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.005045+0000","last_change":"2024-04-22T19:51:36.574227+0000","last_active":"2024-04-22T20:48:03.005045+0000","last_peered":"2024-04-22T20:48:03.005045+0000","last_clean":"2024-04-22T20:48:03.005045+0000","last_became_active":"2024-04-22T19:51:36.573936+0000","last_became_peered":"2024-04-22T19:51:36.573936+0000","last_unstale":"2024-04-22T20:48:03.005045+0000","last_undegraded":"2024-04-22T20:48:03.005045+0000","last_fullsized":"2024-04-22T20:48:03.005045+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:05:17.804766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":284,"num_write":451,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"261'231","reported_seq":917,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.154075+0000","last_change":"2024-04-22T19:51:28.508566+0000","last_active":"2024-04-22T20:45:10.154075+0000","last_peered":"2024-04-22T20:45:10.154075+0000","last_clean":"2024-04-22T20:45:10.154075+0000","last_became_active":"2024-04-22T19:51:28.508307+0000","last_became_peered":"2024-04-22T19:51:28.508307+0000","last_unstale":"2024-04-22T20:45:10.154075+0000","last_undegraded":"2024-04-22T20:45:10.154075+0000","last_fullsized":"2024-04-22T20:45:10.154075+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:48.292329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":959,"num_read_kb":784,"num_write":348,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"261'268","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180692+0000","last_change":"2024-04-22T19:51:26.501920+0000","last_active":"2024-04-22T20:44:40.180692+0000","last_peered":"2024-04-22T20:44:40.180692+0000","last_clean":"2024-04-22T20:44:40.180692+0000","last_became_active":"2024-04-22T19:51:26.501477+0000","last_became_peered":"2024-04-22T19:51:26.501477+0000","last_unstale":"2024-04-22T20:44:40.180692+0000","last_undegraded":"2024-04-22T20:44:40.180692+0000","last_fullsized":"2024-04-22T20:44:40.180692+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:06.904602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16045811,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":422,"num_read_kb":46522,"num_write":1225,"num_write_kb":35021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060725+0000","last_change":"2024-04-22T19:51:30.527405+0000","last_active":"2024-04-22T20:45:10.060725+0000","last_peered":"2024-04-22T20:45:10.060725+0000","last_clean":"2024-04-22T20:45:10.060725+0000","last_became_active":"2024-04-22T19:51:30.527076+0000","last_became_peered":"2024-04-22T19:51:30.527076+0000","last_unstale":"2024-04-22T20:45:10.060725+0000","last_undegraded":"2024-04-22T20:45:10.060725+0000","last_fullsized":"2024-04-22T20:45:10.060725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:38.558657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"261'297","reported_seq":892,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171120+0000","last_change":"2024-04-22T19:51:32.542586+0000","last_active":"2024-04-22T20:47:35.171120+0000","last_peered":"2024-04-22T20:47:35.171120+0000","last_clean":"2024-04-22T20:47:35.171120+0000","last_became_active":"2024-04-22T19:51:32.542262+0000","last_became_peered":"2024-04-22T19:51:32.542262+0000","last_unstale":"2024-04-22T20:47:35.171120+0000","last_undegraded":"2024-04-22T20:47:35.171120+0000","last_fullsized":"2024-04-22T20:47:35.171120+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:52:10.014200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55208,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":478,"num_read_kb":917,"num_write":378,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.313200+0000","last_change":"2024-04-22T19:51:34.554113+0000","last_active":"2024-04-22T20:45:15.313200+0000","last_peered":"2024-04-22T20:45:15.313200+0000","last_clean":"2024-04-22T20:45:15.313200+0000","last_became_active":"2024-04-22T19:51:34.553691+0000","last_became_peered":"2024-04-22T19:51:34.553691+0000","last_unstale":"2024-04-22T20:45:15.313200+0000","last_undegraded":"2024-04-22T20:45:15.313200+0000","last_fullsized":"2024-04-22T20:45:15.313200+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:08:07.000668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"261'135","reported_seq":616,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:10.176306+0000","last_change":"2024-04-22T19:51:36.571557+0000","last_active":"2024-04-22T20:44:10.176306+0000","last_peered":"2024-04-22T20:44:10.176306+0000","last_clean":"2024-04-22T20:44:10.176306+0000","last_became_active":"2024-04-22T19:51:36.571290+0000","last_became_peered":"2024-04-22T19:51:36.571290+0000","last_unstale":"2024-04-22T20:44:10.176306+0000","last_undegraded":"2024-04-22T20:44:10.176306+0000","last_fullsized":"2024-04-22T20:44:10.176306+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:42:28.669764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":111,"num_write":334,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"261'185","reported_seq":854,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:01.148987+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:47:01.148987+0000","last_peered":"2024-04-22T20:47:01.148987+0000","last_clean":"2024-04-22T20:47:01.148987+0000","last_became_active":"2024-04-22T19:51:28.506615+0000","last_became_peered":"2024-04-22T19:51:28.506615+0000","last_unstale":"2024-04-22T20:47:01.148987+0000","last_undegraded":"2024-04-22T20:47:01.148987+0000","last_fullsized":"2024-04-22T20:47:01.148987+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:25.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":757,"num_read_kb":623,"num_write":269,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'270","reported_seq":911,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:43.414545+0000","last_change":"2024-04-22T19:51:26.503480+0000","last_active":"2024-04-22T20:44:43.414545+0000","last_peered":"2024-04-22T20:44:43.414545+0000","last_clean":"2024-04-22T20:44:43.414545+0000","last_became_active":"2024-04-22T19:51:26.502938+0000","last_became_peered":"2024-04-22T19:51:26.502938+0000","last_unstale":"2024-04-22T20:44:43.414545+0000","last_undegraded":"2024-04-22T20:44:43.414545+0000","last_fullsized":"2024-04-22T20:44:43.414545+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:25.122842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11884442,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":410,"num_read_kb":28738,"num_write":1201,"num_write_kb":45425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.312594+0000","last_change":"2024-04-22T19:51:30.527273+0000","last_active":"2024-04-22T20:45:10.312594+0000","last_peered":"2024-04-22T20:45:10.312594+0000","last_clean":"2024-04-22T20:45:10.312594+0000","last_became_active":"2024-04-22T19:51:30.526994+0000","last_became_peered":"2024-04-22T19:51:30.526994+0000","last_unstale":"2024-04-22T20:45:10.312594+0000","last_undegraded":"2024-04-22T20:45:10.312594+0000","last_fullsized":"2024-04-22T20:45:10.312594+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:05:35.522811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"261'246","reported_seq":812,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.082501+0000","last_change":"2024-04-22T19:51:32.538193+0000","last_active":"2024-04-22T20:47:35.082501+0000","last_peered":"2024-04-22T20:47:35.082501+0000","last_clean":"2024-04-22T20:47:35.082501+0000","last_became_active":"2024-04-22T19:51:32.537875+0000","last_became_peered":"2024-04-22T19:51:32.537875+0000","last_unstale":"2024-04-22T20:47:35.082501+0000","last_undegraded":"2024-04-22T20:47:35.082501+0000","last_fullsized":"2024-04-22T20:47:35.082501+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:43.382645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50977,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":409,"num_read_kb":848,"num_write":332,"num_write_kb":266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179968+0000","last_change":"2024-04-22T19:51:34.554251+0000","last_active":"2024-04-22T20:45:15.179968+0000","last_peered":"2024-04-22T20:45:15.179968+0000","last_clean":"2024-04-22T20:45:15.179968+0000","last_became_active":"2024-04-22T19:51:34.553954+0000","last_became_peered":"2024-04-22T19:51:34.553954+0000","last_unstale":"2024-04-22T20:45:15.179968+0000","last_undegraded":"2024-04-22T20:45:15.179968+0000","last_fullsized":"2024-04-22T20:45:15.179968+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:25:45.427696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"261'192","reported_seq":830,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317164+0000","last_change":"2024-04-22T19:51:28.514880+0000","last_active":"2024-04-22T20:45:40.317164+0000","last_peered":"2024-04-22T20:45:40.317164+0000","last_clean":"2024-04-22T20:45:40.317164+0000","last_became_active":"2024-04-22T19:51:28.514593+0000","last_became_peered":"2024-04-22T19:51:28.514593+0000","last_unstale":"2024-04-22T20:45:40.317164+0000","last_undegraded":"2024-04-22T20:45:40.317164+0000","last_fullsized":"2024-04-22T20:45:40.317164+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:10:57.618723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":788,"num_read_kb":638,"num_write":274,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"261'282","reported_seq":944,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:16.752867+0000","last_change":"2024-04-22T19:51:26.502507+0000","last_active":"2024-04-22T20:47:16.752867+0000","last_peered":"2024-04-22T20:47:16.752867+0000","last_clean":"2024-04-22T20:47:16.752867+0000","last_became_active":"2024-04-22T19:51:26.501812+0000","last_became_peered":"2024-04-22T19:51:26.501812+0000","last_unstale":"2024-04-22T20:47:16.752867+0000","last_undegraded":"2024-04-22T20:47:16.752867+0000","last_fullsized":"2024-04-22T20:47:16.752867+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:47:54.670392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1914449,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":492,"num_read_kb":55834,"num_write":1102,"num_write_kb":35884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183098+0000","last_change":"2024-04-22T19:51:30.525750+0000","last_active":"2024-04-22T20:45:40.183098+0000","last_peered":"2024-04-22T20:45:40.183098+0000","last_clean":"2024-04-22T20:45:40.183098+0000","last_became_active":"2024-04-22T19:51:30.525450+0000","last_became_peered":"2024-04-22T19:51:30.525450+0000","last_unstale":"2024-04-22T20:45:40.183098+0000","last_undegraded":"2024-04-22T20:45:40.183098+0000","last_fullsized":"2024-04-22T20:45:40.183098+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:32.196222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"261'474","reported_seq":1092,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.506230+0000","last_change":"2024-04-22T19:51:32.541007+0000","last_active":"2024-04-22T20:48:08.506230+0000","last_peered":"2024-04-22T20:48:08.506230+0000","last_clean":"2024-04-22T20:48:08.506230+0000","last_became_active":"2024-04-22T19:51:32.540755+0000","last_became_peered":"2024-04-22T19:51:32.540755+0000","last_unstale":"2024-04-22T20:48:08.506230+0000","last_undegraded":"2024-04-22T20:48:08.506230+0000","last_fullsized":"2024-04-22T20:48:08.506230+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":400,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:18.375444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112197,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":849,"num_read_kb":1808,"num_write":744,"num_write_kb":667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":482,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.175603+0000","last_change":"2024-04-22T19:51:34.553700+0000","last_active":"2024-04-22T20:44:05.175603+0000","last_peered":"2024-04-22T20:44:05.175603+0000","last_clean":"2024-04-22T20:44:05.175603+0000","last_became_active":"2024-04-22T19:51:34.553363+0000","last_became_peered":"2024-04-22T19:51:34.553363+0000","last_unstale":"2024-04-22T20:44:05.175603+0000","last_undegraded":"2024-04-22T20:44:05.175603+0000","last_fullsized":"2024-04-22T20:44:05.175603+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:13:11.618765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"261'159","reported_seq":673,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:02.819260+0000","last_change":"2024-04-22T19:51:36.569083+0000","last_active":"2024-04-22T20:48:02.819260+0000","last_peered":"2024-04-22T20:48:02.819260+0000","last_clean":"2024-04-22T20:48:02.819260+0000","last_became_active":"2024-04-22T19:51:36.568373+0000","last_became_peered":"2024-04-22T19:51:36.568373+0000","last_unstale":"2024-04-22T20:48:02.819260+0000","last_undegraded":"2024-04-22T20:48:02.819260+0000","last_fullsized":"2024-04-22T20:48:02.819260+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:00:29.149286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":213,"num_write":373,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"261'100","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:11.966474+0000","last_change":"2024-04-22T19:51:28.510922+0000","last_active":"2024-04-22T20:47:11.966474+0000","last_peered":"2024-04-22T20:47:11.966474+0000","last_clean":"2024-04-22T20:47:11.966474+0000","last_became_active":"2024-04-22T19:51:28.510629+0000","last_became_peered":"2024-04-22T19:51:28.510629+0000","last_unstale":"2024-04-22T20:47:11.966474+0000","last_undegraded":"2024-04-22T20:47:11.966474+0000","last_fullsized":"2024-04-22T20:47:11.966474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:55:55.876914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":435,"num_read_kb":357,"num_write":161,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"261'250","reported_seq":883,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:14.134249+0000","last_change":"2024-04-22T19:51:26.497497+0000","last_active":"2024-04-22T20:47:14.134249+0000","last_peered":"2024-04-22T20:47:14.134249+0000","last_clean":"2024-04-22T20:47:14.134249+0000","last_became_active":"2024-04-22T19:51:26.497165+0000","last_became_peered":"2024-04-22T19:51:26.497165+0000","last_unstale":"2024-04-22T20:47:14.134249+0000","last_undegraded":"2024-04-22T20:47:14.134249+0000","last_fullsized":"2024-04-22T20:47:14.134249+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:31.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13593714,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":438,"num_read_kb":39207,"num_write":1067,"num_write_kb":42297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317184+0000","last_change":"2024-04-22T19:51:30.526861+0000","last_active":"2024-04-22T20:45:40.317184+0000","last_peered":"2024-04-22T20:45:40.317184+0000","last_clean":"2024-04-22T20:45:40.317184+0000","last_became_active":"2024-04-22T19:51:30.526597+0000","last_became_peered":"2024-04-22T19:51:30.526597+0000","last_unstale":"2024-04-22T20:45:40.317184+0000","last_undegraded":"2024-04-22T20:45:40.317184+0000","last_fullsized":"2024-04-22T20:45:40.317184+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:24.696235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"261'306","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.074224+0000","last_change":"2024-04-22T19:51:32.537791+0000","last_active":"2024-04-22T20:46:35.074224+0000","last_peered":"2024-04-22T20:46:35.074224+0000","last_clean":"2024-04-22T20:46:35.074224+0000","last_became_active":"2024-04-22T19:51:32.537416+0000","last_became_peered":"2024-04-22T19:51:32.537416+0000","last_unstale":"2024-04-22T20:46:35.074224+0000","last_undegraded":"2024-04-22T20:46:35.074224+0000","last_fullsized":"2024-04-22T20:46:35.074224+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:35:00.012988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55718,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":984,"num_write":430,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":484,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.035331+0000","last_change":"2024-04-22T19:51:34.561345+0000","last_active":"2024-04-22T20:44:05.035331+0000","last_peered":"2024-04-22T20:44:05.035331+0000","last_clean":"2024-04-22T20:44:05.035331+0000","last_became_active":"2024-04-22T19:51:34.561096+0000","last_became_peered":"2024-04-22T19:51:34.561096+0000","last_unstale":"2024-04-22T20:44:05.035331+0000","last_undegraded":"2024-04-22T20:44:05.035331+0000","last_fullsized":"2024-04-22T20:44:05.035331+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:38.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"261'161","reported_seq":661,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314006+0000","last_change":"2024-04-22T19:51:36.571812+0000","last_active":"2024-04-22T20:45:20.314006+0000","last_peered":"2024-04-22T20:45:20.314006+0000","last_clean":"2024-04-22T20:45:20.314006+0000","last_became_active":"2024-04-22T19:51:36.571565+0000","last_became_peered":"2024-04-22T19:51:36.571565+0000","last_unstale":"2024-04-22T20:45:20.314006+0000","last_undegraded":"2024-04-22T20:45:20.314006+0000","last_fullsized":"2024-04-22T20:45:20.314006+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:55.978840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":180,"num_write":473,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"261'44","reported_seq":609,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145600+0000","last_change":"2024-04-22T19:51:28.507235+0000","last_active":"2024-04-22T20:44:00.145600+0000","last_peered":"2024-04-22T20:44:00.145600+0000","last_clean":"2024-04-22T20:44:00.145600+0000","last_became_active":"2024-04-22T19:51:28.506835+0000","last_became_peered":"2024-04-22T19:51:28.506835+0000","last_unstale":"2024-04-22T20:44:00.145600+0000","last_undegraded":"2024-04-22T20:44:00.145600+0000","last_fullsized":"2024-04-22T20:44:00.145600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:22:38.074432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":202,"num_read_kb":174,"num_write":58,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"261'309","reported_seq":936,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.304458+0000","last_change":"2024-04-22T19:52:04.618218+0000","last_active":"2024-04-22T20:47:38.304458+0000","last_peered":"2024-04-22T20:47:38.304458+0000","last_clean":"2024-04-22T20:47:38.304458+0000","last_became_active":"2024-04-22T19:52:04.617820+0000","last_became_peered":"2024-04-22T19:52:04.617820+0000","last_unstale":"2024-04-22T20:47:38.304458+0000","last_undegraded":"2024-04-22T20:47:38.304458+0000","last_fullsized":"2024-04-22T20:47:38.304458+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:04:41.066429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588803,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":467,"num_read_kb":31540,"num_write":1405,"num_write_kb":43247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166735+0000","last_change":"2024-04-22T19:51:30.529632+0000","last_active":"2024-04-22T20:44:00.166735+0000","last_peered":"2024-04-22T20:44:00.166735+0000","last_clean":"2024-04-22T20:44:00.166735+0000","last_became_active":"2024-04-22T19:51:30.529322+0000","last_became_peered":"2024-04-22T19:51:30.529322+0000","last_unstale":"2024-04-22T20:44:00.166735+0000","last_undegraded":"2024-04-22T20:44:00.166735+0000","last_fullsized":"2024-04-22T20:44:00.166735+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:43:15.680192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"261'266","reported_seq":848,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.025839+0000","last_change":"2024-04-22T19:51:33.248631+0000","last_active":"2024-04-22T20:48:03.025839+0000","last_peered":"2024-04-22T20:48:03.025839+0000","last_clean":"2024-04-22T20:48:03.025839+0000","last_became_active":"2024-04-22T19:51:33.248293+0000","last_became_peered":"2024-04-22T19:51:33.248293+0000","last_unstale":"2024-04-22T20:48:03.025839+0000","last_undegraded":"2024-04-22T20:48:03.025839+0000","last_fullsized":"2024-04-22T20:48:03.025839+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:49.051719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66888,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":442,"num_read_kb":925,"num_write":358,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.046532+0000","last_change":"2024-04-22T19:51:34.561273+0000","last_active":"2024-04-22T20:45:45.046532+0000","last_peered":"2024-04-22T20:45:45.046532+0000","last_clean":"2024-04-22T20:45:45.046532+0000","last_became_active":"2024-04-22T19:51:34.560955+0000","last_became_peered":"2024-04-22T19:51:34.560955+0000","last_unstale":"2024-04-22T20:45:45.046532+0000","last_undegraded":"2024-04-22T20:45:45.046532+0000","last_fullsized":"2024-04-22T20:45:45.046532+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:03:55.584983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"261'119","reported_seq":602,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.536618+0000","last_change":"2024-04-22T19:51:36.567411+0000","last_active":"2024-04-22T20:48:08.536618+0000","last_peered":"2024-04-22T20:48:08.536618+0000","last_clean":"2024-04-22T20:48:08.536618+0000","last_became_active":"2024-04-22T19:51:36.567139+0000","last_became_peered":"2024-04-22T19:51:36.567139+0000","last_unstale":"2024-04-22T20:48:08.536618+0000","last_undegraded":"2024-04-22T20:48:08.536618+0000","last_fullsized":"2024-04-22T20:48:08.536618+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:39:35.365237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":92,"num_write":268,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"261'106","reported_seq":688,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.058728+0000","last_change":"2024-04-22T19:51:28.512939+0000","last_active":"2024-04-22T20:44:00.058728+0000","last_peered":"2024-04-22T20:44:00.058728+0000","last_clean":"2024-04-22T20:44:00.058728+0000","last_became_active":"2024-04-22T19:51:28.512642+0000","last_became_peered":"2024-04-22T19:51:28.512642+0000","last_unstale":"2024-04-22T20:44:00.058728+0000","last_undegraded":"2024-04-22T20:44:00.058728+0000","last_fullsized":"2024-04-22T20:44:00.058728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:36.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":349,"num_write":169,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"261'262","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.171114+0000","last_change":"2024-04-22T19:51:26.512313+0000","last_active":"2024-04-22T20:44:00.171114+0000","last_peered":"2024-04-22T20:44:00.171114+0000","last_clean":"2024-04-22T20:44:00.171114+0000","last_became_active":"2024-04-22T19:51:26.511966+0000","last_became_peered":"2024-04-22T19:51:26.511966+0000","last_unstale":"2024-04-22T20:44:00.171114+0000","last_undegraded":"2024-04-22T20:44:00.171114+0000","last_fullsized":"2024-04-22T20:44:00.171114+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:52.017119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7663079,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":370,"num_read_kb":38272,"num_write":1038,"num_write_kb":51509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.034785+0000","last_change":"2024-04-22T19:51:30.527261+0000","last_active":"2024-04-22T20:44:00.034785+0000","last_peered":"2024-04-22T20:44:00.034785+0000","last_clean":"2024-04-22T20:44:00.034785+0000","last_became_active":"2024-04-22T19:51:30.526949+0000","last_became_peered":"2024-04-22T19:51:30.526949+0000","last_unstale":"2024-04-22T20:44:00.034785+0000","last_undegraded":"2024-04-22T20:44:00.034785+0000","last_fullsized":"2024-04-22T20:44:00.034785+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:13:41.016976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"261'103","reported_seq":629,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182111+0000","last_change":"2024-04-22T19:51:33.248558+0000","last_active":"2024-04-22T20:45:45.182111+0000","last_peered":"2024-04-22T20:45:45.182111+0000","last_clean":"2024-04-22T20:45:45.182111+0000","last_became_active":"2024-04-22T19:51:33.248113+0000","last_became_peered":"2024-04-22T19:51:33.248113+0000","last_unstale":"2024-04-22T20:45:45.182111+0000","last_undegraded":"2024-04-22T20:45:45.182111+0000","last_fullsized":"2024-04-22T20:45:45.182111+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:09.138747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":119,"num_read_kb":113,"num_write":116,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"261'515","reported_seq":2294,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.550861+0000","last_change":"2024-04-22T19:51:34.557105+0000","last_active":"2024-04-22T20:48:08.550861+0000","last_peered":"2024-04-22T20:48:08.550861+0000","last_clean":"2024-04-22T20:48:08.550861+0000","last_became_active":"2024-04-22T19:51:34.556789+0000","last_became_peered":"2024-04-22T19:51:34.556789+0000","last_unstale":"2024-04-22T20:48:08.550861+0000","last_undegraded":"2024-04-22T20:48:08.550861+0000","last_fullsized":"2024-04-22T20:48:08.550861+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:07.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"261'96","reported_seq":594,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062299+0000","last_change":"2024-04-22T19:51:36.571445+0000","last_active":"2024-04-22T20:45:20.062299+0000","last_peered":"2024-04-22T20:45:20.062299+0000","last_clean":"2024-04-22T20:45:20.062299+0000","last_became_active":"2024-04-22T19:51:36.571175+0000","last_became_peered":"2024-04-22T19:51:36.571175+0000","last_unstale":"2024-04-22T20:45:20.062299+0000","last_undegraded":"2024-04-22T20:45:20.062299+0000","last_fullsized":"2024-04-22T20:45:20.062299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:18.054128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":221,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"261'131","reported_seq":708,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:14.252216+0000","last_change":"2024-04-22T19:51:28.509477+0000","last_active":"2024-04-22T20:45:14.252216+0000","last_peered":"2024-04-22T20:45:14.252216+0000","last_clean":"2024-04-22T20:45:14.252216+0000","last_became_active":"2024-04-22T19:51:28.509110+0000","last_became_peered":"2024-04-22T19:51:28.509110+0000","last_unstale":"2024-04-22T20:45:14.252216+0000","last_undegraded":"2024-04-22T20:45:14.252216+0000","last_fullsized":"2024-04-22T20:45:14.252216+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:10:52.641173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":519,"num_read_kb":420,"num_write":188,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"261'333","reported_seq":959,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:08.194615+0000","last_change":"2024-04-22T19:52:04.619673+0000","last_active":"2024-04-22T20:45:08.194615+0000","last_peered":"2024-04-22T20:45:08.194615+0000","last_clean":"2024-04-22T20:45:08.194615+0000","last_became_active":"2024-04-22T19:52:04.619385+0000","last_became_peered":"2024-04-22T19:52:04.619385+0000","last_unstale":"2024-04-22T20:45:08.194615+0000","last_undegraded":"2024-04-22T20:45:08.194615+0000","last_fullsized":"2024-04-22T20:45:08.194615+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:11.756964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8224737,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":472,"num_read_kb":20998,"num_write":1385,"num_write_kb":33571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.049743+0000","last_change":"2024-04-22T19:51:30.527064+0000","last_active":"2024-04-22T20:44:00.049743+0000","last_peered":"2024-04-22T20:44:00.049743+0000","last_clean":"2024-04-22T20:44:00.049743+0000","last_became_active":"2024-04-22T19:51:30.526469+0000","last_became_peered":"2024-04-22T19:51:30.526469+0000","last_unstale":"2024-04-22T20:44:00.049743+0000","last_undegraded":"2024-04-22T20:44:00.049743+0000","last_fullsized":"2024-04-22T20:44:00.049743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:45:04.946224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"261'301","reported_seq":880,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.204151+0000","last_change":"2024-04-22T19:51:32.540830+0000","last_active":"2024-04-22T20:47:20.204151+0000","last_peered":"2024-04-22T20:47:20.204151+0000","last_clean":"2024-04-22T20:47:20.204151+0000","last_became_active":"2024-04-22T19:51:32.540573+0000","last_became_peered":"2024-04-22T19:51:32.540573+0000","last_unstale":"2024-04-22T20:47:20.204151+0000","last_undegraded":"2024-04-22T20:47:20.204151+0000","last_fullsized":"2024-04-22T20:47:20.204151+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:49.905368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61803,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":483,"num_read_kb":946,"num_write":388,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.176362+0000","last_change":"2024-04-22T19:51:34.554124+0000","last_active":"2024-04-22T20:44:45.176362+0000","last_peered":"2024-04-22T20:44:45.176362+0000","last_clean":"2024-04-22T20:44:45.176362+0000","last_became_active":"2024-04-22T19:51:34.553857+0000","last_became_peered":"2024-04-22T19:51:34.553857+0000","last_unstale":"2024-04-22T20:44:45.176362+0000","last_undegraded":"2024-04-22T20:44:45.176362+0000","last_fullsized":"2024-04-22T20:44:45.176362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:38:56.150478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"261'153","reported_seq":654,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:15.079446+0000","last_change":"2024-04-22T19:51:36.575509+0000","last_active":"2024-04-22T20:47:15.079446+0000","last_peered":"2024-04-22T20:47:15.079446+0000","last_clean":"2024-04-22T20:47:15.079446+0000","last_became_active":"2024-04-22T19:51:36.575211+0000","last_became_peered":"2024-04-22T19:51:36.575211+0000","last_unstale":"2024-04-22T20:47:15.079446+0000","last_undegraded":"2024-04-22T20:47:15.079446+0000","last_fullsized":"2024-04-22T20:47:15.079446+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:46:54.630277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":158,"num_write":375,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"261'85","reported_seq":680,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.024455+0000","last_change":"2024-04-22T19:51:28.508354+0000","last_active":"2024-04-22T20:48:04.024455+0000","last_peered":"2024-04-22T20:48:04.024455+0000","last_clean":"2024-04-22T20:48:04.024455+0000","last_became_active":"2024-04-22T19:51:28.508145+0000","last_became_peered":"2024-04-22T19:51:28.508145+0000","last_unstale":"2024-04-22T20:48:04.024455+0000","last_undegraded":"2024-04-22T20:48:04.024455+0000","last_fullsized":"2024-04-22T20:48:04.024455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:54:32.701108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":376,"num_read_kb":320,"num_write":115,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"261'220","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174972+0000","last_change":"2024-04-22T19:51:26.504671+0000","last_active":"2024-04-22T20:44:00.174972+0000","last_peered":"2024-04-22T20:44:00.174972+0000","last_clean":"2024-04-22T20:44:00.174972+0000","last_became_active":"2024-04-22T19:51:26.504301+0000","last_became_peered":"2024-04-22T19:51:26.504301+0000","last_unstale":"2024-04-22T20:44:00.174972+0000","last_undegraded":"2024-04-22T20:44:00.174972+0000","last_fullsized":"2024-04-22T20:44:00.174972+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:56:36.267794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4884274,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":384,"num_read_kb":4553,"num_write":1039,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174938+0000","last_change":"2024-04-22T19:51:30.524895+0000","last_active":"2024-04-22T20:44:00.174938+0000","last_peered":"2024-04-22T20:44:00.174938+0000","last_clean":"2024-04-22T20:44:00.174938+0000","last_became_active":"2024-04-22T19:51:30.524417+0000","last_became_peered":"2024-04-22T19:51:30.524417+0000","last_unstale":"2024-04-22T20:44:00.174938+0000","last_undegraded":"2024-04-22T20:44:00.174938+0000","last_fullsized":"2024-04-22T20:44:00.174938+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:39:15.056196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"261'225","reported_seq":790,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.193293+0000","last_change":"2024-04-22T19:51:32.540923+0000","last_active":"2024-04-22T20:47:10.193293+0000","last_peered":"2024-04-22T20:47:10.193293+0000","last_clean":"2024-04-22T20:47:10.193293+0000","last_became_active":"2024-04-22T19:51:32.540659+0000","last_became_peered":"2024-04-22T19:51:32.540659+0000","last_unstale":"2024-04-22T20:47:10.193293+0000","last_undegraded":"2024-04-22T20:47:10.193293+0000","last_fullsized":"2024-04-22T20:47:10.193293+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:42:58.310584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51883,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":376,"num_read_kb":774,"num_write":298,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309863+0000","last_change":"2024-04-22T19:51:34.552587+0000","last_active":"2024-04-22T20:44:50.309863+0000","last_peered":"2024-04-22T20:44:50.309863+0000","last_clean":"2024-04-22T20:44:50.309863+0000","last_became_active":"2024-04-22T19:51:34.552344+0000","last_became_peered":"2024-04-22T19:51:34.552344+0000","last_unstale":"2024-04-22T20:44:50.309863+0000","last_undegraded":"2024-04-22T20:44:50.309863+0000","last_fullsized":"2024-04-22T20:44:50.309863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:41.886857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"261'137","reported_seq":675,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:01.745300+0000","last_change":"2024-04-22T19:51:36.567945+0000","last_active":"2024-04-22T20:48:01.745300+0000","last_peered":"2024-04-22T20:48:01.745300+0000","last_clean":"2024-04-22T20:48:01.745300+0000","last_became_active":"2024-04-22T19:51:36.567594+0000","last_became_peered":"2024-04-22T19:51:36.567594+0000","last_unstale":"2024-04-22T20:48:01.745300+0000","last_undegraded":"2024-04-22T20:48:01.745300+0000","last_fullsized":"2024-04-22T20:48:01.745300+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:25:38.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":214,"num_write":335,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"261'87","reported_seq":648,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.151277+0000","last_change":"2024-04-22T19:51:28.519780+0000","last_active":"2024-04-22T20:44:45.151277+0000","last_peered":"2024-04-22T20:44:45.151277+0000","last_clean":"2024-04-22T20:44:45.151277+0000","last_became_active":"2024-04-22T19:51:28.519542+0000","last_became_peered":"2024-04-22T19:51:28.519542+0000","last_unstale":"2024-04-22T20:44:45.151277+0000","last_undegraded":"2024-04-22T20:44:45.151277+0000","last_fullsized":"2024-04-22T20:44:45.151277+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:24:55.634254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":347,"num_read_kb":277,"num_write":137,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"261'332","reported_seq":1176,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157148+0000","last_change":"2024-04-22T19:51:26.500077+0000","last_active":"2024-04-22T20:45:35.157148+0000","last_peered":"2024-04-22T20:45:35.157148+0000","last_clean":"2024-04-22T20:45:35.157148+0000","last_became_active":"2024-04-22T19:51:26.499390+0000","last_became_peered":"2024-04-22T19:51:26.499390+0000","last_unstale":"2024-04-22T20:45:35.157148+0000","last_undegraded":"2024-04-22T20:45:35.157148+0000","last_fullsized":"2024-04-22T20:45:35.157148+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:40:50.316762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24547368,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":977,"num_read_kb":88369,"num_write":1347,"num_write_kb":41583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172368+0000","last_change":"2024-04-22T19:51:30.526678+0000","last_active":"2024-04-22T20:44:40.172368+0000","last_peered":"2024-04-22T20:44:40.172368+0000","last_clean":"2024-04-22T20:44:40.172368+0000","last_became_active":"2024-04-22T19:51:30.526183+0000","last_became_peered":"2024-04-22T19:51:30.526183+0000","last_unstale":"2024-04-22T20:44:40.172368+0000","last_undegraded":"2024-04-22T20:44:40.172368+0000","last_fullsized":"2024-04-22T20:44:40.172368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:40.232417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"261'243","reported_seq":802,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.164215+0000","last_change":"2024-04-22T19:51:32.538965+0000","last_active":"2024-04-22T20:46:35.164215+0000","last_peered":"2024-04-22T20:46:35.164215+0000","last_clean":"2024-04-22T20:46:35.164215+0000","last_became_active":"2024-04-22T19:51:32.538644+0000","last_became_peered":"2024-04-22T19:51:32.538644+0000","last_unstale":"2024-04-22T20:46:35.164215+0000","last_undegraded":"2024-04-22T20:46:35.164215+0000","last_fullsized":"2024-04-22T20:46:35.164215+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:18:02.949702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52954,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":389,"num_read_kb":836,"num_write":326,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":486,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.167484+0000","last_change":"2024-04-22T19:51:34.551365+0000","last_active":"2024-04-22T20:44:05.167484+0000","last_peered":"2024-04-22T20:44:05.167484+0000","last_clean":"2024-04-22T20:44:05.167484+0000","last_became_active":"2024-04-22T19:51:34.551085+0000","last_became_peered":"2024-04-22T19:51:34.551085+0000","last_unstale":"2024-04-22T20:44:05.167484+0000","last_undegraded":"2024-04-22T20:44:05.167484+0000","last_fullsized":"2024-04-22T20:44:05.167484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:50:23.004035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"261'179","reported_seq":903,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.372427+0000","last_change":"2024-04-22T19:51:36.569537+0000","last_active":"2024-04-22T20:48:00.372427+0000","last_peered":"2024-04-22T20:48:00.372427+0000","last_clean":"2024-04-22T20:48:00.372427+0000","last_became_active":"2024-04-22T19:51:36.569234+0000","last_became_peered":"2024-04-22T19:51:36.569234+0000","last_unstale":"2024-04-22T20:48:00.372427+0000","last_undegraded":"2024-04-22T20:48:00.372427+0000","last_fullsized":"2024-04-22T20:48:00.372427+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:56:31.796866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":419,"num_read_kb":454,"num_write":429,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"261'180","reported_seq":750,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172395+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:44:40.172395+0000","last_peered":"2024-04-22T20:44:40.172395+0000","last_clean":"2024-04-22T20:44:40.172395+0000","last_became_active":"2024-04-22T19:51:28.515506+0000","last_became_peered":"2024-04-22T19:51:28.515506+0000","last_unstale":"2024-04-22T20:44:40.172395+0000","last_undegraded":"2024-04-22T20:44:40.172395+0000","last_fullsized":"2024-04-22T20:44:40.172395+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:14:29.010830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":678,"num_read_kb":529,"num_write":279,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"261'227","reported_seq":839,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.881098+0000","last_change":"2024-04-22T19:51:26.504948+0000","last_active":"2024-04-22T20:46:41.881098+0000","last_peered":"2024-04-22T20:46:41.881098+0000","last_clean":"2024-04-22T20:46:41.881098+0000","last_became_active":"2024-04-22T19:51:26.504604+0000","last_became_peered":"2024-04-22T19:51:26.504604+0000","last_unstale":"2024-04-22T20:46:41.881098+0000","last_undegraded":"2024-04-22T20:46:41.881098+0000","last_fullsized":"2024-04-22T20:46:41.881098+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:11.092305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11653248,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":302,"num_read_kb":9007,"num_write":882,"num_write_kb":46630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"25'1","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040244+0000","last_change":"2024-04-22T19:51:30.523630+0000","last_active":"2024-04-22T20:44:50.040244+0000","last_peered":"2024-04-22T20:44:50.040244+0000","last_clean":"2024-04-22T20:44:50.040244+0000","last_became_active":"2024-04-22T19:51:30.523338+0000","last_became_peered":"2024-04-22T19:51:30.523338+0000","last_unstale":"2024-04-22T20:44:50.040244+0000","last_undegraded":"2024-04-22T20:44:50.040244+0000","last_fullsized":"2024-04-22T20:44:50.040244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:00.508499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"261'305","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.188922+0000","last_change":"2024-04-22T19:51:32.537701+0000","last_active":"2024-04-22T20:46:35.188922+0000","last_peered":"2024-04-22T20:46:35.188922+0000","last_clean":"2024-04-22T20:46:35.188922+0000","last_became_active":"2024-04-22T19:51:32.537373+0000","last_became_peered":"2024-04-22T19:51:32.537373+0000","last_unstale":"2024-04-22T20:46:35.188922+0000","last_undegraded":"2024-04-22T20:46:35.188922+0000","last_fullsized":"2024-04-22T20:46:35.188922+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:43:27.042829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59361,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":482,"num_read_kb":951,"num_write":388,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":486,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.171634+0000","last_change":"2024-04-22T19:51:34.555960+0000","last_active":"2024-04-22T20:44:05.171634+0000","last_peered":"2024-04-22T20:44:05.171634+0000","last_clean":"2024-04-22T20:44:05.171634+0000","last_became_active":"2024-04-22T19:51:34.555709+0000","last_became_peered":"2024-04-22T19:51:34.555709+0000","last_unstale":"2024-04-22T20:44:05.171634+0000","last_undegraded":"2024-04-22T20:44:05.171634+0000","last_fullsized":"2024-04-22T20:44:05.171634+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:02.499776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"261'144","reported_seq":628,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:32.358361+0000","last_change":"2024-04-22T19:51:36.574316+0000","last_active":"2024-04-22T20:46:32.358361+0000","last_peered":"2024-04-22T20:46:32.358361+0000","last_clean":"2024-04-22T20:46:32.358361+0000","last_became_active":"2024-04-22T19:51:36.574012+0000","last_became_peered":"2024-04-22T19:51:36.574012+0000","last_unstale":"2024-04-22T20:46:32.358361+0000","last_undegraded":"2024-04-22T20:46:32.358361+0000","last_fullsized":"2024-04-22T20:46:32.358361+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:01:01.700368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":109,"num_write":348,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"261'372","reported_seq":1037,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045977+0000","last_change":"2024-04-22T19:51:28.518589+0000","last_active":"2024-04-22T20:45:40.045977+0000","last_peered":"2024-04-22T20:45:40.045977+0000","last_clean":"2024-04-22T20:45:40.045977+0000","last_became_active":"2024-04-22T19:51:28.518339+0000","last_became_peered":"2024-04-22T19:51:28.518339+0000","last_unstale":"2024-04-22T20:45:40.045977+0000","last_undegraded":"2024-04-22T20:45:40.045977+0000","last_fullsized":"2024-04-22T20:45:40.045977+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:15:44.877848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1526,"num_read_kb":1211,"num_write":619,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"261'252","reported_seq":951,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.316997+0000","last_change":"2024-04-22T19:51:26.503151+0000","last_active":"2024-04-22T20:45:40.316997+0000","last_peered":"2024-04-22T20:45:40.316997+0000","last_clean":"2024-04-22T20:45:40.316997+0000","last_became_active":"2024-04-22T19:51:26.502773+0000","last_became_peered":"2024-04-22T19:51:26.502773+0000","last_unstale":"2024-04-22T20:45:40.316997+0000","last_undegraded":"2024-04-22T20:45:40.316997+0000","last_fullsized":"2024-04-22T20:45:40.316997+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:34:49.559184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6843432,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":522,"num_read_kb":14380,"num_write":1027,"num_write_kb":17858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"261'524","reported_seq":1286,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.175644+0000","last_change":"2024-04-22T19:52:04.617284+0000","last_active":"2024-04-22T20:44:05.175644+0000","last_peered":"2024-04-22T20:44:05.175644+0000","last_clean":"2024-04-22T20:44:05.175644+0000","last_became_active":"2024-04-22T19:52:04.617036+0000","last_became_peered":"2024-04-22T19:52:04.617036+0000","last_unstale":"2024-04-22T20:44:05.175644+0000","last_undegraded":"2024-04-22T20:44:05.175644+0000","last_fullsized":"2024-04-22T20:44:05.175644+0000","mapping_epoch":33,"log_start":"261'500","ondisk_log_start":"261'500","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":500,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:59:14.488482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2230,"num_read_kb":1847,"num_write":935,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"261'226","reported_seq":865,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:43.145525+0000","last_change":"2024-04-22T19:51:26.496262+0000","last_active":"2024-04-22T20:46:43.145525+0000","last_peered":"2024-04-22T20:46:43.145525+0000","last_clean":"2024-04-22T20:46:43.145525+0000","last_became_active":"2024-04-22T19:51:26.495948+0000","last_became_peered":"2024-04-22T19:51:26.495948+0000","last_unstale":"2024-04-22T20:46:43.145525+0000","last_undegraded":"2024-04-22T20:46:43.145525+0000","last_fullsized":"2024-04-22T20:46:43.145525+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":200,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:25.588593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1295176,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":394,"num_read_kb":10115,"num_write":945,"num_write_kb":22835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"261'208","reported_seq":841,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189450+0000","last_change":"2024-04-22T19:51:28.513290+0000","last_active":"2024-04-22T20:45:40.189450+0000","last_peered":"2024-04-22T20:45:40.189450+0000","last_clean":"2024-04-22T20:45:40.189450+0000","last_became_active":"2024-04-22T19:51:28.513026+0000","last_became_peered":"2024-04-22T19:51:28.513026+0000","last_unstale":"2024-04-22T20:45:40.189450+0000","last_undegraded":"2024-04-22T20:45:40.189450+0000","last_fullsized":"2024-04-22T20:45:40.189450+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:09:31.873692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":827,"num_read_kb":665,"num_write":342,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"261'251","reported_seq":870,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:43.136633+0000","last_change":"2024-04-22T19:51:26.507266+0000","last_active":"2024-04-22T20:46:43.136633+0000","last_peered":"2024-04-22T20:46:43.136633+0000","last_clean":"2024-04-22T20:46:43.136633+0000","last_became_active":"2024-04-22T19:51:26.506748+0000","last_became_peered":"2024-04-22T19:51:26.506748+0000","last_unstale":"2024-04-22T20:46:43.136633+0000","last_undegraded":"2024-04-22T20:46:43.136633+0000","last_fullsized":"2024-04-22T20:46:43.136633+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:15.321922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6641638,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":354,"num_read_kb":12674,"num_write":1040,"num_write_kb":41347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"261'4175","reported_seq":4791,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:29.357349+0000","last_change":"2024-04-22T19:51:28.511560+0000","last_active":"2024-04-22T20:46:29.357349+0000","last_peered":"2024-04-22T20:46:29.357349+0000","last_clean":"2024-04-22T20:46:29.357349+0000","last_became_active":"2024-04-22T19:51:28.511312+0000","last_became_peered":"2024-04-22T19:51:28.511312+0000","last_unstale":"2024-04-22T20:46:29.357349+0000","last_undegraded":"2024-04-22T20:46:29.357349+0000","last_fullsized":"2024-04-22T20:46:29.357349+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":3000,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:28:56.675918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14718,"num_read_kb":11919,"num_write":6278,"num_write_kb":3133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"261'248","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.060540+0000","last_change":"2024-04-22T19:52:05.115180+0000","last_active":"2024-04-22T20:44:20.060540+0000","last_peered":"2024-04-22T20:44:20.060540+0000","last_clean":"2024-04-22T20:44:20.060540+0000","last_became_active":"2024-04-22T19:52:05.114734+0000","last_became_peered":"2024-04-22T19:52:05.114734+0000","last_unstale":"2024-04-22T20:44:20.060540+0000","last_undegraded":"2024-04-22T20:44:20.060540+0000","last_fullsized":"2024-04-22T20:44:20.060540+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:08:25.577231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11535134,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":357,"num_read_kb":5518,"num_write":1024,"num_write_kb":21667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.23","version":"261'170","reported_seq":851,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.006667+0000","last_change":"2024-04-22T19:51:28.516445+0000","last_active":"2024-04-22T20:48:03.006667+0000","last_peered":"2024-04-22T20:48:03.006667+0000","last_clean":"2024-04-22T20:48:03.006667+0000","last_became_active":"2024-04-22T19:51:28.516125+0000","last_became_peered":"2024-04-22T19:51:28.516125+0000","last_unstale":"2024-04-22T20:48:03.006667+0000","last_undegraded":"2024-04-22T20:48:03.006667+0000","last_fullsized":"2024-04-22T20:48:03.006667+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:01:29.663998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":741,"num_read_kb":616,"num_write":243,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"261'299","reported_seq":945,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.135884+0000","last_change":"2024-04-22T19:52:04.619831+0000","last_active":"2024-04-22T20:47:38.135884+0000","last_peered":"2024-04-22T20:47:38.135884+0000","last_clean":"2024-04-22T20:47:38.135884+0000","last_became_active":"2024-04-22T19:52:04.619286+0000","last_became_peered":"2024-04-22T19:52:04.619286+0000","last_unstale":"2024-04-22T20:47:38.135884+0000","last_undegraded":"2024-04-22T20:47:38.135884+0000","last_fullsized":"2024-04-22T20:47:38.135884+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:58.235831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538009,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":519,"num_read_kb":69891,"num_write":1172,"num_write_kb":36151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"261'122","reported_seq":726,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181359+0000","last_change":"2024-04-22T19:51:28.516690+0000","last_active":"2024-04-22T20:45:40.181359+0000","last_peered":"2024-04-22T20:45:40.181359+0000","last_clean":"2024-04-22T20:45:40.181359+0000","last_became_active":"2024-04-22T19:51:28.516437+0000","last_became_peered":"2024-04-22T19:51:28.516437+0000","last_unstale":"2024-04-22T20:45:40.181359+0000","last_undegraded":"2024-04-22T20:45:40.181359+0000","last_fullsized":"2024-04-22T20:45:40.181359+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:44.084365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":423,"num_write":178,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"261'257","reported_seq":991,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045950+0000","last_change":"2024-04-22T19:51:26.498349+0000","last_active":"2024-04-22T20:45:40.045950+0000","last_peered":"2024-04-22T20:45:40.045950+0000","last_clean":"2024-04-22T20:45:40.045950+0000","last_became_active":"2024-04-22T19:51:26.498039+0000","last_became_peered":"2024-04-22T19:51:26.498039+0000","last_unstale":"2024-04-22T20:45:40.045950+0000","last_undegraded":"2024-04-22T20:45:40.045950+0000","last_fullsized":"2024-04-22T20:45:40.045950+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:17.714586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14707922,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":706,"num_read_kb":22382,"num_write":1193,"num_write_kb":18139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"261'199","reported_seq":848,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:49.466099+0000","last_change":"2024-04-22T19:51:28.520853+0000","last_active":"2024-04-22T20:46:49.466099+0000","last_peered":"2024-04-22T20:46:49.466099+0000","last_clean":"2024-04-22T20:46:49.466099+0000","last_became_active":"2024-04-22T19:51:28.520612+0000","last_became_peered":"2024-04-22T19:51:28.520612+0000","last_unstale":"2024-04-22T20:46:49.466099+0000","last_undegraded":"2024-04-22T20:46:49.466099+0000","last_fullsized":"2024-04-22T20:46:49.466099+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:46:57.197381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":735,"num_read_kb":593,"num_write":262,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"261'291","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317145+0000","last_change":"2024-04-22T19:51:26.499154+0000","last_active":"2024-04-22T20:45:40.317145+0000","last_peered":"2024-04-22T20:45:40.317145+0000","last_clean":"2024-04-22T20:45:40.317145+0000","last_became_active":"2024-04-22T19:51:26.498845+0000","last_became_peered":"2024-04-22T19:51:26.498845+0000","last_unstale":"2024-04-22T20:45:40.317145+0000","last_undegraded":"2024-04-22T20:45:40.317145+0000","last_fullsized":"2024-04-22T20:45:40.317145+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:50.165203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7476623,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":432,"num_read_kb":33253,"num_write":1174,"num_write_kb":30860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"261'133","reported_seq":766,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317028+0000","last_change":"2024-04-22T19:51:28.520128+0000","last_active":"2024-04-22T20:45:40.317028+0000","last_peered":"2024-04-22T20:45:40.317028+0000","last_clean":"2024-04-22T20:45:40.317028+0000","last_became_active":"2024-04-22T19:51:28.519669+0000","last_became_peered":"2024-04-22T19:51:28.519669+0000","last_unstale":"2024-04-22T20:45:40.317028+0000","last_undegraded":"2024-04-22T20:45:40.317028+0000","last_fullsized":"2024-04-22T20:45:40.317028+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:18.121387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":560,"num_read_kb":466,"num_write":196,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"261'257","reported_seq":852,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:07.068523+0000","last_change":"2024-04-22T19:52:04.618077+0000","last_active":"2024-04-22T20:44:07.068523+0000","last_peered":"2024-04-22T20:44:07.068523+0000","last_clean":"2024-04-22T20:44:07.068523+0000","last_became_active":"2024-04-22T19:52:04.617649+0000","last_became_peered":"2024-04-22T19:52:04.617649+0000","last_unstale":"2024-04-22T20:44:07.068523+0000","last_undegraded":"2024-04-22T20:44:07.068523+0000","last_fullsized":"2024-04-22T20:44:07.068523+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:58.404399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587123,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":386,"num_read_kb":29068,"num_write":962,"num_write_kb":22627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"261'128","reported_seq":709,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.179387+0000","last_change":"2024-04-22T19:51:28.521764+0000","last_active":"2024-04-22T20:45:10.179387+0000","last_peered":"2024-04-22T20:45:10.179387+0000","last_clean":"2024-04-22T20:45:10.179387+0000","last_became_active":"2024-04-22T19:51:28.521516+0000","last_became_peered":"2024-04-22T19:51:28.521516+0000","last_unstale":"2024-04-22T20:45:10.179387+0000","last_undegraded":"2024-04-22T20:45:10.179387+0000","last_fullsized":"2024-04-22T20:45:10.179387+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:13:11.122213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":542,"num_read_kb":442,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"261'278","reported_seq":958,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:52.078309+0000","last_change":"2024-04-22T19:51:26.510292+0000","last_active":"2024-04-22T20:46:52.078309+0000","last_peered":"2024-04-22T20:46:52.078309+0000","last_clean":"2024-04-22T20:46:52.078309+0000","last_became_active":"2024-04-22T19:51:26.507843+0000","last_became_peered":"2024-04-22T19:51:26.507843+0000","last_unstale":"2024-04-22T20:46:52.078309+0000","last_undegraded":"2024-04-22T20:46:52.078309+0000","last_fullsized":"2024-04-22T20:46:52.078309+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:43:15.698499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10798875,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":447,"num_read_kb":136369,"num_write":1173,"num_write_kb":55788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"261'235","reported_seq":860,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176799+0000","last_change":"2024-04-22T19:51:28.515473+0000","last_active":"2024-04-22T20:45:10.176799+0000","last_peered":"2024-04-22T20:45:10.176799+0000","last_clean":"2024-04-22T20:45:10.176799+0000","last_became_active":"2024-04-22T19:51:28.515164+0000","last_became_peered":"2024-04-22T19:51:28.515164+0000","last_unstale":"2024-04-22T20:45:10.176799+0000","last_undegraded":"2024-04-22T20:45:10.176799+0000","last_fullsized":"2024-04-22T20:45:10.176799+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:38:12.199672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":950,"num_read_kb":750,"num_write":364,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"261'436","reported_seq":1136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.156482+0000","last_change":"2024-04-22T19:51:26.494217+0000","last_active":"2024-04-22T20:45:30.156482+0000","last_peered":"2024-04-22T20:45:30.156482+0000","last_clean":"2024-04-22T20:45:30.156482+0000","last_became_active":"2024-04-22T19:51:26.493860+0000","last_became_peered":"2024-04-22T19:51:26.493860+0000","last_unstale":"2024-04-22T20:45:30.156482+0000","last_undegraded":"2024-04-22T20:45:30.156482+0000","last_fullsized":"2024-04-22T20:45:30.156482+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:56:54.837990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6300944,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":885,"num_read_kb":13872,"num_write":1373,"num_write_kb":37693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"261'251","reported_seq":929,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:37.822141+0000","last_change":"2024-04-22T19:51:28.515833+0000","last_active":"2024-04-22T20:46:37.822141+0000","last_peered":"2024-04-22T20:46:37.822141+0000","last_clean":"2024-04-22T20:46:37.822141+0000","last_became_active":"2024-04-22T19:51:28.515547+0000","last_became_peered":"2024-04-22T19:51:28.515547+0000","last_unstale":"2024-04-22T20:46:37.822141+0000","last_undegraded":"2024-04-22T20:46:37.822141+0000","last_fullsized":"2024-04-22T20:46:37.822141+0000","mapping_epoch":22,"log_start":"247'200","ondisk_log_start":"247'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:00:39.299916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1120,"num_read_kb":973,"num_write":365,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"261'258","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.187879+0000","last_change":"2024-04-22T19:51:26.500937+0000","last_active":"2024-04-22T20:45:30.187879+0000","last_peered":"2024-04-22T20:45:30.187879+0000","last_clean":"2024-04-22T20:45:30.187879+0000","last_became_active":"2024-04-22T19:51:26.500566+0000","last_became_peered":"2024-04-22T19:51:26.500566+0000","last_unstale":"2024-04-22T20:45:30.187879+0000","last_undegraded":"2024-04-22T20:45:30.187879+0000","last_fullsized":"2024-04-22T20:45:30.187879+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:25.795560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4539865,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":358,"num_read_kb":45844,"num_write":1054,"num_write_kb":42819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"261'152","reported_seq":750,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:42.563356+0000","last_change":"2024-04-22T19:51:28.516044+0000","last_active":"2024-04-22T20:46:42.563356+0000","last_peered":"2024-04-22T20:46:42.563356+0000","last_clean":"2024-04-22T20:46:42.563356+0000","last_became_active":"2024-04-22T19:51:28.515764+0000","last_became_peered":"2024-04-22T19:51:28.515764+0000","last_unstale":"2024-04-22T20:46:42.563356+0000","last_undegraded":"2024-04-22T20:46:42.563356+0000","last_fullsized":"2024-04-22T20:46:42.563356+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:31.975652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":633,"num_read_kb":515,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"261'212","reported_seq":821,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181737+0000","last_change":"2024-04-22T19:51:26.512810+0000","last_active":"2024-04-22T20:45:30.181737+0000","last_peered":"2024-04-22T20:45:30.181737+0000","last_clean":"2024-04-22T20:45:30.181737+0000","last_became_active":"2024-04-22T19:51:26.512467+0000","last_became_peered":"2024-04-22T19:51:26.512467+0000","last_unstale":"2024-04-22T20:45:30.181737+0000","last_undegraded":"2024-04-22T20:45:30.181737+0000","last_fullsized":"2024-04-22T20:45:30.181737+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:04.495670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12585759,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":328,"num_read_kb":17902,"num_write":889,"num_write_kb":43703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"261'27","reported_seq":554,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.063941+0000","last_change":"2024-04-22T19:51:28.510362+0000","last_active":"2024-04-22T20:45:30.063941+0000","last_peered":"2024-04-22T20:45:30.063941+0000","last_clean":"2024-04-22T20:45:30.063941+0000","last_became_active":"2024-04-22T19:51:28.510091+0000","last_became_peered":"2024-04-22T19:51:28.510091+0000","last_unstale":"2024-04-22T20:45:30.063941+0000","last_undegraded":"2024-04-22T20:45:30.063941+0000","last_fullsized":"2024-04-22T20:45:30.063941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:48.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":95,"num_write":39,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"261'277","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:11.047810+0000","last_change":"2024-04-22T19:51:26.502070+0000","last_active":"2024-04-22T20:47:11.047810+0000","last_peered":"2024-04-22T20:47:11.047810+0000","last_clean":"2024-04-22T20:47:11.047810+0000","last_became_active":"2024-04-22T19:51:26.501700+0000","last_became_peered":"2024-04-22T19:51:26.501700+0000","last_unstale":"2024-04-22T20:47:11.047810+0000","last_undegraded":"2024-04-22T20:47:11.047810+0000","last_fullsized":"2024-04-22T20:47:11.047810+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:26:11.753132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4607139,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":443,"num_read_kb":6050,"num_write":1178,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"261'54","reported_seq":603,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181693+0000","last_change":"2024-04-22T19:51:28.509002+0000","last_active":"2024-04-22T20:45:30.181693+0000","last_peered":"2024-04-22T20:45:30.181693+0000","last_clean":"2024-04-22T20:45:30.181693+0000","last_became_active":"2024-04-22T19:51:28.508768+0000","last_became_peered":"2024-04-22T19:51:28.508768+0000","last_unstale":"2024-04-22T20:45:30.181693+0000","last_undegraded":"2024-04-22T20:45:30.181693+0000","last_fullsized":"2024-04-22T20:45:30.181693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:32:52.855073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":238,"num_read_kb":196,"num_write":80,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"261'285","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.477365+0000","last_change":"2024-04-22T19:51:26.497503+0000","last_active":"2024-04-22T20:47:44.477365+0000","last_peered":"2024-04-22T20:47:44.477365+0000","last_clean":"2024-04-22T20:47:44.477365+0000","last_became_active":"2024-04-22T19:51:26.497156+0000","last_became_peered":"2024-04-22T19:51:26.497156+0000","last_unstale":"2024-04-22T20:47:44.477365+0000","last_undegraded":"2024-04-22T20:47:44.477365+0000","last_fullsized":"2024-04-22T20:47:44.477365+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:23:31.844540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15135500,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":466,"num_read_kb":35895,"num_write":1174,"num_write_kb":35286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"261'4050","reported_seq":4612,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.156504+0000","last_change":"2024-04-22T19:51:28.515076+0000","last_active":"2024-04-22T20:45:30.156504+0000","last_peered":"2024-04-22T20:45:30.156504+0000","last_clean":"2024-04-22T20:45:30.156504+0000","last_became_active":"2024-04-22T19:51:28.514780+0000","last_became_peered":"2024-04-22T19:51:28.514780+0000","last_unstale":"2024-04-22T20:45:30.156504+0000","last_undegraded":"2024-04-22T20:45:30.156504+0000","last_fullsized":"2024-04-22T20:45:30.156504+0000","mapping_epoch":22,"log_start":"261'4000","ondisk_log_start":"261'4000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":3000,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:22:22.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14224,"num_read_kb":11535,"num_write":6070,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"261'243","reported_seq":1002,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.065500+0000","last_change":"2024-04-22T19:51:26.496549+0000","last_active":"2024-04-22T20:45:10.065500+0000","last_peered":"2024-04-22T20:45:10.065500+0000","last_clean":"2024-04-22T20:45:10.065500+0000","last_became_active":"2024-04-22T19:51:26.496214+0000","last_became_peered":"2024-04-22T19:51:26.496214+0000","last_unstale":"2024-04-22T20:45:10.065500+0000","last_undegraded":"2024-04-22T20:45:10.065500+0000","last_fullsized":"2024-04-22T20:45:10.065500+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:15:54.981915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6811717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":525,"num_read_kb":13796,"num_write":1122,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"261'298","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.067136+0000","last_change":"2024-04-22T19:51:28.508937+0000","last_active":"2024-04-22T20:45:50.067136+0000","last_peered":"2024-04-22T20:45:50.067136+0000","last_clean":"2024-04-22T20:45:50.067136+0000","last_became_active":"2024-04-22T19:51:28.508681+0000","last_became_peered":"2024-04-22T19:51:28.508681+0000","last_unstale":"2024-04-22T20:45:50.067136+0000","last_undegraded":"2024-04-22T20:45:50.067136+0000","last_fullsized":"2024-04-22T20:45:50.067136+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:21:44.423395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1231,"num_read_kb":1000,"num_write":489,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"261'215","reported_seq":824,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.184935+0000","last_change":"2024-04-22T19:51:26.505139+0000","last_active":"2024-04-22T20:45:10.184935+0000","last_peered":"2024-04-22T20:45:10.184935+0000","last_clean":"2024-04-22T20:45:10.184935+0000","last_became_active":"2024-04-22T19:51:26.502438+0000","last_became_peered":"2024-04-22T19:51:26.502438+0000","last_unstale":"2024-04-22T20:45:10.184935+0000","last_undegraded":"2024-04-22T20:45:10.184935+0000","last_fullsized":"2024-04-22T20:45:10.184935+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:27.926512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13702713,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":319,"num_read_kb":36636,"num_write":870,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"261'300","reported_seq":1179,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:52.068642+0000","last_change":"2024-04-22T19:51:26.505871+0000","last_active":"2024-04-22T20:46:52.068642+0000","last_peered":"2024-04-22T20:46:52.068642+0000","last_clean":"2024-04-22T20:46:52.068642+0000","last_became_active":"2024-04-22T19:51:26.504878+0000","last_became_peered":"2024-04-22T19:51:26.504878+0000","last_unstale":"2024-04-22T20:46:52.068642+0000","last_undegraded":"2024-04-22T20:46:52.068642+0000","last_fullsized":"2024-04-22T20:46:52.068642+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:57.296051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787489,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":944,"num_read_kb":55238,"num_write":1333,"num_write_kb":52838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"261'127","reported_seq":728,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.177008+0000","last_change":"2024-04-22T19:51:28.521641+0000","last_active":"2024-04-22T20:44:50.177008+0000","last_peered":"2024-04-22T20:44:50.177008+0000","last_clean":"2024-04-22T20:44:50.177008+0000","last_became_active":"2024-04-22T19:51:28.521372+0000","last_became_peered":"2024-04-22T19:51:28.521372+0000","last_unstale":"2024-04-22T20:44:50.177008+0000","last_undegraded":"2024-04-22T20:44:50.177008+0000","last_fullsized":"2024-04-22T20:44:50.177008+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:16.451481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":525,"num_read_kb":437,"num_write":198,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"261'224","reported_seq":878,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:08.497364+0000","last_change":"2024-04-22T19:52:04.618718+0000","last_active":"2024-04-22T20:47:08.497364+0000","last_peered":"2024-04-22T20:47:08.497364+0000","last_clean":"2024-04-22T20:47:08.497364+0000","last_became_active":"2024-04-22T19:52:04.618273+0000","last_became_peered":"2024-04-22T19:52:04.618273+0000","last_unstale":"2024-04-22T20:47:08.497364+0000","last_undegraded":"2024-04-22T20:47:08.497364+0000","last_fullsized":"2024-04-22T20:47:08.497364+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:09.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19315969,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":402,"num_read_kb":73466,"num_write":926,"num_write_kb":40708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"261'95","reported_seq":648,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180705+0000","last_change":"2024-04-22T19:51:28.513718+0000","last_active":"2024-04-22T20:44:40.180705+0000","last_peered":"2024-04-22T20:44:40.180705+0000","last_clean":"2024-04-22T20:44:40.180705+0000","last_became_active":"2024-04-22T19:51:28.513450+0000","last_became_peered":"2024-04-22T19:51:28.513450+0000","last_unstale":"2024-04-22T20:44:40.180705+0000","last_undegraded":"2024-04-22T20:44:40.180705+0000","last_fullsized":"2024-04-22T20:44:40.180705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:50.937435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":382,"num_read_kb":302,"num_write":152,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5385,"num_read_kb":3813,"num_write":2124,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1241,"ondisk_log_size":1241,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19082,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":13196,"num_read_kb":13004,"num_write":15956,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":38164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1853,"ondisk_log_size":1853,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1817836,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":16703,"num_read_kb":31506,"num_write":14213,"num_write_kb":11110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3293184,"data_stored":2776426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2244,"ondisk_log_size":2244,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1617,"num_read_kb":1920,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":85960,"num_read_kb":69763,"num_write":34673,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3889,"ondisk_log_size":3889,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":684407196,"num_objects":4507,"num_object_clones":0,"num_object_copies":13521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":30977,"num_read_kb":2216973,"num_write":73272,"num_write_kb":2138644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1075630080,"data_stored":1075630080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3665,"ondisk_log_size":3665,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":91317873,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":141,"num_read_kb":196064,"num_write":151,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182640640,"data_stored":182635746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":91317873,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":139,"num_read_kb":185784,"num_write":151,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182640640,"data_stored":182635746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575536,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":181480,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":185835520,"data_stored":183817698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":286390730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575538,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424572,"kb_used_data":162932,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":93947268,"statfs":{"total":96636764160,"available":96202002432,"internally_reserved":0,"allocated":166842368,"data_stored":164891323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":267909574},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575537,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308012,"kb_used_data":178340,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94063828,"statfs":{"total":96636764160,"available":96321359872,"internally_reserved":0,"allocated":182620160,"data_stored":180552293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575538,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":252084,"kb_used_data":166764,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94119756,"statfs":{"total":96636764160,"available":96378630144,"internally_reserved":0,"allocated":170766336,"data_stored":168845633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575538,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444004,"kb_used_data":196316,"kb_used_omap":1,"kb_used_meta":247678,"kb_avail":93927836,"statfs":{"total":96636764160,"available":96182104064,"internally_reserved":0,"allocated":201027584,"data_stored":199033946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":253622728},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575539,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":264928,"kb_used_data":171352,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94106912,"statfs":{"total":96636764160,"available":96365477888,"internally_reserved":0,"allocated":175464448,"data_stored":173547104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95812037},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575536,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330400,"kb_used_data":186840,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94041440,"statfs":{"total":96636764160,"available":96298434560,"internally_reserved":0,"allocated":191324160,"data_stored":189350803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575538,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434020,"kb_used_data":200156,"kb_used_omap":1,"kb_used_meta":233854,"kb_avail":93937820,"statfs":{"total":96636764160,"available":96192327680,"internally_reserved":0,"allocated":204959744,"data_stored":202964756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":239466950},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":147820544,"data_stored":147820544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139972608,"data_stored":139972608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105086976,"data_stored":105086976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":149049344,"data_stored":149049344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":120315904,"data_stored":120315904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":143065088,"data_stored":143065088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":133967872,"data_stored":133967872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136351744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":353680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":383474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":132055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":479987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":85199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":566114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":248456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":622592,"data_stored":527461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":5948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":4195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":3595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":6129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":4904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28405760,"data_stored":28403313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26214400,"data_stored":26214400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":18923520,"data_stored":18922784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10530816,"data_stored":10529105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24117248,"data_stored":24117248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37748736,"data_stored":37748736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24170496,"data_stored":24168864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24158208,"data_stored":24157393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":15728640,"data_stored":15728640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17879040,"data_stored":17877408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18915328,"data_stored":18914513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T20:48:19.509 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T20:48:19.838 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:19.850 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T20:48:19.853 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":1968,"stamp":"2024-04-22T20:48:18.327450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":869472021,"num_objects":4919,"num_object_clones":0,"num_object_copies":14345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4919,"num_whiteouts":0,"num_read":154246,"num_read_kb":2718939,"num_write":140739,"num_write_kb":2429877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13467,"ondisk_log_size":13467,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2919188,"kb_used_data":1444180,"kb_used_omap":12,"kb_used_meta":1474931,"kb_avail":752055532,"statfs":{"total":773094113280,"available":770104864768,"internally_reserved":0,"allocated":1478840320,"data_stored":1463003556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1510329906},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":34,"num_read_kb":39,"num_write":27,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005878"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.177805+0000","last_change":"2024-04-22T20:19:16.796805+0000","last_active":"2024-04-22T20:44:20.177805+0000","last_peered":"2024-04-22T20:44:20.177805+0000","last_clean":"2024-04-22T20:44:20.177805+0000","last_became_active":"2024-04-22T20:19:16.796520+0000","last_became_peered":"2024-04-22T20:19:16.796520+0000","last_unstale":"2024-04-22T20:44:20.177805+0000","last_undegraded":"2024-04-22T20:44:20.177805+0000","last_fullsized":"2024-04-22T20:44:20.177805+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:42.980562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"261'4","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186412+0000","last_change":"2024-04-22T20:05:17.900640+0000","last_active":"2024-04-22T20:45:20.186412+0000","last_peered":"2024-04-22T20:45:20.186412+0000","last_clean":"2024-04-22T20:45:20.186412+0000","last_became_active":"2024-04-22T20:05:17.900325+0000","last_became_peered":"2024-04-22T20:05:17.900325+0000","last_unstale":"2024-04-22T20:45:20.186412+0000","last_undegraded":"2024-04-22T20:45:20.186412+0000","last_fullsized":"2024-04-22T20:45:20.186412+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:55:20.209199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154687+0000","last_change":"2024-04-22T20:05:13.813157+0000","last_active":"2024-04-22T20:45:15.154687+0000","last_peered":"2024-04-22T20:45:15.154687+0000","last_clean":"2024-04-22T20:45:15.154687+0000","last_became_active":"2024-04-22T20:05:13.812933+0000","last_became_peered":"2024-04-22T20:05:13.812933+0000","last_unstale":"2024-04-22T20:45:15.154687+0000","last_undegraded":"2024-04-22T20:45:15.154687+0000","last_fullsized":"2024-04-22T20:45:15.154687+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:47:15.012252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"261'20","reported_seq":100,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.176985+0000","last_change":"2024-04-22T20:01:04.876250+0000","last_active":"2024-04-22T20:44:50.176985+0000","last_peered":"2024-04-22T20:44:50.176985+0000","last_clean":"2024-04-22T20:44:50.176985+0000","last_became_active":"2024-04-22T20:01:04.875981+0000","last_became_peered":"2024-04-22T20:01:04.875981+0000","last_unstale":"2024-04-22T20:44:50.176985+0000","last_undegraded":"2024-04-22T20:44:50.176985+0000","last_fullsized":"2024-04-22T20:44:50.176985+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:47:50.228681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.060560+0000","last_change":"2024-04-22T20:19:16.790684+0000","last_active":"2024-04-22T20:44:20.060560+0000","last_peered":"2024-04-22T20:44:20.060560+0000","last_clean":"2024-04-22T20:44:20.060560+0000","last_became_active":"2024-04-22T20:19:16.790377+0000","last_became_peered":"2024-04-22T20:19:16.790377+0000","last_unstale":"2024-04-22T20:44:20.060560+0000","last_undegraded":"2024-04-22T20:44:20.060560+0000","last_fullsized":"2024-04-22T20:44:20.060560+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:16:51.462209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.178227+0000","last_change":"2024-04-22T20:05:17.899675+0000","last_active":"2024-04-22T20:45:20.178227+0000","last_peered":"2024-04-22T20:45:20.178227+0000","last_clean":"2024-04-22T20:45:20.178227+0000","last_became_active":"2024-04-22T20:05:17.899338+0000","last_became_peered":"2024-04-22T20:05:17.899338+0000","last_unstale":"2024-04-22T20:45:20.178227+0000","last_undegraded":"2024-04-22T20:45:20.178227+0000","last_fullsized":"2024-04-22T20:45:20.178227+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:09:36.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"258'4","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.043137+0000","last_change":"2024-04-22T20:05:13.808152+0000","last_active":"2024-04-22T20:45:15.043137+0000","last_peered":"2024-04-22T20:45:15.043137+0000","last_clean":"2024-04-22T20:45:15.043137+0000","last_became_active":"2024-04-22T20:05:13.807231+0000","last_became_peered":"2024-04-22T20:05:13.807231+0000","last_unstale":"2024-04-22T20:45:15.043137+0000","last_undegraded":"2024-04-22T20:45:15.043137+0000","last_fullsized":"2024-04-22T20:45:15.043137+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:18:02.300257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"261'22","reported_seq":101,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.066043+0000","last_change":"2024-04-22T20:01:04.874850+0000","last_active":"2024-04-22T20:45:15.066043+0000","last_peered":"2024-04-22T20:45:15.066043+0000","last_clean":"2024-04-22T20:45:15.066043+0000","last_became_active":"2024-04-22T20:01:04.874575+0000","last_became_peered":"2024-04-22T20:01:04.874575+0000","last_unstale":"2024-04-22T20:45:15.066043+0000","last_undegraded":"2024-04-22T20:45:15.066043+0000","last_fullsized":"2024-04-22T20:45:15.066043+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:00:34.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":48,"num_write":47,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.298533+0000","last_change":"2024-04-22T20:19:16.794970+0000","last_active":"2024-04-22T20:43:30.298533+0000","last_peered":"2024-04-22T20:43:30.298533+0000","last_clean":"2024-04-22T20:43:30.298533+0000","last_became_active":"2024-04-22T20:19:16.794700+0000","last_became_peered":"2024-04-22T20:19:16.794700+0000","last_unstale":"2024-04-22T20:43:30.298533+0000","last_undegraded":"2024-04-22T20:43:30.298533+0000","last_fullsized":"2024-04-22T20:43:30.298533+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:59.399131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.177554+0000","last_change":"2024-04-22T20:05:18.211663+0000","last_active":"2024-04-22T20:44:55.177554+0000","last_peered":"2024-04-22T20:44:55.177554+0000","last_clean":"2024-04-22T20:44:55.177554+0000","last_became_active":"2024-04-22T20:05:18.211400+0000","last_became_peered":"2024-04-22T20:05:18.211400+0000","last_unstale":"2024-04-22T20:44:55.177554+0000","last_undegraded":"2024-04-22T20:44:55.177554+0000","last_fullsized":"2024-04-22T20:44:55.177554+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:07:00.571570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.067028+0000","last_change":"2024-04-22T20:05:13.809630+0000","last_active":"2024-04-22T20:45:25.067028+0000","last_peered":"2024-04-22T20:45:25.067028+0000","last_clean":"2024-04-22T20:45:25.067028+0000","last_became_active":"2024-04-22T20:05:13.809349+0000","last_became_peered":"2024-04-22T20:05:13.809349+0000","last_unstale":"2024-04-22T20:45:25.067028+0000","last_undegraded":"2024-04-22T20:45:25.067028+0000","last_fullsized":"2024-04-22T20:45:25.067028+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:04:26.130081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"261'24","reported_seq":114,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.167723+0000","last_change":"2024-04-22T20:01:04.870729+0000","last_active":"2024-04-22T20:43:35.167723+0000","last_peered":"2024-04-22T20:43:35.167723+0000","last_clean":"2024-04-22T20:43:35.167723+0000","last_became_active":"2024-04-22T20:01:04.869785+0000","last_became_peered":"2024-04-22T20:01:04.869785+0000","last_unstale":"2024-04-22T20:43:35.167723+0000","last_undegraded":"2024-04-22T20:43:35.167723+0000","last_fullsized":"2024-04-22T20:43:35.167723+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:24:59.438391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":82,"num_write":36,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.141064+0000","last_change":"2024-04-22T20:19:16.795441+0000","last_active":"2024-04-22T20:43:30.141064+0000","last_peered":"2024-04-22T20:43:30.141064+0000","last_clean":"2024-04-22T20:43:30.141064+0000","last_became_active":"2024-04-22T20:19:16.793667+0000","last_became_peered":"2024-04-22T20:19:16.793667+0000","last_unstale":"2024-04-22T20:43:30.141064+0000","last_undegraded":"2024-04-22T20:43:30.141064+0000","last_fullsized":"2024-04-22T20:43:30.141064+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:09.885678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"261'7","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:15.187088+0000","last_change":"2024-04-22T20:06:05.349958+0000","last_active":"2024-04-22T20:46:15.187088+0000","last_peered":"2024-04-22T20:46:15.187088+0000","last_clean":"2024-04-22T20:46:15.187088+0000","last_became_active":"2024-04-22T20:06:05.345554+0000","last_became_peered":"2024-04-22T20:06:05.345554+0000","last_unstale":"2024-04-22T20:46:15.187088+0000","last_undegraded":"2024-04-22T20:46:15.187088+0000","last_fullsized":"2024-04-22T20:46:15.187088+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:50:22.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":17,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1f","version":"258'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.173992+0000","last_change":"2024-04-22T20:05:13.806565+0000","last_active":"2024-04-22T20:44:50.173992+0000","last_peered":"2024-04-22T20:44:50.173992+0000","last_clean":"2024-04-22T20:44:50.173992+0000","last_became_active":"2024-04-22T20:05:13.806246+0000","last_became_peered":"2024-04-22T20:05:13.806246+0000","last_unstale":"2024-04-22T20:44:50.173992+0000","last_undegraded":"2024-04-22T20:44:50.173992+0000","last_fullsized":"2024-04-22T20:44:50.173992+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:04:13.772282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":16720,"num_write":13,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"261'59","reported_seq":198,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.141957+0000","last_change":"2024-04-22T20:01:04.874395+0000","last_active":"2024-04-22T20:43:35.141957+0000","last_peered":"2024-04-22T20:43:35.141957+0000","last_clean":"2024-04-22T20:43:35.141957+0000","last_became_active":"2024-04-22T20:01:04.873899+0000","last_became_peered":"2024-04-22T20:01:04.873899+0000","last_unstale":"2024-04-22T20:43:35.141957+0000","last_undegraded":"2024-04-22T20:43:35.141957+0000","last_fullsized":"2024-04-22T20:43:35.141957+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:23:27.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":183,"num_write":99,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.155406+0000","last_change":"2024-04-22T20:19:16.794646+0000","last_active":"2024-04-22T20:45:20.155406+0000","last_peered":"2024-04-22T20:45:20.155406+0000","last_clean":"2024-04-22T20:45:20.155406+0000","last_became_active":"2024-04-22T20:19:16.794110+0000","last_became_peered":"2024-04-22T20:19:16.794110+0000","last_unstale":"2024-04-22T20:45:20.155406+0000","last_undegraded":"2024-04-22T20:45:20.155406+0000","last_fullsized":"2024-04-22T20:45:20.155406+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:42.561415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"261'1","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:10.176246+0000","last_change":"2024-04-22T20:05:17.898277+0000","last_active":"2024-04-22T20:44:10.176246+0000","last_peered":"2024-04-22T20:44:10.176246+0000","last_clean":"2024-04-22T20:44:10.176246+0000","last_became_active":"2024-04-22T20:05:17.897771+0000","last_became_peered":"2024-04-22T20:05:17.897771+0000","last_unstale":"2024-04-22T20:44:10.176246+0000","last_undegraded":"2024-04-22T20:44:10.176246+0000","last_fullsized":"2024-04-22T20:44:10.176246+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:22:07.487507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"258'5","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.170541+0000","last_change":"2024-04-22T20:05:13.812772+0000","last_active":"2024-04-22T20:43:55.170541+0000","last_peered":"2024-04-22T20:43:55.170541+0000","last_clean":"2024-04-22T20:43:55.170541+0000","last_became_active":"2024-04-22T20:05:13.812308+0000","last_became_peered":"2024-04-22T20:05:13.812308+0000","last_unstale":"2024-04-22T20:43:55.170541+0000","last_undegraded":"2024-04-22T20:43:55.170541+0000","last_fullsized":"2024-04-22T20:43:55.170541+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:52.451966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":9,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"261'8","reported_seq":72,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.141914+0000","last_change":"2024-04-22T20:01:04.877587+0000","last_active":"2024-04-22T20:43:35.141914+0000","last_peered":"2024-04-22T20:43:35.141914+0000","last_clean":"2024-04-22T20:43:35.141914+0000","last_became_active":"2024-04-22T20:01:04.877209+0000","last_became_peered":"2024-04-22T20:01:04.877209+0000","last_unstale":"2024-04-22T20:43:35.141914+0000","last_undegraded":"2024-04-22T20:43:35.141914+0000","last_fullsized":"2024-04-22T20:43:35.141914+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:57.901050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.037056+0000","last_change":"2024-04-22T20:19:16.789917+0000","last_active":"2024-04-22T20:44:20.037056+0000","last_peered":"2024-04-22T20:44:20.037056+0000","last_clean":"2024-04-22T20:44:20.037056+0000","last_became_active":"2024-04-22T20:19:16.789599+0000","last_became_peered":"2024-04-22T20:19:16.789599+0000","last_unstale":"2024-04-22T20:44:20.037056+0000","last_undegraded":"2024-04-22T20:44:20.037056+0000","last_fullsized":"2024-04-22T20:44:20.037056+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:15.659980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"261'1","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.303074+0000","last_change":"2024-04-22T20:05:17.891180+0000","last_active":"2024-04-22T20:44:00.303074+0000","last_peered":"2024-04-22T20:44:00.303074+0000","last_clean":"2024-04-22T20:44:00.303074+0000","last_became_active":"2024-04-22T20:05:17.890841+0000","last_became_peered":"2024-04-22T20:05:17.890841+0000","last_unstale":"2024-04-22T20:44:00.303074+0000","last_undegraded":"2024-04-22T20:44:00.303074+0000","last_fullsized":"2024-04-22T20:44:00.303074+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:53:40.164764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":1,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.167430+0000","last_change":"2024-04-22T20:05:13.807783+0000","last_active":"2024-04-22T20:44:05.167430+0000","last_peered":"2024-04-22T20:44:05.167430+0000","last_clean":"2024-04-22T20:44:05.167430+0000","last_became_active":"2024-04-22T20:05:13.807516+0000","last_became_peered":"2024-04-22T20:05:13.807516+0000","last_unstale":"2024-04-22T20:44:05.167430+0000","last_undegraded":"2024-04-22T20:44:05.167430+0000","last_fullsized":"2024-04-22T20:44:05.167430+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:58:08.969201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"261'77","reported_seq":239,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.299254+0000","last_change":"2024-04-22T20:01:04.873339+0000","last_active":"2024-04-22T20:43:35.299254+0000","last_peered":"2024-04-22T20:43:35.299254+0000","last_clean":"2024-04-22T20:43:35.299254+0000","last_became_active":"2024-04-22T20:01:04.873015+0000","last_became_peered":"2024-04-22T20:01:04.873015+0000","last_unstale":"2024-04-22T20:43:35.299254+0000","last_undegraded":"2024-04-22T20:43:35.299254+0000","last_fullsized":"2024-04-22T20:43:35.299254+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:49:05.613359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":236,"num_write":125,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"261'1","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053060+0000","last_change":"2024-04-22T20:19:16.794883+0000","last_active":"2024-04-22T20:44:20.053060+0000","last_peered":"2024-04-22T20:44:20.053060+0000","last_clean":"2024-04-22T20:44:20.053060+0000","last_became_active":"2024-04-22T20:19:16.794564+0000","last_became_peered":"2024-04-22T20:19:16.794564+0000","last_unstale":"2024-04-22T20:44:20.053060+0000","last_undegraded":"2024-04-22T20:44:20.053060+0000","last_fullsized":"2024-04-22T20:44:20.053060+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:15.121442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"261'3","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314023+0000","last_change":"2024-04-22T20:05:17.901743+0000","last_active":"2024-04-22T20:45:20.314023+0000","last_peered":"2024-04-22T20:45:20.314023+0000","last_clean":"2024-04-22T20:45:20.314023+0000","last_became_active":"2024-04-22T20:05:17.901446+0000","last_became_peered":"2024-04-22T20:05:17.901446+0000","last_unstale":"2024-04-22T20:45:20.314023+0000","last_undegraded":"2024-04-22T20:45:20.314023+0000","last_fullsized":"2024-04-22T20:45:20.314023+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:13:01.401575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":11264,"num_write":8,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"261'1","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.177454+0000","last_change":"2024-04-22T20:05:13.798628+0000","last_active":"2024-04-22T20:45:15.177454+0000","last_peered":"2024-04-22T20:45:15.177454+0000","last_clean":"2024-04-22T20:45:15.177454+0000","last_became_active":"2024-04-22T20:05:13.798311+0000","last_became_peered":"2024-04-22T20:05:13.798311+0000","last_unstale":"2024-04-22T20:45:15.177454+0000","last_undegraded":"2024-04-22T20:45:15.177454+0000","last_fullsized":"2024-04-22T20:45:15.177454+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:21:50.192609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'27","reported_seq":116,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.176374+0000","last_change":"2024-04-22T20:01:04.873715+0000","last_active":"2024-04-22T20:44:45.176374+0000","last_peered":"2024-04-22T20:44:45.176374+0000","last_clean":"2024-04-22T20:44:45.176374+0000","last_became_active":"2024-04-22T20:01:04.873426+0000","last_became_peered":"2024-04-22T20:01:04.873426+0000","last_unstale":"2024-04-22T20:44:45.176374+0000","last_undegraded":"2024-04-22T20:44:45.176374+0000","last_fullsized":"2024-04-22T20:44:45.176374+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:29:40.987253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":80,"num_write":54,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053106+0000","last_change":"2024-04-22T20:19:16.800874+0000","last_active":"2024-04-22T20:44:20.053106+0000","last_peered":"2024-04-22T20:44:20.053106+0000","last_clean":"2024-04-22T20:44:20.053106+0000","last_became_active":"2024-04-22T20:19:16.800611+0000","last_became_peered":"2024-04-22T20:19:16.800611+0000","last_unstale":"2024-04-22T20:44:20.053106+0000","last_undegraded":"2024-04-22T20:44:20.053106+0000","last_fullsized":"2024-04-22T20:44:20.053106+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:27:17.425924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"258'4","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.155286+0000","last_change":"2024-04-22T20:05:17.892900+0000","last_active":"2024-04-22T20:45:20.155286+0000","last_peered":"2024-04-22T20:45:20.155286+0000","last_clean":"2024-04-22T20:45:20.155286+0000","last_became_active":"2024-04-22T20:05:17.892624+0000","last_became_peered":"2024-04-22T20:05:17.892624+0000","last_unstale":"2024-04-22T20:45:20.155286+0000","last_undegraded":"2024-04-22T20:45:20.155286+0000","last_fullsized":"2024-04-22T20:45:20.155286+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:02.296574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3072,"num_write":4,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'1","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179926+0000","last_change":"2024-04-22T20:05:13.816306+0000","last_active":"2024-04-22T20:45:15.179926+0000","last_peered":"2024-04-22T20:45:15.179926+0000","last_clean":"2024-04-22T20:45:15.179926+0000","last_became_active":"2024-04-22T20:05:13.816011+0000","last_became_peered":"2024-04-22T20:05:13.816011+0000","last_unstale":"2024-04-22T20:45:15.179926+0000","last_undegraded":"2024-04-22T20:45:15.179926+0000","last_fullsized":"2024-04-22T20:45:15.179926+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:44.934646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"261'43","reported_seq":150,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.038576+0000","last_change":"2024-04-22T20:01:04.872669+0000","last_active":"2024-04-22T20:44:35.038576+0000","last_peered":"2024-04-22T20:44:35.038576+0000","last_clean":"2024-04-22T20:44:35.038576+0000","last_became_active":"2024-04-22T20:01:04.872356+0000","last_became_peered":"2024-04-22T20:01:04.872356+0000","last_unstale":"2024-04-22T20:44:35.038576+0000","last_undegraded":"2024-04-22T20:44:35.038576+0000","last_fullsized":"2024-04-22T20:44:35.038576+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:11.955524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":119,"num_write":72,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041329+0000","last_change":"2024-04-22T20:19:16.796888+0000","last_active":"2024-04-22T20:45:00.041329+0000","last_peered":"2024-04-22T20:45:00.041329+0000","last_clean":"2024-04-22T20:45:00.041329+0000","last_became_active":"2024-04-22T20:19:16.796155+0000","last_became_peered":"2024-04-22T20:19:16.796155+0000","last_unstale":"2024-04-22T20:45:00.041329+0000","last_undegraded":"2024-04-22T20:45:00.041329+0000","last_fullsized":"2024-04-22T20:45:00.041329+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:25:57.681625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145610+0000","last_change":"2024-04-22T20:05:18.209303+0000","last_active":"2024-04-22T20:44:00.145610+0000","last_peered":"2024-04-22T20:44:00.145610+0000","last_clean":"2024-04-22T20:44:00.145610+0000","last_became_active":"2024-04-22T20:05:18.209023+0000","last_became_peered":"2024-04-22T20:05:18.209023+0000","last_unstale":"2024-04-22T20:44:00.145610+0000","last_undegraded":"2024-04-22T20:44:00.145610+0000","last_fullsized":"2024-04-22T20:44:00.145610+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:14:17.745357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"258'6","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.184202+0000","last_change":"2024-04-22T20:05:13.809878+0000","last_active":"2024-04-22T20:45:05.184202+0000","last_peered":"2024-04-22T20:45:05.184202+0000","last_clean":"2024-04-22T20:45:05.184202+0000","last_became_active":"2024-04-22T20:05:13.809548+0000","last_became_peered":"2024-04-22T20:05:13.809548+0000","last_unstale":"2024-04-22T20:45:05.184202+0000","last_undegraded":"2024-04-22T20:45:05.184202+0000","last_fullsized":"2024-04-22T20:45:05.184202+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:10.962028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":6146,"num_write":5,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'63","reported_seq":241,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.175344+0000","last_change":"2024-04-22T20:01:04.873570+0000","last_active":"2024-04-22T20:44:35.175344+0000","last_peered":"2024-04-22T20:44:35.175344+0000","last_clean":"2024-04-22T20:44:35.175344+0000","last_became_active":"2024-04-22T20:01:04.873236+0000","last_became_peered":"2024-04-22T20:01:04.873236+0000","last_unstale":"2024-04-22T20:44:35.175344+0000","last_undegraded":"2024-04-22T20:44:35.175344+0000","last_fullsized":"2024-04-22T20:44:35.175344+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:38.761169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":250,"num_write":111,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.064494+0000","last_change":"2024-04-22T20:19:16.791895+0000","last_active":"2024-04-22T20:45:00.064494+0000","last_peered":"2024-04-22T20:45:00.064494+0000","last_clean":"2024-04-22T20:45:00.064494+0000","last_became_active":"2024-04-22T20:19:16.791597+0000","last_became_peered":"2024-04-22T20:19:16.791597+0000","last_unstale":"2024-04-22T20:45:00.064494+0000","last_undegraded":"2024-04-22T20:45:00.064494+0000","last_fullsized":"2024-04-22T20:45:00.064494+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:51.409610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":40,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.153541+0000","last_change":"2024-04-22T20:05:18.211042+0000","last_active":"2024-04-22T20:45:05.153541+0000","last_peered":"2024-04-22T20:45:05.153541+0000","last_clean":"2024-04-22T20:45:05.153541+0000","last_became_active":"2024-04-22T20:05:18.210704+0000","last_became_peered":"2024-04-22T20:05:18.210704+0000","last_unstale":"2024-04-22T20:45:05.153541+0000","last_undegraded":"2024-04-22T20:45:05.153541+0000","last_fullsized":"2024-04-22T20:45:05.153541+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:53:47.345090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.048999+0000","last_change":"2024-04-22T20:05:13.811174+0000","last_active":"2024-04-22T20:43:55.048999+0000","last_peered":"2024-04-22T20:43:55.048999+0000","last_clean":"2024-04-22T20:43:55.048999+0000","last_became_active":"2024-04-22T20:05:13.810902+0000","last_became_peered":"2024-04-22T20:05:13.810902+0000","last_unstale":"2024-04-22T20:43:55.048999+0000","last_undegraded":"2024-04-22T20:43:55.048999+0000","last_fullsized":"2024-04-22T20:43:55.048999+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:05:52.847563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"250'9","reported_seq":76,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.149931+0000","last_change":"2024-04-22T20:01:04.874512+0000","last_active":"2024-04-22T20:44:35.149931+0000","last_peered":"2024-04-22T20:44:35.149931+0000","last_clean":"2024-04-22T20:44:35.149931+0000","last_became_active":"2024-04-22T20:01:04.874230+0000","last_became_peered":"2024-04-22T20:01:04.874230+0000","last_unstale":"2024-04-22T20:44:35.149931+0000","last_undegraded":"2024-04-22T20:44:35.149931+0000","last_fullsized":"2024-04-22T20:44:35.149931+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:17:12.953078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.059314+0000","last_change":"2024-04-22T20:19:16.795626+0000","last_active":"2024-04-22T20:45:00.059314+0000","last_peered":"2024-04-22T20:45:00.059314+0000","last_clean":"2024-04-22T20:45:00.059314+0000","last_became_active":"2024-04-22T20:19:16.795139+0000","last_became_peered":"2024-04-22T20:19:16.795139+0000","last_unstale":"2024-04-22T20:45:00.059314+0000","last_undegraded":"2024-04-22T20:45:00.059314+0000","last_fullsized":"2024-04-22T20:45:00.059314+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:17:47.054792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"261'15","reported_seq":74,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145637+0000","last_change":"2024-04-22T20:05:18.212823+0000","last_active":"2024-04-22T20:44:00.145637+0000","last_peered":"2024-04-22T20:44:00.145637+0000","last_clean":"2024-04-22T20:44:00.145637+0000","last_became_active":"2024-04-22T20:05:18.212336+0000","last_became_peered":"2024-04-22T20:05:18.212336+0000","last_unstale":"2024-04-22T20:44:00.145637+0000","last_undegraded":"2024-04-22T20:44:00.145637+0000","last_fullsized":"2024-04-22T20:44:00.145637+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:32.041183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":38918,"num_write":22,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"261'1","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.058251+0000","last_change":"2024-04-22T20:05:13.816425+0000","last_active":"2024-04-22T20:43:55.058251+0000","last_peered":"2024-04-22T20:43:55.058251+0000","last_clean":"2024-04-22T20:43:55.058251+0000","last_became_active":"2024-04-22T20:05:13.816036+0000","last_became_peered":"2024-04-22T20:05:13.816036+0000","last_unstale":"2024-04-22T20:43:55.058251+0000","last_undegraded":"2024-04-22T20:43:55.058251+0000","last_fullsized":"2024-04-22T20:43:55.058251+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:36.892934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'53","reported_seq":176,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.062011+0000","last_change":"2024-04-22T20:01:04.878348+0000","last_active":"2024-04-22T20:44:35.062011+0000","last_peered":"2024-04-22T20:44:35.062011+0000","last_clean":"2024-04-22T20:44:35.062011+0000","last_became_active":"2024-04-22T20:01:04.878067+0000","last_became_peered":"2024-04-22T20:01:04.878067+0000","last_unstale":"2024-04-22T20:44:35.062011+0000","last_undegraded":"2024-04-22T20:44:35.062011+0000","last_fullsized":"2024-04-22T20:44:35.062011+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:52:49.300812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":150,"num_write":99,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.183432+0000","last_change":"2024-04-22T20:19:16.796141+0000","last_active":"2024-04-22T20:45:00.183432+0000","last_peered":"2024-04-22T20:45:00.183432+0000","last_clean":"2024-04-22T20:45:00.183432+0000","last_became_active":"2024-04-22T20:19:16.795855+0000","last_became_peered":"2024-04-22T20:19:16.795855+0000","last_unstale":"2024-04-22T20:45:00.183432+0000","last_undegraded":"2024-04-22T20:45:00.183432+0000","last_fullsized":"2024-04-22T20:45:00.183432+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:02.646044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166753+0000","last_change":"2024-04-22T20:05:17.902267+0000","last_active":"2024-04-22T20:44:00.166753+0000","last_peered":"2024-04-22T20:44:00.166753+0000","last_clean":"2024-04-22T20:44:00.166753+0000","last_became_active":"2024-04-22T20:05:17.901954+0000","last_became_peered":"2024-04-22T20:05:17.901954+0000","last_unstale":"2024-04-22T20:44:00.166753+0000","last_undegraded":"2024-04-22T20:44:00.166753+0000","last_fullsized":"2024-04-22T20:44:00.166753+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:44:04.704303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'2","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.170642+0000","last_change":"2024-04-22T20:05:13.813528+0000","last_active":"2024-04-22T20:43:55.170642+0000","last_peered":"2024-04-22T20:43:55.170642+0000","last_clean":"2024-04-22T20:43:55.170642+0000","last_became_active":"2024-04-22T20:05:13.813256+0000","last_became_peered":"2024-04-22T20:05:13.813256+0000","last_unstale":"2024-04-22T20:43:55.170642+0000","last_undegraded":"2024-04-22T20:43:55.170642+0000","last_fullsized":"2024-04-22T20:43:55.170642+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:19.323647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":44,"num_write":2,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"261'46","reported_seq":164,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.064807+0000","last_change":"2024-04-22T20:01:04.874056+0000","last_active":"2024-04-22T20:45:35.064807+0000","last_peered":"2024-04-22T20:45:35.064807+0000","last_clean":"2024-04-22T20:45:35.064807+0000","last_became_active":"2024-04-22T20:01:04.873533+0000","last_became_peered":"2024-04-22T20:01:04.873533+0000","last_unstale":"2024-04-22T20:45:35.064807+0000","last_undegraded":"2024-04-22T20:45:35.064807+0000","last_fullsized":"2024-04-22T20:45:35.064807+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:43.868445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":192,"num_read_kb":137,"num_write":82,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158918+0000","last_change":"2024-04-22T20:19:16.793148+0000","last_active":"2024-04-22T20:45:50.158918+0000","last_peered":"2024-04-22T20:45:50.158918+0000","last_clean":"2024-04-22T20:45:50.158918+0000","last_became_active":"2024-04-22T20:19:16.792856+0000","last_became_peered":"2024-04-22T20:19:16.792856+0000","last_unstale":"2024-04-22T20:45:50.158918+0000","last_undegraded":"2024-04-22T20:45:50.158918+0000","last_fullsized":"2024-04-22T20:45:50.158918+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:34.522183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'7","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041319+0000","last_change":"2024-04-22T20:05:17.897292+0000","last_active":"2024-04-22T20:45:00.041319+0000","last_peered":"2024-04-22T20:45:00.041319+0000","last_clean":"2024-04-22T20:45:00.041319+0000","last_became_active":"2024-04-22T20:05:17.897040+0000","last_became_peered":"2024-04-22T20:05:17.897040+0000","last_unstale":"2024-04-22T20:45:00.041319+0000","last_undegraded":"2024-04-22T20:45:00.041319+0000","last_fullsized":"2024-04-22T20:45:00.041319+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:13:37.748724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":12290,"num_write":6,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"261'3","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.174581+0000","last_change":"2024-04-22T20:05:13.814475+0000","last_active":"2024-04-22T20:44:55.174581+0000","last_peered":"2024-04-22T20:44:55.174581+0000","last_clean":"2024-04-22T20:44:55.174581+0000","last_became_active":"2024-04-22T20:05:13.814222+0000","last_became_peered":"2024-04-22T20:05:13.814222+0000","last_unstale":"2024-04-22T20:44:55.174581+0000","last_undegraded":"2024-04-22T20:44:55.174581+0000","last_fullsized":"2024-04-22T20:44:55.174581+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:29:09.438132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"261'54","reported_seq":190,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063228+0000","last_change":"2024-04-22T20:01:04.873752+0000","last_active":"2024-04-22T20:45:25.063228+0000","last_peered":"2024-04-22T20:45:25.063228+0000","last_clean":"2024-04-22T20:45:25.063228+0000","last_became_active":"2024-04-22T20:01:04.873422+0000","last_became_peered":"2024-04-22T20:01:04.873422+0000","last_unstale":"2024-04-22T20:45:25.063228+0000","last_undegraded":"2024-04-22T20:45:25.063228+0000","last_fullsized":"2024-04-22T20:45:25.063228+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:49.464347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":184,"num_write":82,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.167040+0000","last_change":"2024-04-22T20:19:16.795990+0000","last_active":"2024-04-22T20:43:30.167040+0000","last_peered":"2024-04-22T20:43:30.167040+0000","last_clean":"2024-04-22T20:43:30.167040+0000","last_became_active":"2024-04-22T20:19:16.795633+0000","last_became_peered":"2024-04-22T20:19:16.795633+0000","last_unstale":"2024-04-22T20:43:30.167040+0000","last_undegraded":"2024-04-22T20:43:30.167040+0000","last_fullsized":"2024-04-22T20:43:30.167040+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:55.584364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"261'8","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041286+0000","last_change":"2024-04-22T20:05:17.900013+0000","last_active":"2024-04-22T20:45:00.041286+0000","last_peered":"2024-04-22T20:45:00.041286+0000","last_clean":"2024-04-22T20:45:00.041286+0000","last_became_active":"2024-04-22T20:05:17.899682+0000","last_became_peered":"2024-04-22T20:05:17.899682+0000","last_unstale":"2024-04-22T20:45:00.041286+0000","last_undegraded":"2024-04-22T20:45:00.041286+0000","last_fullsized":"2024-04-22T20:45:00.041286+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:57:08.679442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":8210,"num_write":12,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"261'5","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.152411+0000","last_change":"2024-04-22T20:05:13.813246+0000","last_active":"2024-04-22T20:44:55.152411+0000","last_peered":"2024-04-22T20:44:55.152411+0000","last_clean":"2024-04-22T20:44:55.152411+0000","last_became_active":"2024-04-22T20:05:13.812893+0000","last_became_peered":"2024-04-22T20:05:13.812893+0000","last_unstale":"2024-04-22T20:44:55.152411+0000","last_undegraded":"2024-04-22T20:44:55.152411+0000","last_fullsized":"2024-04-22T20:44:55.152411+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:26:36.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":15,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"261'47","reported_seq":163,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.181140+0000","last_change":"2024-04-22T20:01:04.878065+0000","last_active":"2024-04-22T20:45:25.181140+0000","last_peered":"2024-04-22T20:45:25.181140+0000","last_clean":"2024-04-22T20:45:25.181140+0000","last_became_active":"2024-04-22T20:01:04.877755+0000","last_became_peered":"2024-04-22T20:01:04.877755+0000","last_unstale":"2024-04-22T20:45:25.181140+0000","last_undegraded":"2024-04-22T20:45:25.181140+0000","last_fullsized":"2024-04-22T20:45:25.181140+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:41:43.369530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":135,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"260'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.178798+0000","last_change":"2024-04-22T20:19:16.793874+0000","last_active":"2024-04-22T20:45:05.178798+0000","last_peered":"2024-04-22T20:45:05.178798+0000","last_clean":"2024-04-22T20:45:05.178798+0000","last_became_active":"2024-04-22T20:19:16.793555+0000","last_became_peered":"2024-04-22T20:19:16.793555+0000","last_unstale":"2024-04-22T20:45:05.178798+0000","last_undegraded":"2024-04-22T20:45:05.178798+0000","last_fullsized":"2024-04-22T20:45:05.178798+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:33:04.108646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"258'2","reported_seq":44,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.152971+0000","last_change":"2024-04-22T20:05:17.890318+0000","last_active":"2024-04-22T20:45:00.152971+0000","last_peered":"2024-04-22T20:45:00.152971+0000","last_clean":"2024-04-22T20:45:00.152971+0000","last_became_active":"2024-04-22T20:05:17.890070+0000","last_became_peered":"2024-04-22T20:05:17.890070+0000","last_unstale":"2024-04-22T20:45:00.152971+0000","last_undegraded":"2024-04-22T20:45:00.152971+0000","last_fullsized":"2024-04-22T20:45:00.152971+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:14:41.324774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"261'3","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.310559+0000","last_change":"2024-04-22T20:05:13.807133+0000","last_active":"2024-04-22T20:44:55.310559+0000","last_peered":"2024-04-22T20:44:55.310559+0000","last_clean":"2024-04-22T20:44:55.310559+0000","last_became_active":"2024-04-22T20:05:13.806882+0000","last_became_peered":"2024-04-22T20:05:13.806882+0000","last_unstale":"2024-04-22T20:44:55.310559+0000","last_undegraded":"2024-04-22T20:44:55.310559+0000","last_fullsized":"2024-04-22T20:44:55.310559+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:52:50.911604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"261'11","reported_seq":80,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.178877+0000","last_change":"2024-04-22T20:01:04.861370+0000","last_active":"2024-04-22T20:45:25.178877+0000","last_peered":"2024-04-22T20:45:25.178877+0000","last_clean":"2024-04-22T20:45:25.178877+0000","last_became_active":"2024-04-22T20:01:04.860991+0000","last_became_peered":"2024-04-22T20:01:04.860991+0000","last_unstale":"2024-04-22T20:45:25.178877+0000","last_undegraded":"2024-04-22T20:45:25.178877+0000","last_fullsized":"2024-04-22T20:45:25.178877+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:26:49.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":31,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158885+0000","last_change":"2024-04-22T20:19:16.794789+0000","last_active":"2024-04-22T20:45:50.158885+0000","last_peered":"2024-04-22T20:45:50.158885+0000","last_clean":"2024-04-22T20:45:50.158885+0000","last_became_active":"2024-04-22T20:19:16.791995+0000","last_became_peered":"2024-04-22T20:19:16.791995+0000","last_unstale":"2024-04-22T20:45:50.158885+0000","last_undegraded":"2024-04-22T20:45:50.158885+0000","last_fullsized":"2024-04-22T20:45:50.158885+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:04:16.246591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"258'4","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.311157+0000","last_change":"2024-04-22T20:05:17.890780+0000","last_active":"2024-04-22T20:45:00.311157+0000","last_peered":"2024-04-22T20:45:00.311157+0000","last_clean":"2024-04-22T20:45:00.311157+0000","last_became_active":"2024-04-22T20:05:17.890408+0000","last_became_peered":"2024-04-22T20:05:17.890408+0000","last_unstale":"2024-04-22T20:45:00.311157+0000","last_undegraded":"2024-04-22T20:45:00.311157+0000","last_fullsized":"2024-04-22T20:45:00.311157+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:30:52.042989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1342,"num_write":4,"num_write_kb":1342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"258'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.182683+0000","last_change":"2024-04-22T20:05:13.808782+0000","last_active":"2024-04-22T20:44:55.182683+0000","last_peered":"2024-04-22T20:44:55.182683+0000","last_clean":"2024-04-22T20:44:55.182683+0000","last_became_active":"2024-04-22T20:05:13.808473+0000","last_became_peered":"2024-04-22T20:05:13.808473+0000","last_unstale":"2024-04-22T20:44:55.182683+0000","last_undegraded":"2024-04-22T20:44:55.182683+0000","last_fullsized":"2024-04-22T20:44:55.182683+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:39:30.056682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"261'46","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.155908+0000","last_change":"2024-04-22T20:01:04.875088+0000","last_active":"2024-04-22T20:45:25.155908+0000","last_peered":"2024-04-22T20:45:25.155908+0000","last_clean":"2024-04-22T20:45:25.155908+0000","last_became_active":"2024-04-22T20:01:04.874816+0000","last_became_peered":"2024-04-22T20:01:04.874816+0000","last_unstale":"2024-04-22T20:45:25.155908+0000","last_undegraded":"2024-04-22T20:45:25.155908+0000","last_fullsized":"2024-04-22T20:45:25.155908+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:59:09.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":291,"num_read_kb":200,"num_write":62,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.066966+0000","last_change":"2024-04-22T20:19:16.795282+0000","last_active":"2024-04-22T20:45:25.066966+0000","last_peered":"2024-04-22T20:45:25.066966+0000","last_clean":"2024-04-22T20:45:25.066966+0000","last_became_active":"2024-04-22T20:19:16.794954+0000","last_became_peered":"2024-04-22T20:19:16.794954+0000","last_unstale":"2024-04-22T20:45:25.066966+0000","last_undegraded":"2024-04-22T20:45:25.066966+0000","last_fullsized":"2024-04-22T20:45:25.066966+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:53:00.610478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"261'3","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158894+0000","last_change":"2024-04-22T20:05:17.889647+0000","last_active":"2024-04-22T20:45:50.158894+0000","last_peered":"2024-04-22T20:45:50.158894+0000","last_clean":"2024-04-22T20:45:50.158894+0000","last_became_active":"2024-04-22T20:05:17.889370+0000","last_became_peered":"2024-04-22T20:05:17.889370+0000","last_unstale":"2024-04-22T20:45:50.158894+0000","last_undegraded":"2024-04-22T20:45:50.158894+0000","last_fullsized":"2024-04-22T20:45:50.158894+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:26:55.859900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5246080,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":4100,"num_write":3,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"261'1","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.066445+0000","last_change":"2024-04-22T20:05:13.815613+0000","last_active":"2024-04-22T20:45:45.066445+0000","last_peered":"2024-04-22T20:45:45.066445+0000","last_clean":"2024-04-22T20:45:45.066445+0000","last_became_active":"2024-04-22T20:05:13.815317+0000","last_became_peered":"2024-04-22T20:05:13.815317+0000","last_unstale":"2024-04-22T20:45:45.066445+0000","last_undegraded":"2024-04-22T20:45:45.066445+0000","last_fullsized":"2024-04-22T20:45:45.066445+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:25:12.295293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"261'28","reported_seq":118,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.145012+0000","last_change":"2024-04-22T20:01:04.878886+0000","last_active":"2024-04-22T20:43:55.145012+0000","last_peered":"2024-04-22T20:43:55.145012+0000","last_clean":"2024-04-22T20:43:55.145012+0000","last_became_active":"2024-04-22T20:01:04.878599+0000","last_became_peered":"2024-04-22T20:01:04.878599+0000","last_unstale":"2024-04-22T20:43:55.145012+0000","last_undegraded":"2024-04-22T20:43:55.145012+0000","last_fullsized":"2024-04-22T20:43:55.145012+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:57:43.301162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":77,"num_write":54,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.050575+0000","last_change":"2024-04-22T20:19:16.794996+0000","last_active":"2024-04-22T20:44:05.050575+0000","last_peered":"2024-04-22T20:44:05.050575+0000","last_clean":"2024-04-22T20:44:05.050575+0000","last_became_active":"2024-04-22T20:19:16.793678+0000","last_became_peered":"2024-04-22T20:19:16.793678+0000","last_unstale":"2024-04-22T20:44:05.050575+0000","last_undegraded":"2024-04-22T20:44:05.050575+0000","last_fullsized":"2024-04-22T20:44:05.050575+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:48:13.172887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158865+0000","last_change":"2024-04-22T20:05:17.892875+0000","last_active":"2024-04-22T20:45:50.158865+0000","last_peered":"2024-04-22T20:45:50.158865+0000","last_clean":"2024-04-22T20:45:50.158865+0000","last_became_active":"2024-04-22T20:05:17.892604+0000","last_became_peered":"2024-04-22T20:05:17.892604+0000","last_unstale":"2024-04-22T20:45:50.158865+0000","last_undegraded":"2024-04-22T20:45:50.158865+0000","last_fullsized":"2024-04-22T20:45:50.158865+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:20.757720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"261'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.068955+0000","last_change":"2024-04-22T20:05:13.814193+0000","last_active":"2024-04-22T20:45:45.068955+0000","last_peered":"2024-04-22T20:45:45.068955+0000","last_clean":"2024-04-22T20:45:45.068955+0000","last_became_active":"2024-04-22T20:05:13.813946+0000","last_became_peered":"2024-04-22T20:05:13.813946+0000","last_unstale":"2024-04-22T20:45:45.068955+0000","last_undegraded":"2024-04-22T20:45:45.068955+0000","last_fullsized":"2024-04-22T20:45:45.068955+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:04:48.745503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":15362,"num_write":8,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"261'28","reported_seq":126,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.173638+0000","last_change":"2024-04-22T20:01:04.873158+0000","last_active":"2024-04-22T20:43:50.173638+0000","last_peered":"2024-04-22T20:43:50.173638+0000","last_clean":"2024-04-22T20:43:50.173638+0000","last_became_active":"2024-04-22T20:01:04.872834+0000","last_became_peered":"2024-04-22T20:01:04.872834+0000","last_unstale":"2024-04-22T20:43:50.173638+0000","last_undegraded":"2024-04-22T20:43:50.173638+0000","last_fullsized":"2024-04-22T20:43:50.173638+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:25.675744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":90,"num_write":50,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.313962+0000","last_change":"2024-04-22T20:19:16.791183+0000","last_active":"2024-04-22T20:45:20.313962+0000","last_peered":"2024-04-22T20:45:20.313962+0000","last_clean":"2024-04-22T20:45:20.313962+0000","last_became_active":"2024-04-22T20:19:16.790917+0000","last_became_peered":"2024-04-22T20:19:16.790917+0000","last_unstale":"2024-04-22T20:45:20.313962+0000","last_undegraded":"2024-04-22T20:45:20.313962+0000","last_fullsized":"2024-04-22T20:45:20.313962+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:12:37.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:00.048158+0000","last_change":"2024-04-22T20:05:18.211747+0000","last_active":"2024-04-22T20:46:00.048158+0000","last_peered":"2024-04-22T20:46:00.048158+0000","last_clean":"2024-04-22T20:46:00.048158+0000","last_became_active":"2024-04-22T20:05:18.211445+0000","last_became_peered":"2024-04-22T20:05:18.211445+0000","last_unstale":"2024-04-22T20:46:00.048158+0000","last_undegraded":"2024-04-22T20:46:00.048158+0000","last_fullsized":"2024-04-22T20:46:00.048158+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:14:07.558996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"258'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183750+0000","last_change":"2024-04-22T20:05:13.809343+0000","last_active":"2024-04-22T20:45:45.183750+0000","last_peered":"2024-04-22T20:45:45.183750+0000","last_clean":"2024-04-22T20:45:45.183750+0000","last_became_active":"2024-04-22T20:05:13.806928+0000","last_became_peered":"2024-04-22T20:05:13.806928+0000","last_unstale":"2024-04-22T20:45:45.183750+0000","last_undegraded":"2024-04-22T20:45:45.183750+0000","last_fullsized":"2024-04-22T20:45:45.183750+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:36:20.049785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"261'28","reported_seq":117,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.301681+0000","last_change":"2024-04-22T20:01:04.869517+0000","last_active":"2024-04-22T20:43:50.301681+0000","last_peered":"2024-04-22T20:43:50.301681+0000","last_clean":"2024-04-22T20:43:50.301681+0000","last_became_active":"2024-04-22T20:01:04.869226+0000","last_became_peered":"2024-04-22T20:01:04.869226+0000","last_unstale":"2024-04-22T20:43:50.301681+0000","last_undegraded":"2024-04-22T20:43:50.301681+0000","last_fullsized":"2024-04-22T20:43:50.301681+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:02:16.025206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":80,"num_write":56,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062354+0000","last_change":"2024-04-22T20:19:16.796398+0000","last_active":"2024-04-22T20:45:20.062354+0000","last_peered":"2024-04-22T20:45:20.062354+0000","last_clean":"2024-04-22T20:45:20.062354+0000","last_became_active":"2024-04-22T20:19:16.795524+0000","last_became_peered":"2024-04-22T20:19:16.795524+0000","last_unstale":"2024-04-22T20:45:20.062354+0000","last_undegraded":"2024-04-22T20:45:20.062354+0000","last_fullsized":"2024-04-22T20:45:20.062354+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:24:50.852300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158909+0000","last_change":"2024-04-22T20:05:17.899183+0000","last_active":"2024-04-22T20:45:50.158909+0000","last_peered":"2024-04-22T20:45:50.158909+0000","last_clean":"2024-04-22T20:45:50.158909+0000","last_became_active":"2024-04-22T20:05:17.898858+0000","last_became_peered":"2024-04-22T20:05:17.898858+0000","last_unstale":"2024-04-22T20:45:50.158909+0000","last_undegraded":"2024-04-22T20:45:50.158909+0000","last_fullsized":"2024-04-22T20:45:50.158909+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:21.640059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"258'3","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.047547+0000","last_change":"2024-04-22T20:05:13.808403+0000","last_active":"2024-04-22T20:45:55.047547+0000","last_peered":"2024-04-22T20:45:55.047547+0000","last_clean":"2024-04-22T20:45:55.047547+0000","last_became_active":"2024-04-22T20:05:13.807945+0000","last_became_peered":"2024-04-22T20:05:13.807945+0000","last_unstale":"2024-04-22T20:45:55.047547+0000","last_undegraded":"2024-04-22T20:45:55.047547+0000","last_fullsized":"2024-04-22T20:45:55.047547+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:02:06.578487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":8,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"261'24","reported_seq":105,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.057658+0000","last_change":"2024-04-22T20:01:04.872099+0000","last_active":"2024-04-22T20:43:50.057658+0000","last_peered":"2024-04-22T20:43:50.057658+0000","last_clean":"2024-04-22T20:43:50.057658+0000","last_became_active":"2024-04-22T20:01:04.871829+0000","last_became_peered":"2024-04-22T20:01:04.871829+0000","last_unstale":"2024-04-22T20:43:50.057658+0000","last_undegraded":"2024-04-22T20:43:50.057658+0000","last_fullsized":"2024-04-22T20:43:50.057658+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:30:28.808258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":61,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145629+0000","last_change":"2024-04-22T20:19:16.794419+0000","last_active":"2024-04-22T20:44:00.145629+0000","last_peered":"2024-04-22T20:44:00.145629+0000","last_clean":"2024-04-22T20:44:00.145629+0000","last_became_active":"2024-04-22T20:19:16.793727+0000","last_became_peered":"2024-04-22T20:19:16.793727+0000","last_unstale":"2024-04-22T20:44:00.145629+0000","last_undegraded":"2024-04-22T20:44:00.145629+0000","last_fullsized":"2024-04-22T20:44:00.145629+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:50.871340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"261'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169594+0000","last_change":"2024-04-22T20:05:17.899691+0000","last_active":"2024-04-22T20:44:20.169594+0000","last_peered":"2024-04-22T20:44:20.169594+0000","last_clean":"2024-04-22T20:44:20.169594+0000","last_became_active":"2024-04-22T20:05:17.899328+0000","last_became_peered":"2024-04-22T20:05:17.899328+0000","last_unstale":"2024-04-22T20:44:20.169594+0000","last_undegraded":"2024-04-22T20:44:20.169594+0000","last_fullsized":"2024-04-22T20:44:20.169594+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:57:36.265424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":9,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.172707+0000","last_change":"2024-04-22T20:05:13.812566+0000","last_active":"2024-04-22T20:44:15.172707+0000","last_peered":"2024-04-22T20:44:15.172707+0000","last_clean":"2024-04-22T20:44:15.172707+0000","last_became_active":"2024-04-22T20:05:13.811986+0000","last_became_peered":"2024-04-22T20:05:13.811986+0000","last_unstale":"2024-04-22T20:44:15.172707+0000","last_undegraded":"2024-04-22T20:44:15.172707+0000","last_fullsized":"2024-04-22T20:44:15.172707+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:29.867735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5291296,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":8240,"num_write":9,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"261'66","reported_seq":211,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:06.776789+0000","last_change":"2024-04-22T20:01:04.875092+0000","last_active":"2024-04-22T20:44:06.776789+0000","last_peered":"2024-04-22T20:44:06.776789+0000","last_clean":"2024-04-22T20:44:06.776789+0000","last_became_active":"2024-04-22T20:01:04.874827+0000","last_became_peered":"2024-04-22T20:01:04.874827+0000","last_unstale":"2024-04-22T20:44:06.776789+0000","last_undegraded":"2024-04-22T20:44:06.776789+0000","last_fullsized":"2024-04-22T20:44:06.776789+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:49.143833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":281,"num_read_kb":200,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.052345+0000","last_change":"2024-04-22T20:19:16.794759+0000","last_active":"2024-04-22T20:44:15.052345+0000","last_peered":"2024-04-22T20:44:15.052345+0000","last_clean":"2024-04-22T20:44:15.052345+0000","last_became_active":"2024-04-22T20:19:16.794434+0000","last_became_peered":"2024-04-22T20:19:16.794434+0000","last_unstale":"2024-04-22T20:44:15.052345+0000","last_undegraded":"2024-04-22T20:44:15.052345+0000","last_fullsized":"2024-04-22T20:44:15.052345+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:47.954843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"261'6","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.052938+0000","last_change":"2024-04-22T20:05:18.209353+0000","last_active":"2024-04-22T20:44:20.052938+0000","last_peered":"2024-04-22T20:44:20.052938+0000","last_clean":"2024-04-22T20:44:20.052938+0000","last_became_active":"2024-04-22T20:05:18.209025+0000","last_became_peered":"2024-04-22T20:05:18.209025+0000","last_unstale":"2024-04-22T20:44:20.052938+0000","last_undegraded":"2024-04-22T20:44:20.052938+0000","last_fullsized":"2024-04-22T20:44:20.052938+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:14:29.836329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":13312,"num_write":11,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"261'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.036411+0000","last_change":"2024-04-22T20:05:13.807877+0000","last_active":"2024-04-22T20:44:15.036411+0000","last_peered":"2024-04-22T20:44:15.036411+0000","last_clean":"2024-04-22T20:44:15.036411+0000","last_became_active":"2024-04-22T20:05:13.807603+0000","last_became_peered":"2024-04-22T20:05:13.807603+0000","last_unstale":"2024-04-22T20:44:15.036411+0000","last_undegraded":"2024-04-22T20:44:15.036411+0000","last_fullsized":"2024-04-22T20:44:15.036411+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:05:08.359131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"258'23","reported_seq":112,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.055220+0000","last_change":"2024-04-22T20:01:04.869170+0000","last_active":"2024-04-22T20:44:35.055220+0000","last_peered":"2024-04-22T20:44:35.055220+0000","last_clean":"2024-04-22T20:44:35.055220+0000","last_became_active":"2024-04-22T20:01:04.868847+0000","last_became_peered":"2024-04-22T20:01:04.868847+0000","last_unstale":"2024-04-22T20:44:35.055220+0000","last_undegraded":"2024-04-22T20:44:35.055220+0000","last_fullsized":"2024-04-22T20:44:35.055220+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:17:28.313341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":76,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.050611+0000","last_change":"2024-04-22T20:19:16.796622+0000","last_active":"2024-04-22T20:44:05.050611+0000","last_peered":"2024-04-22T20:44:05.050611+0000","last_clean":"2024-04-22T20:44:05.050611+0000","last_became_active":"2024-04-22T20:19:16.795613+0000","last_became_peered":"2024-04-22T20:19:16.795613+0000","last_unstale":"2024-04-22T20:44:05.050611+0000","last_undegraded":"2024-04-22T20:44:05.050611+0000","last_fullsized":"2024-04-22T20:44:05.050611+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:22:45.476024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"261'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.047029+0000","last_change":"2024-04-22T20:05:17.891585+0000","last_active":"2024-04-22T20:45:50.047029+0000","last_peered":"2024-04-22T20:45:50.047029+0000","last_clean":"2024-04-22T20:45:50.047029+0000","last_became_active":"2024-04-22T20:05:17.891331+0000","last_became_peered":"2024-04-22T20:05:17.891331+0000","last_unstale":"2024-04-22T20:45:50.047029+0000","last_undegraded":"2024-04-22T20:45:50.047029+0000","last_fullsized":"2024-04-22T20:45:50.047029+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:10.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":14,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.173996+0000","last_change":"2024-04-22T20:05:13.814072+0000","last_active":"2024-04-22T20:44:25.173996+0000","last_peered":"2024-04-22T20:44:25.173996+0000","last_clean":"2024-04-22T20:44:25.173996+0000","last_became_active":"2024-04-22T20:05:13.813811+0000","last_became_peered":"2024-04-22T20:05:13.813811+0000","last_unstale":"2024-04-22T20:44:25.173996+0000","last_undegraded":"2024-04-22T20:44:25.173996+0000","last_fullsized":"2024-04-22T20:44:25.173996+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:57.039895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"258'49","reported_seq":170,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.148654+0000","last_change":"2024-04-22T20:01:04.874422+0000","last_active":"2024-04-22T20:44:25.148654+0000","last_peered":"2024-04-22T20:44:25.148654+0000","last_clean":"2024-04-22T20:44:25.148654+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:44:25.148654+0000","last_undegraded":"2024-04-22T20:44:25.148654+0000","last_fullsized":"2024-04-22T20:44:25.148654+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:56:32.298685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":145,"num_write":78,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.171678+0000","last_change":"2024-04-22T20:19:16.793813+0000","last_active":"2024-04-22T20:44:05.171678+0000","last_peered":"2024-04-22T20:44:05.171678+0000","last_clean":"2024-04-22T20:44:05.171678+0000","last_became_active":"2024-04-22T20:19:16.793486+0000","last_became_peered":"2024-04-22T20:19:16.793486+0000","last_unstale":"2024-04-22T20:44:05.171678+0000","last_undegraded":"2024-04-22T20:44:05.171678+0000","last_fullsized":"2024-04-22T20:44:05.171678+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:47:12.878940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:30.054561+0000","last_change":"2024-04-22T20:05:17.897286+0000","last_active":"2024-04-22T20:44:30.054561+0000","last_peered":"2024-04-22T20:44:30.054561+0000","last_clean":"2024-04-22T20:44:30.054561+0000","last_became_active":"2024-04-22T20:05:17.897008+0000","last_became_peered":"2024-04-22T20:05:17.897008+0000","last_unstale":"2024-04-22T20:44:30.054561+0000","last_undegraded":"2024-04-22T20:44:30.054561+0000","last_fullsized":"2024-04-22T20:44:30.054561+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:56:58.848648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"261'11","reported_seq":72,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182061+0000","last_change":"2024-04-22T20:05:13.798707+0000","last_active":"2024-04-22T20:45:45.182061+0000","last_peered":"2024-04-22T20:45:45.182061+0000","last_clean":"2024-04-22T20:45:45.182061+0000","last_became_active":"2024-04-22T20:05:13.798319+0000","last_became_peered":"2024-04-22T20:05:13.798319+0000","last_unstale":"2024-04-22T20:45:45.182061+0000","last_undegraded":"2024-04-22T20:45:45.182061+0000","last_fullsized":"2024-04-22T20:45:45.182061+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:02.215898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":20,"num_read_kb":16388,"num_write":14,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"261'37","reported_seq":138,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.306289+0000","last_change":"2024-04-22T20:01:04.869368+0000","last_active":"2024-04-22T20:44:25.306289+0000","last_peered":"2024-04-22T20:44:25.306289+0000","last_clean":"2024-04-22T20:44:25.306289+0000","last_became_active":"2024-04-22T20:01:04.869079+0000","last_became_peered":"2024-04-22T20:01:04.869079+0000","last_unstale":"2024-04-22T20:44:25.306289+0000","last_undegraded":"2024-04-22T20:44:25.306289+0000","last_fullsized":"2024-04-22T20:44:25.306289+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:36:35.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":61,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053026+0000","last_change":"2024-04-22T20:19:16.796317+0000","last_active":"2024-04-22T20:44:20.053026+0000","last_peered":"2024-04-22T20:44:20.053026+0000","last_clean":"2024-04-22T20:44:20.053026+0000","last_became_active":"2024-04-22T20:19:16.795462+0000","last_became_peered":"2024-04-22T20:19:16.795462+0000","last_unstale":"2024-04-22T20:44:20.053026+0000","last_undegraded":"2024-04-22T20:44:20.053026+0000","last_fullsized":"2024-04-22T20:44:20.053026+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:38:48.277835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"258'6","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.178822+0000","last_change":"2024-04-22T20:05:17.892351+0000","last_active":"2024-04-22T20:45:05.178822+0000","last_peered":"2024-04-22T20:45:05.178822+0000","last_clean":"2024-04-22T20:45:05.178822+0000","last_became_active":"2024-04-22T20:05:17.892044+0000","last_became_peered":"2024-04-22T20:05:17.892044+0000","last_unstale":"2024-04-22T20:45:05.178822+0000","last_undegraded":"2024-04-22T20:45:05.178822+0000","last_fullsized":"2024-04-22T20:45:05.178822+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:47:37.172131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"258'4","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057775+0000","last_change":"2024-04-22T20:05:13.809668+0000","last_active":"2024-04-22T20:44:50.057775+0000","last_peered":"2024-04-22T20:44:50.057775+0000","last_clean":"2024-04-22T20:44:50.057775+0000","last_became_active":"2024-04-22T20:05:13.809368+0000","last_became_peered":"2024-04-22T20:05:13.809368+0000","last_unstale":"2024-04-22T20:44:50.057775+0000","last_undegraded":"2024-04-22T20:44:50.057775+0000","last_fullsized":"2024-04-22T20:44:50.057775+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:35.986772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"261'35","reported_seq":138,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.174074+0000","last_change":"2024-04-22T20:01:04.874032+0000","last_active":"2024-04-22T20:44:25.174074+0000","last_peered":"2024-04-22T20:44:25.174074+0000","last_clean":"2024-04-22T20:44:25.174074+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:44:25.174074+0000","last_undegraded":"2024-04-22T20:44:25.174074+0000","last_fullsized":"2024-04-22T20:44:25.174074+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:44.638764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":59,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.177834+0000","last_change":"2024-04-22T20:19:16.791940+0000","last_active":"2024-04-22T20:44:20.177834+0000","last_peered":"2024-04-22T20:44:20.177834+0000","last_clean":"2024-04-22T20:44:20.177834+0000","last_became_active":"2024-04-22T20:19:16.791663+0000","last_became_peered":"2024-04-22T20:19:16.791663+0000","last_unstale":"2024-04-22T20:44:20.177834+0000","last_undegraded":"2024-04-22T20:44:20.177834+0000","last_fullsized":"2024-04-22T20:44:20.177834+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:35:00.807867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"261'5","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.177498+0000","last_change":"2024-04-22T20:05:18.211014+0000","last_active":"2024-04-22T20:44:55.177498+0000","last_peered":"2024-04-22T20:44:55.177498+0000","last_clean":"2024-04-22T20:44:55.177498+0000","last_became_active":"2024-04-22T20:05:18.210736+0000","last_became_peered":"2024-04-22T20:05:18.210736+0000","last_unstale":"2024-04-22T20:44:55.177498+0000","last_undegraded":"2024-04-22T20:44:55.177498+0000","last_fullsized":"2024-04-22T20:44:55.177498+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:55:21.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":20482,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"261'5","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.064529+0000","last_change":"2024-04-22T20:05:13.813554+0000","last_active":"2024-04-22T20:45:00.064529+0000","last_peered":"2024-04-22T20:45:00.064529+0000","last_clean":"2024-04-22T20:45:00.064529+0000","last_became_active":"2024-04-22T20:05:13.813277+0000","last_became_peered":"2024-04-22T20:05:13.813277+0000","last_unstale":"2024-04-22T20:45:00.064529+0000","last_undegraded":"2024-04-22T20:45:00.064529+0000","last_fullsized":"2024-04-22T20:45:00.064529+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:43:53.225799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":15,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"261'61","reported_seq":194,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.306363+0000","last_change":"2024-04-22T20:01:04.872160+0000","last_active":"2024-04-22T20:44:25.306363+0000","last_peered":"2024-04-22T20:44:25.306363+0000","last_clean":"2024-04-22T20:44:25.306363+0000","last_became_active":"2024-04-22T20:01:04.871859+0000","last_became_peered":"2024-04-22T20:01:04.871859+0000","last_unstale":"2024-04-22T20:44:25.306363+0000","last_undegraded":"2024-04-22T20:44:25.306363+0000","last_fullsized":"2024-04-22T20:44:25.306363+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:17.093967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":172,"num_write":109,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.044253+0000","last_change":"2024-04-22T20:19:16.790599+0000","last_active":"2024-04-22T20:45:25.044253+0000","last_peered":"2024-04-22T20:45:25.044253+0000","last_clean":"2024-04-22T20:45:25.044253+0000","last_became_active":"2024-04-22T20:19:16.789688+0000","last_became_peered":"2024-04-22T20:19:16.789688+0000","last_unstale":"2024-04-22T20:45:25.044253+0000","last_undegraded":"2024-04-22T20:45:25.044253+0000","last_fullsized":"2024-04-22T20:45:25.044253+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:11.661623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"258'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053135+0000","last_change":"2024-04-22T20:05:18.211736+0000","last_active":"2024-04-22T20:44:20.053135+0000","last_peered":"2024-04-22T20:44:20.053135+0000","last_clean":"2024-04-22T20:44:20.053135+0000","last_became_active":"2024-04-22T20:05:18.211478+0000","last_became_peered":"2024-04-22T20:05:18.211478+0000","last_unstale":"2024-04-22T20:44:20.053135+0000","last_undegraded":"2024-04-22T20:44:20.053135+0000","last_fullsized":"2024-04-22T20:44:20.053135+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:06:10.866642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.177134+0000","last_change":"2024-04-22T20:05:13.811784+0000","last_active":"2024-04-22T20:44:15.177134+0000","last_peered":"2024-04-22T20:44:15.177134+0000","last_clean":"2024-04-22T20:44:15.177134+0000","last_became_active":"2024-04-22T20:05:13.811561+0000","last_became_peered":"2024-04-22T20:05:13.811561+0000","last_unstale":"2024-04-22T20:44:15.177134+0000","last_undegraded":"2024-04-22T20:44:15.177134+0000","last_fullsized":"2024-04-22T20:44:15.177134+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:35:04.990469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":9,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"261'33","reported_seq":135,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.059415+0000","last_change":"2024-04-22T20:01:04.875229+0000","last_active":"2024-04-22T20:45:00.059415+0000","last_peered":"2024-04-22T20:45:00.059415+0000","last_clean":"2024-04-22T20:45:00.059415+0000","last_became_active":"2024-04-22T20:01:04.874790+0000","last_became_peered":"2024-04-22T20:01:04.874790+0000","last_unstale":"2024-04-22T20:45:00.059415+0000","last_undegraded":"2024-04-22T20:45:00.059415+0000","last_fullsized":"2024-04-22T20:45:00.059415+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:48.031491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":100,"num_write":57,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169578+0000","last_change":"2024-04-22T20:19:16.794065+0000","last_active":"2024-04-22T20:44:20.169578+0000","last_peered":"2024-04-22T20:44:20.169578+0000","last_clean":"2024-04-22T20:44:20.169578+0000","last_became_active":"2024-04-22T20:19:16.793188+0000","last_became_peered":"2024-04-22T20:19:16.793188+0000","last_unstale":"2024-04-22T20:44:20.169578+0000","last_undegraded":"2024-04-22T20:44:20.169578+0000","last_fullsized":"2024-04-22T20:44:20.169578+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:39.203315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"261'3","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169675+0000","last_change":"2024-04-22T20:05:17.901392+0000","last_active":"2024-04-22T20:44:20.169675+0000","last_peered":"2024-04-22T20:44:20.169675+0000","last_clean":"2024-04-22T20:44:20.169675+0000","last_became_active":"2024-04-22T20:05:17.901128+0000","last_became_peered":"2024-04-22T20:05:17.901128+0000","last_unstale":"2024-04-22T20:44:20.169675+0000","last_undegraded":"2024-04-22T20:44:20.169675+0000","last_fullsized":"2024-04-22T20:44:20.169675+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:50:08.370833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":8,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"261'4","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.052367+0000","last_change":"2024-04-22T20:05:13.815731+0000","last_active":"2024-04-22T20:44:15.052367+0000","last_peered":"2024-04-22T20:44:15.052367+0000","last_clean":"2024-04-22T20:44:15.052367+0000","last_became_active":"2024-04-22T20:05:13.815332+0000","last_became_peered":"2024-04-22T20:05:13.815332+0000","last_unstale":"2024-04-22T20:44:15.052367+0000","last_undegraded":"2024-04-22T20:44:15.052367+0000","last_fullsized":"2024-04-22T20:44:15.052367+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:07:16.209328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":9216,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'28","reported_seq":124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040298+0000","last_change":"2024-04-22T20:01:04.872148+0000","last_active":"2024-04-22T20:44:50.040298+0000","last_peered":"2024-04-22T20:44:50.040298+0000","last_clean":"2024-04-22T20:44:50.040298+0000","last_became_active":"2024-04-22T20:01:04.871840+0000","last_became_peered":"2024-04-22T20:01:04.871840+0000","last_unstale":"2024-04-22T20:44:50.040298+0000","last_undegraded":"2024-04-22T20:44:50.040298+0000","last_fullsized":"2024-04-22T20:44:50.040298+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:23.933273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":89,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"261'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.066551+0000","last_change":"2024-04-22T20:19:16.794411+0000","last_active":"2024-04-22T20:45:20.066551+0000","last_peered":"2024-04-22T20:45:20.066551+0000","last_clean":"2024-04-22T20:45:20.066551+0000","last_became_active":"2024-04-22T20:19:16.794123+0000","last_became_peered":"2024-04-22T20:19:16.794123+0000","last_unstale":"2024-04-22T20:45:20.066551+0000","last_undegraded":"2024-04-22T20:45:20.066551+0000","last_fullsized":"2024-04-22T20:45:20.066551+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:39.176302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"261'2","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.040820+0000","last_change":"2024-04-22T20:05:17.895539+0000","last_active":"2024-04-22T20:44:55.040820+0000","last_peered":"2024-04-22T20:44:55.040820+0000","last_clean":"2024-04-22T20:44:55.040820+0000","last_became_active":"2024-04-22T20:05:17.895177+0000","last_became_peered":"2024-04-22T20:05:17.895177+0000","last_unstale":"2024-04-22T20:44:55.040820+0000","last_undegraded":"2024-04-22T20:44:55.040820+0000","last_fullsized":"2024-04-22T20:44:55.040820+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:46:34.641098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1024,"num_write":2,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"261'2","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063150+0000","last_change":"2024-04-22T20:05:13.809357+0000","last_active":"2024-04-22T20:45:25.063150+0000","last_peered":"2024-04-22T20:45:25.063150+0000","last_clean":"2024-04-22T20:45:25.063150+0000","last_became_active":"2024-04-22T20:05:13.809047+0000","last_became_peered":"2024-04-22T20:05:13.809047+0000","last_unstale":"2024-04-22T20:45:25.063150+0000","last_undegraded":"2024-04-22T20:45:25.063150+0000","last_fullsized":"2024-04-22T20:45:25.063150+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:17.206006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":2,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"261'33","reported_seq":130,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.181926+0000","last_change":"2024-04-22T20:01:04.872953+0000","last_active":"2024-04-22T20:44:50.181926+0000","last_peered":"2024-04-22T20:44:50.181926+0000","last_clean":"2024-04-22T20:44:50.181926+0000","last_became_active":"2024-04-22T20:01:04.872370+0000","last_became_peered":"2024-04-22T20:01:04.872370+0000","last_unstale":"2024-04-22T20:44:50.181926+0000","last_undegraded":"2024-04-22T20:44:50.181926+0000","last_fullsized":"2024-04-22T20:44:50.181926+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:22:37.771203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":57,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.180655+0000","last_change":"2024-04-22T20:19:16.793728+0000","last_active":"2024-04-22T20:45:20.180655+0000","last_peered":"2024-04-22T20:45:20.180655+0000","last_clean":"2024-04-22T20:45:20.180655+0000","last_became_active":"2024-04-22T20:19:16.793429+0000","last_became_peered":"2024-04-22T20:19:16.793429+0000","last_unstale":"2024-04-22T20:45:20.180655+0000","last_undegraded":"2024-04-22T20:45:20.180655+0000","last_fullsized":"2024-04-22T20:45:20.180655+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:27:12.156563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"261'1","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.187790+0000","last_change":"2024-04-22T20:05:17.901476+0000","last_active":"2024-04-22T20:45:30.187790+0000","last_peered":"2024-04-22T20:45:30.187790+0000","last_clean":"2024-04-22T20:45:30.187790+0000","last_became_active":"2024-04-22T20:05:17.900874+0000","last_became_peered":"2024-04-22T20:05:17.900874+0000","last_unstale":"2024-04-22T20:45:30.187790+0000","last_undegraded":"2024-04-22T20:45:30.187790+0000","last_fullsized":"2024-04-22T20:45:30.187790+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:20:07.700066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057705+0000","last_change":"2024-04-22T20:05:13.814214+0000","last_active":"2024-04-22T20:44:50.057705+0000","last_peered":"2024-04-22T20:44:50.057705+0000","last_clean":"2024-04-22T20:44:50.057705+0000","last_became_active":"2024-04-22T20:05:13.813934+0000","last_became_peered":"2024-04-22T20:05:13.813934+0000","last_unstale":"2024-04-22T20:44:50.057705+0000","last_undegraded":"2024-04-22T20:44:50.057705+0000","last_fullsized":"2024-04-22T20:44:50.057705+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:15.355174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1024,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"261'37","reported_seq":146,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.173934+0000","last_change":"2024-04-22T20:01:04.869971+0000","last_active":"2024-04-22T20:44:50.173934+0000","last_peered":"2024-04-22T20:44:50.173934+0000","last_clean":"2024-04-22T20:44:50.173934+0000","last_became_active":"2024-04-22T20:01:04.869660+0000","last_became_peered":"2024-04-22T20:01:04.869660+0000","last_unstale":"2024-04-22T20:44:50.173934+0000","last_undegraded":"2024-04-22T20:44:50.173934+0000","last_fullsized":"2024-04-22T20:44:50.173934+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:08:02.708029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":113,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.173415+0000","last_change":"2024-04-22T20:19:16.795977+0000","last_active":"2024-04-22T20:44:20.173415+0000","last_peered":"2024-04-22T20:44:20.173415+0000","last_clean":"2024-04-22T20:44:20.173415+0000","last_became_active":"2024-04-22T20:19:16.795509+0000","last_became_peered":"2024-04-22T20:19:16.795509+0000","last_unstale":"2024-04-22T20:44:20.173415+0000","last_undegraded":"2024-04-22T20:44:20.173415+0000","last_fullsized":"2024-04-22T20:44:20.173415+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:07.456021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.310587+0000","last_change":"2024-04-22T20:05:17.891942+0000","last_active":"2024-04-22T20:44:55.310587+0000","last_peered":"2024-04-22T20:44:55.310587+0000","last_clean":"2024-04-22T20:44:55.310587+0000","last_became_active":"2024-04-22T20:05:17.891523+0000","last_became_peered":"2024-04-22T20:05:17.891523+0000","last_unstale":"2024-04-22T20:44:55.310587+0000","last_undegraded":"2024-04-22T20:44:55.310587+0000","last_fullsized":"2024-04-22T20:44:55.310587+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:38.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040268+0000","last_change":"2024-04-22T20:05:13.805972+0000","last_active":"2024-04-22T20:44:50.040268+0000","last_peered":"2024-04-22T20:44:50.040268+0000","last_clean":"2024-04-22T20:44:50.040268+0000","last_became_active":"2024-04-22T20:05:13.805704+0000","last_became_peered":"2024-04-22T20:05:13.805704+0000","last_unstale":"2024-04-22T20:44:50.040268+0000","last_undegraded":"2024-04-22T20:44:50.040268+0000","last_fullsized":"2024-04-22T20:44:50.040268+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:03.816391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"261'65","reported_seq":224,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309884+0000","last_change":"2024-04-22T20:01:04.866275+0000","last_active":"2024-04-22T20:44:50.309884+0000","last_peered":"2024-04-22T20:44:50.309884+0000","last_clean":"2024-04-22T20:44:50.309884+0000","last_became_active":"2024-04-22T20:01:04.865981+0000","last_became_peered":"2024-04-22T20:01:04.865981+0000","last_unstale":"2024-04-22T20:44:50.309884+0000","last_undegraded":"2024-04-22T20:44:50.309884+0000","last_fullsized":"2024-04-22T20:44:50.309884+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:25.969487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":226,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:30.054611+0000","last_change":"2024-04-22T20:19:16.799325+0000","last_active":"2024-04-22T20:44:30.054611+0000","last_peered":"2024-04-22T20:44:30.054611+0000","last_clean":"2024-04-22T20:44:30.054611+0000","last_became_active":"2024-04-22T20:19:16.799063+0000","last_became_peered":"2024-04-22T20:19:16.799063+0000","last_unstale":"2024-04-22T20:44:30.054611+0000","last_undegraded":"2024-04-22T20:44:30.054611+0000","last_fullsized":"2024-04-22T20:44:30.054611+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:20:02.254828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"261'1","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.174634+0000","last_change":"2024-04-22T20:05:17.901995+0000","last_active":"2024-04-22T20:44:55.174634+0000","last_peered":"2024-04-22T20:44:55.174634+0000","last_clean":"2024-04-22T20:44:55.174634+0000","last_became_active":"2024-04-22T20:05:17.901680+0000","last_became_peered":"2024-04-22T20:05:17.901680+0000","last_unstale":"2024-04-22T20:44:55.174634+0000","last_undegraded":"2024-04-22T20:44:55.174634+0000","last_fullsized":"2024-04-22T20:44:55.174634+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:26.628494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40145,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":80,"num_write":1,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309933+0000","last_change":"2024-04-22T20:05:13.811164+0000","last_active":"2024-04-22T20:44:50.309933+0000","last_peered":"2024-04-22T20:44:50.309933+0000","last_clean":"2024-04-22T20:44:50.309933+0000","last_became_active":"2024-04-22T20:05:13.810865+0000","last_became_peered":"2024-04-22T20:05:13.810865+0000","last_unstale":"2024-04-22T20:44:50.309933+0000","last_undegraded":"2024-04-22T20:44:50.309933+0000","last_fullsized":"2024-04-22T20:44:50.309933+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:47:15.297255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"261'57","reported_seq":180,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.061413+0000","last_change":"2024-04-22T20:01:04.870803+0000","last_active":"2024-04-22T20:45:15.061413+0000","last_peered":"2024-04-22T20:45:15.061413+0000","last_clean":"2024-04-22T20:45:15.061413+0000","last_became_active":"2024-04-22T20:01:04.870253+0000","last_became_peered":"2024-04-22T20:01:04.870253+0000","last_unstale":"2024-04-22T20:45:15.061413+0000","last_undegraded":"2024-04-22T20:45:15.061413+0000","last_fullsized":"2024-04-22T20:45:15.061413+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:32:34.069500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":145,"num_write":102,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"261'281","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.065532+0000","last_change":"2024-04-22T19:53:04.823797+0000","last_active":"2024-04-22T20:45:10.065532+0000","last_peered":"2024-04-22T20:45:10.065532+0000","last_clean":"2024-04-22T20:45:10.065532+0000","last_became_active":"2024-04-22T19:53:04.823351+0000","last_became_peered":"2024-04-22T19:53:04.823351+0000","last_unstale":"2024-04-22T20:45:10.065532+0000","last_undegraded":"2024-04-22T20:45:10.065532+0000","last_fullsized":"2024-04-22T20:45:10.065532+0000","mapping_epoch":35,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:24.940347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9460561,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":447,"num_read_kb":495,"num_write":1235,"num_write_kb":39497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"261'169","reported_seq":763,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174873+0000","last_change":"2024-04-22T19:51:28.511862+0000","last_active":"2024-04-22T20:44:00.174873+0000","last_peered":"2024-04-22T20:44:00.174873+0000","last_clean":"2024-04-22T20:44:00.174873+0000","last_became_active":"2024-04-22T19:51:28.511595+0000","last_became_peered":"2024-04-22T19:51:28.511595+0000","last_unstale":"2024-04-22T20:44:00.174873+0000","last_undegraded":"2024-04-22T20:44:00.174873+0000","last_fullsized":"2024-04-22T20:44:00.174873+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:36.200495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":537,"num_write":262,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"261'248","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:48.785097+0000","last_change":"2024-04-22T19:51:26.496852+0000","last_active":"2024-04-22T20:46:48.785097+0000","last_peered":"2024-04-22T20:46:48.785097+0000","last_clean":"2024-04-22T20:46:48.785097+0000","last_became_active":"2024-04-22T19:51:26.496188+0000","last_became_peered":"2024-04-22T19:51:26.496188+0000","last_unstale":"2024-04-22T20:46:48.785097+0000","last_undegraded":"2024-04-22T20:46:48.785097+0000","last_fullsized":"2024-04-22T20:46:48.785097+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:51:56.333450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10265266,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":429,"num_read_kb":9758,"num_write":1044,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"261'314","reported_seq":886,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:28.101937+0000","last_change":"2024-04-22T19:51:28.519482+0000","last_active":"2024-04-22T20:47:28.101937+0000","last_peered":"2024-04-22T20:47:28.101937+0000","last_clean":"2024-04-22T20:47:28.101937+0000","last_became_active":"2024-04-22T19:51:28.519111+0000","last_became_peered":"2024-04-22T19:51:28.519111+0000","last_unstale":"2024-04-22T20:47:28.101937+0000","last_undegraded":"2024-04-22T20:47:28.101937+0000","last_fullsized":"2024-04-22T20:47:28.101937+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:06:08.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1190,"num_read_kb":913,"num_write":510,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"261'247","reported_seq":868,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:36.787572+0000","last_change":"2024-04-22T19:51:26.503143+0000","last_active":"2024-04-22T20:46:36.787572+0000","last_peered":"2024-04-22T20:46:36.787572+0000","last_clean":"2024-04-22T20:46:36.787572+0000","last_became_active":"2024-04-22T19:51:26.502744+0000","last_became_peered":"2024-04-22T19:51:26.502744+0000","last_unstale":"2024-04-22T20:46:36.787572+0000","last_undegraded":"2024-04-22T20:46:36.787572+0000","last_fullsized":"2024-04-22T20:46:36.787572+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:30:34.457597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9763357,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":370,"num_read_kb":7116,"num_write":962,"num_write_kb":21496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"261'150","reported_seq":726,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166689+0000","last_change":"2024-04-22T19:51:28.518949+0000","last_active":"2024-04-22T20:44:00.166689+0000","last_peered":"2024-04-22T20:44:00.166689+0000","last_clean":"2024-04-22T20:44:00.166689+0000","last_became_active":"2024-04-22T19:51:28.518650+0000","last_became_peered":"2024-04-22T19:51:28.518650+0000","last_unstale":"2024-04-22T20:44:00.166689+0000","last_undegraded":"2024-04-22T20:44:00.166689+0000","last_fullsized":"2024-04-22T20:44:00.166689+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:25:03.391586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":572,"num_read_kb":453,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"261'259","reported_seq":874,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:14.145384+0000","last_change":"2024-04-22T19:51:26.504934+0000","last_active":"2024-04-22T20:47:14.145384+0000","last_peered":"2024-04-22T20:47:14.145384+0000","last_clean":"2024-04-22T20:47:14.145384+0000","last_became_active":"2024-04-22T19:51:26.504450+0000","last_became_peered":"2024-04-22T19:51:26.504450+0000","last_unstale":"2024-04-22T20:47:14.145384+0000","last_undegraded":"2024-04-22T20:47:14.145384+0000","last_fullsized":"2024-04-22T20:47:14.145384+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:47:54.670392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20493175,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":359,"num_read_kb":25508,"num_write":1125,"num_write_kb":37655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"261'81","reported_seq":676,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.171137+0000","last_change":"2024-04-22T19:52:04.620094+0000","last_active":"2024-04-22T20:44:00.171137+0000","last_peered":"2024-04-22T20:44:00.171137+0000","last_clean":"2024-04-22T20:44:00.171137+0000","last_became_active":"2024-04-22T19:52:04.619688+0000","last_became_peered":"2024-04-22T19:52:04.619688+0000","last_unstale":"2024-04-22T20:44:00.171137+0000","last_undegraded":"2024-04-22T20:44:00.171137+0000","last_fullsized":"2024-04-22T20:44:00.171137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:50:37.537623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":353,"num_read_kb":299,"num_write":120,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"261'222","reported_seq":910,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:03.977215+0000","last_change":"2024-04-22T19:51:26.509141+0000","last_active":"2024-04-22T20:47:03.977215+0000","last_peered":"2024-04-22T20:47:03.977215+0000","last_clean":"2024-04-22T20:47:03.977215+0000","last_became_active":"2024-04-22T19:51:26.507185+0000","last_became_peered":"2024-04-22T19:51:26.507185+0000","last_unstale":"2024-04-22T20:47:03.977215+0000","last_undegraded":"2024-04-22T20:47:03.977215+0000","last_fullsized":"2024-04-22T20:47:03.977215+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:14.928877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563190,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":383,"num_read_kb":10200,"num_write":866,"num_write_kb":30630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"261'40","reported_seq":594,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.316183+0000","last_change":"2024-04-22T19:51:28.513981+0000","last_active":"2024-04-22T20:45:35.316183+0000","last_peered":"2024-04-22T20:45:35.316183+0000","last_clean":"2024-04-22T20:45:35.316183+0000","last_became_active":"2024-04-22T19:51:28.513693+0000","last_became_peered":"2024-04-22T19:51:28.513693+0000","last_unstale":"2024-04-22T20:45:35.316183+0000","last_undegraded":"2024-04-22T20:45:35.316183+0000","last_fullsized":"2024-04-22T20:45:35.316183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:54:32.701108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"261'389","reported_seq":1131,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:54.327300+0000","last_change":"2024-04-22T19:51:26.509287+0000","last_active":"2024-04-22T20:46:54.327300+0000","last_peered":"2024-04-22T20:46:54.327300+0000","last_clean":"2024-04-22T20:46:54.327300+0000","last_became_active":"2024-04-22T19:51:26.508716+0000","last_became_peered":"2024-04-22T19:51:26.508716+0000","last_unstale":"2024-04-22T20:46:54.327300+0000","last_undegraded":"2024-04-22T20:46:54.327300+0000","last_fullsized":"2024-04-22T20:46:54.327300+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:42:06.130701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11174792,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":980,"num_read_kb":17244,"num_write":1468,"num_write_kb":25883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"261'144","reported_seq":742,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181403+0000","last_change":"2024-04-22T19:51:28.519053+0000","last_active":"2024-04-22T20:45:40.181403+0000","last_peered":"2024-04-22T20:45:40.181403+0000","last_clean":"2024-04-22T20:45:40.181403+0000","last_became_active":"2024-04-22T19:51:28.518501+0000","last_became_peered":"2024-04-22T19:51:28.518501+0000","last_unstale":"2024-04-22T20:45:40.181403+0000","last_undegraded":"2024-04-22T20:45:40.181403+0000","last_fullsized":"2024-04-22T20:45:40.181403+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:36.369584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":597,"num_read_kb":478,"num_write":226,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"261'300","reported_seq":961,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.049692+0000","last_change":"2024-04-22T19:51:26.508109+0000","last_active":"2024-04-22T20:44:00.049692+0000","last_peered":"2024-04-22T20:44:00.049692+0000","last_clean":"2024-04-22T20:44:00.049692+0000","last_became_active":"2024-04-22T19:51:26.507141+0000","last_became_peered":"2024-04-22T19:51:26.507141+0000","last_unstale":"2024-04-22T20:44:00.049692+0000","last_undegraded":"2024-04-22T20:44:00.049692+0000","last_fullsized":"2024-04-22T20:44:00.049692+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:27.201168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11136278,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":473,"num_read_kb":43743,"num_write":1312,"num_write_kb":32515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"261'151","reported_seq":736,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.067532+0000","last_change":"2024-04-22T19:51:28.516437+0000","last_active":"2024-04-22T20:47:38.067532+0000","last_peered":"2024-04-22T20:47:38.067532+0000","last_clean":"2024-04-22T20:47:38.067532+0000","last_became_active":"2024-04-22T19:51:28.516012+0000","last_became_peered":"2024-04-22T19:51:28.516012+0000","last_unstale":"2024-04-22T20:47:38.067532+0000","last_undegraded":"2024-04-22T20:47:38.067532+0000","last_fullsized":"2024-04-22T20:47:38.067532+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:38.182307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":470,"num_write":219,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"261'268","reported_seq":912,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:28.486641+0000","last_change":"2024-04-22T19:51:26.505832+0000","last_active":"2024-04-22T20:46:28.486641+0000","last_peered":"2024-04-22T20:46:28.486641+0000","last_clean":"2024-04-22T20:46:28.486641+0000","last_became_active":"2024-04-22T19:51:26.502928+0000","last_became_peered":"2024-04-22T19:51:26.502928+0000","last_unstale":"2024-04-22T20:46:28.486641+0000","last_undegraded":"2024-04-22T20:46:28.486641+0000","last_fullsized":"2024-04-22T20:46:28.486641+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:47:30.201226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9686578,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":378,"num_read_kb":51847,"num_write":1138,"num_write_kb":39313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"261'60","reported_seq":629,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:57.506260+0000","last_change":"2024-04-22T19:51:28.510044+0000","last_active":"2024-04-22T20:46:57.506260+0000","last_peered":"2024-04-22T20:46:57.506260+0000","last_clean":"2024-04-22T20:46:57.506260+0000","last_became_active":"2024-04-22T19:51:28.509774+0000","last_became_peered":"2024-04-22T19:51:28.509774+0000","last_unstale":"2024-04-22T20:46:57.506260+0000","last_undegraded":"2024-04-22T20:46:57.506260+0000","last_fullsized":"2024-04-22T20:46:57.506260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:55:37.172957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":211,"num_write":81,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"261'125","reported_seq":734,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176694+0000","last_change":"2024-04-22T19:51:28.505929+0000","last_active":"2024-04-22T20:45:10.176694+0000","last_peered":"2024-04-22T20:45:10.176694+0000","last_clean":"2024-04-22T20:45:10.176694+0000","last_became_active":"2024-04-22T19:51:28.505562+0000","last_became_peered":"2024-04-22T19:51:28.505562+0000","last_unstale":"2024-04-22T20:45:10.176694+0000","last_undegraded":"2024-04-22T20:45:10.176694+0000","last_fullsized":"2024-04-22T20:45:10.176694+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:33:32.464020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":540,"num_read_kb":443,"num_write":193,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"261'253","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057867+0000","last_change":"2024-04-22T19:51:26.507324+0000","last_active":"2024-04-22T20:44:50.057867+0000","last_peered":"2024-04-22T20:44:50.057867+0000","last_clean":"2024-04-22T20:44:50.057867+0000","last_became_active":"2024-04-22T19:51:26.504717+0000","last_became_peered":"2024-04-22T19:51:26.504717+0000","last_unstale":"2024-04-22T20:44:50.057867+0000","last_undegraded":"2024-04-22T20:44:50.057867+0000","last_fullsized":"2024-04-22T20:44:50.057867+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:31:10.322998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6322974,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":420,"num_read_kb":28705,"num_write":1056,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"261'105","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060776+0000","last_change":"2024-04-22T19:51:28.519503+0000","last_active":"2024-04-22T20:45:10.060776+0000","last_peered":"2024-04-22T20:45:10.060776+0000","last_clean":"2024-04-22T20:45:10.060776+0000","last_became_active":"2024-04-22T19:51:28.519230+0000","last_became_peered":"2024-04-22T19:51:28.519230+0000","last_unstale":"2024-04-22T20:45:10.060776+0000","last_undegraded":"2024-04-22T20:45:10.060776+0000","last_fullsized":"2024-04-22T20:45:10.060776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:23:00.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":465,"num_read_kb":383,"num_write":180,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"261'321","reported_seq":978,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.308459+0000","last_change":"2024-04-22T19:51:26.496564+0000","last_active":"2024-04-22T20:44:40.308459+0000","last_peered":"2024-04-22T20:44:40.308459+0000","last_clean":"2024-04-22T20:44:40.308459+0000","last_became_active":"2024-04-22T19:51:26.496156+0000","last_became_peered":"2024-04-22T19:51:26.496156+0000","last_unstale":"2024-04-22T20:44:40.308459+0000","last_undegraded":"2024-04-22T20:44:40.308459+0000","last_fullsized":"2024-04-22T20:44:40.308459+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:25.947249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77420,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":551,"num_read_kb":792,"num_write":1456,"num_write_kb":25865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"261'172","reported_seq":764,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186369+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:45:20.186369+0000","last_peered":"2024-04-22T20:45:20.186369+0000","last_clean":"2024-04-22T20:45:20.186369+0000","last_became_active":"2024-04-22T19:51:28.515673+0000","last_became_peered":"2024-04-22T19:51:28.515673+0000","last_unstale":"2024-04-22T20:45:20.186369+0000","last_undegraded":"2024-04-22T20:45:20.186369+0000","last_fullsized":"2024-04-22T20:45:20.186369+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:06.001104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":666,"num_read_kb":529,"num_write":258,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"261'291","reported_seq":1001,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.308420+0000","last_change":"2024-04-22T19:51:26.502589+0000","last_active":"2024-04-22T20:44:40.308420+0000","last_peered":"2024-04-22T20:44:40.308420+0000","last_clean":"2024-04-22T20:44:40.308420+0000","last_became_active":"2024-04-22T19:51:26.502156+0000","last_became_peered":"2024-04-22T19:51:26.502156+0000","last_unstale":"2024-04-22T20:44:40.308420+0000","last_undegraded":"2024-04-22T20:44:40.308420+0000","last_fullsized":"2024-04-22T20:44:40.308420+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:22:39.535038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23449640,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":550,"num_read_kb":68133,"num_write":1243,"num_write_kb":41095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"261'290","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:09.419441+0000","last_change":"2024-04-22T19:51:28.510666+0000","last_active":"2024-04-22T20:47:09.419441+0000","last_peered":"2024-04-22T20:47:09.419441+0000","last_clean":"2024-04-22T20:47:09.419441+0000","last_became_active":"2024-04-22T19:51:28.510415+0000","last_became_peered":"2024-04-22T19:51:28.510415+0000","last_unstale":"2024-04-22T20:47:09.419441+0000","last_undegraded":"2024-04-22T20:47:09.419441+0000","last_fullsized":"2024-04-22T20:47:09.419441+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:43.003473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1119,"num_read_kb":869,"num_write":475,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"261'251","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172460+0000","last_change":"2024-04-22T19:51:26.507272+0000","last_active":"2024-04-22T20:44:40.172460+0000","last_peered":"2024-04-22T20:44:40.172460+0000","last_clean":"2024-04-22T20:44:40.172460+0000","last_became_active":"2024-04-22T19:51:26.506671+0000","last_became_peered":"2024-04-22T19:51:26.506671+0000","last_unstale":"2024-04-22T20:44:40.172460+0000","last_undegraded":"2024-04-22T20:44:40.172460+0000","last_fullsized":"2024-04-22T20:44:40.172460+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:54:08.993777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8504557,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":414,"num_read_kb":6535,"num_write":1034,"num_write_kb":36003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"261'155","reported_seq":775,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.062504+0000","last_change":"2024-04-22T19:51:28.512453+0000","last_active":"2024-04-22T20:44:40.062504+0000","last_peered":"2024-04-22T20:44:40.062504+0000","last_clean":"2024-04-22T20:44:40.062504+0000","last_became_active":"2024-04-22T19:51:28.511889+0000","last_became_peered":"2024-04-22T19:51:28.511889+0000","last_unstale":"2024-04-22T20:44:40.062504+0000","last_undegraded":"2024-04-22T20:44:40.062504+0000","last_fullsized":"2024-04-22T20:44:40.062504+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:45.195439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":679,"num_read_kb":547,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"261'265","reported_seq":912,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.157609+0000","last_change":"2024-04-22T19:51:26.498087+0000","last_active":"2024-04-22T20:47:38.157609+0000","last_peered":"2024-04-22T20:47:38.157609+0000","last_clean":"2024-04-22T20:47:38.157609+0000","last_became_active":"2024-04-22T19:51:26.497718+0000","last_became_peered":"2024-04-22T19:51:26.497718+0000","last_unstale":"2024-04-22T20:47:38.157609+0000","last_undegraded":"2024-04-22T20:47:38.157609+0000","last_fullsized":"2024-04-22T20:47:38.157609+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:50:38.709806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6162293,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":402,"num_read_kb":30110,"num_write":1095,"num_write_kb":27210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"261'93","reported_seq":699,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:44.069351+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:46:44.069351+0000","last_peered":"2024-04-22T20:46:44.069351+0000","last_clean":"2024-04-22T20:46:44.069351+0000","last_became_active":"2024-04-22T19:51:28.507673+0000","last_became_peered":"2024-04-22T19:51:28.507673+0000","last_unstale":"2024-04-22T20:46:44.069351+0000","last_undegraded":"2024-04-22T20:46:44.069351+0000","last_fullsized":"2024-04-22T20:46:44.069351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:18:40.338876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":404,"num_read_kb":350,"num_write":133,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"261'347","reported_seq":1058,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.243867+0000","last_change":"2024-04-22T19:51:26.511696+0000","last_active":"2024-04-22T20:46:30.243867+0000","last_peered":"2024-04-22T20:46:30.243867+0000","last_clean":"2024-04-22T20:46:30.243867+0000","last_became_active":"2024-04-22T19:51:26.511213+0000","last_became_peered":"2024-04-22T19:51:26.511213+0000","last_unstale":"2024-04-22T20:46:30.243867+0000","last_undegraded":"2024-04-22T20:46:30.243867+0000","last_fullsized":"2024-04-22T20:46:30.243867+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:51:44.984733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13939483,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":664,"num_read_kb":36415,"num_write":1732,"num_write_kb":40524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"261'126","reported_seq":647,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.145449+0000","last_change":"2024-04-22T19:51:36.567880+0000","last_active":"2024-04-22T20:48:04.145449+0000","last_peered":"2024-04-22T20:48:04.145449+0000","last_clean":"2024-04-22T20:48:04.145449+0000","last_became_active":"2024-04-22T19:51:36.567561+0000","last_became_peered":"2024-04-22T19:51:36.567561+0000","last_unstale":"2024-04-22T20:48:04.145449+0000","last_undegraded":"2024-04-22T20:48:04.145449+0000","last_fullsized":"2024-04-22T20:48:04.145449+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:10:43.936463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":168,"num_write":307,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"261'38","reported_seq":584,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154700+0000","last_change":"2024-04-22T19:51:28.520238+0000","last_active":"2024-04-22T20:45:15.154700+0000","last_peered":"2024-04-22T20:45:15.154700+0000","last_clean":"2024-04-22T20:45:15.154700+0000","last_became_active":"2024-04-22T19:51:28.519994+0000","last_became_peered":"2024-04-22T19:51:28.519994+0000","last_unstale":"2024-04-22T20:45:15.154700+0000","last_undegraded":"2024-04-22T20:45:15.154700+0000","last_fullsized":"2024-04-22T20:45:15.154700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:48:21.210710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":138,"num_write":52,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"261'264","reported_seq":997,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060800+0000","last_change":"2024-04-22T19:53:05.051779+0000","last_active":"2024-04-22T20:45:10.060800+0000","last_peered":"2024-04-22T20:45:10.060800+0000","last_clean":"2024-04-22T20:45:10.060800+0000","last_became_active":"2024-04-22T19:53:05.051272+0000","last_became_peered":"2024-04-22T19:53:05.051272+0000","last_unstale":"2024-04-22T20:45:10.060800+0000","last_undegraded":"2024-04-22T20:45:10.060800+0000","last_fullsized":"2024-04-22T20:45:10.060800+0000","mapping_epoch":35,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:14:57.868131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6579999,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":609,"num_read_kb":11069,"num_write":1217,"num_write_kb":33730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180678+0000","last_change":"2024-04-22T19:51:30.521121+0000","last_active":"2024-04-22T20:44:40.180678+0000","last_peered":"2024-04-22T20:44:40.180678+0000","last_clean":"2024-04-22T20:44:40.180678+0000","last_became_active":"2024-04-22T19:51:30.520848+0000","last_became_peered":"2024-04-22T19:51:30.520848+0000","last_unstale":"2024-04-22T20:44:40.180678+0000","last_undegraded":"2024-04-22T20:44:40.180678+0000","last_fullsized":"2024-04-22T20:44:40.180678+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:30.383458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"261'324","reported_seq":923,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.193392+0000","last_change":"2024-04-22T19:51:32.537967+0000","last_active":"2024-04-22T20:47:10.193392+0000","last_peered":"2024-04-22T20:47:10.193392+0000","last_clean":"2024-04-22T20:47:10.193392+0000","last_became_active":"2024-04-22T19:51:32.537695+0000","last_became_peered":"2024-04-22T19:51:32.537695+0000","last_unstale":"2024-04-22T20:47:10.193392+0000","last_undegraded":"2024-04-22T20:47:10.193392+0000","last_fullsized":"2024-04-22T20:47:10.193392+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:12:32.127700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58993,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":537,"num_read_kb":1015,"num_write":432,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.057042+0000","last_change":"2024-04-22T19:51:34.557621+0000","last_active":"2024-04-22T20:44:45.057042+0000","last_peered":"2024-04-22T20:44:45.057042+0000","last_clean":"2024-04-22T20:44:45.057042+0000","last_became_active":"2024-04-22T19:51:34.557285+0000","last_became_peered":"2024-04-22T19:51:34.557285+0000","last_unstale":"2024-04-22T20:44:45.057042+0000","last_undegraded":"2024-04-22T20:44:45.057042+0000","last_fullsized":"2024-04-22T20:44:45.057042+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:14.949781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"261'2399","reported_seq":4258,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.711091+0000","last_change":"2024-04-22T19:51:36.567463+0000","last_active":"2024-04-22T20:48:03.711091+0000","last_peered":"2024-04-22T20:48:03.711091+0000","last_clean":"2024-04-22T20:48:03.711091+0000","last_became_active":"2024-04-22T19:51:36.567195+0000","last_became_peered":"2024-04-22T19:51:36.567195+0000","last_unstale":"2024-04-22T20:48:03.711091+0000","last_undegraded":"2024-04-22T20:48:03.711091+0000","last_fullsized":"2024-04-22T20:48:03.711091+0000","mapping_epoch":30,"log_start":"261'2300","ondisk_log_start":"261'2300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":2300,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:46:00.619550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5986,"num_read_kb":5990,"num_write":4781,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"261'130","reported_seq":720,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:53.028609+0000","last_change":"2024-04-22T19:51:28.513076+0000","last_active":"2024-04-22T20:46:53.028609+0000","last_peered":"2024-04-22T20:46:53.028609+0000","last_clean":"2024-04-22T20:46:53.028609+0000","last_became_active":"2024-04-22T19:51:28.512786+0000","last_became_peered":"2024-04-22T19:51:28.512786+0000","last_unstale":"2024-04-22T20:46:53.028609+0000","last_undegraded":"2024-04-22T20:46:53.028609+0000","last_fullsized":"2024-04-22T20:46:53.028609+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:20:09.250321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":408,"num_write":191,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"261'229","reported_seq":832,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:55.640581+0000","last_change":"2024-04-22T19:51:26.500209+0000","last_active":"2024-04-22T20:46:55.640581+0000","last_peered":"2024-04-22T20:46:55.640581+0000","last_clean":"2024-04-22T20:46:55.640581+0000","last_became_active":"2024-04-22T19:51:26.499885+0000","last_became_peered":"2024-04-22T19:51:26.499885+0000","last_unstale":"2024-04-22T20:46:55.640581+0000","last_undegraded":"2024-04-22T20:46:55.640581+0000","last_fullsized":"2024-04-22T20:46:55.640581+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:06:40.641478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8391210,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":339,"num_read_kb":16305,"num_write":901,"num_write_kb":23193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154709+0000","last_change":"2024-04-22T19:51:30.527682+0000","last_active":"2024-04-22T20:45:15.154709+0000","last_peered":"2024-04-22T20:45:15.154709+0000","last_clean":"2024-04-22T20:45:15.154709+0000","last_became_active":"2024-04-22T19:51:30.526622+0000","last_became_peered":"2024-04-22T19:51:30.526622+0000","last_unstale":"2024-04-22T20:45:15.154709+0000","last_undegraded":"2024-04-22T20:45:15.154709+0000","last_fullsized":"2024-04-22T20:45:15.154709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:20:42.245730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"261'86","reported_seq":611,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.173330+0000","last_change":"2024-04-22T19:51:32.536487+0000","last_active":"2024-04-22T20:44:45.173330+0000","last_peered":"2024-04-22T20:44:45.173330+0000","last_clean":"2024-04-22T20:44:45.173330+0000","last_became_active":"2024-04-22T19:51:32.536180+0000","last_became_peered":"2024-04-22T19:51:32.536180+0000","last_unstale":"2024-04-22T20:44:45.173330+0000","last_undegraded":"2024-04-22T20:44:45.173330+0000","last_fullsized":"2024-04-22T20:44:45.173330+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:21.299001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":95,"num_write":64,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.173306+0000","last_change":"2024-04-22T19:51:34.549833+0000","last_active":"2024-04-22T20:44:45.173306+0000","last_peered":"2024-04-22T20:44:45.173306+0000","last_clean":"2024-04-22T20:44:45.173306+0000","last_became_active":"2024-04-22T19:51:34.549551+0000","last_became_peered":"2024-04-22T19:51:34.549551+0000","last_unstale":"2024-04-22T20:44:45.173306+0000","last_undegraded":"2024-04-22T20:44:45.173306+0000","last_fullsized":"2024-04-22T20:44:45.173306+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:14:36.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"261'146","reported_seq":645,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.823329+0000","last_change":"2024-04-22T19:51:36.573966+0000","last_active":"2024-04-22T20:48:00.823329+0000","last_peered":"2024-04-22T20:48:00.823329+0000","last_clean":"2024-04-22T20:48:00.823329+0000","last_became_active":"2024-04-22T19:51:36.573189+0000","last_became_peered":"2024-04-22T19:51:36.573189+0000","last_unstale":"2024-04-22T20:48:00.823329+0000","last_undegraded":"2024-04-22T20:48:00.823329+0000","last_fullsized":"2024-04-22T20:48:00.823329+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:07:48.890371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":328,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"261'212","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.175878+0000","last_change":"2024-04-22T19:51:28.518334+0000","last_active":"2024-04-22T20:44:40.175878+0000","last_peered":"2024-04-22T20:44:40.175878+0000","last_clean":"2024-04-22T20:44:40.175878+0000","last_became_active":"2024-04-22T19:51:28.517737+0000","last_became_peered":"2024-04-22T19:51:28.517737+0000","last_unstale":"2024-04-22T20:44:40.175878+0000","last_undegraded":"2024-04-22T20:44:40.175878+0000","last_fullsized":"2024-04-22T20:44:40.175878+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:47.145426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":871,"num_read_kb":733,"num_write":338,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"261'241","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:59.594399+0000","last_change":"2024-04-22T19:51:26.509257+0000","last_active":"2024-04-22T20:46:59.594399+0000","last_peered":"2024-04-22T20:46:59.594399+0000","last_clean":"2024-04-22T20:46:59.594399+0000","last_became_active":"2024-04-22T19:51:26.508933+0000","last_became_peered":"2024-04-22T19:51:26.508933+0000","last_unstale":"2024-04-22T20:46:59.594399+0000","last_undegraded":"2024-04-22T20:46:59.594399+0000","last_fullsized":"2024-04-22T20:46:59.594399+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:00:26.499126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8849792,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":456,"num_read_kb":15260,"num_write":1066,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":502,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.150710+0000","last_change":"2024-04-22T19:51:30.528152+0000","last_active":"2024-04-22T20:44:40.150710+0000","last_peered":"2024-04-22T20:44:40.150710+0000","last_clean":"2024-04-22T20:44:40.150710+0000","last_became_active":"2024-04-22T19:51:30.527712+0000","last_became_peered":"2024-04-22T19:51:30.527712+0000","last_unstale":"2024-04-22T20:44:40.150710+0000","last_undegraded":"2024-04-22T20:44:40.150710+0000","last_fullsized":"2024-04-22T20:44:40.150710+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:03:58.918129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"261'112","reported_seq":656,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.066993+0000","last_change":"2024-04-22T19:51:32.536484+0000","last_active":"2024-04-22T20:45:25.066993+0000","last_peered":"2024-04-22T20:45:25.066993+0000","last_clean":"2024-04-22T20:45:25.066993+0000","last_became_active":"2024-04-22T19:51:32.536177+0000","last_became_peered":"2024-04-22T19:51:32.536177+0000","last_unstale":"2024-04-22T20:45:25.066993+0000","last_undegraded":"2024-04-22T20:45:25.066993+0000","last_fullsized":"2024-04-22T20:45:25.066993+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:39:22.960954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"261'519","reported_seq":2079,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.552754+0000","last_change":"2024-04-22T19:51:34.562120+0000","last_active":"2024-04-22T20:48:08.552754+0000","last_peered":"2024-04-22T20:48:08.552754+0000","last_clean":"2024-04-22T20:48:08.552754+0000","last_became_active":"2024-04-22T19:51:34.561828+0000","last_became_peered":"2024-04-22T19:51:34.561828+0000","last_unstale":"2024-04-22T20:48:08.552754+0000","last_undegraded":"2024-04-22T20:48:08.552754+0000","last_fullsized":"2024-04-22T20:48:08.552754+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:46.946439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"261'174","reported_seq":986,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.139062+0000","last_change":"2024-04-22T19:51:36.565908+0000","last_active":"2024-04-22T20:48:03.139062+0000","last_peered":"2024-04-22T20:48:03.139062+0000","last_clean":"2024-04-22T20:48:03.139062+0000","last_became_active":"2024-04-22T19:51:36.565673+0000","last_became_peered":"2024-04-22T19:51:36.565673+0000","last_unstale":"2024-04-22T20:48:03.139062+0000","last_undegraded":"2024-04-22T20:48:03.139062+0000","last_fullsized":"2024-04-22T20:48:03.139062+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:45:46.054789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":579,"num_read_kb":567,"num_write":371,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"261'176","reported_seq":814,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.150685+0000","last_change":"2024-04-22T19:51:28.516658+0000","last_active":"2024-04-22T20:44:40.150685+0000","last_peered":"2024-04-22T20:44:40.150685+0000","last_clean":"2024-04-22T20:44:40.150685+0000","last_became_active":"2024-04-22T19:51:28.515670+0000","last_became_peered":"2024-04-22T19:51:28.515670+0000","last_unstale":"2024-04-22T20:44:40.150685+0000","last_undegraded":"2024-04-22T20:44:40.150685+0000","last_fullsized":"2024-04-22T20:44:40.150685+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:38:08.341570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":729,"num_read_kb":590,"num_write":271,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"261'261","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.305295+0000","last_change":"2024-04-22T19:51:26.503775+0000","last_active":"2024-04-22T20:47:38.305295+0000","last_peered":"2024-04-22T20:47:38.305295+0000","last_clean":"2024-04-22T20:47:38.305295+0000","last_became_active":"2024-04-22T19:51:26.503438+0000","last_became_peered":"2024-04-22T19:51:26.503438+0000","last_unstale":"2024-04-22T20:47:38.305295+0000","last_undegraded":"2024-04-22T20:47:38.305295+0000","last_fullsized":"2024-04-22T20:47:38.305295+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:39:05.281766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13078725,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":525,"num_read_kb":32833,"num_write":1247,"num_write_kb":25399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180715+0000","last_change":"2024-04-22T19:51:30.523415+0000","last_active":"2024-04-22T20:44:40.180715+0000","last_peered":"2024-04-22T20:44:40.180715+0000","last_clean":"2024-04-22T20:44:40.180715+0000","last_became_active":"2024-04-22T19:51:30.523125+0000","last_became_peered":"2024-04-22T19:51:30.523125+0000","last_unstale":"2024-04-22T20:44:40.180715+0000","last_undegraded":"2024-04-22T20:44:40.180715+0000","last_fullsized":"2024-04-22T20:44:40.180715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:33.007742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"261'247","reported_seq":812,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.078728+0000","last_change":"2024-04-22T19:51:33.248486+0000","last_active":"2024-04-22T20:47:10.078728+0000","last_peered":"2024-04-22T20:47:10.078728+0000","last_clean":"2024-04-22T20:47:10.078728+0000","last_became_active":"2024-04-22T19:51:33.248160+0000","last_became_peered":"2024-04-22T19:51:33.248160+0000","last_unstale":"2024-04-22T20:47:10.078728+0000","last_undegraded":"2024-04-22T20:47:10.078728+0000","last_fullsized":"2024-04-22T20:47:10.078728+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:35.418273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51814,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":412,"num_read_kb":843,"num_write":338,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"261'531","reported_seq":4224,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.556421+0000","last_change":"2024-04-22T19:51:34.556255+0000","last_active":"2024-04-22T20:48:08.556421+0000","last_peered":"2024-04-22T20:48:08.556421+0000","last_clean":"2024-04-22T20:48:08.556421+0000","last_became_active":"2024-04-22T19:51:34.555720+0000","last_became_peered":"2024-04-22T19:51:34.555720+0000","last_unstale":"2024-04-22T20:48:08.556421+0000","last_undegraded":"2024-04-22T20:48:08.556421+0000","last_fullsized":"2024-04-22T20:48:08.556421+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":500,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:58.605732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"261'385","reported_seq":1057,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189325+0000","last_change":"2024-04-22T19:51:28.513088+0000","last_active":"2024-04-22T20:45:40.189325+0000","last_peered":"2024-04-22T20:45:40.189325+0000","last_clean":"2024-04-22T20:45:40.189325+0000","last_became_active":"2024-04-22T19:51:28.512125+0000","last_became_peered":"2024-04-22T19:51:28.512125+0000","last_unstale":"2024-04-22T20:45:40.189325+0000","last_undegraded":"2024-04-22T20:45:40.189325+0000","last_fullsized":"2024-04-22T20:45:40.189325+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:22:31.839887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1728,"num_read_kb":1430,"num_write":663,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"261'238","reported_seq":880,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.318260+0000","last_change":"2024-04-22T19:51:26.508318+0000","last_active":"2024-04-22T20:47:44.318260+0000","last_peered":"2024-04-22T20:47:44.318260+0000","last_clean":"2024-04-22T20:47:44.318260+0000","last_became_active":"2024-04-22T19:51:26.504654+0000","last_became_peered":"2024-04-22T19:51:26.504654+0000","last_unstale":"2024-04-22T20:47:44.318260+0000","last_undegraded":"2024-04-22T20:47:44.318260+0000","last_fullsized":"2024-04-22T20:47:44.318260+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:29:46.972228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12959802,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":378,"num_read_kb":80525,"num_write":948,"num_write_kb":26179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.065667+0000","last_change":"2024-04-22T19:51:30.526634+0000","last_active":"2024-04-22T20:45:40.065667+0000","last_peered":"2024-04-22T20:45:40.065667+0000","last_clean":"2024-04-22T20:45:40.065667+0000","last_became_active":"2024-04-22T19:51:30.526306+0000","last_became_peered":"2024-04-22T19:51:30.526306+0000","last_unstale":"2024-04-22T20:45:40.065667+0000","last_undegraded":"2024-04-22T20:45:40.065667+0000","last_fullsized":"2024-04-22T20:45:40.065667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:46.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"261'329","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:05.210994+0000","last_change":"2024-04-22T19:51:33.248375+0000","last_active":"2024-04-22T20:48:05.210994+0000","last_peered":"2024-04-22T20:48:05.210994+0000","last_clean":"2024-04-22T20:48:05.210994+0000","last_became_active":"2024-04-22T19:51:33.248059+0000","last_became_peered":"2024-04-22T19:51:33.248059+0000","last_unstale":"2024-04-22T20:48:05.210994+0000","last_undegraded":"2024-04-22T20:48:05.210994+0000","last_fullsized":"2024-04-22T20:48:05.210994+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:25:25.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48009,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":524,"num_read_kb":923,"num_write":402,"num_write_kb":246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.045374+0000","last_change":"2024-04-22T19:51:34.557926+0000","last_active":"2024-04-22T20:45:35.045374+0000","last_peered":"2024-04-22T20:45:35.045374+0000","last_clean":"2024-04-22T20:45:35.045374+0000","last_became_active":"2024-04-22T19:51:34.557662+0000","last_became_peered":"2024-04-22T19:51:34.557662+0000","last_unstale":"2024-04-22T20:45:35.045374+0000","last_undegraded":"2024-04-22T20:45:35.045374+0000","last_fullsized":"2024-04-22T20:45:35.045374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:25.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"261'562","reported_seq":1370,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.537178+0000","last_change":"2024-04-22T19:51:36.571168+0000","last_active":"2024-04-22T20:48:08.537178+0000","last_peered":"2024-04-22T20:48:08.537178+0000","last_clean":"2024-04-22T20:48:08.537178+0000","last_became_active":"2024-04-22T19:51:36.570520+0000","last_became_peered":"2024-04-22T19:51:36.570520+0000","last_unstale":"2024-04-22T20:48:08.537178+0000","last_undegraded":"2024-04-22T20:48:08.537178+0000","last_fullsized":"2024-04-22T20:48:08.537178+0000","mapping_epoch":30,"log_start":"261'500","ondisk_log_start":"261'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":500,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:56.456085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1283,"num_read_kb":1280,"num_write":1166,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"261'311","reported_seq":959,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181329+0000","last_change":"2024-04-22T19:51:28.512328+0000","last_active":"2024-04-22T20:45:40.181329+0000","last_peered":"2024-04-22T20:45:40.181329+0000","last_clean":"2024-04-22T20:45:40.181329+0000","last_became_active":"2024-04-22T19:51:28.511527+0000","last_became_peered":"2024-04-22T19:51:28.511527+0000","last_unstale":"2024-04-22T20:45:40.181329+0000","last_undegraded":"2024-04-22T20:45:40.181329+0000","last_fullsized":"2024-04-22T20:45:40.181329+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:22.483205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1221,"num_read_kb":977,"num_write":455,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"261'258","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.064025+0000","last_change":"2024-04-22T19:51:26.505351+0000","last_active":"2024-04-22T20:45:30.064025+0000","last_peered":"2024-04-22T20:45:30.064025+0000","last_clean":"2024-04-22T20:45:30.064025+0000","last_became_active":"2024-04-22T19:51:26.504472+0000","last_became_peered":"2024-04-22T19:51:26.504472+0000","last_unstale":"2024-04-22T20:45:30.064025+0000","last_undegraded":"2024-04-22T20:45:30.064025+0000","last_fullsized":"2024-04-22T20:45:30.064025+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:25.100075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11703329,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":411,"num_read_kb":33343,"num_write":1065,"num_write_kb":49829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"32'5","reported_seq":826,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:07.851401+0000","last_change":"2024-04-22T19:51:30.524309+0000","last_active":"2024-04-22T20:48:07.851401+0000","last_peered":"2024-04-22T20:48:07.851401+0000","last_clean":"2024-04-22T20:48:07.851401+0000","last_became_active":"2024-04-22T19:51:30.524043+0000","last_became_peered":"2024-04-22T19:51:30.524043+0000","last_unstale":"2024-04-22T20:48:07.851401+0000","last_undegraded":"2024-04-22T20:48:07.851401+0000","last_fullsized":"2024-04-22T20:48:07.851401+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:51.651333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":489,"num_read_kb":794,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"261'261","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.172994+0000","last_change":"2024-04-22T19:51:32.539254+0000","last_active":"2024-04-22T20:47:50.172994+0000","last_peered":"2024-04-22T20:47:50.172994+0000","last_clean":"2024-04-22T20:47:50.172994+0000","last_became_active":"2024-04-22T19:51:32.537968+0000","last_became_peered":"2024-04-22T19:51:32.537968+0000","last_unstale":"2024-04-22T20:47:50.172994+0000","last_undegraded":"2024-04-22T20:47:50.172994+0000","last_fullsized":"2024-04-22T20:47:50.172994+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:38.345231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51807,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":422,"num_read_kb":877,"num_write":344,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158827+0000","last_change":"2024-04-22T19:51:34.556181+0000","last_active":"2024-04-22T20:45:50.158827+0000","last_peered":"2024-04-22T20:45:50.158827+0000","last_clean":"2024-04-22T20:45:50.158827+0000","last_became_active":"2024-04-22T19:51:34.555889+0000","last_became_peered":"2024-04-22T19:51:34.555889+0000","last_unstale":"2024-04-22T20:45:50.158827+0000","last_undegraded":"2024-04-22T20:45:50.158827+0000","last_fullsized":"2024-04-22T20:45:50.158827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:03:03.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"261'79","reported_seq":577,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.319292+0000","last_change":"2024-04-22T19:51:36.574838+0000","last_active":"2024-04-22T20:45:55.319292+0000","last_peered":"2024-04-22T20:45:55.319292+0000","last_clean":"2024-04-22T20:45:55.319292+0000","last_became_active":"2024-04-22T19:51:36.574555+0000","last_became_peered":"2024-04-22T19:51:36.574555+0000","last_unstale":"2024-04-22T20:45:55.319292+0000","last_undegraded":"2024-04-22T20:45:55.319292+0000","last_fullsized":"2024-04-22T20:45:55.319292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:07.667952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":91,"num_write":194,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"261'188","reported_seq":783,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.115015+0000","last_change":"2024-04-22T19:51:28.519762+0000","last_active":"2024-04-22T20:46:41.115015+0000","last_peered":"2024-04-22T20:46:41.115015+0000","last_clean":"2024-04-22T20:46:41.115015+0000","last_became_active":"2024-04-22T19:51:28.519424+0000","last_became_peered":"2024-04-22T19:51:28.519424+0000","last_unstale":"2024-04-22T20:46:41.115015+0000","last_undegraded":"2024-04-22T20:46:41.115015+0000","last_fullsized":"2024-04-22T20:46:41.115015+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:13.130467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":758,"num_read_kb":601,"num_write":297,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"261'219","reported_seq":841,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.063988+0000","last_change":"2024-04-22T19:51:26.511055+0000","last_active":"2024-04-22T20:45:30.063988+0000","last_peered":"2024-04-22T20:45:30.063988+0000","last_clean":"2024-04-22T20:45:30.063988+0000","last_became_active":"2024-04-22T19:51:26.510273+0000","last_became_peered":"2024-04-22T19:51:26.510273+0000","last_unstale":"2024-04-22T20:45:30.063988+0000","last_undegraded":"2024-04-22T20:45:30.063988+0000","last_fullsized":"2024-04-22T20:45:30.063988+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:53.305766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15039158,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":310,"num_read_kb":64999,"num_write":940,"num_write_kb":29343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.067468+0000","last_change":"2024-04-22T19:51:30.525335+0000","last_active":"2024-04-22T20:45:30.067468+0000","last_peered":"2024-04-22T20:45:30.067468+0000","last_clean":"2024-04-22T20:45:30.067468+0000","last_became_active":"2024-04-22T19:51:30.525018+0000","last_became_peered":"2024-04-22T19:51:30.525018+0000","last_unstale":"2024-04-22T20:45:30.067468+0000","last_undegraded":"2024-04-22T20:45:30.067468+0000","last_fullsized":"2024-04-22T20:45:30.067468+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:49:24.770975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"261'304","reported_seq":892,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.085578+0000","last_change":"2024-04-22T19:51:32.538989+0000","last_active":"2024-04-22T20:47:55.085578+0000","last_peered":"2024-04-22T20:47:55.085578+0000","last_clean":"2024-04-22T20:47:55.085578+0000","last_became_active":"2024-04-22T19:51:32.538656+0000","last_became_peered":"2024-04-22T19:51:32.538656+0000","last_unstale":"2024-04-22T20:47:55.085578+0000","last_undegraded":"2024-04-22T20:47:55.085578+0000","last_fullsized":"2024-04-22T20:47:55.085578+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:36:49.025357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61009,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":485,"num_read_kb":947,"num_write":390,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"261'504","reported_seq":2193,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.557320+0000","last_change":"2024-04-22T19:51:34.561771+0000","last_active":"2024-04-22T20:48:08.557320+0000","last_peered":"2024-04-22T20:48:08.557320+0000","last_clean":"2024-04-22T20:48:08.557320+0000","last_became_active":"2024-04-22T19:51:34.561488+0000","last_became_peered":"2024-04-22T19:51:34.561488+0000","last_unstale":"2024-04-22T20:48:08.557320+0000","last_undegraded":"2024-04-22T20:48:08.557320+0000","last_fullsized":"2024-04-22T20:48:08.557320+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":400,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:08:35.813443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"261'134","reported_seq":623,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.184424+0000","last_change":"2024-04-22T19:51:36.573554+0000","last_active":"2024-04-22T20:45:50.184424+0000","last_peered":"2024-04-22T20:45:50.184424+0000","last_clean":"2024-04-22T20:45:50.184424+0000","last_became_active":"2024-04-22T19:51:36.573167+0000","last_became_peered":"2024-04-22T19:51:36.573167+0000","last_unstale":"2024-04-22T20:45:50.184424+0000","last_undegraded":"2024-04-22T20:45:50.184424+0000","last_fullsized":"2024-04-22T20:45:50.184424+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:24.336268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"261'150","reported_seq":715,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181727+0000","last_change":"2024-04-22T19:51:28.518692+0000","last_active":"2024-04-22T20:45:30.181727+0000","last_peered":"2024-04-22T20:45:30.181727+0000","last_clean":"2024-04-22T20:45:30.181727+0000","last_became_active":"2024-04-22T19:51:28.518398+0000","last_became_peered":"2024-04-22T19:51:28.518398+0000","last_unstale":"2024-04-22T20:45:30.181727+0000","last_undegraded":"2024-04-22T20:45:30.181727+0000","last_fullsized":"2024-04-22T20:45:30.181727+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:33.700338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":596,"num_read_kb":465,"num_write":243,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"261'246","reported_seq":915,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.890178+0000","last_change":"2024-04-22T19:52:05.113375+0000","last_active":"2024-04-22T20:46:41.890178+0000","last_peered":"2024-04-22T20:46:41.890178+0000","last_clean":"2024-04-22T20:46:41.890178+0000","last_became_active":"2024-04-22T19:52:05.112910+0000","last_became_peered":"2024-04-22T19:52:05.112910+0000","last_unstale":"2024-04-22T20:46:41.890178+0000","last_undegraded":"2024-04-22T20:46:41.890178+0000","last_fullsized":"2024-04-22T20:46:41.890178+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:44:17.477228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4708959,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":450,"num_read_kb":39016,"num_write":1025,"num_write_kb":34073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.069399+0000","last_change":"2024-04-22T19:51:30.529388+0000","last_active":"2024-04-22T20:45:50.069399+0000","last_peered":"2024-04-22T20:45:50.069399+0000","last_clean":"2024-04-22T20:45:50.069399+0000","last_became_active":"2024-04-22T19:51:30.529011+0000","last_became_peered":"2024-04-22T19:51:30.529011+0000","last_unstale":"2024-04-22T20:45:50.069399+0000","last_undegraded":"2024-04-22T20:45:50.069399+0000","last_fullsized":"2024-04-22T20:45:50.069399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:02:44.821662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"261'478","reported_seq":1102,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.503866+0000","last_change":"2024-04-22T19:51:32.537571+0000","last_active":"2024-04-22T20:48:08.503866+0000","last_peered":"2024-04-22T20:48:08.503866+0000","last_clean":"2024-04-22T20:48:08.503866+0000","last_became_active":"2024-04-22T19:51:32.537267+0000","last_became_peered":"2024-04-22T19:51:32.537267+0000","last_unstale":"2024-04-22T20:48:08.503866+0000","last_undegraded":"2024-04-22T20:48:08.503866+0000","last_fullsized":"2024-04-22T20:48:08.503866+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":400,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:04.937868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123858,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":830,"num_read_kb":1829,"num_write":692,"num_write_kb":607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.068905+0000","last_change":"2024-04-22T19:51:34.552745+0000","last_active":"2024-04-22T20:45:45.068905+0000","last_peered":"2024-04-22T20:45:45.068905+0000","last_clean":"2024-04-22T20:45:45.068905+0000","last_became_active":"2024-04-22T19:51:34.552446+0000","last_became_peered":"2024-04-22T19:51:34.552446+0000","last_unstale":"2024-04-22T20:45:45.068905+0000","last_undegraded":"2024-04-22T20:45:45.068905+0000","last_fullsized":"2024-04-22T20:45:45.068905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:59.206061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"261'106","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.047062+0000","last_change":"2024-04-22T19:51:36.570982+0000","last_active":"2024-04-22T20:45:50.047062+0000","last_peered":"2024-04-22T20:45:50.047062+0000","last_clean":"2024-04-22T20:45:50.047062+0000","last_became_active":"2024-04-22T19:51:36.570666+0000","last_became_peered":"2024-04-22T19:51:36.570666+0000","last_unstale":"2024-04-22T20:45:50.047062+0000","last_undegraded":"2024-04-22T20:45:50.047062+0000","last_fullsized":"2024-04-22T20:45:50.047062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:18.966614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":242,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"261'168","reported_seq":795,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:16.403532+0000","last_change":"2024-04-22T19:51:28.508639+0000","last_active":"2024-04-22T20:47:16.403532+0000","last_peered":"2024-04-22T20:47:16.403532+0000","last_clean":"2024-04-22T20:47:16.403532+0000","last_became_active":"2024-04-22T19:51:28.508288+0000","last_became_peered":"2024-04-22T19:51:28.508288+0000","last_unstale":"2024-04-22T20:47:16.403532+0000","last_undegraded":"2024-04-22T20:47:16.403532+0000","last_fullsized":"2024-04-22T20:47:16.403532+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:22:30.411824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":576,"num_write":244,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"261'258","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:48.776114+0000","last_change":"2024-04-22T19:51:26.506251+0000","last_active":"2024-04-22T20:46:48.776114+0000","last_peered":"2024-04-22T20:46:48.776114+0000","last_clean":"2024-04-22T20:46:48.776114+0000","last_became_active":"2024-04-22T19:51:26.503669+0000","last_became_peered":"2024-04-22T19:51:26.503669+0000","last_unstale":"2024-04-22T20:46:48.776114+0000","last_undegraded":"2024-04-22T20:46:48.776114+0000","last_fullsized":"2024-04-22T20:46:48.776114+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:52.491938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8753181,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":397,"num_read_kb":4498,"num_write":1038,"num_write_kb":33252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.067499+0000","last_change":"2024-04-22T19:51:30.526698+0000","last_active":"2024-04-22T20:45:30.067499+0000","last_peered":"2024-04-22T20:45:30.067499+0000","last_clean":"2024-04-22T20:45:30.067499+0000","last_became_active":"2024-04-22T19:51:30.526420+0000","last_became_peered":"2024-04-22T19:51:30.526420+0000","last_unstale":"2024-04-22T20:45:30.067499+0000","last_undegraded":"2024-04-22T20:45:30.067499+0000","last_fullsized":"2024-04-22T20:45:30.067499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:14.342753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"261'264","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.199269+0000","last_change":"2024-04-22T19:51:32.536764+0000","last_active":"2024-04-22T20:47:50.199269+0000","last_peered":"2024-04-22T20:47:50.199269+0000","last_clean":"2024-04-22T20:47:50.199269+0000","last_became_active":"2024-04-22T19:51:32.536463+0000","last_became_peered":"2024-04-22T19:51:32.536463+0000","last_unstale":"2024-04-22T20:47:50.199269+0000","last_undegraded":"2024-04-22T20:47:50.199269+0000","last_fullsized":"2024-04-22T20:47:50.199269+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:29:00.474073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48720,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":444,"num_read_kb":848,"num_write":396,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183789+0000","last_change":"2024-04-22T19:51:34.555066+0000","last_active":"2024-04-22T20:45:45.183789+0000","last_peered":"2024-04-22T20:45:45.183789+0000","last_clean":"2024-04-22T20:45:45.183789+0000","last_became_active":"2024-04-22T19:51:34.554737+0000","last_became_peered":"2024-04-22T19:51:34.554737+0000","last_unstale":"2024-04-22T20:45:45.183789+0000","last_undegraded":"2024-04-22T20:45:45.183789+0000","last_fullsized":"2024-04-22T20:45:45.183789+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:09:00.070765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"261'93","reported_seq":590,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:42.565656+0000","last_change":"2024-04-22T19:51:36.571558+0000","last_active":"2024-04-22T20:46:42.565656+0000","last_peered":"2024-04-22T20:46:42.565656+0000","last_clean":"2024-04-22T20:46:42.565656+0000","last_became_active":"2024-04-22T19:51:36.571086+0000","last_became_peered":"2024-04-22T19:51:36.571086+0000","last_unstale":"2024-04-22T20:46:42.565656+0000","last_undegraded":"2024-04-22T20:46:42.565656+0000","last_fullsized":"2024-04-22T20:46:42.565656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:37:15.182216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":72,"num_write":227,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"261'253","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:07.364455+0000","last_change":"2024-04-22T19:51:28.514713+0000","last_active":"2024-04-22T20:46:07.364455+0000","last_peered":"2024-04-22T20:46:07.364455+0000","last_clean":"2024-04-22T20:46:07.364455+0000","last_became_active":"2024-04-22T19:51:28.514445+0000","last_became_peered":"2024-04-22T19:51:28.514445+0000","last_unstale":"2024-04-22T20:46:07.364455+0000","last_undegraded":"2024-04-22T20:46:07.364455+0000","last_fullsized":"2024-04-22T20:46:07.364455+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:39.638389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":723,"num_write":394,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'331","reported_seq":1077,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:59.585410+0000","last_change":"2024-04-22T19:51:26.496789+0000","last_active":"2024-04-22T20:46:59.585410+0000","last_peered":"2024-04-22T20:46:59.585410+0000","last_clean":"2024-04-22T20:46:59.585410+0000","last_became_active":"2024-04-22T19:51:26.496408+0000","last_became_peered":"2024-04-22T19:51:26.496408+0000","last_unstale":"2024-04-22T20:46:59.585410+0000","last_undegraded":"2024-04-22T20:46:59.585410+0000","last_fullsized":"2024-04-22T20:46:59.585410+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:19:23.202029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1053522,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":776,"num_read_kb":8021,"num_write":1416,"num_write_kb":27015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.044898+0000","last_change":"2024-04-22T19:51:30.527588+0000","last_active":"2024-04-22T20:45:30.044898+0000","last_peered":"2024-04-22T20:45:30.044898+0000","last_clean":"2024-04-22T20:45:30.044898+0000","last_became_active":"2024-04-22T19:51:30.527309+0000","last_became_peered":"2024-04-22T19:51:30.527309+0000","last_unstale":"2024-04-22T20:45:30.044898+0000","last_undegraded":"2024-04-22T20:45:30.044898+0000","last_fullsized":"2024-04-22T20:45:30.044898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:22:07.894332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"261'303","reported_seq":894,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.082212+0000","last_change":"2024-04-22T19:51:32.540003+0000","last_active":"2024-04-22T20:48:00.082212+0000","last_peered":"2024-04-22T20:48:00.082212+0000","last_clean":"2024-04-22T20:48:00.082212+0000","last_became_active":"2024-04-22T19:51:32.539732+0000","last_became_peered":"2024-04-22T19:51:32.539732+0000","last_unstale":"2024-04-22T20:48:00.082212+0000","last_undegraded":"2024-04-22T20:48:00.082212+0000","last_fullsized":"2024-04-22T20:48:00.082212+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:19.413189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63653,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":499,"num_read_kb":1000,"num_write":402,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"261'485","reported_seq":2198,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.553773+0000","last_change":"2024-04-22T19:51:34.556537+0000","last_active":"2024-04-22T20:48:08.553773+0000","last_peered":"2024-04-22T20:48:08.553773+0000","last_clean":"2024-04-22T20:48:08.553773+0000","last_became_active":"2024-04-22T19:51:34.556124+0000","last_became_peered":"2024-04-22T19:51:34.556124+0000","last_unstale":"2024-04-22T20:48:08.553773+0000","last_undegraded":"2024-04-22T20:48:08.553773+0000","last_fullsized":"2024-04-22T20:48:08.553773+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":400,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:14:07.299855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"261'116","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.060760+0000","last_change":"2024-04-22T19:51:36.573163+0000","last_active":"2024-04-22T20:47:55.060760+0000","last_peered":"2024-04-22T20:47:55.060760+0000","last_clean":"2024-04-22T20:47:55.060760+0000","last_became_active":"2024-04-22T19:51:36.572913+0000","last_became_peered":"2024-04-22T19:51:36.572913+0000","last_unstale":"2024-04-22T20:47:55.060760+0000","last_undegraded":"2024-04-22T20:47:55.060760+0000","last_fullsized":"2024-04-22T20:47:55.060760+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:40:20.320547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":102,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"261'338","reported_seq":1080,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.179830+0000","last_change":"2024-04-22T19:51:28.516099+0000","last_active":"2024-04-22T20:45:30.179830+0000","last_peered":"2024-04-22T20:45:30.179830+0000","last_clean":"2024-04-22T20:45:30.179830+0000","last_became_active":"2024-04-22T19:51:28.515809+0000","last_became_peered":"2024-04-22T19:51:28.515809+0000","last_unstale":"2024-04-22T20:45:30.179830+0000","last_undegraded":"2024-04-22T20:45:30.179830+0000","last_fullsized":"2024-04-22T20:45:30.179830+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:19.069001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1504,"num_read_kb":1227,"num_write":616,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"261'280","reported_seq":915,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181307+0000","last_change":"2024-04-22T19:51:26.499730+0000","last_active":"2024-04-22T20:45:40.181307+0000","last_peered":"2024-04-22T20:45:40.181307+0000","last_clean":"2024-04-22T20:45:40.181307+0000","last_became_active":"2024-04-22T19:51:26.499362+0000","last_became_peered":"2024-04-22T19:51:26.499362+0000","last_unstale":"2024-04-22T20:45:40.181307+0000","last_undegraded":"2024-04-22T20:45:40.181307+0000","last_fullsized":"2024-04-22T20:45:40.181307+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:28:07.429252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9762556,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":451,"num_read_kb":26712,"num_write":1196,"num_write_kb":36357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"25'2","reported_seq":1134,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.108639+0000","last_change":"2024-04-22T19:51:30.527770+0000","last_active":"2024-04-22T20:48:15.108639+0000","last_peered":"2024-04-22T20:48:15.108639+0000","last_clean":"2024-04-22T20:48:15.108639+0000","last_became_active":"2024-04-22T19:51:30.527490+0000","last_became_peered":"2024-04-22T19:51:30.527490+0000","last_unstale":"2024-04-22T20:48:15.108639+0000","last_undegraded":"2024-04-22T20:48:15.108639+0000","last_fullsized":"2024-04-22T20:48:15.108639+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:39:09.682743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":639,"num_read_kb":639,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"261'90","reported_seq":620,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157133+0000","last_change":"2024-04-22T19:51:33.248919+0000","last_active":"2024-04-22T20:45:35.157133+0000","last_peered":"2024-04-22T20:45:35.157133+0000","last_clean":"2024-04-22T20:45:35.157133+0000","last_became_active":"2024-04-22T19:51:33.248349+0000","last_became_peered":"2024-04-22T19:51:33.248349+0000","last_unstale":"2024-04-22T20:45:35.157133+0000","last_undegraded":"2024-04-22T20:45:35.157133+0000","last_fullsized":"2024-04-22T20:45:35.157133+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:03:31.126924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":60,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"261'515","reported_seq":2069,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.554704+0000","last_change":"2024-04-22T19:51:34.562158+0000","last_active":"2024-04-22T20:48:08.554704+0000","last_peered":"2024-04-22T20:48:08.554704+0000","last_clean":"2024-04-22T20:48:08.554704+0000","last_became_active":"2024-04-22T19:51:34.561825+0000","last_became_peered":"2024-04-22T19:51:34.561825+0000","last_unstale":"2024-04-22T20:48:08.554704+0000","last_undegraded":"2024-04-22T20:48:08.554704+0000","last_fullsized":"2024-04-22T20:48:08.554704+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:22:52.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"261'130","reported_seq":625,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.182787+0000","last_change":"2024-04-22T19:51:36.573308+0000","last_active":"2024-04-22T20:45:50.182787+0000","last_peered":"2024-04-22T20:45:50.182787+0000","last_clean":"2024-04-22T20:45:50.182787+0000","last_became_active":"2024-04-22T19:51:36.573026+0000","last_became_peered":"2024-04-22T19:51:36.573026+0000","last_unstale":"2024-04-22T20:45:50.182787+0000","last_undegraded":"2024-04-22T20:45:50.182787+0000","last_fullsized":"2024-04-22T20:45:50.182787+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:19.108938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":119,"num_write":309,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"261'207","reported_seq":818,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045988+0000","last_change":"2024-04-22T19:51:28.512959+0000","last_active":"2024-04-22T20:45:40.045988+0000","last_peered":"2024-04-22T20:45:40.045988+0000","last_clean":"2024-04-22T20:45:40.045988+0000","last_became_active":"2024-04-22T19:51:28.512275+0000","last_became_peered":"2024-04-22T19:51:28.512275+0000","last_unstale":"2024-04-22T20:45:40.045988+0000","last_undegraded":"2024-04-22T20:45:40.045988+0000","last_fullsized":"2024-04-22T20:45:40.045988+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:24:20.294556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":832,"num_read_kb":677,"num_write":302,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"261'258","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:31.723467+0000","last_change":"2024-04-22T19:51:26.498630+0000","last_active":"2024-04-22T20:46:31.723467+0000","last_peered":"2024-04-22T20:46:31.723467+0000","last_clean":"2024-04-22T20:46:31.723467+0000","last_became_active":"2024-04-22T19:51:26.498276+0000","last_became_peered":"2024-04-22T19:51:26.498276+0000","last_unstale":"2024-04-22T20:46:31.723467+0000","last_undegraded":"2024-04-22T20:46:31.723467+0000","last_fullsized":"2024-04-22T20:46:31.723467+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:43:15.698499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9975117,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":398,"num_read_kb":30145,"num_write":1092,"num_write_kb":23135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"32'3","reported_seq":820,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.478858+0000","last_change":"2024-04-22T19:51:30.530726+0000","last_active":"2024-04-22T20:48:08.478858+0000","last_peered":"2024-04-22T20:48:08.478858+0000","last_clean":"2024-04-22T20:48:08.478858+0000","last_became_active":"2024-04-22T19:51:30.530363+0000","last_became_peered":"2024-04-22T19:51:30.530363+0000","last_unstale":"2024-04-22T20:48:08.478858+0000","last_undegraded":"2024-04-22T20:48:08.478858+0000","last_fullsized":"2024-04-22T20:48:08.478858+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:58:21.578312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":489,"num_read_kb":487,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"261'396","reported_seq":999,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:06.461115+0000","last_change":"2024-04-22T19:51:32.540935+0000","last_active":"2024-04-22T20:48:06.461115+0000","last_peered":"2024-04-22T20:48:06.461115+0000","last_clean":"2024-04-22T20:48:06.461115+0000","last_became_active":"2024-04-22T19:51:32.540622+0000","last_became_peered":"2024-04-22T19:51:32.540622+0000","last_unstale":"2024-04-22T20:48:06.461115+0000","last_undegraded":"2024-04-22T20:48:06.461115+0000","last_fullsized":"2024-04-22T20:48:06.461115+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":300,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:57:03.444994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1216,"num_read_kb":1715,"num_write":1756,"num_write_kb":1624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.316223+0000","last_change":"2024-04-22T19:51:34.551287+0000","last_active":"2024-04-22T20:45:35.316223+0000","last_peered":"2024-04-22T20:45:35.316223+0000","last_clean":"2024-04-22T20:45:35.316223+0000","last_became_active":"2024-04-22T19:51:34.551009+0000","last_became_peered":"2024-04-22T19:51:34.551009+0000","last_unstale":"2024-04-22T20:45:35.316223+0000","last_undegraded":"2024-04-22T20:45:35.316223+0000","last_fullsized":"2024-04-22T20:45:35.316223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:24:58.026337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"261'121","reported_seq":620,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.018220+0000","last_change":"2024-04-22T19:51:36.571807+0000","last_active":"2024-04-22T20:48:04.018220+0000","last_peered":"2024-04-22T20:48:04.018220+0000","last_clean":"2024-04-22T20:48:04.018220+0000","last_became_active":"2024-04-22T19:51:36.571517+0000","last_became_peered":"2024-04-22T19:51:36.571517+0000","last_unstale":"2024-04-22T20:48:04.018220+0000","last_undegraded":"2024-04-22T20:48:04.018220+0000","last_fullsized":"2024-04-22T20:48:04.018220+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:15:52.550381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":272,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"261'4186","reported_seq":4781,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.317812+0000","last_change":"2024-04-22T19:51:28.515172+0000","last_active":"2024-04-22T20:45:45.317812+0000","last_peered":"2024-04-22T20:45:45.317812+0000","last_clean":"2024-04-22T20:45:45.317812+0000","last_became_active":"2024-04-22T19:51:28.514869+0000","last_became_peered":"2024-04-22T19:51:28.514869+0000","last_unstale":"2024-04-22T20:45:45.317812+0000","last_undegraded":"2024-04-22T20:45:45.317812+0000","last_fullsized":"2024-04-22T20:45:45.317812+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":3000,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:44.084365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14767,"num_read_kb":11981,"num_write":6295,"num_write_kb":3142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":520,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063174+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T20:45:25.063174+0000","last_peered":"2024-04-22T20:45:25.063174+0000","last_clean":"2024-04-22T20:45:25.063174+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T20:45:25.063174+0000","last_undegraded":"2024-04-22T20:45:25.063174+0000","last_fullsized":"2024-04-22T20:45:25.063174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"261'278","reported_seq":973,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.326360+0000","last_change":"2024-04-22T19:51:26.506322+0000","last_active":"2024-04-22T20:47:44.326360+0000","last_peered":"2024-04-22T20:47:44.326360+0000","last_clean":"2024-04-22T20:47:44.326360+0000","last_became_active":"2024-04-22T19:51:26.505982+0000","last_became_peered":"2024-04-22T19:51:26.505982+0000","last_unstale":"2024-04-22T20:47:44.326360+0000","last_undegraded":"2024-04-22T20:47:44.326360+0000","last_fullsized":"2024-04-22T20:47:44.326360+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:46.483074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691084,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":484,"num_read_kb":107291,"num_write":1302,"num_write_kb":45616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.065725+0000","last_change":"2024-04-22T19:51:30.527554+0000","last_active":"2024-04-22T20:45:40.065725+0000","last_peered":"2024-04-22T20:45:40.065725+0000","last_clean":"2024-04-22T20:45:40.065725+0000","last_became_active":"2024-04-22T19:51:30.526939+0000","last_became_peered":"2024-04-22T19:51:30.526939+0000","last_unstale":"2024-04-22T20:45:40.065725+0000","last_undegraded":"2024-04-22T20:45:40.065725+0000","last_fullsized":"2024-04-22T20:45:40.065725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:34:00.754603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"261'700","reported_seq":1386,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.522859+0000","last_change":"2024-04-22T19:51:32.543191+0000","last_active":"2024-04-22T20:48:08.522859+0000","last_peered":"2024-04-22T20:48:08.522859+0000","last_clean":"2024-04-22T20:48:08.522859+0000","last_became_active":"2024-04-22T19:51:32.542970+0000","last_became_peered":"2024-04-22T19:51:32.542970+0000","last_unstale":"2024-04-22T20:48:08.522859+0000","last_undegraded":"2024-04-22T20:48:08.522859+0000","last_fullsized":"2024-04-22T20:48:08.522859+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":600,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:42.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171781,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1364,"num_read_kb":2811,"num_write":1300,"num_write_kb":1164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183762+0000","last_change":"2024-04-22T19:51:34.554048+0000","last_active":"2024-04-22T20:45:45.183762+0000","last_peered":"2024-04-22T20:45:45.183762+0000","last_clean":"2024-04-22T20:45:45.183762+0000","last_became_active":"2024-04-22T19:51:34.553704+0000","last_became_peered":"2024-04-22T19:51:34.553704+0000","last_unstale":"2024-04-22T20:45:45.183762+0000","last_undegraded":"2024-04-22T20:45:45.183762+0000","last_fullsized":"2024-04-22T20:45:45.183762+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:19:14.613388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"261'266","reported_seq":1001,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.066411+0000","last_change":"2024-04-22T19:51:36.571003+0000","last_active":"2024-04-22T20:48:04.066411+0000","last_peered":"2024-04-22T20:48:04.066411+0000","last_clean":"2024-04-22T20:48:04.066411+0000","last_became_active":"2024-04-22T19:51:36.570717+0000","last_became_peered":"2024-04-22T19:51:36.570717+0000","last_unstale":"2024-04-22T20:48:04.066411+0000","last_undegraded":"2024-04-22T20:48:04.066411+0000","last_fullsized":"2024-04-22T20:48:04.066411+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:16.336092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":568,"num_write":495,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"261'50","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045937+0000","last_change":"2024-04-22T19:51:28.506462+0000","last_active":"2024-04-22T20:45:40.045937+0000","last_peered":"2024-04-22T20:45:40.045937+0000","last_clean":"2024-04-22T20:45:40.045937+0000","last_became_active":"2024-04-22T19:51:28.505719+0000","last_became_peered":"2024-04-22T19:51:28.505719+0000","last_unstale":"2024-04-22T20:45:40.045937+0000","last_undegraded":"2024-04-22T20:45:40.045937+0000","last_fullsized":"2024-04-22T20:45:40.045937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:02.609039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":69,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":514,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186325+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T20:45:20.186325+0000","last_peered":"2024-04-22T20:45:20.186325+0000","last_clean":"2024-04-22T20:45:20.186325+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T20:45:20.186325+0000","last_undegraded":"2024-04-22T20:45:20.186325+0000","last_fullsized":"2024-04-22T20:45:20.186325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:20:10.880037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"261'278","reported_seq":954,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:18.964925+0000","last_change":"2024-04-22T19:51:26.501306+0000","last_active":"2024-04-22T20:46:18.964925+0000","last_peered":"2024-04-22T20:46:18.964925+0000","last_clean":"2024-04-22T20:46:18.964925+0000","last_became_active":"2024-04-22T19:51:26.500968+0000","last_became_peered":"2024-04-22T19:51:26.500968+0000","last_unstale":"2024-04-22T20:46:18.964925+0000","last_undegraded":"2024-04-22T20:46:18.964925+0000","last_fullsized":"2024-04-22T20:46:18.964925+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:20:53.110538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10522090,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":517,"num_read_kb":30331,"num_write":1214,"num_write_kb":32709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.184374+0000","last_change":"2024-04-22T19:51:30.526621+0000","last_active":"2024-04-22T20:45:50.184374+0000","last_peered":"2024-04-22T20:45:50.184374+0000","last_clean":"2024-04-22T20:45:50.184374+0000","last_became_active":"2024-04-22T19:51:30.526328+0000","last_became_peered":"2024-04-22T19:51:30.526328+0000","last_unstale":"2024-04-22T20:45:50.184374+0000","last_undegraded":"2024-04-22T20:45:50.184374+0000","last_fullsized":"2024-04-22T20:45:50.184374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:59:48.789109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"261'124","reported_seq":677,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.158336+0000","last_change":"2024-04-22T19:51:32.541073+0000","last_active":"2024-04-22T20:45:45.158336+0000","last_peered":"2024-04-22T20:45:45.158336+0000","last_clean":"2024-04-22T20:45:45.158336+0000","last_became_active":"2024-04-22T19:51:32.540805+0000","last_became_peered":"2024-04-22T19:51:32.540805+0000","last_unstale":"2024-04-22T20:45:45.158336+0000","last_undegraded":"2024-04-22T20:45:45.158336+0000","last_fullsized":"2024-04-22T20:45:45.158336+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:55.727535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":112,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.317842+0000","last_change":"2024-04-22T19:51:34.554535+0000","last_active":"2024-04-22T20:45:45.317842+0000","last_peered":"2024-04-22T20:45:45.317842+0000","last_clean":"2024-04-22T20:45:45.317842+0000","last_became_active":"2024-04-22T19:51:34.554307+0000","last_became_peered":"2024-04-22T19:51:34.554307+0000","last_unstale":"2024-04-22T20:45:45.317842+0000","last_undegraded":"2024-04-22T20:45:45.317842+0000","last_fullsized":"2024-04-22T20:45:45.317842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:13.279562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"261'123","reported_seq":625,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.823391+0000","last_change":"2024-04-22T19:51:36.571438+0000","last_active":"2024-04-22T20:48:03.823391+0000","last_peered":"2024-04-22T20:48:03.823391+0000","last_clean":"2024-04-22T20:48:03.823391+0000","last_became_active":"2024-04-22T19:51:36.571130+0000","last_became_peered":"2024-04-22T19:51:36.571130+0000","last_unstale":"2024-04-22T20:48:03.823391+0000","last_undegraded":"2024-04-22T20:48:03.823391+0000","last_fullsized":"2024-04-22T20:48:03.823391+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:10:47.000615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":131,"num_write":305,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"261'90","reported_seq":658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045996+0000","last_change":"2024-04-22T19:51:28.514065+0000","last_active":"2024-04-22T20:45:40.045996+0000","last_peered":"2024-04-22T20:45:40.045996+0000","last_clean":"2024-04-22T20:45:40.045996+0000","last_became_active":"2024-04-22T19:51:28.513451+0000","last_became_peered":"2024-04-22T19:51:28.513451+0000","last_unstale":"2024-04-22T20:45:40.045996+0000","last_undegraded":"2024-04-22T20:45:40.045996+0000","last_fullsized":"2024-04-22T20:45:40.045996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:52:37.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":305,"num_write":130,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":519,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062391+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T20:45:20.062391+0000","last_peered":"2024-04-22T20:45:20.062391+0000","last_clean":"2024-04-22T20:45:20.062391+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T20:45:20.062391+0000","last_undegraded":"2024-04-22T20:45:20.062391+0000","last_fullsized":"2024-04-22T20:45:20.062391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"261'262","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045858+0000","last_change":"2024-04-22T19:51:26.495730+0000","last_active":"2024-04-22T20:45:40.045858+0000","last_peered":"2024-04-22T20:45:40.045858+0000","last_clean":"2024-04-22T20:45:40.045858+0000","last_became_active":"2024-04-22T19:51:26.495405+0000","last_became_peered":"2024-04-22T19:51:26.495405+0000","last_unstale":"2024-04-22T20:45:40.045858+0000","last_undegraded":"2024-04-22T20:45:40.045858+0000","last_fullsized":"2024-04-22T20:45:40.045858+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:00.884012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23787988,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":57834,"num_write":1209,"num_write_kb":50104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"25'1","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.068477+0000","last_change":"2024-04-22T19:51:30.521583+0000","last_active":"2024-04-22T20:45:40.068477+0000","last_peered":"2024-04-22T20:45:40.068477+0000","last_clean":"2024-04-22T20:45:40.068477+0000","last_became_active":"2024-04-22T19:51:30.521134+0000","last_became_peered":"2024-04-22T19:51:30.521134+0000","last_unstale":"2024-04-22T20:45:40.068477+0000","last_undegraded":"2024-04-22T20:45:40.068477+0000","last_fullsized":"2024-04-22T20:45:40.068477+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:49:17.105976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"261'221","reported_seq":1448,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.449853+0000","last_change":"2024-04-22T19:51:33.250383+0000","last_active":"2024-04-22T20:48:08.449853+0000","last_peered":"2024-04-22T20:48:08.449853+0000","last_clean":"2024-04-22T20:48:08.449853+0000","last_became_active":"2024-04-22T19:51:33.250082+0000","last_became_peered":"2024-04-22T19:51:33.250082+0000","last_unstale":"2024-04-22T20:48:08.449853+0000","last_undegraded":"2024-04-22T20:48:08.449853+0000","last_fullsized":"2024-04-22T20:48:08.449853+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:43:34.357226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":188,"num_read_kb":180,"num_write":143,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183725+0000","last_change":"2024-04-22T19:51:34.548541+0000","last_active":"2024-04-22T20:45:45.183725+0000","last_peered":"2024-04-22T20:45:45.183725+0000","last_clean":"2024-04-22T20:45:45.183725+0000","last_became_active":"2024-04-22T19:51:34.546652+0000","last_became_peered":"2024-04-22T19:51:34.546652+0000","last_unstale":"2024-04-22T20:45:45.183725+0000","last_undegraded":"2024-04-22T20:45:45.183725+0000","last_fullsized":"2024-04-22T20:45:45.183725+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:09:15.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"261'254","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.218285+0000","last_change":"2024-04-22T19:51:36.569306+0000","last_active":"2024-04-22T20:48:03.218285+0000","last_peered":"2024-04-22T20:48:03.218285+0000","last_clean":"2024-04-22T20:48:03.218285+0000","last_became_active":"2024-04-22T19:51:36.568767+0000","last_became_peered":"2024-04-22T19:51:36.568767+0000","last_unstale":"2024-04-22T20:48:03.218285+0000","last_undegraded":"2024-04-22T20:48:03.218285+0000","last_fullsized":"2024-04-22T20:48:03.218285+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:39.794900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":625,"num_write":569,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"261'124","reported_seq":751,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183078+0000","last_change":"2024-04-22T19:51:28.518594+0000","last_active":"2024-04-22T20:45:40.183078+0000","last_peered":"2024-04-22T20:45:40.183078+0000","last_clean":"2024-04-22T20:45:40.183078+0000","last_became_active":"2024-04-22T19:51:28.518084+0000","last_became_peered":"2024-04-22T19:51:28.518084+0000","last_unstale":"2024-04-22T20:45:40.183078+0000","last_undegraded":"2024-04-22T20:45:40.183078+0000","last_fullsized":"2024-04-22T20:45:40.183078+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:38:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":533,"num_read_kb":444,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.044325+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T20:45:25.044325+0000","last_peered":"2024-04-22T20:45:25.044325+0000","last_clean":"2024-04-22T20:45:25.044325+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T20:45:25.044325+0000","last_undegraded":"2024-04-22T20:45:25.044325+0000","last_fullsized":"2024-04-22T20:45:25.044325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:22.264706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":668,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063186+0000","last_change":"2024-04-22T19:51:12.892628+0000","last_active":"2024-04-22T20:45:25.063186+0000","last_peered":"2024-04-22T20:45:25.063186+0000","last_clean":"2024-04-22T20:45:25.063186+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T20:45:25.063186+0000","last_undegraded":"2024-04-22T20:45:25.063186+0000","last_fullsized":"2024-04-22T20:45:25.063186+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:34.018008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"261'271","reported_seq":914,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.324539+0000","last_change":"2024-04-22T19:51:26.497957+0000","last_active":"2024-04-22T20:47:44.324539+0000","last_peered":"2024-04-22T20:47:44.324539+0000","last_clean":"2024-04-22T20:47:44.324539+0000","last_became_active":"2024-04-22T19:51:26.497608+0000","last_became_peered":"2024-04-22T19:51:26.497608+0000","last_unstale":"2024-04-22T20:47:44.324539+0000","last_undegraded":"2024-04-22T20:47:44.324539+0000","last_fullsized":"2024-04-22T20:47:44.324539+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:02:40.473180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16410029,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":454,"num_read_kb":42686,"num_write":1180,"num_write_kb":37697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189509+0000","last_change":"2024-04-22T19:51:30.524749+0000","last_active":"2024-04-22T20:45:40.189509+0000","last_peered":"2024-04-22T20:45:40.189509+0000","last_clean":"2024-04-22T20:45:40.189509+0000","last_became_active":"2024-04-22T19:51:30.524437+0000","last_became_peered":"2024-04-22T19:51:30.524437+0000","last_unstale":"2024-04-22T20:45:40.189509+0000","last_undegraded":"2024-04-22T20:45:40.189509+0000","last_fullsized":"2024-04-22T20:45:40.189509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:54.055370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"261'1086","reported_seq":2608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.549870+0000","last_change":"2024-04-22T19:51:32.538859+0000","last_active":"2024-04-22T20:48:08.549870+0000","last_peered":"2024-04-22T20:48:08.549870+0000","last_clean":"2024-04-22T20:48:08.549870+0000","last_became_active":"2024-04-22T19:51:32.538560+0000","last_became_peered":"2024-04-22T19:51:32.538560+0000","last_unstale":"2024-04-22T20:48:08.549870+0000","last_undegraded":"2024-04-22T20:48:08.549870+0000","last_fullsized":"2024-04-22T20:48:08.549870+0000","mapping_epoch":26,"log_start":"261'1000","ondisk_log_start":"261'1000","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":1000,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:19.961941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174953,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1739,"num_read_kb":3173,"num_write":1082,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.067953+0000","last_change":"2024-04-22T19:51:34.557341+0000","last_active":"2024-04-22T20:45:55.067953+0000","last_peered":"2024-04-22T20:45:55.067953+0000","last_clean":"2024-04-22T20:45:55.067953+0000","last_became_active":"2024-04-22T19:51:34.557022+0000","last_became_peered":"2024-04-22T19:51:34.557022+0000","last_unstale":"2024-04-22T20:45:55.067953+0000","last_undegraded":"2024-04-22T20:45:55.067953+0000","last_fullsized":"2024-04-22T20:45:55.067953+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:59:49.539365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"261'163","reported_seq":671,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:00.048103+0000","last_change":"2024-04-22T19:51:36.567874+0000","last_active":"2024-04-22T20:46:00.048103+0000","last_peered":"2024-04-22T20:46:00.048103+0000","last_clean":"2024-04-22T20:46:00.048103+0000","last_became_active":"2024-04-22T19:51:36.567603+0000","last_became_peered":"2024-04-22T19:51:36.567603+0000","last_unstale":"2024-04-22T20:46:00.048103+0000","last_undegraded":"2024-04-22T20:46:00.048103+0000","last_fullsized":"2024-04-22T20:46:00.048103+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:34:18.236728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":641,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":222,"num_read_kb":216,"num_write":373,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"261'364","reported_seq":998,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:32.355984+0000","last_change":"2024-04-22T19:51:28.512889+0000","last_active":"2024-04-22T20:46:32.355984+0000","last_peered":"2024-04-22T20:46:32.355984+0000","last_clean":"2024-04-22T20:46:32.355984+0000","last_became_active":"2024-04-22T19:51:28.512561+0000","last_became_peered":"2024-04-22T19:51:28.512561+0000","last_unstale":"2024-04-22T20:46:32.355984+0000","last_undegraded":"2024-04-22T20:46:32.355984+0000","last_fullsized":"2024-04-22T20:46:32.355984+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:13.130467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1455,"num_read_kb":1128,"num_write":627,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"261'324","reported_seq":1041,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183045+0000","last_change":"2024-04-22T19:51:26.500958+0000","last_active":"2024-04-22T20:45:40.183045+0000","last_peered":"2024-04-22T20:45:40.183045+0000","last_clean":"2024-04-22T20:45:40.183045+0000","last_became_active":"2024-04-22T19:51:26.498178+0000","last_became_peered":"2024-04-22T19:51:26.498178+0000","last_unstale":"2024-04-22T20:45:40.183045+0000","last_undegraded":"2024-04-22T20:45:40.183045+0000","last_fullsized":"2024-04-22T20:45:40.183045+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:54.358145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14073286,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":653,"num_read_kb":47025,"num_write":1473,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.155991+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T20:45:25.155991+0000","last_peered":"2024-04-22T20:45:25.155991+0000","last_clean":"2024-04-22T20:45:25.155991+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T20:45:25.155991+0000","last_undegraded":"2024-04-22T20:45:25.155991+0000","last_fullsized":"2024-04-22T20:45:25.155991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157032+0000","last_change":"2024-04-22T19:51:30.523537+0000","last_active":"2024-04-22T20:45:35.157032+0000","last_peered":"2024-04-22T20:45:35.157032+0000","last_clean":"2024-04-22T20:45:35.157032+0000","last_became_active":"2024-04-22T19:51:30.523242+0000","last_became_peered":"2024-04-22T19:51:30.523242+0000","last_unstale":"2024-04-22T20:45:35.157032+0000","last_undegraded":"2024-04-22T20:45:35.157032+0000","last_fullsized":"2024-04-22T20:45:35.157032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:21.847987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"261'692","reported_seq":1402,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.510075+0000","last_change":"2024-04-22T19:51:33.248507+0000","last_active":"2024-04-22T20:48:08.510075+0000","last_peered":"2024-04-22T20:48:08.510075+0000","last_clean":"2024-04-22T20:48:08.510075+0000","last_became_active":"2024-04-22T19:51:33.248144+0000","last_became_peered":"2024-04-22T19:51:33.248144+0000","last_unstale":"2024-04-22T20:48:08.510075+0000","last_undegraded":"2024-04-22T20:48:08.510075+0000","last_fullsized":"2024-04-22T20:48:08.510075+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":600,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:02:24.314016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":153222,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1224,"num_read_kb":2558,"num_write":1000,"num_write_kb":811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.158258+0000","last_change":"2024-04-22T19:51:34.562290+0000","last_active":"2024-04-22T20:45:45.158258+0000","last_peered":"2024-04-22T20:45:45.158258+0000","last_clean":"2024-04-22T20:45:45.158258+0000","last_became_active":"2024-04-22T19:51:34.562001+0000","last_became_peered":"2024-04-22T19:51:34.562001+0000","last_unstale":"2024-04-22T20:45:45.158258+0000","last_undegraded":"2024-04-22T20:45:45.158258+0000","last_fullsized":"2024-04-22T20:45:45.158258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:30:17.324055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"261'105","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:02.644513+0000","last_change":"2024-04-22T19:51:36.570607+0000","last_active":"2024-04-22T20:48:02.644513+0000","last_peered":"2024-04-22T20:48:02.644513+0000","last_clean":"2024-04-22T20:48:02.644513+0000","last_became_active":"2024-04-22T19:51:36.570134+0000","last_became_peered":"2024-04-22T19:51:36.570134+0000","last_unstale":"2024-04-22T20:48:02.644513+0000","last_undegraded":"2024-04-22T20:48:02.644513+0000","last_fullsized":"2024-04-22T20:48:02.644513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:44.632753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":256,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"261'101","reported_seq":688,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.244225+0000","last_change":"2024-04-22T19:51:28.516624+0000","last_active":"2024-04-22T20:47:05.244225+0000","last_peered":"2024-04-22T20:47:05.244225+0000","last_clean":"2024-04-22T20:47:05.244225+0000","last_became_active":"2024-04-22T19:51:28.516178+0000","last_became_peered":"2024-04-22T19:51:28.516178+0000","last_unstale":"2024-04-22T20:47:05.244225+0000","last_undegraded":"2024-04-22T20:47:05.244225+0000","last_fullsized":"2024-04-22T20:47:05.244225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:27:27.066102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":337,"num_write":140,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"261'295","reported_seq":990,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.157770+0000","last_change":"2024-04-22T19:51:26.503699+0000","last_active":"2024-04-22T20:45:40.157770+0000","last_peered":"2024-04-22T20:45:40.157770+0000","last_clean":"2024-04-22T20:45:40.157770+0000","last_became_active":"2024-04-22T19:51:26.502765+0000","last_became_peered":"2024-04-22T19:51:26.502765+0000","last_unstale":"2024-04-22T20:45:40.157770+0000","last_undegraded":"2024-04-22T20:45:40.157770+0000","last_fullsized":"2024-04-22T20:45:40.157770+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:12:35.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14551057,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":509,"num_read_kb":127053,"num_write":1319,"num_write_kb":34012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":518,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.180533+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T20:45:35.180533+0000","last_peered":"2024-04-22T20:45:35.180533+0000","last_clean":"2024-04-22T20:45:35.180533+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T20:45:35.180533+0000","last_undegraded":"2024-04-22T20:45:35.180533+0000","last_fullsized":"2024-04-22T20:45:35.180533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181286+0000","last_change":"2024-04-22T19:51:30.529790+0000","last_active":"2024-04-22T20:45:40.181286+0000","last_peered":"2024-04-22T20:45:40.181286+0000","last_clean":"2024-04-22T20:45:40.181286+0000","last_became_active":"2024-04-22T19:51:30.529437+0000","last_became_peered":"2024-04-22T19:51:30.529437+0000","last_unstale":"2024-04-22T20:45:40.181286+0000","last_undegraded":"2024-04-22T20:45:40.181286+0000","last_fullsized":"2024-04-22T20:45:40.181286+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:46:59.053426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"261'110","reported_seq":651,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182039+0000","last_change":"2024-04-22T19:51:32.534640+0000","last_active":"2024-04-22T20:45:45.182039+0000","last_peered":"2024-04-22T20:45:45.182039+0000","last_clean":"2024-04-22T20:45:45.182039+0000","last_became_active":"2024-04-22T19:51:32.534280+0000","last_became_peered":"2024-04-22T19:51:32.534280+0000","last_unstale":"2024-04-22T20:45:45.182039+0000","last_undegraded":"2024-04-22T20:45:45.182039+0000","last_fullsized":"2024-04-22T20:45:45.182039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:52.620330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"261'531","reported_seq":2151,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.555610+0000","last_change":"2024-04-22T19:51:34.557012+0000","last_active":"2024-04-22T20:48:08.555610+0000","last_peered":"2024-04-22T20:48:08.555610+0000","last_clean":"2024-04-22T20:48:08.555610+0000","last_became_active":"2024-04-22T19:51:34.556673+0000","last_became_peered":"2024-04-22T19:51:34.556673+0000","last_unstale":"2024-04-22T20:48:08.555610+0000","last_undegraded":"2024-04-22T20:48:08.555610+0000","last_fullsized":"2024-04-22T20:48:08.555610+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":500,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:18.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"261'149","reported_seq":639,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182998+0000","last_change":"2024-04-22T19:51:36.575637+0000","last_active":"2024-04-22T20:45:40.182998+0000","last_peered":"2024-04-22T20:45:40.182998+0000","last_clean":"2024-04-22T20:45:40.182998+0000","last_became_active":"2024-04-22T19:51:36.575351+0000","last_became_peered":"2024-04-22T19:51:36.575351+0000","last_unstale":"2024-04-22T20:45:40.182998+0000","last_undegraded":"2024-04-22T20:45:40.182998+0000","last_fullsized":"2024-04-22T20:45:40.182998+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:38:03.020774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":80,"num_write":319,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"261'146","reported_seq":771,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182955+0000","last_change":"2024-04-22T19:51:28.514202+0000","last_active":"2024-04-22T20:45:40.182955+0000","last_peered":"2024-04-22T20:45:40.182955+0000","last_clean":"2024-04-22T20:45:40.182955+0000","last_became_active":"2024-04-22T19:51:28.513940+0000","last_became_peered":"2024-04-22T19:51:28.513940+0000","last_unstale":"2024-04-22T20:45:40.182955+0000","last_undegraded":"2024-04-22T20:45:40.182955+0000","last_fullsized":"2024-04-22T20:45:40.182955+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:14:36.426996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":550,"num_write":219,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"261'276","reported_seq":914,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:28.475475+0000","last_change":"2024-04-22T19:51:26.506454+0000","last_active":"2024-04-22T20:46:28.475475+0000","last_peered":"2024-04-22T20:46:28.475475+0000","last_clean":"2024-04-22T20:46:28.475475+0000","last_became_active":"2024-04-22T19:51:26.505842+0000","last_became_peered":"2024-04-22T19:51:26.505842+0000","last_unstale":"2024-04-22T20:46:28.475475+0000","last_undegraded":"2024-04-22T20:46:28.475475+0000","last_fullsized":"2024-04-22T20:46:28.475475+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:24:40.890650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10581571,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":407,"num_read_kb":22189,"num_write":1035,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314053+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T20:45:20.314053+0000","last_peered":"2024-04-22T20:45:20.314053+0000","last_clean":"2024-04-22T20:45:20.314053+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T20:45:20.314053+0000","last_undegraded":"2024-04-22T20:45:20.314053+0000","last_fullsized":"2024-04-22T20:45:20.314053+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:01.430732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317090+0000","last_change":"2024-04-22T19:51:30.526076+0000","last_active":"2024-04-22T20:45:40.317090+0000","last_peered":"2024-04-22T20:45:40.317090+0000","last_clean":"2024-04-22T20:45:40.317090+0000","last_became_active":"2024-04-22T19:51:30.525777+0000","last_became_peered":"2024-04-22T19:51:30.525777+0000","last_unstale":"2024-04-22T20:45:40.317090+0000","last_undegraded":"2024-04-22T20:45:40.317090+0000","last_fullsized":"2024-04-22T20:45:40.317090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:18.779258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"261'179","reported_seq":746,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.046459+0000","last_change":"2024-04-22T19:51:33.248819+0000","last_active":"2024-04-22T20:45:45.046459+0000","last_peered":"2024-04-22T20:45:45.046459+0000","last_clean":"2024-04-22T20:45:45.046459+0000","last_became_active":"2024-04-22T19:51:33.248354+0000","last_became_peered":"2024-04-22T19:51:33.248354+0000","last_unstale":"2024-04-22T20:45:45.046459+0000","last_undegraded":"2024-04-22T20:45:45.046459+0000","last_fullsized":"2024-04-22T20:45:45.046459+0000","mapping_epoch":26,"log_start":"258'100","ondisk_log_start":"258'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:02:04.601587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":222,"num_read_kb":211,"num_write":148,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"261'544","reported_seq":4419,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.551803+0000","last_change":"2024-04-22T19:51:34.561431+0000","last_active":"2024-04-22T20:48:08.551803+0000","last_peered":"2024-04-22T20:48:08.551803+0000","last_clean":"2024-04-22T20:48:08.551803+0000","last_became_active":"2024-04-22T19:51:34.561206+0000","last_became_peered":"2024-04-22T19:51:34.561206+0000","last_unstale":"2024-04-22T20:48:08.551803+0000","last_undegraded":"2024-04-22T20:48:08.551803+0000","last_fullsized":"2024-04-22T20:48:08.551803+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":500,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:11:06.654339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"261'182","reported_seq":679,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.174164+0000","last_change":"2024-04-22T19:51:36.569612+0000","last_active":"2024-04-22T20:48:00.174164+0000","last_peered":"2024-04-22T20:48:00.174164+0000","last_clean":"2024-04-22T20:48:00.174164+0000","last_became_active":"2024-04-22T19:51:36.569338+0000","last_became_peered":"2024-04-22T19:51:36.569338+0000","last_unstale":"2024-04-22T20:48:00.174164+0000","last_undegraded":"2024-04-22T20:48:00.174164+0000","last_fullsized":"2024-04-22T20:48:00.174164+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:23:38.495469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":109,"num_write":422,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"261'109","reported_seq":704,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045900+0000","last_change":"2024-04-22T19:51:28.512614+0000","last_active":"2024-04-22T20:45:40.045900+0000","last_peered":"2024-04-22T20:45:40.045900+0000","last_clean":"2024-04-22T20:45:40.045900+0000","last_became_active":"2024-04-22T19:51:28.512337+0000","last_became_peered":"2024-04-22T19:51:28.512337+0000","last_unstale":"2024-04-22T20:45:40.045900+0000","last_undegraded":"2024-04-22T20:45:40.045900+0000","last_fullsized":"2024-04-22T20:45:40.045900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:06:34.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":434,"num_read_kb":362,"num_write":158,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'260","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.190983+0000","last_change":"2024-04-22T19:51:26.508311+0000","last_active":"2024-04-22T20:45:50.190983+0000","last_peered":"2024-04-22T20:45:50.190983+0000","last_clean":"2024-04-22T20:45:50.190983+0000","last_became_active":"2024-04-22T19:51:26.507804+0000","last_became_peered":"2024-04-22T19:51:26.507804+0000","last_unstale":"2024-04-22T20:45:50.190983+0000","last_undegraded":"2024-04-22T20:45:50.190983+0000","last_fullsized":"2024-04-22T20:45:50.190983+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:56:54.837990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111886,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":452,"num_read_kb":11012,"num_write":1175,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314068+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T20:45:20.314068+0000","last_peered":"2024-04-22T20:45:20.314068+0000","last_clean":"2024-04-22T20:45:20.314068+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T20:45:20.314068+0000","last_undegraded":"2024-04-22T20:45:20.314068+0000","last_fullsized":"2024-04-22T20:45:20.314068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:23.800559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182974+0000","last_change":"2024-04-22T19:51:30.527109+0000","last_active":"2024-04-22T20:45:40.182974+0000","last_peered":"2024-04-22T20:45:40.182974+0000","last_clean":"2024-04-22T20:45:40.182974+0000","last_became_active":"2024-04-22T19:51:30.526837+0000","last_became_peered":"2024-04-22T19:51:30.526837+0000","last_unstale":"2024-04-22T20:45:40.182974+0000","last_undegraded":"2024-04-22T20:45:40.182974+0000","last_fullsized":"2024-04-22T20:45:40.182974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:13.442240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"261'532","reported_seq":1192,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.513945+0000","last_change":"2024-04-22T19:51:32.542607+0000","last_active":"2024-04-22T20:48:08.513945+0000","last_peered":"2024-04-22T20:48:08.513945+0000","last_clean":"2024-04-22T20:48:08.513945+0000","last_became_active":"2024-04-22T19:51:32.542344+0000","last_became_peered":"2024-04-22T19:51:32.542344+0000","last_unstale":"2024-04-22T20:48:08.513945+0000","last_undegraded":"2024-04-22T20:48:08.513945+0000","last_fullsized":"2024-04-22T20:48:08.513945+0000","mapping_epoch":26,"log_start":"261'500","ondisk_log_start":"261'500","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":500,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:56:47.214088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115148,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":920,"num_read_kb":1851,"num_write":780,"num_write_kb":630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.181987+0000","last_change":"2024-04-22T19:51:34.551264+0000","last_active":"2024-04-22T20:45:45.181987+0000","last_peered":"2024-04-22T20:45:45.181987+0000","last_clean":"2024-04-22T20:45:45.181987+0000","last_became_active":"2024-04-22T19:51:34.550951+0000","last_became_peered":"2024-04-22T19:51:34.550951+0000","last_unstale":"2024-04-22T20:45:45.181987+0000","last_undegraded":"2024-04-22T20:45:45.181987+0000","last_fullsized":"2024-04-22T20:45:45.181987+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:34:20.504558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"261'112","reported_seq":617,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.198696+0000","last_change":"2024-04-22T19:51:36.573230+0000","last_active":"2024-04-22T20:47:55.198696+0000","last_peered":"2024-04-22T20:47:55.198696+0000","last_clean":"2024-04-22T20:47:55.198696+0000","last_became_active":"2024-04-22T19:51:36.572976+0000","last_became_peered":"2024-04-22T19:51:36.572976+0000","last_unstale":"2024-04-22T20:47:55.198696+0000","last_undegraded":"2024-04-22T20:47:55.198696+0000","last_fullsized":"2024-04-22T20:47:55.198696+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:14.048628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":120,"num_write":270,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"261'130","reported_seq":681,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.160770+0000","last_change":"2024-04-22T19:51:36.574914+0000","last_active":"2024-04-22T20:48:03.160770+0000","last_peered":"2024-04-22T20:48:03.160770+0000","last_clean":"2024-04-22T20:48:03.160770+0000","last_became_active":"2024-04-22T19:51:36.574238+0000","last_became_peered":"2024-04-22T19:51:36.574238+0000","last_unstale":"2024-04-22T20:48:03.160770+0000","last_undegraded":"2024-04-22T20:48:03.160770+0000","last_fullsized":"2024-04-22T20:48:03.160770+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:09.269509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":193,"num_read_kb":192,"num_write":291,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"261'123","reported_seq":739,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176757+0000","last_change":"2024-04-22T19:51:28.515646+0000","last_active":"2024-04-22T20:45:10.176757+0000","last_peered":"2024-04-22T20:45:10.176757+0000","last_clean":"2024-04-22T20:45:10.176757+0000","last_became_active":"2024-04-22T19:51:28.515334+0000","last_became_peered":"2024-04-22T19:51:28.515334+0000","last_unstale":"2024-04-22T20:45:10.176757+0000","last_undegraded":"2024-04-22T20:45:10.176757+0000","last_fullsized":"2024-04-22T20:45:10.176757+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:47:44.805680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":424,"num_write":184,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"261'258","reported_seq":971,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.178176+0000","last_change":"2024-04-22T19:51:26.506527+0000","last_active":"2024-04-22T20:45:20.178176+0000","last_peered":"2024-04-22T20:45:20.178176+0000","last_clean":"2024-04-22T20:45:20.178176+0000","last_became_active":"2024-04-22T19:51:26.506150+0000","last_became_peered":"2024-04-22T19:51:26.506150+0000","last_unstale":"2024-04-22T20:45:20.178176+0000","last_undegraded":"2024-04-22T20:45:20.178176+0000","last_fullsized":"2024-04-22T20:45:20.178176+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:28.117664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7665617,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":650,"num_read_kb":35400,"num_write":1047,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.042550+0000","last_change":"2024-04-22T19:51:30.523906+0000","last_active":"2024-04-22T20:45:10.042550+0000","last_peered":"2024-04-22T20:45:10.042550+0000","last_clean":"2024-04-22T20:45:10.042550+0000","last_became_active":"2024-04-22T19:51:30.522903+0000","last_became_peered":"2024-04-22T19:51:30.522903+0000","last_unstale":"2024-04-22T20:45:10.042550+0000","last_undegraded":"2024-04-22T20:45:10.042550+0000","last_fullsized":"2024-04-22T20:45:10.042550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:12.331782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"261'291","reported_seq":870,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171073+0000","last_change":"2024-04-22T19:51:32.539884+0000","last_active":"2024-04-22T20:47:35.171073+0000","last_peered":"2024-04-22T20:47:35.171073+0000","last_clean":"2024-04-22T20:47:35.171073+0000","last_became_active":"2024-04-22T19:51:32.539631+0000","last_became_peered":"2024-04-22T19:51:32.539631+0000","last_unstale":"2024-04-22T20:47:35.171073+0000","last_undegraded":"2024-04-22T20:47:35.171073+0000","last_fullsized":"2024-04-22T20:47:35.171073+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:00:28.566624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59572,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":482,"num_read_kb":991,"num_write":398,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154587+0000","last_change":"2024-04-22T19:51:34.556940+0000","last_active":"2024-04-22T20:45:15.154587+0000","last_peered":"2024-04-22T20:45:15.154587+0000","last_clean":"2024-04-22T20:45:15.154587+0000","last_became_active":"2024-04-22T19:51:34.556650+0000","last_became_peered":"2024-04-22T19:51:34.556650+0000","last_unstale":"2024-04-22T20:45:15.154587+0000","last_undegraded":"2024-04-22T20:45:15.154587+0000","last_fullsized":"2024-04-22T20:45:15.154587+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:47:48.478954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"261'99","reported_seq":599,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.172452+0000","last_change":"2024-04-22T19:51:36.567251+0000","last_active":"2024-04-22T20:47:45.172452+0000","last_peered":"2024-04-22T20:47:45.172452+0000","last_clean":"2024-04-22T20:47:45.172452+0000","last_became_active":"2024-04-22T19:51:36.566980+0000","last_became_peered":"2024-04-22T19:51:36.566980+0000","last_unstale":"2024-04-22T20:47:45.172452+0000","last_undegraded":"2024-04-22T20:47:45.172452+0000","last_fullsized":"2024-04-22T20:47:45.172452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:01:00.638641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":235,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"261'130","reported_seq":734,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.042558+0000","last_change":"2024-04-22T19:51:28.517914+0000","last_active":"2024-04-22T20:45:10.042558+0000","last_peered":"2024-04-22T20:45:10.042558+0000","last_clean":"2024-04-22T20:45:10.042558+0000","last_became_active":"2024-04-22T19:51:28.517675+0000","last_became_peered":"2024-04-22T19:51:28.517675+0000","last_unstale":"2024-04-22T20:45:10.042558+0000","last_undegraded":"2024-04-22T20:45:10.042558+0000","last_fullsized":"2024-04-22T20:45:10.042558+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:18:10.255589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":543,"num_read_kb":446,"num_write":195,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"261'276","reported_seq":976,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:16.477341+0000","last_change":"2024-04-22T19:51:26.502622+0000","last_active":"2024-04-22T20:45:16.477341+0000","last_peered":"2024-04-22T20:45:16.477341+0000","last_clean":"2024-04-22T20:45:16.477341+0000","last_became_active":"2024-04-22T19:51:26.501897+0000","last_became_peered":"2024-04-22T19:51:26.501897+0000","last_unstale":"2024-04-22T20:45:16.477341+0000","last_undegraded":"2024-04-22T20:45:16.477341+0000","last_fullsized":"2024-04-22T20:45:16.477341+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:33.272475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4946722,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":548,"num_read_kb":41436,"num_write":1081,"num_write_kb":29344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.154110+0000","last_change":"2024-04-22T19:51:30.524665+0000","last_active":"2024-04-22T20:45:10.154110+0000","last_peered":"2024-04-22T20:45:10.154110+0000","last_clean":"2024-04-22T20:45:10.154110+0000","last_became_active":"2024-04-22T19:51:30.524388+0000","last_became_peered":"2024-04-22T19:51:30.524388+0000","last_unstale":"2024-04-22T20:45:10.154110+0000","last_undegraded":"2024-04-22T20:45:10.154110+0000","last_fullsized":"2024-04-22T20:45:10.154110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:58.393800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"261'49","reported_seq":553,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179874+0000","last_change":"2024-04-22T19:51:32.539546+0000","last_active":"2024-04-22T20:45:15.179874+0000","last_peered":"2024-04-22T20:45:15.179874+0000","last_clean":"2024-04-22T20:45:15.179874+0000","last_became_active":"2024-04-22T19:51:32.539260+0000","last_became_peered":"2024-04-22T19:51:32.539260+0000","last_unstale":"2024-04-22T20:45:15.179874+0000","last_undegraded":"2024-04-22T20:45:15.179874+0000","last_fullsized":"2024-04-22T20:45:15.179874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:13:32.092655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":36,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.061461+0000","last_change":"2024-04-22T19:51:34.551584+0000","last_active":"2024-04-22T20:45:15.061461+0000","last_peered":"2024-04-22T20:45:15.061461+0000","last_clean":"2024-04-22T20:45:15.061461+0000","last_became_active":"2024-04-22T19:51:34.551294+0000","last_became_peered":"2024-04-22T19:51:34.551294+0000","last_unstale":"2024-04-22T20:45:15.061461+0000","last_undegraded":"2024-04-22T20:45:15.061461+0000","last_fullsized":"2024-04-22T20:45:15.061461+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:53:58.560478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"261'201","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.005045+0000","last_change":"2024-04-22T19:51:36.574227+0000","last_active":"2024-04-22T20:48:03.005045+0000","last_peered":"2024-04-22T20:48:03.005045+0000","last_clean":"2024-04-22T20:48:03.005045+0000","last_became_active":"2024-04-22T19:51:36.573936+0000","last_became_peered":"2024-04-22T19:51:36.573936+0000","last_unstale":"2024-04-22T20:48:03.005045+0000","last_undegraded":"2024-04-22T20:48:03.005045+0000","last_fullsized":"2024-04-22T20:48:03.005045+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:05:17.804766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":284,"num_write":451,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"261'231","reported_seq":917,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.154075+0000","last_change":"2024-04-22T19:51:28.508566+0000","last_active":"2024-04-22T20:45:10.154075+0000","last_peered":"2024-04-22T20:45:10.154075+0000","last_clean":"2024-04-22T20:45:10.154075+0000","last_became_active":"2024-04-22T19:51:28.508307+0000","last_became_peered":"2024-04-22T19:51:28.508307+0000","last_unstale":"2024-04-22T20:45:10.154075+0000","last_undegraded":"2024-04-22T20:45:10.154075+0000","last_fullsized":"2024-04-22T20:45:10.154075+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:48.292329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":959,"num_read_kb":784,"num_write":348,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"261'268","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180692+0000","last_change":"2024-04-22T19:51:26.501920+0000","last_active":"2024-04-22T20:44:40.180692+0000","last_peered":"2024-04-22T20:44:40.180692+0000","last_clean":"2024-04-22T20:44:40.180692+0000","last_became_active":"2024-04-22T19:51:26.501477+0000","last_became_peered":"2024-04-22T19:51:26.501477+0000","last_unstale":"2024-04-22T20:44:40.180692+0000","last_undegraded":"2024-04-22T20:44:40.180692+0000","last_fullsized":"2024-04-22T20:44:40.180692+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:06.904602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16045811,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":422,"num_read_kb":46522,"num_write":1225,"num_write_kb":35021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060725+0000","last_change":"2024-04-22T19:51:30.527405+0000","last_active":"2024-04-22T20:45:10.060725+0000","last_peered":"2024-04-22T20:45:10.060725+0000","last_clean":"2024-04-22T20:45:10.060725+0000","last_became_active":"2024-04-22T19:51:30.527076+0000","last_became_peered":"2024-04-22T19:51:30.527076+0000","last_unstale":"2024-04-22T20:45:10.060725+0000","last_undegraded":"2024-04-22T20:45:10.060725+0000","last_fullsized":"2024-04-22T20:45:10.060725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:38.558657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"261'297","reported_seq":892,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171120+0000","last_change":"2024-04-22T19:51:32.542586+0000","last_active":"2024-04-22T20:47:35.171120+0000","last_peered":"2024-04-22T20:47:35.171120+0000","last_clean":"2024-04-22T20:47:35.171120+0000","last_became_active":"2024-04-22T19:51:32.542262+0000","last_became_peered":"2024-04-22T19:51:32.542262+0000","last_unstale":"2024-04-22T20:47:35.171120+0000","last_undegraded":"2024-04-22T20:47:35.171120+0000","last_fullsized":"2024-04-22T20:47:35.171120+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:52:10.014200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55208,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":478,"num_read_kb":917,"num_write":378,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.313200+0000","last_change":"2024-04-22T19:51:34.554113+0000","last_active":"2024-04-22T20:45:15.313200+0000","last_peered":"2024-04-22T20:45:15.313200+0000","last_clean":"2024-04-22T20:45:15.313200+0000","last_became_active":"2024-04-22T19:51:34.553691+0000","last_became_peered":"2024-04-22T19:51:34.553691+0000","last_unstale":"2024-04-22T20:45:15.313200+0000","last_undegraded":"2024-04-22T20:45:15.313200+0000","last_fullsized":"2024-04-22T20:45:15.313200+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:08:07.000668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"261'135","reported_seq":616,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:10.176306+0000","last_change":"2024-04-22T19:51:36.571557+0000","last_active":"2024-04-22T20:44:10.176306+0000","last_peered":"2024-04-22T20:44:10.176306+0000","last_clean":"2024-04-22T20:44:10.176306+0000","last_became_active":"2024-04-22T19:51:36.571290+0000","last_became_peered":"2024-04-22T19:51:36.571290+0000","last_unstale":"2024-04-22T20:44:10.176306+0000","last_undegraded":"2024-04-22T20:44:10.176306+0000","last_fullsized":"2024-04-22T20:44:10.176306+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:42:28.669764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":111,"num_write":334,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"261'185","reported_seq":854,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:01.148987+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:47:01.148987+0000","last_peered":"2024-04-22T20:47:01.148987+0000","last_clean":"2024-04-22T20:47:01.148987+0000","last_became_active":"2024-04-22T19:51:28.506615+0000","last_became_peered":"2024-04-22T19:51:28.506615+0000","last_unstale":"2024-04-22T20:47:01.148987+0000","last_undegraded":"2024-04-22T20:47:01.148987+0000","last_fullsized":"2024-04-22T20:47:01.148987+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:25.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":757,"num_read_kb":623,"num_write":269,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'270","reported_seq":911,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:43.414545+0000","last_change":"2024-04-22T19:51:26.503480+0000","last_active":"2024-04-22T20:44:43.414545+0000","last_peered":"2024-04-22T20:44:43.414545+0000","last_clean":"2024-04-22T20:44:43.414545+0000","last_became_active":"2024-04-22T19:51:26.502938+0000","last_became_peered":"2024-04-22T19:51:26.502938+0000","last_unstale":"2024-04-22T20:44:43.414545+0000","last_undegraded":"2024-04-22T20:44:43.414545+0000","last_fullsized":"2024-04-22T20:44:43.414545+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:25.122842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11884442,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":410,"num_read_kb":28738,"num_write":1201,"num_write_kb":45425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.312594+0000","last_change":"2024-04-22T19:51:30.527273+0000","last_active":"2024-04-22T20:45:10.312594+0000","last_peered":"2024-04-22T20:45:10.312594+0000","last_clean":"2024-04-22T20:45:10.312594+0000","last_became_active":"2024-04-22T19:51:30.526994+0000","last_became_peered":"2024-04-22T19:51:30.526994+0000","last_unstale":"2024-04-22T20:45:10.312594+0000","last_undegraded":"2024-04-22T20:45:10.312594+0000","last_fullsized":"2024-04-22T20:45:10.312594+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:05:35.522811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"261'246","reported_seq":812,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.082501+0000","last_change":"2024-04-22T19:51:32.538193+0000","last_active":"2024-04-22T20:47:35.082501+0000","last_peered":"2024-04-22T20:47:35.082501+0000","last_clean":"2024-04-22T20:47:35.082501+0000","last_became_active":"2024-04-22T19:51:32.537875+0000","last_became_peered":"2024-04-22T19:51:32.537875+0000","last_unstale":"2024-04-22T20:47:35.082501+0000","last_undegraded":"2024-04-22T20:47:35.082501+0000","last_fullsized":"2024-04-22T20:47:35.082501+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:43.382645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50977,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":409,"num_read_kb":848,"num_write":332,"num_write_kb":266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179968+0000","last_change":"2024-04-22T19:51:34.554251+0000","last_active":"2024-04-22T20:45:15.179968+0000","last_peered":"2024-04-22T20:45:15.179968+0000","last_clean":"2024-04-22T20:45:15.179968+0000","last_became_active":"2024-04-22T19:51:34.553954+0000","last_became_peered":"2024-04-22T19:51:34.553954+0000","last_unstale":"2024-04-22T20:45:15.179968+0000","last_undegraded":"2024-04-22T20:45:15.179968+0000","last_fullsized":"2024-04-22T20:45:15.179968+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:25:45.427696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"261'192","reported_seq":830,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317164+0000","last_change":"2024-04-22T19:51:28.514880+0000","last_active":"2024-04-22T20:45:40.317164+0000","last_peered":"2024-04-22T20:45:40.317164+0000","last_clean":"2024-04-22T20:45:40.317164+0000","last_became_active":"2024-04-22T19:51:28.514593+0000","last_became_peered":"2024-04-22T19:51:28.514593+0000","last_unstale":"2024-04-22T20:45:40.317164+0000","last_undegraded":"2024-04-22T20:45:40.317164+0000","last_fullsized":"2024-04-22T20:45:40.317164+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:10:57.618723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":788,"num_read_kb":638,"num_write":274,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"261'282","reported_seq":944,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:16.752867+0000","last_change":"2024-04-22T19:51:26.502507+0000","last_active":"2024-04-22T20:47:16.752867+0000","last_peered":"2024-04-22T20:47:16.752867+0000","last_clean":"2024-04-22T20:47:16.752867+0000","last_became_active":"2024-04-22T19:51:26.501812+0000","last_became_peered":"2024-04-22T19:51:26.501812+0000","last_unstale":"2024-04-22T20:47:16.752867+0000","last_undegraded":"2024-04-22T20:47:16.752867+0000","last_fullsized":"2024-04-22T20:47:16.752867+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:47:54.670392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1914449,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":492,"num_read_kb":55834,"num_write":1102,"num_write_kb":35884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183098+0000","last_change":"2024-04-22T19:51:30.525750+0000","last_active":"2024-04-22T20:45:40.183098+0000","last_peered":"2024-04-22T20:45:40.183098+0000","last_clean":"2024-04-22T20:45:40.183098+0000","last_became_active":"2024-04-22T19:51:30.525450+0000","last_became_peered":"2024-04-22T19:51:30.525450+0000","last_unstale":"2024-04-22T20:45:40.183098+0000","last_undegraded":"2024-04-22T20:45:40.183098+0000","last_fullsized":"2024-04-22T20:45:40.183098+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:32.196222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"261'474","reported_seq":1092,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.506230+0000","last_change":"2024-04-22T19:51:32.541007+0000","last_active":"2024-04-22T20:48:08.506230+0000","last_peered":"2024-04-22T20:48:08.506230+0000","last_clean":"2024-04-22T20:48:08.506230+0000","last_became_active":"2024-04-22T19:51:32.540755+0000","last_became_peered":"2024-04-22T19:51:32.540755+0000","last_unstale":"2024-04-22T20:48:08.506230+0000","last_undegraded":"2024-04-22T20:48:08.506230+0000","last_fullsized":"2024-04-22T20:48:08.506230+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":400,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:18.375444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112197,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":849,"num_read_kb":1808,"num_write":744,"num_write_kb":667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":482,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.175603+0000","last_change":"2024-04-22T19:51:34.553700+0000","last_active":"2024-04-22T20:44:05.175603+0000","last_peered":"2024-04-22T20:44:05.175603+0000","last_clean":"2024-04-22T20:44:05.175603+0000","last_became_active":"2024-04-22T19:51:34.553363+0000","last_became_peered":"2024-04-22T19:51:34.553363+0000","last_unstale":"2024-04-22T20:44:05.175603+0000","last_undegraded":"2024-04-22T20:44:05.175603+0000","last_fullsized":"2024-04-22T20:44:05.175603+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:13:11.618765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"261'159","reported_seq":673,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:02.819260+0000","last_change":"2024-04-22T19:51:36.569083+0000","last_active":"2024-04-22T20:48:02.819260+0000","last_peered":"2024-04-22T20:48:02.819260+0000","last_clean":"2024-04-22T20:48:02.819260+0000","last_became_active":"2024-04-22T19:51:36.568373+0000","last_became_peered":"2024-04-22T19:51:36.568373+0000","last_unstale":"2024-04-22T20:48:02.819260+0000","last_undegraded":"2024-04-22T20:48:02.819260+0000","last_fullsized":"2024-04-22T20:48:02.819260+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:00:29.149286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":213,"num_write":373,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"261'100","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:11.966474+0000","last_change":"2024-04-22T19:51:28.510922+0000","last_active":"2024-04-22T20:47:11.966474+0000","last_peered":"2024-04-22T20:47:11.966474+0000","last_clean":"2024-04-22T20:47:11.966474+0000","last_became_active":"2024-04-22T19:51:28.510629+0000","last_became_peered":"2024-04-22T19:51:28.510629+0000","last_unstale":"2024-04-22T20:47:11.966474+0000","last_undegraded":"2024-04-22T20:47:11.966474+0000","last_fullsized":"2024-04-22T20:47:11.966474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:55:55.876914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":435,"num_read_kb":357,"num_write":161,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"261'250","reported_seq":883,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:14.134249+0000","last_change":"2024-04-22T19:51:26.497497+0000","last_active":"2024-04-22T20:47:14.134249+0000","last_peered":"2024-04-22T20:47:14.134249+0000","last_clean":"2024-04-22T20:47:14.134249+0000","last_became_active":"2024-04-22T19:51:26.497165+0000","last_became_peered":"2024-04-22T19:51:26.497165+0000","last_unstale":"2024-04-22T20:47:14.134249+0000","last_undegraded":"2024-04-22T20:47:14.134249+0000","last_fullsized":"2024-04-22T20:47:14.134249+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:31.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13593714,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":438,"num_read_kb":39207,"num_write":1067,"num_write_kb":42297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317184+0000","last_change":"2024-04-22T19:51:30.526861+0000","last_active":"2024-04-22T20:45:40.317184+0000","last_peered":"2024-04-22T20:45:40.317184+0000","last_clean":"2024-04-22T20:45:40.317184+0000","last_became_active":"2024-04-22T19:51:30.526597+0000","last_became_peered":"2024-04-22T19:51:30.526597+0000","last_unstale":"2024-04-22T20:45:40.317184+0000","last_undegraded":"2024-04-22T20:45:40.317184+0000","last_fullsized":"2024-04-22T20:45:40.317184+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:24.696235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"261'306","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.074224+0000","last_change":"2024-04-22T19:51:32.537791+0000","last_active":"2024-04-22T20:46:35.074224+0000","last_peered":"2024-04-22T20:46:35.074224+0000","last_clean":"2024-04-22T20:46:35.074224+0000","last_became_active":"2024-04-22T19:51:32.537416+0000","last_became_peered":"2024-04-22T19:51:32.537416+0000","last_unstale":"2024-04-22T20:46:35.074224+0000","last_undegraded":"2024-04-22T20:46:35.074224+0000","last_fullsized":"2024-04-22T20:46:35.074224+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:35:00.012988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55718,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":984,"num_write":430,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":484,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.035331+0000","last_change":"2024-04-22T19:51:34.561345+0000","last_active":"2024-04-22T20:44:05.035331+0000","last_peered":"2024-04-22T20:44:05.035331+0000","last_clean":"2024-04-22T20:44:05.035331+0000","last_became_active":"2024-04-22T19:51:34.561096+0000","last_became_peered":"2024-04-22T19:51:34.561096+0000","last_unstale":"2024-04-22T20:44:05.035331+0000","last_undegraded":"2024-04-22T20:44:05.035331+0000","last_fullsized":"2024-04-22T20:44:05.035331+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:38.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"261'161","reported_seq":661,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314006+0000","last_change":"2024-04-22T19:51:36.571812+0000","last_active":"2024-04-22T20:45:20.314006+0000","last_peered":"2024-04-22T20:45:20.314006+0000","last_clean":"2024-04-22T20:45:20.314006+0000","last_became_active":"2024-04-22T19:51:36.571565+0000","last_became_peered":"2024-04-22T19:51:36.571565+0000","last_unstale":"2024-04-22T20:45:20.314006+0000","last_undegraded":"2024-04-22T20:45:20.314006+0000","last_fullsized":"2024-04-22T20:45:20.314006+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:55.978840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":180,"num_write":473,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"261'44","reported_seq":609,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145600+0000","last_change":"2024-04-22T19:51:28.507235+0000","last_active":"2024-04-22T20:44:00.145600+0000","last_peered":"2024-04-22T20:44:00.145600+0000","last_clean":"2024-04-22T20:44:00.145600+0000","last_became_active":"2024-04-22T19:51:28.506835+0000","last_became_peered":"2024-04-22T19:51:28.506835+0000","last_unstale":"2024-04-22T20:44:00.145600+0000","last_undegraded":"2024-04-22T20:44:00.145600+0000","last_fullsized":"2024-04-22T20:44:00.145600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:22:38.074432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":202,"num_read_kb":174,"num_write":58,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"261'309","reported_seq":936,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.304458+0000","last_change":"2024-04-22T19:52:04.618218+0000","last_active":"2024-04-22T20:47:38.304458+0000","last_peered":"2024-04-22T20:47:38.304458+0000","last_clean":"2024-04-22T20:47:38.304458+0000","last_became_active":"2024-04-22T19:52:04.617820+0000","last_became_peered":"2024-04-22T19:52:04.617820+0000","last_unstale":"2024-04-22T20:47:38.304458+0000","last_undegraded":"2024-04-22T20:47:38.304458+0000","last_fullsized":"2024-04-22T20:47:38.304458+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:04:41.066429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588803,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":467,"num_read_kb":31540,"num_write":1405,"num_write_kb":43247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166735+0000","last_change":"2024-04-22T19:51:30.529632+0000","last_active":"2024-04-22T20:44:00.166735+0000","last_peered":"2024-04-22T20:44:00.166735+0000","last_clean":"2024-04-22T20:44:00.166735+0000","last_became_active":"2024-04-22T19:51:30.529322+0000","last_became_peered":"2024-04-22T19:51:30.529322+0000","last_unstale":"2024-04-22T20:44:00.166735+0000","last_undegraded":"2024-04-22T20:44:00.166735+0000","last_fullsized":"2024-04-22T20:44:00.166735+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:43:15.680192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"261'266","reported_seq":848,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.025839+0000","last_change":"2024-04-22T19:51:33.248631+0000","last_active":"2024-04-22T20:48:03.025839+0000","last_peered":"2024-04-22T20:48:03.025839+0000","last_clean":"2024-04-22T20:48:03.025839+0000","last_became_active":"2024-04-22T19:51:33.248293+0000","last_became_peered":"2024-04-22T19:51:33.248293+0000","last_unstale":"2024-04-22T20:48:03.025839+0000","last_undegraded":"2024-04-22T20:48:03.025839+0000","last_fullsized":"2024-04-22T20:48:03.025839+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:49.051719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66888,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":442,"num_read_kb":925,"num_write":358,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.046532+0000","last_change":"2024-04-22T19:51:34.561273+0000","last_active":"2024-04-22T20:45:45.046532+0000","last_peered":"2024-04-22T20:45:45.046532+0000","last_clean":"2024-04-22T20:45:45.046532+0000","last_became_active":"2024-04-22T19:51:34.560955+0000","last_became_peered":"2024-04-22T19:51:34.560955+0000","last_unstale":"2024-04-22T20:45:45.046532+0000","last_undegraded":"2024-04-22T20:45:45.046532+0000","last_fullsized":"2024-04-22T20:45:45.046532+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:03:55.584983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"261'119","reported_seq":602,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.536618+0000","last_change":"2024-04-22T19:51:36.567411+0000","last_active":"2024-04-22T20:48:08.536618+0000","last_peered":"2024-04-22T20:48:08.536618+0000","last_clean":"2024-04-22T20:48:08.536618+0000","last_became_active":"2024-04-22T19:51:36.567139+0000","last_became_peered":"2024-04-22T19:51:36.567139+0000","last_unstale":"2024-04-22T20:48:08.536618+0000","last_undegraded":"2024-04-22T20:48:08.536618+0000","last_fullsized":"2024-04-22T20:48:08.536618+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:39:35.365237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":92,"num_write":268,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"261'106","reported_seq":688,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.058728+0000","last_change":"2024-04-22T19:51:28.512939+0000","last_active":"2024-04-22T20:44:00.058728+0000","last_peered":"2024-04-22T20:44:00.058728+0000","last_clean":"2024-04-22T20:44:00.058728+0000","last_became_active":"2024-04-22T19:51:28.512642+0000","last_became_peered":"2024-04-22T19:51:28.512642+0000","last_unstale":"2024-04-22T20:44:00.058728+0000","last_undegraded":"2024-04-22T20:44:00.058728+0000","last_fullsized":"2024-04-22T20:44:00.058728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:36.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":349,"num_write":169,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"261'262","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.171114+0000","last_change":"2024-04-22T19:51:26.512313+0000","last_active":"2024-04-22T20:44:00.171114+0000","last_peered":"2024-04-22T20:44:00.171114+0000","last_clean":"2024-04-22T20:44:00.171114+0000","last_became_active":"2024-04-22T19:51:26.511966+0000","last_became_peered":"2024-04-22T19:51:26.511966+0000","last_unstale":"2024-04-22T20:44:00.171114+0000","last_undegraded":"2024-04-22T20:44:00.171114+0000","last_fullsized":"2024-04-22T20:44:00.171114+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:52.017119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7663079,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":370,"num_read_kb":38272,"num_write":1038,"num_write_kb":51509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.034785+0000","last_change":"2024-04-22T19:51:30.527261+0000","last_active":"2024-04-22T20:44:00.034785+0000","last_peered":"2024-04-22T20:44:00.034785+0000","last_clean":"2024-04-22T20:44:00.034785+0000","last_became_active":"2024-04-22T19:51:30.526949+0000","last_became_peered":"2024-04-22T19:51:30.526949+0000","last_unstale":"2024-04-22T20:44:00.034785+0000","last_undegraded":"2024-04-22T20:44:00.034785+0000","last_fullsized":"2024-04-22T20:44:00.034785+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:13:41.016976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"261'103","reported_seq":629,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182111+0000","last_change":"2024-04-22T19:51:33.248558+0000","last_active":"2024-04-22T20:45:45.182111+0000","last_peered":"2024-04-22T20:45:45.182111+0000","last_clean":"2024-04-22T20:45:45.182111+0000","last_became_active":"2024-04-22T19:51:33.248113+0000","last_became_peered":"2024-04-22T19:51:33.248113+0000","last_unstale":"2024-04-22T20:45:45.182111+0000","last_undegraded":"2024-04-22T20:45:45.182111+0000","last_fullsized":"2024-04-22T20:45:45.182111+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:09.138747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":119,"num_read_kb":113,"num_write":116,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"261'515","reported_seq":2294,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.550861+0000","last_change":"2024-04-22T19:51:34.557105+0000","last_active":"2024-04-22T20:48:08.550861+0000","last_peered":"2024-04-22T20:48:08.550861+0000","last_clean":"2024-04-22T20:48:08.550861+0000","last_became_active":"2024-04-22T19:51:34.556789+0000","last_became_peered":"2024-04-22T19:51:34.556789+0000","last_unstale":"2024-04-22T20:48:08.550861+0000","last_undegraded":"2024-04-22T20:48:08.550861+0000","last_fullsized":"2024-04-22T20:48:08.550861+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:07.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"261'96","reported_seq":594,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062299+0000","last_change":"2024-04-22T19:51:36.571445+0000","last_active":"2024-04-22T20:45:20.062299+0000","last_peered":"2024-04-22T20:45:20.062299+0000","last_clean":"2024-04-22T20:45:20.062299+0000","last_became_active":"2024-04-22T19:51:36.571175+0000","last_became_peered":"2024-04-22T19:51:36.571175+0000","last_unstale":"2024-04-22T20:45:20.062299+0000","last_undegraded":"2024-04-22T20:45:20.062299+0000","last_fullsized":"2024-04-22T20:45:20.062299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:18.054128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":221,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"261'131","reported_seq":708,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:14.252216+0000","last_change":"2024-04-22T19:51:28.509477+0000","last_active":"2024-04-22T20:45:14.252216+0000","last_peered":"2024-04-22T20:45:14.252216+0000","last_clean":"2024-04-22T20:45:14.252216+0000","last_became_active":"2024-04-22T19:51:28.509110+0000","last_became_peered":"2024-04-22T19:51:28.509110+0000","last_unstale":"2024-04-22T20:45:14.252216+0000","last_undegraded":"2024-04-22T20:45:14.252216+0000","last_fullsized":"2024-04-22T20:45:14.252216+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:10:52.641173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":519,"num_read_kb":420,"num_write":188,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"261'333","reported_seq":959,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:08.194615+0000","last_change":"2024-04-22T19:52:04.619673+0000","last_active":"2024-04-22T20:45:08.194615+0000","last_peered":"2024-04-22T20:45:08.194615+0000","last_clean":"2024-04-22T20:45:08.194615+0000","last_became_active":"2024-04-22T19:52:04.619385+0000","last_became_peered":"2024-04-22T19:52:04.619385+0000","last_unstale":"2024-04-22T20:45:08.194615+0000","last_undegraded":"2024-04-22T20:45:08.194615+0000","last_fullsized":"2024-04-22T20:45:08.194615+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:11.756964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8224737,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":472,"num_read_kb":20998,"num_write":1385,"num_write_kb":33571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.049743+0000","last_change":"2024-04-22T19:51:30.527064+0000","last_active":"2024-04-22T20:44:00.049743+0000","last_peered":"2024-04-22T20:44:00.049743+0000","last_clean":"2024-04-22T20:44:00.049743+0000","last_became_active":"2024-04-22T19:51:30.526469+0000","last_became_peered":"2024-04-22T19:51:30.526469+0000","last_unstale":"2024-04-22T20:44:00.049743+0000","last_undegraded":"2024-04-22T20:44:00.049743+0000","last_fullsized":"2024-04-22T20:44:00.049743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:45:04.946224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"261'301","reported_seq":880,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.204151+0000","last_change":"2024-04-22T19:51:32.540830+0000","last_active":"2024-04-22T20:47:20.204151+0000","last_peered":"2024-04-22T20:47:20.204151+0000","last_clean":"2024-04-22T20:47:20.204151+0000","last_became_active":"2024-04-22T19:51:32.540573+0000","last_became_peered":"2024-04-22T19:51:32.540573+0000","last_unstale":"2024-04-22T20:47:20.204151+0000","last_undegraded":"2024-04-22T20:47:20.204151+0000","last_fullsized":"2024-04-22T20:47:20.204151+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:49.905368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61803,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":483,"num_read_kb":946,"num_write":388,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.176362+0000","last_change":"2024-04-22T19:51:34.554124+0000","last_active":"2024-04-22T20:44:45.176362+0000","last_peered":"2024-04-22T20:44:45.176362+0000","last_clean":"2024-04-22T20:44:45.176362+0000","last_became_active":"2024-04-22T19:51:34.553857+0000","last_became_peered":"2024-04-22T19:51:34.553857+0000","last_unstale":"2024-04-22T20:44:45.176362+0000","last_undegraded":"2024-04-22T20:44:45.176362+0000","last_fullsized":"2024-04-22T20:44:45.176362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:38:56.150478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"261'153","reported_seq":654,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:15.079446+0000","last_change":"2024-04-22T19:51:36.575509+0000","last_active":"2024-04-22T20:47:15.079446+0000","last_peered":"2024-04-22T20:47:15.079446+0000","last_clean":"2024-04-22T20:47:15.079446+0000","last_became_active":"2024-04-22T19:51:36.575211+0000","last_became_peered":"2024-04-22T19:51:36.575211+0000","last_unstale":"2024-04-22T20:47:15.079446+0000","last_undegraded":"2024-04-22T20:47:15.079446+0000","last_fullsized":"2024-04-22T20:47:15.079446+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:46:54.630277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":158,"num_write":375,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"261'85","reported_seq":680,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.024455+0000","last_change":"2024-04-22T19:51:28.508354+0000","last_active":"2024-04-22T20:48:04.024455+0000","last_peered":"2024-04-22T20:48:04.024455+0000","last_clean":"2024-04-22T20:48:04.024455+0000","last_became_active":"2024-04-22T19:51:28.508145+0000","last_became_peered":"2024-04-22T19:51:28.508145+0000","last_unstale":"2024-04-22T20:48:04.024455+0000","last_undegraded":"2024-04-22T20:48:04.024455+0000","last_fullsized":"2024-04-22T20:48:04.024455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:54:32.701108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":376,"num_read_kb":320,"num_write":115,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"261'220","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174972+0000","last_change":"2024-04-22T19:51:26.504671+0000","last_active":"2024-04-22T20:44:00.174972+0000","last_peered":"2024-04-22T20:44:00.174972+0000","last_clean":"2024-04-22T20:44:00.174972+0000","last_became_active":"2024-04-22T19:51:26.504301+0000","last_became_peered":"2024-04-22T19:51:26.504301+0000","last_unstale":"2024-04-22T20:44:00.174972+0000","last_undegraded":"2024-04-22T20:44:00.174972+0000","last_fullsized":"2024-04-22T20:44:00.174972+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:56:36.267794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4884274,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":384,"num_read_kb":4553,"num_write":1039,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174938+0000","last_change":"2024-04-22T19:51:30.524895+0000","last_active":"2024-04-22T20:44:00.174938+0000","last_peered":"2024-04-22T20:44:00.174938+0000","last_clean":"2024-04-22T20:44:00.174938+0000","last_became_active":"2024-04-22T19:51:30.524417+0000","last_became_peered":"2024-04-22T19:51:30.524417+0000","last_unstale":"2024-04-22T20:44:00.174938+0000","last_undegraded":"2024-04-22T20:44:00.174938+0000","last_fullsized":"2024-04-22T20:44:00.174938+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:39:15.056196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"261'225","reported_seq":790,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.193293+0000","last_change":"2024-04-22T19:51:32.540923+0000","last_active":"2024-04-22T20:47:10.193293+0000","last_peered":"2024-04-22T20:47:10.193293+0000","last_clean":"2024-04-22T20:47:10.193293+0000","last_became_active":"2024-04-22T19:51:32.540659+0000","last_became_peered":"2024-04-22T19:51:32.540659+0000","last_unstale":"2024-04-22T20:47:10.193293+0000","last_undegraded":"2024-04-22T20:47:10.193293+0000","last_fullsized":"2024-04-22T20:47:10.193293+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:42:58.310584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51883,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":376,"num_read_kb":774,"num_write":298,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309863+0000","last_change":"2024-04-22T19:51:34.552587+0000","last_active":"2024-04-22T20:44:50.309863+0000","last_peered":"2024-04-22T20:44:50.309863+0000","last_clean":"2024-04-22T20:44:50.309863+0000","last_became_active":"2024-04-22T19:51:34.552344+0000","last_became_peered":"2024-04-22T19:51:34.552344+0000","last_unstale":"2024-04-22T20:44:50.309863+0000","last_undegraded":"2024-04-22T20:44:50.309863+0000","last_fullsized":"2024-04-22T20:44:50.309863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:41.886857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"261'137","reported_seq":675,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:01.745300+0000","last_change":"2024-04-22T19:51:36.567945+0000","last_active":"2024-04-22T20:48:01.745300+0000","last_peered":"2024-04-22T20:48:01.745300+0000","last_clean":"2024-04-22T20:48:01.745300+0000","last_became_active":"2024-04-22T19:51:36.567594+0000","last_became_peered":"2024-04-22T19:51:36.567594+0000","last_unstale":"2024-04-22T20:48:01.745300+0000","last_undegraded":"2024-04-22T20:48:01.745300+0000","last_fullsized":"2024-04-22T20:48:01.745300+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:25:38.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":214,"num_write":335,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"261'87","reported_seq":648,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.151277+0000","last_change":"2024-04-22T19:51:28.519780+0000","last_active":"2024-04-22T20:44:45.151277+0000","last_peered":"2024-04-22T20:44:45.151277+0000","last_clean":"2024-04-22T20:44:45.151277+0000","last_became_active":"2024-04-22T19:51:28.519542+0000","last_became_peered":"2024-04-22T19:51:28.519542+0000","last_unstale":"2024-04-22T20:44:45.151277+0000","last_undegraded":"2024-04-22T20:44:45.151277+0000","last_fullsized":"2024-04-22T20:44:45.151277+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:24:55.634254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":347,"num_read_kb":277,"num_write":137,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"261'332","reported_seq":1176,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157148+0000","last_change":"2024-04-22T19:51:26.500077+0000","last_active":"2024-04-22T20:45:35.157148+0000","last_peered":"2024-04-22T20:45:35.157148+0000","last_clean":"2024-04-22T20:45:35.157148+0000","last_became_active":"2024-04-22T19:51:26.499390+0000","last_became_peered":"2024-04-22T19:51:26.499390+0000","last_unstale":"2024-04-22T20:45:35.157148+0000","last_undegraded":"2024-04-22T20:45:35.157148+0000","last_fullsized":"2024-04-22T20:45:35.157148+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:40:50.316762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24547368,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":977,"num_read_kb":88369,"num_write":1347,"num_write_kb":41583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172368+0000","last_change":"2024-04-22T19:51:30.526678+0000","last_active":"2024-04-22T20:44:40.172368+0000","last_peered":"2024-04-22T20:44:40.172368+0000","last_clean":"2024-04-22T20:44:40.172368+0000","last_became_active":"2024-04-22T19:51:30.526183+0000","last_became_peered":"2024-04-22T19:51:30.526183+0000","last_unstale":"2024-04-22T20:44:40.172368+0000","last_undegraded":"2024-04-22T20:44:40.172368+0000","last_fullsized":"2024-04-22T20:44:40.172368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:40.232417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"261'243","reported_seq":802,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.164215+0000","last_change":"2024-04-22T19:51:32.538965+0000","last_active":"2024-04-22T20:46:35.164215+0000","last_peered":"2024-04-22T20:46:35.164215+0000","last_clean":"2024-04-22T20:46:35.164215+0000","last_became_active":"2024-04-22T19:51:32.538644+0000","last_became_peered":"2024-04-22T19:51:32.538644+0000","last_unstale":"2024-04-22T20:46:35.164215+0000","last_undegraded":"2024-04-22T20:46:35.164215+0000","last_fullsized":"2024-04-22T20:46:35.164215+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:18:02.949702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52954,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":389,"num_read_kb":836,"num_write":326,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":486,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.167484+0000","last_change":"2024-04-22T19:51:34.551365+0000","last_active":"2024-04-22T20:44:05.167484+0000","last_peered":"2024-04-22T20:44:05.167484+0000","last_clean":"2024-04-22T20:44:05.167484+0000","last_became_active":"2024-04-22T19:51:34.551085+0000","last_became_peered":"2024-04-22T19:51:34.551085+0000","last_unstale":"2024-04-22T20:44:05.167484+0000","last_undegraded":"2024-04-22T20:44:05.167484+0000","last_fullsized":"2024-04-22T20:44:05.167484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:50:23.004035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"261'179","reported_seq":903,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.372427+0000","last_change":"2024-04-22T19:51:36.569537+0000","last_active":"2024-04-22T20:48:00.372427+0000","last_peered":"2024-04-22T20:48:00.372427+0000","last_clean":"2024-04-22T20:48:00.372427+0000","last_became_active":"2024-04-22T19:51:36.569234+0000","last_became_peered":"2024-04-22T19:51:36.569234+0000","last_unstale":"2024-04-22T20:48:00.372427+0000","last_undegraded":"2024-04-22T20:48:00.372427+0000","last_fullsized":"2024-04-22T20:48:00.372427+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:56:31.796866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":419,"num_read_kb":454,"num_write":429,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"261'180","reported_seq":750,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172395+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:44:40.172395+0000","last_peered":"2024-04-22T20:44:40.172395+0000","last_clean":"2024-04-22T20:44:40.172395+0000","last_became_active":"2024-04-22T19:51:28.515506+0000","last_became_peered":"2024-04-22T19:51:28.515506+0000","last_unstale":"2024-04-22T20:44:40.172395+0000","last_undegraded":"2024-04-22T20:44:40.172395+0000","last_fullsized":"2024-04-22T20:44:40.172395+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:14:29.010830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":678,"num_read_kb":529,"num_write":279,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"261'227","reported_seq":839,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.881098+0000","last_change":"2024-04-22T19:51:26.504948+0000","last_active":"2024-04-22T20:46:41.881098+0000","last_peered":"2024-04-22T20:46:41.881098+0000","last_clean":"2024-04-22T20:46:41.881098+0000","last_became_active":"2024-04-22T19:51:26.504604+0000","last_became_peered":"2024-04-22T19:51:26.504604+0000","last_unstale":"2024-04-22T20:46:41.881098+0000","last_undegraded":"2024-04-22T20:46:41.881098+0000","last_fullsized":"2024-04-22T20:46:41.881098+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:11.092305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11653248,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":302,"num_read_kb":9007,"num_write":882,"num_write_kb":46630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"25'1","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040244+0000","last_change":"2024-04-22T19:51:30.523630+0000","last_active":"2024-04-22T20:44:50.040244+0000","last_peered":"2024-04-22T20:44:50.040244+0000","last_clean":"2024-04-22T20:44:50.040244+0000","last_became_active":"2024-04-22T19:51:30.523338+0000","last_became_peered":"2024-04-22T19:51:30.523338+0000","last_unstale":"2024-04-22T20:44:50.040244+0000","last_undegraded":"2024-04-22T20:44:50.040244+0000","last_fullsized":"2024-04-22T20:44:50.040244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:00.508499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"261'305","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.188922+0000","last_change":"2024-04-22T19:51:32.537701+0000","last_active":"2024-04-22T20:46:35.188922+0000","last_peered":"2024-04-22T20:46:35.188922+0000","last_clean":"2024-04-22T20:46:35.188922+0000","last_became_active":"2024-04-22T19:51:32.537373+0000","last_became_peered":"2024-04-22T19:51:32.537373+0000","last_unstale":"2024-04-22T20:46:35.188922+0000","last_undegraded":"2024-04-22T20:46:35.188922+0000","last_fullsized":"2024-04-22T20:46:35.188922+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:43:27.042829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59361,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":482,"num_read_kb":951,"num_write":388,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":486,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.171634+0000","last_change":"2024-04-22T19:51:34.555960+0000","last_active":"2024-04-22T20:44:05.171634+0000","last_peered":"2024-04-22T20:44:05.171634+0000","last_clean":"2024-04-22T20:44:05.171634+0000","last_became_active":"2024-04-22T19:51:34.555709+0000","last_became_peered":"2024-04-22T19:51:34.555709+0000","last_unstale":"2024-04-22T20:44:05.171634+0000","last_undegraded":"2024-04-22T20:44:05.171634+0000","last_fullsized":"2024-04-22T20:44:05.171634+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:02.499776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"261'144","reported_seq":628,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:32.358361+0000","last_change":"2024-04-22T19:51:36.574316+0000","last_active":"2024-04-22T20:46:32.358361+0000","last_peered":"2024-04-22T20:46:32.358361+0000","last_clean":"2024-04-22T20:46:32.358361+0000","last_became_active":"2024-04-22T19:51:36.574012+0000","last_became_peered":"2024-04-22T19:51:36.574012+0000","last_unstale":"2024-04-22T20:46:32.358361+0000","last_undegraded":"2024-04-22T20:46:32.358361+0000","last_fullsized":"2024-04-22T20:46:32.358361+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:01:01.700368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":109,"num_write":348,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"261'372","reported_seq":1037,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045977+0000","last_change":"2024-04-22T19:51:28.518589+0000","last_active":"2024-04-22T20:45:40.045977+0000","last_peered":"2024-04-22T20:45:40.045977+0000","last_clean":"2024-04-22T20:45:40.045977+0000","last_became_active":"2024-04-22T19:51:28.518339+0000","last_became_peered":"2024-04-22T19:51:28.518339+0000","last_unstale":"2024-04-22T20:45:40.045977+0000","last_undegraded":"2024-04-22T20:45:40.045977+0000","last_fullsized":"2024-04-22T20:45:40.045977+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:15:44.877848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1526,"num_read_kb":1211,"num_write":619,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"261'252","reported_seq":951,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.316997+0000","last_change":"2024-04-22T19:51:26.503151+0000","last_active":"2024-04-22T20:45:40.316997+0000","last_peered":"2024-04-22T20:45:40.316997+0000","last_clean":"2024-04-22T20:45:40.316997+0000","last_became_active":"2024-04-22T19:51:26.502773+0000","last_became_peered":"2024-04-22T19:51:26.502773+0000","last_unstale":"2024-04-22T20:45:40.316997+0000","last_undegraded":"2024-04-22T20:45:40.316997+0000","last_fullsized":"2024-04-22T20:45:40.316997+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:34:49.559184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6843432,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":522,"num_read_kb":14380,"num_write":1027,"num_write_kb":17858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"261'524","reported_seq":1286,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.175644+0000","last_change":"2024-04-22T19:52:04.617284+0000","last_active":"2024-04-22T20:44:05.175644+0000","last_peered":"2024-04-22T20:44:05.175644+0000","last_clean":"2024-04-22T20:44:05.175644+0000","last_became_active":"2024-04-22T19:52:04.617036+0000","last_became_peered":"2024-04-22T19:52:04.617036+0000","last_unstale":"2024-04-22T20:44:05.175644+0000","last_undegraded":"2024-04-22T20:44:05.175644+0000","last_fullsized":"2024-04-22T20:44:05.175644+0000","mapping_epoch":33,"log_start":"261'500","ondisk_log_start":"261'500","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":500,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:59:14.488482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2230,"num_read_kb":1847,"num_write":935,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"261'226","reported_seq":865,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:43.145525+0000","last_change":"2024-04-22T19:51:26.496262+0000","last_active":"2024-04-22T20:46:43.145525+0000","last_peered":"2024-04-22T20:46:43.145525+0000","last_clean":"2024-04-22T20:46:43.145525+0000","last_became_active":"2024-04-22T19:51:26.495948+0000","last_became_peered":"2024-04-22T19:51:26.495948+0000","last_unstale":"2024-04-22T20:46:43.145525+0000","last_undegraded":"2024-04-22T20:46:43.145525+0000","last_fullsized":"2024-04-22T20:46:43.145525+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":200,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:25.588593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1295176,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":394,"num_read_kb":10115,"num_write":945,"num_write_kb":22835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"261'208","reported_seq":841,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189450+0000","last_change":"2024-04-22T19:51:28.513290+0000","last_active":"2024-04-22T20:45:40.189450+0000","last_peered":"2024-04-22T20:45:40.189450+0000","last_clean":"2024-04-22T20:45:40.189450+0000","last_became_active":"2024-04-22T19:51:28.513026+0000","last_became_peered":"2024-04-22T19:51:28.513026+0000","last_unstale":"2024-04-22T20:45:40.189450+0000","last_undegraded":"2024-04-22T20:45:40.189450+0000","last_fullsized":"2024-04-22T20:45:40.189450+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:09:31.873692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":827,"num_read_kb":665,"num_write":342,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"261'251","reported_seq":870,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:43.136633+0000","last_change":"2024-04-22T19:51:26.507266+0000","last_active":"2024-04-22T20:46:43.136633+0000","last_peered":"2024-04-22T20:46:43.136633+0000","last_clean":"2024-04-22T20:46:43.136633+0000","last_became_active":"2024-04-22T19:51:26.506748+0000","last_became_peered":"2024-04-22T19:51:26.506748+0000","last_unstale":"2024-04-22T20:46:43.136633+0000","last_undegraded":"2024-04-22T20:46:43.136633+0000","last_fullsized":"2024-04-22T20:46:43.136633+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:15.321922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6641638,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":354,"num_read_kb":12674,"num_write":1040,"num_write_kb":41347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"261'4175","reported_seq":4791,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:29.357349+0000","last_change":"2024-04-22T19:51:28.511560+0000","last_active":"2024-04-22T20:46:29.357349+0000","last_peered":"2024-04-22T20:46:29.357349+0000","last_clean":"2024-04-22T20:46:29.357349+0000","last_became_active":"2024-04-22T19:51:28.511312+0000","last_became_peered":"2024-04-22T19:51:28.511312+0000","last_unstale":"2024-04-22T20:46:29.357349+0000","last_undegraded":"2024-04-22T20:46:29.357349+0000","last_fullsized":"2024-04-22T20:46:29.357349+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":3000,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:28:56.675918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14718,"num_read_kb":11919,"num_write":6278,"num_write_kb":3133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"261'248","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.060540+0000","last_change":"2024-04-22T19:52:05.115180+0000","last_active":"2024-04-22T20:44:20.060540+0000","last_peered":"2024-04-22T20:44:20.060540+0000","last_clean":"2024-04-22T20:44:20.060540+0000","last_became_active":"2024-04-22T19:52:05.114734+0000","last_became_peered":"2024-04-22T19:52:05.114734+0000","last_unstale":"2024-04-22T20:44:20.060540+0000","last_undegraded":"2024-04-22T20:44:20.060540+0000","last_fullsized":"2024-04-22T20:44:20.060540+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:08:25.577231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11535134,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":357,"num_read_kb":5518,"num_write":1024,"num_write_kb":21667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.23","version":"261'170","reported_seq":851,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.006667+0000","last_change":"2024-04-22T19:51:28.516445+0000","last_active":"2024-04-22T20:48:03.006667+0000","last_peered":"2024-04-22T20:48:03.006667+0000","last_clean":"2024-04-22T20:48:03.006667+0000","last_became_active":"2024-04-22T19:51:28.516125+0000","last_became_peered":"2024-04-22T19:51:28.516125+0000","last_unstale":"2024-04-22T20:48:03.006667+0000","last_undegraded":"2024-04-22T20:48:03.006667+0000","last_fullsized":"2024-04-22T20:48:03.006667+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:01:29.663998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":741,"num_read_kb":616,"num_write":243,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"261'299","reported_seq":945,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.135884+0000","last_change":"2024-04-22T19:52:04.619831+0000","last_active":"2024-04-22T20:47:38.135884+0000","last_peered":"2024-04-22T20:47:38.135884+0000","last_clean":"2024-04-22T20:47:38.135884+0000","last_became_active":"2024-04-22T19:52:04.619286+0000","last_became_peered":"2024-04-22T19:52:04.619286+0000","last_unstale":"2024-04-22T20:47:38.135884+0000","last_undegraded":"2024-04-22T20:47:38.135884+0000","last_fullsized":"2024-04-22T20:47:38.135884+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:58.235831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538009,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":519,"num_read_kb":69891,"num_write":1172,"num_write_kb":36151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"261'122","reported_seq":726,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181359+0000","last_change":"2024-04-22T19:51:28.516690+0000","last_active":"2024-04-22T20:45:40.181359+0000","last_peered":"2024-04-22T20:45:40.181359+0000","last_clean":"2024-04-22T20:45:40.181359+0000","last_became_active":"2024-04-22T19:51:28.516437+0000","last_became_peered":"2024-04-22T19:51:28.516437+0000","last_unstale":"2024-04-22T20:45:40.181359+0000","last_undegraded":"2024-04-22T20:45:40.181359+0000","last_fullsized":"2024-04-22T20:45:40.181359+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:44.084365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":423,"num_write":178,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"261'257","reported_seq":991,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045950+0000","last_change":"2024-04-22T19:51:26.498349+0000","last_active":"2024-04-22T20:45:40.045950+0000","last_peered":"2024-04-22T20:45:40.045950+0000","last_clean":"2024-04-22T20:45:40.045950+0000","last_became_active":"2024-04-22T19:51:26.498039+0000","last_became_peered":"2024-04-22T19:51:26.498039+0000","last_unstale":"2024-04-22T20:45:40.045950+0000","last_undegraded":"2024-04-22T20:45:40.045950+0000","last_fullsized":"2024-04-22T20:45:40.045950+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:17.714586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14707922,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":706,"num_read_kb":22382,"num_write":1193,"num_write_kb":18139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"261'199","reported_seq":848,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:49.466099+0000","last_change":"2024-04-22T19:51:28.520853+0000","last_active":"2024-04-22T20:46:49.466099+0000","last_peered":"2024-04-22T20:46:49.466099+0000","last_clean":"2024-04-22T20:46:49.466099+0000","last_became_active":"2024-04-22T19:51:28.520612+0000","last_became_peered":"2024-04-22T19:51:28.520612+0000","last_unstale":"2024-04-22T20:46:49.466099+0000","last_undegraded":"2024-04-22T20:46:49.466099+0000","last_fullsized":"2024-04-22T20:46:49.466099+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:46:57.197381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":735,"num_read_kb":593,"num_write":262,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"261'291","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317145+0000","last_change":"2024-04-22T19:51:26.499154+0000","last_active":"2024-04-22T20:45:40.317145+0000","last_peered":"2024-04-22T20:45:40.317145+0000","last_clean":"2024-04-22T20:45:40.317145+0000","last_became_active":"2024-04-22T19:51:26.498845+0000","last_became_peered":"2024-04-22T19:51:26.498845+0000","last_unstale":"2024-04-22T20:45:40.317145+0000","last_undegraded":"2024-04-22T20:45:40.317145+0000","last_fullsized":"2024-04-22T20:45:40.317145+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:50.165203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7476623,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":432,"num_read_kb":33253,"num_write":1174,"num_write_kb":30860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"261'133","reported_seq":766,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317028+0000","last_change":"2024-04-22T19:51:28.520128+0000","last_active":"2024-04-22T20:45:40.317028+0000","last_peered":"2024-04-22T20:45:40.317028+0000","last_clean":"2024-04-22T20:45:40.317028+0000","last_became_active":"2024-04-22T19:51:28.519669+0000","last_became_peered":"2024-04-22T19:51:28.519669+0000","last_unstale":"2024-04-22T20:45:40.317028+0000","last_undegraded":"2024-04-22T20:45:40.317028+0000","last_fullsized":"2024-04-22T20:45:40.317028+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:18.121387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":560,"num_read_kb":466,"num_write":196,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"261'257","reported_seq":852,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:07.068523+0000","last_change":"2024-04-22T19:52:04.618077+0000","last_active":"2024-04-22T20:44:07.068523+0000","last_peered":"2024-04-22T20:44:07.068523+0000","last_clean":"2024-04-22T20:44:07.068523+0000","last_became_active":"2024-04-22T19:52:04.617649+0000","last_became_peered":"2024-04-22T19:52:04.617649+0000","last_unstale":"2024-04-22T20:44:07.068523+0000","last_undegraded":"2024-04-22T20:44:07.068523+0000","last_fullsized":"2024-04-22T20:44:07.068523+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:58.404399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587123,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":386,"num_read_kb":29068,"num_write":962,"num_write_kb":22627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"261'128","reported_seq":709,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.179387+0000","last_change":"2024-04-22T19:51:28.521764+0000","last_active":"2024-04-22T20:45:10.179387+0000","last_peered":"2024-04-22T20:45:10.179387+0000","last_clean":"2024-04-22T20:45:10.179387+0000","last_became_active":"2024-04-22T19:51:28.521516+0000","last_became_peered":"2024-04-22T19:51:28.521516+0000","last_unstale":"2024-04-22T20:45:10.179387+0000","last_undegraded":"2024-04-22T20:45:10.179387+0000","last_fullsized":"2024-04-22T20:45:10.179387+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:13:11.122213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":542,"num_read_kb":442,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"261'278","reported_seq":958,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:52.078309+0000","last_change":"2024-04-22T19:51:26.510292+0000","last_active":"2024-04-22T20:46:52.078309+0000","last_peered":"2024-04-22T20:46:52.078309+0000","last_clean":"2024-04-22T20:46:52.078309+0000","last_became_active":"2024-04-22T19:51:26.507843+0000","last_became_peered":"2024-04-22T19:51:26.507843+0000","last_unstale":"2024-04-22T20:46:52.078309+0000","last_undegraded":"2024-04-22T20:46:52.078309+0000","last_fullsized":"2024-04-22T20:46:52.078309+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:43:15.698499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10798875,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":447,"num_read_kb":136369,"num_write":1173,"num_write_kb":55788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"261'235","reported_seq":860,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176799+0000","last_change":"2024-04-22T19:51:28.515473+0000","last_active":"2024-04-22T20:45:10.176799+0000","last_peered":"2024-04-22T20:45:10.176799+0000","last_clean":"2024-04-22T20:45:10.176799+0000","last_became_active":"2024-04-22T19:51:28.515164+0000","last_became_peered":"2024-04-22T19:51:28.515164+0000","last_unstale":"2024-04-22T20:45:10.176799+0000","last_undegraded":"2024-04-22T20:45:10.176799+0000","last_fullsized":"2024-04-22T20:45:10.176799+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:38:12.199672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":950,"num_read_kb":750,"num_write":364,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"261'436","reported_seq":1136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.156482+0000","last_change":"2024-04-22T19:51:26.494217+0000","last_active":"2024-04-22T20:45:30.156482+0000","last_peered":"2024-04-22T20:45:30.156482+0000","last_clean":"2024-04-22T20:45:30.156482+0000","last_became_active":"2024-04-22T19:51:26.493860+0000","last_became_peered":"2024-04-22T19:51:26.493860+0000","last_unstale":"2024-04-22T20:45:30.156482+0000","last_undegraded":"2024-04-22T20:45:30.156482+0000","last_fullsized":"2024-04-22T20:45:30.156482+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:56:54.837990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6300944,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":885,"num_read_kb":13872,"num_write":1373,"num_write_kb":37693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"261'251","reported_seq":929,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:37.822141+0000","last_change":"2024-04-22T19:51:28.515833+0000","last_active":"2024-04-22T20:46:37.822141+0000","last_peered":"2024-04-22T20:46:37.822141+0000","last_clean":"2024-04-22T20:46:37.822141+0000","last_became_active":"2024-04-22T19:51:28.515547+0000","last_became_peered":"2024-04-22T19:51:28.515547+0000","last_unstale":"2024-04-22T20:46:37.822141+0000","last_undegraded":"2024-04-22T20:46:37.822141+0000","last_fullsized":"2024-04-22T20:46:37.822141+0000","mapping_epoch":22,"log_start":"247'200","ondisk_log_start":"247'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:00:39.299916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1120,"num_read_kb":973,"num_write":365,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"261'258","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.187879+0000","last_change":"2024-04-22T19:51:26.500937+0000","last_active":"2024-04-22T20:45:30.187879+0000","last_peered":"2024-04-22T20:45:30.187879+0000","last_clean":"2024-04-22T20:45:30.187879+0000","last_became_active":"2024-04-22T19:51:26.500566+0000","last_became_peered":"2024-04-22T19:51:26.500566+0000","last_unstale":"2024-04-22T20:45:30.187879+0000","last_undegraded":"2024-04-22T20:45:30.187879+0000","last_fullsized":"2024-04-22T20:45:30.187879+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:25.795560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4539865,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":358,"num_read_kb":45844,"num_write":1054,"num_write_kb":42819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"261'152","reported_seq":750,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:42.563356+0000","last_change":"2024-04-22T19:51:28.516044+0000","last_active":"2024-04-22T20:46:42.563356+0000","last_peered":"2024-04-22T20:46:42.563356+0000","last_clean":"2024-04-22T20:46:42.563356+0000","last_became_active":"2024-04-22T19:51:28.515764+0000","last_became_peered":"2024-04-22T19:51:28.515764+0000","last_unstale":"2024-04-22T20:46:42.563356+0000","last_undegraded":"2024-04-22T20:46:42.563356+0000","last_fullsized":"2024-04-22T20:46:42.563356+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:31.975652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":633,"num_read_kb":515,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"261'212","reported_seq":821,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181737+0000","last_change":"2024-04-22T19:51:26.512810+0000","last_active":"2024-04-22T20:45:30.181737+0000","last_peered":"2024-04-22T20:45:30.181737+0000","last_clean":"2024-04-22T20:45:30.181737+0000","last_became_active":"2024-04-22T19:51:26.512467+0000","last_became_peered":"2024-04-22T19:51:26.512467+0000","last_unstale":"2024-04-22T20:45:30.181737+0000","last_undegraded":"2024-04-22T20:45:30.181737+0000","last_fullsized":"2024-04-22T20:45:30.181737+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:04.495670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12585759,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":328,"num_read_kb":17902,"num_write":889,"num_write_kb":43703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"261'27","reported_seq":554,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.063941+0000","last_change":"2024-04-22T19:51:28.510362+0000","last_active":"2024-04-22T20:45:30.063941+0000","last_peered":"2024-04-22T20:45:30.063941+0000","last_clean":"2024-04-22T20:45:30.063941+0000","last_became_active":"2024-04-22T19:51:28.510091+0000","last_became_peered":"2024-04-22T19:51:28.510091+0000","last_unstale":"2024-04-22T20:45:30.063941+0000","last_undegraded":"2024-04-22T20:45:30.063941+0000","last_fullsized":"2024-04-22T20:45:30.063941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:48.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":95,"num_write":39,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"261'277","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:11.047810+0000","last_change":"2024-04-22T19:51:26.502070+0000","last_active":"2024-04-22T20:47:11.047810+0000","last_peered":"2024-04-22T20:47:11.047810+0000","last_clean":"2024-04-22T20:47:11.047810+0000","last_became_active":"2024-04-22T19:51:26.501700+0000","last_became_peered":"2024-04-22T19:51:26.501700+0000","last_unstale":"2024-04-22T20:47:11.047810+0000","last_undegraded":"2024-04-22T20:47:11.047810+0000","last_fullsized":"2024-04-22T20:47:11.047810+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:26:11.753132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4607139,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":443,"num_read_kb":6050,"num_write":1178,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"261'54","reported_seq":603,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181693+0000","last_change":"2024-04-22T19:51:28.509002+0000","last_active":"2024-04-22T20:45:30.181693+0000","last_peered":"2024-04-22T20:45:30.181693+0000","last_clean":"2024-04-22T20:45:30.181693+0000","last_became_active":"2024-04-22T19:51:28.508768+0000","last_became_peered":"2024-04-22T19:51:28.508768+0000","last_unstale":"2024-04-22T20:45:30.181693+0000","last_undegraded":"2024-04-22T20:45:30.181693+0000","last_fullsized":"2024-04-22T20:45:30.181693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:32:52.855073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":238,"num_read_kb":196,"num_write":80,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"261'285","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.477365+0000","last_change":"2024-04-22T19:51:26.497503+0000","last_active":"2024-04-22T20:47:44.477365+0000","last_peered":"2024-04-22T20:47:44.477365+0000","last_clean":"2024-04-22T20:47:44.477365+0000","last_became_active":"2024-04-22T19:51:26.497156+0000","last_became_peered":"2024-04-22T19:51:26.497156+0000","last_unstale":"2024-04-22T20:47:44.477365+0000","last_undegraded":"2024-04-22T20:47:44.477365+0000","last_fullsized":"2024-04-22T20:47:44.477365+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:23:31.844540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15135500,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":466,"num_read_kb":35895,"num_write":1174,"num_write_kb":35286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"261'4050","reported_seq":4612,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.156504+0000","last_change":"2024-04-22T19:51:28.515076+0000","last_active":"2024-04-22T20:45:30.156504+0000","last_peered":"2024-04-22T20:45:30.156504+0000","last_clean":"2024-04-22T20:45:30.156504+0000","last_became_active":"2024-04-22T19:51:28.514780+0000","last_became_peered":"2024-04-22T19:51:28.514780+0000","last_unstale":"2024-04-22T20:45:30.156504+0000","last_undegraded":"2024-04-22T20:45:30.156504+0000","last_fullsized":"2024-04-22T20:45:30.156504+0000","mapping_epoch":22,"log_start":"261'4000","ondisk_log_start":"261'4000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":3000,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:22:22.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14224,"num_read_kb":11535,"num_write":6070,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"261'243","reported_seq":1002,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.065500+0000","last_change":"2024-04-22T19:51:26.496549+0000","last_active":"2024-04-22T20:45:10.065500+0000","last_peered":"2024-04-22T20:45:10.065500+0000","last_clean":"2024-04-22T20:45:10.065500+0000","last_became_active":"2024-04-22T19:51:26.496214+0000","last_became_peered":"2024-04-22T19:51:26.496214+0000","last_unstale":"2024-04-22T20:45:10.065500+0000","last_undegraded":"2024-04-22T20:45:10.065500+0000","last_fullsized":"2024-04-22T20:45:10.065500+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:15:54.981915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6811717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":525,"num_read_kb":13796,"num_write":1122,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"261'298","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.067136+0000","last_change":"2024-04-22T19:51:28.508937+0000","last_active":"2024-04-22T20:45:50.067136+0000","last_peered":"2024-04-22T20:45:50.067136+0000","last_clean":"2024-04-22T20:45:50.067136+0000","last_became_active":"2024-04-22T19:51:28.508681+0000","last_became_peered":"2024-04-22T19:51:28.508681+0000","last_unstale":"2024-04-22T20:45:50.067136+0000","last_undegraded":"2024-04-22T20:45:50.067136+0000","last_fullsized":"2024-04-22T20:45:50.067136+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:21:44.423395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1231,"num_read_kb":1000,"num_write":489,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"261'215","reported_seq":824,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.184935+0000","last_change":"2024-04-22T19:51:26.505139+0000","last_active":"2024-04-22T20:45:10.184935+0000","last_peered":"2024-04-22T20:45:10.184935+0000","last_clean":"2024-04-22T20:45:10.184935+0000","last_became_active":"2024-04-22T19:51:26.502438+0000","last_became_peered":"2024-04-22T19:51:26.502438+0000","last_unstale":"2024-04-22T20:45:10.184935+0000","last_undegraded":"2024-04-22T20:45:10.184935+0000","last_fullsized":"2024-04-22T20:45:10.184935+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:27.926512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13702713,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":319,"num_read_kb":36636,"num_write":870,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"261'300","reported_seq":1179,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:52.068642+0000","last_change":"2024-04-22T19:51:26.505871+0000","last_active":"2024-04-22T20:46:52.068642+0000","last_peered":"2024-04-22T20:46:52.068642+0000","last_clean":"2024-04-22T20:46:52.068642+0000","last_became_active":"2024-04-22T19:51:26.504878+0000","last_became_peered":"2024-04-22T19:51:26.504878+0000","last_unstale":"2024-04-22T20:46:52.068642+0000","last_undegraded":"2024-04-22T20:46:52.068642+0000","last_fullsized":"2024-04-22T20:46:52.068642+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:57.296051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787489,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":944,"num_read_kb":55238,"num_write":1333,"num_write_kb":52838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"261'127","reported_seq":728,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.177008+0000","last_change":"2024-04-22T19:51:28.521641+0000","last_active":"2024-04-22T20:44:50.177008+0000","last_peered":"2024-04-22T20:44:50.177008+0000","last_clean":"2024-04-22T20:44:50.177008+0000","last_became_active":"2024-04-22T19:51:28.521372+0000","last_became_peered":"2024-04-22T19:51:28.521372+0000","last_unstale":"2024-04-22T20:44:50.177008+0000","last_undegraded":"2024-04-22T20:44:50.177008+0000","last_fullsized":"2024-04-22T20:44:50.177008+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:16.451481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":525,"num_read_kb":437,"num_write":198,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"261'224","reported_seq":878,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:08.497364+0000","last_change":"2024-04-22T19:52:04.618718+0000","last_active":"2024-04-22T20:47:08.497364+0000","last_peered":"2024-04-22T20:47:08.497364+0000","last_clean":"2024-04-22T20:47:08.497364+0000","last_became_active":"2024-04-22T19:52:04.618273+0000","last_became_peered":"2024-04-22T19:52:04.618273+0000","last_unstale":"2024-04-22T20:47:08.497364+0000","last_undegraded":"2024-04-22T20:47:08.497364+0000","last_fullsized":"2024-04-22T20:47:08.497364+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:09.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19315969,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":402,"num_read_kb":73466,"num_write":926,"num_write_kb":40708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"261'95","reported_seq":648,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180705+0000","last_change":"2024-04-22T19:51:28.513718+0000","last_active":"2024-04-22T20:44:40.180705+0000","last_peered":"2024-04-22T20:44:40.180705+0000","last_clean":"2024-04-22T20:44:40.180705+0000","last_became_active":"2024-04-22T19:51:28.513450+0000","last_became_peered":"2024-04-22T19:51:28.513450+0000","last_unstale":"2024-04-22T20:44:40.180705+0000","last_undegraded":"2024-04-22T20:44:40.180705+0000","last_fullsized":"2024-04-22T20:44:40.180705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:50.937435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":382,"num_read_kb":302,"num_write":152,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5385,"num_read_kb":3813,"num_write":2124,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1241,"ondisk_log_size":1241,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19082,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":13196,"num_read_kb":13004,"num_write":15956,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":38164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1853,"ondisk_log_size":1853,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1817836,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":16703,"num_read_kb":31506,"num_write":14213,"num_write_kb":11110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3293184,"data_stored":2776426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2244,"ondisk_log_size":2244,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1617,"num_read_kb":1920,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":85960,"num_read_kb":69763,"num_write":34673,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3889,"ondisk_log_size":3889,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":684407196,"num_objects":4507,"num_object_clones":0,"num_object_copies":13521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":30977,"num_read_kb":2216973,"num_write":73272,"num_write_kb":2138644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1075630080,"data_stored":1075630080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3665,"ondisk_log_size":3665,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":91317873,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":141,"num_read_kb":196064,"num_write":151,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182640640,"data_stored":182635746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":91317873,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":139,"num_read_kb":185784,"num_write":151,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182640640,"data_stored":182635746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575536,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":181480,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":185835520,"data_stored":183817698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":286390730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575538,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424572,"kb_used_data":162932,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":93947268,"statfs":{"total":96636764160,"available":96202002432,"internally_reserved":0,"allocated":166842368,"data_stored":164891323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":267909574},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575537,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308012,"kb_used_data":178340,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94063828,"statfs":{"total":96636764160,"available":96321359872,"internally_reserved":0,"allocated":182620160,"data_stored":180552293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575538,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":252084,"kb_used_data":166764,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94119756,"statfs":{"total":96636764160,"available":96378630144,"internally_reserved":0,"allocated":170766336,"data_stored":168845633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575538,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444004,"kb_used_data":196316,"kb_used_omap":1,"kb_used_meta":247678,"kb_avail":93927836,"statfs":{"total":96636764160,"available":96182104064,"internally_reserved":0,"allocated":201027584,"data_stored":199033946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":253622728},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575539,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":264928,"kb_used_data":171352,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94106912,"statfs":{"total":96636764160,"available":96365477888,"internally_reserved":0,"allocated":175464448,"data_stored":173547104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95812037},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575536,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330400,"kb_used_data":186840,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94041440,"statfs":{"total":96636764160,"available":96298434560,"internally_reserved":0,"allocated":191324160,"data_stored":189350803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575538,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434020,"kb_used_data":200156,"kb_used_omap":1,"kb_used_meta":233854,"kb_avail":93937820,"statfs":{"total":96636764160,"available":96192327680,"internally_reserved":0,"allocated":204959744,"data_stored":202964756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":239466950},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":147820544,"data_stored":147820544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139972608,"data_stored":139972608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105086976,"data_stored":105086976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":149049344,"data_stored":149049344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":120315904,"data_stored":120315904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":143065088,"data_stored":143065088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":133967872,"data_stored":133967872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136351744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":353680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":383474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":132055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":479987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":85199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":566114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":248456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":622592,"data_stored":527461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":5948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":4195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":3595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":6129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":4904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28405760,"data_stored":28403313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26214400,"data_stored":26214400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":18923520,"data_stored":18922784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10530816,"data_stored":10529105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24117248,"data_stored":24117248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37748736,"data_stored":37748736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24170496,"data_stored":24168864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24158208,"data_stored":24157393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":15728640,"data_stored":15728640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17879040,"data_stored":17877408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18915328,"data_stored":18914513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T20:48:19.867 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-22T20:48:19.867 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T20:48:20.210 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:20.222 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T20:48:20.224 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":1968,"stamp":"2024-04-22T20:48:18.327450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":869472021,"num_objects":4919,"num_object_clones":0,"num_object_copies":14345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4919,"num_whiteouts":0,"num_read":154246,"num_read_kb":2718939,"num_write":140739,"num_write_kb":2429877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13467,"ondisk_log_size":13467,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2919188,"kb_used_data":1444180,"kb_used_omap":12,"kb_used_meta":1474931,"kb_avail":752055532,"statfs":{"total":773094113280,"available":770104864768,"internally_reserved":0,"allocated":1478840320,"data_stored":1463003556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1510329906},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-520,"num_objects":-4,"num_object_clones":0,"num_object_copies":-8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4,"num_whiteouts":0,"num_read":34,"num_read_kb":39,"num_write":27,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005878"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.177805+0000","last_change":"2024-04-22T20:19:16.796805+0000","last_active":"2024-04-22T20:44:20.177805+0000","last_peered":"2024-04-22T20:44:20.177805+0000","last_clean":"2024-04-22T20:44:20.177805+0000","last_became_active":"2024-04-22T20:19:16.796520+0000","last_became_peered":"2024-04-22T20:19:16.796520+0000","last_unstale":"2024-04-22T20:44:20.177805+0000","last_undegraded":"2024-04-22T20:44:20.177805+0000","last_fullsized":"2024-04-22T20:44:20.177805+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:42.980562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"261'4","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186412+0000","last_change":"2024-04-22T20:05:17.900640+0000","last_active":"2024-04-22T20:45:20.186412+0000","last_peered":"2024-04-22T20:45:20.186412+0000","last_clean":"2024-04-22T20:45:20.186412+0000","last_became_active":"2024-04-22T20:05:17.900325+0000","last_became_peered":"2024-04-22T20:05:17.900325+0000","last_unstale":"2024-04-22T20:45:20.186412+0000","last_undegraded":"2024-04-22T20:45:20.186412+0000","last_fullsized":"2024-04-22T20:45:20.186412+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:55:20.209199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154687+0000","last_change":"2024-04-22T20:05:13.813157+0000","last_active":"2024-04-22T20:45:15.154687+0000","last_peered":"2024-04-22T20:45:15.154687+0000","last_clean":"2024-04-22T20:45:15.154687+0000","last_became_active":"2024-04-22T20:05:13.812933+0000","last_became_peered":"2024-04-22T20:05:13.812933+0000","last_unstale":"2024-04-22T20:45:15.154687+0000","last_undegraded":"2024-04-22T20:45:15.154687+0000","last_fullsized":"2024-04-22T20:45:15.154687+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:47:15.012252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"261'20","reported_seq":100,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.176985+0000","last_change":"2024-04-22T20:01:04.876250+0000","last_active":"2024-04-22T20:44:50.176985+0000","last_peered":"2024-04-22T20:44:50.176985+0000","last_clean":"2024-04-22T20:44:50.176985+0000","last_became_active":"2024-04-22T20:01:04.875981+0000","last_became_peered":"2024-04-22T20:01:04.875981+0000","last_unstale":"2024-04-22T20:44:50.176985+0000","last_undegraded":"2024-04-22T20:44:50.176985+0000","last_fullsized":"2024-04-22T20:44:50.176985+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:47:50.228681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.060560+0000","last_change":"2024-04-22T20:19:16.790684+0000","last_active":"2024-04-22T20:44:20.060560+0000","last_peered":"2024-04-22T20:44:20.060560+0000","last_clean":"2024-04-22T20:44:20.060560+0000","last_became_active":"2024-04-22T20:19:16.790377+0000","last_became_peered":"2024-04-22T20:19:16.790377+0000","last_unstale":"2024-04-22T20:44:20.060560+0000","last_undegraded":"2024-04-22T20:44:20.060560+0000","last_fullsized":"2024-04-22T20:44:20.060560+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:16:51.462209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.178227+0000","last_change":"2024-04-22T20:05:17.899675+0000","last_active":"2024-04-22T20:45:20.178227+0000","last_peered":"2024-04-22T20:45:20.178227+0000","last_clean":"2024-04-22T20:45:20.178227+0000","last_became_active":"2024-04-22T20:05:17.899338+0000","last_became_peered":"2024-04-22T20:05:17.899338+0000","last_unstale":"2024-04-22T20:45:20.178227+0000","last_undegraded":"2024-04-22T20:45:20.178227+0000","last_fullsized":"2024-04-22T20:45:20.178227+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:09:36.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"258'4","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.043137+0000","last_change":"2024-04-22T20:05:13.808152+0000","last_active":"2024-04-22T20:45:15.043137+0000","last_peered":"2024-04-22T20:45:15.043137+0000","last_clean":"2024-04-22T20:45:15.043137+0000","last_became_active":"2024-04-22T20:05:13.807231+0000","last_became_peered":"2024-04-22T20:05:13.807231+0000","last_unstale":"2024-04-22T20:45:15.043137+0000","last_undegraded":"2024-04-22T20:45:15.043137+0000","last_fullsized":"2024-04-22T20:45:15.043137+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:18:02.300257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"261'22","reported_seq":101,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.066043+0000","last_change":"2024-04-22T20:01:04.874850+0000","last_active":"2024-04-22T20:45:15.066043+0000","last_peered":"2024-04-22T20:45:15.066043+0000","last_clean":"2024-04-22T20:45:15.066043+0000","last_became_active":"2024-04-22T20:01:04.874575+0000","last_became_peered":"2024-04-22T20:01:04.874575+0000","last_unstale":"2024-04-22T20:45:15.066043+0000","last_undegraded":"2024-04-22T20:45:15.066043+0000","last_fullsized":"2024-04-22T20:45:15.066043+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:00:34.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":48,"num_write":47,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.298533+0000","last_change":"2024-04-22T20:19:16.794970+0000","last_active":"2024-04-22T20:43:30.298533+0000","last_peered":"2024-04-22T20:43:30.298533+0000","last_clean":"2024-04-22T20:43:30.298533+0000","last_became_active":"2024-04-22T20:19:16.794700+0000","last_became_peered":"2024-04-22T20:19:16.794700+0000","last_unstale":"2024-04-22T20:43:30.298533+0000","last_undegraded":"2024-04-22T20:43:30.298533+0000","last_fullsized":"2024-04-22T20:43:30.298533+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:59.399131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.177554+0000","last_change":"2024-04-22T20:05:18.211663+0000","last_active":"2024-04-22T20:44:55.177554+0000","last_peered":"2024-04-22T20:44:55.177554+0000","last_clean":"2024-04-22T20:44:55.177554+0000","last_became_active":"2024-04-22T20:05:18.211400+0000","last_became_peered":"2024-04-22T20:05:18.211400+0000","last_unstale":"2024-04-22T20:44:55.177554+0000","last_undegraded":"2024-04-22T20:44:55.177554+0000","last_fullsized":"2024-04-22T20:44:55.177554+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:07:00.571570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.067028+0000","last_change":"2024-04-22T20:05:13.809630+0000","last_active":"2024-04-22T20:45:25.067028+0000","last_peered":"2024-04-22T20:45:25.067028+0000","last_clean":"2024-04-22T20:45:25.067028+0000","last_became_active":"2024-04-22T20:05:13.809349+0000","last_became_peered":"2024-04-22T20:05:13.809349+0000","last_unstale":"2024-04-22T20:45:25.067028+0000","last_undegraded":"2024-04-22T20:45:25.067028+0000","last_fullsized":"2024-04-22T20:45:25.067028+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:04:26.130081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"261'24","reported_seq":114,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.167723+0000","last_change":"2024-04-22T20:01:04.870729+0000","last_active":"2024-04-22T20:43:35.167723+0000","last_peered":"2024-04-22T20:43:35.167723+0000","last_clean":"2024-04-22T20:43:35.167723+0000","last_became_active":"2024-04-22T20:01:04.869785+0000","last_became_peered":"2024-04-22T20:01:04.869785+0000","last_unstale":"2024-04-22T20:43:35.167723+0000","last_undegraded":"2024-04-22T20:43:35.167723+0000","last_fullsized":"2024-04-22T20:43:35.167723+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:24:59.438391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":82,"num_write":36,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.141064+0000","last_change":"2024-04-22T20:19:16.795441+0000","last_active":"2024-04-22T20:43:30.141064+0000","last_peered":"2024-04-22T20:43:30.141064+0000","last_clean":"2024-04-22T20:43:30.141064+0000","last_became_active":"2024-04-22T20:19:16.793667+0000","last_became_peered":"2024-04-22T20:19:16.793667+0000","last_unstale":"2024-04-22T20:43:30.141064+0000","last_undegraded":"2024-04-22T20:43:30.141064+0000","last_fullsized":"2024-04-22T20:43:30.141064+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:09.885678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"261'7","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:15.187088+0000","last_change":"2024-04-22T20:06:05.349958+0000","last_active":"2024-04-22T20:46:15.187088+0000","last_peered":"2024-04-22T20:46:15.187088+0000","last_clean":"2024-04-22T20:46:15.187088+0000","last_became_active":"2024-04-22T20:06:05.345554+0000","last_became_peered":"2024-04-22T20:06:05.345554+0000","last_unstale":"2024-04-22T20:46:15.187088+0000","last_undegraded":"2024-04-22T20:46:15.187088+0000","last_fullsized":"2024-04-22T20:46:15.187088+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:50:22.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6291456,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":17,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1f","version":"258'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.173992+0000","last_change":"2024-04-22T20:05:13.806565+0000","last_active":"2024-04-22T20:44:50.173992+0000","last_peered":"2024-04-22T20:44:50.173992+0000","last_clean":"2024-04-22T20:44:50.173992+0000","last_became_active":"2024-04-22T20:05:13.806246+0000","last_became_peered":"2024-04-22T20:05:13.806246+0000","last_unstale":"2024-04-22T20:44:50.173992+0000","last_undegraded":"2024-04-22T20:44:50.173992+0000","last_fullsized":"2024-04-22T20:44:50.173992+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:04:13.772282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":16720,"num_write":13,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"261'59","reported_seq":198,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.141957+0000","last_change":"2024-04-22T20:01:04.874395+0000","last_active":"2024-04-22T20:43:35.141957+0000","last_peered":"2024-04-22T20:43:35.141957+0000","last_clean":"2024-04-22T20:43:35.141957+0000","last_became_active":"2024-04-22T20:01:04.873899+0000","last_became_peered":"2024-04-22T20:01:04.873899+0000","last_unstale":"2024-04-22T20:43:35.141957+0000","last_undegraded":"2024-04-22T20:43:35.141957+0000","last_fullsized":"2024-04-22T20:43:35.141957+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:23:27.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":183,"num_write":99,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.155406+0000","last_change":"2024-04-22T20:19:16.794646+0000","last_active":"2024-04-22T20:45:20.155406+0000","last_peered":"2024-04-22T20:45:20.155406+0000","last_clean":"2024-04-22T20:45:20.155406+0000","last_became_active":"2024-04-22T20:19:16.794110+0000","last_became_peered":"2024-04-22T20:19:16.794110+0000","last_unstale":"2024-04-22T20:45:20.155406+0000","last_undegraded":"2024-04-22T20:45:20.155406+0000","last_fullsized":"2024-04-22T20:45:20.155406+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:42.561415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"261'1","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:10.176246+0000","last_change":"2024-04-22T20:05:17.898277+0000","last_active":"2024-04-22T20:44:10.176246+0000","last_peered":"2024-04-22T20:44:10.176246+0000","last_clean":"2024-04-22T20:44:10.176246+0000","last_became_active":"2024-04-22T20:05:17.897771+0000","last_became_peered":"2024-04-22T20:05:17.897771+0000","last_unstale":"2024-04-22T20:44:10.176246+0000","last_undegraded":"2024-04-22T20:44:10.176246+0000","last_fullsized":"2024-04-22T20:44:10.176246+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:22:07.487507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"258'5","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.170541+0000","last_change":"2024-04-22T20:05:13.812772+0000","last_active":"2024-04-22T20:43:55.170541+0000","last_peered":"2024-04-22T20:43:55.170541+0000","last_clean":"2024-04-22T20:43:55.170541+0000","last_became_active":"2024-04-22T20:05:13.812308+0000","last_became_peered":"2024-04-22T20:05:13.812308+0000","last_unstale":"2024-04-22T20:43:55.170541+0000","last_undegraded":"2024-04-22T20:43:55.170541+0000","last_fullsized":"2024-04-22T20:43:55.170541+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:52.451966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":9,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"261'8","reported_seq":72,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.141914+0000","last_change":"2024-04-22T20:01:04.877587+0000","last_active":"2024-04-22T20:43:35.141914+0000","last_peered":"2024-04-22T20:43:35.141914+0000","last_clean":"2024-04-22T20:43:35.141914+0000","last_became_active":"2024-04-22T20:01:04.877209+0000","last_became_peered":"2024-04-22T20:01:04.877209+0000","last_unstale":"2024-04-22T20:43:35.141914+0000","last_undegraded":"2024-04-22T20:43:35.141914+0000","last_fullsized":"2024-04-22T20:43:35.141914+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:57.901050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.037056+0000","last_change":"2024-04-22T20:19:16.789917+0000","last_active":"2024-04-22T20:44:20.037056+0000","last_peered":"2024-04-22T20:44:20.037056+0000","last_clean":"2024-04-22T20:44:20.037056+0000","last_became_active":"2024-04-22T20:19:16.789599+0000","last_became_peered":"2024-04-22T20:19:16.789599+0000","last_unstale":"2024-04-22T20:44:20.037056+0000","last_undegraded":"2024-04-22T20:44:20.037056+0000","last_fullsized":"2024-04-22T20:44:20.037056+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:15.659980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"261'1","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.303074+0000","last_change":"2024-04-22T20:05:17.891180+0000","last_active":"2024-04-22T20:44:00.303074+0000","last_peered":"2024-04-22T20:44:00.303074+0000","last_clean":"2024-04-22T20:44:00.303074+0000","last_became_active":"2024-04-22T20:05:17.890841+0000","last_became_peered":"2024-04-22T20:05:17.890841+0000","last_unstale":"2024-04-22T20:44:00.303074+0000","last_undegraded":"2024-04-22T20:44:00.303074+0000","last_fullsized":"2024-04-22T20:44:00.303074+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:53:40.164764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48416,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":48,"num_write":1,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.167430+0000","last_change":"2024-04-22T20:05:13.807783+0000","last_active":"2024-04-22T20:44:05.167430+0000","last_peered":"2024-04-22T20:44:05.167430+0000","last_clean":"2024-04-22T20:44:05.167430+0000","last_became_active":"2024-04-22T20:05:13.807516+0000","last_became_peered":"2024-04-22T20:05:13.807516+0000","last_unstale":"2024-04-22T20:44:05.167430+0000","last_undegraded":"2024-04-22T20:44:05.167430+0000","last_fullsized":"2024-04-22T20:44:05.167430+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:58:08.969201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"261'77","reported_seq":239,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.299254+0000","last_change":"2024-04-22T20:01:04.873339+0000","last_active":"2024-04-22T20:43:35.299254+0000","last_peered":"2024-04-22T20:43:35.299254+0000","last_clean":"2024-04-22T20:43:35.299254+0000","last_became_active":"2024-04-22T20:01:04.873015+0000","last_became_peered":"2024-04-22T20:01:04.873015+0000","last_unstale":"2024-04-22T20:43:35.299254+0000","last_undegraded":"2024-04-22T20:43:35.299254+0000","last_fullsized":"2024-04-22T20:43:35.299254+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:49:05.613359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":236,"num_write":125,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"261'1","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053060+0000","last_change":"2024-04-22T20:19:16.794883+0000","last_active":"2024-04-22T20:44:20.053060+0000","last_peered":"2024-04-22T20:44:20.053060+0000","last_clean":"2024-04-22T20:44:20.053060+0000","last_became_active":"2024-04-22T20:19:16.794564+0000","last_became_peered":"2024-04-22T20:19:16.794564+0000","last_unstale":"2024-04-22T20:44:20.053060+0000","last_undegraded":"2024-04-22T20:44:20.053060+0000","last_fullsized":"2024-04-22T20:44:20.053060+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:15.121442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"261'3","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314023+0000","last_change":"2024-04-22T20:05:17.901743+0000","last_active":"2024-04-22T20:45:20.314023+0000","last_peered":"2024-04-22T20:45:20.314023+0000","last_clean":"2024-04-22T20:45:20.314023+0000","last_became_active":"2024-04-22T20:05:17.901446+0000","last_became_peered":"2024-04-22T20:05:17.901446+0000","last_unstale":"2024-04-22T20:45:20.314023+0000","last_undegraded":"2024-04-22T20:45:20.314023+0000","last_fullsized":"2024-04-22T20:45:20.314023+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:13:01.401575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":11264,"num_write":8,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"261'1","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.177454+0000","last_change":"2024-04-22T20:05:13.798628+0000","last_active":"2024-04-22T20:45:15.177454+0000","last_peered":"2024-04-22T20:45:15.177454+0000","last_clean":"2024-04-22T20:45:15.177454+0000","last_became_active":"2024-04-22T20:05:13.798311+0000","last_became_peered":"2024-04-22T20:05:13.798311+0000","last_unstale":"2024-04-22T20:45:15.177454+0000","last_undegraded":"2024-04-22T20:45:15.177454+0000","last_fullsized":"2024-04-22T20:45:15.177454+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:21:50.192609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2048,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'27","reported_seq":116,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.176374+0000","last_change":"2024-04-22T20:01:04.873715+0000","last_active":"2024-04-22T20:44:45.176374+0000","last_peered":"2024-04-22T20:44:45.176374+0000","last_clean":"2024-04-22T20:44:45.176374+0000","last_became_active":"2024-04-22T20:01:04.873426+0000","last_became_peered":"2024-04-22T20:01:04.873426+0000","last_unstale":"2024-04-22T20:44:45.176374+0000","last_undegraded":"2024-04-22T20:44:45.176374+0000","last_fullsized":"2024-04-22T20:44:45.176374+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:29:40.987253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":80,"num_write":54,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053106+0000","last_change":"2024-04-22T20:19:16.800874+0000","last_active":"2024-04-22T20:44:20.053106+0000","last_peered":"2024-04-22T20:44:20.053106+0000","last_clean":"2024-04-22T20:44:20.053106+0000","last_became_active":"2024-04-22T20:19:16.800611+0000","last_became_peered":"2024-04-22T20:19:16.800611+0000","last_unstale":"2024-04-22T20:44:20.053106+0000","last_undegraded":"2024-04-22T20:44:20.053106+0000","last_fullsized":"2024-04-22T20:44:20.053106+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:27:17.425924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"258'4","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.155286+0000","last_change":"2024-04-22T20:05:17.892900+0000","last_active":"2024-04-22T20:45:20.155286+0000","last_peered":"2024-04-22T20:45:20.155286+0000","last_clean":"2024-04-22T20:45:20.155286+0000","last_became_active":"2024-04-22T20:05:17.892624+0000","last_became_peered":"2024-04-22T20:05:17.892624+0000","last_unstale":"2024-04-22T20:45:20.155286+0000","last_undegraded":"2024-04-22T20:45:20.155286+0000","last_fullsized":"2024-04-22T20:45:20.155286+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:02.296574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3072,"num_write":4,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'1","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179926+0000","last_change":"2024-04-22T20:05:13.816306+0000","last_active":"2024-04-22T20:45:15.179926+0000","last_peered":"2024-04-22T20:45:15.179926+0000","last_clean":"2024-04-22T20:45:15.179926+0000","last_became_active":"2024-04-22T20:05:13.816011+0000","last_became_peered":"2024-04-22T20:05:13.816011+0000","last_unstale":"2024-04-22T20:45:15.179926+0000","last_undegraded":"2024-04-22T20:45:15.179926+0000","last_fullsized":"2024-04-22T20:45:15.179926+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:44.934646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"261'43","reported_seq":150,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.038576+0000","last_change":"2024-04-22T20:01:04.872669+0000","last_active":"2024-04-22T20:44:35.038576+0000","last_peered":"2024-04-22T20:44:35.038576+0000","last_clean":"2024-04-22T20:44:35.038576+0000","last_became_active":"2024-04-22T20:01:04.872356+0000","last_became_peered":"2024-04-22T20:01:04.872356+0000","last_unstale":"2024-04-22T20:44:35.038576+0000","last_undegraded":"2024-04-22T20:44:35.038576+0000","last_fullsized":"2024-04-22T20:44:35.038576+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:11.955524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":119,"num_write":72,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041329+0000","last_change":"2024-04-22T20:19:16.796888+0000","last_active":"2024-04-22T20:45:00.041329+0000","last_peered":"2024-04-22T20:45:00.041329+0000","last_clean":"2024-04-22T20:45:00.041329+0000","last_became_active":"2024-04-22T20:19:16.796155+0000","last_became_peered":"2024-04-22T20:19:16.796155+0000","last_unstale":"2024-04-22T20:45:00.041329+0000","last_undegraded":"2024-04-22T20:45:00.041329+0000","last_fullsized":"2024-04-22T20:45:00.041329+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:25:57.681625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145610+0000","last_change":"2024-04-22T20:05:18.209303+0000","last_active":"2024-04-22T20:44:00.145610+0000","last_peered":"2024-04-22T20:44:00.145610+0000","last_clean":"2024-04-22T20:44:00.145610+0000","last_became_active":"2024-04-22T20:05:18.209023+0000","last_became_peered":"2024-04-22T20:05:18.209023+0000","last_unstale":"2024-04-22T20:44:00.145610+0000","last_undegraded":"2024-04-22T20:44:00.145610+0000","last_fullsized":"2024-04-22T20:44:00.145610+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:14:17.745357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"258'6","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.184202+0000","last_change":"2024-04-22T20:05:13.809878+0000","last_active":"2024-04-22T20:45:05.184202+0000","last_peered":"2024-04-22T20:45:05.184202+0000","last_clean":"2024-04-22T20:45:05.184202+0000","last_became_active":"2024-04-22T20:05:13.809548+0000","last_became_peered":"2024-04-22T20:05:13.809548+0000","last_unstale":"2024-04-22T20:45:05.184202+0000","last_undegraded":"2024-04-22T20:45:05.184202+0000","last_fullsized":"2024-04-22T20:45:05.184202+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:10.962028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":6146,"num_write":5,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'63","reported_seq":241,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.175344+0000","last_change":"2024-04-22T20:01:04.873570+0000","last_active":"2024-04-22T20:44:35.175344+0000","last_peered":"2024-04-22T20:44:35.175344+0000","last_clean":"2024-04-22T20:44:35.175344+0000","last_became_active":"2024-04-22T20:01:04.873236+0000","last_became_peered":"2024-04-22T20:01:04.873236+0000","last_unstale":"2024-04-22T20:44:35.175344+0000","last_undegraded":"2024-04-22T20:44:35.175344+0000","last_fullsized":"2024-04-22T20:44:35.175344+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:38.761169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":250,"num_write":111,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.064494+0000","last_change":"2024-04-22T20:19:16.791895+0000","last_active":"2024-04-22T20:45:00.064494+0000","last_peered":"2024-04-22T20:45:00.064494+0000","last_clean":"2024-04-22T20:45:00.064494+0000","last_became_active":"2024-04-22T20:19:16.791597+0000","last_became_peered":"2024-04-22T20:19:16.791597+0000","last_unstale":"2024-04-22T20:45:00.064494+0000","last_undegraded":"2024-04-22T20:45:00.064494+0000","last_fullsized":"2024-04-22T20:45:00.064494+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:51.409610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":40,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.153541+0000","last_change":"2024-04-22T20:05:18.211042+0000","last_active":"2024-04-22T20:45:05.153541+0000","last_peered":"2024-04-22T20:45:05.153541+0000","last_clean":"2024-04-22T20:45:05.153541+0000","last_became_active":"2024-04-22T20:05:18.210704+0000","last_became_peered":"2024-04-22T20:05:18.210704+0000","last_unstale":"2024-04-22T20:45:05.153541+0000","last_undegraded":"2024-04-22T20:45:05.153541+0000","last_fullsized":"2024-04-22T20:45:05.153541+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:53:47.345090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.048999+0000","last_change":"2024-04-22T20:05:13.811174+0000","last_active":"2024-04-22T20:43:55.048999+0000","last_peered":"2024-04-22T20:43:55.048999+0000","last_clean":"2024-04-22T20:43:55.048999+0000","last_became_active":"2024-04-22T20:05:13.810902+0000","last_became_peered":"2024-04-22T20:05:13.810902+0000","last_unstale":"2024-04-22T20:43:55.048999+0000","last_undegraded":"2024-04-22T20:43:55.048999+0000","last_fullsized":"2024-04-22T20:43:55.048999+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:05:52.847563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"250'9","reported_seq":76,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.149931+0000","last_change":"2024-04-22T20:01:04.874512+0000","last_active":"2024-04-22T20:44:35.149931+0000","last_peered":"2024-04-22T20:44:35.149931+0000","last_clean":"2024-04-22T20:44:35.149931+0000","last_became_active":"2024-04-22T20:01:04.874230+0000","last_became_peered":"2024-04-22T20:01:04.874230+0000","last_unstale":"2024-04-22T20:44:35.149931+0000","last_undegraded":"2024-04-22T20:44:35.149931+0000","last_fullsized":"2024-04-22T20:44:35.149931+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:17:12.953078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.059314+0000","last_change":"2024-04-22T20:19:16.795626+0000","last_active":"2024-04-22T20:45:00.059314+0000","last_peered":"2024-04-22T20:45:00.059314+0000","last_clean":"2024-04-22T20:45:00.059314+0000","last_became_active":"2024-04-22T20:19:16.795139+0000","last_became_peered":"2024-04-22T20:19:16.795139+0000","last_unstale":"2024-04-22T20:45:00.059314+0000","last_undegraded":"2024-04-22T20:45:00.059314+0000","last_fullsized":"2024-04-22T20:45:00.059314+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:17:47.054792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"261'15","reported_seq":74,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145637+0000","last_change":"2024-04-22T20:05:18.212823+0000","last_active":"2024-04-22T20:44:00.145637+0000","last_peered":"2024-04-22T20:44:00.145637+0000","last_clean":"2024-04-22T20:44:00.145637+0000","last_became_active":"2024-04-22T20:05:18.212336+0000","last_became_peered":"2024-04-22T20:05:18.212336+0000","last_unstale":"2024-04-22T20:44:00.145637+0000","last_undegraded":"2024-04-22T20:44:00.145637+0000","last_fullsized":"2024-04-22T20:44:00.145637+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:32.041183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":38918,"num_write":22,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"261'1","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.058251+0000","last_change":"2024-04-22T20:05:13.816425+0000","last_active":"2024-04-22T20:43:55.058251+0000","last_peered":"2024-04-22T20:43:55.058251+0000","last_clean":"2024-04-22T20:43:55.058251+0000","last_became_active":"2024-04-22T20:05:13.816036+0000","last_became_peered":"2024-04-22T20:05:13.816036+0000","last_unstale":"2024-04-22T20:43:55.058251+0000","last_undegraded":"2024-04-22T20:43:55.058251+0000","last_fullsized":"2024-04-22T20:43:55.058251+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:36.892934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'53","reported_seq":176,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.062011+0000","last_change":"2024-04-22T20:01:04.878348+0000","last_active":"2024-04-22T20:44:35.062011+0000","last_peered":"2024-04-22T20:44:35.062011+0000","last_clean":"2024-04-22T20:44:35.062011+0000","last_became_active":"2024-04-22T20:01:04.878067+0000","last_became_peered":"2024-04-22T20:01:04.878067+0000","last_unstale":"2024-04-22T20:44:35.062011+0000","last_undegraded":"2024-04-22T20:44:35.062011+0000","last_fullsized":"2024-04-22T20:44:35.062011+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:52:49.300812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":150,"num_write":99,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.183432+0000","last_change":"2024-04-22T20:19:16.796141+0000","last_active":"2024-04-22T20:45:00.183432+0000","last_peered":"2024-04-22T20:45:00.183432+0000","last_clean":"2024-04-22T20:45:00.183432+0000","last_became_active":"2024-04-22T20:19:16.795855+0000","last_became_peered":"2024-04-22T20:19:16.795855+0000","last_unstale":"2024-04-22T20:45:00.183432+0000","last_undegraded":"2024-04-22T20:45:00.183432+0000","last_fullsized":"2024-04-22T20:45:00.183432+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:02.646044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166753+0000","last_change":"2024-04-22T20:05:17.902267+0000","last_active":"2024-04-22T20:44:00.166753+0000","last_peered":"2024-04-22T20:44:00.166753+0000","last_clean":"2024-04-22T20:44:00.166753+0000","last_became_active":"2024-04-22T20:05:17.901954+0000","last_became_peered":"2024-04-22T20:05:17.901954+0000","last_unstale":"2024-04-22T20:44:00.166753+0000","last_undegraded":"2024-04-22T20:44:00.166753+0000","last_fullsized":"2024-04-22T20:44:00.166753+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:44:04.704303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'2","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.170642+0000","last_change":"2024-04-22T20:05:13.813528+0000","last_active":"2024-04-22T20:43:55.170642+0000","last_peered":"2024-04-22T20:43:55.170642+0000","last_clean":"2024-04-22T20:43:55.170642+0000","last_became_active":"2024-04-22T20:05:13.813256+0000","last_became_peered":"2024-04-22T20:05:13.813256+0000","last_unstale":"2024-04-22T20:43:55.170642+0000","last_undegraded":"2024-04-22T20:43:55.170642+0000","last_fullsized":"2024-04-22T20:43:55.170642+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:19.323647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":44,"num_write":2,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"261'46","reported_seq":164,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.064807+0000","last_change":"2024-04-22T20:01:04.874056+0000","last_active":"2024-04-22T20:45:35.064807+0000","last_peered":"2024-04-22T20:45:35.064807+0000","last_clean":"2024-04-22T20:45:35.064807+0000","last_became_active":"2024-04-22T20:01:04.873533+0000","last_became_peered":"2024-04-22T20:01:04.873533+0000","last_unstale":"2024-04-22T20:45:35.064807+0000","last_undegraded":"2024-04-22T20:45:35.064807+0000","last_fullsized":"2024-04-22T20:45:35.064807+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:43.868445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":192,"num_read_kb":137,"num_write":82,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158918+0000","last_change":"2024-04-22T20:19:16.793148+0000","last_active":"2024-04-22T20:45:50.158918+0000","last_peered":"2024-04-22T20:45:50.158918+0000","last_clean":"2024-04-22T20:45:50.158918+0000","last_became_active":"2024-04-22T20:19:16.792856+0000","last_became_peered":"2024-04-22T20:19:16.792856+0000","last_unstale":"2024-04-22T20:45:50.158918+0000","last_undegraded":"2024-04-22T20:45:50.158918+0000","last_fullsized":"2024-04-22T20:45:50.158918+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:34.522183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'7","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041319+0000","last_change":"2024-04-22T20:05:17.897292+0000","last_active":"2024-04-22T20:45:00.041319+0000","last_peered":"2024-04-22T20:45:00.041319+0000","last_clean":"2024-04-22T20:45:00.041319+0000","last_became_active":"2024-04-22T20:05:17.897040+0000","last_became_peered":"2024-04-22T20:05:17.897040+0000","last_unstale":"2024-04-22T20:45:00.041319+0000","last_undegraded":"2024-04-22T20:45:00.041319+0000","last_fullsized":"2024-04-22T20:45:00.041319+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:13:37.748724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":12290,"num_write":6,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"261'3","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.174581+0000","last_change":"2024-04-22T20:05:13.814475+0000","last_active":"2024-04-22T20:44:55.174581+0000","last_peered":"2024-04-22T20:44:55.174581+0000","last_clean":"2024-04-22T20:44:55.174581+0000","last_became_active":"2024-04-22T20:05:13.814222+0000","last_became_peered":"2024-04-22T20:05:13.814222+0000","last_unstale":"2024-04-22T20:44:55.174581+0000","last_undegraded":"2024-04-22T20:44:55.174581+0000","last_fullsized":"2024-04-22T20:44:55.174581+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:29:09.438132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"261'54","reported_seq":190,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063228+0000","last_change":"2024-04-22T20:01:04.873752+0000","last_active":"2024-04-22T20:45:25.063228+0000","last_peered":"2024-04-22T20:45:25.063228+0000","last_clean":"2024-04-22T20:45:25.063228+0000","last_became_active":"2024-04-22T20:01:04.873422+0000","last_became_peered":"2024-04-22T20:01:04.873422+0000","last_unstale":"2024-04-22T20:45:25.063228+0000","last_undegraded":"2024-04-22T20:45:25.063228+0000","last_fullsized":"2024-04-22T20:45:25.063228+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:49.464347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":184,"num_write":82,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:30.167040+0000","last_change":"2024-04-22T20:19:16.795990+0000","last_active":"2024-04-22T20:43:30.167040+0000","last_peered":"2024-04-22T20:43:30.167040+0000","last_clean":"2024-04-22T20:43:30.167040+0000","last_became_active":"2024-04-22T20:19:16.795633+0000","last_became_peered":"2024-04-22T20:19:16.795633+0000","last_unstale":"2024-04-22T20:43:30.167040+0000","last_undegraded":"2024-04-22T20:43:30.167040+0000","last_fullsized":"2024-04-22T20:43:30.167040+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:55.584364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"261'8","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041286+0000","last_change":"2024-04-22T20:05:17.900013+0000","last_active":"2024-04-22T20:45:00.041286+0000","last_peered":"2024-04-22T20:45:00.041286+0000","last_clean":"2024-04-22T20:45:00.041286+0000","last_became_active":"2024-04-22T20:05:17.899682+0000","last_became_peered":"2024-04-22T20:05:17.899682+0000","last_unstale":"2024-04-22T20:45:00.041286+0000","last_undegraded":"2024-04-22T20:45:00.041286+0000","last_fullsized":"2024-04-22T20:45:00.041286+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:57:08.679442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":8210,"num_write":12,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"261'5","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.152411+0000","last_change":"2024-04-22T20:05:13.813246+0000","last_active":"2024-04-22T20:44:55.152411+0000","last_peered":"2024-04-22T20:44:55.152411+0000","last_clean":"2024-04-22T20:44:55.152411+0000","last_became_active":"2024-04-22T20:05:13.812893+0000","last_became_peered":"2024-04-22T20:05:13.812893+0000","last_unstale":"2024-04-22T20:44:55.152411+0000","last_undegraded":"2024-04-22T20:44:55.152411+0000","last_fullsized":"2024-04-22T20:44:55.152411+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:26:36.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":15,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"261'47","reported_seq":163,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.181140+0000","last_change":"2024-04-22T20:01:04.878065+0000","last_active":"2024-04-22T20:45:25.181140+0000","last_peered":"2024-04-22T20:45:25.181140+0000","last_clean":"2024-04-22T20:45:25.181140+0000","last_became_active":"2024-04-22T20:01:04.877755+0000","last_became_peered":"2024-04-22T20:01:04.877755+0000","last_unstale":"2024-04-22T20:45:25.181140+0000","last_undegraded":"2024-04-22T20:45:25.181140+0000","last_fullsized":"2024-04-22T20:45:25.181140+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:41:43.369530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":135,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"260'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.178798+0000","last_change":"2024-04-22T20:19:16.793874+0000","last_active":"2024-04-22T20:45:05.178798+0000","last_peered":"2024-04-22T20:45:05.178798+0000","last_clean":"2024-04-22T20:45:05.178798+0000","last_became_active":"2024-04-22T20:19:16.793555+0000","last_became_peered":"2024-04-22T20:19:16.793555+0000","last_unstale":"2024-04-22T20:45:05.178798+0000","last_undegraded":"2024-04-22T20:45:05.178798+0000","last_fullsized":"2024-04-22T20:45:05.178798+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:33:04.108646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"258'2","reported_seq":44,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.152971+0000","last_change":"2024-04-22T20:05:17.890318+0000","last_active":"2024-04-22T20:45:00.152971+0000","last_peered":"2024-04-22T20:45:00.152971+0000","last_clean":"2024-04-22T20:45:00.152971+0000","last_became_active":"2024-04-22T20:05:17.890070+0000","last_became_peered":"2024-04-22T20:05:17.890070+0000","last_unstale":"2024-04-22T20:45:00.152971+0000","last_undegraded":"2024-04-22T20:45:00.152971+0000","last_fullsized":"2024-04-22T20:45:00.152971+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:14:41.324774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"261'3","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.310559+0000","last_change":"2024-04-22T20:05:13.807133+0000","last_active":"2024-04-22T20:44:55.310559+0000","last_peered":"2024-04-22T20:44:55.310559+0000","last_clean":"2024-04-22T20:44:55.310559+0000","last_became_active":"2024-04-22T20:05:13.806882+0000","last_became_peered":"2024-04-22T20:05:13.806882+0000","last_unstale":"2024-04-22T20:44:55.310559+0000","last_undegraded":"2024-04-22T20:44:55.310559+0000","last_fullsized":"2024-04-22T20:44:55.310559+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:52:50.911604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"261'11","reported_seq":80,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.178877+0000","last_change":"2024-04-22T20:01:04.861370+0000","last_active":"2024-04-22T20:45:25.178877+0000","last_peered":"2024-04-22T20:45:25.178877+0000","last_clean":"2024-04-22T20:45:25.178877+0000","last_became_active":"2024-04-22T20:01:04.860991+0000","last_became_peered":"2024-04-22T20:01:04.860991+0000","last_unstale":"2024-04-22T20:45:25.178877+0000","last_undegraded":"2024-04-22T20:45:25.178877+0000","last_fullsized":"2024-04-22T20:45:25.178877+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:26:49.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":31,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158885+0000","last_change":"2024-04-22T20:19:16.794789+0000","last_active":"2024-04-22T20:45:50.158885+0000","last_peered":"2024-04-22T20:45:50.158885+0000","last_clean":"2024-04-22T20:45:50.158885+0000","last_became_active":"2024-04-22T20:19:16.791995+0000","last_became_peered":"2024-04-22T20:19:16.791995+0000","last_unstale":"2024-04-22T20:45:50.158885+0000","last_undegraded":"2024-04-22T20:45:50.158885+0000","last_fullsized":"2024-04-22T20:45:50.158885+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:04:16.246591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"258'4","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.311157+0000","last_change":"2024-04-22T20:05:17.890780+0000","last_active":"2024-04-22T20:45:00.311157+0000","last_peered":"2024-04-22T20:45:00.311157+0000","last_clean":"2024-04-22T20:45:00.311157+0000","last_became_active":"2024-04-22T20:05:17.890408+0000","last_became_peered":"2024-04-22T20:05:17.890408+0000","last_unstale":"2024-04-22T20:45:00.311157+0000","last_undegraded":"2024-04-22T20:45:00.311157+0000","last_fullsized":"2024-04-22T20:45:00.311157+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:30:52.042989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1342,"num_write":4,"num_write_kb":1342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"258'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.182683+0000","last_change":"2024-04-22T20:05:13.808782+0000","last_active":"2024-04-22T20:44:55.182683+0000","last_peered":"2024-04-22T20:44:55.182683+0000","last_clean":"2024-04-22T20:44:55.182683+0000","last_became_active":"2024-04-22T20:05:13.808473+0000","last_became_peered":"2024-04-22T20:05:13.808473+0000","last_unstale":"2024-04-22T20:44:55.182683+0000","last_undegraded":"2024-04-22T20:44:55.182683+0000","last_fullsized":"2024-04-22T20:44:55.182683+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:39:30.056682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"261'46","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.155908+0000","last_change":"2024-04-22T20:01:04.875088+0000","last_active":"2024-04-22T20:45:25.155908+0000","last_peered":"2024-04-22T20:45:25.155908+0000","last_clean":"2024-04-22T20:45:25.155908+0000","last_became_active":"2024-04-22T20:01:04.874816+0000","last_became_peered":"2024-04-22T20:01:04.874816+0000","last_unstale":"2024-04-22T20:45:25.155908+0000","last_undegraded":"2024-04-22T20:45:25.155908+0000","last_fullsized":"2024-04-22T20:45:25.155908+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:59:09.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":291,"num_read_kb":200,"num_write":62,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.066966+0000","last_change":"2024-04-22T20:19:16.795282+0000","last_active":"2024-04-22T20:45:25.066966+0000","last_peered":"2024-04-22T20:45:25.066966+0000","last_clean":"2024-04-22T20:45:25.066966+0000","last_became_active":"2024-04-22T20:19:16.794954+0000","last_became_peered":"2024-04-22T20:19:16.794954+0000","last_unstale":"2024-04-22T20:45:25.066966+0000","last_undegraded":"2024-04-22T20:45:25.066966+0000","last_fullsized":"2024-04-22T20:45:25.066966+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:53:00.610478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"261'3","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158894+0000","last_change":"2024-04-22T20:05:17.889647+0000","last_active":"2024-04-22T20:45:50.158894+0000","last_peered":"2024-04-22T20:45:50.158894+0000","last_clean":"2024-04-22T20:45:50.158894+0000","last_became_active":"2024-04-22T20:05:17.889370+0000","last_became_peered":"2024-04-22T20:05:17.889370+0000","last_unstale":"2024-04-22T20:45:50.158894+0000","last_undegraded":"2024-04-22T20:45:50.158894+0000","last_fullsized":"2024-04-22T20:45:50.158894+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:26:55.859900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5246080,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":4100,"num_write":3,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"261'1","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.066445+0000","last_change":"2024-04-22T20:05:13.815613+0000","last_active":"2024-04-22T20:45:45.066445+0000","last_peered":"2024-04-22T20:45:45.066445+0000","last_clean":"2024-04-22T20:45:45.066445+0000","last_became_active":"2024-04-22T20:05:13.815317+0000","last_became_peered":"2024-04-22T20:05:13.815317+0000","last_unstale":"2024-04-22T20:45:45.066445+0000","last_undegraded":"2024-04-22T20:45:45.066445+0000","last_fullsized":"2024-04-22T20:45:45.066445+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:25:12.295293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"261'28","reported_seq":118,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.145012+0000","last_change":"2024-04-22T20:01:04.878886+0000","last_active":"2024-04-22T20:43:55.145012+0000","last_peered":"2024-04-22T20:43:55.145012+0000","last_clean":"2024-04-22T20:43:55.145012+0000","last_became_active":"2024-04-22T20:01:04.878599+0000","last_became_peered":"2024-04-22T20:01:04.878599+0000","last_unstale":"2024-04-22T20:43:55.145012+0000","last_undegraded":"2024-04-22T20:43:55.145012+0000","last_fullsized":"2024-04-22T20:43:55.145012+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:57:43.301162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":77,"num_write":54,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.050575+0000","last_change":"2024-04-22T20:19:16.794996+0000","last_active":"2024-04-22T20:44:05.050575+0000","last_peered":"2024-04-22T20:44:05.050575+0000","last_clean":"2024-04-22T20:44:05.050575+0000","last_became_active":"2024-04-22T20:19:16.793678+0000","last_became_peered":"2024-04-22T20:19:16.793678+0000","last_unstale":"2024-04-22T20:44:05.050575+0000","last_undegraded":"2024-04-22T20:44:05.050575+0000","last_fullsized":"2024-04-22T20:44:05.050575+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:48:13.172887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158865+0000","last_change":"2024-04-22T20:05:17.892875+0000","last_active":"2024-04-22T20:45:50.158865+0000","last_peered":"2024-04-22T20:45:50.158865+0000","last_clean":"2024-04-22T20:45:50.158865+0000","last_became_active":"2024-04-22T20:05:17.892604+0000","last_became_peered":"2024-04-22T20:05:17.892604+0000","last_unstale":"2024-04-22T20:45:50.158865+0000","last_undegraded":"2024-04-22T20:45:50.158865+0000","last_fullsized":"2024-04-22T20:45:50.158865+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:20.757720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"261'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.068955+0000","last_change":"2024-04-22T20:05:13.814193+0000","last_active":"2024-04-22T20:45:45.068955+0000","last_peered":"2024-04-22T20:45:45.068955+0000","last_clean":"2024-04-22T20:45:45.068955+0000","last_became_active":"2024-04-22T20:05:13.813946+0000","last_became_peered":"2024-04-22T20:05:13.813946+0000","last_unstale":"2024-04-22T20:45:45.068955+0000","last_undegraded":"2024-04-22T20:45:45.068955+0000","last_fullsized":"2024-04-22T20:45:45.068955+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:04:48.745503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":15362,"num_write":8,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"261'28","reported_seq":126,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.173638+0000","last_change":"2024-04-22T20:01:04.873158+0000","last_active":"2024-04-22T20:43:50.173638+0000","last_peered":"2024-04-22T20:43:50.173638+0000","last_clean":"2024-04-22T20:43:50.173638+0000","last_became_active":"2024-04-22T20:01:04.872834+0000","last_became_peered":"2024-04-22T20:01:04.872834+0000","last_unstale":"2024-04-22T20:43:50.173638+0000","last_undegraded":"2024-04-22T20:43:50.173638+0000","last_fullsized":"2024-04-22T20:43:50.173638+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:25.675744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":90,"num_write":50,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.313962+0000","last_change":"2024-04-22T20:19:16.791183+0000","last_active":"2024-04-22T20:45:20.313962+0000","last_peered":"2024-04-22T20:45:20.313962+0000","last_clean":"2024-04-22T20:45:20.313962+0000","last_became_active":"2024-04-22T20:19:16.790917+0000","last_became_peered":"2024-04-22T20:19:16.790917+0000","last_unstale":"2024-04-22T20:45:20.313962+0000","last_undegraded":"2024-04-22T20:45:20.313962+0000","last_fullsized":"2024-04-22T20:45:20.313962+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:12:37.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:00.048158+0000","last_change":"2024-04-22T20:05:18.211747+0000","last_active":"2024-04-22T20:46:00.048158+0000","last_peered":"2024-04-22T20:46:00.048158+0000","last_clean":"2024-04-22T20:46:00.048158+0000","last_became_active":"2024-04-22T20:05:18.211445+0000","last_became_peered":"2024-04-22T20:05:18.211445+0000","last_unstale":"2024-04-22T20:46:00.048158+0000","last_undegraded":"2024-04-22T20:46:00.048158+0000","last_fullsized":"2024-04-22T20:46:00.048158+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:14:07.558996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"258'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183750+0000","last_change":"2024-04-22T20:05:13.809343+0000","last_active":"2024-04-22T20:45:45.183750+0000","last_peered":"2024-04-22T20:45:45.183750+0000","last_clean":"2024-04-22T20:45:45.183750+0000","last_became_active":"2024-04-22T20:05:13.806928+0000","last_became_peered":"2024-04-22T20:05:13.806928+0000","last_unstale":"2024-04-22T20:45:45.183750+0000","last_undegraded":"2024-04-22T20:45:45.183750+0000","last_fullsized":"2024-04-22T20:45:45.183750+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:36:20.049785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"261'28","reported_seq":117,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.301681+0000","last_change":"2024-04-22T20:01:04.869517+0000","last_active":"2024-04-22T20:43:50.301681+0000","last_peered":"2024-04-22T20:43:50.301681+0000","last_clean":"2024-04-22T20:43:50.301681+0000","last_became_active":"2024-04-22T20:01:04.869226+0000","last_became_peered":"2024-04-22T20:01:04.869226+0000","last_unstale":"2024-04-22T20:43:50.301681+0000","last_undegraded":"2024-04-22T20:43:50.301681+0000","last_fullsized":"2024-04-22T20:43:50.301681+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:02:16.025206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":80,"num_write":56,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062354+0000","last_change":"2024-04-22T20:19:16.796398+0000","last_active":"2024-04-22T20:45:20.062354+0000","last_peered":"2024-04-22T20:45:20.062354+0000","last_clean":"2024-04-22T20:45:20.062354+0000","last_became_active":"2024-04-22T20:19:16.795524+0000","last_became_peered":"2024-04-22T20:19:16.795524+0000","last_unstale":"2024-04-22T20:45:20.062354+0000","last_undegraded":"2024-04-22T20:45:20.062354+0000","last_fullsized":"2024-04-22T20:45:20.062354+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:24:50.852300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'3","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158909+0000","last_change":"2024-04-22T20:05:17.899183+0000","last_active":"2024-04-22T20:45:50.158909+0000","last_peered":"2024-04-22T20:45:50.158909+0000","last_clean":"2024-04-22T20:45:50.158909+0000","last_became_active":"2024-04-22T20:05:17.898858+0000","last_became_peered":"2024-04-22T20:05:17.898858+0000","last_unstale":"2024-04-22T20:45:50.158909+0000","last_undegraded":"2024-04-22T20:45:50.158909+0000","last_fullsized":"2024-04-22T20:45:50.158909+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:21.640059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"258'3","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.047547+0000","last_change":"2024-04-22T20:05:13.808403+0000","last_active":"2024-04-22T20:45:55.047547+0000","last_peered":"2024-04-22T20:45:55.047547+0000","last_clean":"2024-04-22T20:45:55.047547+0000","last_became_active":"2024-04-22T20:05:13.807945+0000","last_became_peered":"2024-04-22T20:05:13.807945+0000","last_unstale":"2024-04-22T20:45:55.047547+0000","last_undegraded":"2024-04-22T20:45:55.047547+0000","last_fullsized":"2024-04-22T20:45:55.047547+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:02:06.578487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":8,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"261'24","reported_seq":105,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:50.057658+0000","last_change":"2024-04-22T20:01:04.872099+0000","last_active":"2024-04-22T20:43:50.057658+0000","last_peered":"2024-04-22T20:43:50.057658+0000","last_clean":"2024-04-22T20:43:50.057658+0000","last_became_active":"2024-04-22T20:01:04.871829+0000","last_became_peered":"2024-04-22T20:01:04.871829+0000","last_unstale":"2024-04-22T20:43:50.057658+0000","last_undegraded":"2024-04-22T20:43:50.057658+0000","last_fullsized":"2024-04-22T20:43:50.057658+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:30:28.808258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":61,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145629+0000","last_change":"2024-04-22T20:19:16.794419+0000","last_active":"2024-04-22T20:44:00.145629+0000","last_peered":"2024-04-22T20:44:00.145629+0000","last_clean":"2024-04-22T20:44:00.145629+0000","last_became_active":"2024-04-22T20:19:16.793727+0000","last_became_peered":"2024-04-22T20:19:16.793727+0000","last_unstale":"2024-04-22T20:44:00.145629+0000","last_undegraded":"2024-04-22T20:44:00.145629+0000","last_fullsized":"2024-04-22T20:44:00.145629+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:50.871340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"261'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169594+0000","last_change":"2024-04-22T20:05:17.899691+0000","last_active":"2024-04-22T20:44:20.169594+0000","last_peered":"2024-04-22T20:44:20.169594+0000","last_clean":"2024-04-22T20:44:20.169594+0000","last_became_active":"2024-04-22T20:05:17.899328+0000","last_became_peered":"2024-04-22T20:05:17.899328+0000","last_unstale":"2024-04-22T20:44:20.169594+0000","last_undegraded":"2024-04-22T20:44:20.169594+0000","last_fullsized":"2024-04-22T20:44:20.169594+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:57:36.265424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":9,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.172707+0000","last_change":"2024-04-22T20:05:13.812566+0000","last_active":"2024-04-22T20:44:15.172707+0000","last_peered":"2024-04-22T20:44:15.172707+0000","last_clean":"2024-04-22T20:44:15.172707+0000","last_became_active":"2024-04-22T20:05:13.811986+0000","last_became_peered":"2024-04-22T20:05:13.811986+0000","last_unstale":"2024-04-22T20:44:15.172707+0000","last_undegraded":"2024-04-22T20:44:15.172707+0000","last_fullsized":"2024-04-22T20:44:15.172707+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:29.867735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5291296,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":8240,"num_write":9,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"261'66","reported_seq":211,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:06.776789+0000","last_change":"2024-04-22T20:01:04.875092+0000","last_active":"2024-04-22T20:44:06.776789+0000","last_peered":"2024-04-22T20:44:06.776789+0000","last_clean":"2024-04-22T20:44:06.776789+0000","last_became_active":"2024-04-22T20:01:04.874827+0000","last_became_peered":"2024-04-22T20:01:04.874827+0000","last_unstale":"2024-04-22T20:44:06.776789+0000","last_undegraded":"2024-04-22T20:44:06.776789+0000","last_fullsized":"2024-04-22T20:44:06.776789+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:49.143833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":281,"num_read_kb":200,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.052345+0000","last_change":"2024-04-22T20:19:16.794759+0000","last_active":"2024-04-22T20:44:15.052345+0000","last_peered":"2024-04-22T20:44:15.052345+0000","last_clean":"2024-04-22T20:44:15.052345+0000","last_became_active":"2024-04-22T20:19:16.794434+0000","last_became_peered":"2024-04-22T20:19:16.794434+0000","last_unstale":"2024-04-22T20:44:15.052345+0000","last_undegraded":"2024-04-22T20:44:15.052345+0000","last_fullsized":"2024-04-22T20:44:15.052345+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:47.954843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"261'6","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.052938+0000","last_change":"2024-04-22T20:05:18.209353+0000","last_active":"2024-04-22T20:44:20.052938+0000","last_peered":"2024-04-22T20:44:20.052938+0000","last_clean":"2024-04-22T20:44:20.052938+0000","last_became_active":"2024-04-22T20:05:18.209025+0000","last_became_peered":"2024-04-22T20:05:18.209025+0000","last_unstale":"2024-04-22T20:44:20.052938+0000","last_undegraded":"2024-04-22T20:44:20.052938+0000","last_fullsized":"2024-04-22T20:44:20.052938+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:14:29.836329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":13312,"num_write":11,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"261'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.036411+0000","last_change":"2024-04-22T20:05:13.807877+0000","last_active":"2024-04-22T20:44:15.036411+0000","last_peered":"2024-04-22T20:44:15.036411+0000","last_clean":"2024-04-22T20:44:15.036411+0000","last_became_active":"2024-04-22T20:05:13.807603+0000","last_became_peered":"2024-04-22T20:05:13.807603+0000","last_unstale":"2024-04-22T20:44:15.036411+0000","last_undegraded":"2024-04-22T20:44:15.036411+0000","last_fullsized":"2024-04-22T20:44:15.036411+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:05:08.359131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":5120,"num_write":2,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"258'23","reported_seq":112,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.055220+0000","last_change":"2024-04-22T20:01:04.869170+0000","last_active":"2024-04-22T20:44:35.055220+0000","last_peered":"2024-04-22T20:44:35.055220+0000","last_clean":"2024-04-22T20:44:35.055220+0000","last_became_active":"2024-04-22T20:01:04.868847+0000","last_became_peered":"2024-04-22T20:01:04.868847+0000","last_unstale":"2024-04-22T20:44:35.055220+0000","last_undegraded":"2024-04-22T20:44:35.055220+0000","last_fullsized":"2024-04-22T20:44:35.055220+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:17:28.313341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":76,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.050611+0000","last_change":"2024-04-22T20:19:16.796622+0000","last_active":"2024-04-22T20:44:05.050611+0000","last_peered":"2024-04-22T20:44:05.050611+0000","last_clean":"2024-04-22T20:44:05.050611+0000","last_became_active":"2024-04-22T20:19:16.795613+0000","last_became_peered":"2024-04-22T20:19:16.795613+0000","last_unstale":"2024-04-22T20:44:05.050611+0000","last_undegraded":"2024-04-22T20:44:05.050611+0000","last_fullsized":"2024-04-22T20:44:05.050611+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:22:45.476024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"261'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.047029+0000","last_change":"2024-04-22T20:05:17.891585+0000","last_active":"2024-04-22T20:45:50.047029+0000","last_peered":"2024-04-22T20:45:50.047029+0000","last_clean":"2024-04-22T20:45:50.047029+0000","last_became_active":"2024-04-22T20:05:17.891331+0000","last_became_peered":"2024-04-22T20:05:17.891331+0000","last_unstale":"2024-04-22T20:45:50.047029+0000","last_undegraded":"2024-04-22T20:45:50.047029+0000","last_fullsized":"2024-04-22T20:45:50.047029+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:10.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":10240,"num_write":14,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.173996+0000","last_change":"2024-04-22T20:05:13.814072+0000","last_active":"2024-04-22T20:44:25.173996+0000","last_peered":"2024-04-22T20:44:25.173996+0000","last_clean":"2024-04-22T20:44:25.173996+0000","last_became_active":"2024-04-22T20:05:13.813811+0000","last_became_peered":"2024-04-22T20:05:13.813811+0000","last_unstale":"2024-04-22T20:44:25.173996+0000","last_undegraded":"2024-04-22T20:44:25.173996+0000","last_fullsized":"2024-04-22T20:44:25.173996+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:57.039895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"258'49","reported_seq":170,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.148654+0000","last_change":"2024-04-22T20:01:04.874422+0000","last_active":"2024-04-22T20:44:25.148654+0000","last_peered":"2024-04-22T20:44:25.148654+0000","last_clean":"2024-04-22T20:44:25.148654+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:44:25.148654+0000","last_undegraded":"2024-04-22T20:44:25.148654+0000","last_fullsized":"2024-04-22T20:44:25.148654+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:56:32.298685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":145,"num_write":78,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.171678+0000","last_change":"2024-04-22T20:19:16.793813+0000","last_active":"2024-04-22T20:44:05.171678+0000","last_peered":"2024-04-22T20:44:05.171678+0000","last_clean":"2024-04-22T20:44:05.171678+0000","last_became_active":"2024-04-22T20:19:16.793486+0000","last_became_peered":"2024-04-22T20:19:16.793486+0000","last_unstale":"2024-04-22T20:44:05.171678+0000","last_undegraded":"2024-04-22T20:44:05.171678+0000","last_fullsized":"2024-04-22T20:44:05.171678+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:47:12.878940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:30.054561+0000","last_change":"2024-04-22T20:05:17.897286+0000","last_active":"2024-04-22T20:44:30.054561+0000","last_peered":"2024-04-22T20:44:30.054561+0000","last_clean":"2024-04-22T20:44:30.054561+0000","last_became_active":"2024-04-22T20:05:17.897008+0000","last_became_peered":"2024-04-22T20:05:17.897008+0000","last_unstale":"2024-04-22T20:44:30.054561+0000","last_undegraded":"2024-04-22T20:44:30.054561+0000","last_fullsized":"2024-04-22T20:44:30.054561+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:56:58.848648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"261'11","reported_seq":72,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182061+0000","last_change":"2024-04-22T20:05:13.798707+0000","last_active":"2024-04-22T20:45:45.182061+0000","last_peered":"2024-04-22T20:45:45.182061+0000","last_clean":"2024-04-22T20:45:45.182061+0000","last_became_active":"2024-04-22T20:05:13.798319+0000","last_became_peered":"2024-04-22T20:05:13.798319+0000","last_unstale":"2024-04-22T20:45:45.182061+0000","last_undegraded":"2024-04-22T20:45:45.182061+0000","last_fullsized":"2024-04-22T20:45:45.182061+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:02.215898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":20,"num_read_kb":16388,"num_write":14,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"261'37","reported_seq":138,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.306289+0000","last_change":"2024-04-22T20:01:04.869368+0000","last_active":"2024-04-22T20:44:25.306289+0000","last_peered":"2024-04-22T20:44:25.306289+0000","last_clean":"2024-04-22T20:44:25.306289+0000","last_became_active":"2024-04-22T20:01:04.869079+0000","last_became_peered":"2024-04-22T20:01:04.869079+0000","last_unstale":"2024-04-22T20:44:25.306289+0000","last_undegraded":"2024-04-22T20:44:25.306289+0000","last_fullsized":"2024-04-22T20:44:25.306289+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:36:35.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":61,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053026+0000","last_change":"2024-04-22T20:19:16.796317+0000","last_active":"2024-04-22T20:44:20.053026+0000","last_peered":"2024-04-22T20:44:20.053026+0000","last_clean":"2024-04-22T20:44:20.053026+0000","last_became_active":"2024-04-22T20:19:16.795462+0000","last_became_peered":"2024-04-22T20:19:16.795462+0000","last_unstale":"2024-04-22T20:44:20.053026+0000","last_undegraded":"2024-04-22T20:44:20.053026+0000","last_fullsized":"2024-04-22T20:44:20.053026+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:38:48.277835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"258'6","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.178822+0000","last_change":"2024-04-22T20:05:17.892351+0000","last_active":"2024-04-22T20:45:05.178822+0000","last_peered":"2024-04-22T20:45:05.178822+0000","last_clean":"2024-04-22T20:45:05.178822+0000","last_became_active":"2024-04-22T20:05:17.892044+0000","last_became_peered":"2024-04-22T20:05:17.892044+0000","last_unstale":"2024-04-22T20:45:05.178822+0000","last_undegraded":"2024-04-22T20:45:05.178822+0000","last_fullsized":"2024-04-22T20:45:05.178822+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:47:37.172131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"258'4","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057775+0000","last_change":"2024-04-22T20:05:13.809668+0000","last_active":"2024-04-22T20:44:50.057775+0000","last_peered":"2024-04-22T20:44:50.057775+0000","last_clean":"2024-04-22T20:44:50.057775+0000","last_became_active":"2024-04-22T20:05:13.809368+0000","last_became_peered":"2024-04-22T20:05:13.809368+0000","last_unstale":"2024-04-22T20:44:50.057775+0000","last_undegraded":"2024-04-22T20:44:50.057775+0000","last_fullsized":"2024-04-22T20:44:50.057775+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:35.986772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"261'35","reported_seq":138,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.174074+0000","last_change":"2024-04-22T20:01:04.874032+0000","last_active":"2024-04-22T20:44:25.174074+0000","last_peered":"2024-04-22T20:44:25.174074+0000","last_clean":"2024-04-22T20:44:25.174074+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:44:25.174074+0000","last_undegraded":"2024-04-22T20:44:25.174074+0000","last_fullsized":"2024-04-22T20:44:25.174074+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:44.638764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":59,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.177834+0000","last_change":"2024-04-22T20:19:16.791940+0000","last_active":"2024-04-22T20:44:20.177834+0000","last_peered":"2024-04-22T20:44:20.177834+0000","last_clean":"2024-04-22T20:44:20.177834+0000","last_became_active":"2024-04-22T20:19:16.791663+0000","last_became_peered":"2024-04-22T20:19:16.791663+0000","last_unstale":"2024-04-22T20:44:20.177834+0000","last_undegraded":"2024-04-22T20:44:20.177834+0000","last_fullsized":"2024-04-22T20:44:20.177834+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:35:00.807867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"261'5","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.177498+0000","last_change":"2024-04-22T20:05:18.211014+0000","last_active":"2024-04-22T20:44:55.177498+0000","last_peered":"2024-04-22T20:44:55.177498+0000","last_clean":"2024-04-22T20:44:55.177498+0000","last_became_active":"2024-04-22T20:05:18.210736+0000","last_became_peered":"2024-04-22T20:05:18.210736+0000","last_unstale":"2024-04-22T20:44:55.177498+0000","last_undegraded":"2024-04-22T20:44:55.177498+0000","last_fullsized":"2024-04-22T20:44:55.177498+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:55:21.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":20482,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"261'5","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.064529+0000","last_change":"2024-04-22T20:05:13.813554+0000","last_active":"2024-04-22T20:45:00.064529+0000","last_peered":"2024-04-22T20:45:00.064529+0000","last_clean":"2024-04-22T20:45:00.064529+0000","last_became_active":"2024-04-22T20:05:13.813277+0000","last_became_peered":"2024-04-22T20:05:13.813277+0000","last_unstale":"2024-04-22T20:45:00.064529+0000","last_undegraded":"2024-04-22T20:45:00.064529+0000","last_fullsized":"2024-04-22T20:45:00.064529+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:43:53.225799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":15,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"261'61","reported_seq":194,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.306363+0000","last_change":"2024-04-22T20:01:04.872160+0000","last_active":"2024-04-22T20:44:25.306363+0000","last_peered":"2024-04-22T20:44:25.306363+0000","last_clean":"2024-04-22T20:44:25.306363+0000","last_became_active":"2024-04-22T20:01:04.871859+0000","last_became_peered":"2024-04-22T20:01:04.871859+0000","last_unstale":"2024-04-22T20:44:25.306363+0000","last_undegraded":"2024-04-22T20:44:25.306363+0000","last_fullsized":"2024-04-22T20:44:25.306363+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:17.093967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":172,"num_write":109,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.044253+0000","last_change":"2024-04-22T20:19:16.790599+0000","last_active":"2024-04-22T20:45:25.044253+0000","last_peered":"2024-04-22T20:45:25.044253+0000","last_clean":"2024-04-22T20:45:25.044253+0000","last_became_active":"2024-04-22T20:19:16.789688+0000","last_became_peered":"2024-04-22T20:19:16.789688+0000","last_unstale":"2024-04-22T20:45:25.044253+0000","last_undegraded":"2024-04-22T20:45:25.044253+0000","last_fullsized":"2024-04-22T20:45:25.044253+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:11.661623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"258'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053135+0000","last_change":"2024-04-22T20:05:18.211736+0000","last_active":"2024-04-22T20:44:20.053135+0000","last_peered":"2024-04-22T20:44:20.053135+0000","last_clean":"2024-04-22T20:44:20.053135+0000","last_became_active":"2024-04-22T20:05:18.211478+0000","last_became_peered":"2024-04-22T20:05:18.211478+0000","last_unstale":"2024-04-22T20:44:20.053135+0000","last_undegraded":"2024-04-22T20:44:20.053135+0000","last_fullsized":"2024-04-22T20:44:20.053135+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:06:10.866642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.177134+0000","last_change":"2024-04-22T20:05:13.811784+0000","last_active":"2024-04-22T20:44:15.177134+0000","last_peered":"2024-04-22T20:44:15.177134+0000","last_clean":"2024-04-22T20:44:15.177134+0000","last_became_active":"2024-04-22T20:05:13.811561+0000","last_became_peered":"2024-04-22T20:05:13.811561+0000","last_unstale":"2024-04-22T20:44:15.177134+0000","last_undegraded":"2024-04-22T20:44:15.177134+0000","last_fullsized":"2024-04-22T20:44:15.177134+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:35:04.990469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":9,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"261'33","reported_seq":135,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.059415+0000","last_change":"2024-04-22T20:01:04.875229+0000","last_active":"2024-04-22T20:45:00.059415+0000","last_peered":"2024-04-22T20:45:00.059415+0000","last_clean":"2024-04-22T20:45:00.059415+0000","last_became_active":"2024-04-22T20:01:04.874790+0000","last_became_peered":"2024-04-22T20:01:04.874790+0000","last_unstale":"2024-04-22T20:45:00.059415+0000","last_undegraded":"2024-04-22T20:45:00.059415+0000","last_fullsized":"2024-04-22T20:45:00.059415+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:48.031491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":100,"num_write":57,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169578+0000","last_change":"2024-04-22T20:19:16.794065+0000","last_active":"2024-04-22T20:44:20.169578+0000","last_peered":"2024-04-22T20:44:20.169578+0000","last_clean":"2024-04-22T20:44:20.169578+0000","last_became_active":"2024-04-22T20:19:16.793188+0000","last_became_peered":"2024-04-22T20:19:16.793188+0000","last_unstale":"2024-04-22T20:44:20.169578+0000","last_undegraded":"2024-04-22T20:44:20.169578+0000","last_fullsized":"2024-04-22T20:44:20.169578+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:39.203315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"261'3","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169675+0000","last_change":"2024-04-22T20:05:17.901392+0000","last_active":"2024-04-22T20:44:20.169675+0000","last_peered":"2024-04-22T20:44:20.169675+0000","last_clean":"2024-04-22T20:44:20.169675+0000","last_became_active":"2024-04-22T20:05:17.901128+0000","last_became_peered":"2024-04-22T20:05:17.901128+0000","last_unstale":"2024-04-22T20:44:20.169675+0000","last_undegraded":"2024-04-22T20:44:20.169675+0000","last_fullsized":"2024-04-22T20:44:20.169675+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:50:08.370833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":8,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"261'4","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.052367+0000","last_change":"2024-04-22T20:05:13.815731+0000","last_active":"2024-04-22T20:44:15.052367+0000","last_peered":"2024-04-22T20:44:15.052367+0000","last_clean":"2024-04-22T20:44:15.052367+0000","last_became_active":"2024-04-22T20:05:13.815332+0000","last_became_peered":"2024-04-22T20:05:13.815332+0000","last_unstale":"2024-04-22T20:44:15.052367+0000","last_undegraded":"2024-04-22T20:44:15.052367+0000","last_fullsized":"2024-04-22T20:44:15.052367+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:07:16.209328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":9216,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'28","reported_seq":124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040298+0000","last_change":"2024-04-22T20:01:04.872148+0000","last_active":"2024-04-22T20:44:50.040298+0000","last_peered":"2024-04-22T20:44:50.040298+0000","last_clean":"2024-04-22T20:44:50.040298+0000","last_became_active":"2024-04-22T20:01:04.871840+0000","last_became_peered":"2024-04-22T20:01:04.871840+0000","last_unstale":"2024-04-22T20:44:50.040298+0000","last_undegraded":"2024-04-22T20:44:50.040298+0000","last_fullsized":"2024-04-22T20:44:50.040298+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:23.933273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":89,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"261'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.066551+0000","last_change":"2024-04-22T20:19:16.794411+0000","last_active":"2024-04-22T20:45:20.066551+0000","last_peered":"2024-04-22T20:45:20.066551+0000","last_clean":"2024-04-22T20:45:20.066551+0000","last_became_active":"2024-04-22T20:19:16.794123+0000","last_became_peered":"2024-04-22T20:19:16.794123+0000","last_unstale":"2024-04-22T20:45:20.066551+0000","last_undegraded":"2024-04-22T20:45:20.066551+0000","last_fullsized":"2024-04-22T20:45:20.066551+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:39.176302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"261'2","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.040820+0000","last_change":"2024-04-22T20:05:17.895539+0000","last_active":"2024-04-22T20:44:55.040820+0000","last_peered":"2024-04-22T20:44:55.040820+0000","last_clean":"2024-04-22T20:44:55.040820+0000","last_became_active":"2024-04-22T20:05:17.895177+0000","last_became_peered":"2024-04-22T20:05:17.895177+0000","last_unstale":"2024-04-22T20:44:55.040820+0000","last_undegraded":"2024-04-22T20:44:55.040820+0000","last_fullsized":"2024-04-22T20:44:55.040820+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:46:34.641098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1024,"num_write":2,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"261'2","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063150+0000","last_change":"2024-04-22T20:05:13.809357+0000","last_active":"2024-04-22T20:45:25.063150+0000","last_peered":"2024-04-22T20:45:25.063150+0000","last_clean":"2024-04-22T20:45:25.063150+0000","last_became_active":"2024-04-22T20:05:13.809047+0000","last_became_peered":"2024-04-22T20:05:13.809047+0000","last_unstale":"2024-04-22T20:45:25.063150+0000","last_undegraded":"2024-04-22T20:45:25.063150+0000","last_fullsized":"2024-04-22T20:45:25.063150+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:17.206006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":2,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"261'33","reported_seq":130,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.181926+0000","last_change":"2024-04-22T20:01:04.872953+0000","last_active":"2024-04-22T20:44:50.181926+0000","last_peered":"2024-04-22T20:44:50.181926+0000","last_clean":"2024-04-22T20:44:50.181926+0000","last_became_active":"2024-04-22T20:01:04.872370+0000","last_became_peered":"2024-04-22T20:01:04.872370+0000","last_unstale":"2024-04-22T20:44:50.181926+0000","last_undegraded":"2024-04-22T20:44:50.181926+0000","last_fullsized":"2024-04-22T20:44:50.181926+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:22:37.771203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":57,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.180655+0000","last_change":"2024-04-22T20:19:16.793728+0000","last_active":"2024-04-22T20:45:20.180655+0000","last_peered":"2024-04-22T20:45:20.180655+0000","last_clean":"2024-04-22T20:45:20.180655+0000","last_became_active":"2024-04-22T20:19:16.793429+0000","last_became_peered":"2024-04-22T20:19:16.793429+0000","last_unstale":"2024-04-22T20:45:20.180655+0000","last_undegraded":"2024-04-22T20:45:20.180655+0000","last_fullsized":"2024-04-22T20:45:20.180655+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:27:12.156563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"261'1","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.187790+0000","last_change":"2024-04-22T20:05:17.901476+0000","last_active":"2024-04-22T20:45:30.187790+0000","last_peered":"2024-04-22T20:45:30.187790+0000","last_clean":"2024-04-22T20:45:30.187790+0000","last_became_active":"2024-04-22T20:05:17.900874+0000","last_became_peered":"2024-04-22T20:05:17.900874+0000","last_unstale":"2024-04-22T20:45:30.187790+0000","last_undegraded":"2024-04-22T20:45:30.187790+0000","last_fullsized":"2024-04-22T20:45:30.187790+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:20:07.700066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057705+0000","last_change":"2024-04-22T20:05:13.814214+0000","last_active":"2024-04-22T20:44:50.057705+0000","last_peered":"2024-04-22T20:44:50.057705+0000","last_clean":"2024-04-22T20:44:50.057705+0000","last_became_active":"2024-04-22T20:05:13.813934+0000","last_became_peered":"2024-04-22T20:05:13.813934+0000","last_unstale":"2024-04-22T20:44:50.057705+0000","last_undegraded":"2024-04-22T20:44:50.057705+0000","last_fullsized":"2024-04-22T20:44:50.057705+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:15.355174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1024,"num_write":1,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"261'37","reported_seq":146,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.173934+0000","last_change":"2024-04-22T20:01:04.869971+0000","last_active":"2024-04-22T20:44:50.173934+0000","last_peered":"2024-04-22T20:44:50.173934+0000","last_clean":"2024-04-22T20:44:50.173934+0000","last_became_active":"2024-04-22T20:01:04.869660+0000","last_became_peered":"2024-04-22T20:01:04.869660+0000","last_unstale":"2024-04-22T20:44:50.173934+0000","last_undegraded":"2024-04-22T20:44:50.173934+0000","last_fullsized":"2024-04-22T20:44:50.173934+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:08:02.708029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":113,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.173415+0000","last_change":"2024-04-22T20:19:16.795977+0000","last_active":"2024-04-22T20:44:20.173415+0000","last_peered":"2024-04-22T20:44:20.173415+0000","last_clean":"2024-04-22T20:44:20.173415+0000","last_became_active":"2024-04-22T20:19:16.795509+0000","last_became_peered":"2024-04-22T20:19:16.795509+0000","last_unstale":"2024-04-22T20:44:20.173415+0000","last_undegraded":"2024-04-22T20:44:20.173415+0000","last_fullsized":"2024-04-22T20:44:20.173415+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:07.456021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.310587+0000","last_change":"2024-04-22T20:05:17.891942+0000","last_active":"2024-04-22T20:44:55.310587+0000","last_peered":"2024-04-22T20:44:55.310587+0000","last_clean":"2024-04-22T20:44:55.310587+0000","last_became_active":"2024-04-22T20:05:17.891523+0000","last_became_peered":"2024-04-22T20:05:17.891523+0000","last_unstale":"2024-04-22T20:44:55.310587+0000","last_undegraded":"2024-04-22T20:44:55.310587+0000","last_fullsized":"2024-04-22T20:44:55.310587+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:38.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"261'1","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040268+0000","last_change":"2024-04-22T20:05:13.805972+0000","last_active":"2024-04-22T20:44:50.040268+0000","last_peered":"2024-04-22T20:44:50.040268+0000","last_clean":"2024-04-22T20:44:50.040268+0000","last_became_active":"2024-04-22T20:05:13.805704+0000","last_became_peered":"2024-04-22T20:05:13.805704+0000","last_unstale":"2024-04-22T20:44:50.040268+0000","last_undegraded":"2024-04-22T20:44:50.040268+0000","last_fullsized":"2024-04-22T20:44:50.040268+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:03.816391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"261'65","reported_seq":224,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309884+0000","last_change":"2024-04-22T20:01:04.866275+0000","last_active":"2024-04-22T20:44:50.309884+0000","last_peered":"2024-04-22T20:44:50.309884+0000","last_clean":"2024-04-22T20:44:50.309884+0000","last_became_active":"2024-04-22T20:01:04.865981+0000","last_became_peered":"2024-04-22T20:01:04.865981+0000","last_unstale":"2024-04-22T20:44:50.309884+0000","last_undegraded":"2024-04-22T20:44:50.309884+0000","last_fullsized":"2024-04-22T20:44:50.309884+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:25.969487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":226,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:30.054611+0000","last_change":"2024-04-22T20:19:16.799325+0000","last_active":"2024-04-22T20:44:30.054611+0000","last_peered":"2024-04-22T20:44:30.054611+0000","last_clean":"2024-04-22T20:44:30.054611+0000","last_became_active":"2024-04-22T20:19:16.799063+0000","last_became_peered":"2024-04-22T20:19:16.799063+0000","last_unstale":"2024-04-22T20:44:30.054611+0000","last_undegraded":"2024-04-22T20:44:30.054611+0000","last_fullsized":"2024-04-22T20:44:30.054611+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:20:02.254828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"261'1","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.174634+0000","last_change":"2024-04-22T20:05:17.901995+0000","last_active":"2024-04-22T20:44:55.174634+0000","last_peered":"2024-04-22T20:44:55.174634+0000","last_clean":"2024-04-22T20:44:55.174634+0000","last_became_active":"2024-04-22T20:05:17.901680+0000","last_became_peered":"2024-04-22T20:05:17.901680+0000","last_unstale":"2024-04-22T20:44:55.174634+0000","last_undegraded":"2024-04-22T20:44:55.174634+0000","last_fullsized":"2024-04-22T20:44:55.174634+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:26.628494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40145,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":80,"num_write":1,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309933+0000","last_change":"2024-04-22T20:05:13.811164+0000","last_active":"2024-04-22T20:44:50.309933+0000","last_peered":"2024-04-22T20:44:50.309933+0000","last_clean":"2024-04-22T20:44:50.309933+0000","last_became_active":"2024-04-22T20:05:13.810865+0000","last_became_peered":"2024-04-22T20:05:13.810865+0000","last_unstale":"2024-04-22T20:44:50.309933+0000","last_undegraded":"2024-04-22T20:44:50.309933+0000","last_fullsized":"2024-04-22T20:44:50.309933+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:47:15.297255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"261'57","reported_seq":180,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.061413+0000","last_change":"2024-04-22T20:01:04.870803+0000","last_active":"2024-04-22T20:45:15.061413+0000","last_peered":"2024-04-22T20:45:15.061413+0000","last_clean":"2024-04-22T20:45:15.061413+0000","last_became_active":"2024-04-22T20:01:04.870253+0000","last_became_peered":"2024-04-22T20:01:04.870253+0000","last_unstale":"2024-04-22T20:45:15.061413+0000","last_undegraded":"2024-04-22T20:45:15.061413+0000","last_fullsized":"2024-04-22T20:45:15.061413+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:32:34.069500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":145,"num_write":102,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"261'281","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.065532+0000","last_change":"2024-04-22T19:53:04.823797+0000","last_active":"2024-04-22T20:45:10.065532+0000","last_peered":"2024-04-22T20:45:10.065532+0000","last_clean":"2024-04-22T20:45:10.065532+0000","last_became_active":"2024-04-22T19:53:04.823351+0000","last_became_peered":"2024-04-22T19:53:04.823351+0000","last_unstale":"2024-04-22T20:45:10.065532+0000","last_undegraded":"2024-04-22T20:45:10.065532+0000","last_fullsized":"2024-04-22T20:45:10.065532+0000","mapping_epoch":35,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:24.940347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9460561,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":447,"num_read_kb":495,"num_write":1235,"num_write_kb":39497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"261'169","reported_seq":763,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174873+0000","last_change":"2024-04-22T19:51:28.511862+0000","last_active":"2024-04-22T20:44:00.174873+0000","last_peered":"2024-04-22T20:44:00.174873+0000","last_clean":"2024-04-22T20:44:00.174873+0000","last_became_active":"2024-04-22T19:51:28.511595+0000","last_became_peered":"2024-04-22T19:51:28.511595+0000","last_unstale":"2024-04-22T20:44:00.174873+0000","last_undegraded":"2024-04-22T20:44:00.174873+0000","last_fullsized":"2024-04-22T20:44:00.174873+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:36.200495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":537,"num_write":262,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"261'248","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:48.785097+0000","last_change":"2024-04-22T19:51:26.496852+0000","last_active":"2024-04-22T20:46:48.785097+0000","last_peered":"2024-04-22T20:46:48.785097+0000","last_clean":"2024-04-22T20:46:48.785097+0000","last_became_active":"2024-04-22T19:51:26.496188+0000","last_became_peered":"2024-04-22T19:51:26.496188+0000","last_unstale":"2024-04-22T20:46:48.785097+0000","last_undegraded":"2024-04-22T20:46:48.785097+0000","last_fullsized":"2024-04-22T20:46:48.785097+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:51:56.333450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10265266,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":429,"num_read_kb":9758,"num_write":1044,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"261'314","reported_seq":886,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:28.101937+0000","last_change":"2024-04-22T19:51:28.519482+0000","last_active":"2024-04-22T20:47:28.101937+0000","last_peered":"2024-04-22T20:47:28.101937+0000","last_clean":"2024-04-22T20:47:28.101937+0000","last_became_active":"2024-04-22T19:51:28.519111+0000","last_became_peered":"2024-04-22T19:51:28.519111+0000","last_unstale":"2024-04-22T20:47:28.101937+0000","last_undegraded":"2024-04-22T20:47:28.101937+0000","last_fullsized":"2024-04-22T20:47:28.101937+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:06:08.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1190,"num_read_kb":913,"num_write":510,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"261'247","reported_seq":868,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:36.787572+0000","last_change":"2024-04-22T19:51:26.503143+0000","last_active":"2024-04-22T20:46:36.787572+0000","last_peered":"2024-04-22T20:46:36.787572+0000","last_clean":"2024-04-22T20:46:36.787572+0000","last_became_active":"2024-04-22T19:51:26.502744+0000","last_became_peered":"2024-04-22T19:51:26.502744+0000","last_unstale":"2024-04-22T20:46:36.787572+0000","last_undegraded":"2024-04-22T20:46:36.787572+0000","last_fullsized":"2024-04-22T20:46:36.787572+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:30:34.457597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9763357,"num_objects":80,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":370,"num_read_kb":7116,"num_write":962,"num_write_kb":21496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"261'150","reported_seq":726,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166689+0000","last_change":"2024-04-22T19:51:28.518949+0000","last_active":"2024-04-22T20:44:00.166689+0000","last_peered":"2024-04-22T20:44:00.166689+0000","last_clean":"2024-04-22T20:44:00.166689+0000","last_became_active":"2024-04-22T19:51:28.518650+0000","last_became_peered":"2024-04-22T19:51:28.518650+0000","last_unstale":"2024-04-22T20:44:00.166689+0000","last_undegraded":"2024-04-22T20:44:00.166689+0000","last_fullsized":"2024-04-22T20:44:00.166689+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:25:03.391586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":572,"num_read_kb":453,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"261'259","reported_seq":874,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:14.145384+0000","last_change":"2024-04-22T19:51:26.504934+0000","last_active":"2024-04-22T20:47:14.145384+0000","last_peered":"2024-04-22T20:47:14.145384+0000","last_clean":"2024-04-22T20:47:14.145384+0000","last_became_active":"2024-04-22T19:51:26.504450+0000","last_became_peered":"2024-04-22T19:51:26.504450+0000","last_unstale":"2024-04-22T20:47:14.145384+0000","last_undegraded":"2024-04-22T20:47:14.145384+0000","last_fullsized":"2024-04-22T20:47:14.145384+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":200,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:47:54.670392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20493175,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":359,"num_read_kb":25508,"num_write":1125,"num_write_kb":37655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"261'81","reported_seq":676,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.171137+0000","last_change":"2024-04-22T19:52:04.620094+0000","last_active":"2024-04-22T20:44:00.171137+0000","last_peered":"2024-04-22T20:44:00.171137+0000","last_clean":"2024-04-22T20:44:00.171137+0000","last_became_active":"2024-04-22T19:52:04.619688+0000","last_became_peered":"2024-04-22T19:52:04.619688+0000","last_unstale":"2024-04-22T20:44:00.171137+0000","last_undegraded":"2024-04-22T20:44:00.171137+0000","last_fullsized":"2024-04-22T20:44:00.171137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:50:37.537623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":353,"num_read_kb":299,"num_write":120,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"261'222","reported_seq":910,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:03.977215+0000","last_change":"2024-04-22T19:51:26.509141+0000","last_active":"2024-04-22T20:47:03.977215+0000","last_peered":"2024-04-22T20:47:03.977215+0000","last_clean":"2024-04-22T20:47:03.977215+0000","last_became_active":"2024-04-22T19:51:26.507185+0000","last_became_peered":"2024-04-22T19:51:26.507185+0000","last_unstale":"2024-04-22T20:47:03.977215+0000","last_undegraded":"2024-04-22T20:47:03.977215+0000","last_fullsized":"2024-04-22T20:47:03.977215+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":200,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:14.928877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563190,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":383,"num_read_kb":10200,"num_write":866,"num_write_kb":30630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"261'40","reported_seq":594,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.316183+0000","last_change":"2024-04-22T19:51:28.513981+0000","last_active":"2024-04-22T20:45:35.316183+0000","last_peered":"2024-04-22T20:45:35.316183+0000","last_clean":"2024-04-22T20:45:35.316183+0000","last_became_active":"2024-04-22T19:51:28.513693+0000","last_became_peered":"2024-04-22T19:51:28.513693+0000","last_unstale":"2024-04-22T20:45:35.316183+0000","last_undegraded":"2024-04-22T20:45:35.316183+0000","last_fullsized":"2024-04-22T20:45:35.316183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:54:32.701108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"261'389","reported_seq":1131,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:54.327300+0000","last_change":"2024-04-22T19:51:26.509287+0000","last_active":"2024-04-22T20:46:54.327300+0000","last_peered":"2024-04-22T20:46:54.327300+0000","last_clean":"2024-04-22T20:46:54.327300+0000","last_became_active":"2024-04-22T19:51:26.508716+0000","last_became_peered":"2024-04-22T19:51:26.508716+0000","last_unstale":"2024-04-22T20:46:54.327300+0000","last_undegraded":"2024-04-22T20:46:54.327300+0000","last_fullsized":"2024-04-22T20:46:54.327300+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":300,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:42:06.130701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11174792,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":980,"num_read_kb":17244,"num_write":1468,"num_write_kb":25883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"261'144","reported_seq":742,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181403+0000","last_change":"2024-04-22T19:51:28.519053+0000","last_active":"2024-04-22T20:45:40.181403+0000","last_peered":"2024-04-22T20:45:40.181403+0000","last_clean":"2024-04-22T20:45:40.181403+0000","last_became_active":"2024-04-22T19:51:28.518501+0000","last_became_peered":"2024-04-22T19:51:28.518501+0000","last_unstale":"2024-04-22T20:45:40.181403+0000","last_undegraded":"2024-04-22T20:45:40.181403+0000","last_fullsized":"2024-04-22T20:45:40.181403+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:36.369584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":597,"num_read_kb":478,"num_write":226,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"261'300","reported_seq":961,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.049692+0000","last_change":"2024-04-22T19:51:26.508109+0000","last_active":"2024-04-22T20:44:00.049692+0000","last_peered":"2024-04-22T20:44:00.049692+0000","last_clean":"2024-04-22T20:44:00.049692+0000","last_became_active":"2024-04-22T19:51:26.507141+0000","last_became_peered":"2024-04-22T19:51:26.507141+0000","last_unstale":"2024-04-22T20:44:00.049692+0000","last_undegraded":"2024-04-22T20:44:00.049692+0000","last_fullsized":"2024-04-22T20:44:00.049692+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:27.201168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11136278,"num_objects":86,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":473,"num_read_kb":43743,"num_write":1312,"num_write_kb":32515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"261'151","reported_seq":736,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.067532+0000","last_change":"2024-04-22T19:51:28.516437+0000","last_active":"2024-04-22T20:47:38.067532+0000","last_peered":"2024-04-22T20:47:38.067532+0000","last_clean":"2024-04-22T20:47:38.067532+0000","last_became_active":"2024-04-22T19:51:28.516012+0000","last_became_peered":"2024-04-22T19:51:28.516012+0000","last_unstale":"2024-04-22T20:47:38.067532+0000","last_undegraded":"2024-04-22T20:47:38.067532+0000","last_fullsized":"2024-04-22T20:47:38.067532+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:38.182307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":470,"num_write":219,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"261'268","reported_seq":912,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:28.486641+0000","last_change":"2024-04-22T19:51:26.505832+0000","last_active":"2024-04-22T20:46:28.486641+0000","last_peered":"2024-04-22T20:46:28.486641+0000","last_clean":"2024-04-22T20:46:28.486641+0000","last_became_active":"2024-04-22T19:51:26.502928+0000","last_became_peered":"2024-04-22T19:51:26.502928+0000","last_unstale":"2024-04-22T20:46:28.486641+0000","last_undegraded":"2024-04-22T20:46:28.486641+0000","last_fullsized":"2024-04-22T20:46:28.486641+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:47:30.201226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9686578,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":378,"num_read_kb":51847,"num_write":1138,"num_write_kb":39313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"261'60","reported_seq":629,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:57.506260+0000","last_change":"2024-04-22T19:51:28.510044+0000","last_active":"2024-04-22T20:46:57.506260+0000","last_peered":"2024-04-22T20:46:57.506260+0000","last_clean":"2024-04-22T20:46:57.506260+0000","last_became_active":"2024-04-22T19:51:28.509774+0000","last_became_peered":"2024-04-22T19:51:28.509774+0000","last_unstale":"2024-04-22T20:46:57.506260+0000","last_undegraded":"2024-04-22T20:46:57.506260+0000","last_fullsized":"2024-04-22T20:46:57.506260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:55:37.172957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":211,"num_write":81,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"261'125","reported_seq":734,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176694+0000","last_change":"2024-04-22T19:51:28.505929+0000","last_active":"2024-04-22T20:45:10.176694+0000","last_peered":"2024-04-22T20:45:10.176694+0000","last_clean":"2024-04-22T20:45:10.176694+0000","last_became_active":"2024-04-22T19:51:28.505562+0000","last_became_peered":"2024-04-22T19:51:28.505562+0000","last_unstale":"2024-04-22T20:45:10.176694+0000","last_undegraded":"2024-04-22T20:45:10.176694+0000","last_fullsized":"2024-04-22T20:45:10.176694+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:33:32.464020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":540,"num_read_kb":443,"num_write":193,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"261'253","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057867+0000","last_change":"2024-04-22T19:51:26.507324+0000","last_active":"2024-04-22T20:44:50.057867+0000","last_peered":"2024-04-22T20:44:50.057867+0000","last_clean":"2024-04-22T20:44:50.057867+0000","last_became_active":"2024-04-22T19:51:26.504717+0000","last_became_peered":"2024-04-22T19:51:26.504717+0000","last_unstale":"2024-04-22T20:44:50.057867+0000","last_undegraded":"2024-04-22T20:44:50.057867+0000","last_fullsized":"2024-04-22T20:44:50.057867+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:31:10.322998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6322974,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":420,"num_read_kb":28705,"num_write":1056,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"261'105","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060776+0000","last_change":"2024-04-22T19:51:28.519503+0000","last_active":"2024-04-22T20:45:10.060776+0000","last_peered":"2024-04-22T20:45:10.060776+0000","last_clean":"2024-04-22T20:45:10.060776+0000","last_became_active":"2024-04-22T19:51:28.519230+0000","last_became_peered":"2024-04-22T19:51:28.519230+0000","last_unstale":"2024-04-22T20:45:10.060776+0000","last_undegraded":"2024-04-22T20:45:10.060776+0000","last_fullsized":"2024-04-22T20:45:10.060776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:23:00.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":465,"num_read_kb":383,"num_write":180,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"261'321","reported_seq":978,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.308459+0000","last_change":"2024-04-22T19:51:26.496564+0000","last_active":"2024-04-22T20:44:40.308459+0000","last_peered":"2024-04-22T20:44:40.308459+0000","last_clean":"2024-04-22T20:44:40.308459+0000","last_became_active":"2024-04-22T19:51:26.496156+0000","last_became_peered":"2024-04-22T19:51:26.496156+0000","last_unstale":"2024-04-22T20:44:40.308459+0000","last_undegraded":"2024-04-22T20:44:40.308459+0000","last_fullsized":"2024-04-22T20:44:40.308459+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:25.947249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77420,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":551,"num_read_kb":792,"num_write":1456,"num_write_kb":25865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"261'172","reported_seq":764,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186369+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:45:20.186369+0000","last_peered":"2024-04-22T20:45:20.186369+0000","last_clean":"2024-04-22T20:45:20.186369+0000","last_became_active":"2024-04-22T19:51:28.515673+0000","last_became_peered":"2024-04-22T19:51:28.515673+0000","last_unstale":"2024-04-22T20:45:20.186369+0000","last_undegraded":"2024-04-22T20:45:20.186369+0000","last_fullsized":"2024-04-22T20:45:20.186369+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:06.001104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":666,"num_read_kb":529,"num_write":258,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"261'291","reported_seq":1001,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.308420+0000","last_change":"2024-04-22T19:51:26.502589+0000","last_active":"2024-04-22T20:44:40.308420+0000","last_peered":"2024-04-22T20:44:40.308420+0000","last_clean":"2024-04-22T20:44:40.308420+0000","last_became_active":"2024-04-22T19:51:26.502156+0000","last_became_peered":"2024-04-22T19:51:26.502156+0000","last_unstale":"2024-04-22T20:44:40.308420+0000","last_undegraded":"2024-04-22T20:44:40.308420+0000","last_fullsized":"2024-04-22T20:44:40.308420+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:22:39.535038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23449640,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":550,"num_read_kb":68133,"num_write":1243,"num_write_kb":41095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"261'290","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:09.419441+0000","last_change":"2024-04-22T19:51:28.510666+0000","last_active":"2024-04-22T20:47:09.419441+0000","last_peered":"2024-04-22T20:47:09.419441+0000","last_clean":"2024-04-22T20:47:09.419441+0000","last_became_active":"2024-04-22T19:51:28.510415+0000","last_became_peered":"2024-04-22T19:51:28.510415+0000","last_unstale":"2024-04-22T20:47:09.419441+0000","last_undegraded":"2024-04-22T20:47:09.419441+0000","last_fullsized":"2024-04-22T20:47:09.419441+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:43.003473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1119,"num_read_kb":869,"num_write":475,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"261'251","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172460+0000","last_change":"2024-04-22T19:51:26.507272+0000","last_active":"2024-04-22T20:44:40.172460+0000","last_peered":"2024-04-22T20:44:40.172460+0000","last_clean":"2024-04-22T20:44:40.172460+0000","last_became_active":"2024-04-22T19:51:26.506671+0000","last_became_peered":"2024-04-22T19:51:26.506671+0000","last_unstale":"2024-04-22T20:44:40.172460+0000","last_undegraded":"2024-04-22T20:44:40.172460+0000","last_fullsized":"2024-04-22T20:44:40.172460+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:54:08.993777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8504557,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":414,"num_read_kb":6535,"num_write":1034,"num_write_kb":36003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"261'155","reported_seq":775,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.062504+0000","last_change":"2024-04-22T19:51:28.512453+0000","last_active":"2024-04-22T20:44:40.062504+0000","last_peered":"2024-04-22T20:44:40.062504+0000","last_clean":"2024-04-22T20:44:40.062504+0000","last_became_active":"2024-04-22T19:51:28.511889+0000","last_became_peered":"2024-04-22T19:51:28.511889+0000","last_unstale":"2024-04-22T20:44:40.062504+0000","last_undegraded":"2024-04-22T20:44:40.062504+0000","last_fullsized":"2024-04-22T20:44:40.062504+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:45.195439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":679,"num_read_kb":547,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"261'265","reported_seq":912,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.157609+0000","last_change":"2024-04-22T19:51:26.498087+0000","last_active":"2024-04-22T20:47:38.157609+0000","last_peered":"2024-04-22T20:47:38.157609+0000","last_clean":"2024-04-22T20:47:38.157609+0000","last_became_active":"2024-04-22T19:51:26.497718+0000","last_became_peered":"2024-04-22T19:51:26.497718+0000","last_unstale":"2024-04-22T20:47:38.157609+0000","last_undegraded":"2024-04-22T20:47:38.157609+0000","last_fullsized":"2024-04-22T20:47:38.157609+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":200,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:50:38.709806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6162293,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":402,"num_read_kb":30110,"num_write":1095,"num_write_kb":27210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"261'93","reported_seq":699,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:44.069351+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:46:44.069351+0000","last_peered":"2024-04-22T20:46:44.069351+0000","last_clean":"2024-04-22T20:46:44.069351+0000","last_became_active":"2024-04-22T19:51:28.507673+0000","last_became_peered":"2024-04-22T19:51:28.507673+0000","last_unstale":"2024-04-22T20:46:44.069351+0000","last_undegraded":"2024-04-22T20:46:44.069351+0000","last_fullsized":"2024-04-22T20:46:44.069351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:18:40.338876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":404,"num_read_kb":350,"num_write":133,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"261'347","reported_seq":1058,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.243867+0000","last_change":"2024-04-22T19:51:26.511696+0000","last_active":"2024-04-22T20:46:30.243867+0000","last_peered":"2024-04-22T20:46:30.243867+0000","last_clean":"2024-04-22T20:46:30.243867+0000","last_became_active":"2024-04-22T19:51:26.511213+0000","last_became_peered":"2024-04-22T19:51:26.511213+0000","last_unstale":"2024-04-22T20:46:30.243867+0000","last_undegraded":"2024-04-22T20:46:30.243867+0000","last_fullsized":"2024-04-22T20:46:30.243867+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:51:44.984733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13939483,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":664,"num_read_kb":36415,"num_write":1732,"num_write_kb":40524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"261'126","reported_seq":647,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.145449+0000","last_change":"2024-04-22T19:51:36.567880+0000","last_active":"2024-04-22T20:48:04.145449+0000","last_peered":"2024-04-22T20:48:04.145449+0000","last_clean":"2024-04-22T20:48:04.145449+0000","last_became_active":"2024-04-22T19:51:36.567561+0000","last_became_peered":"2024-04-22T19:51:36.567561+0000","last_unstale":"2024-04-22T20:48:04.145449+0000","last_undegraded":"2024-04-22T20:48:04.145449+0000","last_fullsized":"2024-04-22T20:48:04.145449+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:10:43.936463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":168,"num_write":307,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"261'38","reported_seq":584,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154700+0000","last_change":"2024-04-22T19:51:28.520238+0000","last_active":"2024-04-22T20:45:15.154700+0000","last_peered":"2024-04-22T20:45:15.154700+0000","last_clean":"2024-04-22T20:45:15.154700+0000","last_became_active":"2024-04-22T19:51:28.519994+0000","last_became_peered":"2024-04-22T19:51:28.519994+0000","last_unstale":"2024-04-22T20:45:15.154700+0000","last_undegraded":"2024-04-22T20:45:15.154700+0000","last_fullsized":"2024-04-22T20:45:15.154700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:48:21.210710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":138,"num_write":52,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"261'264","reported_seq":997,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060800+0000","last_change":"2024-04-22T19:53:05.051779+0000","last_active":"2024-04-22T20:45:10.060800+0000","last_peered":"2024-04-22T20:45:10.060800+0000","last_clean":"2024-04-22T20:45:10.060800+0000","last_became_active":"2024-04-22T19:53:05.051272+0000","last_became_peered":"2024-04-22T19:53:05.051272+0000","last_unstale":"2024-04-22T20:45:10.060800+0000","last_undegraded":"2024-04-22T20:45:10.060800+0000","last_fullsized":"2024-04-22T20:45:10.060800+0000","mapping_epoch":35,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:14:57.868131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6579999,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":609,"num_read_kb":11069,"num_write":1217,"num_write_kb":33730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180678+0000","last_change":"2024-04-22T19:51:30.521121+0000","last_active":"2024-04-22T20:44:40.180678+0000","last_peered":"2024-04-22T20:44:40.180678+0000","last_clean":"2024-04-22T20:44:40.180678+0000","last_became_active":"2024-04-22T19:51:30.520848+0000","last_became_peered":"2024-04-22T19:51:30.520848+0000","last_unstale":"2024-04-22T20:44:40.180678+0000","last_undegraded":"2024-04-22T20:44:40.180678+0000","last_fullsized":"2024-04-22T20:44:40.180678+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:30.383458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"261'324","reported_seq":923,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.193392+0000","last_change":"2024-04-22T19:51:32.537967+0000","last_active":"2024-04-22T20:47:10.193392+0000","last_peered":"2024-04-22T20:47:10.193392+0000","last_clean":"2024-04-22T20:47:10.193392+0000","last_became_active":"2024-04-22T19:51:32.537695+0000","last_became_peered":"2024-04-22T19:51:32.537695+0000","last_unstale":"2024-04-22T20:47:10.193392+0000","last_undegraded":"2024-04-22T20:47:10.193392+0000","last_fullsized":"2024-04-22T20:47:10.193392+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:12:32.127700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58993,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":537,"num_read_kb":1015,"num_write":432,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.057042+0000","last_change":"2024-04-22T19:51:34.557621+0000","last_active":"2024-04-22T20:44:45.057042+0000","last_peered":"2024-04-22T20:44:45.057042+0000","last_clean":"2024-04-22T20:44:45.057042+0000","last_became_active":"2024-04-22T19:51:34.557285+0000","last_became_peered":"2024-04-22T19:51:34.557285+0000","last_unstale":"2024-04-22T20:44:45.057042+0000","last_undegraded":"2024-04-22T20:44:45.057042+0000","last_fullsized":"2024-04-22T20:44:45.057042+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:14.949781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"261'2399","reported_seq":4258,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.711091+0000","last_change":"2024-04-22T19:51:36.567463+0000","last_active":"2024-04-22T20:48:03.711091+0000","last_peered":"2024-04-22T20:48:03.711091+0000","last_clean":"2024-04-22T20:48:03.711091+0000","last_became_active":"2024-04-22T19:51:36.567195+0000","last_became_peered":"2024-04-22T19:51:36.567195+0000","last_unstale":"2024-04-22T20:48:03.711091+0000","last_undegraded":"2024-04-22T20:48:03.711091+0000","last_fullsized":"2024-04-22T20:48:03.711091+0000","mapping_epoch":30,"log_start":"261'2300","ondisk_log_start":"261'2300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":2300,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:46:00.619550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5986,"num_read_kb":5990,"num_write":4781,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"261'130","reported_seq":720,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:53.028609+0000","last_change":"2024-04-22T19:51:28.513076+0000","last_active":"2024-04-22T20:46:53.028609+0000","last_peered":"2024-04-22T20:46:53.028609+0000","last_clean":"2024-04-22T20:46:53.028609+0000","last_became_active":"2024-04-22T19:51:28.512786+0000","last_became_peered":"2024-04-22T19:51:28.512786+0000","last_unstale":"2024-04-22T20:46:53.028609+0000","last_undegraded":"2024-04-22T20:46:53.028609+0000","last_fullsized":"2024-04-22T20:46:53.028609+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:20:09.250321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":408,"num_write":191,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"261'229","reported_seq":832,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:55.640581+0000","last_change":"2024-04-22T19:51:26.500209+0000","last_active":"2024-04-22T20:46:55.640581+0000","last_peered":"2024-04-22T20:46:55.640581+0000","last_clean":"2024-04-22T20:46:55.640581+0000","last_became_active":"2024-04-22T19:51:26.499885+0000","last_became_peered":"2024-04-22T19:51:26.499885+0000","last_unstale":"2024-04-22T20:46:55.640581+0000","last_undegraded":"2024-04-22T20:46:55.640581+0000","last_fullsized":"2024-04-22T20:46:55.640581+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:06:40.641478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8391210,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":339,"num_read_kb":16305,"num_write":901,"num_write_kb":23193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154709+0000","last_change":"2024-04-22T19:51:30.527682+0000","last_active":"2024-04-22T20:45:15.154709+0000","last_peered":"2024-04-22T20:45:15.154709+0000","last_clean":"2024-04-22T20:45:15.154709+0000","last_became_active":"2024-04-22T19:51:30.526622+0000","last_became_peered":"2024-04-22T19:51:30.526622+0000","last_unstale":"2024-04-22T20:45:15.154709+0000","last_undegraded":"2024-04-22T20:45:15.154709+0000","last_fullsized":"2024-04-22T20:45:15.154709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:20:42.245730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"261'86","reported_seq":611,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.173330+0000","last_change":"2024-04-22T19:51:32.536487+0000","last_active":"2024-04-22T20:44:45.173330+0000","last_peered":"2024-04-22T20:44:45.173330+0000","last_clean":"2024-04-22T20:44:45.173330+0000","last_became_active":"2024-04-22T19:51:32.536180+0000","last_became_peered":"2024-04-22T19:51:32.536180+0000","last_unstale":"2024-04-22T20:44:45.173330+0000","last_undegraded":"2024-04-22T20:44:45.173330+0000","last_fullsized":"2024-04-22T20:44:45.173330+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:21.299001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":95,"num_write":64,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.173306+0000","last_change":"2024-04-22T19:51:34.549833+0000","last_active":"2024-04-22T20:44:45.173306+0000","last_peered":"2024-04-22T20:44:45.173306+0000","last_clean":"2024-04-22T20:44:45.173306+0000","last_became_active":"2024-04-22T19:51:34.549551+0000","last_became_peered":"2024-04-22T19:51:34.549551+0000","last_unstale":"2024-04-22T20:44:45.173306+0000","last_undegraded":"2024-04-22T20:44:45.173306+0000","last_fullsized":"2024-04-22T20:44:45.173306+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:14:36.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"261'146","reported_seq":645,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.823329+0000","last_change":"2024-04-22T19:51:36.573966+0000","last_active":"2024-04-22T20:48:00.823329+0000","last_peered":"2024-04-22T20:48:00.823329+0000","last_clean":"2024-04-22T20:48:00.823329+0000","last_became_active":"2024-04-22T19:51:36.573189+0000","last_became_peered":"2024-04-22T19:51:36.573189+0000","last_unstale":"2024-04-22T20:48:00.823329+0000","last_undegraded":"2024-04-22T20:48:00.823329+0000","last_fullsized":"2024-04-22T20:48:00.823329+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:07:48.890371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":328,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"261'212","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.175878+0000","last_change":"2024-04-22T19:51:28.518334+0000","last_active":"2024-04-22T20:44:40.175878+0000","last_peered":"2024-04-22T20:44:40.175878+0000","last_clean":"2024-04-22T20:44:40.175878+0000","last_became_active":"2024-04-22T19:51:28.517737+0000","last_became_peered":"2024-04-22T19:51:28.517737+0000","last_unstale":"2024-04-22T20:44:40.175878+0000","last_undegraded":"2024-04-22T20:44:40.175878+0000","last_fullsized":"2024-04-22T20:44:40.175878+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:47.145426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":871,"num_read_kb":733,"num_write":338,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"261'241","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:59.594399+0000","last_change":"2024-04-22T19:51:26.509257+0000","last_active":"2024-04-22T20:46:59.594399+0000","last_peered":"2024-04-22T20:46:59.594399+0000","last_clean":"2024-04-22T20:46:59.594399+0000","last_became_active":"2024-04-22T19:51:26.508933+0000","last_became_peered":"2024-04-22T19:51:26.508933+0000","last_unstale":"2024-04-22T20:46:59.594399+0000","last_undegraded":"2024-04-22T20:46:59.594399+0000","last_fullsized":"2024-04-22T20:46:59.594399+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":200,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:00:26.499126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8849792,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":456,"num_read_kb":15260,"num_write":1066,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":502,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.150710+0000","last_change":"2024-04-22T19:51:30.528152+0000","last_active":"2024-04-22T20:44:40.150710+0000","last_peered":"2024-04-22T20:44:40.150710+0000","last_clean":"2024-04-22T20:44:40.150710+0000","last_became_active":"2024-04-22T19:51:30.527712+0000","last_became_peered":"2024-04-22T19:51:30.527712+0000","last_unstale":"2024-04-22T20:44:40.150710+0000","last_undegraded":"2024-04-22T20:44:40.150710+0000","last_fullsized":"2024-04-22T20:44:40.150710+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:03:58.918129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"261'112","reported_seq":656,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.066993+0000","last_change":"2024-04-22T19:51:32.536484+0000","last_active":"2024-04-22T20:45:25.066993+0000","last_peered":"2024-04-22T20:45:25.066993+0000","last_clean":"2024-04-22T20:45:25.066993+0000","last_became_active":"2024-04-22T19:51:32.536177+0000","last_became_peered":"2024-04-22T19:51:32.536177+0000","last_unstale":"2024-04-22T20:45:25.066993+0000","last_undegraded":"2024-04-22T20:45:25.066993+0000","last_fullsized":"2024-04-22T20:45:25.066993+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:39:22.960954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"261'519","reported_seq":2079,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.552754+0000","last_change":"2024-04-22T19:51:34.562120+0000","last_active":"2024-04-22T20:48:08.552754+0000","last_peered":"2024-04-22T20:48:08.552754+0000","last_clean":"2024-04-22T20:48:08.552754+0000","last_became_active":"2024-04-22T19:51:34.561828+0000","last_became_peered":"2024-04-22T19:51:34.561828+0000","last_unstale":"2024-04-22T20:48:08.552754+0000","last_undegraded":"2024-04-22T20:48:08.552754+0000","last_fullsized":"2024-04-22T20:48:08.552754+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":500,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:46.946439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"261'174","reported_seq":986,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.139062+0000","last_change":"2024-04-22T19:51:36.565908+0000","last_active":"2024-04-22T20:48:03.139062+0000","last_peered":"2024-04-22T20:48:03.139062+0000","last_clean":"2024-04-22T20:48:03.139062+0000","last_became_active":"2024-04-22T19:51:36.565673+0000","last_became_peered":"2024-04-22T19:51:36.565673+0000","last_unstale":"2024-04-22T20:48:03.139062+0000","last_undegraded":"2024-04-22T20:48:03.139062+0000","last_fullsized":"2024-04-22T20:48:03.139062+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:45:46.054789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":579,"num_read_kb":567,"num_write":371,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"261'176","reported_seq":814,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.150685+0000","last_change":"2024-04-22T19:51:28.516658+0000","last_active":"2024-04-22T20:44:40.150685+0000","last_peered":"2024-04-22T20:44:40.150685+0000","last_clean":"2024-04-22T20:44:40.150685+0000","last_became_active":"2024-04-22T19:51:28.515670+0000","last_became_peered":"2024-04-22T19:51:28.515670+0000","last_unstale":"2024-04-22T20:44:40.150685+0000","last_undegraded":"2024-04-22T20:44:40.150685+0000","last_fullsized":"2024-04-22T20:44:40.150685+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:38:08.341570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":729,"num_read_kb":590,"num_write":271,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"261'261","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.305295+0000","last_change":"2024-04-22T19:51:26.503775+0000","last_active":"2024-04-22T20:47:38.305295+0000","last_peered":"2024-04-22T20:47:38.305295+0000","last_clean":"2024-04-22T20:47:38.305295+0000","last_became_active":"2024-04-22T19:51:26.503438+0000","last_became_peered":"2024-04-22T19:51:26.503438+0000","last_unstale":"2024-04-22T20:47:38.305295+0000","last_undegraded":"2024-04-22T20:47:38.305295+0000","last_fullsized":"2024-04-22T20:47:38.305295+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:39:05.281766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13078725,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":525,"num_read_kb":32833,"num_write":1247,"num_write_kb":25399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180715+0000","last_change":"2024-04-22T19:51:30.523415+0000","last_active":"2024-04-22T20:44:40.180715+0000","last_peered":"2024-04-22T20:44:40.180715+0000","last_clean":"2024-04-22T20:44:40.180715+0000","last_became_active":"2024-04-22T19:51:30.523125+0000","last_became_peered":"2024-04-22T19:51:30.523125+0000","last_unstale":"2024-04-22T20:44:40.180715+0000","last_undegraded":"2024-04-22T20:44:40.180715+0000","last_fullsized":"2024-04-22T20:44:40.180715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:33.007742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"261'247","reported_seq":812,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.078728+0000","last_change":"2024-04-22T19:51:33.248486+0000","last_active":"2024-04-22T20:47:10.078728+0000","last_peered":"2024-04-22T20:47:10.078728+0000","last_clean":"2024-04-22T20:47:10.078728+0000","last_became_active":"2024-04-22T19:51:33.248160+0000","last_became_peered":"2024-04-22T19:51:33.248160+0000","last_unstale":"2024-04-22T20:47:10.078728+0000","last_undegraded":"2024-04-22T20:47:10.078728+0000","last_fullsized":"2024-04-22T20:47:10.078728+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:35.418273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51814,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":412,"num_read_kb":843,"num_write":338,"num_write_kb":269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"261'531","reported_seq":4224,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.556421+0000","last_change":"2024-04-22T19:51:34.556255+0000","last_active":"2024-04-22T20:48:08.556421+0000","last_peered":"2024-04-22T20:48:08.556421+0000","last_clean":"2024-04-22T20:48:08.556421+0000","last_became_active":"2024-04-22T19:51:34.555720+0000","last_became_peered":"2024-04-22T19:51:34.555720+0000","last_unstale":"2024-04-22T20:48:08.556421+0000","last_undegraded":"2024-04-22T20:48:08.556421+0000","last_fullsized":"2024-04-22T20:48:08.556421+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":500,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:58.605732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"261'385","reported_seq":1057,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189325+0000","last_change":"2024-04-22T19:51:28.513088+0000","last_active":"2024-04-22T20:45:40.189325+0000","last_peered":"2024-04-22T20:45:40.189325+0000","last_clean":"2024-04-22T20:45:40.189325+0000","last_became_active":"2024-04-22T19:51:28.512125+0000","last_became_peered":"2024-04-22T19:51:28.512125+0000","last_unstale":"2024-04-22T20:45:40.189325+0000","last_undegraded":"2024-04-22T20:45:40.189325+0000","last_fullsized":"2024-04-22T20:45:40.189325+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:22:31.839887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1728,"num_read_kb":1430,"num_write":663,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"261'238","reported_seq":880,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.318260+0000","last_change":"2024-04-22T19:51:26.508318+0000","last_active":"2024-04-22T20:47:44.318260+0000","last_peered":"2024-04-22T20:47:44.318260+0000","last_clean":"2024-04-22T20:47:44.318260+0000","last_became_active":"2024-04-22T19:51:26.504654+0000","last_became_peered":"2024-04-22T19:51:26.504654+0000","last_unstale":"2024-04-22T20:47:44.318260+0000","last_undegraded":"2024-04-22T20:47:44.318260+0000","last_fullsized":"2024-04-22T20:47:44.318260+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":200,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:29:46.972228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12959802,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":378,"num_read_kb":80525,"num_write":948,"num_write_kb":26179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.065667+0000","last_change":"2024-04-22T19:51:30.526634+0000","last_active":"2024-04-22T20:45:40.065667+0000","last_peered":"2024-04-22T20:45:40.065667+0000","last_clean":"2024-04-22T20:45:40.065667+0000","last_became_active":"2024-04-22T19:51:30.526306+0000","last_became_peered":"2024-04-22T19:51:30.526306+0000","last_unstale":"2024-04-22T20:45:40.065667+0000","last_undegraded":"2024-04-22T20:45:40.065667+0000","last_fullsized":"2024-04-22T20:45:40.065667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:46.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"261'329","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:05.210994+0000","last_change":"2024-04-22T19:51:33.248375+0000","last_active":"2024-04-22T20:48:05.210994+0000","last_peered":"2024-04-22T20:48:05.210994+0000","last_clean":"2024-04-22T20:48:05.210994+0000","last_became_active":"2024-04-22T19:51:33.248059+0000","last_became_peered":"2024-04-22T19:51:33.248059+0000","last_unstale":"2024-04-22T20:48:05.210994+0000","last_undegraded":"2024-04-22T20:48:05.210994+0000","last_fullsized":"2024-04-22T20:48:05.210994+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:25:25.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48009,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":524,"num_read_kb":923,"num_write":402,"num_write_kb":246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.045374+0000","last_change":"2024-04-22T19:51:34.557926+0000","last_active":"2024-04-22T20:45:35.045374+0000","last_peered":"2024-04-22T20:45:35.045374+0000","last_clean":"2024-04-22T20:45:35.045374+0000","last_became_active":"2024-04-22T19:51:34.557662+0000","last_became_peered":"2024-04-22T19:51:34.557662+0000","last_unstale":"2024-04-22T20:45:35.045374+0000","last_undegraded":"2024-04-22T20:45:35.045374+0000","last_fullsized":"2024-04-22T20:45:35.045374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:25.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"261'562","reported_seq":1370,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.537178+0000","last_change":"2024-04-22T19:51:36.571168+0000","last_active":"2024-04-22T20:48:08.537178+0000","last_peered":"2024-04-22T20:48:08.537178+0000","last_clean":"2024-04-22T20:48:08.537178+0000","last_became_active":"2024-04-22T19:51:36.570520+0000","last_became_peered":"2024-04-22T19:51:36.570520+0000","last_unstale":"2024-04-22T20:48:08.537178+0000","last_undegraded":"2024-04-22T20:48:08.537178+0000","last_fullsized":"2024-04-22T20:48:08.537178+0000","mapping_epoch":30,"log_start":"261'500","ondisk_log_start":"261'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":500,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:56.456085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1283,"num_read_kb":1280,"num_write":1166,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"261'311","reported_seq":959,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181329+0000","last_change":"2024-04-22T19:51:28.512328+0000","last_active":"2024-04-22T20:45:40.181329+0000","last_peered":"2024-04-22T20:45:40.181329+0000","last_clean":"2024-04-22T20:45:40.181329+0000","last_became_active":"2024-04-22T19:51:28.511527+0000","last_became_peered":"2024-04-22T19:51:28.511527+0000","last_unstale":"2024-04-22T20:45:40.181329+0000","last_undegraded":"2024-04-22T20:45:40.181329+0000","last_fullsized":"2024-04-22T20:45:40.181329+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:22.483205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1221,"num_read_kb":977,"num_write":455,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"261'258","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.064025+0000","last_change":"2024-04-22T19:51:26.505351+0000","last_active":"2024-04-22T20:45:30.064025+0000","last_peered":"2024-04-22T20:45:30.064025+0000","last_clean":"2024-04-22T20:45:30.064025+0000","last_became_active":"2024-04-22T19:51:26.504472+0000","last_became_peered":"2024-04-22T19:51:26.504472+0000","last_unstale":"2024-04-22T20:45:30.064025+0000","last_undegraded":"2024-04-22T20:45:30.064025+0000","last_fullsized":"2024-04-22T20:45:30.064025+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:25.100075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11703329,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":411,"num_read_kb":33343,"num_write":1065,"num_write_kb":49829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"32'5","reported_seq":826,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:07.851401+0000","last_change":"2024-04-22T19:51:30.524309+0000","last_active":"2024-04-22T20:48:07.851401+0000","last_peered":"2024-04-22T20:48:07.851401+0000","last_clean":"2024-04-22T20:48:07.851401+0000","last_became_active":"2024-04-22T19:51:30.524043+0000","last_became_peered":"2024-04-22T19:51:30.524043+0000","last_unstale":"2024-04-22T20:48:07.851401+0000","last_undegraded":"2024-04-22T20:48:07.851401+0000","last_fullsized":"2024-04-22T20:48:07.851401+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:51.651333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":489,"num_read_kb":794,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"261'261","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.172994+0000","last_change":"2024-04-22T19:51:32.539254+0000","last_active":"2024-04-22T20:47:50.172994+0000","last_peered":"2024-04-22T20:47:50.172994+0000","last_clean":"2024-04-22T20:47:50.172994+0000","last_became_active":"2024-04-22T19:51:32.537968+0000","last_became_peered":"2024-04-22T19:51:32.537968+0000","last_unstale":"2024-04-22T20:47:50.172994+0000","last_undegraded":"2024-04-22T20:47:50.172994+0000","last_fullsized":"2024-04-22T20:47:50.172994+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:38.345231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51807,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":422,"num_read_kb":877,"num_write":344,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158827+0000","last_change":"2024-04-22T19:51:34.556181+0000","last_active":"2024-04-22T20:45:50.158827+0000","last_peered":"2024-04-22T20:45:50.158827+0000","last_clean":"2024-04-22T20:45:50.158827+0000","last_became_active":"2024-04-22T19:51:34.555889+0000","last_became_peered":"2024-04-22T19:51:34.555889+0000","last_unstale":"2024-04-22T20:45:50.158827+0000","last_undegraded":"2024-04-22T20:45:50.158827+0000","last_fullsized":"2024-04-22T20:45:50.158827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:03:03.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"261'79","reported_seq":577,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.319292+0000","last_change":"2024-04-22T19:51:36.574838+0000","last_active":"2024-04-22T20:45:55.319292+0000","last_peered":"2024-04-22T20:45:55.319292+0000","last_clean":"2024-04-22T20:45:55.319292+0000","last_became_active":"2024-04-22T19:51:36.574555+0000","last_became_peered":"2024-04-22T19:51:36.574555+0000","last_unstale":"2024-04-22T20:45:55.319292+0000","last_undegraded":"2024-04-22T20:45:55.319292+0000","last_fullsized":"2024-04-22T20:45:55.319292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:07.667952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":91,"num_write":194,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"261'188","reported_seq":783,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.115015+0000","last_change":"2024-04-22T19:51:28.519762+0000","last_active":"2024-04-22T20:46:41.115015+0000","last_peered":"2024-04-22T20:46:41.115015+0000","last_clean":"2024-04-22T20:46:41.115015+0000","last_became_active":"2024-04-22T19:51:28.519424+0000","last_became_peered":"2024-04-22T19:51:28.519424+0000","last_unstale":"2024-04-22T20:46:41.115015+0000","last_undegraded":"2024-04-22T20:46:41.115015+0000","last_fullsized":"2024-04-22T20:46:41.115015+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:13.130467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":758,"num_read_kb":601,"num_write":297,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"261'219","reported_seq":841,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.063988+0000","last_change":"2024-04-22T19:51:26.511055+0000","last_active":"2024-04-22T20:45:30.063988+0000","last_peered":"2024-04-22T20:45:30.063988+0000","last_clean":"2024-04-22T20:45:30.063988+0000","last_became_active":"2024-04-22T19:51:26.510273+0000","last_became_peered":"2024-04-22T19:51:26.510273+0000","last_unstale":"2024-04-22T20:45:30.063988+0000","last_undegraded":"2024-04-22T20:45:30.063988+0000","last_fullsized":"2024-04-22T20:45:30.063988+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":200,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:53.305766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15039158,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":310,"num_read_kb":64999,"num_write":940,"num_write_kb":29343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.067468+0000","last_change":"2024-04-22T19:51:30.525335+0000","last_active":"2024-04-22T20:45:30.067468+0000","last_peered":"2024-04-22T20:45:30.067468+0000","last_clean":"2024-04-22T20:45:30.067468+0000","last_became_active":"2024-04-22T19:51:30.525018+0000","last_became_peered":"2024-04-22T19:51:30.525018+0000","last_unstale":"2024-04-22T20:45:30.067468+0000","last_undegraded":"2024-04-22T20:45:30.067468+0000","last_fullsized":"2024-04-22T20:45:30.067468+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:49:24.770975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"261'304","reported_seq":892,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.085578+0000","last_change":"2024-04-22T19:51:32.538989+0000","last_active":"2024-04-22T20:47:55.085578+0000","last_peered":"2024-04-22T20:47:55.085578+0000","last_clean":"2024-04-22T20:47:55.085578+0000","last_became_active":"2024-04-22T19:51:32.538656+0000","last_became_peered":"2024-04-22T19:51:32.538656+0000","last_unstale":"2024-04-22T20:47:55.085578+0000","last_undegraded":"2024-04-22T20:47:55.085578+0000","last_fullsized":"2024-04-22T20:47:55.085578+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:36:49.025357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61009,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":485,"num_read_kb":947,"num_write":390,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"261'504","reported_seq":2193,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.557320+0000","last_change":"2024-04-22T19:51:34.561771+0000","last_active":"2024-04-22T20:48:08.557320+0000","last_peered":"2024-04-22T20:48:08.557320+0000","last_clean":"2024-04-22T20:48:08.557320+0000","last_became_active":"2024-04-22T19:51:34.561488+0000","last_became_peered":"2024-04-22T19:51:34.561488+0000","last_unstale":"2024-04-22T20:48:08.557320+0000","last_undegraded":"2024-04-22T20:48:08.557320+0000","last_fullsized":"2024-04-22T20:48:08.557320+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":400,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:08:35.813443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"261'134","reported_seq":623,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.184424+0000","last_change":"2024-04-22T19:51:36.573554+0000","last_active":"2024-04-22T20:45:50.184424+0000","last_peered":"2024-04-22T20:45:50.184424+0000","last_clean":"2024-04-22T20:45:50.184424+0000","last_became_active":"2024-04-22T19:51:36.573167+0000","last_became_peered":"2024-04-22T19:51:36.573167+0000","last_unstale":"2024-04-22T20:45:50.184424+0000","last_undegraded":"2024-04-22T20:45:50.184424+0000","last_fullsized":"2024-04-22T20:45:50.184424+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:24.336268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"261'150","reported_seq":715,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181727+0000","last_change":"2024-04-22T19:51:28.518692+0000","last_active":"2024-04-22T20:45:30.181727+0000","last_peered":"2024-04-22T20:45:30.181727+0000","last_clean":"2024-04-22T20:45:30.181727+0000","last_became_active":"2024-04-22T19:51:28.518398+0000","last_became_peered":"2024-04-22T19:51:28.518398+0000","last_unstale":"2024-04-22T20:45:30.181727+0000","last_undegraded":"2024-04-22T20:45:30.181727+0000","last_fullsized":"2024-04-22T20:45:30.181727+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:33.700338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":596,"num_read_kb":465,"num_write":243,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"261'246","reported_seq":915,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.890178+0000","last_change":"2024-04-22T19:52:05.113375+0000","last_active":"2024-04-22T20:46:41.890178+0000","last_peered":"2024-04-22T20:46:41.890178+0000","last_clean":"2024-04-22T20:46:41.890178+0000","last_became_active":"2024-04-22T19:52:05.112910+0000","last_became_peered":"2024-04-22T19:52:05.112910+0000","last_unstale":"2024-04-22T20:46:41.890178+0000","last_undegraded":"2024-04-22T20:46:41.890178+0000","last_fullsized":"2024-04-22T20:46:41.890178+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:44:17.477228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4708959,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":450,"num_read_kb":39016,"num_write":1025,"num_write_kb":34073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.069399+0000","last_change":"2024-04-22T19:51:30.529388+0000","last_active":"2024-04-22T20:45:50.069399+0000","last_peered":"2024-04-22T20:45:50.069399+0000","last_clean":"2024-04-22T20:45:50.069399+0000","last_became_active":"2024-04-22T19:51:30.529011+0000","last_became_peered":"2024-04-22T19:51:30.529011+0000","last_unstale":"2024-04-22T20:45:50.069399+0000","last_undegraded":"2024-04-22T20:45:50.069399+0000","last_fullsized":"2024-04-22T20:45:50.069399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:02:44.821662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"261'478","reported_seq":1102,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.503866+0000","last_change":"2024-04-22T19:51:32.537571+0000","last_active":"2024-04-22T20:48:08.503866+0000","last_peered":"2024-04-22T20:48:08.503866+0000","last_clean":"2024-04-22T20:48:08.503866+0000","last_became_active":"2024-04-22T19:51:32.537267+0000","last_became_peered":"2024-04-22T19:51:32.537267+0000","last_unstale":"2024-04-22T20:48:08.503866+0000","last_undegraded":"2024-04-22T20:48:08.503866+0000","last_fullsized":"2024-04-22T20:48:08.503866+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":400,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:04.937868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123858,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":830,"num_read_kb":1829,"num_write":692,"num_write_kb":607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.068905+0000","last_change":"2024-04-22T19:51:34.552745+0000","last_active":"2024-04-22T20:45:45.068905+0000","last_peered":"2024-04-22T20:45:45.068905+0000","last_clean":"2024-04-22T20:45:45.068905+0000","last_became_active":"2024-04-22T19:51:34.552446+0000","last_became_peered":"2024-04-22T19:51:34.552446+0000","last_unstale":"2024-04-22T20:45:45.068905+0000","last_undegraded":"2024-04-22T20:45:45.068905+0000","last_fullsized":"2024-04-22T20:45:45.068905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:59.206061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"261'106","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.047062+0000","last_change":"2024-04-22T19:51:36.570982+0000","last_active":"2024-04-22T20:45:50.047062+0000","last_peered":"2024-04-22T20:45:50.047062+0000","last_clean":"2024-04-22T20:45:50.047062+0000","last_became_active":"2024-04-22T19:51:36.570666+0000","last_became_peered":"2024-04-22T19:51:36.570666+0000","last_unstale":"2024-04-22T20:45:50.047062+0000","last_undegraded":"2024-04-22T20:45:50.047062+0000","last_fullsized":"2024-04-22T20:45:50.047062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:18.966614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":242,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"261'168","reported_seq":795,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:16.403532+0000","last_change":"2024-04-22T19:51:28.508639+0000","last_active":"2024-04-22T20:47:16.403532+0000","last_peered":"2024-04-22T20:47:16.403532+0000","last_clean":"2024-04-22T20:47:16.403532+0000","last_became_active":"2024-04-22T19:51:28.508288+0000","last_became_peered":"2024-04-22T19:51:28.508288+0000","last_unstale":"2024-04-22T20:47:16.403532+0000","last_undegraded":"2024-04-22T20:47:16.403532+0000","last_fullsized":"2024-04-22T20:47:16.403532+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:22:30.411824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":576,"num_write":244,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"261'258","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:48.776114+0000","last_change":"2024-04-22T19:51:26.506251+0000","last_active":"2024-04-22T20:46:48.776114+0000","last_peered":"2024-04-22T20:46:48.776114+0000","last_clean":"2024-04-22T20:46:48.776114+0000","last_became_active":"2024-04-22T19:51:26.503669+0000","last_became_peered":"2024-04-22T19:51:26.503669+0000","last_unstale":"2024-04-22T20:46:48.776114+0000","last_undegraded":"2024-04-22T20:46:48.776114+0000","last_fullsized":"2024-04-22T20:46:48.776114+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:52.491938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8753181,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":397,"num_read_kb":4498,"num_write":1038,"num_write_kb":33252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.067499+0000","last_change":"2024-04-22T19:51:30.526698+0000","last_active":"2024-04-22T20:45:30.067499+0000","last_peered":"2024-04-22T20:45:30.067499+0000","last_clean":"2024-04-22T20:45:30.067499+0000","last_became_active":"2024-04-22T19:51:30.526420+0000","last_became_peered":"2024-04-22T19:51:30.526420+0000","last_unstale":"2024-04-22T20:45:30.067499+0000","last_undegraded":"2024-04-22T20:45:30.067499+0000","last_fullsized":"2024-04-22T20:45:30.067499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:14.342753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"261'264","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.199269+0000","last_change":"2024-04-22T19:51:32.536764+0000","last_active":"2024-04-22T20:47:50.199269+0000","last_peered":"2024-04-22T20:47:50.199269+0000","last_clean":"2024-04-22T20:47:50.199269+0000","last_became_active":"2024-04-22T19:51:32.536463+0000","last_became_peered":"2024-04-22T19:51:32.536463+0000","last_unstale":"2024-04-22T20:47:50.199269+0000","last_undegraded":"2024-04-22T20:47:50.199269+0000","last_fullsized":"2024-04-22T20:47:50.199269+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":200,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:29:00.474073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48720,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":444,"num_read_kb":848,"num_write":396,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183789+0000","last_change":"2024-04-22T19:51:34.555066+0000","last_active":"2024-04-22T20:45:45.183789+0000","last_peered":"2024-04-22T20:45:45.183789+0000","last_clean":"2024-04-22T20:45:45.183789+0000","last_became_active":"2024-04-22T19:51:34.554737+0000","last_became_peered":"2024-04-22T19:51:34.554737+0000","last_unstale":"2024-04-22T20:45:45.183789+0000","last_undegraded":"2024-04-22T20:45:45.183789+0000","last_fullsized":"2024-04-22T20:45:45.183789+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:09:00.070765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"261'93","reported_seq":590,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:42.565656+0000","last_change":"2024-04-22T19:51:36.571558+0000","last_active":"2024-04-22T20:46:42.565656+0000","last_peered":"2024-04-22T20:46:42.565656+0000","last_clean":"2024-04-22T20:46:42.565656+0000","last_became_active":"2024-04-22T19:51:36.571086+0000","last_became_peered":"2024-04-22T19:51:36.571086+0000","last_unstale":"2024-04-22T20:46:42.565656+0000","last_undegraded":"2024-04-22T20:46:42.565656+0000","last_fullsized":"2024-04-22T20:46:42.565656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:37:15.182216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":72,"num_write":227,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"261'253","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:07.364455+0000","last_change":"2024-04-22T19:51:28.514713+0000","last_active":"2024-04-22T20:46:07.364455+0000","last_peered":"2024-04-22T20:46:07.364455+0000","last_clean":"2024-04-22T20:46:07.364455+0000","last_became_active":"2024-04-22T19:51:28.514445+0000","last_became_peered":"2024-04-22T19:51:28.514445+0000","last_unstale":"2024-04-22T20:46:07.364455+0000","last_undegraded":"2024-04-22T20:46:07.364455+0000","last_fullsized":"2024-04-22T20:46:07.364455+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:39.638389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":723,"num_write":394,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'331","reported_seq":1077,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:59.585410+0000","last_change":"2024-04-22T19:51:26.496789+0000","last_active":"2024-04-22T20:46:59.585410+0000","last_peered":"2024-04-22T20:46:59.585410+0000","last_clean":"2024-04-22T20:46:59.585410+0000","last_became_active":"2024-04-22T19:51:26.496408+0000","last_became_peered":"2024-04-22T19:51:26.496408+0000","last_unstale":"2024-04-22T20:46:59.585410+0000","last_undegraded":"2024-04-22T20:46:59.585410+0000","last_fullsized":"2024-04-22T20:46:59.585410+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:19:23.202029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1053522,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":776,"num_read_kb":8021,"num_write":1416,"num_write_kb":27015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.044898+0000","last_change":"2024-04-22T19:51:30.527588+0000","last_active":"2024-04-22T20:45:30.044898+0000","last_peered":"2024-04-22T20:45:30.044898+0000","last_clean":"2024-04-22T20:45:30.044898+0000","last_became_active":"2024-04-22T19:51:30.527309+0000","last_became_peered":"2024-04-22T19:51:30.527309+0000","last_unstale":"2024-04-22T20:45:30.044898+0000","last_undegraded":"2024-04-22T20:45:30.044898+0000","last_fullsized":"2024-04-22T20:45:30.044898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:22:07.894332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"261'303","reported_seq":894,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.082212+0000","last_change":"2024-04-22T19:51:32.540003+0000","last_active":"2024-04-22T20:48:00.082212+0000","last_peered":"2024-04-22T20:48:00.082212+0000","last_clean":"2024-04-22T20:48:00.082212+0000","last_became_active":"2024-04-22T19:51:32.539732+0000","last_became_peered":"2024-04-22T19:51:32.539732+0000","last_unstale":"2024-04-22T20:48:00.082212+0000","last_undegraded":"2024-04-22T20:48:00.082212+0000","last_fullsized":"2024-04-22T20:48:00.082212+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":200,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:19.413189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63653,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":499,"num_read_kb":1000,"num_write":402,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"261'485","reported_seq":2198,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.553773+0000","last_change":"2024-04-22T19:51:34.556537+0000","last_active":"2024-04-22T20:48:08.553773+0000","last_peered":"2024-04-22T20:48:08.553773+0000","last_clean":"2024-04-22T20:48:08.553773+0000","last_became_active":"2024-04-22T19:51:34.556124+0000","last_became_peered":"2024-04-22T19:51:34.556124+0000","last_unstale":"2024-04-22T20:48:08.553773+0000","last_undegraded":"2024-04-22T20:48:08.553773+0000","last_fullsized":"2024-04-22T20:48:08.553773+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":400,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:14:07.299855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"261'116","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.060760+0000","last_change":"2024-04-22T19:51:36.573163+0000","last_active":"2024-04-22T20:47:55.060760+0000","last_peered":"2024-04-22T20:47:55.060760+0000","last_clean":"2024-04-22T20:47:55.060760+0000","last_became_active":"2024-04-22T19:51:36.572913+0000","last_became_peered":"2024-04-22T19:51:36.572913+0000","last_unstale":"2024-04-22T20:47:55.060760+0000","last_undegraded":"2024-04-22T20:47:55.060760+0000","last_fullsized":"2024-04-22T20:47:55.060760+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:40:20.320547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":102,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"261'338","reported_seq":1080,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.179830+0000","last_change":"2024-04-22T19:51:28.516099+0000","last_active":"2024-04-22T20:45:30.179830+0000","last_peered":"2024-04-22T20:45:30.179830+0000","last_clean":"2024-04-22T20:45:30.179830+0000","last_became_active":"2024-04-22T19:51:28.515809+0000","last_became_peered":"2024-04-22T19:51:28.515809+0000","last_unstale":"2024-04-22T20:45:30.179830+0000","last_undegraded":"2024-04-22T20:45:30.179830+0000","last_fullsized":"2024-04-22T20:45:30.179830+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:19.069001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1504,"num_read_kb":1227,"num_write":616,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"261'280","reported_seq":915,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181307+0000","last_change":"2024-04-22T19:51:26.499730+0000","last_active":"2024-04-22T20:45:40.181307+0000","last_peered":"2024-04-22T20:45:40.181307+0000","last_clean":"2024-04-22T20:45:40.181307+0000","last_became_active":"2024-04-22T19:51:26.499362+0000","last_became_peered":"2024-04-22T19:51:26.499362+0000","last_unstale":"2024-04-22T20:45:40.181307+0000","last_undegraded":"2024-04-22T20:45:40.181307+0000","last_fullsized":"2024-04-22T20:45:40.181307+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":200,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:28:07.429252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9762556,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":451,"num_read_kb":26712,"num_write":1196,"num_write_kb":36357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"25'2","reported_seq":1134,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.108639+0000","last_change":"2024-04-22T19:51:30.527770+0000","last_active":"2024-04-22T20:48:15.108639+0000","last_peered":"2024-04-22T20:48:15.108639+0000","last_clean":"2024-04-22T20:48:15.108639+0000","last_became_active":"2024-04-22T19:51:30.527490+0000","last_became_peered":"2024-04-22T19:51:30.527490+0000","last_unstale":"2024-04-22T20:48:15.108639+0000","last_undegraded":"2024-04-22T20:48:15.108639+0000","last_fullsized":"2024-04-22T20:48:15.108639+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:39:09.682743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":639,"num_read_kb":639,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"261'90","reported_seq":620,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157133+0000","last_change":"2024-04-22T19:51:33.248919+0000","last_active":"2024-04-22T20:45:35.157133+0000","last_peered":"2024-04-22T20:45:35.157133+0000","last_clean":"2024-04-22T20:45:35.157133+0000","last_became_active":"2024-04-22T19:51:33.248349+0000","last_became_peered":"2024-04-22T19:51:33.248349+0000","last_unstale":"2024-04-22T20:45:35.157133+0000","last_undegraded":"2024-04-22T20:45:35.157133+0000","last_fullsized":"2024-04-22T20:45:35.157133+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:03:31.126924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":60,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"261'515","reported_seq":2069,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.554704+0000","last_change":"2024-04-22T19:51:34.562158+0000","last_active":"2024-04-22T20:48:08.554704+0000","last_peered":"2024-04-22T20:48:08.554704+0000","last_clean":"2024-04-22T20:48:08.554704+0000","last_became_active":"2024-04-22T19:51:34.561825+0000","last_became_peered":"2024-04-22T19:51:34.561825+0000","last_unstale":"2024-04-22T20:48:08.554704+0000","last_undegraded":"2024-04-22T20:48:08.554704+0000","last_fullsized":"2024-04-22T20:48:08.554704+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:22:52.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"261'130","reported_seq":625,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.182787+0000","last_change":"2024-04-22T19:51:36.573308+0000","last_active":"2024-04-22T20:45:50.182787+0000","last_peered":"2024-04-22T20:45:50.182787+0000","last_clean":"2024-04-22T20:45:50.182787+0000","last_became_active":"2024-04-22T19:51:36.573026+0000","last_became_peered":"2024-04-22T19:51:36.573026+0000","last_unstale":"2024-04-22T20:45:50.182787+0000","last_undegraded":"2024-04-22T20:45:50.182787+0000","last_fullsized":"2024-04-22T20:45:50.182787+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:19.108938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":119,"num_write":309,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"261'207","reported_seq":818,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045988+0000","last_change":"2024-04-22T19:51:28.512959+0000","last_active":"2024-04-22T20:45:40.045988+0000","last_peered":"2024-04-22T20:45:40.045988+0000","last_clean":"2024-04-22T20:45:40.045988+0000","last_became_active":"2024-04-22T19:51:28.512275+0000","last_became_peered":"2024-04-22T19:51:28.512275+0000","last_unstale":"2024-04-22T20:45:40.045988+0000","last_undegraded":"2024-04-22T20:45:40.045988+0000","last_fullsized":"2024-04-22T20:45:40.045988+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:24:20.294556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":832,"num_read_kb":677,"num_write":302,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"261'258","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:31.723467+0000","last_change":"2024-04-22T19:51:26.498630+0000","last_active":"2024-04-22T20:46:31.723467+0000","last_peered":"2024-04-22T20:46:31.723467+0000","last_clean":"2024-04-22T20:46:31.723467+0000","last_became_active":"2024-04-22T19:51:26.498276+0000","last_became_peered":"2024-04-22T19:51:26.498276+0000","last_unstale":"2024-04-22T20:46:31.723467+0000","last_undegraded":"2024-04-22T20:46:31.723467+0000","last_fullsized":"2024-04-22T20:46:31.723467+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:43:15.698499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9975117,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":398,"num_read_kb":30145,"num_write":1092,"num_write_kb":23135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"32'3","reported_seq":820,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.478858+0000","last_change":"2024-04-22T19:51:30.530726+0000","last_active":"2024-04-22T20:48:08.478858+0000","last_peered":"2024-04-22T20:48:08.478858+0000","last_clean":"2024-04-22T20:48:08.478858+0000","last_became_active":"2024-04-22T19:51:30.530363+0000","last_became_peered":"2024-04-22T19:51:30.530363+0000","last_unstale":"2024-04-22T20:48:08.478858+0000","last_undegraded":"2024-04-22T20:48:08.478858+0000","last_fullsized":"2024-04-22T20:48:08.478858+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:58:21.578312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":489,"num_read_kb":487,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"261'396","reported_seq":999,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:06.461115+0000","last_change":"2024-04-22T19:51:32.540935+0000","last_active":"2024-04-22T20:48:06.461115+0000","last_peered":"2024-04-22T20:48:06.461115+0000","last_clean":"2024-04-22T20:48:06.461115+0000","last_became_active":"2024-04-22T19:51:32.540622+0000","last_became_peered":"2024-04-22T19:51:32.540622+0000","last_unstale":"2024-04-22T20:48:06.461115+0000","last_undegraded":"2024-04-22T20:48:06.461115+0000","last_fullsized":"2024-04-22T20:48:06.461115+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":300,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:57:03.444994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1216,"num_read_kb":1715,"num_write":1756,"num_write_kb":1624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.316223+0000","last_change":"2024-04-22T19:51:34.551287+0000","last_active":"2024-04-22T20:45:35.316223+0000","last_peered":"2024-04-22T20:45:35.316223+0000","last_clean":"2024-04-22T20:45:35.316223+0000","last_became_active":"2024-04-22T19:51:34.551009+0000","last_became_peered":"2024-04-22T19:51:34.551009+0000","last_unstale":"2024-04-22T20:45:35.316223+0000","last_undegraded":"2024-04-22T20:45:35.316223+0000","last_fullsized":"2024-04-22T20:45:35.316223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:24:58.026337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"261'121","reported_seq":620,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.018220+0000","last_change":"2024-04-22T19:51:36.571807+0000","last_active":"2024-04-22T20:48:04.018220+0000","last_peered":"2024-04-22T20:48:04.018220+0000","last_clean":"2024-04-22T20:48:04.018220+0000","last_became_active":"2024-04-22T19:51:36.571517+0000","last_became_peered":"2024-04-22T19:51:36.571517+0000","last_unstale":"2024-04-22T20:48:04.018220+0000","last_undegraded":"2024-04-22T20:48:04.018220+0000","last_fullsized":"2024-04-22T20:48:04.018220+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:15:52.550381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":272,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"261'4186","reported_seq":4781,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.317812+0000","last_change":"2024-04-22T19:51:28.515172+0000","last_active":"2024-04-22T20:45:45.317812+0000","last_peered":"2024-04-22T20:45:45.317812+0000","last_clean":"2024-04-22T20:45:45.317812+0000","last_became_active":"2024-04-22T19:51:28.514869+0000","last_became_peered":"2024-04-22T19:51:28.514869+0000","last_unstale":"2024-04-22T20:45:45.317812+0000","last_undegraded":"2024-04-22T20:45:45.317812+0000","last_fullsized":"2024-04-22T20:45:45.317812+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":3000,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:44.084365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14767,"num_read_kb":11981,"num_write":6295,"num_write_kb":3142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":520,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063174+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T20:45:25.063174+0000","last_peered":"2024-04-22T20:45:25.063174+0000","last_clean":"2024-04-22T20:45:25.063174+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T20:45:25.063174+0000","last_undegraded":"2024-04-22T20:45:25.063174+0000","last_fullsized":"2024-04-22T20:45:25.063174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"261'278","reported_seq":973,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.326360+0000","last_change":"2024-04-22T19:51:26.506322+0000","last_active":"2024-04-22T20:47:44.326360+0000","last_peered":"2024-04-22T20:47:44.326360+0000","last_clean":"2024-04-22T20:47:44.326360+0000","last_became_active":"2024-04-22T19:51:26.505982+0000","last_became_peered":"2024-04-22T19:51:26.505982+0000","last_unstale":"2024-04-22T20:47:44.326360+0000","last_undegraded":"2024-04-22T20:47:44.326360+0000","last_fullsized":"2024-04-22T20:47:44.326360+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:46.483074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691084,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":484,"num_read_kb":107291,"num_write":1302,"num_write_kb":45616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.065725+0000","last_change":"2024-04-22T19:51:30.527554+0000","last_active":"2024-04-22T20:45:40.065725+0000","last_peered":"2024-04-22T20:45:40.065725+0000","last_clean":"2024-04-22T20:45:40.065725+0000","last_became_active":"2024-04-22T19:51:30.526939+0000","last_became_peered":"2024-04-22T19:51:30.526939+0000","last_unstale":"2024-04-22T20:45:40.065725+0000","last_undegraded":"2024-04-22T20:45:40.065725+0000","last_fullsized":"2024-04-22T20:45:40.065725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:34:00.754603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"261'700","reported_seq":1386,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.522859+0000","last_change":"2024-04-22T19:51:32.543191+0000","last_active":"2024-04-22T20:48:08.522859+0000","last_peered":"2024-04-22T20:48:08.522859+0000","last_clean":"2024-04-22T20:48:08.522859+0000","last_became_active":"2024-04-22T19:51:32.542970+0000","last_became_peered":"2024-04-22T19:51:32.542970+0000","last_unstale":"2024-04-22T20:48:08.522859+0000","last_undegraded":"2024-04-22T20:48:08.522859+0000","last_fullsized":"2024-04-22T20:48:08.522859+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":600,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:42.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171781,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1364,"num_read_kb":2811,"num_write":1300,"num_write_kb":1164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183762+0000","last_change":"2024-04-22T19:51:34.554048+0000","last_active":"2024-04-22T20:45:45.183762+0000","last_peered":"2024-04-22T20:45:45.183762+0000","last_clean":"2024-04-22T20:45:45.183762+0000","last_became_active":"2024-04-22T19:51:34.553704+0000","last_became_peered":"2024-04-22T19:51:34.553704+0000","last_unstale":"2024-04-22T20:45:45.183762+0000","last_undegraded":"2024-04-22T20:45:45.183762+0000","last_fullsized":"2024-04-22T20:45:45.183762+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:19:14.613388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"261'266","reported_seq":1001,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.066411+0000","last_change":"2024-04-22T19:51:36.571003+0000","last_active":"2024-04-22T20:48:04.066411+0000","last_peered":"2024-04-22T20:48:04.066411+0000","last_clean":"2024-04-22T20:48:04.066411+0000","last_became_active":"2024-04-22T19:51:36.570717+0000","last_became_peered":"2024-04-22T19:51:36.570717+0000","last_unstale":"2024-04-22T20:48:04.066411+0000","last_undegraded":"2024-04-22T20:48:04.066411+0000","last_fullsized":"2024-04-22T20:48:04.066411+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:16.336092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":568,"num_write":495,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"261'50","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045937+0000","last_change":"2024-04-22T19:51:28.506462+0000","last_active":"2024-04-22T20:45:40.045937+0000","last_peered":"2024-04-22T20:45:40.045937+0000","last_clean":"2024-04-22T20:45:40.045937+0000","last_became_active":"2024-04-22T19:51:28.505719+0000","last_became_peered":"2024-04-22T19:51:28.505719+0000","last_unstale":"2024-04-22T20:45:40.045937+0000","last_undegraded":"2024-04-22T20:45:40.045937+0000","last_fullsized":"2024-04-22T20:45:40.045937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:02.609039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":69,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":514,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186325+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T20:45:20.186325+0000","last_peered":"2024-04-22T20:45:20.186325+0000","last_clean":"2024-04-22T20:45:20.186325+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T20:45:20.186325+0000","last_undegraded":"2024-04-22T20:45:20.186325+0000","last_fullsized":"2024-04-22T20:45:20.186325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:20:10.880037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"261'278","reported_seq":954,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:18.964925+0000","last_change":"2024-04-22T19:51:26.501306+0000","last_active":"2024-04-22T20:46:18.964925+0000","last_peered":"2024-04-22T20:46:18.964925+0000","last_clean":"2024-04-22T20:46:18.964925+0000","last_became_active":"2024-04-22T19:51:26.500968+0000","last_became_peered":"2024-04-22T19:51:26.500968+0000","last_unstale":"2024-04-22T20:46:18.964925+0000","last_undegraded":"2024-04-22T20:46:18.964925+0000","last_fullsized":"2024-04-22T20:46:18.964925+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:20:53.110538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10522090,"num_objects":87,"num_object_clones":0,"num_object_copies":261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":517,"num_read_kb":30331,"num_write":1214,"num_write_kb":32709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.184374+0000","last_change":"2024-04-22T19:51:30.526621+0000","last_active":"2024-04-22T20:45:50.184374+0000","last_peered":"2024-04-22T20:45:50.184374+0000","last_clean":"2024-04-22T20:45:50.184374+0000","last_became_active":"2024-04-22T19:51:30.526328+0000","last_became_peered":"2024-04-22T19:51:30.526328+0000","last_unstale":"2024-04-22T20:45:50.184374+0000","last_undegraded":"2024-04-22T20:45:50.184374+0000","last_fullsized":"2024-04-22T20:45:50.184374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:59:48.789109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"261'124","reported_seq":677,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.158336+0000","last_change":"2024-04-22T19:51:32.541073+0000","last_active":"2024-04-22T20:45:45.158336+0000","last_peered":"2024-04-22T20:45:45.158336+0000","last_clean":"2024-04-22T20:45:45.158336+0000","last_became_active":"2024-04-22T19:51:32.540805+0000","last_became_peered":"2024-04-22T19:51:32.540805+0000","last_unstale":"2024-04-22T20:45:45.158336+0000","last_undegraded":"2024-04-22T20:45:45.158336+0000","last_fullsized":"2024-04-22T20:45:45.158336+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:55.727535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":112,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.317842+0000","last_change":"2024-04-22T19:51:34.554535+0000","last_active":"2024-04-22T20:45:45.317842+0000","last_peered":"2024-04-22T20:45:45.317842+0000","last_clean":"2024-04-22T20:45:45.317842+0000","last_became_active":"2024-04-22T19:51:34.554307+0000","last_became_peered":"2024-04-22T19:51:34.554307+0000","last_unstale":"2024-04-22T20:45:45.317842+0000","last_undegraded":"2024-04-22T20:45:45.317842+0000","last_fullsized":"2024-04-22T20:45:45.317842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:13.279562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"261'123","reported_seq":625,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.823391+0000","last_change":"2024-04-22T19:51:36.571438+0000","last_active":"2024-04-22T20:48:03.823391+0000","last_peered":"2024-04-22T20:48:03.823391+0000","last_clean":"2024-04-22T20:48:03.823391+0000","last_became_active":"2024-04-22T19:51:36.571130+0000","last_became_peered":"2024-04-22T19:51:36.571130+0000","last_unstale":"2024-04-22T20:48:03.823391+0000","last_undegraded":"2024-04-22T20:48:03.823391+0000","last_fullsized":"2024-04-22T20:48:03.823391+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:10:47.000615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":131,"num_write":305,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"261'90","reported_seq":658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045996+0000","last_change":"2024-04-22T19:51:28.514065+0000","last_active":"2024-04-22T20:45:40.045996+0000","last_peered":"2024-04-22T20:45:40.045996+0000","last_clean":"2024-04-22T20:45:40.045996+0000","last_became_active":"2024-04-22T19:51:28.513451+0000","last_became_peered":"2024-04-22T19:51:28.513451+0000","last_unstale":"2024-04-22T20:45:40.045996+0000","last_undegraded":"2024-04-22T20:45:40.045996+0000","last_fullsized":"2024-04-22T20:45:40.045996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:52:37.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":305,"num_write":130,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":519,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062391+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T20:45:20.062391+0000","last_peered":"2024-04-22T20:45:20.062391+0000","last_clean":"2024-04-22T20:45:20.062391+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T20:45:20.062391+0000","last_undegraded":"2024-04-22T20:45:20.062391+0000","last_fullsized":"2024-04-22T20:45:20.062391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"261'262","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045858+0000","last_change":"2024-04-22T19:51:26.495730+0000","last_active":"2024-04-22T20:45:40.045858+0000","last_peered":"2024-04-22T20:45:40.045858+0000","last_clean":"2024-04-22T20:45:40.045858+0000","last_became_active":"2024-04-22T19:51:26.495405+0000","last_became_peered":"2024-04-22T19:51:26.495405+0000","last_unstale":"2024-04-22T20:45:40.045858+0000","last_undegraded":"2024-04-22T20:45:40.045858+0000","last_fullsized":"2024-04-22T20:45:40.045858+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:00.884012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23787988,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":57834,"num_write":1209,"num_write_kb":50104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"25'1","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.068477+0000","last_change":"2024-04-22T19:51:30.521583+0000","last_active":"2024-04-22T20:45:40.068477+0000","last_peered":"2024-04-22T20:45:40.068477+0000","last_clean":"2024-04-22T20:45:40.068477+0000","last_became_active":"2024-04-22T19:51:30.521134+0000","last_became_peered":"2024-04-22T19:51:30.521134+0000","last_unstale":"2024-04-22T20:45:40.068477+0000","last_undegraded":"2024-04-22T20:45:40.068477+0000","last_fullsized":"2024-04-22T20:45:40.068477+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:49:17.105976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"261'221","reported_seq":1448,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.449853+0000","last_change":"2024-04-22T19:51:33.250383+0000","last_active":"2024-04-22T20:48:08.449853+0000","last_peered":"2024-04-22T20:48:08.449853+0000","last_clean":"2024-04-22T20:48:08.449853+0000","last_became_active":"2024-04-22T19:51:33.250082+0000","last_became_peered":"2024-04-22T19:51:33.250082+0000","last_unstale":"2024-04-22T20:48:08.449853+0000","last_undegraded":"2024-04-22T20:48:08.449853+0000","last_fullsized":"2024-04-22T20:48:08.449853+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:43:34.357226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":188,"num_read_kb":180,"num_write":143,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183725+0000","last_change":"2024-04-22T19:51:34.548541+0000","last_active":"2024-04-22T20:45:45.183725+0000","last_peered":"2024-04-22T20:45:45.183725+0000","last_clean":"2024-04-22T20:45:45.183725+0000","last_became_active":"2024-04-22T19:51:34.546652+0000","last_became_peered":"2024-04-22T19:51:34.546652+0000","last_unstale":"2024-04-22T20:45:45.183725+0000","last_undegraded":"2024-04-22T20:45:45.183725+0000","last_fullsized":"2024-04-22T20:45:45.183725+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:09:15.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"261'254","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.218285+0000","last_change":"2024-04-22T19:51:36.569306+0000","last_active":"2024-04-22T20:48:03.218285+0000","last_peered":"2024-04-22T20:48:03.218285+0000","last_clean":"2024-04-22T20:48:03.218285+0000","last_became_active":"2024-04-22T19:51:36.568767+0000","last_became_peered":"2024-04-22T19:51:36.568767+0000","last_unstale":"2024-04-22T20:48:03.218285+0000","last_undegraded":"2024-04-22T20:48:03.218285+0000","last_fullsized":"2024-04-22T20:48:03.218285+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:39.794900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":625,"num_write":569,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"261'124","reported_seq":751,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183078+0000","last_change":"2024-04-22T19:51:28.518594+0000","last_active":"2024-04-22T20:45:40.183078+0000","last_peered":"2024-04-22T20:45:40.183078+0000","last_clean":"2024-04-22T20:45:40.183078+0000","last_became_active":"2024-04-22T19:51:28.518084+0000","last_became_peered":"2024-04-22T19:51:28.518084+0000","last_unstale":"2024-04-22T20:45:40.183078+0000","last_undegraded":"2024-04-22T20:45:40.183078+0000","last_fullsized":"2024-04-22T20:45:40.183078+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:38:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":533,"num_read_kb":444,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.044325+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T20:45:25.044325+0000","last_peered":"2024-04-22T20:45:25.044325+0000","last_clean":"2024-04-22T20:45:25.044325+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T20:45:25.044325+0000","last_undegraded":"2024-04-22T20:45:25.044325+0000","last_fullsized":"2024-04-22T20:45:25.044325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:22.264706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":668,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063186+0000","last_change":"2024-04-22T19:51:12.892628+0000","last_active":"2024-04-22T20:45:25.063186+0000","last_peered":"2024-04-22T20:45:25.063186+0000","last_clean":"2024-04-22T20:45:25.063186+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T20:45:25.063186+0000","last_undegraded":"2024-04-22T20:45:25.063186+0000","last_fullsized":"2024-04-22T20:45:25.063186+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:34.018008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"261'271","reported_seq":914,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.324539+0000","last_change":"2024-04-22T19:51:26.497957+0000","last_active":"2024-04-22T20:47:44.324539+0000","last_peered":"2024-04-22T20:47:44.324539+0000","last_clean":"2024-04-22T20:47:44.324539+0000","last_became_active":"2024-04-22T19:51:26.497608+0000","last_became_peered":"2024-04-22T19:51:26.497608+0000","last_unstale":"2024-04-22T20:47:44.324539+0000","last_undegraded":"2024-04-22T20:47:44.324539+0000","last_fullsized":"2024-04-22T20:47:44.324539+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:02:40.473180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16410029,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":454,"num_read_kb":42686,"num_write":1180,"num_write_kb":37697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189509+0000","last_change":"2024-04-22T19:51:30.524749+0000","last_active":"2024-04-22T20:45:40.189509+0000","last_peered":"2024-04-22T20:45:40.189509+0000","last_clean":"2024-04-22T20:45:40.189509+0000","last_became_active":"2024-04-22T19:51:30.524437+0000","last_became_peered":"2024-04-22T19:51:30.524437+0000","last_unstale":"2024-04-22T20:45:40.189509+0000","last_undegraded":"2024-04-22T20:45:40.189509+0000","last_fullsized":"2024-04-22T20:45:40.189509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:54.055370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"261'1086","reported_seq":2608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.549870+0000","last_change":"2024-04-22T19:51:32.538859+0000","last_active":"2024-04-22T20:48:08.549870+0000","last_peered":"2024-04-22T20:48:08.549870+0000","last_clean":"2024-04-22T20:48:08.549870+0000","last_became_active":"2024-04-22T19:51:32.538560+0000","last_became_peered":"2024-04-22T19:51:32.538560+0000","last_unstale":"2024-04-22T20:48:08.549870+0000","last_undegraded":"2024-04-22T20:48:08.549870+0000","last_fullsized":"2024-04-22T20:48:08.549870+0000","mapping_epoch":26,"log_start":"261'1000","ondisk_log_start":"261'1000","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":1000,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:19.961941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174953,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1739,"num_read_kb":3173,"num_write":1082,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.067953+0000","last_change":"2024-04-22T19:51:34.557341+0000","last_active":"2024-04-22T20:45:55.067953+0000","last_peered":"2024-04-22T20:45:55.067953+0000","last_clean":"2024-04-22T20:45:55.067953+0000","last_became_active":"2024-04-22T19:51:34.557022+0000","last_became_peered":"2024-04-22T19:51:34.557022+0000","last_unstale":"2024-04-22T20:45:55.067953+0000","last_undegraded":"2024-04-22T20:45:55.067953+0000","last_fullsized":"2024-04-22T20:45:55.067953+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:59:49.539365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"261'163","reported_seq":671,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:00.048103+0000","last_change":"2024-04-22T19:51:36.567874+0000","last_active":"2024-04-22T20:46:00.048103+0000","last_peered":"2024-04-22T20:46:00.048103+0000","last_clean":"2024-04-22T20:46:00.048103+0000","last_became_active":"2024-04-22T19:51:36.567603+0000","last_became_peered":"2024-04-22T19:51:36.567603+0000","last_unstale":"2024-04-22T20:46:00.048103+0000","last_undegraded":"2024-04-22T20:46:00.048103+0000","last_fullsized":"2024-04-22T20:46:00.048103+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:34:18.236728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":641,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":222,"num_read_kb":216,"num_write":373,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"261'364","reported_seq":998,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:32.355984+0000","last_change":"2024-04-22T19:51:28.512889+0000","last_active":"2024-04-22T20:46:32.355984+0000","last_peered":"2024-04-22T20:46:32.355984+0000","last_clean":"2024-04-22T20:46:32.355984+0000","last_became_active":"2024-04-22T19:51:28.512561+0000","last_became_peered":"2024-04-22T19:51:28.512561+0000","last_unstale":"2024-04-22T20:46:32.355984+0000","last_undegraded":"2024-04-22T20:46:32.355984+0000","last_fullsized":"2024-04-22T20:46:32.355984+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:13.130467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1455,"num_read_kb":1128,"num_write":627,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"261'324","reported_seq":1041,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183045+0000","last_change":"2024-04-22T19:51:26.500958+0000","last_active":"2024-04-22T20:45:40.183045+0000","last_peered":"2024-04-22T20:45:40.183045+0000","last_clean":"2024-04-22T20:45:40.183045+0000","last_became_active":"2024-04-22T19:51:26.498178+0000","last_became_peered":"2024-04-22T19:51:26.498178+0000","last_unstale":"2024-04-22T20:45:40.183045+0000","last_undegraded":"2024-04-22T20:45:40.183045+0000","last_fullsized":"2024-04-22T20:45:40.183045+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:54.358145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14073286,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":653,"num_read_kb":47025,"num_write":1473,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.155991+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T20:45:25.155991+0000","last_peered":"2024-04-22T20:45:25.155991+0000","last_clean":"2024-04-22T20:45:25.155991+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T20:45:25.155991+0000","last_undegraded":"2024-04-22T20:45:25.155991+0000","last_fullsized":"2024-04-22T20:45:25.155991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157032+0000","last_change":"2024-04-22T19:51:30.523537+0000","last_active":"2024-04-22T20:45:35.157032+0000","last_peered":"2024-04-22T20:45:35.157032+0000","last_clean":"2024-04-22T20:45:35.157032+0000","last_became_active":"2024-04-22T19:51:30.523242+0000","last_became_peered":"2024-04-22T19:51:30.523242+0000","last_unstale":"2024-04-22T20:45:35.157032+0000","last_undegraded":"2024-04-22T20:45:35.157032+0000","last_fullsized":"2024-04-22T20:45:35.157032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:21.847987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"261'692","reported_seq":1402,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.510075+0000","last_change":"2024-04-22T19:51:33.248507+0000","last_active":"2024-04-22T20:48:08.510075+0000","last_peered":"2024-04-22T20:48:08.510075+0000","last_clean":"2024-04-22T20:48:08.510075+0000","last_became_active":"2024-04-22T19:51:33.248144+0000","last_became_peered":"2024-04-22T19:51:33.248144+0000","last_unstale":"2024-04-22T20:48:08.510075+0000","last_undegraded":"2024-04-22T20:48:08.510075+0000","last_fullsized":"2024-04-22T20:48:08.510075+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":600,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:02:24.314016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":153222,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1224,"num_read_kb":2558,"num_write":1000,"num_write_kb":811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.158258+0000","last_change":"2024-04-22T19:51:34.562290+0000","last_active":"2024-04-22T20:45:45.158258+0000","last_peered":"2024-04-22T20:45:45.158258+0000","last_clean":"2024-04-22T20:45:45.158258+0000","last_became_active":"2024-04-22T19:51:34.562001+0000","last_became_peered":"2024-04-22T19:51:34.562001+0000","last_unstale":"2024-04-22T20:45:45.158258+0000","last_undegraded":"2024-04-22T20:45:45.158258+0000","last_fullsized":"2024-04-22T20:45:45.158258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:30:17.324055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"261'105","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:02.644513+0000","last_change":"2024-04-22T19:51:36.570607+0000","last_active":"2024-04-22T20:48:02.644513+0000","last_peered":"2024-04-22T20:48:02.644513+0000","last_clean":"2024-04-22T20:48:02.644513+0000","last_became_active":"2024-04-22T19:51:36.570134+0000","last_became_peered":"2024-04-22T19:51:36.570134+0000","last_unstale":"2024-04-22T20:48:02.644513+0000","last_undegraded":"2024-04-22T20:48:02.644513+0000","last_fullsized":"2024-04-22T20:48:02.644513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:44.632753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":256,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"261'101","reported_seq":688,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.244225+0000","last_change":"2024-04-22T19:51:28.516624+0000","last_active":"2024-04-22T20:47:05.244225+0000","last_peered":"2024-04-22T20:47:05.244225+0000","last_clean":"2024-04-22T20:47:05.244225+0000","last_became_active":"2024-04-22T19:51:28.516178+0000","last_became_peered":"2024-04-22T19:51:28.516178+0000","last_unstale":"2024-04-22T20:47:05.244225+0000","last_undegraded":"2024-04-22T20:47:05.244225+0000","last_fullsized":"2024-04-22T20:47:05.244225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:27:27.066102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":337,"num_write":140,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"261'295","reported_seq":990,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.157770+0000","last_change":"2024-04-22T19:51:26.503699+0000","last_active":"2024-04-22T20:45:40.157770+0000","last_peered":"2024-04-22T20:45:40.157770+0000","last_clean":"2024-04-22T20:45:40.157770+0000","last_became_active":"2024-04-22T19:51:26.502765+0000","last_became_peered":"2024-04-22T19:51:26.502765+0000","last_unstale":"2024-04-22T20:45:40.157770+0000","last_undegraded":"2024-04-22T20:45:40.157770+0000","last_fullsized":"2024-04-22T20:45:40.157770+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":200,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:12:35.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14551057,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":509,"num_read_kb":127053,"num_write":1319,"num_write_kb":34012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":518,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.180533+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T20:45:35.180533+0000","last_peered":"2024-04-22T20:45:35.180533+0000","last_clean":"2024-04-22T20:45:35.180533+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T20:45:35.180533+0000","last_undegraded":"2024-04-22T20:45:35.180533+0000","last_fullsized":"2024-04-22T20:45:35.180533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181286+0000","last_change":"2024-04-22T19:51:30.529790+0000","last_active":"2024-04-22T20:45:40.181286+0000","last_peered":"2024-04-22T20:45:40.181286+0000","last_clean":"2024-04-22T20:45:40.181286+0000","last_became_active":"2024-04-22T19:51:30.529437+0000","last_became_peered":"2024-04-22T19:51:30.529437+0000","last_unstale":"2024-04-22T20:45:40.181286+0000","last_undegraded":"2024-04-22T20:45:40.181286+0000","last_fullsized":"2024-04-22T20:45:40.181286+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:46:59.053426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"261'110","reported_seq":651,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182039+0000","last_change":"2024-04-22T19:51:32.534640+0000","last_active":"2024-04-22T20:45:45.182039+0000","last_peered":"2024-04-22T20:45:45.182039+0000","last_clean":"2024-04-22T20:45:45.182039+0000","last_became_active":"2024-04-22T19:51:32.534280+0000","last_became_peered":"2024-04-22T19:51:32.534280+0000","last_unstale":"2024-04-22T20:45:45.182039+0000","last_undegraded":"2024-04-22T20:45:45.182039+0000","last_fullsized":"2024-04-22T20:45:45.182039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:52.620330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"261'531","reported_seq":2151,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.555610+0000","last_change":"2024-04-22T19:51:34.557012+0000","last_active":"2024-04-22T20:48:08.555610+0000","last_peered":"2024-04-22T20:48:08.555610+0000","last_clean":"2024-04-22T20:48:08.555610+0000","last_became_active":"2024-04-22T19:51:34.556673+0000","last_became_peered":"2024-04-22T19:51:34.556673+0000","last_unstale":"2024-04-22T20:48:08.555610+0000","last_undegraded":"2024-04-22T20:48:08.555610+0000","last_fullsized":"2024-04-22T20:48:08.555610+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":500,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:18.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"261'149","reported_seq":639,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182998+0000","last_change":"2024-04-22T19:51:36.575637+0000","last_active":"2024-04-22T20:45:40.182998+0000","last_peered":"2024-04-22T20:45:40.182998+0000","last_clean":"2024-04-22T20:45:40.182998+0000","last_became_active":"2024-04-22T19:51:36.575351+0000","last_became_peered":"2024-04-22T19:51:36.575351+0000","last_unstale":"2024-04-22T20:45:40.182998+0000","last_undegraded":"2024-04-22T20:45:40.182998+0000","last_fullsized":"2024-04-22T20:45:40.182998+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:38:03.020774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":80,"num_write":319,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"261'146","reported_seq":771,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182955+0000","last_change":"2024-04-22T19:51:28.514202+0000","last_active":"2024-04-22T20:45:40.182955+0000","last_peered":"2024-04-22T20:45:40.182955+0000","last_clean":"2024-04-22T20:45:40.182955+0000","last_became_active":"2024-04-22T19:51:28.513940+0000","last_became_peered":"2024-04-22T19:51:28.513940+0000","last_unstale":"2024-04-22T20:45:40.182955+0000","last_undegraded":"2024-04-22T20:45:40.182955+0000","last_fullsized":"2024-04-22T20:45:40.182955+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:14:36.426996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":550,"num_write":219,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"261'276","reported_seq":914,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:28.475475+0000","last_change":"2024-04-22T19:51:26.506454+0000","last_active":"2024-04-22T20:46:28.475475+0000","last_peered":"2024-04-22T20:46:28.475475+0000","last_clean":"2024-04-22T20:46:28.475475+0000","last_became_active":"2024-04-22T19:51:26.505842+0000","last_became_peered":"2024-04-22T19:51:26.505842+0000","last_unstale":"2024-04-22T20:46:28.475475+0000","last_undegraded":"2024-04-22T20:46:28.475475+0000","last_fullsized":"2024-04-22T20:46:28.475475+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:24:40.890650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10581571,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":407,"num_read_kb":22189,"num_write":1035,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314053+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T20:45:20.314053+0000","last_peered":"2024-04-22T20:45:20.314053+0000","last_clean":"2024-04-22T20:45:20.314053+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T20:45:20.314053+0000","last_undegraded":"2024-04-22T20:45:20.314053+0000","last_fullsized":"2024-04-22T20:45:20.314053+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:01.430732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317090+0000","last_change":"2024-04-22T19:51:30.526076+0000","last_active":"2024-04-22T20:45:40.317090+0000","last_peered":"2024-04-22T20:45:40.317090+0000","last_clean":"2024-04-22T20:45:40.317090+0000","last_became_active":"2024-04-22T19:51:30.525777+0000","last_became_peered":"2024-04-22T19:51:30.525777+0000","last_unstale":"2024-04-22T20:45:40.317090+0000","last_undegraded":"2024-04-22T20:45:40.317090+0000","last_fullsized":"2024-04-22T20:45:40.317090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:18.779258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"261'179","reported_seq":746,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.046459+0000","last_change":"2024-04-22T19:51:33.248819+0000","last_active":"2024-04-22T20:45:45.046459+0000","last_peered":"2024-04-22T20:45:45.046459+0000","last_clean":"2024-04-22T20:45:45.046459+0000","last_became_active":"2024-04-22T19:51:33.248354+0000","last_became_peered":"2024-04-22T19:51:33.248354+0000","last_unstale":"2024-04-22T20:45:45.046459+0000","last_undegraded":"2024-04-22T20:45:45.046459+0000","last_fullsized":"2024-04-22T20:45:45.046459+0000","mapping_epoch":26,"log_start":"258'100","ondisk_log_start":"258'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:02:04.601587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":222,"num_read_kb":211,"num_write":148,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"261'544","reported_seq":4419,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.551803+0000","last_change":"2024-04-22T19:51:34.561431+0000","last_active":"2024-04-22T20:48:08.551803+0000","last_peered":"2024-04-22T20:48:08.551803+0000","last_clean":"2024-04-22T20:48:08.551803+0000","last_became_active":"2024-04-22T19:51:34.561206+0000","last_became_peered":"2024-04-22T19:51:34.561206+0000","last_unstale":"2024-04-22T20:48:08.551803+0000","last_undegraded":"2024-04-22T20:48:08.551803+0000","last_fullsized":"2024-04-22T20:48:08.551803+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":500,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:11:06.654339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"261'182","reported_seq":679,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.174164+0000","last_change":"2024-04-22T19:51:36.569612+0000","last_active":"2024-04-22T20:48:00.174164+0000","last_peered":"2024-04-22T20:48:00.174164+0000","last_clean":"2024-04-22T20:48:00.174164+0000","last_became_active":"2024-04-22T19:51:36.569338+0000","last_became_peered":"2024-04-22T19:51:36.569338+0000","last_unstale":"2024-04-22T20:48:00.174164+0000","last_undegraded":"2024-04-22T20:48:00.174164+0000","last_fullsized":"2024-04-22T20:48:00.174164+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:23:38.495469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":109,"num_write":422,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"261'109","reported_seq":704,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045900+0000","last_change":"2024-04-22T19:51:28.512614+0000","last_active":"2024-04-22T20:45:40.045900+0000","last_peered":"2024-04-22T20:45:40.045900+0000","last_clean":"2024-04-22T20:45:40.045900+0000","last_became_active":"2024-04-22T19:51:28.512337+0000","last_became_peered":"2024-04-22T19:51:28.512337+0000","last_unstale":"2024-04-22T20:45:40.045900+0000","last_undegraded":"2024-04-22T20:45:40.045900+0000","last_fullsized":"2024-04-22T20:45:40.045900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:06:34.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":434,"num_read_kb":362,"num_write":158,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'260","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.190983+0000","last_change":"2024-04-22T19:51:26.508311+0000","last_active":"2024-04-22T20:45:50.190983+0000","last_peered":"2024-04-22T20:45:50.190983+0000","last_clean":"2024-04-22T20:45:50.190983+0000","last_became_active":"2024-04-22T19:51:26.507804+0000","last_became_peered":"2024-04-22T19:51:26.507804+0000","last_unstale":"2024-04-22T20:45:50.190983+0000","last_undegraded":"2024-04-22T20:45:50.190983+0000","last_fullsized":"2024-04-22T20:45:50.190983+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:56:54.837990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1111886,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":452,"num_read_kb":11012,"num_write":1175,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314068+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T20:45:20.314068+0000","last_peered":"2024-04-22T20:45:20.314068+0000","last_clean":"2024-04-22T20:45:20.314068+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T20:45:20.314068+0000","last_undegraded":"2024-04-22T20:45:20.314068+0000","last_fullsized":"2024-04-22T20:45:20.314068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:23.800559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182974+0000","last_change":"2024-04-22T19:51:30.527109+0000","last_active":"2024-04-22T20:45:40.182974+0000","last_peered":"2024-04-22T20:45:40.182974+0000","last_clean":"2024-04-22T20:45:40.182974+0000","last_became_active":"2024-04-22T19:51:30.526837+0000","last_became_peered":"2024-04-22T19:51:30.526837+0000","last_unstale":"2024-04-22T20:45:40.182974+0000","last_undegraded":"2024-04-22T20:45:40.182974+0000","last_fullsized":"2024-04-22T20:45:40.182974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:13.442240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"261'532","reported_seq":1192,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.513945+0000","last_change":"2024-04-22T19:51:32.542607+0000","last_active":"2024-04-22T20:48:08.513945+0000","last_peered":"2024-04-22T20:48:08.513945+0000","last_clean":"2024-04-22T20:48:08.513945+0000","last_became_active":"2024-04-22T19:51:32.542344+0000","last_became_peered":"2024-04-22T19:51:32.542344+0000","last_unstale":"2024-04-22T20:48:08.513945+0000","last_undegraded":"2024-04-22T20:48:08.513945+0000","last_fullsized":"2024-04-22T20:48:08.513945+0000","mapping_epoch":26,"log_start":"261'500","ondisk_log_start":"261'500","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":500,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:56:47.214088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115148,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":920,"num_read_kb":1851,"num_write":780,"num_write_kb":630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.181987+0000","last_change":"2024-04-22T19:51:34.551264+0000","last_active":"2024-04-22T20:45:45.181987+0000","last_peered":"2024-04-22T20:45:45.181987+0000","last_clean":"2024-04-22T20:45:45.181987+0000","last_became_active":"2024-04-22T19:51:34.550951+0000","last_became_peered":"2024-04-22T19:51:34.550951+0000","last_unstale":"2024-04-22T20:45:45.181987+0000","last_undegraded":"2024-04-22T20:45:45.181987+0000","last_fullsized":"2024-04-22T20:45:45.181987+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:34:20.504558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"261'112","reported_seq":617,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.198696+0000","last_change":"2024-04-22T19:51:36.573230+0000","last_active":"2024-04-22T20:47:55.198696+0000","last_peered":"2024-04-22T20:47:55.198696+0000","last_clean":"2024-04-22T20:47:55.198696+0000","last_became_active":"2024-04-22T19:51:36.572976+0000","last_became_peered":"2024-04-22T19:51:36.572976+0000","last_unstale":"2024-04-22T20:47:55.198696+0000","last_undegraded":"2024-04-22T20:47:55.198696+0000","last_fullsized":"2024-04-22T20:47:55.198696+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:14.048628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":120,"num_write":270,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"261'130","reported_seq":681,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.160770+0000","last_change":"2024-04-22T19:51:36.574914+0000","last_active":"2024-04-22T20:48:03.160770+0000","last_peered":"2024-04-22T20:48:03.160770+0000","last_clean":"2024-04-22T20:48:03.160770+0000","last_became_active":"2024-04-22T19:51:36.574238+0000","last_became_peered":"2024-04-22T19:51:36.574238+0000","last_unstale":"2024-04-22T20:48:03.160770+0000","last_undegraded":"2024-04-22T20:48:03.160770+0000","last_fullsized":"2024-04-22T20:48:03.160770+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:09.269509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":193,"num_read_kb":192,"num_write":291,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"261'123","reported_seq":739,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176757+0000","last_change":"2024-04-22T19:51:28.515646+0000","last_active":"2024-04-22T20:45:10.176757+0000","last_peered":"2024-04-22T20:45:10.176757+0000","last_clean":"2024-04-22T20:45:10.176757+0000","last_became_active":"2024-04-22T19:51:28.515334+0000","last_became_peered":"2024-04-22T19:51:28.515334+0000","last_unstale":"2024-04-22T20:45:10.176757+0000","last_undegraded":"2024-04-22T20:45:10.176757+0000","last_fullsized":"2024-04-22T20:45:10.176757+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:47:44.805680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":424,"num_write":184,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"261'258","reported_seq":971,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.178176+0000","last_change":"2024-04-22T19:51:26.506527+0000","last_active":"2024-04-22T20:45:20.178176+0000","last_peered":"2024-04-22T20:45:20.178176+0000","last_clean":"2024-04-22T20:45:20.178176+0000","last_became_active":"2024-04-22T19:51:26.506150+0000","last_became_peered":"2024-04-22T19:51:26.506150+0000","last_unstale":"2024-04-22T20:45:20.178176+0000","last_undegraded":"2024-04-22T20:45:20.178176+0000","last_fullsized":"2024-04-22T20:45:20.178176+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:28.117664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7665617,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":650,"num_read_kb":35400,"num_write":1047,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.042550+0000","last_change":"2024-04-22T19:51:30.523906+0000","last_active":"2024-04-22T20:45:10.042550+0000","last_peered":"2024-04-22T20:45:10.042550+0000","last_clean":"2024-04-22T20:45:10.042550+0000","last_became_active":"2024-04-22T19:51:30.522903+0000","last_became_peered":"2024-04-22T19:51:30.522903+0000","last_unstale":"2024-04-22T20:45:10.042550+0000","last_undegraded":"2024-04-22T20:45:10.042550+0000","last_fullsized":"2024-04-22T20:45:10.042550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:12.331782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"261'291","reported_seq":870,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171073+0000","last_change":"2024-04-22T19:51:32.539884+0000","last_active":"2024-04-22T20:47:35.171073+0000","last_peered":"2024-04-22T20:47:35.171073+0000","last_clean":"2024-04-22T20:47:35.171073+0000","last_became_active":"2024-04-22T19:51:32.539631+0000","last_became_peered":"2024-04-22T19:51:32.539631+0000","last_unstale":"2024-04-22T20:47:35.171073+0000","last_undegraded":"2024-04-22T20:47:35.171073+0000","last_fullsized":"2024-04-22T20:47:35.171073+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:00:28.566624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59572,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":482,"num_read_kb":991,"num_write":398,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154587+0000","last_change":"2024-04-22T19:51:34.556940+0000","last_active":"2024-04-22T20:45:15.154587+0000","last_peered":"2024-04-22T20:45:15.154587+0000","last_clean":"2024-04-22T20:45:15.154587+0000","last_became_active":"2024-04-22T19:51:34.556650+0000","last_became_peered":"2024-04-22T19:51:34.556650+0000","last_unstale":"2024-04-22T20:45:15.154587+0000","last_undegraded":"2024-04-22T20:45:15.154587+0000","last_fullsized":"2024-04-22T20:45:15.154587+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:47:48.478954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"261'99","reported_seq":599,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.172452+0000","last_change":"2024-04-22T19:51:36.567251+0000","last_active":"2024-04-22T20:47:45.172452+0000","last_peered":"2024-04-22T20:47:45.172452+0000","last_clean":"2024-04-22T20:47:45.172452+0000","last_became_active":"2024-04-22T19:51:36.566980+0000","last_became_peered":"2024-04-22T19:51:36.566980+0000","last_unstale":"2024-04-22T20:47:45.172452+0000","last_undegraded":"2024-04-22T20:47:45.172452+0000","last_fullsized":"2024-04-22T20:47:45.172452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:01:00.638641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":235,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"261'130","reported_seq":734,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.042558+0000","last_change":"2024-04-22T19:51:28.517914+0000","last_active":"2024-04-22T20:45:10.042558+0000","last_peered":"2024-04-22T20:45:10.042558+0000","last_clean":"2024-04-22T20:45:10.042558+0000","last_became_active":"2024-04-22T19:51:28.517675+0000","last_became_peered":"2024-04-22T19:51:28.517675+0000","last_unstale":"2024-04-22T20:45:10.042558+0000","last_undegraded":"2024-04-22T20:45:10.042558+0000","last_fullsized":"2024-04-22T20:45:10.042558+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:18:10.255589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":543,"num_read_kb":446,"num_write":195,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"261'276","reported_seq":976,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:16.477341+0000","last_change":"2024-04-22T19:51:26.502622+0000","last_active":"2024-04-22T20:45:16.477341+0000","last_peered":"2024-04-22T20:45:16.477341+0000","last_clean":"2024-04-22T20:45:16.477341+0000","last_became_active":"2024-04-22T19:51:26.501897+0000","last_became_peered":"2024-04-22T19:51:26.501897+0000","last_unstale":"2024-04-22T20:45:16.477341+0000","last_undegraded":"2024-04-22T20:45:16.477341+0000","last_fullsized":"2024-04-22T20:45:16.477341+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:33.272475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4946722,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":548,"num_read_kb":41436,"num_write":1081,"num_write_kb":29344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.154110+0000","last_change":"2024-04-22T19:51:30.524665+0000","last_active":"2024-04-22T20:45:10.154110+0000","last_peered":"2024-04-22T20:45:10.154110+0000","last_clean":"2024-04-22T20:45:10.154110+0000","last_became_active":"2024-04-22T19:51:30.524388+0000","last_became_peered":"2024-04-22T19:51:30.524388+0000","last_unstale":"2024-04-22T20:45:10.154110+0000","last_undegraded":"2024-04-22T20:45:10.154110+0000","last_fullsized":"2024-04-22T20:45:10.154110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:58.393800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"261'49","reported_seq":553,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179874+0000","last_change":"2024-04-22T19:51:32.539546+0000","last_active":"2024-04-22T20:45:15.179874+0000","last_peered":"2024-04-22T20:45:15.179874+0000","last_clean":"2024-04-22T20:45:15.179874+0000","last_became_active":"2024-04-22T19:51:32.539260+0000","last_became_peered":"2024-04-22T19:51:32.539260+0000","last_unstale":"2024-04-22T20:45:15.179874+0000","last_undegraded":"2024-04-22T20:45:15.179874+0000","last_fullsized":"2024-04-22T20:45:15.179874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:13:32.092655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":36,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.061461+0000","last_change":"2024-04-22T19:51:34.551584+0000","last_active":"2024-04-22T20:45:15.061461+0000","last_peered":"2024-04-22T20:45:15.061461+0000","last_clean":"2024-04-22T20:45:15.061461+0000","last_became_active":"2024-04-22T19:51:34.551294+0000","last_became_peered":"2024-04-22T19:51:34.551294+0000","last_unstale":"2024-04-22T20:45:15.061461+0000","last_undegraded":"2024-04-22T20:45:15.061461+0000","last_fullsized":"2024-04-22T20:45:15.061461+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:53:58.560478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"261'201","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.005045+0000","last_change":"2024-04-22T19:51:36.574227+0000","last_active":"2024-04-22T20:48:03.005045+0000","last_peered":"2024-04-22T20:48:03.005045+0000","last_clean":"2024-04-22T20:48:03.005045+0000","last_became_active":"2024-04-22T19:51:36.573936+0000","last_became_peered":"2024-04-22T19:51:36.573936+0000","last_unstale":"2024-04-22T20:48:03.005045+0000","last_undegraded":"2024-04-22T20:48:03.005045+0000","last_fullsized":"2024-04-22T20:48:03.005045+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:05:17.804766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":284,"num_write":451,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"261'231","reported_seq":917,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.154075+0000","last_change":"2024-04-22T19:51:28.508566+0000","last_active":"2024-04-22T20:45:10.154075+0000","last_peered":"2024-04-22T20:45:10.154075+0000","last_clean":"2024-04-22T20:45:10.154075+0000","last_became_active":"2024-04-22T19:51:28.508307+0000","last_became_peered":"2024-04-22T19:51:28.508307+0000","last_unstale":"2024-04-22T20:45:10.154075+0000","last_undegraded":"2024-04-22T20:45:10.154075+0000","last_fullsized":"2024-04-22T20:45:10.154075+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:48.292329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":959,"num_read_kb":784,"num_write":348,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"261'268","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180692+0000","last_change":"2024-04-22T19:51:26.501920+0000","last_active":"2024-04-22T20:44:40.180692+0000","last_peered":"2024-04-22T20:44:40.180692+0000","last_clean":"2024-04-22T20:44:40.180692+0000","last_became_active":"2024-04-22T19:51:26.501477+0000","last_became_peered":"2024-04-22T19:51:26.501477+0000","last_unstale":"2024-04-22T20:44:40.180692+0000","last_undegraded":"2024-04-22T20:44:40.180692+0000","last_fullsized":"2024-04-22T20:44:40.180692+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":200,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:06.904602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16045811,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":422,"num_read_kb":46522,"num_write":1225,"num_write_kb":35021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060725+0000","last_change":"2024-04-22T19:51:30.527405+0000","last_active":"2024-04-22T20:45:10.060725+0000","last_peered":"2024-04-22T20:45:10.060725+0000","last_clean":"2024-04-22T20:45:10.060725+0000","last_became_active":"2024-04-22T19:51:30.527076+0000","last_became_peered":"2024-04-22T19:51:30.527076+0000","last_unstale":"2024-04-22T20:45:10.060725+0000","last_undegraded":"2024-04-22T20:45:10.060725+0000","last_fullsized":"2024-04-22T20:45:10.060725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:38.558657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"261'297","reported_seq":892,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171120+0000","last_change":"2024-04-22T19:51:32.542586+0000","last_active":"2024-04-22T20:47:35.171120+0000","last_peered":"2024-04-22T20:47:35.171120+0000","last_clean":"2024-04-22T20:47:35.171120+0000","last_became_active":"2024-04-22T19:51:32.542262+0000","last_became_peered":"2024-04-22T19:51:32.542262+0000","last_unstale":"2024-04-22T20:47:35.171120+0000","last_undegraded":"2024-04-22T20:47:35.171120+0000","last_fullsized":"2024-04-22T20:47:35.171120+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:52:10.014200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55208,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":478,"num_read_kb":917,"num_write":378,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.313200+0000","last_change":"2024-04-22T19:51:34.554113+0000","last_active":"2024-04-22T20:45:15.313200+0000","last_peered":"2024-04-22T20:45:15.313200+0000","last_clean":"2024-04-22T20:45:15.313200+0000","last_became_active":"2024-04-22T19:51:34.553691+0000","last_became_peered":"2024-04-22T19:51:34.553691+0000","last_unstale":"2024-04-22T20:45:15.313200+0000","last_undegraded":"2024-04-22T20:45:15.313200+0000","last_fullsized":"2024-04-22T20:45:15.313200+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:08:07.000668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"261'135","reported_seq":616,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:10.176306+0000","last_change":"2024-04-22T19:51:36.571557+0000","last_active":"2024-04-22T20:44:10.176306+0000","last_peered":"2024-04-22T20:44:10.176306+0000","last_clean":"2024-04-22T20:44:10.176306+0000","last_became_active":"2024-04-22T19:51:36.571290+0000","last_became_peered":"2024-04-22T19:51:36.571290+0000","last_unstale":"2024-04-22T20:44:10.176306+0000","last_undegraded":"2024-04-22T20:44:10.176306+0000","last_fullsized":"2024-04-22T20:44:10.176306+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:42:28.669764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":111,"num_write":334,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"261'185","reported_seq":854,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:01.148987+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:47:01.148987+0000","last_peered":"2024-04-22T20:47:01.148987+0000","last_clean":"2024-04-22T20:47:01.148987+0000","last_became_active":"2024-04-22T19:51:28.506615+0000","last_became_peered":"2024-04-22T19:51:28.506615+0000","last_unstale":"2024-04-22T20:47:01.148987+0000","last_undegraded":"2024-04-22T20:47:01.148987+0000","last_fullsized":"2024-04-22T20:47:01.148987+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:25.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":757,"num_read_kb":623,"num_write":269,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'270","reported_seq":911,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:43.414545+0000","last_change":"2024-04-22T19:51:26.503480+0000","last_active":"2024-04-22T20:44:43.414545+0000","last_peered":"2024-04-22T20:44:43.414545+0000","last_clean":"2024-04-22T20:44:43.414545+0000","last_became_active":"2024-04-22T19:51:26.502938+0000","last_became_peered":"2024-04-22T19:51:26.502938+0000","last_unstale":"2024-04-22T20:44:43.414545+0000","last_undegraded":"2024-04-22T20:44:43.414545+0000","last_fullsized":"2024-04-22T20:44:43.414545+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:25.122842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11884442,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":410,"num_read_kb":28738,"num_write":1201,"num_write_kb":45425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.312594+0000","last_change":"2024-04-22T19:51:30.527273+0000","last_active":"2024-04-22T20:45:10.312594+0000","last_peered":"2024-04-22T20:45:10.312594+0000","last_clean":"2024-04-22T20:45:10.312594+0000","last_became_active":"2024-04-22T19:51:30.526994+0000","last_became_peered":"2024-04-22T19:51:30.526994+0000","last_unstale":"2024-04-22T20:45:10.312594+0000","last_undegraded":"2024-04-22T20:45:10.312594+0000","last_fullsized":"2024-04-22T20:45:10.312594+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:05:35.522811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"261'246","reported_seq":812,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.082501+0000","last_change":"2024-04-22T19:51:32.538193+0000","last_active":"2024-04-22T20:47:35.082501+0000","last_peered":"2024-04-22T20:47:35.082501+0000","last_clean":"2024-04-22T20:47:35.082501+0000","last_became_active":"2024-04-22T19:51:32.537875+0000","last_became_peered":"2024-04-22T19:51:32.537875+0000","last_unstale":"2024-04-22T20:47:35.082501+0000","last_undegraded":"2024-04-22T20:47:35.082501+0000","last_fullsized":"2024-04-22T20:47:35.082501+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:43.382645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50977,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":409,"num_read_kb":848,"num_write":332,"num_write_kb":266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179968+0000","last_change":"2024-04-22T19:51:34.554251+0000","last_active":"2024-04-22T20:45:15.179968+0000","last_peered":"2024-04-22T20:45:15.179968+0000","last_clean":"2024-04-22T20:45:15.179968+0000","last_became_active":"2024-04-22T19:51:34.553954+0000","last_became_peered":"2024-04-22T19:51:34.553954+0000","last_unstale":"2024-04-22T20:45:15.179968+0000","last_undegraded":"2024-04-22T20:45:15.179968+0000","last_fullsized":"2024-04-22T20:45:15.179968+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:25:45.427696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"261'192","reported_seq":830,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317164+0000","last_change":"2024-04-22T19:51:28.514880+0000","last_active":"2024-04-22T20:45:40.317164+0000","last_peered":"2024-04-22T20:45:40.317164+0000","last_clean":"2024-04-22T20:45:40.317164+0000","last_became_active":"2024-04-22T19:51:28.514593+0000","last_became_peered":"2024-04-22T19:51:28.514593+0000","last_unstale":"2024-04-22T20:45:40.317164+0000","last_undegraded":"2024-04-22T20:45:40.317164+0000","last_fullsized":"2024-04-22T20:45:40.317164+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:10:57.618723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":788,"num_read_kb":638,"num_write":274,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"261'282","reported_seq":944,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:16.752867+0000","last_change":"2024-04-22T19:51:26.502507+0000","last_active":"2024-04-22T20:47:16.752867+0000","last_peered":"2024-04-22T20:47:16.752867+0000","last_clean":"2024-04-22T20:47:16.752867+0000","last_became_active":"2024-04-22T19:51:26.501812+0000","last_became_peered":"2024-04-22T19:51:26.501812+0000","last_unstale":"2024-04-22T20:47:16.752867+0000","last_undegraded":"2024-04-22T20:47:16.752867+0000","last_fullsized":"2024-04-22T20:47:16.752867+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:47:54.670392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1914449,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":492,"num_read_kb":55834,"num_write":1102,"num_write_kb":35884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183098+0000","last_change":"2024-04-22T19:51:30.525750+0000","last_active":"2024-04-22T20:45:40.183098+0000","last_peered":"2024-04-22T20:45:40.183098+0000","last_clean":"2024-04-22T20:45:40.183098+0000","last_became_active":"2024-04-22T19:51:30.525450+0000","last_became_peered":"2024-04-22T19:51:30.525450+0000","last_unstale":"2024-04-22T20:45:40.183098+0000","last_undegraded":"2024-04-22T20:45:40.183098+0000","last_fullsized":"2024-04-22T20:45:40.183098+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:32.196222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"261'474","reported_seq":1092,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.506230+0000","last_change":"2024-04-22T19:51:32.541007+0000","last_active":"2024-04-22T20:48:08.506230+0000","last_peered":"2024-04-22T20:48:08.506230+0000","last_clean":"2024-04-22T20:48:08.506230+0000","last_became_active":"2024-04-22T19:51:32.540755+0000","last_became_peered":"2024-04-22T19:51:32.540755+0000","last_unstale":"2024-04-22T20:48:08.506230+0000","last_undegraded":"2024-04-22T20:48:08.506230+0000","last_fullsized":"2024-04-22T20:48:08.506230+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":400,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:18.375444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112197,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":849,"num_read_kb":1808,"num_write":744,"num_write_kb":667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":482,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.175603+0000","last_change":"2024-04-22T19:51:34.553700+0000","last_active":"2024-04-22T20:44:05.175603+0000","last_peered":"2024-04-22T20:44:05.175603+0000","last_clean":"2024-04-22T20:44:05.175603+0000","last_became_active":"2024-04-22T19:51:34.553363+0000","last_became_peered":"2024-04-22T19:51:34.553363+0000","last_unstale":"2024-04-22T20:44:05.175603+0000","last_undegraded":"2024-04-22T20:44:05.175603+0000","last_fullsized":"2024-04-22T20:44:05.175603+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:13:11.618765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"261'159","reported_seq":673,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:02.819260+0000","last_change":"2024-04-22T19:51:36.569083+0000","last_active":"2024-04-22T20:48:02.819260+0000","last_peered":"2024-04-22T20:48:02.819260+0000","last_clean":"2024-04-22T20:48:02.819260+0000","last_became_active":"2024-04-22T19:51:36.568373+0000","last_became_peered":"2024-04-22T19:51:36.568373+0000","last_unstale":"2024-04-22T20:48:02.819260+0000","last_undegraded":"2024-04-22T20:48:02.819260+0000","last_fullsized":"2024-04-22T20:48:02.819260+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:00:29.149286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":213,"num_write":373,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"261'100","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:11.966474+0000","last_change":"2024-04-22T19:51:28.510922+0000","last_active":"2024-04-22T20:47:11.966474+0000","last_peered":"2024-04-22T20:47:11.966474+0000","last_clean":"2024-04-22T20:47:11.966474+0000","last_became_active":"2024-04-22T19:51:28.510629+0000","last_became_peered":"2024-04-22T19:51:28.510629+0000","last_unstale":"2024-04-22T20:47:11.966474+0000","last_undegraded":"2024-04-22T20:47:11.966474+0000","last_fullsized":"2024-04-22T20:47:11.966474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:55:55.876914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":435,"num_read_kb":357,"num_write":161,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"261'250","reported_seq":883,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:14.134249+0000","last_change":"2024-04-22T19:51:26.497497+0000","last_active":"2024-04-22T20:47:14.134249+0000","last_peered":"2024-04-22T20:47:14.134249+0000","last_clean":"2024-04-22T20:47:14.134249+0000","last_became_active":"2024-04-22T19:51:26.497165+0000","last_became_peered":"2024-04-22T19:51:26.497165+0000","last_unstale":"2024-04-22T20:47:14.134249+0000","last_undegraded":"2024-04-22T20:47:14.134249+0000","last_fullsized":"2024-04-22T20:47:14.134249+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":200,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:31.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13593714,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":438,"num_read_kb":39207,"num_write":1067,"num_write_kb":42297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317184+0000","last_change":"2024-04-22T19:51:30.526861+0000","last_active":"2024-04-22T20:45:40.317184+0000","last_peered":"2024-04-22T20:45:40.317184+0000","last_clean":"2024-04-22T20:45:40.317184+0000","last_became_active":"2024-04-22T19:51:30.526597+0000","last_became_peered":"2024-04-22T19:51:30.526597+0000","last_unstale":"2024-04-22T20:45:40.317184+0000","last_undegraded":"2024-04-22T20:45:40.317184+0000","last_fullsized":"2024-04-22T20:45:40.317184+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:24.696235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"261'306","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.074224+0000","last_change":"2024-04-22T19:51:32.537791+0000","last_active":"2024-04-22T20:46:35.074224+0000","last_peered":"2024-04-22T20:46:35.074224+0000","last_clean":"2024-04-22T20:46:35.074224+0000","last_became_active":"2024-04-22T19:51:32.537416+0000","last_became_peered":"2024-04-22T19:51:32.537416+0000","last_unstale":"2024-04-22T20:46:35.074224+0000","last_undegraded":"2024-04-22T20:46:35.074224+0000","last_fullsized":"2024-04-22T20:46:35.074224+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:35:00.012988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55718,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":984,"num_write":430,"num_write_kb":328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":484,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.035331+0000","last_change":"2024-04-22T19:51:34.561345+0000","last_active":"2024-04-22T20:44:05.035331+0000","last_peered":"2024-04-22T20:44:05.035331+0000","last_clean":"2024-04-22T20:44:05.035331+0000","last_became_active":"2024-04-22T19:51:34.561096+0000","last_became_peered":"2024-04-22T19:51:34.561096+0000","last_unstale":"2024-04-22T20:44:05.035331+0000","last_undegraded":"2024-04-22T20:44:05.035331+0000","last_fullsized":"2024-04-22T20:44:05.035331+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:38.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"261'161","reported_seq":661,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314006+0000","last_change":"2024-04-22T19:51:36.571812+0000","last_active":"2024-04-22T20:45:20.314006+0000","last_peered":"2024-04-22T20:45:20.314006+0000","last_clean":"2024-04-22T20:45:20.314006+0000","last_became_active":"2024-04-22T19:51:36.571565+0000","last_became_peered":"2024-04-22T19:51:36.571565+0000","last_unstale":"2024-04-22T20:45:20.314006+0000","last_undegraded":"2024-04-22T20:45:20.314006+0000","last_fullsized":"2024-04-22T20:45:20.314006+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:55.978840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":180,"num_write":473,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"261'44","reported_seq":609,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145600+0000","last_change":"2024-04-22T19:51:28.507235+0000","last_active":"2024-04-22T20:44:00.145600+0000","last_peered":"2024-04-22T20:44:00.145600+0000","last_clean":"2024-04-22T20:44:00.145600+0000","last_became_active":"2024-04-22T19:51:28.506835+0000","last_became_peered":"2024-04-22T19:51:28.506835+0000","last_unstale":"2024-04-22T20:44:00.145600+0000","last_undegraded":"2024-04-22T20:44:00.145600+0000","last_fullsized":"2024-04-22T20:44:00.145600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:22:38.074432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":202,"num_read_kb":174,"num_write":58,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"261'309","reported_seq":936,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.304458+0000","last_change":"2024-04-22T19:52:04.618218+0000","last_active":"2024-04-22T20:47:38.304458+0000","last_peered":"2024-04-22T20:47:38.304458+0000","last_clean":"2024-04-22T20:47:38.304458+0000","last_became_active":"2024-04-22T19:52:04.617820+0000","last_became_peered":"2024-04-22T19:52:04.617820+0000","last_unstale":"2024-04-22T20:47:38.304458+0000","last_undegraded":"2024-04-22T20:47:38.304458+0000","last_fullsized":"2024-04-22T20:47:38.304458+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:04:41.066429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588803,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":467,"num_read_kb":31540,"num_write":1405,"num_write_kb":43247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166735+0000","last_change":"2024-04-22T19:51:30.529632+0000","last_active":"2024-04-22T20:44:00.166735+0000","last_peered":"2024-04-22T20:44:00.166735+0000","last_clean":"2024-04-22T20:44:00.166735+0000","last_became_active":"2024-04-22T19:51:30.529322+0000","last_became_peered":"2024-04-22T19:51:30.529322+0000","last_unstale":"2024-04-22T20:44:00.166735+0000","last_undegraded":"2024-04-22T20:44:00.166735+0000","last_fullsized":"2024-04-22T20:44:00.166735+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:43:15.680192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"261'266","reported_seq":848,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.025839+0000","last_change":"2024-04-22T19:51:33.248631+0000","last_active":"2024-04-22T20:48:03.025839+0000","last_peered":"2024-04-22T20:48:03.025839+0000","last_clean":"2024-04-22T20:48:03.025839+0000","last_became_active":"2024-04-22T19:51:33.248293+0000","last_became_peered":"2024-04-22T19:51:33.248293+0000","last_unstale":"2024-04-22T20:48:03.025839+0000","last_undegraded":"2024-04-22T20:48:03.025839+0000","last_fullsized":"2024-04-22T20:48:03.025839+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:49.051719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66888,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":442,"num_read_kb":925,"num_write":358,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.046532+0000","last_change":"2024-04-22T19:51:34.561273+0000","last_active":"2024-04-22T20:45:45.046532+0000","last_peered":"2024-04-22T20:45:45.046532+0000","last_clean":"2024-04-22T20:45:45.046532+0000","last_became_active":"2024-04-22T19:51:34.560955+0000","last_became_peered":"2024-04-22T19:51:34.560955+0000","last_unstale":"2024-04-22T20:45:45.046532+0000","last_undegraded":"2024-04-22T20:45:45.046532+0000","last_fullsized":"2024-04-22T20:45:45.046532+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:03:55.584983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"261'119","reported_seq":602,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.536618+0000","last_change":"2024-04-22T19:51:36.567411+0000","last_active":"2024-04-22T20:48:08.536618+0000","last_peered":"2024-04-22T20:48:08.536618+0000","last_clean":"2024-04-22T20:48:08.536618+0000","last_became_active":"2024-04-22T19:51:36.567139+0000","last_became_peered":"2024-04-22T19:51:36.567139+0000","last_unstale":"2024-04-22T20:48:08.536618+0000","last_undegraded":"2024-04-22T20:48:08.536618+0000","last_fullsized":"2024-04-22T20:48:08.536618+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:39:35.365237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":92,"num_write":268,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"261'106","reported_seq":688,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.058728+0000","last_change":"2024-04-22T19:51:28.512939+0000","last_active":"2024-04-22T20:44:00.058728+0000","last_peered":"2024-04-22T20:44:00.058728+0000","last_clean":"2024-04-22T20:44:00.058728+0000","last_became_active":"2024-04-22T19:51:28.512642+0000","last_became_peered":"2024-04-22T19:51:28.512642+0000","last_unstale":"2024-04-22T20:44:00.058728+0000","last_undegraded":"2024-04-22T20:44:00.058728+0000","last_fullsized":"2024-04-22T20:44:00.058728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:36.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":349,"num_write":169,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"261'262","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.171114+0000","last_change":"2024-04-22T19:51:26.512313+0000","last_active":"2024-04-22T20:44:00.171114+0000","last_peered":"2024-04-22T20:44:00.171114+0000","last_clean":"2024-04-22T20:44:00.171114+0000","last_became_active":"2024-04-22T19:51:26.511966+0000","last_became_peered":"2024-04-22T19:51:26.511966+0000","last_unstale":"2024-04-22T20:44:00.171114+0000","last_undegraded":"2024-04-22T20:44:00.171114+0000","last_fullsized":"2024-04-22T20:44:00.171114+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:52.017119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7663079,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":370,"num_read_kb":38272,"num_write":1038,"num_write_kb":51509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.034785+0000","last_change":"2024-04-22T19:51:30.527261+0000","last_active":"2024-04-22T20:44:00.034785+0000","last_peered":"2024-04-22T20:44:00.034785+0000","last_clean":"2024-04-22T20:44:00.034785+0000","last_became_active":"2024-04-22T19:51:30.526949+0000","last_became_peered":"2024-04-22T19:51:30.526949+0000","last_unstale":"2024-04-22T20:44:00.034785+0000","last_undegraded":"2024-04-22T20:44:00.034785+0000","last_fullsized":"2024-04-22T20:44:00.034785+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:13:41.016976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"261'103","reported_seq":629,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182111+0000","last_change":"2024-04-22T19:51:33.248558+0000","last_active":"2024-04-22T20:45:45.182111+0000","last_peered":"2024-04-22T20:45:45.182111+0000","last_clean":"2024-04-22T20:45:45.182111+0000","last_became_active":"2024-04-22T19:51:33.248113+0000","last_became_peered":"2024-04-22T19:51:33.248113+0000","last_unstale":"2024-04-22T20:45:45.182111+0000","last_undegraded":"2024-04-22T20:45:45.182111+0000","last_fullsized":"2024-04-22T20:45:45.182111+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:09.138747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":119,"num_read_kb":113,"num_write":116,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"261'515","reported_seq":2294,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.550861+0000","last_change":"2024-04-22T19:51:34.557105+0000","last_active":"2024-04-22T20:48:08.550861+0000","last_peered":"2024-04-22T20:48:08.550861+0000","last_clean":"2024-04-22T20:48:08.550861+0000","last_became_active":"2024-04-22T19:51:34.556789+0000","last_became_peered":"2024-04-22T19:51:34.556789+0000","last_unstale":"2024-04-22T20:48:08.550861+0000","last_undegraded":"2024-04-22T20:48:08.550861+0000","last_fullsized":"2024-04-22T20:48:08.550861+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:07.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"261'96","reported_seq":594,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062299+0000","last_change":"2024-04-22T19:51:36.571445+0000","last_active":"2024-04-22T20:45:20.062299+0000","last_peered":"2024-04-22T20:45:20.062299+0000","last_clean":"2024-04-22T20:45:20.062299+0000","last_became_active":"2024-04-22T19:51:36.571175+0000","last_became_peered":"2024-04-22T19:51:36.571175+0000","last_unstale":"2024-04-22T20:45:20.062299+0000","last_undegraded":"2024-04-22T20:45:20.062299+0000","last_fullsized":"2024-04-22T20:45:20.062299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:18.054128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":221,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"261'131","reported_seq":708,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:14.252216+0000","last_change":"2024-04-22T19:51:28.509477+0000","last_active":"2024-04-22T20:45:14.252216+0000","last_peered":"2024-04-22T20:45:14.252216+0000","last_clean":"2024-04-22T20:45:14.252216+0000","last_became_active":"2024-04-22T19:51:28.509110+0000","last_became_peered":"2024-04-22T19:51:28.509110+0000","last_unstale":"2024-04-22T20:45:14.252216+0000","last_undegraded":"2024-04-22T20:45:14.252216+0000","last_fullsized":"2024-04-22T20:45:14.252216+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:10:52.641173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":519,"num_read_kb":420,"num_write":188,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"261'333","reported_seq":959,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:08.194615+0000","last_change":"2024-04-22T19:52:04.619673+0000","last_active":"2024-04-22T20:45:08.194615+0000","last_peered":"2024-04-22T20:45:08.194615+0000","last_clean":"2024-04-22T20:45:08.194615+0000","last_became_active":"2024-04-22T19:52:04.619385+0000","last_became_peered":"2024-04-22T19:52:04.619385+0000","last_unstale":"2024-04-22T20:45:08.194615+0000","last_undegraded":"2024-04-22T20:45:08.194615+0000","last_fullsized":"2024-04-22T20:45:08.194615+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:11.756964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8224737,"num_objects":76,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":472,"num_read_kb":20998,"num_write":1385,"num_write_kb":33571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.049743+0000","last_change":"2024-04-22T19:51:30.527064+0000","last_active":"2024-04-22T20:44:00.049743+0000","last_peered":"2024-04-22T20:44:00.049743+0000","last_clean":"2024-04-22T20:44:00.049743+0000","last_became_active":"2024-04-22T19:51:30.526469+0000","last_became_peered":"2024-04-22T19:51:30.526469+0000","last_unstale":"2024-04-22T20:44:00.049743+0000","last_undegraded":"2024-04-22T20:44:00.049743+0000","last_fullsized":"2024-04-22T20:44:00.049743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:45:04.946224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"261'301","reported_seq":880,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.204151+0000","last_change":"2024-04-22T19:51:32.540830+0000","last_active":"2024-04-22T20:47:20.204151+0000","last_peered":"2024-04-22T20:47:20.204151+0000","last_clean":"2024-04-22T20:47:20.204151+0000","last_became_active":"2024-04-22T19:51:32.540573+0000","last_became_peered":"2024-04-22T19:51:32.540573+0000","last_unstale":"2024-04-22T20:47:20.204151+0000","last_undegraded":"2024-04-22T20:47:20.204151+0000","last_fullsized":"2024-04-22T20:47:20.204151+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:49.905368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61803,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":483,"num_read_kb":946,"num_write":388,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.176362+0000","last_change":"2024-04-22T19:51:34.554124+0000","last_active":"2024-04-22T20:44:45.176362+0000","last_peered":"2024-04-22T20:44:45.176362+0000","last_clean":"2024-04-22T20:44:45.176362+0000","last_became_active":"2024-04-22T19:51:34.553857+0000","last_became_peered":"2024-04-22T19:51:34.553857+0000","last_unstale":"2024-04-22T20:44:45.176362+0000","last_undegraded":"2024-04-22T20:44:45.176362+0000","last_fullsized":"2024-04-22T20:44:45.176362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:38:56.150478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"261'153","reported_seq":654,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:15.079446+0000","last_change":"2024-04-22T19:51:36.575509+0000","last_active":"2024-04-22T20:47:15.079446+0000","last_peered":"2024-04-22T20:47:15.079446+0000","last_clean":"2024-04-22T20:47:15.079446+0000","last_became_active":"2024-04-22T19:51:36.575211+0000","last_became_peered":"2024-04-22T19:51:36.575211+0000","last_unstale":"2024-04-22T20:47:15.079446+0000","last_undegraded":"2024-04-22T20:47:15.079446+0000","last_fullsized":"2024-04-22T20:47:15.079446+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:46:54.630277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":158,"num_write":375,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"261'85","reported_seq":680,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.024455+0000","last_change":"2024-04-22T19:51:28.508354+0000","last_active":"2024-04-22T20:48:04.024455+0000","last_peered":"2024-04-22T20:48:04.024455+0000","last_clean":"2024-04-22T20:48:04.024455+0000","last_became_active":"2024-04-22T19:51:28.508145+0000","last_became_peered":"2024-04-22T19:51:28.508145+0000","last_unstale":"2024-04-22T20:48:04.024455+0000","last_undegraded":"2024-04-22T20:48:04.024455+0000","last_fullsized":"2024-04-22T20:48:04.024455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:54:32.701108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":376,"num_read_kb":320,"num_write":115,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"261'220","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174972+0000","last_change":"2024-04-22T19:51:26.504671+0000","last_active":"2024-04-22T20:44:00.174972+0000","last_peered":"2024-04-22T20:44:00.174972+0000","last_clean":"2024-04-22T20:44:00.174972+0000","last_became_active":"2024-04-22T19:51:26.504301+0000","last_became_peered":"2024-04-22T19:51:26.504301+0000","last_unstale":"2024-04-22T20:44:00.174972+0000","last_undegraded":"2024-04-22T20:44:00.174972+0000","last_fullsized":"2024-04-22T20:44:00.174972+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":200,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:56:36.267794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4884274,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":384,"num_read_kb":4553,"num_write":1039,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174938+0000","last_change":"2024-04-22T19:51:30.524895+0000","last_active":"2024-04-22T20:44:00.174938+0000","last_peered":"2024-04-22T20:44:00.174938+0000","last_clean":"2024-04-22T20:44:00.174938+0000","last_became_active":"2024-04-22T19:51:30.524417+0000","last_became_peered":"2024-04-22T19:51:30.524417+0000","last_unstale":"2024-04-22T20:44:00.174938+0000","last_undegraded":"2024-04-22T20:44:00.174938+0000","last_fullsized":"2024-04-22T20:44:00.174938+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:39:15.056196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"261'225","reported_seq":790,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.193293+0000","last_change":"2024-04-22T19:51:32.540923+0000","last_active":"2024-04-22T20:47:10.193293+0000","last_peered":"2024-04-22T20:47:10.193293+0000","last_clean":"2024-04-22T20:47:10.193293+0000","last_became_active":"2024-04-22T19:51:32.540659+0000","last_became_peered":"2024-04-22T19:51:32.540659+0000","last_unstale":"2024-04-22T20:47:10.193293+0000","last_undegraded":"2024-04-22T20:47:10.193293+0000","last_fullsized":"2024-04-22T20:47:10.193293+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:42:58.310584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51883,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":376,"num_read_kb":774,"num_write":298,"num_write_kb":236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309863+0000","last_change":"2024-04-22T19:51:34.552587+0000","last_active":"2024-04-22T20:44:50.309863+0000","last_peered":"2024-04-22T20:44:50.309863+0000","last_clean":"2024-04-22T20:44:50.309863+0000","last_became_active":"2024-04-22T19:51:34.552344+0000","last_became_peered":"2024-04-22T19:51:34.552344+0000","last_unstale":"2024-04-22T20:44:50.309863+0000","last_undegraded":"2024-04-22T20:44:50.309863+0000","last_fullsized":"2024-04-22T20:44:50.309863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:41.886857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"261'137","reported_seq":675,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:01.745300+0000","last_change":"2024-04-22T19:51:36.567945+0000","last_active":"2024-04-22T20:48:01.745300+0000","last_peered":"2024-04-22T20:48:01.745300+0000","last_clean":"2024-04-22T20:48:01.745300+0000","last_became_active":"2024-04-22T19:51:36.567594+0000","last_became_peered":"2024-04-22T19:51:36.567594+0000","last_unstale":"2024-04-22T20:48:01.745300+0000","last_undegraded":"2024-04-22T20:48:01.745300+0000","last_fullsized":"2024-04-22T20:48:01.745300+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:25:38.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":214,"num_write":335,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"261'87","reported_seq":648,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.151277+0000","last_change":"2024-04-22T19:51:28.519780+0000","last_active":"2024-04-22T20:44:45.151277+0000","last_peered":"2024-04-22T20:44:45.151277+0000","last_clean":"2024-04-22T20:44:45.151277+0000","last_became_active":"2024-04-22T19:51:28.519542+0000","last_became_peered":"2024-04-22T19:51:28.519542+0000","last_unstale":"2024-04-22T20:44:45.151277+0000","last_undegraded":"2024-04-22T20:44:45.151277+0000","last_fullsized":"2024-04-22T20:44:45.151277+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:24:55.634254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":347,"num_read_kb":277,"num_write":137,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"261'332","reported_seq":1176,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157148+0000","last_change":"2024-04-22T19:51:26.500077+0000","last_active":"2024-04-22T20:45:35.157148+0000","last_peered":"2024-04-22T20:45:35.157148+0000","last_clean":"2024-04-22T20:45:35.157148+0000","last_became_active":"2024-04-22T19:51:26.499390+0000","last_became_peered":"2024-04-22T19:51:26.499390+0000","last_unstale":"2024-04-22T20:45:35.157148+0000","last_undegraded":"2024-04-22T20:45:35.157148+0000","last_fullsized":"2024-04-22T20:45:35.157148+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:40:50.316762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24547368,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":977,"num_read_kb":88369,"num_write":1347,"num_write_kb":41583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172368+0000","last_change":"2024-04-22T19:51:30.526678+0000","last_active":"2024-04-22T20:44:40.172368+0000","last_peered":"2024-04-22T20:44:40.172368+0000","last_clean":"2024-04-22T20:44:40.172368+0000","last_became_active":"2024-04-22T19:51:30.526183+0000","last_became_peered":"2024-04-22T19:51:30.526183+0000","last_unstale":"2024-04-22T20:44:40.172368+0000","last_undegraded":"2024-04-22T20:44:40.172368+0000","last_fullsized":"2024-04-22T20:44:40.172368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:40.232417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"261'243","reported_seq":802,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.164215+0000","last_change":"2024-04-22T19:51:32.538965+0000","last_active":"2024-04-22T20:46:35.164215+0000","last_peered":"2024-04-22T20:46:35.164215+0000","last_clean":"2024-04-22T20:46:35.164215+0000","last_became_active":"2024-04-22T19:51:32.538644+0000","last_became_peered":"2024-04-22T19:51:32.538644+0000","last_unstale":"2024-04-22T20:46:35.164215+0000","last_undegraded":"2024-04-22T20:46:35.164215+0000","last_fullsized":"2024-04-22T20:46:35.164215+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:18:02.949702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52954,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":389,"num_read_kb":836,"num_write":326,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":486,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.167484+0000","last_change":"2024-04-22T19:51:34.551365+0000","last_active":"2024-04-22T20:44:05.167484+0000","last_peered":"2024-04-22T20:44:05.167484+0000","last_clean":"2024-04-22T20:44:05.167484+0000","last_became_active":"2024-04-22T19:51:34.551085+0000","last_became_peered":"2024-04-22T19:51:34.551085+0000","last_unstale":"2024-04-22T20:44:05.167484+0000","last_undegraded":"2024-04-22T20:44:05.167484+0000","last_fullsized":"2024-04-22T20:44:05.167484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:50:23.004035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"261'179","reported_seq":903,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.372427+0000","last_change":"2024-04-22T19:51:36.569537+0000","last_active":"2024-04-22T20:48:00.372427+0000","last_peered":"2024-04-22T20:48:00.372427+0000","last_clean":"2024-04-22T20:48:00.372427+0000","last_became_active":"2024-04-22T19:51:36.569234+0000","last_became_peered":"2024-04-22T19:51:36.569234+0000","last_unstale":"2024-04-22T20:48:00.372427+0000","last_undegraded":"2024-04-22T20:48:00.372427+0000","last_fullsized":"2024-04-22T20:48:00.372427+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:56:31.796866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":419,"num_read_kb":454,"num_write":429,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"261'180","reported_seq":750,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172395+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:44:40.172395+0000","last_peered":"2024-04-22T20:44:40.172395+0000","last_clean":"2024-04-22T20:44:40.172395+0000","last_became_active":"2024-04-22T19:51:28.515506+0000","last_became_peered":"2024-04-22T19:51:28.515506+0000","last_unstale":"2024-04-22T20:44:40.172395+0000","last_undegraded":"2024-04-22T20:44:40.172395+0000","last_fullsized":"2024-04-22T20:44:40.172395+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:14:29.010830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":678,"num_read_kb":529,"num_write":279,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"261'227","reported_seq":839,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.881098+0000","last_change":"2024-04-22T19:51:26.504948+0000","last_active":"2024-04-22T20:46:41.881098+0000","last_peered":"2024-04-22T20:46:41.881098+0000","last_clean":"2024-04-22T20:46:41.881098+0000","last_became_active":"2024-04-22T19:51:26.504604+0000","last_became_peered":"2024-04-22T19:51:26.504604+0000","last_unstale":"2024-04-22T20:46:41.881098+0000","last_undegraded":"2024-04-22T20:46:41.881098+0000","last_fullsized":"2024-04-22T20:46:41.881098+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":200,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:11.092305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11653248,"num_objects":79,"num_object_clones":0,"num_object_copies":237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":302,"num_read_kb":9007,"num_write":882,"num_write_kb":46630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"25'1","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040244+0000","last_change":"2024-04-22T19:51:30.523630+0000","last_active":"2024-04-22T20:44:50.040244+0000","last_peered":"2024-04-22T20:44:50.040244+0000","last_clean":"2024-04-22T20:44:50.040244+0000","last_became_active":"2024-04-22T19:51:30.523338+0000","last_became_peered":"2024-04-22T19:51:30.523338+0000","last_unstale":"2024-04-22T20:44:50.040244+0000","last_undegraded":"2024-04-22T20:44:50.040244+0000","last_fullsized":"2024-04-22T20:44:50.040244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:00.508499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"261'305","reported_seq":885,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.188922+0000","last_change":"2024-04-22T19:51:32.537701+0000","last_active":"2024-04-22T20:46:35.188922+0000","last_peered":"2024-04-22T20:46:35.188922+0000","last_clean":"2024-04-22T20:46:35.188922+0000","last_became_active":"2024-04-22T19:51:32.537373+0000","last_became_peered":"2024-04-22T19:51:32.537373+0000","last_unstale":"2024-04-22T20:46:35.188922+0000","last_undegraded":"2024-04-22T20:46:35.188922+0000","last_fullsized":"2024-04-22T20:46:35.188922+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:43:27.042829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59361,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":482,"num_read_kb":951,"num_write":388,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":486,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.171634+0000","last_change":"2024-04-22T19:51:34.555960+0000","last_active":"2024-04-22T20:44:05.171634+0000","last_peered":"2024-04-22T20:44:05.171634+0000","last_clean":"2024-04-22T20:44:05.171634+0000","last_became_active":"2024-04-22T19:51:34.555709+0000","last_became_peered":"2024-04-22T19:51:34.555709+0000","last_unstale":"2024-04-22T20:44:05.171634+0000","last_undegraded":"2024-04-22T20:44:05.171634+0000","last_fullsized":"2024-04-22T20:44:05.171634+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:02.499776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"261'144","reported_seq":628,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:32.358361+0000","last_change":"2024-04-22T19:51:36.574316+0000","last_active":"2024-04-22T20:46:32.358361+0000","last_peered":"2024-04-22T20:46:32.358361+0000","last_clean":"2024-04-22T20:46:32.358361+0000","last_became_active":"2024-04-22T19:51:36.574012+0000","last_became_peered":"2024-04-22T19:51:36.574012+0000","last_unstale":"2024-04-22T20:46:32.358361+0000","last_undegraded":"2024-04-22T20:46:32.358361+0000","last_fullsized":"2024-04-22T20:46:32.358361+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:01:01.700368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":109,"num_write":348,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"261'372","reported_seq":1037,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045977+0000","last_change":"2024-04-22T19:51:28.518589+0000","last_active":"2024-04-22T20:45:40.045977+0000","last_peered":"2024-04-22T20:45:40.045977+0000","last_clean":"2024-04-22T20:45:40.045977+0000","last_became_active":"2024-04-22T19:51:28.518339+0000","last_became_peered":"2024-04-22T19:51:28.518339+0000","last_unstale":"2024-04-22T20:45:40.045977+0000","last_undegraded":"2024-04-22T20:45:40.045977+0000","last_fullsized":"2024-04-22T20:45:40.045977+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:15:44.877848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1526,"num_read_kb":1211,"num_write":619,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"261'252","reported_seq":951,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.316997+0000","last_change":"2024-04-22T19:51:26.503151+0000","last_active":"2024-04-22T20:45:40.316997+0000","last_peered":"2024-04-22T20:45:40.316997+0000","last_clean":"2024-04-22T20:45:40.316997+0000","last_became_active":"2024-04-22T19:51:26.502773+0000","last_became_peered":"2024-04-22T19:51:26.502773+0000","last_unstale":"2024-04-22T20:45:40.316997+0000","last_undegraded":"2024-04-22T20:45:40.316997+0000","last_fullsized":"2024-04-22T20:45:40.316997+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:34:49.559184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6843432,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":522,"num_read_kb":14380,"num_write":1027,"num_write_kb":17858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"261'524","reported_seq":1286,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.175644+0000","last_change":"2024-04-22T19:52:04.617284+0000","last_active":"2024-04-22T20:44:05.175644+0000","last_peered":"2024-04-22T20:44:05.175644+0000","last_clean":"2024-04-22T20:44:05.175644+0000","last_became_active":"2024-04-22T19:52:04.617036+0000","last_became_peered":"2024-04-22T19:52:04.617036+0000","last_unstale":"2024-04-22T20:44:05.175644+0000","last_undegraded":"2024-04-22T20:44:05.175644+0000","last_fullsized":"2024-04-22T20:44:05.175644+0000","mapping_epoch":33,"log_start":"261'500","ondisk_log_start":"261'500","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":500,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:59:14.488482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2230,"num_read_kb":1847,"num_write":935,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"261'226","reported_seq":865,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:43.145525+0000","last_change":"2024-04-22T19:51:26.496262+0000","last_active":"2024-04-22T20:46:43.145525+0000","last_peered":"2024-04-22T20:46:43.145525+0000","last_clean":"2024-04-22T20:46:43.145525+0000","last_became_active":"2024-04-22T19:51:26.495948+0000","last_became_peered":"2024-04-22T19:51:26.495948+0000","last_unstale":"2024-04-22T20:46:43.145525+0000","last_undegraded":"2024-04-22T20:46:43.145525+0000","last_fullsized":"2024-04-22T20:46:43.145525+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":200,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:25.588593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1295176,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":394,"num_read_kb":10115,"num_write":945,"num_write_kb":22835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"261'208","reported_seq":841,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189450+0000","last_change":"2024-04-22T19:51:28.513290+0000","last_active":"2024-04-22T20:45:40.189450+0000","last_peered":"2024-04-22T20:45:40.189450+0000","last_clean":"2024-04-22T20:45:40.189450+0000","last_became_active":"2024-04-22T19:51:28.513026+0000","last_became_peered":"2024-04-22T19:51:28.513026+0000","last_unstale":"2024-04-22T20:45:40.189450+0000","last_undegraded":"2024-04-22T20:45:40.189450+0000","last_fullsized":"2024-04-22T20:45:40.189450+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:09:31.873692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":827,"num_read_kb":665,"num_write":342,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"261'251","reported_seq":870,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:43.136633+0000","last_change":"2024-04-22T19:51:26.507266+0000","last_active":"2024-04-22T20:46:43.136633+0000","last_peered":"2024-04-22T20:46:43.136633+0000","last_clean":"2024-04-22T20:46:43.136633+0000","last_became_active":"2024-04-22T19:51:26.506748+0000","last_became_peered":"2024-04-22T19:51:26.506748+0000","last_unstale":"2024-04-22T20:46:43.136633+0000","last_undegraded":"2024-04-22T20:46:43.136633+0000","last_fullsized":"2024-04-22T20:46:43.136633+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:15.321922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6641638,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":354,"num_read_kb":12674,"num_write":1040,"num_write_kb":41347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"261'4175","reported_seq":4791,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:29.357349+0000","last_change":"2024-04-22T19:51:28.511560+0000","last_active":"2024-04-22T20:46:29.357349+0000","last_peered":"2024-04-22T20:46:29.357349+0000","last_clean":"2024-04-22T20:46:29.357349+0000","last_became_active":"2024-04-22T19:51:28.511312+0000","last_became_peered":"2024-04-22T19:51:28.511312+0000","last_unstale":"2024-04-22T20:46:29.357349+0000","last_undegraded":"2024-04-22T20:46:29.357349+0000","last_fullsized":"2024-04-22T20:46:29.357349+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":3000,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:28:56.675918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14718,"num_read_kb":11919,"num_write":6278,"num_write_kb":3133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"261'248","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.060540+0000","last_change":"2024-04-22T19:52:05.115180+0000","last_active":"2024-04-22T20:44:20.060540+0000","last_peered":"2024-04-22T20:44:20.060540+0000","last_clean":"2024-04-22T20:44:20.060540+0000","last_became_active":"2024-04-22T19:52:05.114734+0000","last_became_peered":"2024-04-22T19:52:05.114734+0000","last_unstale":"2024-04-22T20:44:20.060540+0000","last_undegraded":"2024-04-22T20:44:20.060540+0000","last_fullsized":"2024-04-22T20:44:20.060540+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:08:25.577231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11535134,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":357,"num_read_kb":5518,"num_write":1024,"num_write_kb":21667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.23","version":"261'170","reported_seq":851,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.006667+0000","last_change":"2024-04-22T19:51:28.516445+0000","last_active":"2024-04-22T20:48:03.006667+0000","last_peered":"2024-04-22T20:48:03.006667+0000","last_clean":"2024-04-22T20:48:03.006667+0000","last_became_active":"2024-04-22T19:51:28.516125+0000","last_became_peered":"2024-04-22T19:51:28.516125+0000","last_unstale":"2024-04-22T20:48:03.006667+0000","last_undegraded":"2024-04-22T20:48:03.006667+0000","last_fullsized":"2024-04-22T20:48:03.006667+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:01:29.663998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":741,"num_read_kb":616,"num_write":243,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"261'299","reported_seq":945,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.135884+0000","last_change":"2024-04-22T19:52:04.619831+0000","last_active":"2024-04-22T20:47:38.135884+0000","last_peered":"2024-04-22T20:47:38.135884+0000","last_clean":"2024-04-22T20:47:38.135884+0000","last_became_active":"2024-04-22T19:52:04.619286+0000","last_became_peered":"2024-04-22T19:52:04.619286+0000","last_unstale":"2024-04-22T20:47:38.135884+0000","last_undegraded":"2024-04-22T20:47:38.135884+0000","last_fullsized":"2024-04-22T20:47:38.135884+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":200,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:58.235831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10538009,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":519,"num_read_kb":69891,"num_write":1172,"num_write_kb":36151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"261'122","reported_seq":726,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181359+0000","last_change":"2024-04-22T19:51:28.516690+0000","last_active":"2024-04-22T20:45:40.181359+0000","last_peered":"2024-04-22T20:45:40.181359+0000","last_clean":"2024-04-22T20:45:40.181359+0000","last_became_active":"2024-04-22T19:51:28.516437+0000","last_became_peered":"2024-04-22T19:51:28.516437+0000","last_unstale":"2024-04-22T20:45:40.181359+0000","last_undegraded":"2024-04-22T20:45:40.181359+0000","last_fullsized":"2024-04-22T20:45:40.181359+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:44.084365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":423,"num_write":178,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"261'257","reported_seq":991,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045950+0000","last_change":"2024-04-22T19:51:26.498349+0000","last_active":"2024-04-22T20:45:40.045950+0000","last_peered":"2024-04-22T20:45:40.045950+0000","last_clean":"2024-04-22T20:45:40.045950+0000","last_became_active":"2024-04-22T19:51:26.498039+0000","last_became_peered":"2024-04-22T19:51:26.498039+0000","last_unstale":"2024-04-22T20:45:40.045950+0000","last_undegraded":"2024-04-22T20:45:40.045950+0000","last_fullsized":"2024-04-22T20:45:40.045950+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:17.714586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14707922,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":706,"num_read_kb":22382,"num_write":1193,"num_write_kb":18139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"261'199","reported_seq":848,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:49.466099+0000","last_change":"2024-04-22T19:51:28.520853+0000","last_active":"2024-04-22T20:46:49.466099+0000","last_peered":"2024-04-22T20:46:49.466099+0000","last_clean":"2024-04-22T20:46:49.466099+0000","last_became_active":"2024-04-22T19:51:28.520612+0000","last_became_peered":"2024-04-22T19:51:28.520612+0000","last_unstale":"2024-04-22T20:46:49.466099+0000","last_undegraded":"2024-04-22T20:46:49.466099+0000","last_fullsized":"2024-04-22T20:46:49.466099+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:46:57.197381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":735,"num_read_kb":593,"num_write":262,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"261'291","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317145+0000","last_change":"2024-04-22T19:51:26.499154+0000","last_active":"2024-04-22T20:45:40.317145+0000","last_peered":"2024-04-22T20:45:40.317145+0000","last_clean":"2024-04-22T20:45:40.317145+0000","last_became_active":"2024-04-22T19:51:26.498845+0000","last_became_peered":"2024-04-22T19:51:26.498845+0000","last_unstale":"2024-04-22T20:45:40.317145+0000","last_undegraded":"2024-04-22T20:45:40.317145+0000","last_fullsized":"2024-04-22T20:45:40.317145+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":200,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:50.165203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7476623,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":432,"num_read_kb":33253,"num_write":1174,"num_write_kb":30860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"261'133","reported_seq":766,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317028+0000","last_change":"2024-04-22T19:51:28.520128+0000","last_active":"2024-04-22T20:45:40.317028+0000","last_peered":"2024-04-22T20:45:40.317028+0000","last_clean":"2024-04-22T20:45:40.317028+0000","last_became_active":"2024-04-22T19:51:28.519669+0000","last_became_peered":"2024-04-22T19:51:28.519669+0000","last_unstale":"2024-04-22T20:45:40.317028+0000","last_undegraded":"2024-04-22T20:45:40.317028+0000","last_fullsized":"2024-04-22T20:45:40.317028+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:18.121387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":560,"num_read_kb":466,"num_write":196,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"261'257","reported_seq":852,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:07.068523+0000","last_change":"2024-04-22T19:52:04.618077+0000","last_active":"2024-04-22T20:44:07.068523+0000","last_peered":"2024-04-22T20:44:07.068523+0000","last_clean":"2024-04-22T20:44:07.068523+0000","last_became_active":"2024-04-22T19:52:04.617649+0000","last_became_peered":"2024-04-22T19:52:04.617649+0000","last_unstale":"2024-04-22T20:44:07.068523+0000","last_undegraded":"2024-04-22T20:44:07.068523+0000","last_fullsized":"2024-04-22T20:44:07.068523+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:58.404399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587123,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":386,"num_read_kb":29068,"num_write":962,"num_write_kb":22627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"261'128","reported_seq":709,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.179387+0000","last_change":"2024-04-22T19:51:28.521764+0000","last_active":"2024-04-22T20:45:10.179387+0000","last_peered":"2024-04-22T20:45:10.179387+0000","last_clean":"2024-04-22T20:45:10.179387+0000","last_became_active":"2024-04-22T19:51:28.521516+0000","last_became_peered":"2024-04-22T19:51:28.521516+0000","last_unstale":"2024-04-22T20:45:10.179387+0000","last_undegraded":"2024-04-22T20:45:10.179387+0000","last_fullsized":"2024-04-22T20:45:10.179387+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:13:11.122213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":542,"num_read_kb":442,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"261'278","reported_seq":958,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:52.078309+0000","last_change":"2024-04-22T19:51:26.510292+0000","last_active":"2024-04-22T20:46:52.078309+0000","last_peered":"2024-04-22T20:46:52.078309+0000","last_clean":"2024-04-22T20:46:52.078309+0000","last_became_active":"2024-04-22T19:51:26.507843+0000","last_became_peered":"2024-04-22T19:51:26.507843+0000","last_unstale":"2024-04-22T20:46:52.078309+0000","last_undegraded":"2024-04-22T20:46:52.078309+0000","last_fullsized":"2024-04-22T20:46:52.078309+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":200,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:43:15.698499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10798875,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":447,"num_read_kb":136369,"num_write":1173,"num_write_kb":55788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"261'235","reported_seq":860,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176799+0000","last_change":"2024-04-22T19:51:28.515473+0000","last_active":"2024-04-22T20:45:10.176799+0000","last_peered":"2024-04-22T20:45:10.176799+0000","last_clean":"2024-04-22T20:45:10.176799+0000","last_became_active":"2024-04-22T19:51:28.515164+0000","last_became_peered":"2024-04-22T19:51:28.515164+0000","last_unstale":"2024-04-22T20:45:10.176799+0000","last_undegraded":"2024-04-22T20:45:10.176799+0000","last_fullsized":"2024-04-22T20:45:10.176799+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:38:12.199672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":950,"num_read_kb":750,"num_write":364,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"261'436","reported_seq":1136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.156482+0000","last_change":"2024-04-22T19:51:26.494217+0000","last_active":"2024-04-22T20:45:30.156482+0000","last_peered":"2024-04-22T20:45:30.156482+0000","last_clean":"2024-04-22T20:45:30.156482+0000","last_became_active":"2024-04-22T19:51:26.493860+0000","last_became_peered":"2024-04-22T19:51:26.493860+0000","last_unstale":"2024-04-22T20:45:30.156482+0000","last_undegraded":"2024-04-22T20:45:30.156482+0000","last_fullsized":"2024-04-22T20:45:30.156482+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":400,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:56:54.837990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6300944,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":885,"num_read_kb":13872,"num_write":1373,"num_write_kb":37693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"261'251","reported_seq":929,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:37.822141+0000","last_change":"2024-04-22T19:51:28.515833+0000","last_active":"2024-04-22T20:46:37.822141+0000","last_peered":"2024-04-22T20:46:37.822141+0000","last_clean":"2024-04-22T20:46:37.822141+0000","last_became_active":"2024-04-22T19:51:28.515547+0000","last_became_peered":"2024-04-22T19:51:28.515547+0000","last_unstale":"2024-04-22T20:46:37.822141+0000","last_undegraded":"2024-04-22T20:46:37.822141+0000","last_fullsized":"2024-04-22T20:46:37.822141+0000","mapping_epoch":22,"log_start":"247'200","ondisk_log_start":"247'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:00:39.299916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1120,"num_read_kb":973,"num_write":365,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"261'258","reported_seq":888,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.187879+0000","last_change":"2024-04-22T19:51:26.500937+0000","last_active":"2024-04-22T20:45:30.187879+0000","last_peered":"2024-04-22T20:45:30.187879+0000","last_clean":"2024-04-22T20:45:30.187879+0000","last_became_active":"2024-04-22T19:51:26.500566+0000","last_became_peered":"2024-04-22T19:51:26.500566+0000","last_unstale":"2024-04-22T20:45:30.187879+0000","last_undegraded":"2024-04-22T20:45:30.187879+0000","last_fullsized":"2024-04-22T20:45:30.187879+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:25.795560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4539865,"num_objects":73,"num_object_clones":0,"num_object_copies":219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":358,"num_read_kb":45844,"num_write":1054,"num_write_kb":42819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"261'152","reported_seq":750,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:42.563356+0000","last_change":"2024-04-22T19:51:28.516044+0000","last_active":"2024-04-22T20:46:42.563356+0000","last_peered":"2024-04-22T20:46:42.563356+0000","last_clean":"2024-04-22T20:46:42.563356+0000","last_became_active":"2024-04-22T19:51:28.515764+0000","last_became_peered":"2024-04-22T19:51:28.515764+0000","last_unstale":"2024-04-22T20:46:42.563356+0000","last_undegraded":"2024-04-22T20:46:42.563356+0000","last_fullsized":"2024-04-22T20:46:42.563356+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:31.975652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":633,"num_read_kb":515,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"261'212","reported_seq":821,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181737+0000","last_change":"2024-04-22T19:51:26.512810+0000","last_active":"2024-04-22T20:45:30.181737+0000","last_peered":"2024-04-22T20:45:30.181737+0000","last_clean":"2024-04-22T20:45:30.181737+0000","last_became_active":"2024-04-22T19:51:26.512467+0000","last_became_peered":"2024-04-22T19:51:26.512467+0000","last_unstale":"2024-04-22T20:45:30.181737+0000","last_undegraded":"2024-04-22T20:45:30.181737+0000","last_fullsized":"2024-04-22T20:45:30.181737+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:04.495670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12585759,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":328,"num_read_kb":17902,"num_write":889,"num_write_kb":43703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"261'27","reported_seq":554,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.063941+0000","last_change":"2024-04-22T19:51:28.510362+0000","last_active":"2024-04-22T20:45:30.063941+0000","last_peered":"2024-04-22T20:45:30.063941+0000","last_clean":"2024-04-22T20:45:30.063941+0000","last_became_active":"2024-04-22T19:51:28.510091+0000","last_became_peered":"2024-04-22T19:51:28.510091+0000","last_unstale":"2024-04-22T20:45:30.063941+0000","last_undegraded":"2024-04-22T20:45:30.063941+0000","last_fullsized":"2024-04-22T20:45:30.063941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:48.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":95,"num_write":39,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"261'277","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:11.047810+0000","last_change":"2024-04-22T19:51:26.502070+0000","last_active":"2024-04-22T20:47:11.047810+0000","last_peered":"2024-04-22T20:47:11.047810+0000","last_clean":"2024-04-22T20:47:11.047810+0000","last_became_active":"2024-04-22T19:51:26.501700+0000","last_became_peered":"2024-04-22T19:51:26.501700+0000","last_unstale":"2024-04-22T20:47:11.047810+0000","last_undegraded":"2024-04-22T20:47:11.047810+0000","last_fullsized":"2024-04-22T20:47:11.047810+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":200,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:26:11.753132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4607139,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":443,"num_read_kb":6050,"num_write":1178,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"261'54","reported_seq":603,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181693+0000","last_change":"2024-04-22T19:51:28.509002+0000","last_active":"2024-04-22T20:45:30.181693+0000","last_peered":"2024-04-22T20:45:30.181693+0000","last_clean":"2024-04-22T20:45:30.181693+0000","last_became_active":"2024-04-22T19:51:28.508768+0000","last_became_peered":"2024-04-22T19:51:28.508768+0000","last_unstale":"2024-04-22T20:45:30.181693+0000","last_undegraded":"2024-04-22T20:45:30.181693+0000","last_fullsized":"2024-04-22T20:45:30.181693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:32:52.855073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":238,"num_read_kb":196,"num_write":80,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"261'285","reported_seq":924,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:44.477365+0000","last_change":"2024-04-22T19:51:26.497503+0000","last_active":"2024-04-22T20:47:44.477365+0000","last_peered":"2024-04-22T20:47:44.477365+0000","last_clean":"2024-04-22T20:47:44.477365+0000","last_became_active":"2024-04-22T19:51:26.497156+0000","last_became_peered":"2024-04-22T19:51:26.497156+0000","last_unstale":"2024-04-22T20:47:44.477365+0000","last_undegraded":"2024-04-22T20:47:44.477365+0000","last_fullsized":"2024-04-22T20:47:44.477365+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:23:31.844540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15135500,"num_objects":78,"num_object_clones":0,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":466,"num_read_kb":35895,"num_write":1174,"num_write_kb":35286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"261'4050","reported_seq":4612,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.156504+0000","last_change":"2024-04-22T19:51:28.515076+0000","last_active":"2024-04-22T20:45:30.156504+0000","last_peered":"2024-04-22T20:45:30.156504+0000","last_clean":"2024-04-22T20:45:30.156504+0000","last_became_active":"2024-04-22T19:51:28.514780+0000","last_became_peered":"2024-04-22T19:51:28.514780+0000","last_unstale":"2024-04-22T20:45:30.156504+0000","last_undegraded":"2024-04-22T20:45:30.156504+0000","last_fullsized":"2024-04-22T20:45:30.156504+0000","mapping_epoch":22,"log_start":"261'4000","ondisk_log_start":"261'4000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":3000,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:22:22.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14224,"num_read_kb":11535,"num_write":6070,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"261'243","reported_seq":1002,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.065500+0000","last_change":"2024-04-22T19:51:26.496549+0000","last_active":"2024-04-22T20:45:10.065500+0000","last_peered":"2024-04-22T20:45:10.065500+0000","last_clean":"2024-04-22T20:45:10.065500+0000","last_became_active":"2024-04-22T19:51:26.496214+0000","last_became_peered":"2024-04-22T19:51:26.496214+0000","last_unstale":"2024-04-22T20:45:10.065500+0000","last_undegraded":"2024-04-22T20:45:10.065500+0000","last_fullsized":"2024-04-22T20:45:10.065500+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":200,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:15:54.981915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6811717,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":525,"num_read_kb":13796,"num_write":1122,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"261'298","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.067136+0000","last_change":"2024-04-22T19:51:28.508937+0000","last_active":"2024-04-22T20:45:50.067136+0000","last_peered":"2024-04-22T20:45:50.067136+0000","last_clean":"2024-04-22T20:45:50.067136+0000","last_became_active":"2024-04-22T19:51:28.508681+0000","last_became_peered":"2024-04-22T19:51:28.508681+0000","last_unstale":"2024-04-22T20:45:50.067136+0000","last_undegraded":"2024-04-22T20:45:50.067136+0000","last_fullsized":"2024-04-22T20:45:50.067136+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:21:44.423395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1231,"num_read_kb":1000,"num_write":489,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"261'215","reported_seq":824,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.184935+0000","last_change":"2024-04-22T19:51:26.505139+0000","last_active":"2024-04-22T20:45:10.184935+0000","last_peered":"2024-04-22T20:45:10.184935+0000","last_clean":"2024-04-22T20:45:10.184935+0000","last_became_active":"2024-04-22T19:51:26.502438+0000","last_became_peered":"2024-04-22T19:51:26.502438+0000","last_unstale":"2024-04-22T20:45:10.184935+0000","last_undegraded":"2024-04-22T20:45:10.184935+0000","last_fullsized":"2024-04-22T20:45:10.184935+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":200,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:27.926512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13702713,"num_objects":72,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":319,"num_read_kb":36636,"num_write":870,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"261'300","reported_seq":1179,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:52.068642+0000","last_change":"2024-04-22T19:51:26.505871+0000","last_active":"2024-04-22T20:46:52.068642+0000","last_peered":"2024-04-22T20:46:52.068642+0000","last_clean":"2024-04-22T20:46:52.068642+0000","last_became_active":"2024-04-22T19:51:26.504878+0000","last_became_peered":"2024-04-22T19:51:26.504878+0000","last_unstale":"2024-04-22T20:46:52.068642+0000","last_undegraded":"2024-04-22T20:46:52.068642+0000","last_fullsized":"2024-04-22T20:46:52.068642+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":200,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:57.296051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787489,"num_objects":82,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":944,"num_read_kb":55238,"num_write":1333,"num_write_kb":52838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"261'127","reported_seq":728,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.177008+0000","last_change":"2024-04-22T19:51:28.521641+0000","last_active":"2024-04-22T20:44:50.177008+0000","last_peered":"2024-04-22T20:44:50.177008+0000","last_clean":"2024-04-22T20:44:50.177008+0000","last_became_active":"2024-04-22T19:51:28.521372+0000","last_became_peered":"2024-04-22T19:51:28.521372+0000","last_unstale":"2024-04-22T20:44:50.177008+0000","last_undegraded":"2024-04-22T20:44:50.177008+0000","last_fullsized":"2024-04-22T20:44:50.177008+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:16.451481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":525,"num_read_kb":437,"num_write":198,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"261'224","reported_seq":878,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:08.497364+0000","last_change":"2024-04-22T19:52:04.618718+0000","last_active":"2024-04-22T20:47:08.497364+0000","last_peered":"2024-04-22T20:47:08.497364+0000","last_clean":"2024-04-22T20:47:08.497364+0000","last_became_active":"2024-04-22T19:52:04.618273+0000","last_became_peered":"2024-04-22T19:52:04.618273+0000","last_unstale":"2024-04-22T20:47:08.497364+0000","last_undegraded":"2024-04-22T20:47:08.497364+0000","last_fullsized":"2024-04-22T20:47:08.497364+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":200,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:09.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19315969,"num_objects":70,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":402,"num_read_kb":73466,"num_write":926,"num_write_kb":40708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"261'95","reported_seq":648,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180705+0000","last_change":"2024-04-22T19:51:28.513718+0000","last_active":"2024-04-22T20:44:40.180705+0000","last_peered":"2024-04-22T20:44:40.180705+0000","last_clean":"2024-04-22T20:44:40.180705+0000","last_became_active":"2024-04-22T19:51:28.513450+0000","last_became_peered":"2024-04-22T19:51:28.513450+0000","last_unstale":"2024-04-22T20:44:40.180705+0000","last_undegraded":"2024-04-22T20:44:40.180705+0000","last_fullsized":"2024-04-22T20:44:40.180705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:50.937435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":382,"num_read_kb":302,"num_write":152,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5385,"num_read_kb":3813,"num_write":2124,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1241,"ondisk_log_size":1241,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19082,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":13196,"num_read_kb":13004,"num_write":15956,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":38164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1853,"ondisk_log_size":1853,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1817836,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":16703,"num_read_kb":31506,"num_write":14213,"num_write_kb":11110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3293184,"data_stored":2776426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2244,"ondisk_log_size":2244,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1617,"num_read_kb":1920,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":85960,"num_read_kb":69763,"num_write":34673,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3889,"ondisk_log_size":3889,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":684407196,"num_objects":4507,"num_object_clones":0,"num_object_copies":13521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":30977,"num_read_kb":2216973,"num_write":73272,"num_write_kb":2138644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1075630080,"data_stored":1075630080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3665,"ondisk_log_size":3665,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":91317873,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":141,"num_read_kb":196064,"num_write":151,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182640640,"data_stored":182635746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":91317873,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":139,"num_read_kb":185784,"num_write":151,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":182640640,"data_stored":182635746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575536,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461168,"kb_used_data":181480,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":93910672,"statfs":{"total":96636764160,"available":96164528128,"internally_reserved":0,"allocated":185835520,"data_stored":183817698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":286390730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575538,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424572,"kb_used_data":162932,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":93947268,"statfs":{"total":96636764160,"available":96202002432,"internally_reserved":0,"allocated":166842368,"data_stored":164891323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":267909574},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575537,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308012,"kb_used_data":178340,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94063828,"statfs":{"total":96636764160,"available":96321359872,"internally_reserved":0,"allocated":182620160,"data_stored":180552293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575538,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":252084,"kb_used_data":166764,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94119756,"statfs":{"total":96636764160,"available":96378630144,"internally_reserved":0,"allocated":170766336,"data_stored":168845633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575538,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444004,"kb_used_data":196316,"kb_used_omap":1,"kb_used_meta":247678,"kb_avail":93927836,"statfs":{"total":96636764160,"available":96182104064,"internally_reserved":0,"allocated":201027584,"data_stored":199033946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":253622728},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575539,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":264928,"kb_used_data":171352,"kb_used_omap":1,"kb_used_meta":93566,"kb_avail":94106912,"statfs":{"total":96636764160,"available":96365477888,"internally_reserved":0,"allocated":175464448,"data_stored":173547104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95812037},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575536,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330400,"kb_used_data":186840,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94041440,"statfs":{"total":96636764160,"available":96298434560,"internally_reserved":0,"allocated":191324160,"data_stored":189350803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575538,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434020,"kb_used_data":200156,"kb_used_omap":1,"kb_used_meta":233854,"kb_avail":93937820,"statfs":{"total":96636764160,"available":96192327680,"internally_reserved":0,"allocated":204959744,"data_stored":202964756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":239466950},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":147820544,"data_stored":147820544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139972608,"data_stored":139972608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105086976,"data_stored":105086976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":149049344,"data_stored":149049344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":120315904,"data_stored":120315904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":143065088,"data_stored":143065088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":133967872,"data_stored":133967872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136351744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":353680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":383474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":132055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":479987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":85199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":566114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":248456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":622592,"data_stored":527461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":5948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":4195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":3595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":6129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":4904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28405760,"data_stored":28403313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":26214400,"data_stored":26214400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28311552,"data_stored":28311552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":23068672,"data_stored":23068672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22020096,"data_stored":22020096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":18923520,"data_stored":18922784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10530816,"data_stored":10529105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24117248,"data_stored":24117248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19922944,"data_stored":19922944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37748736,"data_stored":37748736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24170496,"data_stored":24168864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24158208,"data_stored":24157393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":15728640,"data_stored":15728640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17879040,"data_stored":17877408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18915328,"data_stored":18914513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T20:48:20.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-22T20:48:20.558 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:20.559 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":261,"fsid":"b2f95cfa-7b26-4a63-b13e-7105b011d9c4","created":"2024-04-22T19:50:58.750498+0000","modified":"2024-04-22T20:19:17.783552+0000","last_up_change":"2024-04-22T19:51:09.418727+0000","last_in_change":"2024-04-22T19:51:01.138291+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":15,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":68,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-22T19:51:09.800954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-22T19:51:10.966181+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-04-22T19:51:24.469002+0000","flags":1,"flags_names":"hashpspool","type":3,"size":3,"min_size":2,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"client.0","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-04-22T19:51:26.804502+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-04-22T19:51:28.630054+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-04-22T19:51:31.193244+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-04-22T19:51:32.540653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-04-22T19:51:34.625821+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":65,"pool_name":"default.rgw.buckets.non-ec","create_time":"2024-04-22T20:01:03.169847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":66,"pool_name":"default.rgw.buckets.data.frozen","create_time":"2024-04-22T20:05:12.478390+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":67,"pool_name":"default.rgw.buckets.data.lukewarm","create_time":"2024-04-22T20:05:16.721698+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"256","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":68,"pool_name":"default.rgw.otp","create_time":"2024-04-22T20:19:15.115668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"261","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8c300310-767d-4fee-84ea-37d38622ac19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":259,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6801","nonce":2747506017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6803","nonce":2747506017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6807","nonce":2747506017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2747506017},{"type":"v1","addr":"172.21.15.136:6805","nonce":2747506017}]},"public_addr":"172.21.15.136:6801/2747506017","cluster_addr":"172.21.15.136:6803/2747506017","heartbeat_back_addr":"172.21.15.136:6807/2747506017","heartbeat_front_addr":"172.21.15.136:6805/2747506017","state":["exists","up"]},{"osd":1,"uuid":"d15e0912-05cd-4fea-bae1-79d478c1ec69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":259,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6817","nonce":3821859466}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6819","nonce":3821859466}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6823","nonce":3821859466}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3821859466},{"type":"v1","addr":"172.21.15.136:6821","nonce":3821859466}]},"public_addr":"172.21.15.136:6817/3821859466","cluster_addr":"172.21.15.136:6819/3821859466","heartbeat_back_addr":"172.21.15.136:6823/3821859466","heartbeat_front_addr":"172.21.15.136:6821/3821859466","state":["exists","up"]},{"osd":2,"uuid":"4b191e79-cb0c-4d90-a66d-7bd1e50b7d18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":259,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6809","nonce":253893352}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6811","nonce":253893352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6815","nonce":253893352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":253893352},{"type":"v1","addr":"172.21.15.136:6813","nonce":253893352}]},"public_addr":"172.21.15.136:6809/253893352","cluster_addr":"172.21.15.136:6811/253893352","heartbeat_back_addr":"172.21.15.136:6815/253893352","heartbeat_front_addr":"172.21.15.136:6813/253893352","state":["exists","up"]},{"osd":3,"uuid":"1062e5ff-d3f9-4d3a-9d13-37c8f2f9a90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":259,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6825","nonce":3414382223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6827","nonce":3414382223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6831","nonce":3414382223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3414382223},{"type":"v1","addr":"172.21.15.136:6829","nonce":3414382223}]},"public_addr":"172.21.15.136:6825/3414382223","cluster_addr":"172.21.15.136:6827/3414382223","heartbeat_back_addr":"172.21.15.136:6831/3414382223","heartbeat_front_addr":"172.21.15.136:6829/3414382223","state":["exists","up"]},{"osd":4,"uuid":"85b19339-9132-4b74-af11-bf2838de4cba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":259,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6809","nonce":3291801237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6811","nonce":3291801237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6815","nonce":3291801237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":3291801237},{"type":"v1","addr":"172.21.15.164:6813","nonce":3291801237}]},"public_addr":"172.21.15.164:6809/3291801237","cluster_addr":"172.21.15.164:6811/3291801237","heartbeat_back_addr":"172.21.15.164:6815/3291801237","heartbeat_front_addr":"172.21.15.164:6813/3291801237","state":["exists","up"]},{"osd":5,"uuid":"ca31b1f5-b528-4f5e-85e0-2c38fa53f7ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":259,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6825","nonce":100873265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6827","nonce":100873265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6831","nonce":100873265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":100873265},{"type":"v1","addr":"172.21.15.164:6829","nonce":100873265}]},"public_addr":"172.21.15.164:6825/100873265","cluster_addr":"172.21.15.164:6827/100873265","heartbeat_back_addr":"172.21.15.164:6831/100873265","heartbeat_front_addr":"172.21.15.164:6829/100873265","state":["exists","up"]},{"osd":6,"uuid":"d2a9730f-bc66-45f1-8b7f-5e982423c8d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":259,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6817","nonce":2410758924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6819","nonce":2410758924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6823","nonce":2410758924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":2410758924},{"type":"v1","addr":"172.21.15.164:6821","nonce":2410758924}]},"public_addr":"172.21.15.164:6817/2410758924","cluster_addr":"172.21.15.164:6819/2410758924","heartbeat_back_addr":"172.21.15.164:6823/2410758924","heartbeat_front_addr":"172.21.15.164:6821/2410758924","state":["exists","up"]},{"osd":7,"uuid":"845c3a0b-1d2c-4750-9cc2-cabb32ded81e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":259,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6801","nonce":4018161608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6803","nonce":4018161608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6807","nonce":4018161608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":4018161608},{"type":"v1","addr":"172.21.15.164:6805","nonce":4018161608}]},"public_addr":"172.21.15.164:6801/4018161608","cluster_addr":"172.21.15.164:6803/4018161608","heartbeat_back_addr":"172.21.15.164:6807/4018161608","heartbeat_front_addr":"172.21.15.164:6805/4018161608","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.275552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.318182+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.342147+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.342562+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.315740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.359563+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.278972+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-22T19:51:06.255612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":2}]},{"pgid":"3.13","mappings":[{"from":6,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":2}]},{"pgid":"3.1c","mappings":[{"from":4,"to":2}]},{"pgid":"3.23","mappings":[{"from":0,"to":2}]},{"pgid":"3.24","mappings":[{"from":3,"to":2},{"from":4,"to":1}]},{"pgid":"3.27","mappings":[{"from":3,"to":1}]},{"pgid":"3.31","mappings":[{"from":4,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]},{"pgid":"4.3b","mappings":[{"from":3,"to":2}]},{"pgid":"67.1e","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"client.0":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":26,"snaps":[{"begin":1,"length":2}]},{"pool":31,"snaps":[{"begin":1,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-22T20:48:21.572 INFO:tasks.ceph:Scrubbing osd.0 2024-04-22T20:48:21.572 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-22T20:48:21.722 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:48:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T20:48:21.722 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:48:21.733 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-22T20:48:22.029 INFO:teuthology.orchestra.run.smithi136.stderr:instructed osd(s) 0 to deep-scrub 2024-04-22T20:48:22.043 INFO:tasks.ceph:Scrubbing osd.1 2024-04-22T20:48:22.043 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-22T20:48:22.178 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:48:22.178 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T20:48:22.178 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:48:22.189 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-22T20:48:22.489 INFO:teuthology.orchestra.run.smithi136.stderr:instructed osd(s) 1 to deep-scrub 2024-04-22T20:48:22.502 INFO:tasks.ceph:Scrubbing osd.2 2024-04-22T20:48:22.503 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-22T20:48:22.639 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:48:22.639 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T20:48:22.639 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:48:22.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-22T20:48:22.950 INFO:teuthology.orchestra.run.smithi136.stderr:instructed osd(s) 2 to deep-scrub 2024-04-22T20:48:22.964 INFO:tasks.ceph:Scrubbing osd.3 2024-04-22T20:48:22.964 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-22T20:48:23.099 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:48:23.099 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T20:48:23.099 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:48:23.110 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-22T20:48:23.411 INFO:teuthology.orchestra.run.smithi136.stderr:instructed osd(s) 3 to deep-scrub 2024-04-22T20:48:23.425 INFO:tasks.ceph:Scrubbing osd.4 2024-04-22T20:48:23.425 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-22T20:48:23.559 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:48:23.560 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T20:48:23.560 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:48:23.570 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-22T20:48:23.871 INFO:teuthology.orchestra.run.smithi136.stderr:instructed osd(s) 4 to deep-scrub 2024-04-22T20:48:23.884 INFO:tasks.ceph:Scrubbing osd.5 2024-04-22T20:48:23.884 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-22T20:48:24.019 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:48:24.020 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T20:48:24.020 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:48:24.031 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-22T20:48:24.331 INFO:teuthology.orchestra.run.smithi136.stderr:instructed osd(s) 5 to deep-scrub 2024-04-22T20:48:24.344 INFO:tasks.ceph:Scrubbing osd.6 2024-04-22T20:48:24.345 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-22T20:48:24.484 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:48:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T20:48:24.485 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:48:24.496 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-22T20:48:24.794 INFO:teuthology.orchestra.run.smithi136.stderr:instructed osd(s) 6 to deep-scrub 2024-04-22T20:48:24.808 INFO:tasks.ceph:Scrubbing osd.7 2024-04-22T20:48:24.808 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-22T20:48:24.943 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-22T20:48:24.944 INFO:teuthology.orchestra.run.smithi136.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-22T20:48:24.944 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-22T20:48:24.955 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-22T20:48:25.252 INFO:teuthology.orchestra.run.smithi136.stderr:instructed osd(s) 7 to deep-scrub 2024-04-22T20:48:25.265 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T20:48:25.597 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:25.608 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T20:48:25.610 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":1971,"stamp":"2024-04-22T20:48:24.329917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":267312719,"num_objects":825,"num_object_clones":0,"num_object_copies":2119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":158844,"num_read_kb":2723065,"num_write":145049,"num_write_kb":2429940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13242,"ondisk_log_size":13242,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1953564,"kb_used_data":441188,"kb_used_omap":12,"kb_used_meta":1511027,"kb_avail":753021156,"statfs":{"total":773094113280,"available":771093663744,"internally_reserved":0,"allocated":451776512,"data_stored":436195170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1547292210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-602159302,"num_objects":-4094,"num_object_clones":0,"num_object_copies":-12226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4094,"num_whiteouts":0,"num_read":4603,"num_read_kb":4131,"num_write":4310,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005385"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.177805+0000","last_change":"2024-04-22T20:19:16.796805+0000","last_active":"2024-04-22T20:44:20.177805+0000","last_peered":"2024-04-22T20:44:20.177805+0000","last_clean":"2024-04-22T20:44:20.177805+0000","last_became_active":"2024-04-22T20:19:16.796520+0000","last_became_peered":"2024-04-22T20:19:16.796520+0000","last_unstale":"2024-04-22T20:44:20.177805+0000","last_undegraded":"2024-04-22T20:44:20.177805+0000","last_fullsized":"2024-04-22T20:44:20.177805+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:30:42.980562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"261'5","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.208924+0000","last_change":"2024-04-22T20:05:17.900640+0000","last_active":"2024-04-22T20:47:50.208924+0000","last_peered":"2024-04-22T20:47:50.208924+0000","last_clean":"2024-04-22T20:47:50.208924+0000","last_became_active":"2024-04-22T20:05:17.900325+0000","last_became_peered":"2024-04-22T20:05:17.900325+0000","last_unstale":"2024-04-22T20:47:50.208924+0000","last_undegraded":"2024-04-22T20:47:50.208924+0000","last_fullsized":"2024-04-22T20:47:50.208924+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:55:20.209199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154687+0000","last_change":"2024-04-22T20:05:13.813157+0000","last_active":"2024-04-22T20:45:15.154687+0000","last_peered":"2024-04-22T20:45:15.154687+0000","last_clean":"2024-04-22T20:45:15.154687+0000","last_became_active":"2024-04-22T20:05:13.812933+0000","last_became_peered":"2024-04-22T20:05:13.812933+0000","last_unstale":"2024-04-22T20:45:15.154687+0000","last_undegraded":"2024-04-22T20:45:15.154687+0000","last_fullsized":"2024-04-22T20:45:15.154687+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:47:15.012252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"261'20","reported_seq":100,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.176985+0000","last_change":"2024-04-22T20:01:04.876250+0000","last_active":"2024-04-22T20:44:50.176985+0000","last_peered":"2024-04-22T20:44:50.176985+0000","last_clean":"2024-04-22T20:44:50.176985+0000","last_became_active":"2024-04-22T20:01:04.875981+0000","last_became_peered":"2024-04-22T20:01:04.875981+0000","last_unstale":"2024-04-22T20:44:50.176985+0000","last_undegraded":"2024-04-22T20:44:50.176985+0000","last_fullsized":"2024-04-22T20:44:50.176985+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:47:50.228681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.060560+0000","last_change":"2024-04-22T20:19:16.790684+0000","last_active":"2024-04-22T20:44:20.060560+0000","last_peered":"2024-04-22T20:44:20.060560+0000","last_clean":"2024-04-22T20:44:20.060560+0000","last_became_active":"2024-04-22T20:19:16.790377+0000","last_became_peered":"2024-04-22T20:19:16.790377+0000","last_unstale":"2024-04-22T20:44:20.060560+0000","last_undegraded":"2024-04-22T20:44:20.060560+0000","last_fullsized":"2024-04-22T20:44:20.060560+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:16:51.462209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.178227+0000","last_change":"2024-04-22T20:05:17.899675+0000","last_active":"2024-04-22T20:45:20.178227+0000","last_peered":"2024-04-22T20:45:20.178227+0000","last_clean":"2024-04-22T20:45:20.178227+0000","last_became_active":"2024-04-22T20:05:17.899338+0000","last_became_peered":"2024-04-22T20:05:17.899338+0000","last_unstale":"2024-04-22T20:45:20.178227+0000","last_undegraded":"2024-04-22T20:45:20.178227+0000","last_fullsized":"2024-04-22T20:45:20.178227+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:09:36.841164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"258'4","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.043137+0000","last_change":"2024-04-22T20:05:13.808152+0000","last_active":"2024-04-22T20:45:15.043137+0000","last_peered":"2024-04-22T20:45:15.043137+0000","last_clean":"2024-04-22T20:45:15.043137+0000","last_became_active":"2024-04-22T20:05:13.807231+0000","last_became_peered":"2024-04-22T20:05:13.807231+0000","last_unstale":"2024-04-22T20:45:15.043137+0000","last_undegraded":"2024-04-22T20:45:15.043137+0000","last_fullsized":"2024-04-22T20:45:15.043137+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:18:02.300257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"261'22","reported_seq":101,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.066043+0000","last_change":"2024-04-22T20:01:04.874850+0000","last_active":"2024-04-22T20:45:15.066043+0000","last_peered":"2024-04-22T20:45:15.066043+0000","last_clean":"2024-04-22T20:45:15.066043+0000","last_became_active":"2024-04-22T20:01:04.874575+0000","last_became_peered":"2024-04-22T20:01:04.874575+0000","last_unstale":"2024-04-22T20:45:15.066043+0000","last_undegraded":"2024-04-22T20:45:15.066043+0000","last_fullsized":"2024-04-22T20:45:15.066043+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:00:34.689708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":48,"num_write":47,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.319216+0000","last_change":"2024-04-22T20:19:16.794970+0000","last_active":"2024-04-22T20:45:55.319216+0000","last_peered":"2024-04-22T20:45:55.319216+0000","last_clean":"2024-04-22T20:45:55.319216+0000","last_became_active":"2024-04-22T20:19:16.794700+0000","last_became_peered":"2024-04-22T20:19:16.794700+0000","last_unstale":"2024-04-22T20:45:55.319216+0000","last_undegraded":"2024-04-22T20:45:55.319216+0000","last_fullsized":"2024-04-22T20:45:55.319216+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:59.399131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.177554+0000","last_change":"2024-04-22T20:05:18.211663+0000","last_active":"2024-04-22T20:44:55.177554+0000","last_peered":"2024-04-22T20:44:55.177554+0000","last_clean":"2024-04-22T20:44:55.177554+0000","last_became_active":"2024-04-22T20:05:18.211400+0000","last_became_peered":"2024-04-22T20:05:18.211400+0000","last_unstale":"2024-04-22T20:44:55.177554+0000","last_undegraded":"2024-04-22T20:44:55.177554+0000","last_fullsized":"2024-04-22T20:44:55.177554+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:07:00.571570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"261'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.078600+0000","last_change":"2024-04-22T20:05:13.809630+0000","last_active":"2024-04-22T20:47:25.078600+0000","last_peered":"2024-04-22T20:47:25.078600+0000","last_clean":"2024-04-22T20:47:25.078600+0000","last_became_active":"2024-04-22T20:05:13.809349+0000","last_became_peered":"2024-04-22T20:05:13.809349+0000","last_unstale":"2024-04-22T20:47:25.078600+0000","last_undegraded":"2024-04-22T20:47:25.078600+0000","last_fullsized":"2024-04-22T20:47:25.078600+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:04:26.130081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"261'24","reported_seq":114,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.167723+0000","last_change":"2024-04-22T20:01:04.870729+0000","last_active":"2024-04-22T20:43:35.167723+0000","last_peered":"2024-04-22T20:43:35.167723+0000","last_clean":"2024-04-22T20:43:35.167723+0000","last_became_active":"2024-04-22T20:01:04.869785+0000","last_became_peered":"2024-04-22T20:01:04.869785+0000","last_unstale":"2024-04-22T20:43:35.167723+0000","last_undegraded":"2024-04-22T20:43:35.167723+0000","last_fullsized":"2024-04-22T20:43:35.167723+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:24:59.438391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":82,"num_write":36,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.159414+0000","last_change":"2024-04-22T20:19:16.795441+0000","last_active":"2024-04-22T20:45:55.159414+0000","last_peered":"2024-04-22T20:45:55.159414+0000","last_clean":"2024-04-22T20:45:55.159414+0000","last_became_active":"2024-04-22T20:19:16.793667+0000","last_became_peered":"2024-04-22T20:19:16.793667+0000","last_unstale":"2024-04-22T20:45:55.159414+0000","last_undegraded":"2024-04-22T20:45:55.159414+0000","last_fullsized":"2024-04-22T20:45:55.159414+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:09.885678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"261'10","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.490673+0000","last_change":"2024-04-22T20:06:05.349958+0000","last_active":"2024-04-22T20:48:17.490673+0000","last_peered":"2024-04-22T20:48:17.490673+0000","last_clean":"2024-04-22T20:48:17.490673+0000","last_became_active":"2024-04-22T20:06:05.345554+0000","last_became_peered":"2024-04-22T20:06:05.345554+0000","last_unstale":"2024-04-22T20:48:17.490673+0000","last_undegraded":"2024-04-22T20:48:17.490673+0000","last_fullsized":"2024-04-22T20:48:17.490673+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:50:22.244962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":20,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1f","version":"258'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.173992+0000","last_change":"2024-04-22T20:05:13.806565+0000","last_active":"2024-04-22T20:44:50.173992+0000","last_peered":"2024-04-22T20:44:50.173992+0000","last_clean":"2024-04-22T20:44:50.173992+0000","last_became_active":"2024-04-22T20:05:13.806246+0000","last_became_peered":"2024-04-22T20:05:13.806246+0000","last_unstale":"2024-04-22T20:44:50.173992+0000","last_undegraded":"2024-04-22T20:44:50.173992+0000","last_fullsized":"2024-04-22T20:44:50.173992+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:04:13.772282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":16720,"num_write":13,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"261'59","reported_seq":198,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.141957+0000","last_change":"2024-04-22T20:01:04.874395+0000","last_active":"2024-04-22T20:43:35.141957+0000","last_peered":"2024-04-22T20:43:35.141957+0000","last_clean":"2024-04-22T20:43:35.141957+0000","last_became_active":"2024-04-22T20:01:04.873899+0000","last_became_peered":"2024-04-22T20:01:04.873899+0000","last_unstale":"2024-04-22T20:43:35.141957+0000","last_undegraded":"2024-04-22T20:43:35.141957+0000","last_fullsized":"2024-04-22T20:43:35.141957+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:23:27.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":183,"num_write":99,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.155406+0000","last_change":"2024-04-22T20:19:16.794646+0000","last_active":"2024-04-22T20:45:20.155406+0000","last_peered":"2024-04-22T20:45:20.155406+0000","last_clean":"2024-04-22T20:45:20.155406+0000","last_became_active":"2024-04-22T20:19:16.794110+0000","last_became_peered":"2024-04-22T20:19:16.794110+0000","last_unstale":"2024-04-22T20:45:20.155406+0000","last_undegraded":"2024-04-22T20:45:20.155406+0000","last_fullsized":"2024-04-22T20:45:20.155406+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:42.561415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"261'2","reported_seq":43,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.197686+0000","last_change":"2024-04-22T20:05:17.898277+0000","last_active":"2024-04-22T20:46:35.197686+0000","last_peered":"2024-04-22T20:46:35.197686+0000","last_clean":"2024-04-22T20:46:35.197686+0000","last_became_active":"2024-04-22T20:05:17.897771+0000","last_became_peered":"2024-04-22T20:05:17.897771+0000","last_unstale":"2024-04-22T20:46:35.197686+0000","last_undegraded":"2024-04-22T20:46:35.197686+0000","last_fullsized":"2024-04-22T20:46:35.197686+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:22:07.487507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"258'5","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.170541+0000","last_change":"2024-04-22T20:05:13.812772+0000","last_active":"2024-04-22T20:43:55.170541+0000","last_peered":"2024-04-22T20:43:55.170541+0000","last_clean":"2024-04-22T20:43:55.170541+0000","last_became_active":"2024-04-22T20:05:13.812308+0000","last_became_peered":"2024-04-22T20:05:13.812308+0000","last_unstale":"2024-04-22T20:43:55.170541+0000","last_undegraded":"2024-04-22T20:43:55.170541+0000","last_fullsized":"2024-04-22T20:43:55.170541+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:52.451966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":9,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"261'8","reported_seq":72,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.141914+0000","last_change":"2024-04-22T20:01:04.877587+0000","last_active":"2024-04-22T20:43:35.141914+0000","last_peered":"2024-04-22T20:43:35.141914+0000","last_clean":"2024-04-22T20:43:35.141914+0000","last_became_active":"2024-04-22T20:01:04.877209+0000","last_became_peered":"2024-04-22T20:01:04.877209+0000","last_unstale":"2024-04-22T20:43:35.141914+0000","last_undegraded":"2024-04-22T20:43:35.141914+0000","last_fullsized":"2024-04-22T20:43:35.141914+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:57.901050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.037056+0000","last_change":"2024-04-22T20:19:16.789917+0000","last_active":"2024-04-22T20:44:20.037056+0000","last_peered":"2024-04-22T20:44:20.037056+0000","last_clean":"2024-04-22T20:44:20.037056+0000","last_became_active":"2024-04-22T20:19:16.789599+0000","last_became_peered":"2024-04-22T20:19:16.789599+0000","last_unstale":"2024-04-22T20:44:20.037056+0000","last_undegraded":"2024-04-22T20:44:20.037056+0000","last_fullsized":"2024-04-22T20:44:20.037056+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:15.659980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"261'2","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:25.323653+0000","last_change":"2024-04-22T20:05:17.891180+0000","last_active":"2024-04-22T20:46:25.323653+0000","last_peered":"2024-04-22T20:46:25.323653+0000","last_clean":"2024-04-22T20:46:25.323653+0000","last_became_active":"2024-04-22T20:05:17.890841+0000","last_became_peered":"2024-04-22T20:05:17.890841+0000","last_unstale":"2024-04-22T20:46:25.323653+0000","last_undegraded":"2024-04-22T20:46:25.323653+0000","last_fullsized":"2024-04-22T20:46:25.323653+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:53:40.164764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.167430+0000","last_change":"2024-04-22T20:05:13.807783+0000","last_active":"2024-04-22T20:44:05.167430+0000","last_peered":"2024-04-22T20:44:05.167430+0000","last_clean":"2024-04-22T20:44:05.167430+0000","last_became_active":"2024-04-22T20:05:13.807516+0000","last_became_peered":"2024-04-22T20:05:13.807516+0000","last_unstale":"2024-04-22T20:44:05.167430+0000","last_undegraded":"2024-04-22T20:44:05.167430+0000","last_fullsized":"2024-04-22T20:44:05.167430+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:58:08.969201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"261'77","reported_seq":239,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:35.299254+0000","last_change":"2024-04-22T20:01:04.873339+0000","last_active":"2024-04-22T20:43:35.299254+0000","last_peered":"2024-04-22T20:43:35.299254+0000","last_clean":"2024-04-22T20:43:35.299254+0000","last_became_active":"2024-04-22T20:01:04.873015+0000","last_became_peered":"2024-04-22T20:01:04.873015+0000","last_unstale":"2024-04-22T20:43:35.299254+0000","last_undegraded":"2024-04-22T20:43:35.299254+0000","last_fullsized":"2024-04-22T20:43:35.299254+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:49:05.613359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":236,"num_write":125,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"261'1","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053060+0000","last_change":"2024-04-22T20:19:16.794883+0000","last_active":"2024-04-22T20:44:20.053060+0000","last_peered":"2024-04-22T20:44:20.053060+0000","last_clean":"2024-04-22T20:44:20.053060+0000","last_became_active":"2024-04-22T20:19:16.794564+0000","last_became_peered":"2024-04-22T20:19:16.794564+0000","last_unstale":"2024-04-22T20:44:20.053060+0000","last_undegraded":"2024-04-22T20:44:20.053060+0000","last_fullsized":"2024-04-22T20:44:20.053060+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:15.121442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"261'5","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.887612+0000","last_change":"2024-04-22T20:05:17.901743+0000","last_active":"2024-04-22T20:48:15.887612+0000","last_peered":"2024-04-22T20:48:15.887612+0000","last_clean":"2024-04-22T20:48:15.887612+0000","last_became_active":"2024-04-22T20:05:17.901446+0000","last_became_peered":"2024-04-22T20:05:17.901446+0000","last_unstale":"2024-04-22T20:48:15.887612+0000","last_undegraded":"2024-04-22T20:48:15.887612+0000","last_fullsized":"2024-04-22T20:48:15.887612+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:13:01.401575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":11264,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"261'2","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198472+0000","last_change":"2024-04-22T20:05:13.798628+0000","last_active":"2024-04-22T20:47:45.198472+0000","last_peered":"2024-04-22T20:47:45.198472+0000","last_clean":"2024-04-22T20:47:45.198472+0000","last_became_active":"2024-04-22T20:05:13.798311+0000","last_became_peered":"2024-04-22T20:05:13.798311+0000","last_unstale":"2024-04-22T20:47:45.198472+0000","last_undegraded":"2024-04-22T20:47:45.198472+0000","last_fullsized":"2024-04-22T20:47:45.198472+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:21:50.192609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'27","reported_seq":116,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.176374+0000","last_change":"2024-04-22T20:01:04.873715+0000","last_active":"2024-04-22T20:44:45.176374+0000","last_peered":"2024-04-22T20:44:45.176374+0000","last_clean":"2024-04-22T20:44:45.176374+0000","last_became_active":"2024-04-22T20:01:04.873426+0000","last_became_peered":"2024-04-22T20:01:04.873426+0000","last_unstale":"2024-04-22T20:44:45.176374+0000","last_undegraded":"2024-04-22T20:44:45.176374+0000","last_fullsized":"2024-04-22T20:44:45.176374+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:29:40.987253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":80,"num_write":54,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":23,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053106+0000","last_change":"2024-04-22T20:19:16.800874+0000","last_active":"2024-04-22T20:44:20.053106+0000","last_peered":"2024-04-22T20:44:20.053106+0000","last_clean":"2024-04-22T20:44:20.053106+0000","last_became_active":"2024-04-22T20:19:16.800611+0000","last_became_peered":"2024-04-22T20:19:16.800611+0000","last_unstale":"2024-04-22T20:44:20.053106+0000","last_undegraded":"2024-04-22T20:44:20.053106+0000","last_fullsized":"2024-04-22T20:44:20.053106+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:27:17.425924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"258'4","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.155286+0000","last_change":"2024-04-22T20:05:17.892900+0000","last_active":"2024-04-22T20:45:20.155286+0000","last_peered":"2024-04-22T20:45:20.155286+0000","last_clean":"2024-04-22T20:45:20.155286+0000","last_became_active":"2024-04-22T20:05:17.892624+0000","last_became_peered":"2024-04-22T20:05:17.892624+0000","last_unstale":"2024-04-22T20:45:20.155286+0000","last_undegraded":"2024-04-22T20:45:20.155286+0000","last_fullsized":"2024-04-22T20:45:20.155286+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:02.296574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3072,"num_write":4,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.197460+0000","last_change":"2024-04-22T20:05:13.816306+0000","last_active":"2024-04-22T20:47:45.197460+0000","last_peered":"2024-04-22T20:47:45.197460+0000","last_clean":"2024-04-22T20:47:45.197460+0000","last_became_active":"2024-04-22T20:05:13.816011+0000","last_became_peered":"2024-04-22T20:05:13.816011+0000","last_unstale":"2024-04-22T20:47:45.197460+0000","last_undegraded":"2024-04-22T20:47:45.197460+0000","last_fullsized":"2024-04-22T20:47:45.197460+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:44.934646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"261'43","reported_seq":150,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.038576+0000","last_change":"2024-04-22T20:01:04.872669+0000","last_active":"2024-04-22T20:44:35.038576+0000","last_peered":"2024-04-22T20:44:35.038576+0000","last_clean":"2024-04-22T20:44:35.038576+0000","last_became_active":"2024-04-22T20:01:04.872356+0000","last_became_peered":"2024-04-22T20:01:04.872356+0000","last_unstale":"2024-04-22T20:44:35.038576+0000","last_undegraded":"2024-04-22T20:44:35.038576+0000","last_fullsized":"2024-04-22T20:44:35.038576+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:11.955524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":119,"num_write":72,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.041329+0000","last_change":"2024-04-22T20:19:16.796888+0000","last_active":"2024-04-22T20:45:00.041329+0000","last_peered":"2024-04-22T20:45:00.041329+0000","last_clean":"2024-04-22T20:45:00.041329+0000","last_became_active":"2024-04-22T20:19:16.796155+0000","last_became_peered":"2024-04-22T20:19:16.796155+0000","last_unstale":"2024-04-22T20:45:00.041329+0000","last_undegraded":"2024-04-22T20:45:00.041329+0000","last_fullsized":"2024-04-22T20:45:00.041329+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:25:57.681625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145610+0000","last_change":"2024-04-22T20:05:18.209303+0000","last_active":"2024-04-22T20:44:00.145610+0000","last_peered":"2024-04-22T20:44:00.145610+0000","last_clean":"2024-04-22T20:44:00.145610+0000","last_became_active":"2024-04-22T20:05:18.209023+0000","last_became_peered":"2024-04-22T20:05:18.209023+0000","last_unstale":"2024-04-22T20:44:00.145610+0000","last_undegraded":"2024-04-22T20:44:00.145610+0000","last_fullsized":"2024-04-22T20:44:00.145610+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:14:17.745357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"258'6","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.184202+0000","last_change":"2024-04-22T20:05:13.809878+0000","last_active":"2024-04-22T20:45:05.184202+0000","last_peered":"2024-04-22T20:45:05.184202+0000","last_clean":"2024-04-22T20:45:05.184202+0000","last_became_active":"2024-04-22T20:05:13.809548+0000","last_became_peered":"2024-04-22T20:05:13.809548+0000","last_unstale":"2024-04-22T20:45:05.184202+0000","last_undegraded":"2024-04-22T20:45:05.184202+0000","last_fullsized":"2024-04-22T20:45:05.184202+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:10.962028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":6146,"num_write":5,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'63","reported_seq":241,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.175344+0000","last_change":"2024-04-22T20:01:04.873570+0000","last_active":"2024-04-22T20:44:35.175344+0000","last_peered":"2024-04-22T20:44:35.175344+0000","last_clean":"2024-04-22T20:44:35.175344+0000","last_became_active":"2024-04-22T20:01:04.873236+0000","last_became_peered":"2024-04-22T20:01:04.873236+0000","last_unstale":"2024-04-22T20:44:35.175344+0000","last_undegraded":"2024-04-22T20:44:35.175344+0000","last_fullsized":"2024-04-22T20:44:35.175344+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:38.761169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":250,"num_write":111,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.064494+0000","last_change":"2024-04-22T20:19:16.791895+0000","last_active":"2024-04-22T20:45:00.064494+0000","last_peered":"2024-04-22T20:45:00.064494+0000","last_clean":"2024-04-22T20:45:00.064494+0000","last_became_active":"2024-04-22T20:19:16.791597+0000","last_became_peered":"2024-04-22T20:19:16.791597+0000","last_unstale":"2024-04-22T20:45:00.064494+0000","last_undegraded":"2024-04-22T20:45:00.064494+0000","last_fullsized":"2024-04-22T20:45:00.064494+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:51.409610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":40,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.153541+0000","last_change":"2024-04-22T20:05:18.211042+0000","last_active":"2024-04-22T20:45:05.153541+0000","last_peered":"2024-04-22T20:45:05.153541+0000","last_clean":"2024-04-22T20:45:05.153541+0000","last_became_active":"2024-04-22T20:05:18.210704+0000","last_became_peered":"2024-04-22T20:05:18.210704+0000","last_unstale":"2024-04-22T20:45:05.153541+0000","last_undegraded":"2024-04-22T20:45:05.153541+0000","last_fullsized":"2024-04-22T20:45:05.153541+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:53:47.345090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.048999+0000","last_change":"2024-04-22T20:05:13.811174+0000","last_active":"2024-04-22T20:43:55.048999+0000","last_peered":"2024-04-22T20:43:55.048999+0000","last_clean":"2024-04-22T20:43:55.048999+0000","last_became_active":"2024-04-22T20:05:13.810902+0000","last_became_peered":"2024-04-22T20:05:13.810902+0000","last_unstale":"2024-04-22T20:43:55.048999+0000","last_undegraded":"2024-04-22T20:43:55.048999+0000","last_fullsized":"2024-04-22T20:43:55.048999+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:05:52.847563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"250'9","reported_seq":76,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.149931+0000","last_change":"2024-04-22T20:01:04.874512+0000","last_active":"2024-04-22T20:44:35.149931+0000","last_peered":"2024-04-22T20:44:35.149931+0000","last_clean":"2024-04-22T20:44:35.149931+0000","last_became_active":"2024-04-22T20:01:04.874230+0000","last_became_peered":"2024-04-22T20:01:04.874230+0000","last_unstale":"2024-04-22T20:44:35.149931+0000","last_undegraded":"2024-04-22T20:44:35.149931+0000","last_fullsized":"2024-04-22T20:44:35.149931+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:17:12.953078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.059314+0000","last_change":"2024-04-22T20:19:16.795626+0000","last_active":"2024-04-22T20:45:00.059314+0000","last_peered":"2024-04-22T20:45:00.059314+0000","last_clean":"2024-04-22T20:45:00.059314+0000","last_became_active":"2024-04-22T20:19:16.795139+0000","last_became_peered":"2024-04-22T20:19:16.795139+0000","last_unstale":"2024-04-22T20:45:00.059314+0000","last_undegraded":"2024-04-22T20:45:00.059314+0000","last_fullsized":"2024-04-22T20:45:00.059314+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:17:47.054792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"261'16","reported_seq":75,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:25.163079+0000","last_change":"2024-04-22T20:05:18.212823+0000","last_active":"2024-04-22T20:46:25.163079+0000","last_peered":"2024-04-22T20:46:25.163079+0000","last_clean":"2024-04-22T20:46:25.163079+0000","last_became_active":"2024-04-22T20:05:18.212336+0000","last_became_peered":"2024-04-22T20:05:18.212336+0000","last_unstale":"2024-04-22T20:46:25.163079+0000","last_undegraded":"2024-04-22T20:46:25.163079+0000","last_fullsized":"2024-04-22T20:46:25.163079+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:32.041183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437184,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":38918,"num_write":23,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"261'2","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:20.072214+0000","last_change":"2024-04-22T20:05:13.816425+0000","last_active":"2024-04-22T20:46:20.072214+0000","last_peered":"2024-04-22T20:46:20.072214+0000","last_clean":"2024-04-22T20:46:20.072214+0000","last_became_active":"2024-04-22T20:05:13.816036+0000","last_became_peered":"2024-04-22T20:05:13.816036+0000","last_unstale":"2024-04-22T20:46:20.072214+0000","last_undegraded":"2024-04-22T20:46:20.072214+0000","last_fullsized":"2024-04-22T20:46:20.072214+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:36.892934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'53","reported_seq":176,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.062011+0000","last_change":"2024-04-22T20:01:04.878348+0000","last_active":"2024-04-22T20:44:35.062011+0000","last_peered":"2024-04-22T20:44:35.062011+0000","last_clean":"2024-04-22T20:44:35.062011+0000","last_became_active":"2024-04-22T20:01:04.878067+0000","last_became_peered":"2024-04-22T20:01:04.878067+0000","last_unstale":"2024-04-22T20:44:35.062011+0000","last_undegraded":"2024-04-22T20:44:35.062011+0000","last_fullsized":"2024-04-22T20:44:35.062011+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:52:49.300812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":150,"num_write":99,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.183432+0000","last_change":"2024-04-22T20:19:16.796141+0000","last_active":"2024-04-22T20:45:00.183432+0000","last_peered":"2024-04-22T20:45:00.183432+0000","last_clean":"2024-04-22T20:45:00.183432+0000","last_became_active":"2024-04-22T20:19:16.795855+0000","last_became_peered":"2024-04-22T20:19:16.795855+0000","last_unstale":"2024-04-22T20:45:00.183432+0000","last_undegraded":"2024-04-22T20:45:00.183432+0000","last_fullsized":"2024-04-22T20:45:00.183432+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:02.646044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"0'0","reported_seq":39,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166753+0000","last_change":"2024-04-22T20:05:17.902267+0000","last_active":"2024-04-22T20:44:00.166753+0000","last_peered":"2024-04-22T20:44:00.166753+0000","last_clean":"2024-04-22T20:44:00.166753+0000","last_became_active":"2024-04-22T20:05:17.901954+0000","last_became_peered":"2024-04-22T20:05:17.901954+0000","last_unstale":"2024-04-22T20:44:00.166753+0000","last_undegraded":"2024-04-22T20:44:00.166753+0000","last_fullsized":"2024-04-22T20:44:00.166753+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:44:04.704303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'4","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.563865+0000","last_change":"2024-04-22T20:05:13.813528+0000","last_active":"2024-04-22T20:48:16.563865+0000","last_peered":"2024-04-22T20:48:16.563865+0000","last_clean":"2024-04-22T20:48:16.563865+0000","last_became_active":"2024-04-22T20:05:13.813256+0000","last_became_peered":"2024-04-22T20:05:13.813256+0000","last_unstale":"2024-04-22T20:48:16.563865+0000","last_undegraded":"2024-04-22T20:48:16.563865+0000","last_fullsized":"2024-04-22T20:48:16.563865+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:19.323647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":44,"num_write":4,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"261'46","reported_seq":164,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.064807+0000","last_change":"2024-04-22T20:01:04.874056+0000","last_active":"2024-04-22T20:45:35.064807+0000","last_peered":"2024-04-22T20:45:35.064807+0000","last_clean":"2024-04-22T20:45:35.064807+0000","last_became_active":"2024-04-22T20:01:04.873533+0000","last_became_peered":"2024-04-22T20:01:04.873533+0000","last_unstale":"2024-04-22T20:45:35.064807+0000","last_undegraded":"2024-04-22T20:45:35.064807+0000","last_fullsized":"2024-04-22T20:45:35.064807+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:43.868445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":192,"num_read_kb":137,"num_write":82,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158918+0000","last_change":"2024-04-22T20:19:16.793148+0000","last_active":"2024-04-22T20:45:50.158918+0000","last_peered":"2024-04-22T20:45:50.158918+0000","last_clean":"2024-04-22T20:45:50.158918+0000","last_became_active":"2024-04-22T20:19:16.792856+0000","last_became_peered":"2024-04-22T20:19:16.792856+0000","last_unstale":"2024-04-22T20:45:50.158918+0000","last_undegraded":"2024-04-22T20:45:50.158918+0000","last_fullsized":"2024-04-22T20:45:50.158918+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:34.522183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'8","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.056835+0000","last_change":"2024-04-22T20:05:17.897292+0000","last_active":"2024-04-22T20:47:20.056835+0000","last_peered":"2024-04-22T20:47:20.056835+0000","last_clean":"2024-04-22T20:47:20.056835+0000","last_became_active":"2024-04-22T20:05:17.897040+0000","last_became_peered":"2024-04-22T20:05:17.897040+0000","last_unstale":"2024-04-22T20:47:20.056835+0000","last_undegraded":"2024-04-22T20:47:20.056835+0000","last_fullsized":"2024-04-22T20:47:20.056835+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:13:37.748724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":12290,"num_write":7,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"261'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:15.194625+0000","last_change":"2024-04-22T20:05:13.814475+0000","last_active":"2024-04-22T20:47:15.194625+0000","last_peered":"2024-04-22T20:47:15.194625+0000","last_clean":"2024-04-22T20:47:15.194625+0000","last_became_active":"2024-04-22T20:05:13.814222+0000","last_became_peered":"2024-04-22T20:05:13.814222+0000","last_unstale":"2024-04-22T20:47:15.194625+0000","last_undegraded":"2024-04-22T20:47:15.194625+0000","last_fullsized":"2024-04-22T20:47:15.194625+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:29:09.438132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"261'54","reported_seq":190,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063228+0000","last_change":"2024-04-22T20:01:04.873752+0000","last_active":"2024-04-22T20:45:25.063228+0000","last_peered":"2024-04-22T20:45:25.063228+0000","last_clean":"2024-04-22T20:45:25.063228+0000","last_became_active":"2024-04-22T20:01:04.873422+0000","last_became_peered":"2024-04-22T20:01:04.873422+0000","last_unstale":"2024-04-22T20:45:25.063228+0000","last_undegraded":"2024-04-22T20:45:25.063228+0000","last_fullsized":"2024-04-22T20:45:25.063228+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:49.464347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":184,"num_write":82,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.184955+0000","last_change":"2024-04-22T20:19:16.795990+0000","last_active":"2024-04-22T20:45:55.184955+0000","last_peered":"2024-04-22T20:45:55.184955+0000","last_clean":"2024-04-22T20:45:55.184955+0000","last_became_active":"2024-04-22T20:19:16.795633+0000","last_became_peered":"2024-04-22T20:19:16.795633+0000","last_unstale":"2024-04-22T20:45:55.184955+0000","last_undegraded":"2024-04-22T20:45:55.184955+0000","last_fullsized":"2024-04-22T20:45:55.184955+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:55.584364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"261'9","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.056802+0000","last_change":"2024-04-22T20:05:17.900013+0000","last_active":"2024-04-22T20:47:20.056802+0000","last_peered":"2024-04-22T20:47:20.056802+0000","last_clean":"2024-04-22T20:47:20.056802+0000","last_became_active":"2024-04-22T20:05:17.899682+0000","last_became_peered":"2024-04-22T20:05:17.899682+0000","last_unstale":"2024-04-22T20:47:20.056802+0000","last_undegraded":"2024-04-22T20:47:20.056802+0000","last_fullsized":"2024-04-22T20:47:20.056802+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:57:08.679442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":8210,"num_write":13,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"261'6","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:15.168776+0000","last_change":"2024-04-22T20:05:13.813246+0000","last_active":"2024-04-22T20:47:15.168776+0000","last_peered":"2024-04-22T20:47:15.168776+0000","last_clean":"2024-04-22T20:47:15.168776+0000","last_became_active":"2024-04-22T20:05:13.812893+0000","last_became_peered":"2024-04-22T20:05:13.812893+0000","last_unstale":"2024-04-22T20:47:15.168776+0000","last_undegraded":"2024-04-22T20:47:15.168776+0000","last_fullsized":"2024-04-22T20:47:15.168776+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:26:36.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4096,"num_write":16,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"261'47","reported_seq":163,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.181140+0000","last_change":"2024-04-22T20:01:04.878065+0000","last_active":"2024-04-22T20:45:25.181140+0000","last_peered":"2024-04-22T20:45:25.181140+0000","last_clean":"2024-04-22T20:45:25.181140+0000","last_became_active":"2024-04-22T20:01:04.877755+0000","last_became_peered":"2024-04-22T20:01:04.877755+0000","last_unstale":"2024-04-22T20:45:25.181140+0000","last_undegraded":"2024-04-22T20:45:25.181140+0000","last_fullsized":"2024-04-22T20:45:25.181140+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:41:43.369530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":135,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"260'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.178798+0000","last_change":"2024-04-22T20:19:16.793874+0000","last_active":"2024-04-22T20:45:05.178798+0000","last_peered":"2024-04-22T20:45:05.178798+0000","last_clean":"2024-04-22T20:45:05.178798+0000","last_became_active":"2024-04-22T20:19:16.793555+0000","last_became_peered":"2024-04-22T20:19:16.793555+0000","last_unstale":"2024-04-22T20:45:05.178798+0000","last_undegraded":"2024-04-22T20:45:05.178798+0000","last_fullsized":"2024-04-22T20:45:05.178798+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:33:04.108646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"258'2","reported_seq":44,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.152971+0000","last_change":"2024-04-22T20:05:17.890318+0000","last_active":"2024-04-22T20:45:00.152971+0000","last_peered":"2024-04-22T20:45:00.152971+0000","last_clean":"2024-04-22T20:45:00.152971+0000","last_became_active":"2024-04-22T20:05:17.890070+0000","last_became_peered":"2024-04-22T20:05:17.890070+0000","last_unstale":"2024-04-22T20:45:00.152971+0000","last_undegraded":"2024-04-22T20:45:00.152971+0000","last_fullsized":"2024-04-22T20:45:00.152971+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:14:41.324774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"261'5","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.642182+0000","last_change":"2024-04-22T20:05:13.807133+0000","last_active":"2024-04-22T20:48:16.642182+0000","last_peered":"2024-04-22T20:48:16.642182+0000","last_clean":"2024-04-22T20:48:16.642182+0000","last_became_active":"2024-04-22T20:05:13.806882+0000","last_became_peered":"2024-04-22T20:05:13.806882+0000","last_unstale":"2024-04-22T20:48:16.642182+0000","last_undegraded":"2024-04-22T20:48:16.642182+0000","last_fullsized":"2024-04-22T20:48:16.642182+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:52:50.911604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"261'11","reported_seq":80,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.178877+0000","last_change":"2024-04-22T20:01:04.861370+0000","last_active":"2024-04-22T20:45:25.178877+0000","last_peered":"2024-04-22T20:45:25.178877+0000","last_clean":"2024-04-22T20:45:25.178877+0000","last_became_active":"2024-04-22T20:01:04.860991+0000","last_became_peered":"2024-04-22T20:01:04.860991+0000","last_unstale":"2024-04-22T20:45:25.178877+0000","last_undegraded":"2024-04-22T20:45:25.178877+0000","last_fullsized":"2024-04-22T20:45:25.178877+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:26:49.483447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":31,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158885+0000","last_change":"2024-04-22T20:19:16.794789+0000","last_active":"2024-04-22T20:45:50.158885+0000","last_peered":"2024-04-22T20:45:50.158885+0000","last_clean":"2024-04-22T20:45:50.158885+0000","last_became_active":"2024-04-22T20:19:16.791995+0000","last_became_peered":"2024-04-22T20:19:16.791995+0000","last_unstale":"2024-04-22T20:45:50.158885+0000","last_undegraded":"2024-04-22T20:45:50.158885+0000","last_fullsized":"2024-04-22T20:45:50.158885+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:04:16.246591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"258'4","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.311157+0000","last_change":"2024-04-22T20:05:17.890780+0000","last_active":"2024-04-22T20:45:00.311157+0000","last_peered":"2024-04-22T20:45:00.311157+0000","last_clean":"2024-04-22T20:45:00.311157+0000","last_became_active":"2024-04-22T20:05:17.890408+0000","last_became_peered":"2024-04-22T20:05:17.890408+0000","last_unstale":"2024-04-22T20:45:00.311157+0000","last_undegraded":"2024-04-22T20:45:00.311157+0000","last_fullsized":"2024-04-22T20:45:00.311157+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:30:52.042989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1342,"num_write":4,"num_write_kb":1342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"258'4","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.182683+0000","last_change":"2024-04-22T20:05:13.808782+0000","last_active":"2024-04-22T20:44:55.182683+0000","last_peered":"2024-04-22T20:44:55.182683+0000","last_clean":"2024-04-22T20:44:55.182683+0000","last_became_active":"2024-04-22T20:05:13.808473+0000","last_became_peered":"2024-04-22T20:05:13.808473+0000","last_unstale":"2024-04-22T20:44:55.182683+0000","last_undegraded":"2024-04-22T20:44:55.182683+0000","last_fullsized":"2024-04-22T20:44:55.182683+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:39:30.056682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"261'46","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.155908+0000","last_change":"2024-04-22T20:01:04.875088+0000","last_active":"2024-04-22T20:45:25.155908+0000","last_peered":"2024-04-22T20:45:25.155908+0000","last_clean":"2024-04-22T20:45:25.155908+0000","last_became_active":"2024-04-22T20:01:04.874816+0000","last_became_peered":"2024-04-22T20:01:04.874816+0000","last_unstale":"2024-04-22T20:45:25.155908+0000","last_undegraded":"2024-04-22T20:45:25.155908+0000","last_fullsized":"2024-04-22T20:45:25.155908+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:59:09.574988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":291,"num_read_kb":200,"num_write":62,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.066966+0000","last_change":"2024-04-22T20:19:16.795282+0000","last_active":"2024-04-22T20:45:25.066966+0000","last_peered":"2024-04-22T20:45:25.066966+0000","last_clean":"2024-04-22T20:45:25.066966+0000","last_became_active":"2024-04-22T20:19:16.794954+0000","last_became_peered":"2024-04-22T20:19:16.794954+0000","last_unstale":"2024-04-22T20:45:25.066966+0000","last_undegraded":"2024-04-22T20:45:25.066966+0000","last_fullsized":"2024-04-22T20:45:25.066966+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:53:00.610478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"261'6","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.260135+0000","last_change":"2024-04-22T20:05:17.889647+0000","last_active":"2024-04-22T20:48:16.260135+0000","last_peered":"2024-04-22T20:48:16.260135+0000","last_clean":"2024-04-22T20:48:16.260135+0000","last_became_active":"2024-04-22T20:05:17.889370+0000","last_became_peered":"2024-04-22T20:05:17.889370+0000","last_unstale":"2024-04-22T20:48:16.260135+0000","last_undegraded":"2024-04-22T20:48:16.260135+0000","last_fullsized":"2024-04-22T20:48:16.260135+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:26:55.859900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":6,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"261'2","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.086369+0000","last_change":"2024-04-22T20:05:13.815613+0000","last_active":"2024-04-22T20:48:00.086369+0000","last_peered":"2024-04-22T20:48:00.086369+0000","last_clean":"2024-04-22T20:48:00.086369+0000","last_became_active":"2024-04-22T20:05:13.815317+0000","last_became_peered":"2024-04-22T20:05:13.815317+0000","last_unstale":"2024-04-22T20:48:00.086369+0000","last_undegraded":"2024-04-22T20:48:00.086369+0000","last_fullsized":"2024-04-22T20:48:00.086369+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:25:12.295293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"261'28","reported_seq":118,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:43:55.145012+0000","last_change":"2024-04-22T20:01:04.878886+0000","last_active":"2024-04-22T20:43:55.145012+0000","last_peered":"2024-04-22T20:43:55.145012+0000","last_clean":"2024-04-22T20:43:55.145012+0000","last_became_active":"2024-04-22T20:01:04.878599+0000","last_became_peered":"2024-04-22T20:01:04.878599+0000","last_unstale":"2024-04-22T20:43:55.145012+0000","last_undegraded":"2024-04-22T20:43:55.145012+0000","last_fullsized":"2024-04-22T20:43:55.145012+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:57:43.301162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":77,"num_write":54,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.050575+0000","last_change":"2024-04-22T20:19:16.794996+0000","last_active":"2024-04-22T20:44:05.050575+0000","last_peered":"2024-04-22T20:44:05.050575+0000","last_clean":"2024-04-22T20:44:05.050575+0000","last_became_active":"2024-04-22T20:19:16.793678+0000","last_became_peered":"2024-04-22T20:19:16.793678+0000","last_unstale":"2024-04-22T20:44:05.050575+0000","last_undegraded":"2024-04-22T20:44:05.050575+0000","last_fullsized":"2024-04-22T20:44:05.050575+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:48:13.172887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158865+0000","last_change":"2024-04-22T20:05:17.892875+0000","last_active":"2024-04-22T20:45:50.158865+0000","last_peered":"2024-04-22T20:45:50.158865+0000","last_clean":"2024-04-22T20:45:50.158865+0000","last_became_active":"2024-04-22T20:05:17.892604+0000","last_became_peered":"2024-04-22T20:05:17.892604+0000","last_unstale":"2024-04-22T20:45:50.158865+0000","last_undegraded":"2024-04-22T20:45:50.158865+0000","last_fullsized":"2024-04-22T20:45:50.158865+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:20.757720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"261'10","reported_seq":68,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.082186+0000","last_change":"2024-04-22T20:05:13.814193+0000","last_active":"2024-04-22T20:48:00.082186+0000","last_peered":"2024-04-22T20:48:00.082186+0000","last_clean":"2024-04-22T20:48:00.082186+0000","last_became_active":"2024-04-22T20:05:13.813946+0000","last_became_peered":"2024-04-22T20:05:13.813946+0000","last_unstale":"2024-04-22T20:48:00.082186+0000","last_undegraded":"2024-04-22T20:48:00.082186+0000","last_fullsized":"2024-04-22T20:48:00.082186+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:04:48.745503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":15362,"num_write":9,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"261'28","reported_seq":127,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:05.193167+0000","last_change":"2024-04-22T20:01:04.873158+0000","last_active":"2024-04-22T20:46:05.193167+0000","last_peered":"2024-04-22T20:46:05.193167+0000","last_clean":"2024-04-22T20:46:05.193167+0000","last_became_active":"2024-04-22T20:01:04.872834+0000","last_became_peered":"2024-04-22T20:01:04.872834+0000","last_unstale":"2024-04-22T20:46:05.193167+0000","last_undegraded":"2024-04-22T20:46:05.193167+0000","last_fullsized":"2024-04-22T20:46:05.193167+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:25.675744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":90,"num_write":50,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.313962+0000","last_change":"2024-04-22T20:19:16.791183+0000","last_active":"2024-04-22T20:45:20.313962+0000","last_peered":"2024-04-22T20:45:20.313962+0000","last_clean":"2024-04-22T20:45:20.313962+0000","last_became_active":"2024-04-22T20:19:16.790917+0000","last_became_peered":"2024-04-22T20:19:16.790917+0000","last_unstale":"2024-04-22T20:45:20.313962+0000","last_undegraded":"2024-04-22T20:45:20.313962+0000","last_fullsized":"2024-04-22T20:45:20.313962+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:12:37.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:00.048158+0000","last_change":"2024-04-22T20:05:18.211747+0000","last_active":"2024-04-22T20:46:00.048158+0000","last_peered":"2024-04-22T20:46:00.048158+0000","last_clean":"2024-04-22T20:46:00.048158+0000","last_became_active":"2024-04-22T20:05:18.211445+0000","last_became_peered":"2024-04-22T20:05:18.211445+0000","last_unstale":"2024-04-22T20:46:00.048158+0000","last_undegraded":"2024-04-22T20:46:00.048158+0000","last_fullsized":"2024-04-22T20:46:00.048158+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:14:07.558996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"258'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183750+0000","last_change":"2024-04-22T20:05:13.809343+0000","last_active":"2024-04-22T20:45:45.183750+0000","last_peered":"2024-04-22T20:45:45.183750+0000","last_clean":"2024-04-22T20:45:45.183750+0000","last_became_active":"2024-04-22T20:05:13.806928+0000","last_became_peered":"2024-04-22T20:05:13.806928+0000","last_unstale":"2024-04-22T20:45:45.183750+0000","last_undegraded":"2024-04-22T20:45:45.183750+0000","last_fullsized":"2024-04-22T20:45:45.183750+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:36:20.049785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"261'28","reported_seq":118,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:05.320617+0000","last_change":"2024-04-22T20:01:04.869517+0000","last_active":"2024-04-22T20:46:05.320617+0000","last_peered":"2024-04-22T20:46:05.320617+0000","last_clean":"2024-04-22T20:46:05.320617+0000","last_became_active":"2024-04-22T20:01:04.869226+0000","last_became_peered":"2024-04-22T20:01:04.869226+0000","last_unstale":"2024-04-22T20:46:05.320617+0000","last_undegraded":"2024-04-22T20:46:05.320617+0000","last_fullsized":"2024-04-22T20:46:05.320617+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:02:16.025206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":80,"num_write":56,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062354+0000","last_change":"2024-04-22T20:19:16.796398+0000","last_active":"2024-04-22T20:45:20.062354+0000","last_peered":"2024-04-22T20:45:20.062354+0000","last_clean":"2024-04-22T20:45:20.062354+0000","last_became_active":"2024-04-22T20:19:16.795524+0000","last_became_peered":"2024-04-22T20:19:16.795524+0000","last_unstale":"2024-04-22T20:45:20.062354+0000","last_undegraded":"2024-04-22T20:45:20.062354+0000","last_fullsized":"2024-04-22T20:45:20.062354+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:24:50.852300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:05.174798+0000","last_change":"2024-04-22T20:05:17.899183+0000","last_active":"2024-04-22T20:48:05.174798+0000","last_peered":"2024-04-22T20:48:05.174798+0000","last_clean":"2024-04-22T20:48:05.174798+0000","last_became_active":"2024-04-22T20:05:17.898858+0000","last_became_peered":"2024-04-22T20:05:17.898858+0000","last_unstale":"2024-04-22T20:48:05.174798+0000","last_undegraded":"2024-04-22T20:48:05.174798+0000","last_fullsized":"2024-04-22T20:48:05.174798+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:21.640059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"258'3","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.047547+0000","last_change":"2024-04-22T20:05:13.808403+0000","last_active":"2024-04-22T20:45:55.047547+0000","last_peered":"2024-04-22T20:45:55.047547+0000","last_clean":"2024-04-22T20:45:55.047547+0000","last_became_active":"2024-04-22T20:05:13.807945+0000","last_became_peered":"2024-04-22T20:05:13.807945+0000","last_unstale":"2024-04-22T20:45:55.047547+0000","last_undegraded":"2024-04-22T20:45:55.047547+0000","last_fullsized":"2024-04-22T20:45:55.047547+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:02:06.578487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":8,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"261'24","reported_seq":106,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:05.070750+0000","last_change":"2024-04-22T20:01:04.872099+0000","last_active":"2024-04-22T20:46:05.070750+0000","last_peered":"2024-04-22T20:46:05.070750+0000","last_clean":"2024-04-22T20:46:05.070750+0000","last_became_active":"2024-04-22T20:01:04.871829+0000","last_became_peered":"2024-04-22T20:01:04.871829+0000","last_unstale":"2024-04-22T20:46:05.070750+0000","last_undegraded":"2024-04-22T20:46:05.070750+0000","last_fullsized":"2024-04-22T20:46:05.070750+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:30:28.808258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":61,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145629+0000","last_change":"2024-04-22T20:19:16.794419+0000","last_active":"2024-04-22T20:44:00.145629+0000","last_peered":"2024-04-22T20:44:00.145629+0000","last_clean":"2024-04-22T20:44:00.145629+0000","last_became_active":"2024-04-22T20:19:16.793727+0000","last_became_peered":"2024-04-22T20:19:16.793727+0000","last_unstale":"2024-04-22T20:44:00.145629+0000","last_undegraded":"2024-04-22T20:44:00.145629+0000","last_fullsized":"2024-04-22T20:44:00.145629+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:50.871340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"261'5","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.188147+0000","last_change":"2024-04-22T20:05:17.899691+0000","last_active":"2024-04-22T20:46:30.188147+0000","last_peered":"2024-04-22T20:46:30.188147+0000","last_clean":"2024-04-22T20:46:30.188147+0000","last_became_active":"2024-04-22T20:05:17.899328+0000","last_became_peered":"2024-04-22T20:05:17.899328+0000","last_unstale":"2024-04-22T20:46:30.188147+0000","last_undegraded":"2024-04-22T20:46:30.188147+0000","last_fullsized":"2024-04-22T20:46:30.188147+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:57:36.265424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'7","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.717082+0000","last_change":"2024-04-22T20:05:13.812566+0000","last_active":"2024-04-22T20:48:16.717082+0000","last_peered":"2024-04-22T20:48:16.717082+0000","last_clean":"2024-04-22T20:48:16.717082+0000","last_became_active":"2024-04-22T20:05:13.811986+0000","last_became_peered":"2024-04-22T20:05:13.811986+0000","last_unstale":"2024-04-22T20:48:16.717082+0000","last_undegraded":"2024-04-22T20:48:16.717082+0000","last_fullsized":"2024-04-22T20:48:16.717082+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:29.867735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8240,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"261'66","reported_seq":212,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:05.185906+0000","last_change":"2024-04-22T20:01:04.875092+0000","last_active":"2024-04-22T20:46:05.185906+0000","last_peered":"2024-04-22T20:46:05.185906+0000","last_clean":"2024-04-22T20:46:05.185906+0000","last_became_active":"2024-04-22T20:01:04.874827+0000","last_became_peered":"2024-04-22T20:01:04.874827+0000","last_unstale":"2024-04-22T20:46:05.185906+0000","last_undegraded":"2024-04-22T20:46:05.185906+0000","last_fullsized":"2024-04-22T20:46:05.185906+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:49.143833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":281,"num_read_kb":200,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:15.052345+0000","last_change":"2024-04-22T20:19:16.794759+0000","last_active":"2024-04-22T20:44:15.052345+0000","last_peered":"2024-04-22T20:44:15.052345+0000","last_clean":"2024-04-22T20:44:15.052345+0000","last_became_active":"2024-04-22T20:19:16.794434+0000","last_became_peered":"2024-04-22T20:19:16.794434+0000","last_unstale":"2024-04-22T20:44:15.052345+0000","last_undegraded":"2024-04-22T20:44:15.052345+0000","last_fullsized":"2024-04-22T20:44:15.052345+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:47.954843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"261'9","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.888977+0000","last_change":"2024-04-22T20:05:18.209353+0000","last_active":"2024-04-22T20:48:15.888977+0000","last_peered":"2024-04-22T20:48:15.888977+0000","last_clean":"2024-04-22T20:48:15.888977+0000","last_became_active":"2024-04-22T20:05:18.209025+0000","last_became_peered":"2024-04-22T20:05:18.209025+0000","last_unstale":"2024-04-22T20:48:15.888977+0000","last_undegraded":"2024-04-22T20:48:15.888977+0000","last_fullsized":"2024-04-22T20:48:15.888977+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:14:29.836329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":13312,"num_write":14,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"261'4","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.192528+0000","last_change":"2024-04-22T20:05:13.807877+0000","last_active":"2024-04-22T20:48:16.192528+0000","last_peered":"2024-04-22T20:48:16.192528+0000","last_clean":"2024-04-22T20:48:16.192528+0000","last_became_active":"2024-04-22T20:05:13.807603+0000","last_became_peered":"2024-04-22T20:05:13.807603+0000","last_unstale":"2024-04-22T20:48:16.192528+0000","last_undegraded":"2024-04-22T20:48:16.192528+0000","last_fullsized":"2024-04-22T20:48:16.192528+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:05:08.359131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"258'23","reported_seq":112,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:35.055220+0000","last_change":"2024-04-22T20:01:04.869170+0000","last_active":"2024-04-22T20:44:35.055220+0000","last_peered":"2024-04-22T20:44:35.055220+0000","last_clean":"2024-04-22T20:44:35.055220+0000","last_became_active":"2024-04-22T20:01:04.868847+0000","last_became_peered":"2024-04-22T20:01:04.868847+0000","last_unstale":"2024-04-22T20:44:35.055220+0000","last_undegraded":"2024-04-22T20:44:35.055220+0000","last_fullsized":"2024-04-22T20:44:35.055220+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:17:28.313341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":76,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.050611+0000","last_change":"2024-04-22T20:19:16.796622+0000","last_active":"2024-04-22T20:44:05.050611+0000","last_peered":"2024-04-22T20:44:05.050611+0000","last_clean":"2024-04-22T20:44:05.050611+0000","last_became_active":"2024-04-22T20:19:16.795613+0000","last_became_peered":"2024-04-22T20:19:16.795613+0000","last_unstale":"2024-04-22T20:44:05.050611+0000","last_undegraded":"2024-04-22T20:44:05.050611+0000","last_fullsized":"2024-04-22T20:44:05.050611+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:22:45.476024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"261'6","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.126009+0000","last_change":"2024-04-22T20:05:17.891585+0000","last_active":"2024-04-22T20:48:17.126009+0000","last_peered":"2024-04-22T20:48:17.126009+0000","last_clean":"2024-04-22T20:48:17.126009+0000","last_became_active":"2024-04-22T20:05:17.891331+0000","last_became_peered":"2024-04-22T20:05:17.891331+0000","last_unstale":"2024-04-22T20:48:17.126009+0000","last_undegraded":"2024-04-22T20:48:17.126009+0000","last_fullsized":"2024-04-22T20:48:17.126009+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:46:10.241060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.173996+0000","last_change":"2024-04-22T20:05:13.814072+0000","last_active":"2024-04-22T20:44:25.173996+0000","last_peered":"2024-04-22T20:44:25.173996+0000","last_clean":"2024-04-22T20:44:25.173996+0000","last_became_active":"2024-04-22T20:05:13.813811+0000","last_became_peered":"2024-04-22T20:05:13.813811+0000","last_unstale":"2024-04-22T20:44:25.173996+0000","last_undegraded":"2024-04-22T20:44:25.173996+0000","last_fullsized":"2024-04-22T20:44:25.173996+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:57.039895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"258'49","reported_seq":170,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.148654+0000","last_change":"2024-04-22T20:01:04.874422+0000","last_active":"2024-04-22T20:44:25.148654+0000","last_peered":"2024-04-22T20:44:25.148654+0000","last_clean":"2024-04-22T20:44:25.148654+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:44:25.148654+0000","last_undegraded":"2024-04-22T20:44:25.148654+0000","last_fullsized":"2024-04-22T20:44:25.148654+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:56:32.298685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":145,"num_write":78,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.171678+0000","last_change":"2024-04-22T20:19:16.793813+0000","last_active":"2024-04-22T20:44:05.171678+0000","last_peered":"2024-04-22T20:44:05.171678+0000","last_clean":"2024-04-22T20:44:05.171678+0000","last_became_active":"2024-04-22T20:19:16.793486+0000","last_became_peered":"2024-04-22T20:19:16.793486+0000","last_unstale":"2024-04-22T20:44:05.171678+0000","last_undegraded":"2024-04-22T20:44:05.171678+0000","last_fullsized":"2024-04-22T20:44:05.171678+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:47:12.878940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:30.054561+0000","last_change":"2024-04-22T20:05:17.897286+0000","last_active":"2024-04-22T20:44:30.054561+0000","last_peered":"2024-04-22T20:44:30.054561+0000","last_clean":"2024-04-22T20:44:30.054561+0000","last_became_active":"2024-04-22T20:05:17.897008+0000","last_became_peered":"2024-04-22T20:05:17.897008+0000","last_unstale":"2024-04-22T20:44:30.054561+0000","last_undegraded":"2024-04-22T20:44:30.054561+0000","last_fullsized":"2024-04-22T20:44:30.054561+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:56:58.848648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"261'13","reported_seq":74,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.642198+0000","last_change":"2024-04-22T20:05:13.798707+0000","last_active":"2024-04-22T20:48:16.642198+0000","last_peered":"2024-04-22T20:48:16.642198+0000","last_clean":"2024-04-22T20:48:16.642198+0000","last_became_active":"2024-04-22T20:05:13.798319+0000","last_became_peered":"2024-04-22T20:05:13.798319+0000","last_unstale":"2024-04-22T20:48:16.642198+0000","last_undegraded":"2024-04-22T20:48:16.642198+0000","last_fullsized":"2024-04-22T20:48:16.642198+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:02.215898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":16388,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"261'37","reported_seq":138,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.306289+0000","last_change":"2024-04-22T20:01:04.869368+0000","last_active":"2024-04-22T20:44:25.306289+0000","last_peered":"2024-04-22T20:44:25.306289+0000","last_clean":"2024-04-22T20:44:25.306289+0000","last_became_active":"2024-04-22T20:01:04.869079+0000","last_became_peered":"2024-04-22T20:01:04.869079+0000","last_unstale":"2024-04-22T20:44:25.306289+0000","last_undegraded":"2024-04-22T20:44:25.306289+0000","last_fullsized":"2024-04-22T20:44:25.306289+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:36:35.177072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":61,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053026+0000","last_change":"2024-04-22T20:19:16.796317+0000","last_active":"2024-04-22T20:44:20.053026+0000","last_peered":"2024-04-22T20:44:20.053026+0000","last_clean":"2024-04-22T20:44:20.053026+0000","last_became_active":"2024-04-22T20:19:16.795462+0000","last_became_peered":"2024-04-22T20:19:16.795462+0000","last_unstale":"2024-04-22T20:44:20.053026+0000","last_undegraded":"2024-04-22T20:44:20.053026+0000","last_fullsized":"2024-04-22T20:44:20.053026+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:38:48.277835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"258'6","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:05.178822+0000","last_change":"2024-04-22T20:05:17.892351+0000","last_active":"2024-04-22T20:45:05.178822+0000","last_peered":"2024-04-22T20:45:05.178822+0000","last_clean":"2024-04-22T20:45:05.178822+0000","last_became_active":"2024-04-22T20:05:17.892044+0000","last_became_peered":"2024-04-22T20:05:17.892044+0000","last_unstale":"2024-04-22T20:45:05.178822+0000","last_undegraded":"2024-04-22T20:45:05.178822+0000","last_fullsized":"2024-04-22T20:45:05.178822+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:47:37.172131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"258'4","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.057775+0000","last_change":"2024-04-22T20:05:13.809668+0000","last_active":"2024-04-22T20:44:50.057775+0000","last_peered":"2024-04-22T20:44:50.057775+0000","last_clean":"2024-04-22T20:44:50.057775+0000","last_became_active":"2024-04-22T20:05:13.809368+0000","last_became_peered":"2024-04-22T20:05:13.809368+0000","last_unstale":"2024-04-22T20:44:50.057775+0000","last_undegraded":"2024-04-22T20:44:50.057775+0000","last_fullsized":"2024-04-22T20:44:50.057775+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:35.986772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"261'35","reported_seq":138,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.174074+0000","last_change":"2024-04-22T20:01:04.874032+0000","last_active":"2024-04-22T20:44:25.174074+0000","last_peered":"2024-04-22T20:44:25.174074+0000","last_clean":"2024-04-22T20:44:25.174074+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:44:25.174074+0000","last_undegraded":"2024-04-22T20:44:25.174074+0000","last_fullsized":"2024-04-22T20:44:25.174074+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:36:44.638764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":59,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.177834+0000","last_change":"2024-04-22T20:19:16.791940+0000","last_active":"2024-04-22T20:44:20.177834+0000","last_peered":"2024-04-22T20:44:20.177834+0000","last_clean":"2024-04-22T20:44:20.177834+0000","last_became_active":"2024-04-22T20:19:16.791663+0000","last_became_peered":"2024-04-22T20:19:16.791663+0000","last_unstale":"2024-04-22T20:44:20.177834+0000","last_undegraded":"2024-04-22T20:44:20.177834+0000","last_fullsized":"2024-04-22T20:44:20.177834+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:35:00.807867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"261'6","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:00.192199+0000","last_change":"2024-04-22T20:05:18.211014+0000","last_active":"2024-04-22T20:47:00.192199+0000","last_peered":"2024-04-22T20:47:00.192199+0000","last_clean":"2024-04-22T20:47:00.192199+0000","last_became_active":"2024-04-22T20:05:18.210736+0000","last_became_peered":"2024-04-22T20:05:18.210736+0000","last_unstale":"2024-04-22T20:47:00.192199+0000","last_undegraded":"2024-04-22T20:47:00.192199+0000","last_fullsized":"2024-04-22T20:47:00.192199+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:55:21.676210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20482,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"261'6","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.076639+0000","last_change":"2024-04-22T20:05:13.813554+0000","last_active":"2024-04-22T20:47:05.076639+0000","last_peered":"2024-04-22T20:47:05.076639+0000","last_clean":"2024-04-22T20:47:05.076639+0000","last_became_active":"2024-04-22T20:05:13.813277+0000","last_became_peered":"2024-04-22T20:05:13.813277+0000","last_unstale":"2024-04-22T20:47:05.076639+0000","last_undegraded":"2024-04-22T20:47:05.076639+0000","last_fullsized":"2024-04-22T20:47:05.076639+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:43:53.225799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1024,"num_write":16,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"261'61","reported_seq":194,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:25.306363+0000","last_change":"2024-04-22T20:01:04.872160+0000","last_active":"2024-04-22T20:44:25.306363+0000","last_peered":"2024-04-22T20:44:25.306363+0000","last_clean":"2024-04-22T20:44:25.306363+0000","last_became_active":"2024-04-22T20:01:04.871859+0000","last_became_peered":"2024-04-22T20:01:04.871859+0000","last_unstale":"2024-04-22T20:44:25.306363+0000","last_undegraded":"2024-04-22T20:44:25.306363+0000","last_fullsized":"2024-04-22T20:44:25.306363+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:17.093967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":172,"num_write":109,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.044253+0000","last_change":"2024-04-22T20:19:16.790599+0000","last_active":"2024-04-22T20:45:25.044253+0000","last_peered":"2024-04-22T20:45:25.044253+0000","last_clean":"2024-04-22T20:45:25.044253+0000","last_became_active":"2024-04-22T20:19:16.789688+0000","last_became_peered":"2024-04-22T20:19:16.789688+0000","last_unstale":"2024-04-22T20:45:25.044253+0000","last_undegraded":"2024-04-22T20:45:25.044253+0000","last_fullsized":"2024-04-22T20:45:25.044253+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:11.661623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"258'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.053135+0000","last_change":"2024-04-22T20:05:18.211736+0000","last_active":"2024-04-22T20:44:20.053135+0000","last_peered":"2024-04-22T20:44:20.053135+0000","last_clean":"2024-04-22T20:44:20.053135+0000","last_became_active":"2024-04-22T20:05:18.211478+0000","last_became_peered":"2024-04-22T20:05:18.211478+0000","last_unstale":"2024-04-22T20:44:20.053135+0000","last_undegraded":"2024-04-22T20:44:20.053135+0000","last_fullsized":"2024-04-22T20:44:20.053135+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:06:10.866642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'7","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.246312+0000","last_change":"2024-04-22T20:05:13.811784+0000","last_active":"2024-04-22T20:48:17.246312+0000","last_peered":"2024-04-22T20:48:17.246312+0000","last_clean":"2024-04-22T20:48:17.246312+0000","last_became_active":"2024-04-22T20:05:13.811561+0000","last_became_peered":"2024-04-22T20:05:13.811561+0000","last_unstale":"2024-04-22T20:48:17.246312+0000","last_undegraded":"2024-04-22T20:48:17.246312+0000","last_fullsized":"2024-04-22T20:48:17.246312+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:35:04.990469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":12,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"261'33","reported_seq":135,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:00.059415+0000","last_change":"2024-04-22T20:01:04.875229+0000","last_active":"2024-04-22T20:45:00.059415+0000","last_peered":"2024-04-22T20:45:00.059415+0000","last_clean":"2024-04-22T20:45:00.059415+0000","last_became_active":"2024-04-22T20:01:04.874790+0000","last_became_peered":"2024-04-22T20:01:04.874790+0000","last_unstale":"2024-04-22T20:45:00.059415+0000","last_undegraded":"2024-04-22T20:45:00.059415+0000","last_fullsized":"2024-04-22T20:45:00.059415+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:37:48.031491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":100,"num_write":57,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.169578+0000","last_change":"2024-04-22T20:19:16.794065+0000","last_active":"2024-04-22T20:44:20.169578+0000","last_peered":"2024-04-22T20:44:20.169578+0000","last_clean":"2024-04-22T20:44:20.169578+0000","last_became_active":"2024-04-22T20:19:16.793188+0000","last_became_peered":"2024-04-22T20:19:16.793188+0000","last_unstale":"2024-04-22T20:44:20.169578+0000","last_undegraded":"2024-04-22T20:44:20.169578+0000","last_fullsized":"2024-04-22T20:44:20.169578+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:00:39.203315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"261'5","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.736453+0000","last_change":"2024-04-22T20:05:17.901392+0000","last_active":"2024-04-22T20:48:15.736453+0000","last_peered":"2024-04-22T20:48:15.736453+0000","last_clean":"2024-04-22T20:48:15.736453+0000","last_became_active":"2024-04-22T20:05:17.901128+0000","last_became_peered":"2024-04-22T20:05:17.901128+0000","last_unstale":"2024-04-22T20:48:15.736453+0000","last_undegraded":"2024-04-22T20:48:15.736453+0000","last_fullsized":"2024-04-22T20:48:15.736453+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:50:08.370833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":8192,"num_write":10,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"261'5","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:25.072353+0000","last_change":"2024-04-22T20:05:13.815731+0000","last_active":"2024-04-22T20:46:25.072353+0000","last_peered":"2024-04-22T20:46:25.072353+0000","last_clean":"2024-04-22T20:46:25.072353+0000","last_became_active":"2024-04-22T20:05:13.815332+0000","last_became_peered":"2024-04-22T20:05:13.815332+0000","last_unstale":"2024-04-22T20:46:25.072353+0000","last_undegraded":"2024-04-22T20:46:25.072353+0000","last_fullsized":"2024-04-22T20:46:25.072353+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:07:16.209328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":9216,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'28","reported_seq":124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040298+0000","last_change":"2024-04-22T20:01:04.872148+0000","last_active":"2024-04-22T20:44:50.040298+0000","last_peered":"2024-04-22T20:44:50.040298+0000","last_clean":"2024-04-22T20:44:50.040298+0000","last_became_active":"2024-04-22T20:01:04.871840+0000","last_became_peered":"2024-04-22T20:01:04.871840+0000","last_unstale":"2024-04-22T20:44:50.040298+0000","last_undegraded":"2024-04-22T20:44:50.040298+0000","last_fullsized":"2024-04-22T20:44:50.040298+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:23.933273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":89,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"261'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.066551+0000","last_change":"2024-04-22T20:19:16.794411+0000","last_active":"2024-04-22T20:45:20.066551+0000","last_peered":"2024-04-22T20:45:20.066551+0000","last_clean":"2024-04-22T20:45:20.066551+0000","last_became_active":"2024-04-22T20:19:16.794123+0000","last_became_peered":"2024-04-22T20:19:16.794123+0000","last_unstale":"2024-04-22T20:45:20.066551+0000","last_undegraded":"2024-04-22T20:45:20.066551+0000","last_fullsized":"2024-04-22T20:45:20.066551+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:39.176302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"261'4","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.735524+0000","last_change":"2024-04-22T20:05:17.895539+0000","last_active":"2024-04-22T20:48:15.735524+0000","last_peered":"2024-04-22T20:48:15.735524+0000","last_clean":"2024-04-22T20:48:15.735524+0000","last_became_active":"2024-04-22T20:05:17.895177+0000","last_became_peered":"2024-04-22T20:05:17.895177+0000","last_unstale":"2024-04-22T20:48:15.735524+0000","last_undegraded":"2024-04-22T20:48:15.735524+0000","last_fullsized":"2024-04-22T20:48:15.735524+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:46:34.641098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":4,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"261'4","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.248000+0000","last_change":"2024-04-22T20:05:13.809357+0000","last_active":"2024-04-22T20:48:17.248000+0000","last_peered":"2024-04-22T20:48:17.248000+0000","last_clean":"2024-04-22T20:48:17.248000+0000","last_became_active":"2024-04-22T20:05:13.809047+0000","last_became_peered":"2024-04-22T20:05:13.809047+0000","last_unstale":"2024-04-22T20:48:17.248000+0000","last_undegraded":"2024-04-22T20:48:17.248000+0000","last_fullsized":"2024-04-22T20:48:17.248000+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:17.206006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"261'33","reported_seq":130,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.181926+0000","last_change":"2024-04-22T20:01:04.872953+0000","last_active":"2024-04-22T20:44:50.181926+0000","last_peered":"2024-04-22T20:44:50.181926+0000","last_clean":"2024-04-22T20:44:50.181926+0000","last_became_active":"2024-04-22T20:01:04.872370+0000","last_became_peered":"2024-04-22T20:01:04.872370+0000","last_unstale":"2024-04-22T20:44:50.181926+0000","last_undegraded":"2024-04-22T20:44:50.181926+0000","last_fullsized":"2024-04-22T20:44:50.181926+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:22:37.771203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":57,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.180655+0000","last_change":"2024-04-22T20:19:16.793728+0000","last_active":"2024-04-22T20:45:20.180655+0000","last_peered":"2024-04-22T20:45:20.180655+0000","last_clean":"2024-04-22T20:45:20.180655+0000","last_became_active":"2024-04-22T20:19:16.793429+0000","last_became_peered":"2024-04-22T20:19:16.793429+0000","last_unstale":"2024-04-22T20:45:20.180655+0000","last_undegraded":"2024-04-22T20:45:20.180655+0000","last_fullsized":"2024-04-22T20:45:20.180655+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:27:12.156563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"261'2","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.205910+0000","last_change":"2024-04-22T20:05:17.901476+0000","last_active":"2024-04-22T20:47:30.205910+0000","last_peered":"2024-04-22T20:47:30.205910+0000","last_clean":"2024-04-22T20:47:30.205910+0000","last_became_active":"2024-04-22T20:05:17.900874+0000","last_became_peered":"2024-04-22T20:05:17.900874+0000","last_unstale":"2024-04-22T20:47:30.205910+0000","last_undegraded":"2024-04-22T20:47:30.205910+0000","last_fullsized":"2024-04-22T20:47:30.205910+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:20:07.700066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"261'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:55.076775+0000","last_change":"2024-04-22T20:05:13.814214+0000","last_active":"2024-04-22T20:46:55.076775+0000","last_peered":"2024-04-22T20:46:55.076775+0000","last_clean":"2024-04-22T20:46:55.076775+0000","last_became_active":"2024-04-22T20:05:13.813934+0000","last_became_peered":"2024-04-22T20:05:13.813934+0000","last_unstale":"2024-04-22T20:46:55.076775+0000","last_undegraded":"2024-04-22T20:46:55.076775+0000","last_fullsized":"2024-04-22T20:46:55.076775+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:15.355174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1024,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"261'37","reported_seq":146,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.173934+0000","last_change":"2024-04-22T20:01:04.869971+0000","last_active":"2024-04-22T20:44:50.173934+0000","last_peered":"2024-04-22T20:44:50.173934+0000","last_clean":"2024-04-22T20:44:50.173934+0000","last_became_active":"2024-04-22T20:01:04.869660+0000","last_became_peered":"2024-04-22T20:01:04.869660+0000","last_unstale":"2024-04-22T20:44:50.173934+0000","last_undegraded":"2024-04-22T20:44:50.173934+0000","last_fullsized":"2024-04-22T20:44:50.173934+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:08:02.708029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":113,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:20.173415+0000","last_change":"2024-04-22T20:19:16.795977+0000","last_active":"2024-04-22T20:44:20.173415+0000","last_peered":"2024-04-22T20:44:20.173415+0000","last_clean":"2024-04-22T20:44:20.173415+0000","last_became_active":"2024-04-22T20:19:16.795509+0000","last_became_peered":"2024-04-22T20:19:16.795509+0000","last_unstale":"2024-04-22T20:44:20.173415+0000","last_undegraded":"2024-04-22T20:44:20.173415+0000","last_fullsized":"2024-04-22T20:44:20.173415+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:07.456021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:55.310587+0000","last_change":"2024-04-22T20:05:17.891942+0000","last_active":"2024-04-22T20:44:55.310587+0000","last_peered":"2024-04-22T20:44:55.310587+0000","last_clean":"2024-04-22T20:44:55.310587+0000","last_became_active":"2024-04-22T20:05:17.891523+0000","last_became_peered":"2024-04-22T20:05:17.891523+0000","last_unstale":"2024-04-22T20:44:55.310587+0000","last_undegraded":"2024-04-22T20:44:55.310587+0000","last_fullsized":"2024-04-22T20:44:55.310587+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:38.553189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"261'2","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:55.054041+0000","last_change":"2024-04-22T20:05:13.805972+0000","last_active":"2024-04-22T20:46:55.054041+0000","last_peered":"2024-04-22T20:46:55.054041+0000","last_clean":"2024-04-22T20:46:55.054041+0000","last_became_active":"2024-04-22T20:05:13.805704+0000","last_became_peered":"2024-04-22T20:05:13.805704+0000","last_unstale":"2024-04-22T20:46:55.054041+0000","last_undegraded":"2024-04-22T20:46:55.054041+0000","last_fullsized":"2024-04-22T20:46:55.054041+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:03.816391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"261'65","reported_seq":224,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309884+0000","last_change":"2024-04-22T20:01:04.866275+0000","last_active":"2024-04-22T20:44:50.309884+0000","last_peered":"2024-04-22T20:44:50.309884+0000","last_clean":"2024-04-22T20:44:50.309884+0000","last_became_active":"2024-04-22T20:01:04.865981+0000","last_became_peered":"2024-04-22T20:01:04.865981+0000","last_unstale":"2024-04-22T20:44:50.309884+0000","last_undegraded":"2024-04-22T20:44:50.309884+0000","last_fullsized":"2024-04-22T20:44:50.309884+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:25.969487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":226,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:30.054611+0000","last_change":"2024-04-22T20:19:16.799325+0000","last_active":"2024-04-22T20:44:30.054611+0000","last_peered":"2024-04-22T20:44:30.054611+0000","last_clean":"2024-04-22T20:44:30.054611+0000","last_became_active":"2024-04-22T20:19:16.799063+0000","last_became_peered":"2024-04-22T20:19:16.799063+0000","last_unstale":"2024-04-22T20:44:30.054611+0000","last_undegraded":"2024-04-22T20:44:30.054611+0000","last_fullsized":"2024-04-22T20:44:30.054611+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:20:02.254828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"261'2","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:00.192644+0000","last_change":"2024-04-22T20:05:17.901995+0000","last_active":"2024-04-22T20:47:00.192644+0000","last_peered":"2024-04-22T20:47:00.192644+0000","last_clean":"2024-04-22T20:47:00.192644+0000","last_became_active":"2024-04-22T20:05:17.901680+0000","last_became_peered":"2024-04-22T20:05:17.901680+0000","last_unstale":"2024-04-22T20:47:00.192644+0000","last_undegraded":"2024-04-22T20:47:00.192644+0000","last_fullsized":"2024-04-22T20:47:00.192644+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:26.628494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":80,"num_write":2,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309933+0000","last_change":"2024-04-22T20:05:13.811164+0000","last_active":"2024-04-22T20:44:50.309933+0000","last_peered":"2024-04-22T20:44:50.309933+0000","last_clean":"2024-04-22T20:44:50.309933+0000","last_became_active":"2024-04-22T20:05:13.810865+0000","last_became_peered":"2024-04-22T20:05:13.810865+0000","last_unstale":"2024-04-22T20:44:50.309933+0000","last_undegraded":"2024-04-22T20:44:50.309933+0000","last_fullsized":"2024-04-22T20:44:50.309933+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:47:15.297255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"261'57","reported_seq":180,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.061413+0000","last_change":"2024-04-22T20:01:04.870803+0000","last_active":"2024-04-22T20:45:15.061413+0000","last_peered":"2024-04-22T20:45:15.061413+0000","last_clean":"2024-04-22T20:45:15.061413+0000","last_became_active":"2024-04-22T20:01:04.870253+0000","last_became_peered":"2024-04-22T20:01:04.870253+0000","last_unstale":"2024-04-22T20:45:15.061413+0000","last_undegraded":"2024-04-22T20:45:15.061413+0000","last_fullsized":"2024-04-22T20:45:15.061413+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:01:03.850280+0000","last_clean_scrub_stamp":"2024-04-22T20:01:03.850280+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:32:34.069500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":145,"num_write":102,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"261'344","reported_seq":951,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.915979+0000","last_change":"2024-04-22T19:53:04.823797+0000","last_active":"2024-04-22T20:48:17.915979+0000","last_peered":"2024-04-22T20:48:17.915979+0000","last_clean":"2024-04-22T20:48:17.915979+0000","last_became_active":"2024-04-22T19:53:04.823351+0000","last_became_peered":"2024-04-22T19:53:04.823351+0000","last_unstale":"2024-04-22T20:48:17.915979+0000","last_undegraded":"2024-04-22T20:48:17.915979+0000","last_fullsized":"2024-04-22T20:48:17.915979+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:36:24.940347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":495,"num_write":1298,"num_write_kb":39497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"261'169","reported_seq":763,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174873+0000","last_change":"2024-04-22T19:51:28.511862+0000","last_active":"2024-04-22T20:44:00.174873+0000","last_peered":"2024-04-22T20:44:00.174873+0000","last_clean":"2024-04-22T20:44:00.174873+0000","last_became_active":"2024-04-22T19:51:28.511595+0000","last_became_peered":"2024-04-22T19:51:28.511595+0000","last_unstale":"2024-04-22T20:44:00.174873+0000","last_undegraded":"2024-04-22T20:44:00.174873+0000","last_fullsized":"2024-04-22T20:44:00.174873+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:36.200495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":537,"num_write":262,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"261'316","reported_seq":967,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.907148+0000","last_change":"2024-04-22T19:51:26.496852+0000","last_active":"2024-04-22T20:48:17.907148+0000","last_peered":"2024-04-22T20:48:17.907148+0000","last_clean":"2024-04-22T20:48:17.907148+0000","last_became_active":"2024-04-22T19:51:26.496188+0000","last_became_peered":"2024-04-22T19:51:26.496188+0000","last_unstale":"2024-04-22T20:48:17.907148+0000","last_undegraded":"2024-04-22T20:48:17.907148+0000","last_fullsized":"2024-04-22T20:48:17.907148+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:51:56.333450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5719773,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":497,"num_read_kb":9758,"num_write":1112,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"261'314","reported_seq":886,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:28.101937+0000","last_change":"2024-04-22T19:51:28.519482+0000","last_active":"2024-04-22T20:47:28.101937+0000","last_peered":"2024-04-22T20:47:28.101937+0000","last_clean":"2024-04-22T20:47:28.101937+0000","last_became_active":"2024-04-22T19:51:28.519111+0000","last_became_peered":"2024-04-22T19:51:28.519111+0000","last_unstale":"2024-04-22T20:47:28.101937+0000","last_undegraded":"2024-04-22T20:47:28.101937+0000","last_fullsized":"2024-04-22T20:47:28.101937+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:06:08.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1190,"num_read_kb":913,"num_write":510,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"261'320","reported_seq":941,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.882599+0000","last_change":"2024-04-22T19:51:26.503143+0000","last_active":"2024-04-22T20:48:17.882599+0000","last_peered":"2024-04-22T20:48:17.882599+0000","last_clean":"2024-04-22T20:48:17.882599+0000","last_became_active":"2024-04-22T19:51:26.502744+0000","last_became_peered":"2024-04-22T19:51:26.502744+0000","last_unstale":"2024-04-22T20:48:17.882599+0000","last_undegraded":"2024-04-22T20:48:17.882599+0000","last_fullsized":"2024-04-22T20:48:17.882599+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:30:34.457597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437211,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":443,"num_read_kb":7116,"num_write":1035,"num_write_kb":21496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"261'150","reported_seq":726,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166689+0000","last_change":"2024-04-22T19:51:28.518949+0000","last_active":"2024-04-22T20:44:00.166689+0000","last_peered":"2024-04-22T20:44:00.166689+0000","last_clean":"2024-04-22T20:44:00.166689+0000","last_became_active":"2024-04-22T19:51:28.518650+0000","last_became_peered":"2024-04-22T19:51:28.518650+0000","last_unstale":"2024-04-22T20:44:00.166689+0000","last_undegraded":"2024-04-22T20:44:00.166689+0000","last_fullsized":"2024-04-22T20:44:00.166689+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:25:03.391586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":572,"num_read_kb":453,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"261'324","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.932240+0000","last_change":"2024-04-22T19:51:26.504934+0000","last_active":"2024-04-22T20:48:17.932240+0000","last_peered":"2024-04-22T20:48:17.932240+0000","last_clean":"2024-04-22T20:48:17.932240+0000","last_became_active":"2024-04-22T19:51:26.504450+0000","last_became_peered":"2024-04-22T19:51:26.504450+0000","last_unstale":"2024-04-22T20:48:17.932240+0000","last_undegraded":"2024-04-22T20:48:17.932240+0000","last_fullsized":"2024-04-22T20:48:17.932240+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:47:54.670392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8563195,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":424,"num_read_kb":25508,"num_write":1190,"num_write_kb":37655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"261'81","reported_seq":676,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.171137+0000","last_change":"2024-04-22T19:52:04.620094+0000","last_active":"2024-04-22T20:44:00.171137+0000","last_peered":"2024-04-22T20:44:00.171137+0000","last_clean":"2024-04-22T20:44:00.171137+0000","last_became_active":"2024-04-22T19:52:04.619688+0000","last_became_peered":"2024-04-22T19:52:04.619688+0000","last_unstale":"2024-04-22T20:44:00.171137+0000","last_undegraded":"2024-04-22T20:44:00.171137+0000","last_fullsized":"2024-04-22T20:44:00.171137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:50:37.537623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":353,"num_read_kb":299,"num_write":120,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"261'281","reported_seq":969,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.923799+0000","last_change":"2024-04-22T19:51:26.509141+0000","last_active":"2024-04-22T20:48:17.923799+0000","last_peered":"2024-04-22T20:48:17.923799+0000","last_clean":"2024-04-22T20:48:17.923799+0000","last_became_active":"2024-04-22T19:51:26.507185+0000","last_became_peered":"2024-04-22T19:51:26.507185+0000","last_unstale":"2024-04-22T20:48:17.923799+0000","last_undegraded":"2024-04-22T20:48:17.923799+0000","last_fullsized":"2024-04-22T20:48:17.923799+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:14.928877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3508486,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":442,"num_read_kb":10200,"num_write":925,"num_write_kb":30630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"261'40","reported_seq":594,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.316183+0000","last_change":"2024-04-22T19:51:28.513981+0000","last_active":"2024-04-22T20:45:35.316183+0000","last_peered":"2024-04-22T20:45:35.316183+0000","last_clean":"2024-04-22T20:45:35.316183+0000","last_became_active":"2024-04-22T19:51:28.513693+0000","last_became_peered":"2024-04-22T19:51:28.513693+0000","last_unstale":"2024-04-22T20:45:35.316183+0000","last_undegraded":"2024-04-22T20:45:35.316183+0000","last_fullsized":"2024-04-22T20:45:35.316183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:54:32.701108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"261'444","reported_seq":1186,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.807802+0000","last_change":"2024-04-22T19:51:26.509287+0000","last_active":"2024-04-22T20:48:17.807802+0000","last_peered":"2024-04-22T20:48:17.807802+0000","last_clean":"2024-04-22T20:48:17.807802+0000","last_became_active":"2024-04-22T19:51:26.508716+0000","last_became_peered":"2024-04-22T19:51:26.508716+0000","last_unstale":"2024-04-22T20:48:17.807802+0000","last_undegraded":"2024-04-22T20:48:17.807802+0000","last_fullsized":"2024-04-22T20:48:17.807802+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":400,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:42:06.130701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661200,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1035,"num_read_kb":17244,"num_write":1523,"num_write_kb":25883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"261'144","reported_seq":742,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181403+0000","last_change":"2024-04-22T19:51:28.519053+0000","last_active":"2024-04-22T20:45:40.181403+0000","last_peered":"2024-04-22T20:45:40.181403+0000","last_clean":"2024-04-22T20:45:40.181403+0000","last_became_active":"2024-04-22T19:51:28.518501+0000","last_became_peered":"2024-04-22T19:51:28.518501+0000","last_unstale":"2024-04-22T20:45:40.181403+0000","last_undegraded":"2024-04-22T20:45:40.181403+0000","last_fullsized":"2024-04-22T20:45:40.181403+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:16:36.369584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":597,"num_read_kb":478,"num_write":226,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"261'377","reported_seq":1038,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.928955+0000","last_change":"2024-04-22T19:51:26.508109+0000","last_active":"2024-04-22T20:48:17.928955+0000","last_peered":"2024-04-22T20:48:17.928955+0000","last_clean":"2024-04-22T20:48:17.928955+0000","last_became_active":"2024-04-22T19:51:26.507141+0000","last_became_peered":"2024-04-22T19:51:26.507141+0000","last_unstale":"2024-04-22T20:48:17.928955+0000","last_undegraded":"2024-04-22T20:48:17.928955+0000","last_fullsized":"2024-04-22T20:48:17.928955+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:27.201168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2367620,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":550,"num_read_kb":43743,"num_write":1389,"num_write_kb":32515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"261'151","reported_seq":736,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:38.067532+0000","last_change":"2024-04-22T19:51:28.516437+0000","last_active":"2024-04-22T20:47:38.067532+0000","last_peered":"2024-04-22T20:47:38.067532+0000","last_clean":"2024-04-22T20:47:38.067532+0000","last_became_active":"2024-04-22T19:51:28.516012+0000","last_became_peered":"2024-04-22T19:51:28.516012+0000","last_unstale":"2024-04-22T20:47:38.067532+0000","last_undegraded":"2024-04-22T20:47:38.067532+0000","last_fullsized":"2024-04-22T20:47:38.067532+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:38.182307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":470,"num_write":219,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"261'333","reported_seq":977,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.910458+0000","last_change":"2024-04-22T19:51:26.505832+0000","last_active":"2024-04-22T20:48:17.910458+0000","last_peered":"2024-04-22T20:48:17.910458+0000","last_clean":"2024-04-22T20:48:17.910458+0000","last_became_active":"2024-04-22T19:51:26.502928+0000","last_became_peered":"2024-04-22T19:51:26.502928+0000","last_unstale":"2024-04-22T20:48:17.910458+0000","last_undegraded":"2024-04-22T20:48:17.910458+0000","last_fullsized":"2024-04-22T20:48:17.910458+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:47:30.201226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5414404,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":443,"num_read_kb":51849,"num_write":1202,"num_write_kb":39313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"261'60","reported_seq":629,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:57.506260+0000","last_change":"2024-04-22T19:51:28.510044+0000","last_active":"2024-04-22T20:46:57.506260+0000","last_peered":"2024-04-22T20:46:57.506260+0000","last_clean":"2024-04-22T20:46:57.506260+0000","last_became_active":"2024-04-22T19:51:28.509774+0000","last_became_peered":"2024-04-22T19:51:28.509774+0000","last_unstale":"2024-04-22T20:46:57.506260+0000","last_undegraded":"2024-04-22T20:46:57.506260+0000","last_fullsized":"2024-04-22T20:46:57.506260+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:55:37.172957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":211,"num_write":81,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"261'125","reported_seq":734,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176694+0000","last_change":"2024-04-22T19:51:28.505929+0000","last_active":"2024-04-22T20:45:10.176694+0000","last_peered":"2024-04-22T20:45:10.176694+0000","last_clean":"2024-04-22T20:45:10.176694+0000","last_became_active":"2024-04-22T19:51:28.505562+0000","last_became_peered":"2024-04-22T19:51:28.505562+0000","last_unstale":"2024-04-22T20:45:10.176694+0000","last_undegraded":"2024-04-22T20:45:10.176694+0000","last_fullsized":"2024-04-22T20:45:10.176694+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:33:32.464020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":540,"num_read_kb":443,"num_write":193,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"261'307","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.820884+0000","last_change":"2024-04-22T19:51:26.507324+0000","last_active":"2024-04-22T20:48:17.820884+0000","last_peered":"2024-04-22T20:48:17.820884+0000","last_clean":"2024-04-22T20:48:17.820884+0000","last_became_active":"2024-04-22T19:51:26.504717+0000","last_became_peered":"2024-04-22T19:51:26.504717+0000","last_unstale":"2024-04-22T20:48:17.820884+0000","last_undegraded":"2024-04-22T20:48:17.820884+0000","last_fullsized":"2024-04-22T20:48:17.820884+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:31:10.322998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10266,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":28705,"num_write":1110,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"261'105","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060776+0000","last_change":"2024-04-22T19:51:28.519503+0000","last_active":"2024-04-22T20:45:10.060776+0000","last_peered":"2024-04-22T20:45:10.060776+0000","last_clean":"2024-04-22T20:45:10.060776+0000","last_became_active":"2024-04-22T19:51:28.519230+0000","last_became_peered":"2024-04-22T19:51:28.519230+0000","last_unstale":"2024-04-22T20:45:10.060776+0000","last_undegraded":"2024-04-22T20:45:10.060776+0000","last_fullsized":"2024-04-22T20:45:10.060776+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:23:00.702297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":465,"num_read_kb":383,"num_write":180,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"261'381","reported_seq":1038,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.866410+0000","last_change":"2024-04-22T19:51:26.496564+0000","last_active":"2024-04-22T20:48:17.866410+0000","last_peered":"2024-04-22T20:48:17.866410+0000","last_clean":"2024-04-22T20:48:17.866410+0000","last_became_active":"2024-04-22T19:51:26.496156+0000","last_became_peered":"2024-04-22T19:51:26.496156+0000","last_unstale":"2024-04-22T20:48:17.866410+0000","last_undegraded":"2024-04-22T20:48:17.866410+0000","last_fullsized":"2024-04-22T20:48:17.866410+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":300,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:25.947249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":611,"num_read_kb":792,"num_write":1516,"num_write_kb":25865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"261'172","reported_seq":764,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186369+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:45:20.186369+0000","last_peered":"2024-04-22T20:45:20.186369+0000","last_clean":"2024-04-22T20:45:20.186369+0000","last_became_active":"2024-04-22T19:51:28.515673+0000","last_became_peered":"2024-04-22T19:51:28.515673+0000","last_unstale":"2024-04-22T20:45:20.186369+0000","last_undegraded":"2024-04-22T20:45:20.186369+0000","last_fullsized":"2024-04-22T20:45:20.186369+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:06.001104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":666,"num_read_kb":529,"num_write":258,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"261'360","reported_seq":1070,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.945626+0000","last_change":"2024-04-22T19:51:26.502589+0000","last_active":"2024-04-22T20:48:17.945626+0000","last_peered":"2024-04-22T20:48:17.945626+0000","last_clean":"2024-04-22T20:48:17.945626+0000","last_became_active":"2024-04-22T19:51:26.502156+0000","last_became_peered":"2024-04-22T19:51:26.502156+0000","last_unstale":"2024-04-22T20:48:17.945626+0000","last_undegraded":"2024-04-22T20:48:17.945626+0000","last_fullsized":"2024-04-22T20:48:17.945626+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:22:39.535038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1373470,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":619,"num_read_kb":68133,"num_write":1312,"num_write_kb":41095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"261'290","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:09.419441+0000","last_change":"2024-04-22T19:51:28.510666+0000","last_active":"2024-04-22T20:47:09.419441+0000","last_peered":"2024-04-22T20:47:09.419441+0000","last_clean":"2024-04-22T20:47:09.419441+0000","last_became_active":"2024-04-22T19:51:28.510415+0000","last_became_peered":"2024-04-22T19:51:28.510415+0000","last_unstale":"2024-04-22T20:47:09.419441+0000","last_undegraded":"2024-04-22T20:47:09.419441+0000","last_fullsized":"2024-04-22T20:47:09.419441+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:37:43.003473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1119,"num_read_kb":869,"num_write":475,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"261'320","reported_seq":965,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.832969+0000","last_change":"2024-04-22T19:51:26.507272+0000","last_active":"2024-04-22T20:48:17.832969+0000","last_peered":"2024-04-22T20:48:17.832969+0000","last_clean":"2024-04-22T20:48:17.832969+0000","last_became_active":"2024-04-22T19:51:26.506671+0000","last_became_peered":"2024-04-22T19:51:26.506671+0000","last_unstale":"2024-04-22T20:48:17.832969+0000","last_undegraded":"2024-04-22T20:48:17.832969+0000","last_fullsized":"2024-04-22T20:48:17.832969+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:54:08.993777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194361,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":6535,"num_write":1103,"num_write_kb":36003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"261'155","reported_seq":775,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.062504+0000","last_change":"2024-04-22T19:51:28.512453+0000","last_active":"2024-04-22T20:44:40.062504+0000","last_peered":"2024-04-22T20:44:40.062504+0000","last_clean":"2024-04-22T20:44:40.062504+0000","last_became_active":"2024-04-22T19:51:28.511889+0000","last_became_peered":"2024-04-22T19:51:28.511889+0000","last_unstale":"2024-04-22T20:44:40.062504+0000","last_undegraded":"2024-04-22T20:44:40.062504+0000","last_fullsized":"2024-04-22T20:44:40.062504+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:45.195439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":679,"num_read_kb":547,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"261'334","reported_seq":981,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.922476+0000","last_change":"2024-04-22T19:51:26.498087+0000","last_active":"2024-04-22T20:48:17.922476+0000","last_peered":"2024-04-22T20:48:17.922476+0000","last_clean":"2024-04-22T20:48:17.922476+0000","last_became_active":"2024-04-22T19:51:26.497718+0000","last_became_peered":"2024-04-22T19:51:26.497718+0000","last_unstale":"2024-04-22T20:48:17.922476+0000","last_undegraded":"2024-04-22T20:48:17.922476+0000","last_fullsized":"2024-04-22T20:48:17.922476+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:50:38.709806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050340,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":471,"num_read_kb":30112,"num_write":1163,"num_write_kb":27210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"261'93","reported_seq":699,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:44.069351+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:46:44.069351+0000","last_peered":"2024-04-22T20:46:44.069351+0000","last_clean":"2024-04-22T20:46:44.069351+0000","last_became_active":"2024-04-22T19:51:28.507673+0000","last_became_peered":"2024-04-22T19:51:28.507673+0000","last_unstale":"2024-04-22T20:46:44.069351+0000","last_undegraded":"2024-04-22T20:46:44.069351+0000","last_fullsized":"2024-04-22T20:46:44.069351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:18:40.338876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":404,"num_read_kb":350,"num_write":133,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"261'413","reported_seq":1124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.882831+0000","last_change":"2024-04-22T19:51:26.511696+0000","last_active":"2024-04-22T20:48:17.882831+0000","last_peered":"2024-04-22T20:48:17.882831+0000","last_clean":"2024-04-22T20:48:17.882831+0000","last_became_active":"2024-04-22T19:51:26.511213+0000","last_became_peered":"2024-04-22T19:51:26.511213+0000","last_unstale":"2024-04-22T20:48:17.882831+0000","last_undegraded":"2024-04-22T20:48:17.882831+0000","last_fullsized":"2024-04-22T20:48:17.882831+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:51:44.984733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306877,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":730,"num_read_kb":36415,"num_write":1798,"num_write_kb":40524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"261'126","reported_seq":647,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.145449+0000","last_change":"2024-04-22T19:51:36.567880+0000","last_active":"2024-04-22T20:48:04.145449+0000","last_peered":"2024-04-22T20:48:04.145449+0000","last_clean":"2024-04-22T20:48:04.145449+0000","last_became_active":"2024-04-22T19:51:36.567561+0000","last_became_peered":"2024-04-22T19:51:36.567561+0000","last_unstale":"2024-04-22T20:48:04.145449+0000","last_undegraded":"2024-04-22T20:48:04.145449+0000","last_fullsized":"2024-04-22T20:48:04.145449+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:10:43.936463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":168,"num_write":307,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"261'38","reported_seq":584,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154700+0000","last_change":"2024-04-22T19:51:28.520238+0000","last_active":"2024-04-22T20:45:15.154700+0000","last_peered":"2024-04-22T20:45:15.154700+0000","last_clean":"2024-04-22T20:45:15.154700+0000","last_became_active":"2024-04-22T19:51:28.519994+0000","last_became_peered":"2024-04-22T19:51:28.519994+0000","last_unstale":"2024-04-22T20:45:15.154700+0000","last_undegraded":"2024-04-22T20:45:15.154700+0000","last_fullsized":"2024-04-22T20:45:15.154700+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:48:21.210710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":138,"num_write":52,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"261'317","reported_seq":1050,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.907502+0000","last_change":"2024-04-22T19:53:05.051779+0000","last_active":"2024-04-22T20:48:17.907502+0000","last_peered":"2024-04-22T20:48:17.907502+0000","last_clean":"2024-04-22T20:48:17.907502+0000","last_became_active":"2024-04-22T19:53:05.051272+0000","last_became_peered":"2024-04-22T19:53:05.051272+0000","last_unstale":"2024-04-22T20:48:17.907502+0000","last_undegraded":"2024-04-22T20:48:17.907502+0000","last_fullsized":"2024-04-22T20:48:17.907502+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:14:57.868131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263217,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":662,"num_read_kb":11069,"num_write":1270,"num_write_kb":33730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180678+0000","last_change":"2024-04-22T19:51:30.521121+0000","last_active":"2024-04-22T20:44:40.180678+0000","last_peered":"2024-04-22T20:44:40.180678+0000","last_clean":"2024-04-22T20:44:40.180678+0000","last_became_active":"2024-04-22T19:51:30.520848+0000","last_became_peered":"2024-04-22T19:51:30.520848+0000","last_unstale":"2024-04-22T20:44:40.180678+0000","last_undegraded":"2024-04-22T20:44:40.180678+0000","last_fullsized":"2024-04-22T20:44:40.180678+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:30.383458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"261'329","reported_seq":933,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.474258+0000","last_change":"2024-04-22T19:51:32.537967+0000","last_active":"2024-04-22T20:48:16.474258+0000","last_peered":"2024-04-22T20:48:16.474258+0000","last_clean":"2024-04-22T20:48:16.474258+0000","last_became_active":"2024-04-22T19:51:32.537695+0000","last_became_peered":"2024-04-22T19:51:32.537695+0000","last_unstale":"2024-04-22T20:48:16.474258+0000","last_undegraded":"2024-04-22T20:48:16.474258+0000","last_fullsized":"2024-04-22T20:48:16.474258+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:12:32.127700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39872,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":550,"num_read_kb":1178,"num_write":438,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.057042+0000","last_change":"2024-04-22T19:51:34.557621+0000","last_active":"2024-04-22T20:44:45.057042+0000","last_peered":"2024-04-22T20:44:45.057042+0000","last_clean":"2024-04-22T20:44:45.057042+0000","last_became_active":"2024-04-22T19:51:34.557285+0000","last_became_peered":"2024-04-22T19:51:34.557285+0000","last_unstale":"2024-04-22T20:44:45.057042+0000","last_undegraded":"2024-04-22T20:44:45.057042+0000","last_fullsized":"2024-04-22T20:44:45.057042+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:14.949781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"261'2399","reported_seq":4258,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.711091+0000","last_change":"2024-04-22T19:51:36.567463+0000","last_active":"2024-04-22T20:48:03.711091+0000","last_peered":"2024-04-22T20:48:03.711091+0000","last_clean":"2024-04-22T20:48:03.711091+0000","last_became_active":"2024-04-22T19:51:36.567195+0000","last_became_peered":"2024-04-22T19:51:36.567195+0000","last_unstale":"2024-04-22T20:48:03.711091+0000","last_undegraded":"2024-04-22T20:48:03.711091+0000","last_fullsized":"2024-04-22T20:48:03.711091+0000","mapping_epoch":30,"log_start":"261'2300","ondisk_log_start":"261'2300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":2300,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:46:00.619550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5986,"num_read_kb":5990,"num_write":4781,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"261'130","reported_seq":720,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:53.028609+0000","last_change":"2024-04-22T19:51:28.513076+0000","last_active":"2024-04-22T20:46:53.028609+0000","last_peered":"2024-04-22T20:46:53.028609+0000","last_clean":"2024-04-22T20:46:53.028609+0000","last_became_active":"2024-04-22T19:51:28.512786+0000","last_became_peered":"2024-04-22T19:51:28.512786+0000","last_unstale":"2024-04-22T20:46:53.028609+0000","last_undegraded":"2024-04-22T20:46:53.028609+0000","last_fullsized":"2024-04-22T20:46:53.028609+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:20:09.250321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":408,"num_write":191,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"261'290","reported_seq":893,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.922185+0000","last_change":"2024-04-22T19:51:26.500209+0000","last_active":"2024-04-22T20:48:17.922185+0000","last_peered":"2024-04-22T20:48:17.922185+0000","last_clean":"2024-04-22T20:48:17.922185+0000","last_became_active":"2024-04-22T19:51:26.499885+0000","last_became_peered":"2024-04-22T19:51:26.499885+0000","last_unstale":"2024-04-22T20:48:17.922185+0000","last_undegraded":"2024-04-22T20:48:17.922185+0000","last_fullsized":"2024-04-22T20:48:17.922185+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:06:40.641478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050193,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":16305,"num_write":962,"num_write_kb":23193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154709+0000","last_change":"2024-04-22T19:51:30.527682+0000","last_active":"2024-04-22T20:45:15.154709+0000","last_peered":"2024-04-22T20:45:15.154709+0000","last_clean":"2024-04-22T20:45:15.154709+0000","last_became_active":"2024-04-22T19:51:30.526622+0000","last_became_peered":"2024-04-22T19:51:30.526622+0000","last_unstale":"2024-04-22T20:45:15.154709+0000","last_undegraded":"2024-04-22T20:45:15.154709+0000","last_fullsized":"2024-04-22T20:45:15.154709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:20:42.245730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"261'86","reported_seq":611,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.173330+0000","last_change":"2024-04-22T19:51:32.536487+0000","last_active":"2024-04-22T20:44:45.173330+0000","last_peered":"2024-04-22T20:44:45.173330+0000","last_clean":"2024-04-22T20:44:45.173330+0000","last_became_active":"2024-04-22T19:51:32.536180+0000","last_became_peered":"2024-04-22T19:51:32.536180+0000","last_unstale":"2024-04-22T20:44:45.173330+0000","last_undegraded":"2024-04-22T20:44:45.173330+0000","last_fullsized":"2024-04-22T20:44:45.173330+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:21.299001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":95,"num_write":64,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.173306+0000","last_change":"2024-04-22T19:51:34.549833+0000","last_active":"2024-04-22T20:44:45.173306+0000","last_peered":"2024-04-22T20:44:45.173306+0000","last_clean":"2024-04-22T20:44:45.173306+0000","last_became_active":"2024-04-22T19:51:34.549551+0000","last_became_peered":"2024-04-22T19:51:34.549551+0000","last_unstale":"2024-04-22T20:44:45.173306+0000","last_undegraded":"2024-04-22T20:44:45.173306+0000","last_fullsized":"2024-04-22T20:44:45.173306+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:14:36.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"261'146","reported_seq":645,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.823329+0000","last_change":"2024-04-22T19:51:36.573966+0000","last_active":"2024-04-22T20:48:00.823329+0000","last_peered":"2024-04-22T20:48:00.823329+0000","last_clean":"2024-04-22T20:48:00.823329+0000","last_became_active":"2024-04-22T19:51:36.573189+0000","last_became_peered":"2024-04-22T19:51:36.573189+0000","last_unstale":"2024-04-22T20:48:00.823329+0000","last_undegraded":"2024-04-22T20:48:00.823329+0000","last_fullsized":"2024-04-22T20:48:00.823329+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:07:48.890371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":328,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"261'212","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.175878+0000","last_change":"2024-04-22T19:51:28.518334+0000","last_active":"2024-04-22T20:44:40.175878+0000","last_peered":"2024-04-22T20:44:40.175878+0000","last_clean":"2024-04-22T20:44:40.175878+0000","last_became_active":"2024-04-22T19:51:28.517737+0000","last_became_peered":"2024-04-22T19:51:28.517737+0000","last_unstale":"2024-04-22T20:44:40.175878+0000","last_undegraded":"2024-04-22T20:44:40.175878+0000","last_fullsized":"2024-04-22T20:44:40.175878+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:06:47.145426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":871,"num_read_kb":733,"num_write":338,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"261'298","reported_seq":942,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.941210+0000","last_change":"2024-04-22T19:51:26.509257+0000","last_active":"2024-04-22T20:48:17.941210+0000","last_peered":"2024-04-22T20:48:17.941210+0000","last_clean":"2024-04-22T20:48:17.941210+0000","last_became_active":"2024-04-22T19:51:26.508933+0000","last_became_peered":"2024-04-22T19:51:26.508933+0000","last_unstale":"2024-04-22T20:48:17.941210+0000","last_undegraded":"2024-04-22T20:48:17.941210+0000","last_fullsized":"2024-04-22T20:48:17.941210+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:00:26.499126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632139,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":513,"num_read_kb":15262,"num_write":1122,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":502,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.150710+0000","last_change":"2024-04-22T19:51:30.528152+0000","last_active":"2024-04-22T20:44:40.150710+0000","last_peered":"2024-04-22T20:44:40.150710+0000","last_clean":"2024-04-22T20:44:40.150710+0000","last_became_active":"2024-04-22T19:51:30.527712+0000","last_became_peered":"2024-04-22T19:51:30.527712+0000","last_unstale":"2024-04-22T20:44:40.150710+0000","last_undegraded":"2024-04-22T20:44:40.150710+0000","last_fullsized":"2024-04-22T20:44:40.150710+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:03:58.918129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"261'112","reported_seq":656,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.066993+0000","last_change":"2024-04-22T19:51:32.536484+0000","last_active":"2024-04-22T20:45:25.066993+0000","last_peered":"2024-04-22T20:45:25.066993+0000","last_clean":"2024-04-22T20:45:25.066993+0000","last_became_active":"2024-04-22T19:51:32.536177+0000","last_became_peered":"2024-04-22T19:51:32.536177+0000","last_unstale":"2024-04-22T20:45:25.066993+0000","last_undegraded":"2024-04-22T20:45:25.066993+0000","last_fullsized":"2024-04-22T20:45:25.066993+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:39:22.960954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"261'523","reported_seq":2083,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.313356+0000","last_change":"2024-04-22T19:51:34.562120+0000","last_active":"2024-04-22T20:48:17.313356+0000","last_peered":"2024-04-22T20:48:17.313356+0000","last_clean":"2024-04-22T20:48:17.313356+0000","last_became_active":"2024-04-22T19:51:34.561828+0000","last_became_peered":"2024-04-22T19:51:34.561828+0000","last_unstale":"2024-04-22T20:48:17.313356+0000","last_undegraded":"2024-04-22T20:48:17.313356+0000","last_fullsized":"2024-04-22T20:48:17.313356+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":500,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:46.946439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"261'174","reported_seq":986,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.139062+0000","last_change":"2024-04-22T19:51:36.565908+0000","last_active":"2024-04-22T20:48:03.139062+0000","last_peered":"2024-04-22T20:48:03.139062+0000","last_clean":"2024-04-22T20:48:03.139062+0000","last_became_active":"2024-04-22T19:51:36.565673+0000","last_became_peered":"2024-04-22T19:51:36.565673+0000","last_unstale":"2024-04-22T20:48:03.139062+0000","last_undegraded":"2024-04-22T20:48:03.139062+0000","last_fullsized":"2024-04-22T20:48:03.139062+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:45:46.054789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":579,"num_read_kb":567,"num_write":371,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"261'176","reported_seq":814,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.150685+0000","last_change":"2024-04-22T19:51:28.516658+0000","last_active":"2024-04-22T20:44:40.150685+0000","last_peered":"2024-04-22T20:44:40.150685+0000","last_clean":"2024-04-22T20:44:40.150685+0000","last_became_active":"2024-04-22T19:51:28.515670+0000","last_became_peered":"2024-04-22T19:51:28.515670+0000","last_unstale":"2024-04-22T20:44:40.150685+0000","last_undegraded":"2024-04-22T20:44:40.150685+0000","last_fullsized":"2024-04-22T20:44:40.150685+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:38:08.341570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":729,"num_read_kb":590,"num_write":271,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"261'317","reported_seq":995,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.927650+0000","last_change":"2024-04-22T19:51:26.503775+0000","last_active":"2024-04-22T20:48:17.927650+0000","last_peered":"2024-04-22T20:48:17.927650+0000","last_clean":"2024-04-22T20:48:17.927650+0000","last_became_active":"2024-04-22T19:51:26.503438+0000","last_became_peered":"2024-04-22T19:51:26.503438+0000","last_unstale":"2024-04-22T20:48:17.927650+0000","last_undegraded":"2024-04-22T20:48:17.927650+0000","last_fullsized":"2024-04-22T20:48:17.927650+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:39:05.281766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3601437,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":581,"num_read_kb":32833,"num_write":1303,"num_write_kb":25399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180715+0000","last_change":"2024-04-22T19:51:30.523415+0000","last_active":"2024-04-22T20:44:40.180715+0000","last_peered":"2024-04-22T20:44:40.180715+0000","last_clean":"2024-04-22T20:44:40.180715+0000","last_became_active":"2024-04-22T19:51:30.523125+0000","last_became_peered":"2024-04-22T19:51:30.523125+0000","last_unstale":"2024-04-22T20:44:40.180715+0000","last_undegraded":"2024-04-22T20:44:40.180715+0000","last_fullsized":"2024-04-22T20:44:40.180715+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:17:33.007742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"261'252","reported_seq":822,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.112717+0000","last_change":"2024-04-22T19:51:33.248486+0000","last_active":"2024-04-22T20:48:16.112717+0000","last_peered":"2024-04-22T20:48:16.112717+0000","last_clean":"2024-04-22T20:48:16.112717+0000","last_became_active":"2024-04-22T19:51:33.248160+0000","last_became_peered":"2024-04-22T19:51:33.248160+0000","last_unstale":"2024-04-22T20:48:16.112717+0000","last_undegraded":"2024-04-22T20:48:16.112717+0000","last_fullsized":"2024-04-22T20:48:16.112717+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:35.418273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":425,"num_read_kb":974,"num_write":344,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"261'536","reported_seq":4229,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.191140+0000","last_change":"2024-04-22T19:51:34.556255+0000","last_active":"2024-04-22T20:48:17.191140+0000","last_peered":"2024-04-22T20:48:17.191140+0000","last_clean":"2024-04-22T20:48:17.191140+0000","last_became_active":"2024-04-22T19:51:34.555720+0000","last_became_peered":"2024-04-22T19:51:34.555720+0000","last_unstale":"2024-04-22T20:48:17.191140+0000","last_undegraded":"2024-04-22T20:48:17.191140+0000","last_fullsized":"2024-04-22T20:48:17.191140+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:58.605732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"261'385","reported_seq":1057,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189325+0000","last_change":"2024-04-22T19:51:28.513088+0000","last_active":"2024-04-22T20:45:40.189325+0000","last_peered":"2024-04-22T20:45:40.189325+0000","last_clean":"2024-04-22T20:45:40.189325+0000","last_became_active":"2024-04-22T19:51:28.512125+0000","last_became_peered":"2024-04-22T19:51:28.512125+0000","last_unstale":"2024-04-22T20:45:40.189325+0000","last_undegraded":"2024-04-22T20:45:40.189325+0000","last_fullsized":"2024-04-22T20:45:40.189325+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:22:31.839887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1728,"num_read_kb":1430,"num_write":663,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"261'305","reported_seq":947,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.927736+0000","last_change":"2024-04-22T19:51:26.508318+0000","last_active":"2024-04-22T20:48:17.927736+0000","last_peered":"2024-04-22T20:48:17.927736+0000","last_clean":"2024-04-22T20:48:17.927736+0000","last_became_active":"2024-04-22T19:51:26.504654+0000","last_became_peered":"2024-04-22T19:51:26.504654+0000","last_unstale":"2024-04-22T20:48:17.927736+0000","last_undegraded":"2024-04-22T20:48:17.927736+0000","last_fullsized":"2024-04-22T20:48:17.927736+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:29:46.972228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10487408,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":445,"num_read_kb":80529,"num_write":1013,"num_write_kb":26179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.065667+0000","last_change":"2024-04-22T19:51:30.526634+0000","last_active":"2024-04-22T20:45:40.065667+0000","last_peered":"2024-04-22T20:45:40.065667+0000","last_clean":"2024-04-22T20:45:40.065667+0000","last_became_active":"2024-04-22T19:51:30.526306+0000","last_became_peered":"2024-04-22T19:51:30.526306+0000","last_unstale":"2024-04-22T20:45:40.065667+0000","last_undegraded":"2024-04-22T20:45:40.065667+0000","last_fullsized":"2024-04-22T20:45:40.065667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:46.220150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"261'334","reported_seq":949,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.779237+0000","last_change":"2024-04-22T19:51:33.248375+0000","last_active":"2024-04-22T20:48:16.779237+0000","last_peered":"2024-04-22T20:48:16.779237+0000","last_clean":"2024-04-22T20:48:16.779237+0000","last_became_active":"2024-04-22T19:51:33.248059+0000","last_became_peered":"2024-04-22T19:51:33.248059+0000","last_unstale":"2024-04-22T20:48:16.779237+0000","last_undegraded":"2024-04-22T20:48:16.779237+0000","last_fullsized":"2024-04-22T20:48:16.779237+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:25:25.539547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45321,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":537,"num_read_kb":1022,"num_write":408,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.045374+0000","last_change":"2024-04-22T19:51:34.557926+0000","last_active":"2024-04-22T20:45:35.045374+0000","last_peered":"2024-04-22T20:45:35.045374+0000","last_clean":"2024-04-22T20:45:35.045374+0000","last_became_active":"2024-04-22T19:51:34.557662+0000","last_became_peered":"2024-04-22T19:51:34.557662+0000","last_unstale":"2024-04-22T20:45:35.045374+0000","last_undegraded":"2024-04-22T20:45:35.045374+0000","last_fullsized":"2024-04-22T20:45:35.045374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:25.109053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"261'562","reported_seq":1370,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.537178+0000","last_change":"2024-04-22T19:51:36.571168+0000","last_active":"2024-04-22T20:48:08.537178+0000","last_peered":"2024-04-22T20:48:08.537178+0000","last_clean":"2024-04-22T20:48:08.537178+0000","last_became_active":"2024-04-22T19:51:36.570520+0000","last_became_peered":"2024-04-22T19:51:36.570520+0000","last_unstale":"2024-04-22T20:48:08.537178+0000","last_undegraded":"2024-04-22T20:48:08.537178+0000","last_fullsized":"2024-04-22T20:48:08.537178+0000","mapping_epoch":30,"log_start":"261'500","ondisk_log_start":"261'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":500,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:56.456085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1283,"num_read_kb":1280,"num_write":1166,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"261'311","reported_seq":959,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181329+0000","last_change":"2024-04-22T19:51:28.512328+0000","last_active":"2024-04-22T20:45:40.181329+0000","last_peered":"2024-04-22T20:45:40.181329+0000","last_clean":"2024-04-22T20:45:40.181329+0000","last_became_active":"2024-04-22T19:51:28.511527+0000","last_became_peered":"2024-04-22T19:51:28.511527+0000","last_unstale":"2024-04-22T20:45:40.181329+0000","last_undegraded":"2024-04-22T20:45:40.181329+0000","last_fullsized":"2024-04-22T20:45:40.181329+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:22.483205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1221,"num_read_kb":977,"num_write":455,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"261'330","reported_seq":971,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.924092+0000","last_change":"2024-04-22T19:51:26.505351+0000","last_active":"2024-04-22T20:48:17.924092+0000","last_peered":"2024-04-22T20:48:17.924092+0000","last_clean":"2024-04-22T20:48:17.924092+0000","last_became_active":"2024-04-22T19:51:26.504472+0000","last_became_peered":"2024-04-22T19:51:26.504472+0000","last_unstale":"2024-04-22T20:48:17.924092+0000","last_undegraded":"2024-04-22T20:48:17.924092+0000","last_fullsized":"2024-04-22T20:48:17.924092+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:25.100075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158015,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":483,"num_read_kb":33343,"num_write":1137,"num_write_kb":49829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"32'5","reported_seq":828,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.108308+0000","last_change":"2024-04-22T19:51:30.524309+0000","last_active":"2024-04-22T20:48:15.108308+0000","last_peered":"2024-04-22T20:48:15.108308+0000","last_clean":"2024-04-22T20:48:15.108308+0000","last_became_active":"2024-04-22T19:51:30.524043+0000","last_became_peered":"2024-04-22T19:51:30.524043+0000","last_unstale":"2024-04-22T20:48:15.108308+0000","last_undegraded":"2024-04-22T20:48:15.108308+0000","last_fullsized":"2024-04-22T20:48:15.108308+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:51.651333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":492,"num_read_kb":799,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"261'266","reported_seq":845,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.787421+0000","last_change":"2024-04-22T19:51:32.539254+0000","last_active":"2024-04-22T20:48:17.787421+0000","last_peered":"2024-04-22T20:48:17.787421+0000","last_clean":"2024-04-22T20:48:17.787421+0000","last_became_active":"2024-04-22T19:51:32.537968+0000","last_became_peered":"2024-04-22T19:51:32.537968+0000","last_unstale":"2024-04-22T20:48:17.787421+0000","last_undegraded":"2024-04-22T20:48:17.787421+0000","last_fullsized":"2024-04-22T20:48:17.787421+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:38.345231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43364,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":998,"num_write":350,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.158827+0000","last_change":"2024-04-22T19:51:34.556181+0000","last_active":"2024-04-22T20:45:50.158827+0000","last_peered":"2024-04-22T20:45:50.158827+0000","last_clean":"2024-04-22T20:45:50.158827+0000","last_became_active":"2024-04-22T19:51:34.555889+0000","last_became_peered":"2024-04-22T19:51:34.555889+0000","last_unstale":"2024-04-22T20:45:50.158827+0000","last_undegraded":"2024-04-22T20:45:50.158827+0000","last_fullsized":"2024-04-22T20:45:50.158827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:03:03.044667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"261'79","reported_seq":577,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.319292+0000","last_change":"2024-04-22T19:51:36.574838+0000","last_active":"2024-04-22T20:45:55.319292+0000","last_peered":"2024-04-22T20:45:55.319292+0000","last_clean":"2024-04-22T20:45:55.319292+0000","last_became_active":"2024-04-22T19:51:36.574555+0000","last_became_peered":"2024-04-22T19:51:36.574555+0000","last_unstale":"2024-04-22T20:45:55.319292+0000","last_undegraded":"2024-04-22T20:45:55.319292+0000","last_fullsized":"2024-04-22T20:45:55.319292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:07.667952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":91,"num_write":194,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"261'188","reported_seq":783,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:41.115015+0000","last_change":"2024-04-22T19:51:28.519762+0000","last_active":"2024-04-22T20:46:41.115015+0000","last_peered":"2024-04-22T20:46:41.115015+0000","last_clean":"2024-04-22T20:46:41.115015+0000","last_became_active":"2024-04-22T19:51:28.519424+0000","last_became_peered":"2024-04-22T19:51:28.519424+0000","last_unstale":"2024-04-22T20:46:41.115015+0000","last_undegraded":"2024-04-22T20:46:41.115015+0000","last_fullsized":"2024-04-22T20:46:41.115015+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:13.130467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":758,"num_read_kb":601,"num_write":297,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"261'289","reported_seq":911,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.899453+0000","last_change":"2024-04-22T19:51:26.511055+0000","last_active":"2024-04-22T20:48:17.899453+0000","last_peered":"2024-04-22T20:48:17.899453+0000","last_clean":"2024-04-22T20:48:17.899453+0000","last_became_active":"2024-04-22T19:51:26.510273+0000","last_became_peered":"2024-04-22T19:51:26.510273+0000","last_unstale":"2024-04-22T20:48:17.899453+0000","last_undegraded":"2024-04-22T20:48:17.899453+0000","last_fullsized":"2024-04-22T20:48:17.899453+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:53.305766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2115,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":380,"num_read_kb":65007,"num_write":1004,"num_write_kb":29343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.067468+0000","last_change":"2024-04-22T19:51:30.525335+0000","last_active":"2024-04-22T20:45:30.067468+0000","last_peered":"2024-04-22T20:45:30.067468+0000","last_clean":"2024-04-22T20:45:30.067468+0000","last_became_active":"2024-04-22T19:51:30.525018+0000","last_became_peered":"2024-04-22T19:51:30.525018+0000","last_unstale":"2024-04-22T20:45:30.067468+0000","last_undegraded":"2024-04-22T20:45:30.067468+0000","last_fullsized":"2024-04-22T20:45:30.067468+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:49:24.770975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"261'309","reported_seq":902,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.943989+0000","last_change":"2024-04-22T19:51:32.538989+0000","last_active":"2024-04-22T20:48:16.943989+0000","last_peered":"2024-04-22T20:48:16.943989+0000","last_clean":"2024-04-22T20:48:16.943989+0000","last_became_active":"2024-04-22T19:51:32.538656+0000","last_became_peered":"2024-04-22T19:51:32.538656+0000","last_unstale":"2024-04-22T20:48:16.943989+0000","last_undegraded":"2024-04-22T20:48:16.943989+0000","last_fullsized":"2024-04-22T20:48:16.943989+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:36:49.025357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47724,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":498,"num_read_kb":1086,"num_write":396,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"261'509","reported_seq":2198,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.208931+0000","last_change":"2024-04-22T19:51:34.561771+0000","last_active":"2024-04-22T20:48:16.208931+0000","last_peered":"2024-04-22T20:48:16.208931+0000","last_clean":"2024-04-22T20:48:16.208931+0000","last_became_active":"2024-04-22T19:51:34.561488+0000","last_became_peered":"2024-04-22T19:51:34.561488+0000","last_unstale":"2024-04-22T20:48:16.208931+0000","last_undegraded":"2024-04-22T20:48:16.208931+0000","last_fullsized":"2024-04-22T20:48:16.208931+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:08:35.813443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"261'134","reported_seq":623,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.184424+0000","last_change":"2024-04-22T19:51:36.573554+0000","last_active":"2024-04-22T20:45:50.184424+0000","last_peered":"2024-04-22T20:45:50.184424+0000","last_clean":"2024-04-22T20:45:50.184424+0000","last_became_active":"2024-04-22T19:51:36.573167+0000","last_became_peered":"2024-04-22T19:51:36.573167+0000","last_unstale":"2024-04-22T20:45:50.184424+0000","last_undegraded":"2024-04-22T20:45:50.184424+0000","last_fullsized":"2024-04-22T20:45:50.184424+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:24.336268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"261'150","reported_seq":715,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181727+0000","last_change":"2024-04-22T19:51:28.518692+0000","last_active":"2024-04-22T20:45:30.181727+0000","last_peered":"2024-04-22T20:45:30.181727+0000","last_clean":"2024-04-22T20:45:30.181727+0000","last_became_active":"2024-04-22T19:51:28.518398+0000","last_became_peered":"2024-04-22T19:51:28.518398+0000","last_unstale":"2024-04-22T20:45:30.181727+0000","last_undegraded":"2024-04-22T20:45:30.181727+0000","last_fullsized":"2024-04-22T20:45:30.181727+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:33.700338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":596,"num_read_kb":465,"num_write":243,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"261'313","reported_seq":982,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.925291+0000","last_change":"2024-04-22T19:52:05.113375+0000","last_active":"2024-04-22T20:48:17.925291+0000","last_peered":"2024-04-22T20:48:17.925291+0000","last_clean":"2024-04-22T20:48:17.925291+0000","last_became_active":"2024-04-22T19:52:05.112910+0000","last_became_peered":"2024-04-22T19:52:05.112910+0000","last_unstale":"2024-04-22T20:48:17.925291+0000","last_undegraded":"2024-04-22T20:48:17.925291+0000","last_fullsized":"2024-04-22T20:48:17.925291+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:44:17.477228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":486619,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":517,"num_read_kb":39016,"num_write":1092,"num_write_kb":34073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.069399+0000","last_change":"2024-04-22T19:51:30.529388+0000","last_active":"2024-04-22T20:45:50.069399+0000","last_peered":"2024-04-22T20:45:50.069399+0000","last_clean":"2024-04-22T20:45:50.069399+0000","last_became_active":"2024-04-22T19:51:30.529011+0000","last_became_peered":"2024-04-22T19:51:30.529011+0000","last_unstale":"2024-04-22T20:45:50.069399+0000","last_undegraded":"2024-04-22T20:45:50.069399+0000","last_fullsized":"2024-04-22T20:45:50.069399+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:02:44.821662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"261'488","reported_seq":1122,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.952457+0000","last_change":"2024-04-22T19:51:32.537571+0000","last_active":"2024-04-22T20:48:17.952457+0000","last_peered":"2024-04-22T20:48:17.952457+0000","last_clean":"2024-04-22T20:48:17.952457+0000","last_became_active":"2024-04-22T19:51:32.537267+0000","last_became_peered":"2024-04-22T19:51:32.537267+0000","last_unstale":"2024-04-22T20:48:17.952457+0000","last_undegraded":"2024-04-22T20:48:17.952457+0000","last_fullsized":"2024-04-22T20:48:17.952457+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":400,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:04.937868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":856,"num_read_kb":2097,"num_write":704,"num_write_kb":611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.068905+0000","last_change":"2024-04-22T19:51:34.552745+0000","last_active":"2024-04-22T20:45:45.068905+0000","last_peered":"2024-04-22T20:45:45.068905+0000","last_clean":"2024-04-22T20:45:45.068905+0000","last_became_active":"2024-04-22T19:51:34.552446+0000","last_became_peered":"2024-04-22T19:51:34.552446+0000","last_unstale":"2024-04-22T20:45:45.068905+0000","last_undegraded":"2024-04-22T20:45:45.068905+0000","last_fullsized":"2024-04-22T20:45:45.068905+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:59.206061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"261'106","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.047062+0000","last_change":"2024-04-22T19:51:36.570982+0000","last_active":"2024-04-22T20:45:50.047062+0000","last_peered":"2024-04-22T20:45:50.047062+0000","last_clean":"2024-04-22T20:45:50.047062+0000","last_became_active":"2024-04-22T19:51:36.570666+0000","last_became_peered":"2024-04-22T19:51:36.570666+0000","last_unstale":"2024-04-22T20:45:50.047062+0000","last_undegraded":"2024-04-22T20:45:50.047062+0000","last_fullsized":"2024-04-22T20:45:50.047062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:18.966614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":242,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"261'168","reported_seq":795,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:16.403532+0000","last_change":"2024-04-22T19:51:28.508639+0000","last_active":"2024-04-22T20:47:16.403532+0000","last_peered":"2024-04-22T20:47:16.403532+0000","last_clean":"2024-04-22T20:47:16.403532+0000","last_became_active":"2024-04-22T19:51:28.508288+0000","last_became_peered":"2024-04-22T19:51:28.508288+0000","last_unstale":"2024-04-22T20:47:16.403532+0000","last_undegraded":"2024-04-22T20:47:16.403532+0000","last_fullsized":"2024-04-22T20:47:16.403532+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:22:30.411824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":576,"num_write":244,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"261'317","reported_seq":931,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.903289+0000","last_change":"2024-04-22T19:51:26.506251+0000","last_active":"2024-04-22T20:48:17.903289+0000","last_peered":"2024-04-22T20:48:17.903289+0000","last_clean":"2024-04-22T20:48:17.903289+0000","last_became_active":"2024-04-22T19:51:26.503669+0000","last_became_peered":"2024-04-22T19:51:26.503669+0000","last_unstale":"2024-04-22T20:48:17.903289+0000","last_undegraded":"2024-04-22T20:48:17.903289+0000","last_fullsized":"2024-04-22T20:48:17.903289+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:52.491938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194320,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":456,"num_read_kb":4498,"num_write":1097,"num_write_kb":33252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.067499+0000","last_change":"2024-04-22T19:51:30.526698+0000","last_active":"2024-04-22T20:45:30.067499+0000","last_peered":"2024-04-22T20:45:30.067499+0000","last_clean":"2024-04-22T20:45:30.067499+0000","last_became_active":"2024-04-22T19:51:30.526420+0000","last_became_peered":"2024-04-22T19:51:30.526420+0000","last_unstale":"2024-04-22T20:45:30.067499+0000","last_undegraded":"2024-04-22T20:45:30.067499+0000","last_fullsized":"2024-04-22T20:45:30.067499+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:04:14.342753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"261'269","reported_seq":848,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.665057+0000","last_change":"2024-04-22T19:51:32.536764+0000","last_active":"2024-04-22T20:48:15.665057+0000","last_peered":"2024-04-22T20:48:15.665057+0000","last_clean":"2024-04-22T20:48:15.665057+0000","last_became_active":"2024-04-22T19:51:32.536463+0000","last_became_peered":"2024-04-22T19:51:32.536463+0000","last_unstale":"2024-04-22T20:48:15.665057+0000","last_undegraded":"2024-04-22T20:48:15.665057+0000","last_fullsized":"2024-04-22T20:48:15.665057+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:29:00.474073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42355,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":457,"num_read_kb":963,"num_write":402,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183789+0000","last_change":"2024-04-22T19:51:34.555066+0000","last_active":"2024-04-22T20:45:45.183789+0000","last_peered":"2024-04-22T20:45:45.183789+0000","last_clean":"2024-04-22T20:45:45.183789+0000","last_became_active":"2024-04-22T19:51:34.554737+0000","last_became_peered":"2024-04-22T19:51:34.554737+0000","last_unstale":"2024-04-22T20:45:45.183789+0000","last_undegraded":"2024-04-22T20:45:45.183789+0000","last_fullsized":"2024-04-22T20:45:45.183789+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:09:00.070765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"261'93","reported_seq":590,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:42.565656+0000","last_change":"2024-04-22T19:51:36.571558+0000","last_active":"2024-04-22T20:46:42.565656+0000","last_peered":"2024-04-22T20:46:42.565656+0000","last_clean":"2024-04-22T20:46:42.565656+0000","last_became_active":"2024-04-22T19:51:36.571086+0000","last_became_peered":"2024-04-22T19:51:36.571086+0000","last_unstale":"2024-04-22T20:46:42.565656+0000","last_undegraded":"2024-04-22T20:46:42.565656+0000","last_fullsized":"2024-04-22T20:46:42.565656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:37:15.182216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":72,"num_write":227,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"261'253","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:07.364455+0000","last_change":"2024-04-22T19:51:28.514713+0000","last_active":"2024-04-22T20:46:07.364455+0000","last_peered":"2024-04-22T20:46:07.364455+0000","last_clean":"2024-04-22T20:46:07.364455+0000","last_became_active":"2024-04-22T19:51:28.514445+0000","last_became_peered":"2024-04-22T19:51:28.514445+0000","last_unstale":"2024-04-22T20:46:07.364455+0000","last_undegraded":"2024-04-22T20:46:07.364455+0000","last_fullsized":"2024-04-22T20:46:07.364455+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:39.638389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":723,"num_write":394,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'395","reported_seq":1141,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.918732+0000","last_change":"2024-04-22T19:51:26.496789+0000","last_active":"2024-04-22T20:48:17.918732+0000","last_peered":"2024-04-22T20:48:17.918732+0000","last_clean":"2024-04-22T20:48:17.918732+0000","last_became_active":"2024-04-22T19:51:26.496408+0000","last_became_peered":"2024-04-22T19:51:26.496408+0000","last_unstale":"2024-04-22T20:48:17.918732+0000","last_undegraded":"2024-04-22T20:48:17.918732+0000","last_fullsized":"2024-04-22T20:48:17.918732+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:19:23.202029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":840,"num_read_kb":8023,"num_write":1479,"num_write_kb":27015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.044898+0000","last_change":"2024-04-22T19:51:30.527588+0000","last_active":"2024-04-22T20:45:30.044898+0000","last_peered":"2024-04-22T20:45:30.044898+0000","last_clean":"2024-04-22T20:45:30.044898+0000","last_became_active":"2024-04-22T19:51:30.527309+0000","last_became_peered":"2024-04-22T19:51:30.527309+0000","last_unstale":"2024-04-22T20:45:30.044898+0000","last_undegraded":"2024-04-22T20:45:30.044898+0000","last_fullsized":"2024-04-22T20:45:30.044898+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:22:07.894332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"261'308","reported_seq":904,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.415817+0000","last_change":"2024-04-22T19:51:32.540003+0000","last_active":"2024-04-22T20:48:17.415817+0000","last_peered":"2024-04-22T20:48:17.415817+0000","last_clean":"2024-04-22T20:48:17.415817+0000","last_became_active":"2024-04-22T19:51:32.539732+0000","last_became_peered":"2024-04-22T19:51:32.539732+0000","last_unstale":"2024-04-22T20:48:17.415817+0000","last_undegraded":"2024-04-22T20:48:17.415817+0000","last_fullsized":"2024-04-22T20:48:17.415817+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":200,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:19.413189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52364,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":512,"num_read_kb":1133,"num_write":408,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"261'490","reported_seq":2203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.354013+0000","last_change":"2024-04-22T19:51:34.556537+0000","last_active":"2024-04-22T20:48:17.354013+0000","last_peered":"2024-04-22T20:48:17.354013+0000","last_clean":"2024-04-22T20:48:17.354013+0000","last_became_active":"2024-04-22T19:51:34.556124+0000","last_became_peered":"2024-04-22T19:51:34.556124+0000","last_unstale":"2024-04-22T20:48:17.354013+0000","last_undegraded":"2024-04-22T20:48:17.354013+0000","last_fullsized":"2024-04-22T20:48:17.354013+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":400,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:14:07.299855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"261'116","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.060760+0000","last_change":"2024-04-22T19:51:36.573163+0000","last_active":"2024-04-22T20:47:55.060760+0000","last_peered":"2024-04-22T20:47:55.060760+0000","last_clean":"2024-04-22T20:47:55.060760+0000","last_became_active":"2024-04-22T19:51:36.572913+0000","last_became_peered":"2024-04-22T19:51:36.572913+0000","last_unstale":"2024-04-22T20:47:55.060760+0000","last_undegraded":"2024-04-22T20:47:55.060760+0000","last_fullsized":"2024-04-22T20:47:55.060760+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:40:20.320547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":102,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"261'338","reported_seq":1080,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.179830+0000","last_change":"2024-04-22T19:51:28.516099+0000","last_active":"2024-04-22T20:45:30.179830+0000","last_peered":"2024-04-22T20:45:30.179830+0000","last_clean":"2024-04-22T20:45:30.179830+0000","last_became_active":"2024-04-22T19:51:28.515809+0000","last_became_peered":"2024-04-22T19:51:28.515809+0000","last_unstale":"2024-04-22T20:45:30.179830+0000","last_undegraded":"2024-04-22T20:45:30.179830+0000","last_fullsized":"2024-04-22T20:45:30.179830+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:19.069001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1504,"num_read_kb":1227,"num_write":616,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"261'336","reported_seq":971,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.878732+0000","last_change":"2024-04-22T19:51:26.499730+0000","last_active":"2024-04-22T20:48:17.878732+0000","last_peered":"2024-04-22T20:48:17.878732+0000","last_clean":"2024-04-22T20:48:17.878732+0000","last_became_active":"2024-04-22T19:51:26.499362+0000","last_became_peered":"2024-04-22T19:51:26.499362+0000","last_unstale":"2024-04-22T20:48:17.878732+0000","last_undegraded":"2024-04-22T20:48:17.878732+0000","last_fullsized":"2024-04-22T20:48:17.878732+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:28:07.429252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":26712,"num_write":1252,"num_write_kb":36357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"25'2","reported_seq":1135,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.119125+0000","last_change":"2024-04-22T19:51:30.527770+0000","last_active":"2024-04-22T20:48:15.119125+0000","last_peered":"2024-04-22T20:48:15.119125+0000","last_clean":"2024-04-22T20:48:15.119125+0000","last_became_active":"2024-04-22T19:51:30.527490+0000","last_became_peered":"2024-04-22T19:51:30.527490+0000","last_unstale":"2024-04-22T20:48:15.119125+0000","last_undegraded":"2024-04-22T20:48:15.119125+0000","last_fullsized":"2024-04-22T20:48:15.119125+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:39:09.682743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":640,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"261'90","reported_seq":620,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157133+0000","last_change":"2024-04-22T19:51:33.248919+0000","last_active":"2024-04-22T20:45:35.157133+0000","last_peered":"2024-04-22T20:45:35.157133+0000","last_clean":"2024-04-22T20:45:35.157133+0000","last_became_active":"2024-04-22T19:51:33.248349+0000","last_became_peered":"2024-04-22T19:51:33.248349+0000","last_unstale":"2024-04-22T20:45:35.157133+0000","last_undegraded":"2024-04-22T20:45:35.157133+0000","last_fullsized":"2024-04-22T20:45:35.157133+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:03:31.126924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":60,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"261'518","reported_seq":2072,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.542961+0000","last_change":"2024-04-22T19:51:34.562158+0000","last_active":"2024-04-22T20:48:15.542961+0000","last_peered":"2024-04-22T20:48:15.542961+0000","last_clean":"2024-04-22T20:48:15.542961+0000","last_became_active":"2024-04-22T19:51:34.561825+0000","last_became_peered":"2024-04-22T19:51:34.561825+0000","last_unstale":"2024-04-22T20:48:15.542961+0000","last_undegraded":"2024-04-22T20:48:15.542961+0000","last_fullsized":"2024-04-22T20:48:15.542961+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:22:52.670915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"261'130","reported_seq":625,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.182787+0000","last_change":"2024-04-22T19:51:36.573308+0000","last_active":"2024-04-22T20:45:50.182787+0000","last_peered":"2024-04-22T20:45:50.182787+0000","last_clean":"2024-04-22T20:45:50.182787+0000","last_became_active":"2024-04-22T19:51:36.573026+0000","last_became_peered":"2024-04-22T19:51:36.573026+0000","last_unstale":"2024-04-22T20:45:50.182787+0000","last_undegraded":"2024-04-22T20:45:50.182787+0000","last_fullsized":"2024-04-22T20:45:50.182787+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:19.108938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":119,"num_write":309,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"261'207","reported_seq":818,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045988+0000","last_change":"2024-04-22T19:51:28.512959+0000","last_active":"2024-04-22T20:45:40.045988+0000","last_peered":"2024-04-22T20:45:40.045988+0000","last_clean":"2024-04-22T20:45:40.045988+0000","last_became_active":"2024-04-22T19:51:28.512275+0000","last_became_peered":"2024-04-22T19:51:28.512275+0000","last_unstale":"2024-04-22T20:45:40.045988+0000","last_undegraded":"2024-04-22T20:45:40.045988+0000","last_fullsized":"2024-04-22T20:45:40.045988+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:24:20.294556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":832,"num_read_kb":677,"num_write":302,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"261'328","reported_seq":966,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.890018+0000","last_change":"2024-04-22T19:51:26.498630+0000","last_active":"2024-04-22T20:48:17.890018+0000","last_peered":"2024-04-22T20:48:17.890018+0000","last_clean":"2024-04-22T20:48:17.890018+0000","last_became_active":"2024-04-22T19:51:26.498276+0000","last_became_peered":"2024-04-22T19:51:26.498276+0000","last_unstale":"2024-04-22T20:48:17.890018+0000","last_undegraded":"2024-04-22T20:48:17.890018+0000","last_fullsized":"2024-04-22T20:48:17.890018+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:43:15.698499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1581644,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":468,"num_read_kb":30145,"num_write":1162,"num_write_kb":23135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"32'3","reported_seq":821,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.109517+0000","last_change":"2024-04-22T19:51:30.530726+0000","last_active":"2024-04-22T20:48:15.109517+0000","last_peered":"2024-04-22T20:48:15.109517+0000","last_clean":"2024-04-22T20:48:15.109517+0000","last_became_active":"2024-04-22T19:51:30.530363+0000","last_became_peered":"2024-04-22T19:51:30.530363+0000","last_unstale":"2024-04-22T20:48:15.109517+0000","last_undegraded":"2024-04-22T20:48:15.109517+0000","last_fullsized":"2024-04-22T20:48:15.109517+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:58:21.578312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":490,"num_read_kb":488,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"261'401","reported_seq":1009,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.558192+0000","last_change":"2024-04-22T19:51:32.540935+0000","last_active":"2024-04-22T20:48:16.558192+0000","last_peered":"2024-04-22T20:48:16.558192+0000","last_clean":"2024-04-22T20:48:16.558192+0000","last_became_active":"2024-04-22T19:51:32.540622+0000","last_became_peered":"2024-04-22T19:51:32.540622+0000","last_unstale":"2024-04-22T20:48:16.558192+0000","last_undegraded":"2024-04-22T20:48:16.558192+0000","last_fullsized":"2024-04-22T20:48:16.558192+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":300,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:57:03.444994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56095,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1229,"num_read_kb":1850,"num_write":1762,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.316223+0000","last_change":"2024-04-22T19:51:34.551287+0000","last_active":"2024-04-22T20:45:35.316223+0000","last_peered":"2024-04-22T20:45:35.316223+0000","last_clean":"2024-04-22T20:45:35.316223+0000","last_became_active":"2024-04-22T19:51:34.551009+0000","last_became_peered":"2024-04-22T19:51:34.551009+0000","last_unstale":"2024-04-22T20:45:35.316223+0000","last_undegraded":"2024-04-22T20:45:35.316223+0000","last_fullsized":"2024-04-22T20:45:35.316223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:24:58.026337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"261'121","reported_seq":620,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.018220+0000","last_change":"2024-04-22T19:51:36.571807+0000","last_active":"2024-04-22T20:48:04.018220+0000","last_peered":"2024-04-22T20:48:04.018220+0000","last_clean":"2024-04-22T20:48:04.018220+0000","last_became_active":"2024-04-22T19:51:36.571517+0000","last_became_peered":"2024-04-22T19:51:36.571517+0000","last_unstale":"2024-04-22T20:48:04.018220+0000","last_undegraded":"2024-04-22T20:48:04.018220+0000","last_fullsized":"2024-04-22T20:48:04.018220+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:15:52.550381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":272,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"261'4186","reported_seq":4781,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.317812+0000","last_change":"2024-04-22T19:51:28.515172+0000","last_active":"2024-04-22T20:45:45.317812+0000","last_peered":"2024-04-22T20:45:45.317812+0000","last_clean":"2024-04-22T20:45:45.317812+0000","last_became_active":"2024-04-22T19:51:28.514869+0000","last_became_peered":"2024-04-22T19:51:28.514869+0000","last_unstale":"2024-04-22T20:45:45.317812+0000","last_undegraded":"2024-04-22T20:45:45.317812+0000","last_fullsized":"2024-04-22T20:45:45.317812+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":3000,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:44.084365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14767,"num_read_kb":11981,"num_write":6295,"num_write_kb":3142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":520,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063174+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T20:45:25.063174+0000","last_peered":"2024-04-22T20:45:25.063174+0000","last_clean":"2024-04-22T20:45:25.063174+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T20:45:25.063174+0000","last_undegraded":"2024-04-22T20:45:25.063174+0000","last_fullsized":"2024-04-22T20:45:25.063174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:51:41.710258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"261'336","reported_seq":1031,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.783955+0000","last_change":"2024-04-22T19:51:26.506322+0000","last_active":"2024-04-22T20:48:17.783955+0000","last_peered":"2024-04-22T20:48:17.783955+0000","last_clean":"2024-04-22T20:48:17.783955+0000","last_became_active":"2024-04-22T19:51:26.505982+0000","last_became_peered":"2024-04-22T19:51:26.505982+0000","last_unstale":"2024-04-22T20:48:17.783955+0000","last_undegraded":"2024-04-22T20:48:17.783955+0000","last_fullsized":"2024-04-22T20:48:17.783955+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:46.483074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893637,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":542,"num_read_kb":107293,"num_write":1359,"num_write_kb":45616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.065725+0000","last_change":"2024-04-22T19:51:30.527554+0000","last_active":"2024-04-22T20:45:40.065725+0000","last_peered":"2024-04-22T20:45:40.065725+0000","last_clean":"2024-04-22T20:45:40.065725+0000","last_became_active":"2024-04-22T19:51:30.526939+0000","last_became_peered":"2024-04-22T19:51:30.526939+0000","last_unstale":"2024-04-22T20:45:40.065725+0000","last_undegraded":"2024-04-22T20:45:40.065725+0000","last_fullsized":"2024-04-22T20:45:40.065725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:34:00.754603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"261'715","reported_seq":1416,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.302291+0000","last_change":"2024-04-22T19:51:32.543191+0000","last_active":"2024-04-22T20:48:17.302291+0000","last_peered":"2024-04-22T20:48:17.302291+0000","last_clean":"2024-04-22T20:48:17.302291+0000","last_became_active":"2024-04-22T19:51:32.542970+0000","last_became_peered":"2024-04-22T19:51:32.542970+0000","last_unstale":"2024-04-22T20:48:17.302291+0000","last_undegraded":"2024-04-22T20:48:17.302291+0000","last_fullsized":"2024-04-22T20:48:17.302291+0000","mapping_epoch":26,"log_start":"261'700","ondisk_log_start":"261'700","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":700,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:18:42.030364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142271,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1403,"num_read_kb":3194,"num_write":1318,"num_write_kb":1170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183762+0000","last_change":"2024-04-22T19:51:34.554048+0000","last_active":"2024-04-22T20:45:45.183762+0000","last_peered":"2024-04-22T20:45:45.183762+0000","last_clean":"2024-04-22T20:45:45.183762+0000","last_became_active":"2024-04-22T19:51:34.553704+0000","last_became_peered":"2024-04-22T19:51:34.553704+0000","last_unstale":"2024-04-22T20:45:45.183762+0000","last_undegraded":"2024-04-22T20:45:45.183762+0000","last_fullsized":"2024-04-22T20:45:45.183762+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:19:14.613388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"261'266","reported_seq":1001,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.066411+0000","last_change":"2024-04-22T19:51:36.571003+0000","last_active":"2024-04-22T20:48:04.066411+0000","last_peered":"2024-04-22T20:48:04.066411+0000","last_clean":"2024-04-22T20:48:04.066411+0000","last_became_active":"2024-04-22T19:51:36.570717+0000","last_became_peered":"2024-04-22T19:51:36.570717+0000","last_unstale":"2024-04-22T20:48:04.066411+0000","last_undegraded":"2024-04-22T20:48:04.066411+0000","last_fullsized":"2024-04-22T20:48:04.066411+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:16.336092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":568,"num_write":495,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"261'50","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045937+0000","last_change":"2024-04-22T19:51:28.506462+0000","last_active":"2024-04-22T20:45:40.045937+0000","last_peered":"2024-04-22T20:45:40.045937+0000","last_clean":"2024-04-22T20:45:40.045937+0000","last_became_active":"2024-04-22T19:51:28.505719+0000","last_became_peered":"2024-04-22T19:51:28.505719+0000","last_unstale":"2024-04-22T20:45:40.045937+0000","last_undegraded":"2024-04-22T20:45:40.045937+0000","last_fullsized":"2024-04-22T20:45:40.045937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:02.609039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":69,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":514,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.186325+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T20:45:20.186325+0000","last_peered":"2024-04-22T20:45:20.186325+0000","last_clean":"2024-04-22T20:45:20.186325+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T20:45:20.186325+0000","last_undegraded":"2024-04-22T20:45:20.186325+0000","last_fullsized":"2024-04-22T20:45:20.186325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:20:10.880037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"261'358","reported_seq":1034,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.927315+0000","last_change":"2024-04-22T19:51:26.501306+0000","last_active":"2024-04-22T20:48:17.927315+0000","last_peered":"2024-04-22T20:48:17.927315+0000","last_clean":"2024-04-22T20:48:17.927315+0000","last_became_active":"2024-04-22T19:51:26.500968+0000","last_became_peered":"2024-04-22T19:51:26.500968+0000","last_unstale":"2024-04-22T20:48:17.927315+0000","last_undegraded":"2024-04-22T20:48:17.927315+0000","last_fullsized":"2024-04-22T20:48:17.927315+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:20:53.110538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048617,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":597,"num_read_kb":30331,"num_write":1294,"num_write_kb":32709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.184374+0000","last_change":"2024-04-22T19:51:30.526621+0000","last_active":"2024-04-22T20:45:50.184374+0000","last_peered":"2024-04-22T20:45:50.184374+0000","last_clean":"2024-04-22T20:45:50.184374+0000","last_became_active":"2024-04-22T19:51:30.526328+0000","last_became_peered":"2024-04-22T19:51:30.526328+0000","last_unstale":"2024-04-22T20:45:50.184374+0000","last_undegraded":"2024-04-22T20:45:50.184374+0000","last_fullsized":"2024-04-22T20:45:50.184374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:59:48.789109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"261'124","reported_seq":677,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.158336+0000","last_change":"2024-04-22T19:51:32.541073+0000","last_active":"2024-04-22T20:45:45.158336+0000","last_peered":"2024-04-22T20:45:45.158336+0000","last_clean":"2024-04-22T20:45:45.158336+0000","last_became_active":"2024-04-22T19:51:32.540805+0000","last_became_peered":"2024-04-22T19:51:32.540805+0000","last_unstale":"2024-04-22T20:45:45.158336+0000","last_undegraded":"2024-04-22T20:45:45.158336+0000","last_fullsized":"2024-04-22T20:45:45.158336+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:55.727535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":112,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.317842+0000","last_change":"2024-04-22T19:51:34.554535+0000","last_active":"2024-04-22T20:45:45.317842+0000","last_peered":"2024-04-22T20:45:45.317842+0000","last_clean":"2024-04-22T20:45:45.317842+0000","last_became_active":"2024-04-22T19:51:34.554307+0000","last_became_peered":"2024-04-22T19:51:34.554307+0000","last_unstale":"2024-04-22T20:45:45.317842+0000","last_undegraded":"2024-04-22T20:45:45.317842+0000","last_fullsized":"2024-04-22T20:45:45.317842+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:13.279562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"261'123","reported_seq":625,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.823391+0000","last_change":"2024-04-22T19:51:36.571438+0000","last_active":"2024-04-22T20:48:03.823391+0000","last_peered":"2024-04-22T20:48:03.823391+0000","last_clean":"2024-04-22T20:48:03.823391+0000","last_became_active":"2024-04-22T19:51:36.571130+0000","last_became_peered":"2024-04-22T19:51:36.571130+0000","last_unstale":"2024-04-22T20:48:03.823391+0000","last_undegraded":"2024-04-22T20:48:03.823391+0000","last_fullsized":"2024-04-22T20:48:03.823391+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:10:47.000615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":131,"num_write":305,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"261'90","reported_seq":658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045996+0000","last_change":"2024-04-22T19:51:28.514065+0000","last_active":"2024-04-22T20:45:40.045996+0000","last_peered":"2024-04-22T20:45:40.045996+0000","last_clean":"2024-04-22T20:45:40.045996+0000","last_became_active":"2024-04-22T19:51:28.513451+0000","last_became_peered":"2024-04-22T19:51:28.513451+0000","last_unstale":"2024-04-22T20:45:40.045996+0000","last_undegraded":"2024-04-22T20:45:40.045996+0000","last_fullsized":"2024-04-22T20:45:40.045996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:52:37.460752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":305,"num_write":130,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":519,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062391+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T20:45:20.062391+0000","last_peered":"2024-04-22T20:45:20.062391+0000","last_clean":"2024-04-22T20:45:20.062391+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T20:45:20.062391+0000","last_undegraded":"2024-04-22T20:45:20.062391+0000","last_fullsized":"2024-04-22T20:45:20.062391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:36.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"261'323","reported_seq":985,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.940204+0000","last_change":"2024-04-22T19:51:26.495730+0000","last_active":"2024-04-22T20:48:17.940204+0000","last_peered":"2024-04-22T20:48:17.940204+0000","last_clean":"2024-04-22T20:48:17.940204+0000","last_became_active":"2024-04-22T19:51:26.495405+0000","last_became_peered":"2024-04-22T19:51:26.495405+0000","last_unstale":"2024-04-22T20:48:17.940204+0000","last_undegraded":"2024-04-22T20:48:17.940204+0000","last_fullsized":"2024-04-22T20:48:17.940204+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:00.884012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264208,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":523,"num_read_kb":57840,"num_write":1265,"num_write_kb":50104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"25'1","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.068477+0000","last_change":"2024-04-22T19:51:30.521583+0000","last_active":"2024-04-22T20:45:40.068477+0000","last_peered":"2024-04-22T20:45:40.068477+0000","last_clean":"2024-04-22T20:45:40.068477+0000","last_became_active":"2024-04-22T19:51:30.521134+0000","last_became_peered":"2024-04-22T19:51:30.521134+0000","last_unstale":"2024-04-22T20:45:40.068477+0000","last_undegraded":"2024-04-22T20:45:40.068477+0000","last_fullsized":"2024-04-22T20:45:40.068477+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:49:17.105976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"261'221","reported_seq":1448,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.449853+0000","last_change":"2024-04-22T19:51:33.250383+0000","last_active":"2024-04-22T20:48:08.449853+0000","last_peered":"2024-04-22T20:48:08.449853+0000","last_clean":"2024-04-22T20:48:08.449853+0000","last_became_active":"2024-04-22T19:51:33.250082+0000","last_became_peered":"2024-04-22T19:51:33.250082+0000","last_unstale":"2024-04-22T20:48:08.449853+0000","last_undegraded":"2024-04-22T20:48:08.449853+0000","last_fullsized":"2024-04-22T20:48:08.449853+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:43:34.357226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":188,"num_read_kb":180,"num_write":143,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.183725+0000","last_change":"2024-04-22T19:51:34.548541+0000","last_active":"2024-04-22T20:45:45.183725+0000","last_peered":"2024-04-22T20:45:45.183725+0000","last_clean":"2024-04-22T20:45:45.183725+0000","last_became_active":"2024-04-22T19:51:34.546652+0000","last_became_peered":"2024-04-22T19:51:34.546652+0000","last_unstale":"2024-04-22T20:45:45.183725+0000","last_undegraded":"2024-04-22T20:45:45.183725+0000","last_fullsized":"2024-04-22T20:45:45.183725+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:09:15.877851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"261'254","reported_seq":896,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.218285+0000","last_change":"2024-04-22T19:51:36.569306+0000","last_active":"2024-04-22T20:48:03.218285+0000","last_peered":"2024-04-22T20:48:03.218285+0000","last_clean":"2024-04-22T20:48:03.218285+0000","last_became_active":"2024-04-22T19:51:36.568767+0000","last_became_peered":"2024-04-22T19:51:36.568767+0000","last_unstale":"2024-04-22T20:48:03.218285+0000","last_undegraded":"2024-04-22T20:48:03.218285+0000","last_fullsized":"2024-04-22T20:48:03.218285+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:56:39.794900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":625,"num_write":569,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"261'124","reported_seq":751,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183078+0000","last_change":"2024-04-22T19:51:28.518594+0000","last_active":"2024-04-22T20:45:40.183078+0000","last_peered":"2024-04-22T20:45:40.183078+0000","last_clean":"2024-04-22T20:45:40.183078+0000","last_became_active":"2024-04-22T19:51:28.518084+0000","last_became_peered":"2024-04-22T19:51:28.518084+0000","last_unstale":"2024-04-22T20:45:40.183078+0000","last_undegraded":"2024-04-22T20:45:40.183078+0000","last_fullsized":"2024-04-22T20:45:40.183078+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:38:23.274110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":533,"num_read_kb":444,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.044325+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T20:45:25.044325+0000","last_peered":"2024-04-22T20:45:25.044325+0000","last_clean":"2024-04-22T20:45:25.044325+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T20:45:25.044325+0000","last_undegraded":"2024-04-22T20:45:25.044325+0000","last_fullsized":"2024-04-22T20:45:25.044325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:22.264706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":668,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.063186+0000","last_change":"2024-04-22T19:51:12.892628+0000","last_active":"2024-04-22T20:45:25.063186+0000","last_peered":"2024-04-22T20:45:25.063186+0000","last_clean":"2024-04-22T20:45:25.063186+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T20:45:25.063186+0000","last_undegraded":"2024-04-22T20:45:25.063186+0000","last_fullsized":"2024-04-22T20:45:25.063186+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:34.018008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"261'340","reported_seq":983,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.927535+0000","last_change":"2024-04-22T19:51:26.497957+0000","last_active":"2024-04-22T20:48:17.927535+0000","last_peered":"2024-04-22T20:48:17.927535+0000","last_clean":"2024-04-22T20:48:17.927535+0000","last_became_active":"2024-04-22T19:51:26.497608+0000","last_became_peered":"2024-04-22T19:51:26.497608+0000","last_unstale":"2024-04-22T20:48:17.927535+0000","last_undegraded":"2024-04-22T20:48:17.927535+0000","last_fullsized":"2024-04-22T20:48:17.927535+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:02:40.473180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4198438,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":523,"num_read_kb":42688,"num_write":1248,"num_write_kb":37697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189509+0000","last_change":"2024-04-22T19:51:30.524749+0000","last_active":"2024-04-22T20:45:40.189509+0000","last_peered":"2024-04-22T20:45:40.189509+0000","last_clean":"2024-04-22T20:45:40.189509+0000","last_became_active":"2024-04-22T19:51:30.524437+0000","last_became_peered":"2024-04-22T19:51:30.524437+0000","last_unstale":"2024-04-22T20:45:40.189509+0000","last_undegraded":"2024-04-22T20:45:40.189509+0000","last_fullsized":"2024-04-22T20:45:40.189509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:08:54.055370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"261'1105","reported_seq":2643,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.838091+0000","last_change":"2024-04-22T19:51:32.538859+0000","last_active":"2024-04-22T20:48:17.838091+0000","last_peered":"2024-04-22T20:48:17.838091+0000","last_clean":"2024-04-22T20:48:17.838091+0000","last_became_active":"2024-04-22T19:51:32.538560+0000","last_became_peered":"2024-04-22T19:51:32.538560+0000","last_unstale":"2024-04-22T20:48:17.838091+0000","last_undegraded":"2024-04-22T20:48:17.838091+0000","last_fullsized":"2024-04-22T20:48:17.838091+0000","mapping_epoch":26,"log_start":"261'1000","ondisk_log_start":"261'1000","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":1000,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:19.961941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148242,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1781,"num_read_kb":3545,"num_write":1100,"num_write_kb":929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:55.067953+0000","last_change":"2024-04-22T19:51:34.557341+0000","last_active":"2024-04-22T20:45:55.067953+0000","last_peered":"2024-04-22T20:45:55.067953+0000","last_clean":"2024-04-22T20:45:55.067953+0000","last_became_active":"2024-04-22T19:51:34.557022+0000","last_became_peered":"2024-04-22T19:51:34.557022+0000","last_unstale":"2024-04-22T20:45:55.067953+0000","last_undegraded":"2024-04-22T20:45:55.067953+0000","last_fullsized":"2024-04-22T20:45:55.067953+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T19:59:49.539365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"261'163","reported_seq":671,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:00.048103+0000","last_change":"2024-04-22T19:51:36.567874+0000","last_active":"2024-04-22T20:46:00.048103+0000","last_peered":"2024-04-22T20:46:00.048103+0000","last_clean":"2024-04-22T20:46:00.048103+0000","last_became_active":"2024-04-22T19:51:36.567603+0000","last_became_peered":"2024-04-22T19:51:36.567603+0000","last_unstale":"2024-04-22T20:46:00.048103+0000","last_undegraded":"2024-04-22T20:46:00.048103+0000","last_fullsized":"2024-04-22T20:46:00.048103+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:34:18.236728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":641,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":222,"num_read_kb":216,"num_write":373,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"261'364","reported_seq":998,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:32.355984+0000","last_change":"2024-04-22T19:51:28.512889+0000","last_active":"2024-04-22T20:46:32.355984+0000","last_peered":"2024-04-22T20:46:32.355984+0000","last_clean":"2024-04-22T20:46:32.355984+0000","last_became_active":"2024-04-22T19:51:28.512561+0000","last_became_peered":"2024-04-22T19:51:28.512561+0000","last_unstale":"2024-04-22T20:46:32.355984+0000","last_undegraded":"2024-04-22T20:46:32.355984+0000","last_fullsized":"2024-04-22T20:46:32.355984+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:13.130467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1455,"num_read_kb":1128,"num_write":627,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"261'380","reported_seq":1097,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.855110+0000","last_change":"2024-04-22T19:51:26.500958+0000","last_active":"2024-04-22T20:48:17.855110+0000","last_peered":"2024-04-22T20:48:17.855110+0000","last_clean":"2024-04-22T20:48:17.855110+0000","last_became_active":"2024-04-22T19:51:26.498178+0000","last_became_peered":"2024-04-22T19:51:26.498178+0000","last_unstale":"2024-04-22T20:48:17.855110+0000","last_undegraded":"2024-04-22T20:48:17.855110+0000","last_fullsized":"2024-04-22T20:48:17.855110+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:54.358145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432486,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":709,"num_read_kb":47025,"num_write":1529,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:25.155991+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T20:45:25.155991+0000","last_peered":"2024-04-22T20:45:25.155991+0000","last_clean":"2024-04-22T20:45:25.155991+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T20:45:25.155991+0000","last_undegraded":"2024-04-22T20:45:25.155991+0000","last_fullsized":"2024-04-22T20:45:25.155991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:45.718699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.157032+0000","last_change":"2024-04-22T19:51:30.523537+0000","last_active":"2024-04-22T20:45:35.157032+0000","last_peered":"2024-04-22T20:45:35.157032+0000","last_clean":"2024-04-22T20:45:35.157032+0000","last_became_active":"2024-04-22T19:51:30.523242+0000","last_became_peered":"2024-04-22T19:51:30.523242+0000","last_unstale":"2024-04-22T20:45:35.157032+0000","last_undegraded":"2024-04-22T20:45:35.157032+0000","last_fullsized":"2024-04-22T20:45:35.157032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:21.847987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"261'707","reported_seq":1432,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.893690+0000","last_change":"2024-04-22T19:51:33.248507+0000","last_active":"2024-04-22T20:48:17.893690+0000","last_peered":"2024-04-22T20:48:17.893690+0000","last_clean":"2024-04-22T20:48:17.893690+0000","last_became_active":"2024-04-22T19:51:33.248144+0000","last_became_peered":"2024-04-22T19:51:33.248144+0000","last_unstale":"2024-04-22T20:48:17.893690+0000","last_undegraded":"2024-04-22T20:48:17.893690+0000","last_fullsized":"2024-04-22T20:48:17.893690+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":600,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:02:24.314016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130335,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1263,"num_read_kb":2915,"num_write":1018,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.158258+0000","last_change":"2024-04-22T19:51:34.562290+0000","last_active":"2024-04-22T20:45:45.158258+0000","last_peered":"2024-04-22T20:45:45.158258+0000","last_clean":"2024-04-22T20:45:45.158258+0000","last_became_active":"2024-04-22T19:51:34.562001+0000","last_became_peered":"2024-04-22T19:51:34.562001+0000","last_unstale":"2024-04-22T20:45:45.158258+0000","last_undegraded":"2024-04-22T20:45:45.158258+0000","last_fullsized":"2024-04-22T20:45:45.158258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:30:17.324055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"261'105","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:02.644513+0000","last_change":"2024-04-22T19:51:36.570607+0000","last_active":"2024-04-22T20:48:02.644513+0000","last_peered":"2024-04-22T20:48:02.644513+0000","last_clean":"2024-04-22T20:48:02.644513+0000","last_became_active":"2024-04-22T19:51:36.570134+0000","last_became_peered":"2024-04-22T19:51:36.570134+0000","last_unstale":"2024-04-22T20:48:02.644513+0000","last_undegraded":"2024-04-22T20:48:02.644513+0000","last_fullsized":"2024-04-22T20:48:02.644513+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:44.632753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":256,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"261'101","reported_seq":688,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.244225+0000","last_change":"2024-04-22T19:51:28.516624+0000","last_active":"2024-04-22T20:47:05.244225+0000","last_peered":"2024-04-22T20:47:05.244225+0000","last_clean":"2024-04-22T20:47:05.244225+0000","last_became_active":"2024-04-22T19:51:28.516178+0000","last_became_peered":"2024-04-22T19:51:28.516178+0000","last_unstale":"2024-04-22T20:47:05.244225+0000","last_undegraded":"2024-04-22T20:47:05.244225+0000","last_fullsized":"2024-04-22T20:47:05.244225+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:27:27.066102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":337,"num_write":140,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"261'359","reported_seq":1054,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.827229+0000","last_change":"2024-04-22T19:51:26.503699+0000","last_active":"2024-04-22T20:48:17.827229+0000","last_peered":"2024-04-22T20:48:17.827229+0000","last_clean":"2024-04-22T20:48:17.827229+0000","last_became_active":"2024-04-22T19:51:26.502765+0000","last_became_peered":"2024-04-22T19:51:26.502765+0000","last_unstale":"2024-04-22T20:48:17.827229+0000","last_undegraded":"2024-04-22T20:48:17.827229+0000","last_fullsized":"2024-04-22T20:48:17.827229+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":300,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:12:35.987405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4728363,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":573,"num_read_kb":127053,"num_write":1383,"num_write_kb":34012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":518,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:35.180533+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T20:45:35.180533+0000","last_peered":"2024-04-22T20:45:35.180533+0000","last_clean":"2024-04-22T20:45:35.180533+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T20:45:35.180533+0000","last_undegraded":"2024-04-22T20:45:35.180533+0000","last_fullsized":"2024-04-22T20:45:35.180533+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:03:20.993569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181286+0000","last_change":"2024-04-22T19:51:30.529790+0000","last_active":"2024-04-22T20:45:40.181286+0000","last_peered":"2024-04-22T20:45:40.181286+0000","last_clean":"2024-04-22T20:45:40.181286+0000","last_became_active":"2024-04-22T19:51:30.529437+0000","last_became_peered":"2024-04-22T19:51:30.529437+0000","last_unstale":"2024-04-22T20:45:40.181286+0000","last_undegraded":"2024-04-22T20:45:40.181286+0000","last_fullsized":"2024-04-22T20:45:40.181286+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:46:59.053426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"261'110","reported_seq":651,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182039+0000","last_change":"2024-04-22T19:51:32.534640+0000","last_active":"2024-04-22T20:45:45.182039+0000","last_peered":"2024-04-22T20:45:45.182039+0000","last_clean":"2024-04-22T20:45:45.182039+0000","last_became_active":"2024-04-22T19:51:32.534280+0000","last_became_peered":"2024-04-22T19:51:32.534280+0000","last_unstale":"2024-04-22T20:45:45.182039+0000","last_undegraded":"2024-04-22T20:45:45.182039+0000","last_fullsized":"2024-04-22T20:45:45.182039+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:52.620330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"261'536","reported_seq":2156,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.190511+0000","last_change":"2024-04-22T19:51:34.557012+0000","last_active":"2024-04-22T20:48:17.190511+0000","last_peered":"2024-04-22T20:48:17.190511+0000","last_clean":"2024-04-22T20:48:17.190511+0000","last_became_active":"2024-04-22T19:51:34.556673+0000","last_became_peered":"2024-04-22T19:51:34.556673+0000","last_unstale":"2024-04-22T20:48:17.190511+0000","last_undegraded":"2024-04-22T20:48:17.190511+0000","last_fullsized":"2024-04-22T20:48:17.190511+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:18.070149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"261'149","reported_seq":639,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182998+0000","last_change":"2024-04-22T19:51:36.575637+0000","last_active":"2024-04-22T20:45:40.182998+0000","last_peered":"2024-04-22T20:45:40.182998+0000","last_clean":"2024-04-22T20:45:40.182998+0000","last_became_active":"2024-04-22T19:51:36.575351+0000","last_became_peered":"2024-04-22T19:51:36.575351+0000","last_unstale":"2024-04-22T20:45:40.182998+0000","last_undegraded":"2024-04-22T20:45:40.182998+0000","last_fullsized":"2024-04-22T20:45:40.182998+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:38:03.020774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":80,"num_write":319,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"261'146","reported_seq":771,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182955+0000","last_change":"2024-04-22T19:51:28.514202+0000","last_active":"2024-04-22T20:45:40.182955+0000","last_peered":"2024-04-22T20:45:40.182955+0000","last_clean":"2024-04-22T20:45:40.182955+0000","last_became_active":"2024-04-22T19:51:28.513940+0000","last_became_peered":"2024-04-22T19:51:28.513940+0000","last_unstale":"2024-04-22T20:45:40.182955+0000","last_undegraded":"2024-04-22T20:45:40.182955+0000","last_fullsized":"2024-04-22T20:45:40.182955+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:14:36.426996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":550,"num_write":219,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"261'357","reported_seq":995,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.889207+0000","last_change":"2024-04-22T19:51:26.506454+0000","last_active":"2024-04-22T20:48:17.889207+0000","last_peered":"2024-04-22T20:48:17.889207+0000","last_clean":"2024-04-22T20:48:17.889207+0000","last_became_active":"2024-04-22T19:51:26.505842+0000","last_became_peered":"2024-04-22T19:51:26.505842+0000","last_unstale":"2024-04-22T20:48:17.889207+0000","last_undegraded":"2024-04-22T20:48:17.889207+0000","last_fullsized":"2024-04-22T20:48:17.889207+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:24:40.890650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308777,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":488,"num_read_kb":22191,"num_write":1115,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314053+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T20:45:20.314053+0000","last_peered":"2024-04-22T20:45:20.314053+0000","last_clean":"2024-04-22T20:45:20.314053+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T20:45:20.314053+0000","last_undegraded":"2024-04-22T20:45:20.314053+0000","last_fullsized":"2024-04-22T20:45:20.314053+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:19:01.430732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317090+0000","last_change":"2024-04-22T19:51:30.526076+0000","last_active":"2024-04-22T20:45:40.317090+0000","last_peered":"2024-04-22T20:45:40.317090+0000","last_clean":"2024-04-22T20:45:40.317090+0000","last_became_active":"2024-04-22T19:51:30.525777+0000","last_became_peered":"2024-04-22T19:51:30.525777+0000","last_unstale":"2024-04-22T20:45:40.317090+0000","last_undegraded":"2024-04-22T20:45:40.317090+0000","last_fullsized":"2024-04-22T20:45:40.317090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:18.779258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"261'179","reported_seq":746,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.046459+0000","last_change":"2024-04-22T19:51:33.248819+0000","last_active":"2024-04-22T20:45:45.046459+0000","last_peered":"2024-04-22T20:45:45.046459+0000","last_clean":"2024-04-22T20:45:45.046459+0000","last_became_active":"2024-04-22T19:51:33.248354+0000","last_became_peered":"2024-04-22T19:51:33.248354+0000","last_unstale":"2024-04-22T20:45:45.046459+0000","last_undegraded":"2024-04-22T20:45:45.046459+0000","last_fullsized":"2024-04-22T20:45:45.046459+0000","mapping_epoch":26,"log_start":"258'100","ondisk_log_start":"258'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:02:04.601587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":222,"num_read_kb":211,"num_write":148,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"261'548","reported_seq":4423,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.173841+0000","last_change":"2024-04-22T19:51:34.561431+0000","last_active":"2024-04-22T20:48:17.173841+0000","last_peered":"2024-04-22T20:48:17.173841+0000","last_clean":"2024-04-22T20:48:17.173841+0000","last_became_active":"2024-04-22T19:51:34.561206+0000","last_became_peered":"2024-04-22T19:51:34.561206+0000","last_unstale":"2024-04-22T20:48:17.173841+0000","last_undegraded":"2024-04-22T20:48:17.173841+0000","last_fullsized":"2024-04-22T20:48:17.173841+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":500,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:11:06.654339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"261'182","reported_seq":679,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.174164+0000","last_change":"2024-04-22T19:51:36.569612+0000","last_active":"2024-04-22T20:48:00.174164+0000","last_peered":"2024-04-22T20:48:00.174164+0000","last_clean":"2024-04-22T20:48:00.174164+0000","last_became_active":"2024-04-22T19:51:36.569338+0000","last_became_peered":"2024-04-22T19:51:36.569338+0000","last_unstale":"2024-04-22T20:48:00.174164+0000","last_undegraded":"2024-04-22T20:48:00.174164+0000","last_fullsized":"2024-04-22T20:48:00.174164+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:23:38.495469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":109,"num_write":422,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"261'109","reported_seq":704,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045900+0000","last_change":"2024-04-22T19:51:28.512614+0000","last_active":"2024-04-22T20:45:40.045900+0000","last_peered":"2024-04-22T20:45:40.045900+0000","last_clean":"2024-04-22T20:45:40.045900+0000","last_became_active":"2024-04-22T19:51:28.512337+0000","last_became_peered":"2024-04-22T19:51:28.512337+0000","last_unstale":"2024-04-22T20:45:40.045900+0000","last_undegraded":"2024-04-22T20:45:40.045900+0000","last_fullsized":"2024-04-22T20:45:40.045900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:06:34.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":434,"num_read_kb":362,"num_write":158,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'319","reported_seq":968,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.931920+0000","last_change":"2024-04-22T19:51:26.508311+0000","last_active":"2024-04-22T20:48:17.931920+0000","last_peered":"2024-04-22T20:48:17.931920+0000","last_clean":"2024-04-22T20:48:17.931920+0000","last_became_active":"2024-04-22T19:51:26.507804+0000","last_became_peered":"2024-04-22T19:51:26.507804+0000","last_unstale":"2024-04-22T20:48:17.931920+0000","last_undegraded":"2024-04-22T20:48:17.931920+0000","last_fullsized":"2024-04-22T20:48:17.931920+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:56:54.837990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":511,"num_read_kb":11012,"num_write":1234,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314068+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T20:45:20.314068+0000","last_peered":"2024-04-22T20:45:20.314068+0000","last_clean":"2024-04-22T20:45:20.314068+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T20:45:20.314068+0000","last_undegraded":"2024-04-22T20:45:20.314068+0000","last_fullsized":"2024-04-22T20:45:20.314068+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:23.800559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.182974+0000","last_change":"2024-04-22T19:51:30.527109+0000","last_active":"2024-04-22T20:45:40.182974+0000","last_peered":"2024-04-22T20:45:40.182974+0000","last_clean":"2024-04-22T20:45:40.182974+0000","last_became_active":"2024-04-22T19:51:30.526837+0000","last_became_peered":"2024-04-22T19:51:30.526837+0000","last_unstale":"2024-04-22T20:45:40.182974+0000","last_undegraded":"2024-04-22T20:45:40.182974+0000","last_fullsized":"2024-04-22T20:45:40.182974+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:32:13.442240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"261'542","reported_seq":1212,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.351725+0000","last_change":"2024-04-22T19:51:32.542607+0000","last_active":"2024-04-22T20:48:17.351725+0000","last_peered":"2024-04-22T20:48:17.351725+0000","last_clean":"2024-04-22T20:48:17.351725+0000","last_became_active":"2024-04-22T19:51:32.542344+0000","last_became_peered":"2024-04-22T19:51:32.542344+0000","last_unstale":"2024-04-22T20:48:17.351725+0000","last_undegraded":"2024-04-22T20:48:17.351725+0000","last_fullsized":"2024-04-22T20:48:17.351725+0000","mapping_epoch":26,"log_start":"261'500","ondisk_log_start":"261'500","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":500,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:56:47.214088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97214,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":946,"num_read_kb":2097,"num_write":792,"num_write_kb":634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.181987+0000","last_change":"2024-04-22T19:51:34.551264+0000","last_active":"2024-04-22T20:45:45.181987+0000","last_peered":"2024-04-22T20:45:45.181987+0000","last_clean":"2024-04-22T20:45:45.181987+0000","last_became_active":"2024-04-22T19:51:34.550951+0000","last_became_peered":"2024-04-22T19:51:34.550951+0000","last_unstale":"2024-04-22T20:45:45.181987+0000","last_undegraded":"2024-04-22T20:45:45.181987+0000","last_fullsized":"2024-04-22T20:45:45.181987+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:34:20.504558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"261'112","reported_seq":617,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.198696+0000","last_change":"2024-04-22T19:51:36.573230+0000","last_active":"2024-04-22T20:47:55.198696+0000","last_peered":"2024-04-22T20:47:55.198696+0000","last_clean":"2024-04-22T20:47:55.198696+0000","last_became_active":"2024-04-22T19:51:36.572976+0000","last_became_peered":"2024-04-22T19:51:36.572976+0000","last_unstale":"2024-04-22T20:47:55.198696+0000","last_undegraded":"2024-04-22T20:47:55.198696+0000","last_fullsized":"2024-04-22T20:47:55.198696+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:25:14.048628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":120,"num_write":270,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"261'130","reported_seq":681,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.160770+0000","last_change":"2024-04-22T19:51:36.574914+0000","last_active":"2024-04-22T20:48:03.160770+0000","last_peered":"2024-04-22T20:48:03.160770+0000","last_clean":"2024-04-22T20:48:03.160770+0000","last_became_active":"2024-04-22T19:51:36.574238+0000","last_became_peered":"2024-04-22T19:51:36.574238+0000","last_unstale":"2024-04-22T20:48:03.160770+0000","last_undegraded":"2024-04-22T20:48:03.160770+0000","last_fullsized":"2024-04-22T20:48:03.160770+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:09.269509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":193,"num_read_kb":192,"num_write":291,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"261'123","reported_seq":739,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176757+0000","last_change":"2024-04-22T19:51:28.515646+0000","last_active":"2024-04-22T20:45:10.176757+0000","last_peered":"2024-04-22T20:45:10.176757+0000","last_clean":"2024-04-22T20:45:10.176757+0000","last_became_active":"2024-04-22T19:51:28.515334+0000","last_became_peered":"2024-04-22T19:51:28.515334+0000","last_unstale":"2024-04-22T20:45:10.176757+0000","last_undegraded":"2024-04-22T20:45:10.176757+0000","last_fullsized":"2024-04-22T20:45:10.176757+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:47:44.805680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":424,"num_write":184,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"261'324","reported_seq":1037,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.865356+0000","last_change":"2024-04-22T19:51:26.506527+0000","last_active":"2024-04-22T20:48:17.865356+0000","last_peered":"2024-04-22T20:48:17.865356+0000","last_clean":"2024-04-22T20:48:17.865356+0000","last_became_active":"2024-04-22T19:51:26.506150+0000","last_became_peered":"2024-04-22T19:51:26.506150+0000","last_unstale":"2024-04-22T20:48:17.865356+0000","last_undegraded":"2024-04-22T20:48:17.865356+0000","last_fullsized":"2024-04-22T20:48:17.865356+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:28.117664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097285,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":716,"num_read_kb":35402,"num_write":1112,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.042550+0000","last_change":"2024-04-22T19:51:30.523906+0000","last_active":"2024-04-22T20:45:10.042550+0000","last_peered":"2024-04-22T20:45:10.042550+0000","last_clean":"2024-04-22T20:45:10.042550+0000","last_became_active":"2024-04-22T19:51:30.522903+0000","last_became_peered":"2024-04-22T19:51:30.522903+0000","last_unstale":"2024-04-22T20:45:10.042550+0000","last_undegraded":"2024-04-22T20:45:10.042550+0000","last_fullsized":"2024-04-22T20:45:10.042550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:12.331782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"261'296","reported_seq":880,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.870390+0000","last_change":"2024-04-22T19:51:32.539884+0000","last_active":"2024-04-22T20:48:16.870390+0000","last_peered":"2024-04-22T20:48:16.870390+0000","last_clean":"2024-04-22T20:48:16.870390+0000","last_became_active":"2024-04-22T19:51:32.539631+0000","last_became_peered":"2024-04-22T19:51:32.539631+0000","last_unstale":"2024-04-22T20:48:16.870390+0000","last_undegraded":"2024-04-22T20:48:16.870390+0000","last_fullsized":"2024-04-22T20:48:16.870390+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:00:28.566624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44228,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":495,"num_read_kb":1138,"num_write":404,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.154587+0000","last_change":"2024-04-22T19:51:34.556940+0000","last_active":"2024-04-22T20:45:15.154587+0000","last_peered":"2024-04-22T20:45:15.154587+0000","last_clean":"2024-04-22T20:45:15.154587+0000","last_became_active":"2024-04-22T19:51:34.556650+0000","last_became_peered":"2024-04-22T19:51:34.556650+0000","last_unstale":"2024-04-22T20:45:15.154587+0000","last_undegraded":"2024-04-22T20:45:15.154587+0000","last_fullsized":"2024-04-22T20:45:15.154587+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:47:48.478954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"261'99","reported_seq":599,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.172452+0000","last_change":"2024-04-22T19:51:36.567251+0000","last_active":"2024-04-22T20:47:45.172452+0000","last_peered":"2024-04-22T20:47:45.172452+0000","last_clean":"2024-04-22T20:47:45.172452+0000","last_became_active":"2024-04-22T19:51:36.566980+0000","last_became_peered":"2024-04-22T19:51:36.566980+0000","last_unstale":"2024-04-22T20:47:45.172452+0000","last_undegraded":"2024-04-22T20:47:45.172452+0000","last_fullsized":"2024-04-22T20:47:45.172452+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:01:00.638641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":235,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"261'130","reported_seq":734,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.042558+0000","last_change":"2024-04-22T19:51:28.517914+0000","last_active":"2024-04-22T20:45:10.042558+0000","last_peered":"2024-04-22T20:45:10.042558+0000","last_clean":"2024-04-22T20:45:10.042558+0000","last_became_active":"2024-04-22T19:51:28.517675+0000","last_became_peered":"2024-04-22T19:51:28.517675+0000","last_unstale":"2024-04-22T20:45:10.042558+0000","last_undegraded":"2024-04-22T20:45:10.042558+0000","last_fullsized":"2024-04-22T20:45:10.042558+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:18:10.255589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":543,"num_read_kb":446,"num_write":195,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"261'341","reported_seq":1041,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.856136+0000","last_change":"2024-04-22T19:51:26.502622+0000","last_active":"2024-04-22T20:48:17.856136+0000","last_peered":"2024-04-22T20:48:17.856136+0000","last_clean":"2024-04-22T20:48:17.856136+0000","last_became_active":"2024-04-22T19:51:26.501897+0000","last_became_peered":"2024-04-22T19:51:26.501897+0000","last_unstale":"2024-04-22T20:48:17.856136+0000","last_undegraded":"2024-04-22T20:48:17.856136+0000","last_fullsized":"2024-04-22T20:48:17.856136+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:33.272475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3673455,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":613,"num_read_kb":41436,"num_write":1146,"num_write_kb":29344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.154110+0000","last_change":"2024-04-22T19:51:30.524665+0000","last_active":"2024-04-22T20:45:10.154110+0000","last_peered":"2024-04-22T20:45:10.154110+0000","last_clean":"2024-04-22T20:45:10.154110+0000","last_became_active":"2024-04-22T19:51:30.524388+0000","last_became_peered":"2024-04-22T19:51:30.524388+0000","last_unstale":"2024-04-22T20:45:10.154110+0000","last_undegraded":"2024-04-22T20:45:10.154110+0000","last_fullsized":"2024-04-22T20:45:10.154110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:58.393800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"261'49","reported_seq":553,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179874+0000","last_change":"2024-04-22T19:51:32.539546+0000","last_active":"2024-04-22T20:45:15.179874+0000","last_peered":"2024-04-22T20:45:15.179874+0000","last_clean":"2024-04-22T20:45:15.179874+0000","last_became_active":"2024-04-22T19:51:32.539260+0000","last_became_peered":"2024-04-22T19:51:32.539260+0000","last_unstale":"2024-04-22T20:45:15.179874+0000","last_undegraded":"2024-04-22T20:45:15.179874+0000","last_fullsized":"2024-04-22T20:45:15.179874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:13:32.092655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":36,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.061461+0000","last_change":"2024-04-22T19:51:34.551584+0000","last_active":"2024-04-22T20:45:15.061461+0000","last_peered":"2024-04-22T20:45:15.061461+0000","last_clean":"2024-04-22T20:45:15.061461+0000","last_became_active":"2024-04-22T19:51:34.551294+0000","last_became_peered":"2024-04-22T19:51:34.551294+0000","last_unstale":"2024-04-22T20:45:15.061461+0000","last_undegraded":"2024-04-22T20:45:15.061461+0000","last_fullsized":"2024-04-22T20:45:15.061461+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:53:58.560478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"261'201","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.005045+0000","last_change":"2024-04-22T19:51:36.574227+0000","last_active":"2024-04-22T20:48:03.005045+0000","last_peered":"2024-04-22T20:48:03.005045+0000","last_clean":"2024-04-22T20:48:03.005045+0000","last_became_active":"2024-04-22T19:51:36.573936+0000","last_became_peered":"2024-04-22T19:51:36.573936+0000","last_unstale":"2024-04-22T20:48:03.005045+0000","last_undegraded":"2024-04-22T20:48:03.005045+0000","last_fullsized":"2024-04-22T20:48:03.005045+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:05:17.804766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":284,"num_write":451,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"261'231","reported_seq":917,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.154075+0000","last_change":"2024-04-22T19:51:28.508566+0000","last_active":"2024-04-22T20:45:10.154075+0000","last_peered":"2024-04-22T20:45:10.154075+0000","last_clean":"2024-04-22T20:45:10.154075+0000","last_became_active":"2024-04-22T19:51:28.508307+0000","last_became_peered":"2024-04-22T19:51:28.508307+0000","last_unstale":"2024-04-22T20:45:10.154075+0000","last_undegraded":"2024-04-22T20:45:10.154075+0000","last_fullsized":"2024-04-22T20:45:10.154075+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:48.292329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":959,"num_read_kb":784,"num_write":348,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"261'323","reported_seq":954,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.932259+0000","last_change":"2024-04-22T19:51:26.501920+0000","last_active":"2024-04-22T20:48:17.932259+0000","last_peered":"2024-04-22T20:48:17.932259+0000","last_clean":"2024-04-22T20:48:17.932259+0000","last_became_active":"2024-04-22T19:51:26.501477+0000","last_became_peered":"2024-04-22T19:51:26.501477+0000","last_unstale":"2024-04-22T20:48:17.932259+0000","last_undegraded":"2024-04-22T20:48:17.932259+0000","last_fullsized":"2024-04-22T20:48:17.932259+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:19:06.904602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":46522,"num_write":1280,"num_write_kb":35021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.060725+0000","last_change":"2024-04-22T19:51:30.527405+0000","last_active":"2024-04-22T20:45:10.060725+0000","last_peered":"2024-04-22T20:45:10.060725+0000","last_clean":"2024-04-22T20:45:10.060725+0000","last_became_active":"2024-04-22T19:51:30.527076+0000","last_became_peered":"2024-04-22T19:51:30.527076+0000","last_unstale":"2024-04-22T20:45:10.060725+0000","last_undegraded":"2024-04-22T20:45:10.060725+0000","last_fullsized":"2024-04-22T20:45:10.060725+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:11:38.558657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"261'302","reported_seq":902,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.039881+0000","last_change":"2024-04-22T19:51:32.542586+0000","last_active":"2024-04-22T20:48:16.039881+0000","last_peered":"2024-04-22T20:48:16.039881+0000","last_clean":"2024-04-22T20:48:16.039881+0000","last_became_active":"2024-04-22T19:51:32.542262+0000","last_became_peered":"2024-04-22T19:51:32.542262+0000","last_unstale":"2024-04-22T20:48:16.039881+0000","last_undegraded":"2024-04-22T20:48:16.039881+0000","last_fullsized":"2024-04-22T20:48:16.039881+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:52:10.014200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46510,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":491,"num_read_kb":1038,"num_write":384,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.313200+0000","last_change":"2024-04-22T19:51:34.554113+0000","last_active":"2024-04-22T20:45:15.313200+0000","last_peered":"2024-04-22T20:45:15.313200+0000","last_clean":"2024-04-22T20:45:15.313200+0000","last_became_active":"2024-04-22T19:51:34.553691+0000","last_became_peered":"2024-04-22T19:51:34.553691+0000","last_unstale":"2024-04-22T20:45:15.313200+0000","last_undegraded":"2024-04-22T20:45:15.313200+0000","last_fullsized":"2024-04-22T20:45:15.313200+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:08:07.000668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"261'135","reported_seq":616,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:10.176306+0000","last_change":"2024-04-22T19:51:36.571557+0000","last_active":"2024-04-22T20:44:10.176306+0000","last_peered":"2024-04-22T20:44:10.176306+0000","last_clean":"2024-04-22T20:44:10.176306+0000","last_became_active":"2024-04-22T19:51:36.571290+0000","last_became_peered":"2024-04-22T19:51:36.571290+0000","last_unstale":"2024-04-22T20:44:10.176306+0000","last_undegraded":"2024-04-22T20:44:10.176306+0000","last_fullsized":"2024-04-22T20:44:10.176306+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":100,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:42:28.669764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":111,"num_write":334,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"261'185","reported_seq":854,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:01.148987+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:47:01.148987+0000","last_peered":"2024-04-22T20:47:01.148987+0000","last_clean":"2024-04-22T20:47:01.148987+0000","last_became_active":"2024-04-22T19:51:28.506615+0000","last_became_peered":"2024-04-22T19:51:28.506615+0000","last_unstale":"2024-04-22T20:47:01.148987+0000","last_undegraded":"2024-04-22T20:47:01.148987+0000","last_fullsized":"2024-04-22T20:47:01.148987+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:25.397652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":757,"num_read_kb":623,"num_write":269,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'338","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.924122+0000","last_change":"2024-04-22T19:51:26.503480+0000","last_active":"2024-04-22T20:48:17.924122+0000","last_peered":"2024-04-22T20:48:17.924122+0000","last_clean":"2024-04-22T20:48:17.924122+0000","last_became_active":"2024-04-22T19:51:26.502938+0000","last_became_peered":"2024-04-22T19:51:26.502938+0000","last_unstale":"2024-04-22T20:48:17.924122+0000","last_undegraded":"2024-04-22T20:48:17.924122+0000","last_fullsized":"2024-04-22T20:48:17.924122+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:25.122842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194317,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":478,"num_read_kb":28738,"num_write":1269,"num_write_kb":45425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.312594+0000","last_change":"2024-04-22T19:51:30.527273+0000","last_active":"2024-04-22T20:45:10.312594+0000","last_peered":"2024-04-22T20:45:10.312594+0000","last_clean":"2024-04-22T20:45:10.312594+0000","last_became_active":"2024-04-22T19:51:30.526994+0000","last_became_peered":"2024-04-22T19:51:30.526994+0000","last_unstale":"2024-04-22T20:45:10.312594+0000","last_undegraded":"2024-04-22T20:45:10.312594+0000","last_fullsized":"2024-04-22T20:45:10.312594+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:05:35.522811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"261'251","reported_seq":822,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.792692+0000","last_change":"2024-04-22T19:51:32.538193+0000","last_active":"2024-04-22T20:48:15.792692+0000","last_peered":"2024-04-22T20:48:15.792692+0000","last_clean":"2024-04-22T20:48:15.792692+0000","last_became_active":"2024-04-22T19:51:32.537875+0000","last_became_peered":"2024-04-22T19:51:32.537875+0000","last_unstale":"2024-04-22T20:48:15.792692+0000","last_undegraded":"2024-04-22T20:48:15.792692+0000","last_fullsized":"2024-04-22T20:48:15.792692+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:43.382645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":422,"num_read_kb":961,"num_write":338,"num_write_kb":268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:15.179968+0000","last_change":"2024-04-22T19:51:34.554251+0000","last_active":"2024-04-22T20:45:15.179968+0000","last_peered":"2024-04-22T20:45:15.179968+0000","last_clean":"2024-04-22T20:45:15.179968+0000","last_became_active":"2024-04-22T19:51:34.553954+0000","last_became_peered":"2024-04-22T19:51:34.553954+0000","last_unstale":"2024-04-22T20:45:15.179968+0000","last_undegraded":"2024-04-22T20:45:15.179968+0000","last_fullsized":"2024-04-22T20:45:15.179968+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:25:45.427696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"261'192","reported_seq":830,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317164+0000","last_change":"2024-04-22T19:51:28.514880+0000","last_active":"2024-04-22T20:45:40.317164+0000","last_peered":"2024-04-22T20:45:40.317164+0000","last_clean":"2024-04-22T20:45:40.317164+0000","last_became_active":"2024-04-22T19:51:28.514593+0000","last_became_peered":"2024-04-22T19:51:28.514593+0000","last_unstale":"2024-04-22T20:45:40.317164+0000","last_undegraded":"2024-04-22T20:45:40.317164+0000","last_fullsized":"2024-04-22T20:45:40.317164+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:10:57.618723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":788,"num_read_kb":638,"num_write":274,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"261'342","reported_seq":1004,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.819635+0000","last_change":"2024-04-22T19:51:26.502507+0000","last_active":"2024-04-22T20:48:17.819635+0000","last_peered":"2024-04-22T20:48:17.819635+0000","last_clean":"2024-04-22T20:48:17.819635+0000","last_became_active":"2024-04-22T19:51:26.501812+0000","last_became_peered":"2024-04-22T19:51:26.501812+0000","last_unstale":"2024-04-22T20:48:17.819635+0000","last_undegraded":"2024-04-22T20:48:17.819635+0000","last_fullsized":"2024-04-22T20:48:17.819635+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:47:54.670392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857304,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":552,"num_read_kb":55834,"num_write":1162,"num_write_kb":35884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.183098+0000","last_change":"2024-04-22T19:51:30.525750+0000","last_active":"2024-04-22T20:45:40.183098+0000","last_peered":"2024-04-22T20:45:40.183098+0000","last_clean":"2024-04-22T20:45:40.183098+0000","last_became_active":"2024-04-22T19:51:30.525450+0000","last_became_peered":"2024-04-22T19:51:30.525450+0000","last_unstale":"2024-04-22T20:45:40.183098+0000","last_undegraded":"2024-04-22T20:45:40.183098+0000","last_fullsized":"2024-04-22T20:45:40.183098+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:32.196222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"261'484","reported_seq":1112,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.240507+0000","last_change":"2024-04-22T19:51:32.541007+0000","last_active":"2024-04-22T20:48:17.240507+0000","last_peered":"2024-04-22T20:48:17.240507+0000","last_clean":"2024-04-22T20:48:17.240507+0000","last_became_active":"2024-04-22T19:51:32.540755+0000","last_became_peered":"2024-04-22T19:51:32.540755+0000","last_unstale":"2024-04-22T20:48:17.240507+0000","last_undegraded":"2024-04-22T20:48:17.240507+0000","last_fullsized":"2024-04-22T20:48:17.240507+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":400,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:18.375444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93788,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":875,"num_read_kb":2058,"num_write":756,"num_write_kb":671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":482,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.175603+0000","last_change":"2024-04-22T19:51:34.553700+0000","last_active":"2024-04-22T20:44:05.175603+0000","last_peered":"2024-04-22T20:44:05.175603+0000","last_clean":"2024-04-22T20:44:05.175603+0000","last_became_active":"2024-04-22T19:51:34.553363+0000","last_became_peered":"2024-04-22T19:51:34.553363+0000","last_unstale":"2024-04-22T20:44:05.175603+0000","last_undegraded":"2024-04-22T20:44:05.175603+0000","last_fullsized":"2024-04-22T20:44:05.175603+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:13:11.618765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"261'159","reported_seq":673,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:02.819260+0000","last_change":"2024-04-22T19:51:36.569083+0000","last_active":"2024-04-22T20:48:02.819260+0000","last_peered":"2024-04-22T20:48:02.819260+0000","last_clean":"2024-04-22T20:48:02.819260+0000","last_became_active":"2024-04-22T19:51:36.568373+0000","last_became_peered":"2024-04-22T19:51:36.568373+0000","last_unstale":"2024-04-22T20:48:02.819260+0000","last_undegraded":"2024-04-22T20:48:02.819260+0000","last_fullsized":"2024-04-22T20:48:02.819260+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:00:29.149286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":213,"num_write":373,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"261'100","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:11.966474+0000","last_change":"2024-04-22T19:51:28.510922+0000","last_active":"2024-04-22T20:47:11.966474+0000","last_peered":"2024-04-22T20:47:11.966474+0000","last_clean":"2024-04-22T20:47:11.966474+0000","last_became_active":"2024-04-22T19:51:28.510629+0000","last_became_peered":"2024-04-22T19:51:28.510629+0000","last_unstale":"2024-04-22T20:47:11.966474+0000","last_undegraded":"2024-04-22T20:47:11.966474+0000","last_fullsized":"2024-04-22T20:47:11.966474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:55:55.876914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":435,"num_read_kb":357,"num_write":161,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"261'305","reported_seq":938,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.883008+0000","last_change":"2024-04-22T19:51:26.497497+0000","last_active":"2024-04-22T20:48:17.883008+0000","last_peered":"2024-04-22T20:48:17.883008+0000","last_clean":"2024-04-22T20:48:17.883008+0000","last_became_active":"2024-04-22T19:51:26.497165+0000","last_became_peered":"2024-04-22T19:51:26.497165+0000","last_unstale":"2024-04-22T20:48:17.883008+0000","last_undegraded":"2024-04-22T20:48:17.883008+0000","last_fullsized":"2024-04-22T20:48:17.883008+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:31.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":662323,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":493,"num_read_kb":39207,"num_write":1122,"num_write_kb":42297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317184+0000","last_change":"2024-04-22T19:51:30.526861+0000","last_active":"2024-04-22T20:45:40.317184+0000","last_peered":"2024-04-22T20:45:40.317184+0000","last_clean":"2024-04-22T20:45:40.317184+0000","last_became_active":"2024-04-22T19:51:30.526597+0000","last_became_peered":"2024-04-22T19:51:30.526597+0000","last_unstale":"2024-04-22T20:45:40.317184+0000","last_undegraded":"2024-04-22T20:45:40.317184+0000","last_fullsized":"2024-04-22T20:45:40.317184+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:24.696235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"261'311","reported_seq":898,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.879008+0000","last_change":"2024-04-22T19:51:32.537791+0000","last_active":"2024-04-22T20:48:15.879008+0000","last_peered":"2024-04-22T20:48:15.879008+0000","last_clean":"2024-04-22T20:48:15.879008+0000","last_became_active":"2024-04-22T19:51:32.537416+0000","last_became_peered":"2024-04-22T19:51:32.537416+0000","last_unstale":"2024-04-22T20:48:15.879008+0000","last_undegraded":"2024-04-22T20:48:15.879008+0000","last_fullsized":"2024-04-22T20:48:15.879008+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:35:00.012988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43451,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":521,"num_read_kb":1119,"num_write":436,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":484,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.035331+0000","last_change":"2024-04-22T19:51:34.561345+0000","last_active":"2024-04-22T20:44:05.035331+0000","last_peered":"2024-04-22T20:44:05.035331+0000","last_clean":"2024-04-22T20:44:05.035331+0000","last_became_active":"2024-04-22T19:51:34.561096+0000","last_became_peered":"2024-04-22T19:51:34.561096+0000","last_unstale":"2024-04-22T20:44:05.035331+0000","last_undegraded":"2024-04-22T20:44:05.035331+0000","last_fullsized":"2024-04-22T20:44:05.035331+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:38.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"261'161","reported_seq":661,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.314006+0000","last_change":"2024-04-22T19:51:36.571812+0000","last_active":"2024-04-22T20:45:20.314006+0000","last_peered":"2024-04-22T20:45:20.314006+0000","last_clean":"2024-04-22T20:45:20.314006+0000","last_became_active":"2024-04-22T19:51:36.571565+0000","last_became_peered":"2024-04-22T19:51:36.571565+0000","last_unstale":"2024-04-22T20:45:20.314006+0000","last_undegraded":"2024-04-22T20:45:20.314006+0000","last_fullsized":"2024-04-22T20:45:20.314006+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:55.978840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":180,"num_write":473,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"261'44","reported_seq":609,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.145600+0000","last_change":"2024-04-22T19:51:28.507235+0000","last_active":"2024-04-22T20:44:00.145600+0000","last_peered":"2024-04-22T20:44:00.145600+0000","last_clean":"2024-04-22T20:44:00.145600+0000","last_became_active":"2024-04-22T19:51:28.506835+0000","last_became_peered":"2024-04-22T19:51:28.506835+0000","last_unstale":"2024-04-22T20:44:00.145600+0000","last_undegraded":"2024-04-22T20:44:00.145600+0000","last_fullsized":"2024-04-22T20:44:00.145600+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:22:38.074432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":202,"num_read_kb":174,"num_write":58,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"261'377","reported_seq":1004,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.927654+0000","last_change":"2024-04-22T19:52:04.618218+0000","last_active":"2024-04-22T20:48:17.927654+0000","last_peered":"2024-04-22T20:48:17.927654+0000","last_clean":"2024-04-22T20:48:17.927654+0000","last_became_active":"2024-04-22T19:52:04.617820+0000","last_became_peered":"2024-04-22T19:52:04.617820+0000","last_unstale":"2024-04-22T20:48:17.927654+0000","last_undegraded":"2024-04-22T20:48:17.927654+0000","last_fullsized":"2024-04-22T20:48:17.927654+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:04:41.066429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14463569,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":535,"num_read_kb":31540,"num_write":1473,"num_write_kb":43247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.166735+0000","last_change":"2024-04-22T19:51:30.529632+0000","last_active":"2024-04-22T20:44:00.166735+0000","last_peered":"2024-04-22T20:44:00.166735+0000","last_clean":"2024-04-22T20:44:00.166735+0000","last_became_active":"2024-04-22T19:51:30.529322+0000","last_became_peered":"2024-04-22T19:51:30.529322+0000","last_unstale":"2024-04-22T20:44:00.166735+0000","last_undegraded":"2024-04-22T20:44:00.166735+0000","last_fullsized":"2024-04-22T20:44:00.166735+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:43:15.680192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"261'271","reported_seq":858,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.118724+0000","last_change":"2024-04-22T19:51:33.248631+0000","last_active":"2024-04-22T20:48:17.118724+0000","last_peered":"2024-04-22T20:48:17.118724+0000","last_clean":"2024-04-22T20:48:17.118724+0000","last_became_active":"2024-04-22T19:51:33.248293+0000","last_became_peered":"2024-04-22T19:51:33.248293+0000","last_unstale":"2024-04-22T20:48:17.118724+0000","last_undegraded":"2024-04-22T20:48:17.118724+0000","last_fullsized":"2024-04-22T20:48:17.118724+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:49.051719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56069,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1056,"num_write":364,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.046532+0000","last_change":"2024-04-22T19:51:34.561273+0000","last_active":"2024-04-22T20:45:45.046532+0000","last_peered":"2024-04-22T20:45:45.046532+0000","last_clean":"2024-04-22T20:45:45.046532+0000","last_became_active":"2024-04-22T19:51:34.560955+0000","last_became_peered":"2024-04-22T19:51:34.560955+0000","last_unstale":"2024-04-22T20:45:45.046532+0000","last_undegraded":"2024-04-22T20:45:45.046532+0000","last_fullsized":"2024-04-22T20:45:45.046532+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:03:55.584983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"261'119","reported_seq":602,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.536618+0000","last_change":"2024-04-22T19:51:36.567411+0000","last_active":"2024-04-22T20:48:08.536618+0000","last_peered":"2024-04-22T20:48:08.536618+0000","last_clean":"2024-04-22T20:48:08.536618+0000","last_became_active":"2024-04-22T19:51:36.567139+0000","last_became_peered":"2024-04-22T19:51:36.567139+0000","last_unstale":"2024-04-22T20:48:08.536618+0000","last_undegraded":"2024-04-22T20:48:08.536618+0000","last_fullsized":"2024-04-22T20:48:08.536618+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:39:35.365237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":92,"num_write":268,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"261'106","reported_seq":688,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.058728+0000","last_change":"2024-04-22T19:51:28.512939+0000","last_active":"2024-04-22T20:44:00.058728+0000","last_peered":"2024-04-22T20:44:00.058728+0000","last_clean":"2024-04-22T20:44:00.058728+0000","last_became_active":"2024-04-22T19:51:28.512642+0000","last_became_peered":"2024-04-22T19:51:28.512642+0000","last_unstale":"2024-04-22T20:44:00.058728+0000","last_undegraded":"2024-04-22T20:44:00.058728+0000","last_fullsized":"2024-04-22T20:44:00.058728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:36.902881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":349,"num_write":169,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"261'328","reported_seq":938,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.903251+0000","last_change":"2024-04-22T19:51:26.512313+0000","last_active":"2024-04-22T20:48:17.903251+0000","last_peered":"2024-04-22T20:48:17.903251+0000","last_clean":"2024-04-22T20:48:17.903251+0000","last_became_active":"2024-04-22T19:51:26.511966+0000","last_became_peered":"2024-04-22T19:51:26.511966+0000","last_unstale":"2024-04-22T20:48:17.903251+0000","last_undegraded":"2024-04-22T20:48:17.903251+0000","last_fullsized":"2024-04-22T20:48:17.903251+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:52.017119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262244,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":436,"num_read_kb":38275,"num_write":1102,"num_write_kb":51509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.034785+0000","last_change":"2024-04-22T19:51:30.527261+0000","last_active":"2024-04-22T20:44:00.034785+0000","last_peered":"2024-04-22T20:44:00.034785+0000","last_clean":"2024-04-22T20:44:00.034785+0000","last_became_active":"2024-04-22T19:51:30.526949+0000","last_became_peered":"2024-04-22T19:51:30.526949+0000","last_unstale":"2024-04-22T20:44:00.034785+0000","last_undegraded":"2024-04-22T20:44:00.034785+0000","last_fullsized":"2024-04-22T20:44:00.034785+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:13:41.016976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"261'103","reported_seq":629,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:45.182111+0000","last_change":"2024-04-22T19:51:33.248558+0000","last_active":"2024-04-22T20:45:45.182111+0000","last_peered":"2024-04-22T20:45:45.182111+0000","last_clean":"2024-04-22T20:45:45.182111+0000","last_became_active":"2024-04-22T19:51:33.248113+0000","last_became_peered":"2024-04-22T19:51:33.248113+0000","last_unstale":"2024-04-22T20:45:45.182111+0000","last_undegraded":"2024-04-22T20:45:45.182111+0000","last_fullsized":"2024-04-22T20:45:45.182111+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:09.138747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":119,"num_read_kb":113,"num_write":116,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"261'518","reported_seq":2297,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.538127+0000","last_change":"2024-04-22T19:51:34.557105+0000","last_active":"2024-04-22T20:48:15.538127+0000","last_peered":"2024-04-22T20:48:15.538127+0000","last_clean":"2024-04-22T20:48:15.538127+0000","last_became_active":"2024-04-22T19:51:34.556789+0000","last_became_peered":"2024-04-22T19:51:34.556789+0000","last_unstale":"2024-04-22T20:48:15.538127+0000","last_undegraded":"2024-04-22T20:48:15.538127+0000","last_fullsized":"2024-04-22T20:48:15.538127+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:07.274392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"261'96","reported_seq":594,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:20.062299+0000","last_change":"2024-04-22T19:51:36.571445+0000","last_active":"2024-04-22T20:45:20.062299+0000","last_peered":"2024-04-22T20:45:20.062299+0000","last_clean":"2024-04-22T20:45:20.062299+0000","last_became_active":"2024-04-22T19:51:36.571175+0000","last_became_peered":"2024-04-22T19:51:36.571175+0000","last_unstale":"2024-04-22T20:45:20.062299+0000","last_undegraded":"2024-04-22T20:45:20.062299+0000","last_fullsized":"2024-04-22T20:45:20.062299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:18.054128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":221,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"261'131","reported_seq":708,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:14.252216+0000","last_change":"2024-04-22T19:51:28.509477+0000","last_active":"2024-04-22T20:45:14.252216+0000","last_peered":"2024-04-22T20:45:14.252216+0000","last_clean":"2024-04-22T20:45:14.252216+0000","last_became_active":"2024-04-22T19:51:28.509110+0000","last_became_peered":"2024-04-22T19:51:28.509110+0000","last_unstale":"2024-04-22T20:45:14.252216+0000","last_undegraded":"2024-04-22T20:45:14.252216+0000","last_fullsized":"2024-04-22T20:45:14.252216+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:10:52.641173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":519,"num_read_kb":420,"num_write":188,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"261'402","reported_seq":1028,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.919981+0000","last_change":"2024-04-22T19:52:04.619673+0000","last_active":"2024-04-22T20:48:17.919981+0000","last_peered":"2024-04-22T20:48:17.919981+0000","last_clean":"2024-04-22T20:48:17.919981+0000","last_became_active":"2024-04-22T19:52:04.619385+0000","last_became_peered":"2024-04-22T19:52:04.619385+0000","last_unstale":"2024-04-22T20:48:17.919981+0000","last_undegraded":"2024-04-22T20:48:17.919981+0000","last_fullsized":"2024-04-22T20:48:17.919981+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:11.756964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682319,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":541,"num_read_kb":21001,"num_write":1452,"num_write_kb":33571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.049743+0000","last_change":"2024-04-22T19:51:30.527064+0000","last_active":"2024-04-22T20:44:00.049743+0000","last_peered":"2024-04-22T20:44:00.049743+0000","last_clean":"2024-04-22T20:44:00.049743+0000","last_became_active":"2024-04-22T19:51:30.526469+0000","last_became_peered":"2024-04-22T19:51:30.526469+0000","last_unstale":"2024-04-22T20:44:00.049743+0000","last_undegraded":"2024-04-22T20:44:00.049743+0000","last_fullsized":"2024-04-22T20:44:00.049743+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:45:04.946224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"261'306","reported_seq":890,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.483975+0000","last_change":"2024-04-22T19:51:32.540830+0000","last_active":"2024-04-22T20:48:17.483975+0000","last_peered":"2024-04-22T20:48:17.483975+0000","last_clean":"2024-04-22T20:48:17.483975+0000","last_became_active":"2024-04-22T19:51:32.540573+0000","last_became_peered":"2024-04-22T19:51:32.540573+0000","last_unstale":"2024-04-22T20:48:17.483975+0000","last_undegraded":"2024-04-22T20:48:17.483975+0000","last_fullsized":"2024-04-22T20:48:17.483975+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:49.905368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48433,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":496,"num_read_kb":1091,"num_write":394,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.176362+0000","last_change":"2024-04-22T19:51:34.554124+0000","last_active":"2024-04-22T20:44:45.176362+0000","last_peered":"2024-04-22T20:44:45.176362+0000","last_clean":"2024-04-22T20:44:45.176362+0000","last_became_active":"2024-04-22T19:51:34.553857+0000","last_became_peered":"2024-04-22T19:51:34.553857+0000","last_unstale":"2024-04-22T20:44:45.176362+0000","last_undegraded":"2024-04-22T20:44:45.176362+0000","last_fullsized":"2024-04-22T20:44:45.176362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:38:56.150478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"261'153","reported_seq":654,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:15.079446+0000","last_change":"2024-04-22T19:51:36.575509+0000","last_active":"2024-04-22T20:47:15.079446+0000","last_peered":"2024-04-22T20:47:15.079446+0000","last_clean":"2024-04-22T20:47:15.079446+0000","last_became_active":"2024-04-22T19:51:36.575211+0000","last_became_peered":"2024-04-22T19:51:36.575211+0000","last_unstale":"2024-04-22T20:47:15.079446+0000","last_undegraded":"2024-04-22T20:47:15.079446+0000","last_fullsized":"2024-04-22T20:47:15.079446+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:46:54.630277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":158,"num_write":375,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"261'85","reported_seq":680,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.024455+0000","last_change":"2024-04-22T19:51:28.508354+0000","last_active":"2024-04-22T20:48:04.024455+0000","last_peered":"2024-04-22T20:48:04.024455+0000","last_clean":"2024-04-22T20:48:04.024455+0000","last_became_active":"2024-04-22T19:51:28.508145+0000","last_became_peered":"2024-04-22T19:51:28.508145+0000","last_unstale":"2024-04-22T20:48:04.024455+0000","last_undegraded":"2024-04-22T20:48:04.024455+0000","last_fullsized":"2024-04-22T20:48:04.024455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:54:32.701108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":376,"num_read_kb":320,"num_write":115,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"261'271","reported_seq":889,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.932406+0000","last_change":"2024-04-22T19:51:26.504671+0000","last_active":"2024-04-22T20:48:17.932406+0000","last_peered":"2024-04-22T20:48:17.932406+0000","last_clean":"2024-04-22T20:48:17.932406+0000","last_became_active":"2024-04-22T19:51:26.504301+0000","last_became_peered":"2024-04-22T19:51:26.504301+0000","last_unstale":"2024-04-22T20:48:17.932406+0000","last_undegraded":"2024-04-22T20:48:17.932406+0000","last_fullsized":"2024-04-22T20:48:17.932406+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:56:36.267794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":4555,"num_write":1089,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:00.174938+0000","last_change":"2024-04-22T19:51:30.524895+0000","last_active":"2024-04-22T20:44:00.174938+0000","last_peered":"2024-04-22T20:44:00.174938+0000","last_clean":"2024-04-22T20:44:00.174938+0000","last_became_active":"2024-04-22T19:51:30.524417+0000","last_became_peered":"2024-04-22T19:51:30.524417+0000","last_unstale":"2024-04-22T20:44:00.174938+0000","last_undegraded":"2024-04-22T20:44:00.174938+0000","last_fullsized":"2024-04-22T20:44:00.174938+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:39:15.056196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"261'229","reported_seq":798,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.730107+0000","last_change":"2024-04-22T19:51:32.540923+0000","last_active":"2024-04-22T20:48:17.730107+0000","last_peered":"2024-04-22T20:48:17.730107+0000","last_clean":"2024-04-22T20:48:17.730107+0000","last_became_active":"2024-04-22T19:51:32.540659+0000","last_became_peered":"2024-04-22T19:51:32.540659+0000","last_unstale":"2024-04-22T20:48:17.730107+0000","last_undegraded":"2024-04-22T20:48:17.730107+0000","last_fullsized":"2024-04-22T20:48:17.730107+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:42:58.310584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16866,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":857,"num_write":302,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.309863+0000","last_change":"2024-04-22T19:51:34.552587+0000","last_active":"2024-04-22T20:44:50.309863+0000","last_peered":"2024-04-22T20:44:50.309863+0000","last_clean":"2024-04-22T20:44:50.309863+0000","last_became_active":"2024-04-22T19:51:34.552344+0000","last_became_peered":"2024-04-22T19:51:34.552344+0000","last_unstale":"2024-04-22T20:44:50.309863+0000","last_undegraded":"2024-04-22T20:44:50.309863+0000","last_fullsized":"2024-04-22T20:44:50.309863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:41.886857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"261'137","reported_seq":675,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:01.745300+0000","last_change":"2024-04-22T19:51:36.567945+0000","last_active":"2024-04-22T20:48:01.745300+0000","last_peered":"2024-04-22T20:48:01.745300+0000","last_clean":"2024-04-22T20:48:01.745300+0000","last_became_active":"2024-04-22T19:51:36.567594+0000","last_became_peered":"2024-04-22T19:51:36.567594+0000","last_unstale":"2024-04-22T20:48:01.745300+0000","last_undegraded":"2024-04-22T20:48:01.745300+0000","last_fullsized":"2024-04-22T20:48:01.745300+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:25:38.439273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":214,"num_write":335,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"261'87","reported_seq":648,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:45.151277+0000","last_change":"2024-04-22T19:51:28.519780+0000","last_active":"2024-04-22T20:44:45.151277+0000","last_peered":"2024-04-22T20:44:45.151277+0000","last_clean":"2024-04-22T20:44:45.151277+0000","last_became_active":"2024-04-22T19:51:28.519542+0000","last_became_peered":"2024-04-22T19:51:28.519542+0000","last_unstale":"2024-04-22T20:44:45.151277+0000","last_undegraded":"2024-04-22T20:44:45.151277+0000","last_fullsized":"2024-04-22T20:44:45.151277+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:24:55.634254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":347,"num_read_kb":277,"num_write":137,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"261'398","reported_seq":1242,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.889331+0000","last_change":"2024-04-22T19:51:26.500077+0000","last_active":"2024-04-22T20:48:17.889331+0000","last_peered":"2024-04-22T20:48:17.889331+0000","last_clean":"2024-04-22T20:48:17.889331+0000","last_became_active":"2024-04-22T19:51:26.499390+0000","last_became_peered":"2024-04-22T19:51:26.499390+0000","last_unstale":"2024-04-22T20:48:17.889331+0000","last_undegraded":"2024-04-22T20:48:17.889331+0000","last_fullsized":"2024-04-22T20:48:17.889331+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:40:50.316762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4204544,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1043,"num_read_kb":88369,"num_write":1413,"num_write_kb":41583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172368+0000","last_change":"2024-04-22T19:51:30.526678+0000","last_active":"2024-04-22T20:44:40.172368+0000","last_peered":"2024-04-22T20:44:40.172368+0000","last_clean":"2024-04-22T20:44:40.172368+0000","last_became_active":"2024-04-22T19:51:30.526183+0000","last_became_peered":"2024-04-22T19:51:30.526183+0000","last_unstale":"2024-04-22T20:44:40.172368+0000","last_undegraded":"2024-04-22T20:44:40.172368+0000","last_fullsized":"2024-04-22T20:44:40.172368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:40.232417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"261'248","reported_seq":812,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.999045+0000","last_change":"2024-04-22T19:51:32.538965+0000","last_active":"2024-04-22T20:48:16.999045+0000","last_peered":"2024-04-22T20:48:16.999045+0000","last_clean":"2024-04-22T20:48:16.999045+0000","last_became_active":"2024-04-22T19:51:32.538644+0000","last_became_peered":"2024-04-22T19:51:32.538644+0000","last_unstale":"2024-04-22T20:48:16.999045+0000","last_undegraded":"2024-04-22T20:48:16.999045+0000","last_fullsized":"2024-04-22T20:48:16.999045+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:18:02.949702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45694,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":402,"num_read_kb":955,"num_write":332,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":486,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.167484+0000","last_change":"2024-04-22T19:51:34.551365+0000","last_active":"2024-04-22T20:44:05.167484+0000","last_peered":"2024-04-22T20:44:05.167484+0000","last_clean":"2024-04-22T20:44:05.167484+0000","last_became_active":"2024-04-22T19:51:34.551085+0000","last_became_peered":"2024-04-22T19:51:34.551085+0000","last_unstale":"2024-04-22T20:44:05.167484+0000","last_undegraded":"2024-04-22T20:44:05.167484+0000","last_fullsized":"2024-04-22T20:44:05.167484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:50:23.004035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"261'179","reported_seq":903,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.372427+0000","last_change":"2024-04-22T19:51:36.569537+0000","last_active":"2024-04-22T20:48:00.372427+0000","last_peered":"2024-04-22T20:48:00.372427+0000","last_clean":"2024-04-22T20:48:00.372427+0000","last_became_active":"2024-04-22T19:51:36.569234+0000","last_became_peered":"2024-04-22T19:51:36.569234+0000","last_unstale":"2024-04-22T20:48:00.372427+0000","last_undegraded":"2024-04-22T20:48:00.372427+0000","last_fullsized":"2024-04-22T20:48:00.372427+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:56:31.796866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":419,"num_read_kb":454,"num_write":429,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"261'180","reported_seq":750,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.172395+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:44:40.172395+0000","last_peered":"2024-04-22T20:44:40.172395+0000","last_clean":"2024-04-22T20:44:40.172395+0000","last_became_active":"2024-04-22T19:51:28.515506+0000","last_became_peered":"2024-04-22T19:51:28.515506+0000","last_unstale":"2024-04-22T20:44:40.172395+0000","last_undegraded":"2024-04-22T20:44:40.172395+0000","last_fullsized":"2024-04-22T20:44:40.172395+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:14:29.010830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":678,"num_read_kb":529,"num_write":279,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"261'297","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.873343+0000","last_change":"2024-04-22T19:51:26.504948+0000","last_active":"2024-04-22T20:48:17.873343+0000","last_peered":"2024-04-22T20:48:17.873343+0000","last_clean":"2024-04-22T20:48:17.873343+0000","last_became_active":"2024-04-22T19:51:26.504604+0000","last_became_peered":"2024-04-22T19:51:26.504604+0000","last_unstale":"2024-04-22T20:48:17.873343+0000","last_undegraded":"2024-04-22T20:48:17.873343+0000","last_fullsized":"2024-04-22T20:48:17.873343+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:41:11.092305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242937,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":372,"num_read_kb":9007,"num_write":952,"num_write_kb":46630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"25'1","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.040244+0000","last_change":"2024-04-22T19:51:30.523630+0000","last_active":"2024-04-22T20:44:50.040244+0000","last_peered":"2024-04-22T20:44:50.040244+0000","last_clean":"2024-04-22T20:44:50.040244+0000","last_became_active":"2024-04-22T19:51:30.523338+0000","last_became_peered":"2024-04-22T19:51:30.523338+0000","last_unstale":"2024-04-22T20:44:50.040244+0000","last_undegraded":"2024-04-22T20:44:50.040244+0000","last_fullsized":"2024-04-22T20:44:50.040244+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:00.508499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"261'310","reported_seq":895,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.179859+0000","last_change":"2024-04-22T19:51:32.537701+0000","last_active":"2024-04-22T20:48:17.179859+0000","last_peered":"2024-04-22T20:48:17.179859+0000","last_clean":"2024-04-22T20:48:17.179859+0000","last_became_active":"2024-04-22T19:51:32.537373+0000","last_became_peered":"2024-04-22T19:51:32.537373+0000","last_unstale":"2024-04-22T20:48:17.179859+0000","last_undegraded":"2024-04-22T20:48:17.179859+0000","last_fullsized":"2024-04-22T20:48:17.179859+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":200,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:43:27.042829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49488,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":1080,"num_write":394,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":486,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.171634+0000","last_change":"2024-04-22T19:51:34.555960+0000","last_active":"2024-04-22T20:44:05.171634+0000","last_peered":"2024-04-22T20:44:05.171634+0000","last_clean":"2024-04-22T20:44:05.171634+0000","last_became_active":"2024-04-22T19:51:34.555709+0000","last_became_peered":"2024-04-22T19:51:34.555709+0000","last_unstale":"2024-04-22T20:44:05.171634+0000","last_undegraded":"2024-04-22T20:44:05.171634+0000","last_fullsized":"2024-04-22T20:44:05.171634+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:02.499776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"261'144","reported_seq":628,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:32.358361+0000","last_change":"2024-04-22T19:51:36.574316+0000","last_active":"2024-04-22T20:46:32.358361+0000","last_peered":"2024-04-22T20:46:32.358361+0000","last_clean":"2024-04-22T20:46:32.358361+0000","last_became_active":"2024-04-22T19:51:36.574012+0000","last_became_peered":"2024-04-22T19:51:36.574012+0000","last_unstale":"2024-04-22T20:46:32.358361+0000","last_undegraded":"2024-04-22T20:46:32.358361+0000","last_fullsized":"2024-04-22T20:46:32.358361+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:01:01.700368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":109,"num_write":348,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"261'372","reported_seq":1037,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.045977+0000","last_change":"2024-04-22T19:51:28.518589+0000","last_active":"2024-04-22T20:45:40.045977+0000","last_peered":"2024-04-22T20:45:40.045977+0000","last_clean":"2024-04-22T20:45:40.045977+0000","last_became_active":"2024-04-22T19:51:28.518339+0000","last_became_peered":"2024-04-22T19:51:28.518339+0000","last_unstale":"2024-04-22T20:45:40.045977+0000","last_undegraded":"2024-04-22T20:45:40.045977+0000","last_fullsized":"2024-04-22T20:45:40.045977+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:15:44.877848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1526,"num_read_kb":1211,"num_write":619,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"261'324","reported_seq":1023,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.911130+0000","last_change":"2024-04-22T19:51:26.503151+0000","last_active":"2024-04-22T20:48:17.911130+0000","last_peered":"2024-04-22T20:48:17.911130+0000","last_clean":"2024-04-22T20:48:17.911130+0000","last_became_active":"2024-04-22T19:51:26.502773+0000","last_became_peered":"2024-04-22T19:51:26.502773+0000","last_unstale":"2024-04-22T20:48:17.911130+0000","last_undegraded":"2024-04-22T20:48:17.911130+0000","last_fullsized":"2024-04-22T20:48:17.911130+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:34:49.559184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4371014,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":594,"num_read_kb":14386,"num_write":1094,"num_write_kb":17858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"261'524","reported_seq":1286,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:05.175644+0000","last_change":"2024-04-22T19:52:04.617284+0000","last_active":"2024-04-22T20:44:05.175644+0000","last_peered":"2024-04-22T20:44:05.175644+0000","last_clean":"2024-04-22T20:44:05.175644+0000","last_became_active":"2024-04-22T19:52:04.617036+0000","last_became_peered":"2024-04-22T19:52:04.617036+0000","last_unstale":"2024-04-22T20:44:05.175644+0000","last_undegraded":"2024-04-22T20:44:05.175644+0000","last_fullsized":"2024-04-22T20:44:05.175644+0000","mapping_epoch":33,"log_start":"261'500","ondisk_log_start":"261'500","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":500,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:59:14.488482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2230,"num_read_kb":1847,"num_write":935,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"261'286","reported_seq":925,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.907654+0000","last_change":"2024-04-22T19:51:26.496262+0000","last_active":"2024-04-22T20:48:17.907654+0000","last_peered":"2024-04-22T20:48:17.907654+0000","last_clean":"2024-04-22T20:48:17.907654+0000","last_became_active":"2024-04-22T19:51:26.495948+0000","last_became_peered":"2024-04-22T19:51:26.495948+0000","last_unstale":"2024-04-22T20:48:17.907654+0000","last_undegraded":"2024-04-22T20:48:17.907654+0000","last_fullsized":"2024-04-22T20:48:17.907654+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:25.588593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180512,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":454,"num_read_kb":10117,"num_write":1004,"num_write_kb":22835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"261'208","reported_seq":841,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.189450+0000","last_change":"2024-04-22T19:51:28.513290+0000","last_active":"2024-04-22T20:45:40.189450+0000","last_peered":"2024-04-22T20:45:40.189450+0000","last_clean":"2024-04-22T20:45:40.189450+0000","last_became_active":"2024-04-22T19:51:28.513026+0000","last_became_peered":"2024-04-22T19:51:28.513026+0000","last_unstale":"2024-04-22T20:45:40.189450+0000","last_undegraded":"2024-04-22T20:45:40.189450+0000","last_fullsized":"2024-04-22T20:45:40.189450+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:09:31.873692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":827,"num_read_kb":665,"num_write":342,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"261'313","reported_seq":932,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.944629+0000","last_change":"2024-04-22T19:51:26.507266+0000","last_active":"2024-04-22T20:48:17.944629+0000","last_peered":"2024-04-22T20:48:17.944629+0000","last_clean":"2024-04-22T20:48:17.944629+0000","last_became_active":"2024-04-22T19:51:26.506748+0000","last_became_peered":"2024-04-22T19:51:26.506748+0000","last_unstale":"2024-04-22T20:48:17.944629+0000","last_undegraded":"2024-04-22T20:48:17.944629+0000","last_fullsized":"2024-04-22T20:48:17.944629+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:15.321922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":416,"num_read_kb":12678,"num_write":1099,"num_write_kb":41347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"261'4175","reported_seq":4791,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:29.357349+0000","last_change":"2024-04-22T19:51:28.511560+0000","last_active":"2024-04-22T20:46:29.357349+0000","last_peered":"2024-04-22T20:46:29.357349+0000","last_clean":"2024-04-22T20:46:29.357349+0000","last_became_active":"2024-04-22T19:51:28.511312+0000","last_became_peered":"2024-04-22T19:51:28.511312+0000","last_unstale":"2024-04-22T20:46:29.357349+0000","last_undegraded":"2024-04-22T20:46:29.357349+0000","last_fullsized":"2024-04-22T20:46:29.357349+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":3000,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:28:56.675918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14718,"num_read_kb":11919,"num_write":6278,"num_write_kb":3133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"261'312","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.923309+0000","last_change":"2024-04-22T19:52:05.115180+0000","last_active":"2024-04-22T20:48:17.923309+0000","last_peered":"2024-04-22T20:48:17.923309+0000","last_clean":"2024-04-22T20:48:17.923309+0000","last_became_active":"2024-04-22T19:52:05.114734+0000","last_became_peered":"2024-04-22T19:52:05.114734+0000","last_unstale":"2024-04-22T20:48:17.923309+0000","last_undegraded":"2024-04-22T20:48:17.923309+0000","last_fullsized":"2024-04-22T20:48:17.923309+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:08:25.577231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":5518,"num_write":1088,"num_write_kb":21667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.23","version":"261'170","reported_seq":851,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.006667+0000","last_change":"2024-04-22T19:51:28.516445+0000","last_active":"2024-04-22T20:48:03.006667+0000","last_peered":"2024-04-22T20:48:03.006667+0000","last_clean":"2024-04-22T20:48:03.006667+0000","last_became_active":"2024-04-22T19:51:28.516125+0000","last_became_peered":"2024-04-22T19:51:28.516125+0000","last_unstale":"2024-04-22T20:48:03.006667+0000","last_undegraded":"2024-04-22T20:48:03.006667+0000","last_fullsized":"2024-04-22T20:48:03.006667+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:01:29.663998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":741,"num_read_kb":616,"num_write":243,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"261'368","reported_seq":1014,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.851183+0000","last_change":"2024-04-22T19:52:04.619831+0000","last_active":"2024-04-22T20:48:17.851183+0000","last_peered":"2024-04-22T20:48:17.851183+0000","last_clean":"2024-04-22T20:48:17.851183+0000","last_became_active":"2024-04-22T19:52:04.619286+0000","last_became_peered":"2024-04-22T19:52:04.619286+0000","last_unstale":"2024-04-22T20:48:17.851183+0000","last_undegraded":"2024-04-22T20:48:17.851183+0000","last_fullsized":"2024-04-22T20:48:17.851183+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:58.235831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4719706,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":588,"num_read_kb":69897,"num_write":1236,"num_write_kb":36151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"261'122","reported_seq":726,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.181359+0000","last_change":"2024-04-22T19:51:28.516690+0000","last_active":"2024-04-22T20:45:40.181359+0000","last_peered":"2024-04-22T20:45:40.181359+0000","last_clean":"2024-04-22T20:45:40.181359+0000","last_became_active":"2024-04-22T19:51:28.516437+0000","last_became_peered":"2024-04-22T19:51:28.516437+0000","last_unstale":"2024-04-22T20:45:40.181359+0000","last_undegraded":"2024-04-22T20:45:40.181359+0000","last_fullsized":"2024-04-22T20:45:40.181359+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:58:44.084365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":423,"num_write":178,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"261'317","reported_seq":1051,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.815591+0000","last_change":"2024-04-22T19:51:26.498349+0000","last_active":"2024-04-22T20:48:17.815591+0000","last_peered":"2024-04-22T20:48:17.815591+0000","last_clean":"2024-04-22T20:48:17.815591+0000","last_became_active":"2024-04-22T19:51:26.498039+0000","last_became_peered":"2024-04-22T19:51:26.498039+0000","last_unstale":"2024-04-22T20:48:17.815591+0000","last_undegraded":"2024-04-22T20:48:17.815591+0000","last_fullsized":"2024-04-22T20:48:17.815591+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:12:17.714586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194319,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":766,"num_read_kb":22382,"num_write":1253,"num_write_kb":18139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"261'199","reported_seq":848,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:49.466099+0000","last_change":"2024-04-22T19:51:28.520853+0000","last_active":"2024-04-22T20:46:49.466099+0000","last_peered":"2024-04-22T20:46:49.466099+0000","last_clean":"2024-04-22T20:46:49.466099+0000","last_became_active":"2024-04-22T19:51:28.520612+0000","last_became_peered":"2024-04-22T19:51:28.520612+0000","last_unstale":"2024-04-22T20:46:49.466099+0000","last_undegraded":"2024-04-22T20:46:49.466099+0000","last_fullsized":"2024-04-22T20:46:49.466099+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:46:57.197381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":735,"num_read_kb":593,"num_write":262,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"261'365","reported_seq":1013,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.911246+0000","last_change":"2024-04-22T19:51:26.499154+0000","last_active":"2024-04-22T20:48:17.911246+0000","last_peered":"2024-04-22T20:48:17.911246+0000","last_clean":"2024-04-22T20:48:17.911246+0000","last_became_active":"2024-04-22T19:51:26.498845+0000","last_became_peered":"2024-04-22T19:51:26.498845+0000","last_unstale":"2024-04-22T20:48:17.911246+0000","last_undegraded":"2024-04-22T20:48:17.911246+0000","last_fullsized":"2024-04-22T20:48:17.911246+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:50.165203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050721,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":506,"num_read_kb":33263,"num_write":1241,"num_write_kb":30860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"261'133","reported_seq":766,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:40.317028+0000","last_change":"2024-04-22T19:51:28.520128+0000","last_active":"2024-04-22T20:45:40.317028+0000","last_peered":"2024-04-22T20:45:40.317028+0000","last_clean":"2024-04-22T20:45:40.317028+0000","last_became_active":"2024-04-22T19:51:28.519669+0000","last_became_peered":"2024-04-22T19:51:28.519669+0000","last_unstale":"2024-04-22T20:45:40.317028+0000","last_undegraded":"2024-04-22T20:45:40.317028+0000","last_fullsized":"2024-04-22T20:45:40.317028+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:18.121387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":560,"num_read_kb":466,"num_write":196,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"261'330","reported_seq":925,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.889462+0000","last_change":"2024-04-22T19:52:04.618077+0000","last_active":"2024-04-22T20:48:17.889462+0000","last_peered":"2024-04-22T20:48:17.889462+0000","last_clean":"2024-04-22T20:48:17.889462+0000","last_became_active":"2024-04-22T19:52:04.617649+0000","last_became_peered":"2024-04-22T19:52:04.617649+0000","last_unstale":"2024-04-22T20:48:17.889462+0000","last_undegraded":"2024-04-22T20:48:17.889462+0000","last_fullsized":"2024-04-22T20:48:17.889462+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:58.404399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050919,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":29074,"num_write":1030,"num_write_kb":22627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"261'128","reported_seq":709,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.179387+0000","last_change":"2024-04-22T19:51:28.521764+0000","last_active":"2024-04-22T20:45:10.179387+0000","last_peered":"2024-04-22T20:45:10.179387+0000","last_clean":"2024-04-22T20:45:10.179387+0000","last_became_active":"2024-04-22T19:51:28.521516+0000","last_became_peered":"2024-04-22T19:51:28.521516+0000","last_unstale":"2024-04-22T20:45:10.179387+0000","last_undegraded":"2024-04-22T20:45:10.179387+0000","last_fullsized":"2024-04-22T20:45:10.179387+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:13:11.122213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":542,"num_read_kb":442,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"261'338","reported_seq":1018,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.812716+0000","last_change":"2024-04-22T19:51:26.510292+0000","last_active":"2024-04-22T20:48:17.812716+0000","last_peered":"2024-04-22T20:48:17.812716+0000","last_clean":"2024-04-22T20:48:17.812716+0000","last_became_active":"2024-04-22T19:51:26.507843+0000","last_became_peered":"2024-04-22T19:51:26.507843+0000","last_unstale":"2024-04-22T20:48:17.812716+0000","last_undegraded":"2024-04-22T20:48:17.812716+0000","last_fullsized":"2024-04-22T20:48:17.812716+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:43:15.698499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5531396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":136369,"num_write":1233,"num_write_kb":55788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"261'235","reported_seq":860,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:10.176799+0000","last_change":"2024-04-22T19:51:28.515473+0000","last_active":"2024-04-22T20:45:10.176799+0000","last_peered":"2024-04-22T20:45:10.176799+0000","last_clean":"2024-04-22T20:45:10.176799+0000","last_became_active":"2024-04-22T19:51:28.515164+0000","last_became_peered":"2024-04-22T19:51:28.515164+0000","last_unstale":"2024-04-22T20:45:10.176799+0000","last_undegraded":"2024-04-22T20:45:10.176799+0000","last_fullsized":"2024-04-22T20:45:10.176799+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:38:12.199672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":950,"num_read_kb":750,"num_write":364,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"261'502","reported_seq":1202,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.848193+0000","last_change":"2024-04-22T19:51:26.494217+0000","last_active":"2024-04-22T20:48:17.848193+0000","last_peered":"2024-04-22T20:48:17.848193+0000","last_clean":"2024-04-22T20:48:17.848193+0000","last_became_active":"2024-04-22T19:51:26.493860+0000","last_became_peered":"2024-04-22T19:51:26.493860+0000","last_unstale":"2024-04-22T20:48:17.848193+0000","last_undegraded":"2024-04-22T20:48:17.848193+0000","last_fullsized":"2024-04-22T20:48:17.848193+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":400,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:56:54.837990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6292618,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":951,"num_read_kb":13872,"num_write":1439,"num_write_kb":37693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"261'251","reported_seq":929,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:37.822141+0000","last_change":"2024-04-22T19:51:28.515833+0000","last_active":"2024-04-22T20:46:37.822141+0000","last_peered":"2024-04-22T20:46:37.822141+0000","last_clean":"2024-04-22T20:46:37.822141+0000","last_became_active":"2024-04-22T19:51:28.515547+0000","last_became_peered":"2024-04-22T19:51:28.515547+0000","last_unstale":"2024-04-22T20:46:37.822141+0000","last_undegraded":"2024-04-22T20:46:37.822141+0000","last_fullsized":"2024-04-22T20:46:37.822141+0000","mapping_epoch":22,"log_start":"247'200","ondisk_log_start":"247'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:00:39.299916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1120,"num_read_kb":973,"num_write":365,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"261'324","reported_seq":954,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.900083+0000","last_change":"2024-04-22T19:51:26.500937+0000","last_active":"2024-04-22T20:48:17.900083+0000","last_peered":"2024-04-22T20:48:17.900083+0000","last_clean":"2024-04-22T20:48:17.900083+0000","last_became_active":"2024-04-22T19:51:26.500566+0000","last_became_peered":"2024-04-22T19:51:26.500566+0000","last_unstale":"2024-04-22T20:48:17.900083+0000","last_undegraded":"2024-04-22T20:48:17.900083+0000","last_fullsized":"2024-04-22T20:48:17.900083+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:25.795560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283749,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":424,"num_read_kb":45844,"num_write":1120,"num_write_kb":42819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"261'152","reported_seq":750,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:42.563356+0000","last_change":"2024-04-22T19:51:28.516044+0000","last_active":"2024-04-22T20:46:42.563356+0000","last_peered":"2024-04-22T20:46:42.563356+0000","last_clean":"2024-04-22T20:46:42.563356+0000","last_became_active":"2024-04-22T19:51:28.515764+0000","last_became_peered":"2024-04-22T19:51:28.515764+0000","last_unstale":"2024-04-22T20:46:42.563356+0000","last_undegraded":"2024-04-22T20:46:42.563356+0000","last_fullsized":"2024-04-22T20:46:42.563356+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:16:31.975652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":633,"num_read_kb":515,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"261'261","reported_seq":870,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.804913+0000","last_change":"2024-04-22T19:51:26.512810+0000","last_active":"2024-04-22T20:48:17.804913+0000","last_peered":"2024-04-22T20:48:17.804913+0000","last_clean":"2024-04-22T20:48:17.804913+0000","last_became_active":"2024-04-22T19:51:26.512467+0000","last_became_peered":"2024-04-22T19:51:26.512467+0000","last_unstale":"2024-04-22T20:48:17.804913+0000","last_undegraded":"2024-04-22T20:48:17.804913+0000","last_fullsized":"2024-04-22T20:48:17.804913+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:43:04.495670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11536625,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":377,"num_read_kb":17908,"num_write":933,"num_write_kb":43703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"261'27","reported_seq":554,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.063941+0000","last_change":"2024-04-22T19:51:28.510362+0000","last_active":"2024-04-22T20:45:30.063941+0000","last_peered":"2024-04-22T20:45:30.063941+0000","last_clean":"2024-04-22T20:45:30.063941+0000","last_became_active":"2024-04-22T19:51:28.510091+0000","last_became_peered":"2024-04-22T19:51:28.510091+0000","last_unstale":"2024-04-22T20:45:30.063941+0000","last_undegraded":"2024-04-22T20:45:30.063941+0000","last_fullsized":"2024-04-22T20:45:30.063941+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:48.200414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":95,"num_write":39,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"261'340","reported_seq":987,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.935197+0000","last_change":"2024-04-22T19:51:26.502070+0000","last_active":"2024-04-22T20:48:17.935197+0000","last_peered":"2024-04-22T20:48:17.935197+0000","last_clean":"2024-04-22T20:48:17.935197+0000","last_became_active":"2024-04-22T19:51:26.501700+0000","last_became_peered":"2024-04-22T19:51:26.501700+0000","last_unstale":"2024-04-22T20:48:17.935197+0000","last_undegraded":"2024-04-22T20:48:17.935197+0000","last_fullsized":"2024-04-22T20:48:17.935197+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:26:11.753132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231939,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":506,"num_read_kb":6050,"num_write":1241,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"261'54","reported_seq":603,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.181693+0000","last_change":"2024-04-22T19:51:28.509002+0000","last_active":"2024-04-22T20:45:30.181693+0000","last_peered":"2024-04-22T20:45:30.181693+0000","last_clean":"2024-04-22T20:45:30.181693+0000","last_became_active":"2024-04-22T19:51:28.508768+0000","last_became_peered":"2024-04-22T19:51:28.508768+0000","last_unstale":"2024-04-22T20:45:30.181693+0000","last_undegraded":"2024-04-22T20:45:30.181693+0000","last_fullsized":"2024-04-22T20:45:30.181693+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:32:52.855073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":238,"num_read_kb":196,"num_write":80,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"261'355","reported_seq":994,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.920099+0000","last_change":"2024-04-22T19:51:26.497503+0000","last_active":"2024-04-22T20:48:17.920099+0000","last_peered":"2024-04-22T20:48:17.920099+0000","last_clean":"2024-04-22T20:48:17.920099+0000","last_became_active":"2024-04-22T19:51:26.497156+0000","last_became_peered":"2024-04-22T19:51:26.497156+0000","last_unstale":"2024-04-22T20:48:17.920099+0000","last_undegraded":"2024-04-22T20:48:17.920099+0000","last_fullsized":"2024-04-22T20:48:17.920099+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:23:31.844540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437245,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":35895,"num_write":1244,"num_write_kb":35286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"261'4050","reported_seq":4612,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:30.156504+0000","last_change":"2024-04-22T19:51:28.515076+0000","last_active":"2024-04-22T20:45:30.156504+0000","last_peered":"2024-04-22T20:45:30.156504+0000","last_clean":"2024-04-22T20:45:30.156504+0000","last_became_active":"2024-04-22T19:51:28.514780+0000","last_became_peered":"2024-04-22T19:51:28.514780+0000","last_unstale":"2024-04-22T20:45:30.156504+0000","last_undegraded":"2024-04-22T20:45:30.156504+0000","last_fullsized":"2024-04-22T20:45:30.156504+0000","mapping_epoch":22,"log_start":"261'4000","ondisk_log_start":"261'4000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":3000,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:22:22.893572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14224,"num_read_kb":11535,"num_write":6070,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"261'302","reported_seq":1061,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.899599+0000","last_change":"2024-04-22T19:51:26.496549+0000","last_active":"2024-04-22T20:48:17.899599+0000","last_peered":"2024-04-22T20:48:17.899599+0000","last_clean":"2024-04-22T20:48:17.899599+0000","last_became_active":"2024-04-22T19:51:26.496214+0000","last_became_peered":"2024-04-22T19:51:26.496214+0000","last_unstale":"2024-04-22T20:48:17.899599+0000","last_undegraded":"2024-04-22T20:48:17.899599+0000","last_fullsized":"2024-04-22T20:48:17.899599+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:15:54.981915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1338117,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":584,"num_read_kb":13796,"num_write":1181,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"261'298","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:45:50.067136+0000","last_change":"2024-04-22T19:51:28.508937+0000","last_active":"2024-04-22T20:45:50.067136+0000","last_peered":"2024-04-22T20:45:50.067136+0000","last_clean":"2024-04-22T20:45:50.067136+0000","last_became_active":"2024-04-22T19:51:28.508681+0000","last_became_peered":"2024-04-22T19:51:28.508681+0000","last_unstale":"2024-04-22T20:45:50.067136+0000","last_undegraded":"2024-04-22T20:45:50.067136+0000","last_fullsized":"2024-04-22T20:45:50.067136+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:21:44.423395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1231,"num_read_kb":1000,"num_write":489,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"261'282","reported_seq":891,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.916451+0000","last_change":"2024-04-22T19:51:26.505139+0000","last_active":"2024-04-22T20:48:17.916451+0000","last_peered":"2024-04-22T20:48:17.916451+0000","last_clean":"2024-04-22T20:48:17.916451+0000","last_became_active":"2024-04-22T19:51:26.502438+0000","last_became_peered":"2024-04-22T19:51:26.502438+0000","last_unstale":"2024-04-22T20:48:17.916451+0000","last_undegraded":"2024-04-22T20:48:17.916451+0000","last_fullsized":"2024-04-22T20:48:17.916451+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:45:27.926512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194332,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":386,"num_read_kb":36638,"num_write":936,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"261'374","reported_seq":1253,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.932849+0000","last_change":"2024-04-22T19:51:26.505871+0000","last_active":"2024-04-22T20:48:17.932849+0000","last_peered":"2024-04-22T20:48:17.932849+0000","last_clean":"2024-04-22T20:48:17.932849+0000","last_became_active":"2024-04-22T19:51:26.504878+0000","last_became_peered":"2024-04-22T19:51:26.504878+0000","last_unstale":"2024-04-22T20:48:17.932849+0000","last_undegraded":"2024-04-22T20:48:17.932849+0000","last_fullsized":"2024-04-22T20:48:17.932849+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:15:57.296051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5353839,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1018,"num_read_kb":55238,"num_write":1407,"num_write_kb":52838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"261'127","reported_seq":728,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:50.177008+0000","last_change":"2024-04-22T19:51:28.521641+0000","last_active":"2024-04-22T20:44:50.177008+0000","last_peered":"2024-04-22T20:44:50.177008+0000","last_clean":"2024-04-22T20:44:50.177008+0000","last_became_active":"2024-04-22T19:51:28.521372+0000","last_became_peered":"2024-04-22T19:51:28.521372+0000","last_unstale":"2024-04-22T20:44:50.177008+0000","last_undegraded":"2024-04-22T20:44:50.177008+0000","last_fullsized":"2024-04-22T20:44:50.177008+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:54:16.451481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":525,"num_read_kb":437,"num_write":198,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"261'287","reported_seq":941,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.916528+0000","last_change":"2024-04-22T19:52:04.618718+0000","last_active":"2024-04-22T20:48:17.916528+0000","last_peered":"2024-04-22T20:48:17.916528+0000","last_clean":"2024-04-22T20:48:17.916528+0000","last_became_active":"2024-04-22T19:52:04.618273+0000","last_became_peered":"2024-04-22T19:52:04.618273+0000","last_unstale":"2024-04-22T20:48:17.916528+0000","last_undegraded":"2024-04-22T20:48:17.916528+0000","last_fullsized":"2024-04-22T20:48:17.916528+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:09.689328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1439002,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":73466,"num_write":989,"num_write_kb":40708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"261'95","reported_seq":648,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:44:40.180705+0000","last_change":"2024-04-22T19:51:28.513718+0000","last_active":"2024-04-22T20:44:40.180705+0000","last_peered":"2024-04-22T20:44:40.180705+0000","last_clean":"2024-04-22T20:44:40.180705+0000","last_became_active":"2024-04-22T19:51:28.513450+0000","last_became_peered":"2024-04-22T19:51:28.513450+0000","last_unstale":"2024-04-22T20:44:40.180705+0000","last_undegraded":"2024-04-22T20:44:40.180705+0000","last_fullsized":"2024-04-22T20:44:40.180705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:50.937435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":382,"num_read_kb":302,"num_write":152,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5385,"num_read_kb":3813,"num_write":2124,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1241,"ondisk_log_size":1241,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19082,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":13196,"num_read_kb":13004,"num_write":15956,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":38164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1853,"ondisk_log_size":1853,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":378,"ondisk_log_size":378,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1476154,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":17118,"num_read_kb":35541,"num_write":14403,"num_write_kb":11173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2207,"ondisk_log_size":2207,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1622,"num_read_kb":1927,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":85960,"num_read_kb":69763,"num_write":34673,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3889,"ondisk_log_size":3889,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":231670890,"num_objects":469,"num_object_clones":0,"num_object_copies":1407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":469,"num_whiteouts":0,"num_read":35099,"num_read_kb":2217057,"num_write":77336,"num_write_kb":2138644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":349753344,"data_stored":349753344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3387,"ondisk_log_size":3387,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":169,"num_read_kb":196064,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":185784,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575537,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":339304,"kb_used_data":51116,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":94032536,"statfs":{"total":96636764160,"available":96289316864,"internally_reserved":0,"allocated":52342784,"data_stored":50369134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":294910410},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575539,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317164,"kb_used_data":46940,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":94054676,"statfs":{"total":96636764160,"available":96311988224,"internally_reserved":0,"allocated":48066560,"data_stored":46141552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":276429254},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575538,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175788,"kb_used_data":54196,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94196052,"statfs":{"total":96636764160,"available":96456757248,"internally_reserved":0,"allocated":55496704,"data_stored":53487509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575539,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145616,"kb_used_data":51888,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94226224,"statfs":{"total":96636764160,"available":96487653376,"internally_reserved":0,"allocated":53133312,"data_stored":51218557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575539,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":336668,"kb_used_data":62464,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94035172,"statfs":{"total":96636764160,"available":96292016128,"internally_reserved":0,"allocated":63963136,"data_stored":62012545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":280623560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575540,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144468,"kb_used_data":59116,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94227372,"statfs":{"total":96636764160,"available":96488828928,"internally_reserved":0,"allocated":60534784,"data_stored":58629075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575537,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":189040,"kb_used_data":53668,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94182800,"statfs":{"total":96636764160,"available":96443187200,"internally_reserved":0,"allocated":54956032,"data_stored":53018011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575539,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305516,"kb_used_data":61800,"kb_used_omap":1,"kb_used_meta":243582,"kb_avail":94066324,"statfs":{"total":96636764160,"available":96323915776,"internally_reserved":0,"allocated":63283200,"data_stored":61318787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":249428422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":48562176,"data_stored":48562176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44916736,"data_stored":44916736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37486592,"data_stored":37486592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50307072,"data_stored":50307072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36368384,"data_stored":36368384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44924928,"data_stored":44924928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":43872256,"data_stored":43872256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":43315200,"data_stored":43315200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":5948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":4195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":3595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":6129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":4904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":3145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13631488,"data_stored":13631488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":9437184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T20:48:25.628 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.628 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.629 INFO:tasks.ceph:pgid 66.1c last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.629 INFO:tasks.ceph:pgid 65.1f last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.629 INFO:tasks.ceph:pgid 68.1b last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.629 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.630 INFO:tasks.ceph:pgid 66.1d last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.630 INFO:tasks.ceph:pgid 65.1e last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.630 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.631 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.631 INFO:tasks.ceph:pgid 66.1e last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.631 INFO:tasks.ceph:pgid 65.1d last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.632 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.632 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.633 INFO:tasks.ceph:pgid 66.1f last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.633 INFO:tasks.ceph:pgid 65.1c last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.633 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.633 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.633 INFO:tasks.ceph:pgid 66.18 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.634 INFO:tasks.ceph:pgid 65.1b last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.634 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.634 INFO:tasks.ceph:pgid 67.18 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.634 INFO:tasks.ceph:pgid 66.19 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.635 INFO:tasks.ceph:pgid 65.1a last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.635 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.635 INFO:tasks.ceph:pgid 67.1b last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.635 INFO:tasks.ceph:pgid 66.1a last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.635 INFO:tasks.ceph:pgid 65.19 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.635 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.636 INFO:tasks.ceph:pgid 67.1a last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.636 INFO:tasks.ceph:pgid 66.1b last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.636 INFO:tasks.ceph:pgid 65.18 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.636 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.636 INFO:tasks.ceph:pgid 67.15 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.636 INFO:tasks.ceph:pgid 66.14 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.637 INFO:tasks.ceph:pgid 65.17 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.637 INFO:tasks.ceph:pgid 68.13 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.637 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.637 INFO:tasks.ceph:pgid 66.15 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.637 INFO:tasks.ceph:pgid 65.16 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.637 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.638 INFO:tasks.ceph:pgid 67.17 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.638 INFO:tasks.ceph:pgid 66.16 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.638 INFO:tasks.ceph:pgid 65.15 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.638 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.638 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.639 INFO:tasks.ceph:pgid 66.17 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.639 INFO:tasks.ceph:pgid 65.14 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.639 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.639 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.639 INFO:tasks.ceph:pgid 66.10 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.640 INFO:tasks.ceph:pgid 65.13 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.640 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.640 INFO:tasks.ceph:pgid 67.10 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.640 INFO:tasks.ceph:pgid 66.11 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.640 INFO:tasks.ceph:pgid 65.12 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.640 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.641 INFO:tasks.ceph:pgid 67.13 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.641 INFO:tasks.ceph:pgid 66.12 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.641 INFO:tasks.ceph:pgid 65.11 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.641 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.641 INFO:tasks.ceph:pgid 67.12 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.641 INFO:tasks.ceph:pgid 66.13 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.642 INFO:tasks.ceph:pgid 65.10 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.642 INFO:tasks.ceph:pgid 68.a last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.642 INFO:tasks.ceph:pgid 67.d last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.642 INFO:tasks.ceph:pgid 66.c last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.642 INFO:tasks.ceph:pgid 65.f last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.642 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.643 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.643 INFO:tasks.ceph:pgid 66.d last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.643 INFO:tasks.ceph:pgid 65.e last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.643 INFO:tasks.ceph:pgid 68.8 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.643 INFO:tasks.ceph:pgid 67.f last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.644 INFO:tasks.ceph:pgid 66.e last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.644 INFO:tasks.ceph:pgid 65.d last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.644 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.644 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.644 INFO:tasks.ceph:pgid 66.f last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.644 INFO:tasks.ceph:pgid 65.c last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.645 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.645 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.645 INFO:tasks.ceph:pgid 66.8 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.645 INFO:tasks.ceph:pgid 65.b last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.646 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.646 INFO:tasks.ceph:pgid 67.8 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.646 INFO:tasks.ceph:pgid 66.9 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.646 INFO:tasks.ceph:pgid 65.a last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.646 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.647 INFO:tasks.ceph:pgid 67.b last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.647 INFO:tasks.ceph:pgid 66.a last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.647 INFO:tasks.ceph:pgid 65.9 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.647 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.647 INFO:tasks.ceph:pgid 67.a last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.647 INFO:tasks.ceph:pgid 66.b last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.648 INFO:tasks.ceph:pgid 65.8 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.648 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.648 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.648 INFO:tasks.ceph:pgid 66.4 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.649 INFO:tasks.ceph:pgid 65.7 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.649 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.649 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.649 INFO:tasks.ceph:pgid 66.5 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.649 INFO:tasks.ceph:pgid 65.6 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.649 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.649 INFO:tasks.ceph:pgid 67.7 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.650 INFO:tasks.ceph:pgid 66.6 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.650 INFO:tasks.ceph:pgid 65.5 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.650 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.650 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.650 INFO:tasks.ceph:pgid 66.7 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.650 INFO:tasks.ceph:pgid 65.4 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.651 INFO:tasks.ceph:pgid 68.6 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.651 INFO:tasks.ceph:pgid 67.1 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.651 INFO:tasks.ceph:pgid 66.0 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.651 INFO:tasks.ceph:pgid 65.3 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.651 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.651 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.652 INFO:tasks.ceph:pgid 66.1 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.652 INFO:tasks.ceph:pgid 65.2 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.652 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.652 INFO:tasks.ceph:pgid 67.3 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.652 INFO:tasks.ceph:pgid 66.2 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.652 INFO:tasks.ceph:pgid 65.1 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.653 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.653 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.653 INFO:tasks.ceph:pgid 66.3 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.653 INFO:tasks.ceph:pgid 65.0 last_scrub_stamp 2024-04-22T20:01:03.850280+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=1, tm_sec=3, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.653 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.653 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.653 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.654 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.654 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.654 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.654 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.654 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.654 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.655 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.655 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.655 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.655 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.655 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.655 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.656 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.656 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.656 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.656 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.656 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.656 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.657 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.657 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.657 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.657 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.657 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.657 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.657 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.658 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.658 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.658 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.658 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.658 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.658 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.659 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.661 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.661 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.661 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.661 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.661 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.662 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.662 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.662 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.662 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.662 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.662 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.663 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.663 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.663 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.663 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.663 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.663 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.664 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.664 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.664 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.664 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.664 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.664 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.665 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.665 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.665 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.665 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.665 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.666 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.666 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.666 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.666 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.666 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.666 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.667 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.667 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.667 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.667 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.667 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.667 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.668 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.668 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.668 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.668 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.668 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.668 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.669 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.669 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.669 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.669 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.669 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.669 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.670 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.670 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.670 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.670 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.670 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.670 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.671 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.671 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.671 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.671 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.671 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.671 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.671 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.672 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.672 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.672 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.672 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.672 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.672 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.673 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.673 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.673 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.673 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.673 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.673 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.673 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.674 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.674 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.674 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.674 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.674 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.674 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.675 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.675 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.675 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.675 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.675 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-22T19:51:11.423705+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=11, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.675 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.676 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.676 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.676 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.676 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.676 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.676 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.677 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.677 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.677 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.677 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.677 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.677 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.677 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.678 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.678 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.678 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.678 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.678 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.678 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.679 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.679 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.679 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.679 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.679 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.679 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.680 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.680 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.680 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.680 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.680 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.681 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.681 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.681 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.681 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.681 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.681 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.682 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.682 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.682 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.682 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.682 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.682 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.683 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.683 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.683 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.683 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.683 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.684 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.684 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.684 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.684 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.684 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.684 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.685 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.685 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.685 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.685 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.685 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.685 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.685 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.686 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.686 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.686 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.686 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.686 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.686 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.687 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.687 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.687 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.687 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.687 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.687 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.688 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.688 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.688 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.688 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.688 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.688 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.688 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.689 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.689 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.689 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.689 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.689 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.689 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.689 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.690 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.690 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.690 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.690 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.690 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.690 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.691 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.691 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.691 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.691 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.691 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.691 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.692 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.692 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.692 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.692 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.692 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.692 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.692 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.693 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.693 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.693 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.693 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.693 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.693 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.694 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.694 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.694 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.694 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.694 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.694 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.695 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.695 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.695 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.695 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.695 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.696 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.696 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.696 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.696 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.696 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.697 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.697 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.697 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.697 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.697 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.697 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.698 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.698 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.698 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.698 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.698 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.698 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.699 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:25.699 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-22T20:48:45.700 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T20:48:46.046 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:48:46.057 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T20:48:46.060 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":1981,"stamp":"2024-04-22T20:48:44.338555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":267312719,"num_objects":825,"num_object_clones":0,"num_object_copies":2119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":158844,"num_read_kb":2723065,"num_write":145049,"num_write_kb":2429940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66987,"num_omap_keys":160,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12973,"ondisk_log_size":12973,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1952292,"kb_used_data":441188,"kb_used_omap":12,"kb_used_meta":1511027,"kb_avail":753022428,"statfs":{"total":773094113280,"available":771094966272,"internally_reserved":0,"allocated":451776512,"data_stored":436195170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1547292210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63734,"num_omap_keys":153,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004943"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:50.199771+0000","last_change":"2024-04-22T20:19:16.796805+0000","last_active":"2024-04-22T20:46:50.199771+0000","last_peered":"2024-04-22T20:46:50.199771+0000","last_clean":"2024-04-22T20:46:50.199771+0000","last_became_active":"2024-04-22T20:19:16.796520+0000","last_became_peered":"2024-04-22T20:19:16.796520+0000","last_unstale":"2024-04-22T20:46:50.199771+0000","last_undegraded":"2024-04-22T20:46:50.199771+0000","last_fullsized":"2024-04-22T20:46:50.199771+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"261'5","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.124617+0000","last_change":"2024-04-22T20:05:17.900640+0000","last_active":"2024-04-22T20:48:17.124617+0000","last_peered":"2024-04-22T20:48:17.124617+0000","last_clean":"2024-04-22T20:48:17.124617+0000","last_became_active":"2024-04-22T20:05:17.900325+0000","last_became_peered":"2024-04-22T20:05:17.900325+0000","last_unstale":"2024-04-22T20:48:17.124617+0000","last_undegraded":"2024-04-22T20:48:17.124617+0000","last_fullsized":"2024-04-22T20:48:17.124617+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.497585+0000","last_change":"2024-04-22T20:48:24.497620+0000","last_active":"2024-04-22T20:48:24.497585+0000","last_peered":"2024-04-22T20:48:24.497585+0000","last_clean":"2024-04-22T20:48:24.497585+0000","last_became_active":"2024-04-22T20:05:13.812933+0000","last_became_peered":"2024-04-22T20:05:13.812933+0000","last_unstale":"2024-04-22T20:48:24.497585+0000","last_undegraded":"2024-04-22T20:48:24.497585+0000","last_fullsized":"2024-04-22T20:48:24.497585+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:24.497472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:24.497472+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.497472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:19.340438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"261'20","reported_seq":109,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.612330+0000","last_change":"2024-04-22T20:48:24.612368+0000","last_active":"2024-04-22T20:48:24.612330+0000","last_peered":"2024-04-22T20:48:24.612330+0000","last_clean":"2024-04-22T20:48:24.612330+0000","last_became_active":"2024-04-22T20:01:04.875981+0000","last_became_peered":"2024-04-22T20:01:04.875981+0000","last_unstale":"2024-04-22T20:48:24.612330+0000","last_undegraded":"2024-04-22T20:48:24.612330+0000","last_fullsized":"2024-04-22T20:48:24.612330+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'20","last_scrub_stamp":"2024-04-22T20:48:24.612214+0000","last_deep_scrub":"261'20","last_deep_scrub_stamp":"2024-04-22T20:48:24.612214+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.612214+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:22:32.284247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:50.075066+0000","last_change":"2024-04-22T20:19:16.790684+0000","last_active":"2024-04-22T20:46:50.075066+0000","last_peered":"2024-04-22T20:46:50.075066+0000","last_clean":"2024-04-22T20:46:50.075066+0000","last_became_active":"2024-04-22T20:19:16.790377+0000","last_became_peered":"2024-04-22T20:19:16.790377+0000","last_unstale":"2024-04-22T20:46:50.075066+0000","last_undegraded":"2024-04-22T20:46:50.075066+0000","last_fullsized":"2024-04-22T20:46:50.075066+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":40,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.199235+0000","last_change":"2024-04-22T20:05:17.899675+0000","last_active":"2024-04-22T20:47:50.199235+0000","last_peered":"2024-04-22T20:47:50.199235+0000","last_clean":"2024-04-22T20:47:50.199235+0000","last_became_active":"2024-04-22T20:05:17.899338+0000","last_became_peered":"2024-04-22T20:05:17.899338+0000","last_unstale":"2024-04-22T20:47:50.199235+0000","last_undegraded":"2024-04-22T20:47:50.199235+0000","last_fullsized":"2024-04-22T20:47:50.199235+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"258'4","reported_seq":62,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.584270+0000","last_change":"2024-04-22T20:48:23.584307+0000","last_active":"2024-04-22T20:48:23.584270+0000","last_peered":"2024-04-22T20:48:23.584270+0000","last_clean":"2024-04-22T20:48:23.584270+0000","last_became_active":"2024-04-22T20:05:13.807231+0000","last_became_peered":"2024-04-22T20:05:13.807231+0000","last_unstale":"2024-04-22T20:48:23.584270+0000","last_undegraded":"2024-04-22T20:48:23.584270+0000","last_fullsized":"2024-04-22T20:48:23.584270+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:23.584182+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:23.584182+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.584182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:34.394463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"261'22","reported_seq":110,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.458354+0000","last_change":"2024-04-22T20:48:34.458388+0000","last_active":"2024-04-22T20:48:34.458354+0000","last_peered":"2024-04-22T20:48:34.458354+0000","last_clean":"2024-04-22T20:48:34.458354+0000","last_became_active":"2024-04-22T20:01:04.874575+0000","last_became_peered":"2024-04-22T20:01:04.874575+0000","last_unstale":"2024-04-22T20:48:34.458354+0000","last_undegraded":"2024-04-22T20:48:34.458354+0000","last_fullsized":"2024-04-22T20:48:34.458354+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'22","last_scrub_stamp":"2024-04-22T20:48:34.458264+0000","last_deep_scrub":"261'22","last_deep_scrub_stamp":"2024-04-22T20:48:34.458264+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.458264+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:46:15.278433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":48,"num_write":47,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:20.340132+0000","last_change":"2024-04-22T20:19:16.794970+0000","last_active":"2024-04-22T20:48:20.340132+0000","last_peered":"2024-04-22T20:48:20.340132+0000","last_clean":"2024-04-22T20:48:20.340132+0000","last_became_active":"2024-04-22T20:19:16.794700+0000","last_became_peered":"2024-04-22T20:19:16.794700+0000","last_unstale":"2024-04-22T20:48:20.340132+0000","last_undegraded":"2024-04-22T20:48:20.340132+0000","last_fullsized":"2024-04-22T20:48:20.340132+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":43,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:00.192253+0000","last_change":"2024-04-22T20:05:18.211663+0000","last_active":"2024-04-22T20:47:00.192253+0000","last_peered":"2024-04-22T20:47:00.192253+0000","last_clean":"2024-04-22T20:47:00.192253+0000","last_became_active":"2024-04-22T20:05:18.211400+0000","last_became_peered":"2024-04-22T20:05:18.211400+0000","last_unstale":"2024-04-22T20:47:00.192253+0000","last_undegraded":"2024-04-22T20:47:00.192253+0000","last_fullsized":"2024-04-22T20:47:00.192253+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"261'2","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.260970+0000","last_change":"2024-04-22T20:05:13.809630+0000","last_active":"2024-04-22T20:48:16.260970+0000","last_peered":"2024-04-22T20:48:16.260970+0000","last_clean":"2024-04-22T20:48:16.260970+0000","last_became_active":"2024-04-22T20:05:13.809349+0000","last_became_peered":"2024-04-22T20:05:13.809349+0000","last_unstale":"2024-04-22T20:48:16.260970+0000","last_undegraded":"2024-04-22T20:48:16.260970+0000","last_fullsized":"2024-04-22T20:48:16.260970+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"261'24","reported_seq":124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.511512+0000","last_change":"2024-04-22T20:48:31.511512+0000","last_active":"2024-04-22T20:48:31.511512+0000","last_peered":"2024-04-22T20:48:31.511512+0000","last_clean":"2024-04-22T20:48:31.511512+0000","last_became_active":"2024-04-22T20:01:04.869785+0000","last_became_peered":"2024-04-22T20:01:04.869785+0000","last_unstale":"2024-04-22T20:48:31.511512+0000","last_undegraded":"2024-04-22T20:48:31.511512+0000","last_fullsized":"2024-04-22T20:48:31.511512+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'24","last_scrub_stamp":"2024-04-22T20:48:31.511334+0000","last_deep_scrub":"261'24","last_deep_scrub_stamp":"2024-04-22T20:48:31.511334+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.511334+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:43:54.338193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":82,"num_write":36,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:20.176404+0000","last_change":"2024-04-22T20:19:16.795441+0000","last_active":"2024-04-22T20:48:20.176404+0000","last_peered":"2024-04-22T20:48:20.176404+0000","last_clean":"2024-04-22T20:48:20.176404+0000","last_became_active":"2024-04-22T20:19:16.793667+0000","last_became_peered":"2024-04-22T20:19:16.793667+0000","last_unstale":"2024-04-22T20:48:20.176404+0000","last_undegraded":"2024-04-22T20:48:20.176404+0000","last_fullsized":"2024-04-22T20:48:20.176404+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"261'10","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.491093+0000","last_change":"2024-04-22T20:06:05.349958+0000","last_active":"2024-04-22T20:48:17.491093+0000","last_peered":"2024-04-22T20:48:17.491093+0000","last_clean":"2024-04-22T20:48:17.491093+0000","last_became_active":"2024-04-22T20:06:05.345554+0000","last_became_peered":"2024-04-22T20:06:05.345554+0000","last_unstale":"2024-04-22T20:48:17.491093+0000","last_undegraded":"2024-04-22T20:48:17.491093+0000","last_fullsized":"2024-04-22T20:48:17.491093+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":20,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1f","version":"261'10","reported_seq":77,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.367797+0000","last_change":"2024-04-22T20:48:36.367896+0000","last_active":"2024-04-22T20:48:36.367797+0000","last_peered":"2024-04-22T20:48:36.367797+0000","last_clean":"2024-04-22T20:48:36.367797+0000","last_became_active":"2024-04-22T20:05:13.806246+0000","last_became_peered":"2024-04-22T20:05:13.806246+0000","last_unstale":"2024-04-22T20:48:36.367797+0000","last_undegraded":"2024-04-22T20:48:36.367797+0000","last_fullsized":"2024-04-22T20:48:36.367797+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:48:36.367701+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:48:36.367701+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.367701+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:06:18.240490+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":16720,"num_write":13,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"261'59","reported_seq":208,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.583472+0000","last_change":"2024-04-22T20:48:30.583472+0000","last_active":"2024-04-22T20:48:30.583472+0000","last_peered":"2024-04-22T20:48:30.583472+0000","last_clean":"2024-04-22T20:48:30.583472+0000","last_became_active":"2024-04-22T20:01:04.873899+0000","last_became_peered":"2024-04-22T20:01:04.873899+0000","last_unstale":"2024-04-22T20:48:30.583472+0000","last_undegraded":"2024-04-22T20:48:30.583472+0000","last_fullsized":"2024-04-22T20:48:30.583472+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'59","last_scrub_stamp":"2024-04-22T20:48:30.583311+0000","last_deep_scrub":"261'59","last_deep_scrub_stamp":"2024-04-22T20:48:30.583311+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.583311+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:58:29.692843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":183,"num_write":99,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.169419+0000","last_change":"2024-04-22T20:19:16.794646+0000","last_active":"2024-04-22T20:47:20.169419+0000","last_peered":"2024-04-22T20:47:20.169419+0000","last_clean":"2024-04-22T20:47:20.169419+0000","last_became_active":"2024-04-22T20:19:16.794110+0000","last_became_peered":"2024-04-22T20:19:16.794110+0000","last_unstale":"2024-04-22T20:47:20.169419+0000","last_undegraded":"2024-04-22T20:47:20.169419+0000","last_fullsized":"2024-04-22T20:47:20.169419+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"261'2","reported_seq":44,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.487275+0000","last_change":"2024-04-22T20:05:17.898277+0000","last_active":"2024-04-22T20:48:16.487275+0000","last_peered":"2024-04-22T20:48:16.487275+0000","last_clean":"2024-04-22T20:48:16.487275+0000","last_became_active":"2024-04-22T20:05:17.897771+0000","last_became_peered":"2024-04-22T20:05:17.897771+0000","last_unstale":"2024-04-22T20:48:16.487275+0000","last_undegraded":"2024-04-22T20:48:16.487275+0000","last_fullsized":"2024-04-22T20:48:16.487275+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"261'6","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.520591+0000","last_change":"2024-04-22T20:48:29.520623+0000","last_active":"2024-04-22T20:48:29.520591+0000","last_peered":"2024-04-22T20:48:29.520591+0000","last_clean":"2024-04-22T20:48:29.520591+0000","last_became_active":"2024-04-22T20:05:13.812308+0000","last_became_peered":"2024-04-22T20:05:13.812308+0000","last_unstale":"2024-04-22T20:48:29.520591+0000","last_undegraded":"2024-04-22T20:48:29.520591+0000","last_fullsized":"2024-04-22T20:48:29.520591+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:29.520502+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:29.520502+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.520502+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:11:42.190343+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":9,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"261'8","reported_seq":82,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.526061+0000","last_change":"2024-04-22T20:48:25.526061+0000","last_active":"2024-04-22T20:48:25.526061+0000","last_peered":"2024-04-22T20:48:25.526061+0000","last_clean":"2024-04-22T20:48:25.526061+0000","last_became_active":"2024-04-22T20:01:04.877209+0000","last_became_peered":"2024-04-22T20:01:04.877209+0000","last_unstale":"2024-04-22T20:48:25.526061+0000","last_undegraded":"2024-04-22T20:48:25.526061+0000","last_fullsized":"2024-04-22T20:48:25.526061+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'8","last_scrub_stamp":"2024-04-22T20:48:25.525971+0000","last_deep_scrub":"261'8","last_deep_scrub_stamp":"2024-04-22T20:48:25.525971+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.525971+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:37.255572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.493623+0000","last_change":"2024-04-22T20:19:16.789917+0000","last_active":"2024-04-22T20:48:22.493623+0000","last_peered":"2024-04-22T20:48:22.493623+0000","last_clean":"2024-04-22T20:48:22.493623+0000","last_became_active":"2024-04-22T20:19:16.789599+0000","last_became_peered":"2024-04-22T20:19:16.789599+0000","last_unstale":"2024-04-22T20:48:22.493623+0000","last_undegraded":"2024-04-22T20:48:22.493623+0000","last_fullsized":"2024-04-22T20:48:22.493623+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"261'2","reported_seq":43,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.786744+0000","last_change":"2024-04-22T20:05:17.891180+0000","last_active":"2024-04-22T20:48:16.786744+0000","last_peered":"2024-04-22T20:48:16.786744+0000","last_clean":"2024-04-22T20:48:16.786744+0000","last_became_active":"2024-04-22T20:05:17.890841+0000","last_became_peered":"2024-04-22T20:05:17.890841+0000","last_unstale":"2024-04-22T20:48:16.786744+0000","last_undegraded":"2024-04-22T20:48:16.786744+0000","last_fullsized":"2024-04-22T20:48:16.786744+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":46,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.188174+0000","last_change":"2024-04-22T20:05:13.807783+0000","last_active":"2024-04-22T20:46:30.188174+0000","last_peered":"2024-04-22T20:46:30.188174+0000","last_clean":"2024-04-22T20:46:30.188174+0000","last_became_active":"2024-04-22T20:05:13.807516+0000","last_became_peered":"2024-04-22T20:05:13.807516+0000","last_unstale":"2024-04-22T20:46:30.188174+0000","last_undegraded":"2024-04-22T20:46:30.188174+0000","last_fullsized":"2024-04-22T20:46:30.188174+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"261'77","reported_seq":249,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.378570+0000","last_change":"2024-04-22T20:48:26.378570+0000","last_active":"2024-04-22T20:48:26.378570+0000","last_peered":"2024-04-22T20:48:26.378570+0000","last_clean":"2024-04-22T20:48:26.378570+0000","last_became_active":"2024-04-22T20:01:04.873015+0000","last_became_peered":"2024-04-22T20:01:04.873015+0000","last_unstale":"2024-04-22T20:48:26.378570+0000","last_undegraded":"2024-04-22T20:48:26.378570+0000","last_fullsized":"2024-04-22T20:48:26.378570+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'77","last_scrub_stamp":"2024-04-22T20:48:26.378447+0000","last_deep_scrub":"261'77","last_deep_scrub_stamp":"2024-04-22T20:48:26.378447+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.378447+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:22.636845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":236,"num_write":125,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"261'1","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:50.076213+0000","last_change":"2024-04-22T20:19:16.794883+0000","last_active":"2024-04-22T20:46:50.076213+0000","last_peered":"2024-04-22T20:46:50.076213+0000","last_clean":"2024-04-22T20:46:50.076213+0000","last_became_active":"2024-04-22T20:19:16.794564+0000","last_became_peered":"2024-04-22T20:19:16.794564+0000","last_unstale":"2024-04-22T20:46:50.076213+0000","last_undegraded":"2024-04-22T20:46:50.076213+0000","last_fullsized":"2024-04-22T20:46:50.076213+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"261'5","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.125719+0000","last_change":"2024-04-22T20:05:17.901743+0000","last_active":"2024-04-22T20:48:17.125719+0000","last_peered":"2024-04-22T20:48:17.125719+0000","last_clean":"2024-04-22T20:48:17.125719+0000","last_became_active":"2024-04-22T20:05:17.901446+0000","last_became_peered":"2024-04-22T20:05:17.901446+0000","last_unstale":"2024-04-22T20:48:17.125719+0000","last_undegraded":"2024-04-22T20:48:17.125719+0000","last_fullsized":"2024-04-22T20:48:17.125719+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":11264,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.351494+0000","last_change":"2024-04-22T20:48:28.351557+0000","last_active":"2024-04-22T20:48:28.351494+0000","last_peered":"2024-04-22T20:48:28.351494+0000","last_clean":"2024-04-22T20:48:28.351494+0000","last_became_active":"2024-04-22T20:05:13.798311+0000","last_became_peered":"2024-04-22T20:05:13.798311+0000","last_unstale":"2024-04-22T20:48:28.351494+0000","last_undegraded":"2024-04-22T20:48:28.351494+0000","last_fullsized":"2024-04-22T20:48:28.351494+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:28.351390+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:28.351390+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.351390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:49:42.653069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'27","reported_seq":125,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.545417+0000","last_change":"2024-04-22T20:48:28.545440+0000","last_active":"2024-04-22T20:48:28.545417+0000","last_peered":"2024-04-22T20:48:28.545417+0000","last_clean":"2024-04-22T20:48:28.545417+0000","last_became_active":"2024-04-22T20:01:04.873426+0000","last_became_peered":"2024-04-22T20:01:04.873426+0000","last_unstale":"2024-04-22T20:48:28.545417+0000","last_undegraded":"2024-04-22T20:48:28.545417+0000","last_fullsized":"2024-04-22T20:48:28.545417+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'27","last_scrub_stamp":"2024-04-22T20:48:28.545320+0000","last_deep_scrub":"261'27","last_deep_scrub_stamp":"2024-04-22T20:48:28.545320+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.545320+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:51:52.257563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":80,"num_write":54,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:50.076263+0000","last_change":"2024-04-22T20:19:16.800874+0000","last_active":"2024-04-22T20:46:50.076263+0000","last_peered":"2024-04-22T20:46:50.076263+0000","last_clean":"2024-04-22T20:46:50.076263+0000","last_became_active":"2024-04-22T20:19:16.800611+0000","last_became_peered":"2024-04-22T20:19:16.800611+0000","last_unstale":"2024-04-22T20:46:50.076263+0000","last_undegraded":"2024-04-22T20:46:50.076263+0000","last_fullsized":"2024-04-22T20:46:50.076263+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"258'4","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.172958+0000","last_change":"2024-04-22T20:05:17.892900+0000","last_active":"2024-04-22T20:47:50.172958+0000","last_peered":"2024-04-22T20:47:50.172958+0000","last_clean":"2024-04-22T20:47:50.172958+0000","last_became_active":"2024-04-22T20:05:17.892624+0000","last_became_peered":"2024-04-22T20:05:17.892624+0000","last_unstale":"2024-04-22T20:47:50.172958+0000","last_undegraded":"2024-04-22T20:47:50.172958+0000","last_fullsized":"2024-04-22T20:47:50.172958+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3072,"num_write":4,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.247622+0000","last_change":"2024-04-22T20:05:13.816306+0000","last_active":"2024-04-22T20:48:17.247622+0000","last_peered":"2024-04-22T20:48:17.247622+0000","last_clean":"2024-04-22T20:48:17.247622+0000","last_became_active":"2024-04-22T20:05:13.816011+0000","last_became_peered":"2024-04-22T20:05:13.816011+0000","last_unstale":"2024-04-22T20:48:17.247622+0000","last_undegraded":"2024-04-22T20:48:17.247622+0000","last_fullsized":"2024-04-22T20:48:17.247622+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"261'43","reported_seq":159,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.562937+0000","last_change":"2024-04-22T20:48:25.562960+0000","last_active":"2024-04-22T20:48:25.562937+0000","last_peered":"2024-04-22T20:48:25.562937+0000","last_clean":"2024-04-22T20:48:25.562937+0000","last_became_active":"2024-04-22T20:01:04.872356+0000","last_became_peered":"2024-04-22T20:01:04.872356+0000","last_unstale":"2024-04-22T20:48:25.562937+0000","last_undegraded":"2024-04-22T20:48:25.562937+0000","last_fullsized":"2024-04-22T20:48:25.562937+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'43","last_scrub_stamp":"2024-04-22T20:48:25.562858+0000","last_deep_scrub":"261'43","last_deep_scrub_stamp":"2024-04-22T20:48:25.562858+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.562858+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:39:26.408917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":119,"num_write":72,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.056844+0000","last_change":"2024-04-22T20:19:16.796888+0000","last_active":"2024-04-22T20:47:20.056844+0000","last_peered":"2024-04-22T20:47:20.056844+0000","last_clean":"2024-04-22T20:47:20.056844+0000","last_became_active":"2024-04-22T20:19:16.796155+0000","last_became_peered":"2024-04-22T20:19:16.796155+0000","last_unstale":"2024-04-22T20:47:20.056844+0000","last_undegraded":"2024-04-22T20:47:20.056844+0000","last_fullsized":"2024-04-22T20:47:20.056844+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":40,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:25.163044+0000","last_change":"2024-04-22T20:05:18.209303+0000","last_active":"2024-04-22T20:46:25.163044+0000","last_peered":"2024-04-22T20:46:25.163044+0000","last_clean":"2024-04-22T20:46:25.163044+0000","last_became_active":"2024-04-22T20:05:18.209023+0000","last_became_peered":"2024-04-22T20:05:18.209023+0000","last_unstale":"2024-04-22T20:46:25.163044+0000","last_undegraded":"2024-04-22T20:46:25.163044+0000","last_fullsized":"2024-04-22T20:46:25.163044+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"261'7","reported_seq":70,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.910170+0000","last_change":"2024-04-22T20:48:39.910210+0000","last_active":"2024-04-22T20:48:39.910170+0000","last_peered":"2024-04-22T20:48:39.910170+0000","last_clean":"2024-04-22T20:48:39.910170+0000","last_became_active":"2024-04-22T20:05:13.809548+0000","last_became_peered":"2024-04-22T20:05:13.809548+0000","last_unstale":"2024-04-22T20:48:39.910170+0000","last_undegraded":"2024-04-22T20:48:39.910170+0000","last_fullsized":"2024-04-22T20:48:39.910170+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:39.910080+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:39.910080+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.910080+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:39:16.656357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":6146,"num_write":5,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'64","reported_seq":251,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.544212+0000","last_change":"2024-04-22T20:48:30.544266+0000","last_active":"2024-04-22T20:48:30.544212+0000","last_peered":"2024-04-22T20:48:30.544212+0000","last_clean":"2024-04-22T20:48:30.544212+0000","last_became_active":"2024-04-22T20:01:04.873236+0000","last_became_peered":"2024-04-22T20:01:04.873236+0000","last_unstale":"2024-04-22T20:48:30.544212+0000","last_undegraded":"2024-04-22T20:48:30.544212+0000","last_fullsized":"2024-04-22T20:48:30.544212+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'64","last_scrub_stamp":"2024-04-22T20:48:30.544111+0000","last_deep_scrub":"261'64","last_deep_scrub_stamp":"2024-04-22T20:48:30.544111+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.544111+0000","objects_scrubbed":1,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:51.536623+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":250,"num_write":111,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3950,"num_omap_keys":8,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"261'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.078068+0000","last_change":"2024-04-22T20:19:16.791895+0000","last_active":"2024-04-22T20:47:20.078068+0000","last_peered":"2024-04-22T20:47:20.078068+0000","last_clean":"2024-04-22T20:47:20.078068+0000","last_became_active":"2024-04-22T20:19:16.791597+0000","last_became_peered":"2024-04-22T20:19:16.791597+0000","last_unstale":"2024-04-22T20:47:20.078068+0000","last_undegraded":"2024-04-22T20:47:20.078068+0000","last_fullsized":"2024-04-22T20:47:20.078068+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.169871+0000","last_change":"2024-04-22T20:05:18.211042+0000","last_active":"2024-04-22T20:47:25.169871+0000","last_peered":"2024-04-22T20:47:25.169871+0000","last_clean":"2024-04-22T20:47:25.169871+0000","last_became_active":"2024-04-22T20:05:18.210704+0000","last_became_peered":"2024-04-22T20:05:18.210704+0000","last_unstale":"2024-04-22T20:47:25.169871+0000","last_undegraded":"2024-04-22T20:47:25.169871+0000","last_fullsized":"2024-04-22T20:47:25.169871+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.609458+0000","last_change":"2024-04-22T20:48:27.609480+0000","last_active":"2024-04-22T20:48:27.609458+0000","last_peered":"2024-04-22T20:48:27.609458+0000","last_clean":"2024-04-22T20:48:27.609458+0000","last_became_active":"2024-04-22T20:05:13.810902+0000","last_became_peered":"2024-04-22T20:05:13.810902+0000","last_unstale":"2024-04-22T20:48:27.609458+0000","last_undegraded":"2024-04-22T20:48:27.609458+0000","last_fullsized":"2024-04-22T20:48:27.609458+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:27.609361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:27.609361+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.609361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:08.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"250'9","reported_seq":85,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.596475+0000","last_change":"2024-04-22T20:48:27.596526+0000","last_active":"2024-04-22T20:48:27.596475+0000","last_peered":"2024-04-22T20:48:27.596475+0000","last_clean":"2024-04-22T20:48:27.596475+0000","last_became_active":"2024-04-22T20:01:04.874230+0000","last_became_peered":"2024-04-22T20:01:04.874230+0000","last_unstale":"2024-04-22T20:48:27.596475+0000","last_undegraded":"2024-04-22T20:48:27.596475+0000","last_fullsized":"2024-04-22T20:48:27.596475+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"250'9","last_scrub_stamp":"2024-04-22T20:48:27.596354+0000","last_deep_scrub":"250'9","last_deep_scrub_stamp":"2024-04-22T20:48:27.596354+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.596354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:12:20.356156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.080141+0000","last_change":"2024-04-22T20:19:16.795626+0000","last_active":"2024-04-22T20:47:20.080141+0000","last_peered":"2024-04-22T20:47:20.080141+0000","last_clean":"2024-04-22T20:47:20.080141+0000","last_became_active":"2024-04-22T20:19:16.795139+0000","last_became_peered":"2024-04-22T20:19:16.795139+0000","last_unstale":"2024-04-22T20:47:20.080141+0000","last_undegraded":"2024-04-22T20:47:20.080141+0000","last_fullsized":"2024-04-22T20:47:20.080141+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"261'16","reported_seq":76,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.124108+0000","last_change":"2024-04-22T20:05:18.212823+0000","last_active":"2024-04-22T20:48:17.124108+0000","last_peered":"2024-04-22T20:48:17.124108+0000","last_clean":"2024-04-22T20:48:17.124108+0000","last_became_active":"2024-04-22T20:05:18.212336+0000","last_became_peered":"2024-04-22T20:05:18.212336+0000","last_unstale":"2024-04-22T20:48:17.124108+0000","last_undegraded":"2024-04-22T20:48:17.124108+0000","last_fullsized":"2024-04-22T20:48:17.124108+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437184,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":38918,"num_write":23,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"261'2","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.193080+0000","last_change":"2024-04-22T20:05:13.816425+0000","last_active":"2024-04-22T20:48:16.193080+0000","last_peered":"2024-04-22T20:48:16.193080+0000","last_clean":"2024-04-22T20:48:16.193080+0000","last_became_active":"2024-04-22T20:05:13.816036+0000","last_became_peered":"2024-04-22T20:05:13.816036+0000","last_unstale":"2024-04-22T20:48:16.193080+0000","last_undegraded":"2024-04-22T20:48:16.193080+0000","last_fullsized":"2024-04-22T20:48:16.193080+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'53","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.516499+0000","last_change":"2024-04-22T20:48:36.516524+0000","last_active":"2024-04-22T20:48:36.516499+0000","last_peered":"2024-04-22T20:48:36.516499+0000","last_clean":"2024-04-22T20:48:36.516499+0000","last_became_active":"2024-04-22T20:01:04.878067+0000","last_became_peered":"2024-04-22T20:01:04.878067+0000","last_unstale":"2024-04-22T20:48:36.516499+0000","last_undegraded":"2024-04-22T20:48:36.516499+0000","last_fullsized":"2024-04-22T20:48:36.516499+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'53","last_scrub_stamp":"2024-04-22T20:48:36.516407+0000","last_deep_scrub":"261'53","last_deep_scrub_stamp":"2024-04-22T20:48:36.516407+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.516407+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:26.653699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":150,"num_write":99,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.204124+0000","last_change":"2024-04-22T20:19:16.796141+0000","last_active":"2024-04-22T20:47:20.204124+0000","last_peered":"2024-04-22T20:47:20.204124+0000","last_clean":"2024-04-22T20:47:20.204124+0000","last_became_active":"2024-04-22T20:19:16.795855+0000","last_became_peered":"2024-04-22T20:19:16.795855+0000","last_unstale":"2024-04-22T20:47:20.204124+0000","last_undegraded":"2024-04-22T20:47:20.204124+0000","last_fullsized":"2024-04-22T20:47:20.204124+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"0'0","reported_seq":40,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:25.187494+0000","last_change":"2024-04-22T20:05:17.902267+0000","last_active":"2024-04-22T20:46:25.187494+0000","last_peered":"2024-04-22T20:46:25.187494+0000","last_clean":"2024-04-22T20:46:25.187494+0000","last_became_active":"2024-04-22T20:05:17.901954+0000","last_became_peered":"2024-04-22T20:05:17.901954+0000","last_unstale":"2024-04-22T20:46:25.187494+0000","last_undegraded":"2024-04-22T20:46:25.187494+0000","last_fullsized":"2024-04-22T20:46:25.187494+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'4","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.504192+0000","last_change":"2024-04-22T20:48:33.504218+0000","last_active":"2024-04-22T20:48:33.504192+0000","last_peered":"2024-04-22T20:48:33.504192+0000","last_clean":"2024-04-22T20:48:33.504192+0000","last_became_active":"2024-04-22T20:05:13.813256+0000","last_became_peered":"2024-04-22T20:05:13.813256+0000","last_unstale":"2024-04-22T20:48:33.504192+0000","last_undegraded":"2024-04-22T20:48:33.504192+0000","last_fullsized":"2024-04-22T20:48:33.504192+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:33.504106+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:33.504106+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.504106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:41:49.887327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":44,"num_write":4,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"261'46","reported_seq":173,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.448464+0000","last_change":"2024-04-22T20:48:33.448487+0000","last_active":"2024-04-22T20:48:33.448464+0000","last_peered":"2024-04-22T20:48:33.448464+0000","last_clean":"2024-04-22T20:48:33.448464+0000","last_became_active":"2024-04-22T20:01:04.873533+0000","last_became_peered":"2024-04-22T20:01:04.873533+0000","last_unstale":"2024-04-22T20:48:33.448464+0000","last_undegraded":"2024-04-22T20:48:33.448464+0000","last_fullsized":"2024-04-22T20:48:33.448464+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'46","last_scrub_stamp":"2024-04-22T20:48:33.448376+0000","last_deep_scrub":"261'46","last_deep_scrub_stamp":"2024-04-22T20:48:33.448376+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.448376+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:02:25.135711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":192,"num_read_kb":137,"num_write":82,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.175890+0000","last_change":"2024-04-22T20:19:16.793148+0000","last_active":"2024-04-22T20:48:15.175890+0000","last_peered":"2024-04-22T20:48:15.175890+0000","last_clean":"2024-04-22T20:48:15.175890+0000","last_became_active":"2024-04-22T20:19:16.792856+0000","last_became_peered":"2024-04-22T20:19:16.792856+0000","last_unstale":"2024-04-22T20:48:15.175890+0000","last_undegraded":"2024-04-22T20:48:15.175890+0000","last_fullsized":"2024-04-22T20:48:15.175890+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'8","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.889025+0000","last_change":"2024-04-22T20:05:17.897292+0000","last_active":"2024-04-22T20:48:15.889025+0000","last_peered":"2024-04-22T20:48:15.889025+0000","last_clean":"2024-04-22T20:48:15.889025+0000","last_became_active":"2024-04-22T20:05:17.897040+0000","last_became_peered":"2024-04-22T20:05:17.897040+0000","last_unstale":"2024-04-22T20:48:15.889025+0000","last_undegraded":"2024-04-22T20:48:15.889025+0000","last_fullsized":"2024-04-22T20:48:15.889025+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":12290,"num_write":7,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"261'4","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.949605+0000","last_change":"2024-04-22T20:05:13.814475+0000","last_active":"2024-04-22T20:48:16.949605+0000","last_peered":"2024-04-22T20:48:16.949605+0000","last_clean":"2024-04-22T20:48:16.949605+0000","last_became_active":"2024-04-22T20:05:13.814222+0000","last_became_peered":"2024-04-22T20:05:13.814222+0000","last_unstale":"2024-04-22T20:48:16.949605+0000","last_undegraded":"2024-04-22T20:48:16.949605+0000","last_fullsized":"2024-04-22T20:48:16.949605+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"261'54","reported_seq":199,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.410439+0000","last_change":"2024-04-22T20:48:34.410468+0000","last_active":"2024-04-22T20:48:34.410439+0000","last_peered":"2024-04-22T20:48:34.410439+0000","last_clean":"2024-04-22T20:48:34.410439+0000","last_became_active":"2024-04-22T20:01:04.873422+0000","last_became_peered":"2024-04-22T20:01:04.873422+0000","last_unstale":"2024-04-22T20:48:34.410439+0000","last_undegraded":"2024-04-22T20:48:34.410439+0000","last_fullsized":"2024-04-22T20:48:34.410439+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'54","last_scrub_stamp":"2024-04-22T20:48:34.410341+0000","last_deep_scrub":"261'54","last_deep_scrub_stamp":"2024-04-22T20:48:34.410341+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.410341+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:47:24.001212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":184,"num_write":82,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:20.201608+0000","last_change":"2024-04-22T20:19:16.795990+0000","last_active":"2024-04-22T20:48:20.201608+0000","last_peered":"2024-04-22T20:48:20.201608+0000","last_clean":"2024-04-22T20:48:20.201608+0000","last_became_active":"2024-04-22T20:19:16.795633+0000","last_became_peered":"2024-04-22T20:19:16.795633+0000","last_unstale":"2024-04-22T20:48:20.201608+0000","last_undegraded":"2024-04-22T20:48:20.201608+0000","last_fullsized":"2024-04-22T20:48:20.201608+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"261'9","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.490595+0000","last_change":"2024-04-22T20:05:17.900013+0000","last_active":"2024-04-22T20:48:17.490595+0000","last_peered":"2024-04-22T20:48:17.490595+0000","last_clean":"2024-04-22T20:48:17.490595+0000","last_became_active":"2024-04-22T20:05:17.899682+0000","last_became_peered":"2024-04-22T20:05:17.899682+0000","last_unstale":"2024-04-22T20:48:17.490595+0000","last_undegraded":"2024-04-22T20:48:17.490595+0000","last_fullsized":"2024-04-22T20:48:17.490595+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":8210,"num_write":13,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"261'6","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.625913+0000","last_change":"2024-04-22T20:48:28.625974+0000","last_active":"2024-04-22T20:48:28.625913+0000","last_peered":"2024-04-22T20:48:28.625913+0000","last_clean":"2024-04-22T20:48:28.625913+0000","last_became_active":"2024-04-22T20:05:13.812893+0000","last_became_peered":"2024-04-22T20:05:13.812893+0000","last_unstale":"2024-04-22T20:48:28.625913+0000","last_undegraded":"2024-04-22T20:48:28.625913+0000","last_fullsized":"2024-04-22T20:48:28.625913+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:28.625814+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:28.625814+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.625814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:06.391872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4096,"num_write":16,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"261'47","reported_seq":172,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.628532+0000","last_change":"2024-04-22T20:48:23.628571+0000","last_active":"2024-04-22T20:48:23.628532+0000","last_peered":"2024-04-22T20:48:23.628532+0000","last_clean":"2024-04-22T20:48:23.628532+0000","last_became_active":"2024-04-22T20:01:04.877755+0000","last_became_peered":"2024-04-22T20:01:04.877755+0000","last_unstale":"2024-04-22T20:48:23.628532+0000","last_undegraded":"2024-04-22T20:48:23.628532+0000","last_fullsized":"2024-04-22T20:48:23.628532+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'47","last_scrub_stamp":"2024-04-22T20:48:23.628425+0000","last_deep_scrub":"261'47","last_deep_scrub_stamp":"2024-04-22T20:48:23.628425+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.628425+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:22.945025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":135,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"260'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.195183+0000","last_change":"2024-04-22T20:19:16.793874+0000","last_active":"2024-04-22T20:47:25.195183+0000","last_peered":"2024-04-22T20:47:25.195183+0000","last_clean":"2024-04-22T20:47:25.195183+0000","last_became_active":"2024-04-22T20:19:16.793555+0000","last_became_peered":"2024-04-22T20:19:16.793555+0000","last_unstale":"2024-04-22T20:47:25.195183+0000","last_undegraded":"2024-04-22T20:47:25.195183+0000","last_fullsized":"2024-04-22T20:47:25.195183+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"258'2","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.475045+0000","last_change":"2024-04-22T20:48:36.475071+0000","last_active":"2024-04-22T20:48:36.475045+0000","last_peered":"2024-04-22T20:48:36.475045+0000","last_clean":"2024-04-22T20:48:36.475045+0000","last_became_active":"2024-04-22T20:05:17.890070+0000","last_became_peered":"2024-04-22T20:05:17.890070+0000","last_unstale":"2024-04-22T20:48:36.475045+0000","last_undegraded":"2024-04-22T20:48:36.475045+0000","last_fullsized":"2024-04-22T20:48:36.475045+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'2","last_scrub_stamp":"2024-04-22T20:48:36.474953+0000","last_deep_scrub":"258'2","last_deep_scrub_stamp":"2024-04-22T20:48:36.474953+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.474953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:22:16.222599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"261'5","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.415179+0000","last_change":"2024-04-22T20:48:30.415205+0000","last_active":"2024-04-22T20:48:30.415179+0000","last_peered":"2024-04-22T20:48:30.415179+0000","last_clean":"2024-04-22T20:48:30.415179+0000","last_became_active":"2024-04-22T20:05:13.806882+0000","last_became_peered":"2024-04-22T20:05:13.806882+0000","last_unstale":"2024-04-22T20:48:30.415179+0000","last_undegraded":"2024-04-22T20:48:30.415179+0000","last_fullsized":"2024-04-22T20:48:30.415179+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:30.415089+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:30.415089+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.415089+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:39.469597+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"261'11","reported_seq":89,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.276700+0000","last_change":"2024-04-22T20:48:25.276728+0000","last_active":"2024-04-22T20:48:25.276700+0000","last_peered":"2024-04-22T20:48:25.276700+0000","last_clean":"2024-04-22T20:48:25.276700+0000","last_became_active":"2024-04-22T20:01:04.860991+0000","last_became_peered":"2024-04-22T20:01:04.860991+0000","last_unstale":"2024-04-22T20:48:25.276700+0000","last_undegraded":"2024-04-22T20:48:25.276700+0000","last_fullsized":"2024-04-22T20:48:25.276700+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'11","last_scrub_stamp":"2024-04-22T20:48:25.276600+0000","last_deep_scrub":"261'11","last_deep_scrub_stamp":"2024-04-22T20:48:25.276600+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.276600+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:39:08.585319+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":31,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"261'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.175863+0000","last_change":"2024-04-22T20:19:16.794789+0000","last_active":"2024-04-22T20:48:15.175863+0000","last_peered":"2024-04-22T20:48:15.175863+0000","last_clean":"2024-04-22T20:48:15.175863+0000","last_became_active":"2024-04-22T20:19:16.791995+0000","last_became_peered":"2024-04-22T20:19:16.791995+0000","last_unstale":"2024-04-22T20:48:15.175863+0000","last_undegraded":"2024-04-22T20:48:15.175863+0000","last_fullsized":"2024-04-22T20:48:15.175863+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"258'4","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.331218+0000","last_change":"2024-04-22T20:05:17.890780+0000","last_active":"2024-04-22T20:47:20.331218+0000","last_peered":"2024-04-22T20:47:20.331218+0000","last_clean":"2024-04-22T20:47:20.331218+0000","last_became_active":"2024-04-22T20:05:17.890408+0000","last_became_peered":"2024-04-22T20:05:17.890408+0000","last_unstale":"2024-04-22T20:47:20.331218+0000","last_undegraded":"2024-04-22T20:47:20.331218+0000","last_fullsized":"2024-04-22T20:47:20.331218+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1342,"num_write":4,"num_write_kb":1342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"258'4","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.956463+0000","last_change":"2024-04-22T20:48:35.956522+0000","last_active":"2024-04-22T20:48:35.956463+0000","last_peered":"2024-04-22T20:48:35.956463+0000","last_clean":"2024-04-22T20:48:35.956463+0000","last_became_active":"2024-04-22T20:05:13.808473+0000","last_became_peered":"2024-04-22T20:05:13.808473+0000","last_unstale":"2024-04-22T20:48:35.956463+0000","last_undegraded":"2024-04-22T20:48:35.956463+0000","last_fullsized":"2024-04-22T20:48:35.956463+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:35.956357+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:35.956357+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.956357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:07:38.200109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"261'47","reported_seq":213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.554713+0000","last_change":"2024-04-22T20:48:26.554763+0000","last_active":"2024-04-22T20:48:26.554713+0000","last_peered":"2024-04-22T20:48:26.554713+0000","last_clean":"2024-04-22T20:48:26.554713+0000","last_became_active":"2024-04-22T20:01:04.874816+0000","last_became_peered":"2024-04-22T20:01:04.874816+0000","last_unstale":"2024-04-22T20:48:26.554713+0000","last_undegraded":"2024-04-22T20:48:26.554713+0000","last_fullsized":"2024-04-22T20:48:26.554713+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'47","last_scrub_stamp":"2024-04-22T20:48:26.554614+0000","last_deep_scrub":"261'47","last_deep_scrub_stamp":"2024-04-22T20:48:26.554614+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.554614+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:35:39.583851+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":291,"num_read_kb":200,"num_write":62,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3253,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.079551+0000","last_change":"2024-04-22T20:19:16.795282+0000","last_active":"2024-04-22T20:47:35.079551+0000","last_peered":"2024-04-22T20:47:35.079551+0000","last_clean":"2024-04-22T20:47:35.079551+0000","last_became_active":"2024-04-22T20:19:16.794954+0000","last_became_peered":"2024-04-22T20:19:16.794954+0000","last_unstale":"2024-04-22T20:47:35.079551+0000","last_undegraded":"2024-04-22T20:47:35.079551+0000","last_fullsized":"2024-04-22T20:47:35.079551+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"261'6","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.542755+0000","last_change":"2024-04-22T20:48:32.542810+0000","last_active":"2024-04-22T20:48:32.542755+0000","last_peered":"2024-04-22T20:48:32.542755+0000","last_clean":"2024-04-22T20:48:32.542755+0000","last_became_active":"2024-04-22T20:05:17.889370+0000","last_became_peered":"2024-04-22T20:05:17.889370+0000","last_unstale":"2024-04-22T20:48:32.542755+0000","last_undegraded":"2024-04-22T20:48:32.542755+0000","last_fullsized":"2024-04-22T20:48:32.542755+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:32.542653+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:32.542653+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.542653+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:05:37.965129+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":6,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"261'2","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.599057+0000","last_change":"2024-04-22T20:48:25.599082+0000","last_active":"2024-04-22T20:48:25.599057+0000","last_peered":"2024-04-22T20:48:25.599057+0000","last_clean":"2024-04-22T20:48:25.599057+0000","last_became_active":"2024-04-22T20:05:13.815317+0000","last_became_peered":"2024-04-22T20:05:13.815317+0000","last_unstale":"2024-04-22T20:48:25.599057+0000","last_undegraded":"2024-04-22T20:48:25.599057+0000","last_fullsized":"2024-04-22T20:48:25.599057+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:25.598973+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:25.598973+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.598973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:21:44.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"261'28","reported_seq":128,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.478567+0000","last_change":"2024-04-22T20:48:23.478567+0000","last_active":"2024-04-22T20:48:23.478567+0000","last_peered":"2024-04-22T20:48:23.478567+0000","last_clean":"2024-04-22T20:48:23.478567+0000","last_became_active":"2024-04-22T20:01:04.878599+0000","last_became_peered":"2024-04-22T20:01:04.878599+0000","last_unstale":"2024-04-22T20:48:23.478567+0000","last_undegraded":"2024-04-22T20:48:23.478567+0000","last_fullsized":"2024-04-22T20:48:23.478567+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:23.478425+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:23.478425+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.478425+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:33.610274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":77,"num_write":54,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.257006+0000","last_change":"2024-04-22T20:19:16.794996+0000","last_active":"2024-04-22T20:48:25.257006+0000","last_peered":"2024-04-22T20:48:25.257006+0000","last_clean":"2024-04-22T20:48:25.257006+0000","last_became_active":"2024-04-22T20:19:16.793678+0000","last_became_peered":"2024-04-22T20:19:16.793678+0000","last_unstale":"2024-04-22T20:48:25.257006+0000","last_undegraded":"2024-04-22T20:48:25.257006+0000","last_fullsized":"2024-04-22T20:48:25.257006+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:05.174752+0000","last_change":"2024-04-22T20:05:17.892875+0000","last_active":"2024-04-22T20:48:05.174752+0000","last_peered":"2024-04-22T20:48:05.174752+0000","last_clean":"2024-04-22T20:48:05.174752+0000","last_became_active":"2024-04-22T20:05:17.892604+0000","last_became_peered":"2024-04-22T20:05:17.892604+0000","last_unstale":"2024-04-22T20:48:05.174752+0000","last_undegraded":"2024-04-22T20:48:05.174752+0000","last_fullsized":"2024-04-22T20:48:05.174752+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"261'10","reported_seq":69,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.194087+0000","last_change":"2024-04-22T20:05:13.814193+0000","last_active":"2024-04-22T20:48:16.194087+0000","last_peered":"2024-04-22T20:48:16.194087+0000","last_clean":"2024-04-22T20:48:16.194087+0000","last_became_active":"2024-04-22T20:05:13.813946+0000","last_became_peered":"2024-04-22T20:05:13.813946+0000","last_unstale":"2024-04-22T20:48:16.194087+0000","last_undegraded":"2024-04-22T20:48:16.194087+0000","last_fullsized":"2024-04-22T20:48:16.194087+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":15362,"num_write":9,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"261'28","reported_seq":136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.953873+0000","last_change":"2024-04-22T20:48:31.953898+0000","last_active":"2024-04-22T20:48:31.953873+0000","last_peered":"2024-04-22T20:48:31.953873+0000","last_clean":"2024-04-22T20:48:31.953873+0000","last_became_active":"2024-04-22T20:01:04.872834+0000","last_became_peered":"2024-04-22T20:01:04.872834+0000","last_unstale":"2024-04-22T20:48:31.953873+0000","last_undegraded":"2024-04-22T20:48:31.953873+0000","last_fullsized":"2024-04-22T20:48:31.953873+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:31.953787+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:31.953787+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.953787+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:05:33.171326+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":90,"num_write":50,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.332684+0000","last_change":"2024-04-22T20:19:16.791183+0000","last_active":"2024-04-22T20:47:30.332684+0000","last_peered":"2024-04-22T20:47:30.332684+0000","last_clean":"2024-04-22T20:47:30.332684+0000","last_became_active":"2024-04-22T20:19:16.790917+0000","last_became_peered":"2024-04-22T20:19:16.790917+0000","last_unstale":"2024-04-22T20:47:30.332684+0000","last_undegraded":"2024-04-22T20:47:30.332684+0000","last_fullsized":"2024-04-22T20:47:30.332684+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":42,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.063416+0000","last_change":"2024-04-22T20:05:18.211747+0000","last_active":"2024-04-22T20:48:15.063416+0000","last_peered":"2024-04-22T20:48:15.063416+0000","last_clean":"2024-04-22T20:48:15.063416+0000","last_became_active":"2024-04-22T20:05:18.211445+0000","last_became_peered":"2024-04-22T20:05:18.211445+0000","last_unstale":"2024-04-22T20:48:15.063416+0000","last_undegraded":"2024-04-22T20:48:15.063416+0000","last_fullsized":"2024-04-22T20:48:15.063416+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"258'2","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.661659+0000","last_change":"2024-04-22T20:48:22.661681+0000","last_active":"2024-04-22T20:48:22.661659+0000","last_peered":"2024-04-22T20:48:22.661659+0000","last_clean":"2024-04-22T20:48:22.661659+0000","last_became_active":"2024-04-22T20:05:13.806928+0000","last_became_peered":"2024-04-22T20:05:13.806928+0000","last_unstale":"2024-04-22T20:48:22.661659+0000","last_undegraded":"2024-04-22T20:48:22.661659+0000","last_fullsized":"2024-04-22T20:48:22.661659+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'2","last_scrub_stamp":"2024-04-22T20:48:22.661579+0000","last_deep_scrub":"258'2","last_deep_scrub_stamp":"2024-04-22T20:48:22.661579+0000","last_clean_scrub_stamp":"2024-04-22T20:48:22.661579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:42.533500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"261'28","reported_seq":127,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.409573+0000","last_change":"2024-04-22T20:48:27.409602+0000","last_active":"2024-04-22T20:48:27.409573+0000","last_peered":"2024-04-22T20:48:27.409573+0000","last_clean":"2024-04-22T20:48:27.409573+0000","last_became_active":"2024-04-22T20:01:04.869226+0000","last_became_peered":"2024-04-22T20:01:04.869226+0000","last_unstale":"2024-04-22T20:48:27.409573+0000","last_undegraded":"2024-04-22T20:48:27.409573+0000","last_fullsized":"2024-04-22T20:48:27.409573+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:27.409466+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:27.409466+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.409466+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:07.279566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":80,"num_write":56,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.081712+0000","last_change":"2024-04-22T20:19:16.796398+0000","last_active":"2024-04-22T20:47:30.081712+0000","last_peered":"2024-04-22T20:47:30.081712+0000","last_clean":"2024-04-22T20:47:30.081712+0000","last_became_active":"2024-04-22T20:19:16.795524+0000","last_became_peered":"2024-04-22T20:19:16.795524+0000","last_unstale":"2024-04-22T20:47:30.081712+0000","last_undegraded":"2024-04-22T20:47:30.081712+0000","last_fullsized":"2024-04-22T20:47:30.081712+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'4","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.484657+0000","last_change":"2024-04-22T20:05:17.899183+0000","last_active":"2024-04-22T20:48:16.484657+0000","last_peered":"2024-04-22T20:48:16.484657+0000","last_clean":"2024-04-22T20:48:16.484657+0000","last_became_active":"2024-04-22T20:05:17.898858+0000","last_became_peered":"2024-04-22T20:05:17.898858+0000","last_unstale":"2024-04-22T20:48:16.484657+0000","last_undegraded":"2024-04-22T20:48:16.484657+0000","last_fullsized":"2024-04-22T20:48:16.484657+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"258'3","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.510708+0000","last_change":"2024-04-22T20:48:27.510738+0000","last_active":"2024-04-22T20:48:27.510708+0000","last_peered":"2024-04-22T20:48:27.510708+0000","last_clean":"2024-04-22T20:48:27.510708+0000","last_became_active":"2024-04-22T20:05:13.807945+0000","last_became_peered":"2024-04-22T20:05:13.807945+0000","last_unstale":"2024-04-22T20:48:27.510708+0000","last_undegraded":"2024-04-22T20:48:27.510708+0000","last_fullsized":"2024-04-22T20:48:27.510708+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'3","last_scrub_stamp":"2024-04-22T20:48:27.510574+0000","last_deep_scrub":"258'3","last_deep_scrub_stamp":"2024-04-22T20:48:27.510574+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.510574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:46.475027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":8,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"261'24","reported_seq":115,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.452597+0000","last_change":"2024-04-22T20:48:32.452634+0000","last_active":"2024-04-22T20:48:32.452597+0000","last_peered":"2024-04-22T20:48:32.452597+0000","last_clean":"2024-04-22T20:48:32.452597+0000","last_became_active":"2024-04-22T20:01:04.871829+0000","last_became_peered":"2024-04-22T20:01:04.871829+0000","last_unstale":"2024-04-22T20:48:32.452597+0000","last_undegraded":"2024-04-22T20:48:32.452597+0000","last_fullsized":"2024-04-22T20:48:32.452597+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'24","last_scrub_stamp":"2024-04-22T20:48:32.452510+0000","last_deep_scrub":"261'24","last_deep_scrub_stamp":"2024-04-22T20:48:32.452510+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.452510+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:57.685562+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":61,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.415669+0000","last_change":"2024-04-22T20:19:16.794419+0000","last_active":"2024-04-22T20:48:23.415669+0000","last_peered":"2024-04-22T20:48:23.415669+0000","last_clean":"2024-04-22T20:48:23.415669+0000","last_became_active":"2024-04-22T20:19:16.793727+0000","last_became_peered":"2024-04-22T20:19:16.793727+0000","last_unstale":"2024-04-22T20:48:23.415669+0000","last_undegraded":"2024-04-22T20:48:23.415669+0000","last_fullsized":"2024-04-22T20:48:23.415669+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"261'5","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875813+0000","last_change":"2024-04-22T20:05:17.899691+0000","last_active":"2024-04-22T20:48:23.875813+0000","last_peered":"2024-04-22T20:48:23.875813+0000","last_clean":"2024-04-22T20:48:23.875813+0000","last_became_active":"2024-04-22T20:05:17.899328+0000","last_became_peered":"2024-04-22T20:05:17.899328+0000","last_unstale":"2024-04-22T20:48:23.875813+0000","last_undegraded":"2024-04-22T20:48:23.875813+0000","last_fullsized":"2024-04-22T20:48:23.875813+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'7","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.630926+0000","last_change":"2024-04-22T20:48:25.630926+0000","last_active":"2024-04-22T20:48:25.630926+0000","last_peered":"2024-04-22T20:48:25.630926+0000","last_clean":"2024-04-22T20:48:25.630926+0000","last_became_active":"2024-04-22T20:05:13.811986+0000","last_became_peered":"2024-04-22T20:05:13.811986+0000","last_unstale":"2024-04-22T20:48:25.630926+0000","last_undegraded":"2024-04-22T20:48:25.630926+0000","last_fullsized":"2024-04-22T20:48:25.630926+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:25.630769+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:25.630769+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.630769+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:31:51.985237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8240,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"261'66","reported_seq":221,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.587356+0000","last_change":"2024-04-22T20:48:26.587419+0000","last_active":"2024-04-22T20:48:26.587356+0000","last_peered":"2024-04-22T20:48:26.587356+0000","last_clean":"2024-04-22T20:48:26.587356+0000","last_became_active":"2024-04-22T20:01:04.874827+0000","last_became_peered":"2024-04-22T20:01:04.874827+0000","last_unstale":"2024-04-22T20:48:26.587356+0000","last_undegraded":"2024-04-22T20:48:26.587356+0000","last_fullsized":"2024-04-22T20:48:26.587356+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'66","last_scrub_stamp":"2024-04-22T20:48:26.587233+0000","last_deep_scrub":"261'66","last_deep_scrub_stamp":"2024-04-22T20:48:26.587233+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.587233+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:32.347942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":281,"num_read_kb":200,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.073261+0000","last_change":"2024-04-22T20:19:16.794759+0000","last_active":"2024-04-22T20:46:30.073261+0000","last_peered":"2024-04-22T20:46:30.073261+0000","last_clean":"2024-04-22T20:46:30.073261+0000","last_became_active":"2024-04-22T20:19:16.794434+0000","last_became_peered":"2024-04-22T20:19:16.794434+0000","last_unstale":"2024-04-22T20:46:30.073261+0000","last_undegraded":"2024-04-22T20:46:30.073261+0000","last_fullsized":"2024-04-22T20:46:30.073261+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"261'9","reported_seq":65,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.412462+0000","last_change":"2024-04-22T20:48:36.412462+0000","last_active":"2024-04-22T20:48:36.412462+0000","last_peered":"2024-04-22T20:48:36.412462+0000","last_clean":"2024-04-22T20:48:36.412462+0000","last_became_active":"2024-04-22T20:05:18.209025+0000","last_became_peered":"2024-04-22T20:05:18.209025+0000","last_unstale":"2024-04-22T20:48:36.412462+0000","last_undegraded":"2024-04-22T20:48:36.412462+0000","last_fullsized":"2024-04-22T20:48:36.412462+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-22T20:48:36.412356+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-22T20:48:36.412356+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.412356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:02.300050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":13312,"num_write":14,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"261'4","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.619549+0000","last_change":"2024-04-22T20:48:22.619549+0000","last_active":"2024-04-22T20:48:22.619549+0000","last_peered":"2024-04-22T20:48:22.619549+0000","last_clean":"2024-04-22T20:48:22.619549+0000","last_became_active":"2024-04-22T20:05:13.807603+0000","last_became_peered":"2024-04-22T20:05:13.807603+0000","last_unstale":"2024-04-22T20:48:22.619549+0000","last_undegraded":"2024-04-22T20:48:22.619549+0000","last_fullsized":"2024-04-22T20:48:22.619549+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:22.619402+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:22.619402+0000","last_clean_scrub_stamp":"2024-04-22T20:48:22.619402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:32:09.200890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"258'23","reported_seq":121,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.404462+0000","last_change":"2024-04-22T20:48:37.404488+0000","last_active":"2024-04-22T20:48:37.404462+0000","last_peered":"2024-04-22T20:48:37.404462+0000","last_clean":"2024-04-22T20:48:37.404462+0000","last_became_active":"2024-04-22T20:01:04.868847+0000","last_became_peered":"2024-04-22T20:01:04.868847+0000","last_unstale":"2024-04-22T20:48:37.404462+0000","last_undegraded":"2024-04-22T20:48:37.404462+0000","last_fullsized":"2024-04-22T20:48:37.404462+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'23","last_scrub_stamp":"2024-04-22T20:48:37.404364+0000","last_deep_scrub":"258'23","last_deep_scrub_stamp":"2024-04-22T20:48:37.404364+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.404364+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:31.027402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":76,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"261'1","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.257034+0000","last_change":"2024-04-22T20:19:16.796622+0000","last_active":"2024-04-22T20:48:25.257034+0000","last_peered":"2024-04-22T20:48:25.257034+0000","last_clean":"2024-04-22T20:48:25.257034+0000","last_became_active":"2024-04-22T20:19:16.795613+0000","last_became_peered":"2024-04-22T20:19:16.795613+0000","last_unstale":"2024-04-22T20:48:25.257034+0000","last_undegraded":"2024-04-22T20:48:25.257034+0000","last_fullsized":"2024-04-22T20:48:25.257034+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"261'6","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.538494+0000","last_change":"2024-04-22T20:48:31.538524+0000","last_active":"2024-04-22T20:48:31.538494+0000","last_peered":"2024-04-22T20:48:31.538494+0000","last_clean":"2024-04-22T20:48:31.538494+0000","last_became_active":"2024-04-22T20:05:17.891331+0000","last_became_peered":"2024-04-22T20:05:17.891331+0000","last_unstale":"2024-04-22T20:48:31.538494+0000","last_undegraded":"2024-04-22T20:48:31.538494+0000","last_fullsized":"2024-04-22T20:48:31.538494+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:31.538340+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:31.538340+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.538340+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:36:13.794601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.504485+0000","last_change":"2024-04-22T20:48:37.504525+0000","last_active":"2024-04-22T20:48:37.504485+0000","last_peered":"2024-04-22T20:48:37.504485+0000","last_clean":"2024-04-22T20:48:37.504485+0000","last_became_active":"2024-04-22T20:05:13.813811+0000","last_became_peered":"2024-04-22T20:05:13.813811+0000","last_unstale":"2024-04-22T20:48:37.504485+0000","last_undegraded":"2024-04-22T20:48:37.504485+0000","last_fullsized":"2024-04-22T20:48:37.504485+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:37.504369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:37.504369+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.504369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:22.962064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"258'49","reported_seq":179,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.613692+0000","last_change":"2024-04-22T20:48:29.613719+0000","last_active":"2024-04-22T20:48:29.613692+0000","last_peered":"2024-04-22T20:48:29.613692+0000","last_clean":"2024-04-22T20:48:29.613692+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:48:29.613692+0000","last_undegraded":"2024-04-22T20:48:29.613692+0000","last_fullsized":"2024-04-22T20:48:29.613692+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'49","last_scrub_stamp":"2024-04-22T20:48:29.613592+0000","last_deep_scrub":"258'49","last_deep_scrub_stamp":"2024-04-22T20:48:29.613592+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.613592+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:50:57.860950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":145,"num_write":78,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.034593+0000","last_change":"2024-04-22T20:19:16.793813+0000","last_active":"2024-04-22T20:48:22.034593+0000","last_peered":"2024-04-22T20:48:22.034593+0000","last_clean":"2024-04-22T20:48:22.034593+0000","last_became_active":"2024-04-22T20:19:16.793486+0000","last_became_peered":"2024-04-22T20:19:16.793486+0000","last_unstale":"2024-04-22T20:48:22.034593+0000","last_undegraded":"2024-04-22T20:48:22.034593+0000","last_fullsized":"2024-04-22T20:48:22.034593+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.489168+0000","last_change":"2024-04-22T20:48:32.489207+0000","last_active":"2024-04-22T20:48:32.489168+0000","last_peered":"2024-04-22T20:48:32.489168+0000","last_clean":"2024-04-22T20:48:32.489168+0000","last_became_active":"2024-04-22T20:05:17.897008+0000","last_became_peered":"2024-04-22T20:05:17.897008+0000","last_unstale":"2024-04-22T20:48:32.489168+0000","last_undegraded":"2024-04-22T20:48:32.489168+0000","last_fullsized":"2024-04-22T20:48:32.489168+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:32.489059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:32.489059+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.489059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:11.685871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"261'15","reported_seq":85,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.378030+0000","last_change":"2024-04-22T20:48:32.378065+0000","last_active":"2024-04-22T20:48:32.378030+0000","last_peered":"2024-04-22T20:48:32.378030+0000","last_clean":"2024-04-22T20:48:32.378030+0000","last_became_active":"2024-04-22T20:05:13.798319+0000","last_became_peered":"2024-04-22T20:05:13.798319+0000","last_unstale":"2024-04-22T20:48:32.378030+0000","last_undegraded":"2024-04-22T20:48:32.378030+0000","last_fullsized":"2024-04-22T20:48:32.378030+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'15","last_scrub_stamp":"2024-04-22T20:48:32.377942+0000","last_deep_scrub":"261'15","last_deep_scrub_stamp":"2024-04-22T20:48:32.377942+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.377942+0000","objects_scrubbed":2,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:58.634704+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":16388,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"261'37","reported_seq":147,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.458510+0000","last_change":"2024-04-22T20:48:28.458538+0000","last_active":"2024-04-22T20:48:28.458510+0000","last_peered":"2024-04-22T20:48:28.458510+0000","last_clean":"2024-04-22T20:48:28.458510+0000","last_became_active":"2024-04-22T20:01:04.869079+0000","last_became_peered":"2024-04-22T20:01:04.869079+0000","last_unstale":"2024-04-22T20:48:28.458510+0000","last_undegraded":"2024-04-22T20:48:28.458510+0000","last_fullsized":"2024-04-22T20:48:28.458510+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'37","last_scrub_stamp":"2024-04-22T20:48:28.458400+0000","last_deep_scrub":"261'37","last_deep_scrub_stamp":"2024-04-22T20:48:28.458400+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.458400+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:24:11.187493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":61,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.256974+0000","last_change":"2024-04-22T20:19:16.796317+0000","last_active":"2024-04-22T20:48:25.256974+0000","last_peered":"2024-04-22T20:48:25.256974+0000","last_clean":"2024-04-22T20:48:25.256974+0000","last_became_active":"2024-04-22T20:19:16.795462+0000","last_became_peered":"2024-04-22T20:19:16.795462+0000","last_unstale":"2024-04-22T20:48:25.256974+0000","last_undegraded":"2024-04-22T20:48:25.256974+0000","last_fullsized":"2024-04-22T20:48:25.256974+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"258'6","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.193372+0000","last_change":"2024-04-22T20:05:17.892351+0000","last_active":"2024-04-22T20:47:10.193372+0000","last_peered":"2024-04-22T20:47:10.193372+0000","last_clean":"2024-04-22T20:47:10.193372+0000","last_became_active":"2024-04-22T20:05:17.892044+0000","last_became_peered":"2024-04-22T20:05:17.892044+0000","last_unstale":"2024-04-22T20:47:10.193372+0000","last_undegraded":"2024-04-22T20:47:10.193372+0000","last_fullsized":"2024-04-22T20:47:10.193372+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"261'5","reported_seq":68,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.541530+0000","last_change":"2024-04-22T20:48:29.541577+0000","last_active":"2024-04-22T20:48:29.541530+0000","last_peered":"2024-04-22T20:48:29.541530+0000","last_clean":"2024-04-22T20:48:29.541530+0000","last_became_active":"2024-04-22T20:05:13.809368+0000","last_became_peered":"2024-04-22T20:05:13.809368+0000","last_unstale":"2024-04-22T20:48:29.541530+0000","last_undegraded":"2024-04-22T20:48:29.541530+0000","last_fullsized":"2024-04-22T20:48:29.541530+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:29.541434+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:29.541434+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.541434+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:38:26.695883+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"261'35","reported_seq":147,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.558084+0000","last_change":"2024-04-22T20:48:27.558111+0000","last_active":"2024-04-22T20:48:27.558084+0000","last_peered":"2024-04-22T20:48:27.558084+0000","last_clean":"2024-04-22T20:48:27.558084+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:48:27.558084+0000","last_undegraded":"2024-04-22T20:48:27.558084+0000","last_fullsized":"2024-04-22T20:48:27.558084+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'35","last_scrub_stamp":"2024-04-22T20:48:27.558001+0000","last_deep_scrub":"261'35","last_deep_scrub_stamp":"2024-04-22T20:48:27.558001+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.558001+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:44:57.759608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":59,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.335414+0000","last_change":"2024-04-22T20:19:16.791940+0000","last_active":"2024-04-22T20:48:24.335414+0000","last_peered":"2024-04-22T20:48:24.335414+0000","last_clean":"2024-04-22T20:48:24.335414+0000","last_became_active":"2024-04-22T20:19:16.791663+0000","last_became_peered":"2024-04-22T20:19:16.791663+0000","last_unstale":"2024-04-22T20:48:24.335414+0000","last_undegraded":"2024-04-22T20:48:24.335414+0000","last_fullsized":"2024-04-22T20:48:24.335414+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"261'6","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.564496+0000","last_change":"2024-04-22T20:05:18.211014+0000","last_active":"2024-04-22T20:48:16.564496+0000","last_peered":"2024-04-22T20:48:16.564496+0000","last_clean":"2024-04-22T20:48:16.564496+0000","last_became_active":"2024-04-22T20:05:18.210736+0000","last_became_peered":"2024-04-22T20:05:18.210736+0000","last_unstale":"2024-04-22T20:48:16.564496+0000","last_undegraded":"2024-04-22T20:48:16.564496+0000","last_fullsized":"2024-04-22T20:48:16.564496+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20482,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"261'6","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.260164+0000","last_change":"2024-04-22T20:05:13.813554+0000","last_active":"2024-04-22T20:48:16.260164+0000","last_peered":"2024-04-22T20:48:16.260164+0000","last_clean":"2024-04-22T20:48:16.260164+0000","last_became_active":"2024-04-22T20:05:13.813277+0000","last_became_peered":"2024-04-22T20:05:13.813277+0000","last_unstale":"2024-04-22T20:48:16.260164+0000","last_undegraded":"2024-04-22T20:48:16.260164+0000","last_fullsized":"2024-04-22T20:48:16.260164+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1024,"num_write":16,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"261'61","reported_seq":203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.363674+0000","last_change":"2024-04-22T20:48:25.363698+0000","last_active":"2024-04-22T20:48:25.363674+0000","last_peered":"2024-04-22T20:48:25.363674+0000","last_clean":"2024-04-22T20:48:25.363674+0000","last_became_active":"2024-04-22T20:01:04.871859+0000","last_became_peered":"2024-04-22T20:01:04.871859+0000","last_unstale":"2024-04-22T20:48:25.363674+0000","last_undegraded":"2024-04-22T20:48:25.363674+0000","last_fullsized":"2024-04-22T20:48:25.363674+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'61","last_scrub_stamp":"2024-04-22T20:48:25.363585+0000","last_deep_scrub":"261'61","last_deep_scrub_stamp":"2024-04-22T20:48:25.363585+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.363585+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:50:52.645407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":172,"num_write":109,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.057308+0000","last_change":"2024-04-22T20:19:16.790599+0000","last_active":"2024-04-22T20:47:25.057308+0000","last_peered":"2024-04-22T20:47:25.057308+0000","last_clean":"2024-04-22T20:47:25.057308+0000","last_became_active":"2024-04-22T20:19:16.789688+0000","last_became_peered":"2024-04-22T20:19:16.789688+0000","last_unstale":"2024-04-22T20:47:25.057308+0000","last_undegraded":"2024-04-22T20:47:25.057308+0000","last_fullsized":"2024-04-22T20:47:25.057308+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"258'4","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.256890+0000","last_change":"2024-04-22T20:05:18.211736+0000","last_active":"2024-04-22T20:48:25.256890+0000","last_peered":"2024-04-22T20:48:25.256890+0000","last_clean":"2024-04-22T20:48:25.256890+0000","last_became_active":"2024-04-22T20:05:18.211478+0000","last_became_peered":"2024-04-22T20:05:18.211478+0000","last_unstale":"2024-04-22T20:48:25.256890+0000","last_undegraded":"2024-04-22T20:48:25.256890+0000","last_fullsized":"2024-04-22T20:48:25.256890+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'7","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.335294+0000","last_change":"2024-04-22T20:05:13.811784+0000","last_active":"2024-04-22T20:48:24.335294+0000","last_peered":"2024-04-22T20:48:24.335294+0000","last_clean":"2024-04-22T20:48:24.335294+0000","last_became_active":"2024-04-22T20:05:13.811561+0000","last_became_peered":"2024-04-22T20:05:13.811561+0000","last_unstale":"2024-04-22T20:48:24.335294+0000","last_undegraded":"2024-04-22T20:48:24.335294+0000","last_fullsized":"2024-04-22T20:48:24.335294+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":12,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"261'33","reported_seq":144,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.528994+0000","last_change":"2024-04-22T20:48:31.529020+0000","last_active":"2024-04-22T20:48:31.528994+0000","last_peered":"2024-04-22T20:48:31.528994+0000","last_clean":"2024-04-22T20:48:31.528994+0000","last_became_active":"2024-04-22T20:01:04.874790+0000","last_became_peered":"2024-04-22T20:01:04.874790+0000","last_unstale":"2024-04-22T20:48:31.528994+0000","last_undegraded":"2024-04-22T20:48:31.528994+0000","last_fullsized":"2024-04-22T20:48:31.528994+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'33","last_scrub_stamp":"2024-04-22T20:48:31.528870+0000","last_deep_scrub":"261'33","last_deep_scrub_stamp":"2024-04-22T20:48:31.528870+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.528870+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:43.937257+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":100,"num_write":57,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875852+0000","last_change":"2024-04-22T20:19:16.794065+0000","last_active":"2024-04-22T20:48:23.875852+0000","last_peered":"2024-04-22T20:48:23.875852+0000","last_clean":"2024-04-22T20:48:23.875852+0000","last_became_active":"2024-04-22T20:19:16.793188+0000","last_became_peered":"2024-04-22T20:19:16.793188+0000","last_unstale":"2024-04-22T20:48:23.875852+0000","last_undegraded":"2024-04-22T20:48:23.875852+0000","last_fullsized":"2024-04-22T20:48:23.875852+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"261'5","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875856+0000","last_change":"2024-04-22T20:05:17.901392+0000","last_active":"2024-04-22T20:48:23.875856+0000","last_peered":"2024-04-22T20:48:23.875856+0000","last_clean":"2024-04-22T20:48:23.875856+0000","last_became_active":"2024-04-22T20:05:17.901128+0000","last_became_peered":"2024-04-22T20:05:17.901128+0000","last_unstale":"2024-04-22T20:48:23.875856+0000","last_undegraded":"2024-04-22T20:48:23.875856+0000","last_fullsized":"2024-04-22T20:48:23.875856+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":8192,"num_write":10,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"261'5","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.568012+0000","last_change":"2024-04-22T20:48:28.568012+0000","last_active":"2024-04-22T20:48:28.568012+0000","last_peered":"2024-04-22T20:48:28.568012+0000","last_clean":"2024-04-22T20:48:28.568012+0000","last_became_active":"2024-04-22T20:05:13.815332+0000","last_became_peered":"2024-04-22T20:05:13.815332+0000","last_unstale":"2024-04-22T20:48:28.568012+0000","last_undegraded":"2024-04-22T20:48:28.568012+0000","last_fullsized":"2024-04-22T20:48:28.568012+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:28.567863+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:28.567863+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.567863+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:52:21.493631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":9216,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'28","reported_seq":133,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.541251+0000","last_change":"2024-04-22T20:48:26.541277+0000","last_active":"2024-04-22T20:48:26.541251+0000","last_peered":"2024-04-22T20:48:26.541251+0000","last_clean":"2024-04-22T20:48:26.541251+0000","last_became_active":"2024-04-22T20:01:04.871840+0000","last_became_peered":"2024-04-22T20:01:04.871840+0000","last_unstale":"2024-04-22T20:48:26.541251+0000","last_undegraded":"2024-04-22T20:48:26.541251+0000","last_fullsized":"2024-04-22T20:48:26.541251+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:26.541161+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:26.541161+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.541161+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:03:51.347595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":89,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"261'1","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.079099+0000","last_change":"2024-04-22T20:19:16.794411+0000","last_active":"2024-04-22T20:47:30.079099+0000","last_peered":"2024-04-22T20:47:30.079099+0000","last_clean":"2024-04-22T20:47:30.079099+0000","last_became_active":"2024-04-22T20:19:16.794123+0000","last_became_peered":"2024-04-22T20:19:16.794123+0000","last_unstale":"2024-04-22T20:47:30.079099+0000","last_undegraded":"2024-04-22T20:47:30.079099+0000","last_fullsized":"2024-04-22T20:47:30.079099+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"261'4","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.563993+0000","last_change":"2024-04-22T20:05:17.895539+0000","last_active":"2024-04-22T20:48:16.563993+0000","last_peered":"2024-04-22T20:48:16.563993+0000","last_clean":"2024-04-22T20:48:16.563993+0000","last_became_active":"2024-04-22T20:05:17.895177+0000","last_became_peered":"2024-04-22T20:05:17.895177+0000","last_unstale":"2024-04-22T20:48:16.563993+0000","last_undegraded":"2024-04-22T20:48:16.563993+0000","last_fullsized":"2024-04-22T20:48:16.563993+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":4,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"261'4","reported_seq":65,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.553051+0000","last_change":"2024-04-22T20:48:30.553078+0000","last_active":"2024-04-22T20:48:30.553051+0000","last_peered":"2024-04-22T20:48:30.553051+0000","last_clean":"2024-04-22T20:48:30.553051+0000","last_became_active":"2024-04-22T20:05:13.809047+0000","last_became_peered":"2024-04-22T20:05:13.809047+0000","last_unstale":"2024-04-22T20:48:30.553051+0000","last_undegraded":"2024-04-22T20:48:30.553051+0000","last_fullsized":"2024-04-22T20:48:30.553051+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:30.552959+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:30.552959+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.552959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:44:03.327794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"261'33","reported_seq":139,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.902294+0000","last_change":"2024-04-22T20:48:27.902332+0000","last_active":"2024-04-22T20:48:27.902294+0000","last_peered":"2024-04-22T20:48:27.902294+0000","last_clean":"2024-04-22T20:48:27.902294+0000","last_became_active":"2024-04-22T20:01:04.872370+0000","last_became_peered":"2024-04-22T20:01:04.872370+0000","last_unstale":"2024-04-22T20:48:27.902294+0000","last_undegraded":"2024-04-22T20:48:27.902294+0000","last_fullsized":"2024-04-22T20:48:27.902294+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'33","last_scrub_stamp":"2024-04-22T20:48:27.902204+0000","last_deep_scrub":"261'33","last_deep_scrub_stamp":"2024-04-22T20:48:27.902204+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.902204+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:57:41.905060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":57,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.195802+0000","last_change":"2024-04-22T20:19:16.793728+0000","last_active":"2024-04-22T20:47:30.195802+0000","last_peered":"2024-04-22T20:47:30.195802+0000","last_clean":"2024-04-22T20:47:30.195802+0000","last_became_active":"2024-04-22T20:19:16.793429+0000","last_became_peered":"2024-04-22T20:19:16.793429+0000","last_unstale":"2024-04-22T20:47:30.195802+0000","last_undegraded":"2024-04-22T20:47:30.195802+0000","last_fullsized":"2024-04-22T20:47:30.195802+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"261'2","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.489775+0000","last_change":"2024-04-22T20:05:17.901476+0000","last_active":"2024-04-22T20:48:16.489775+0000","last_peered":"2024-04-22T20:48:16.489775+0000","last_clean":"2024-04-22T20:48:16.489775+0000","last_became_active":"2024-04-22T20:05:17.900874+0000","last_became_peered":"2024-04-22T20:05:17.900874+0000","last_unstale":"2024-04-22T20:48:16.489775+0000","last_undegraded":"2024-04-22T20:48:16.489775+0000","last_fullsized":"2024-04-22T20:48:16.489775+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"261'2","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.612611+0000","last_change":"2024-04-22T20:48:26.612637+0000","last_active":"2024-04-22T20:48:26.612611+0000","last_peered":"2024-04-22T20:48:26.612611+0000","last_clean":"2024-04-22T20:48:26.612611+0000","last_became_active":"2024-04-22T20:05:13.813934+0000","last_became_peered":"2024-04-22T20:05:13.813934+0000","last_unstale":"2024-04-22T20:48:26.612611+0000","last_undegraded":"2024-04-22T20:48:26.612611+0000","last_fullsized":"2024-04-22T20:48:26.612611+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:26.612508+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:26.612508+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.612508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:58:57.908355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1024,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"261'37","reported_seq":155,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.240483+0000","last_change":"2024-04-22T20:48:24.240514+0000","last_active":"2024-04-22T20:48:24.240483+0000","last_peered":"2024-04-22T20:48:24.240483+0000","last_clean":"2024-04-22T20:48:24.240483+0000","last_became_active":"2024-04-22T20:01:04.869660+0000","last_became_peered":"2024-04-22T20:01:04.869660+0000","last_unstale":"2024-04-22T20:48:24.240483+0000","last_undegraded":"2024-04-22T20:48:24.240483+0000","last_fullsized":"2024-04-22T20:48:24.240483+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'37","last_scrub_stamp":"2024-04-22T20:48:24.240383+0000","last_deep_scrub":"261'37","last_deep_scrub_stamp":"2024-04-22T20:48:24.240383+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.240383+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:40:22.419745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":113,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.034566+0000","last_change":"2024-04-22T20:19:16.795977+0000","last_active":"2024-04-22T20:48:22.034566+0000","last_peered":"2024-04-22T20:48:22.034566+0000","last_clean":"2024-04-22T20:48:22.034566+0000","last_became_active":"2024-04-22T20:19:16.795509+0000","last_became_peered":"2024-04-22T20:19:16.795509+0000","last_unstale":"2024-04-22T20:48:22.034566+0000","last_undegraded":"2024-04-22T20:48:22.034566+0000","last_fullsized":"2024-04-22T20:48:22.034566+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":43,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:00.328241+0000","last_change":"2024-04-22T20:05:17.891942+0000","last_active":"2024-04-22T20:47:00.328241+0000","last_peered":"2024-04-22T20:47:00.328241+0000","last_clean":"2024-04-22T20:47:00.328241+0000","last_became_active":"2024-04-22T20:05:17.891523+0000","last_became_peered":"2024-04-22T20:05:17.891523+0000","last_unstale":"2024-04-22T20:47:00.328241+0000","last_undegraded":"2024-04-22T20:47:00.328241+0000","last_fullsized":"2024-04-22T20:47:00.328241+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"261'2","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.553948+0000","last_change":"2024-04-22T20:48:24.553971+0000","last_active":"2024-04-22T20:48:24.553948+0000","last_peered":"2024-04-22T20:48:24.553948+0000","last_clean":"2024-04-22T20:48:24.553948+0000","last_became_active":"2024-04-22T20:05:13.805704+0000","last_became_peered":"2024-04-22T20:05:13.805704+0000","last_unstale":"2024-04-22T20:48:24.553948+0000","last_undegraded":"2024-04-22T20:48:24.553948+0000","last_fullsized":"2024-04-22T20:48:24.553948+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:24.553871+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:24.553871+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.553871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:02.108209+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"261'65","reported_seq":233,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.443217+0000","last_change":"2024-04-22T20:48:29.443246+0000","last_active":"2024-04-22T20:48:29.443217+0000","last_peered":"2024-04-22T20:48:29.443217+0000","last_clean":"2024-04-22T20:48:29.443217+0000","last_became_active":"2024-04-22T20:01:04.865981+0000","last_became_peered":"2024-04-22T20:01:04.865981+0000","last_unstale":"2024-04-22T20:48:29.443217+0000","last_undegraded":"2024-04-22T20:48:29.443217+0000","last_fullsized":"2024-04-22T20:48:29.443217+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'65","last_scrub_stamp":"2024-04-22T20:48:29.443124+0000","last_deep_scrub":"261'65","last_deep_scrub_stamp":"2024-04-22T20:48:29.443124+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.443124+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:12:31.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":226,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.256956+0000","last_change":"2024-04-22T20:19:16.799325+0000","last_active":"2024-04-22T20:48:25.256956+0000","last_peered":"2024-04-22T20:48:25.256956+0000","last_clean":"2024-04-22T20:48:25.256956+0000","last_became_active":"2024-04-22T20:19:16.799063+0000","last_became_peered":"2024-04-22T20:19:16.799063+0000","last_unstale":"2024-04-22T20:48:25.256956+0000","last_undegraded":"2024-04-22T20:48:25.256956+0000","last_fullsized":"2024-04-22T20:48:25.256956+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"261'2","reported_seq":47,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.792867+0000","last_change":"2024-04-22T20:05:17.901995+0000","last_active":"2024-04-22T20:48:17.792867+0000","last_peered":"2024-04-22T20:48:17.792867+0000","last_clean":"2024-04-22T20:48:17.792867+0000","last_became_active":"2024-04-22T20:05:17.901680+0000","last_became_peered":"2024-04-22T20:05:17.901680+0000","last_unstale":"2024-04-22T20:48:17.792867+0000","last_undegraded":"2024-04-22T20:48:17.792867+0000","last_fullsized":"2024-04-22T20:48:17.792867+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":80,"num_write":2,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:55.327650+0000","last_change":"2024-04-22T20:05:13.811164+0000","last_active":"2024-04-22T20:46:55.327650+0000","last_peered":"2024-04-22T20:46:55.327650+0000","last_clean":"2024-04-22T20:46:55.327650+0000","last_became_active":"2024-04-22T20:05:13.810865+0000","last_became_peered":"2024-04-22T20:05:13.810865+0000","last_unstale":"2024-04-22T20:46:55.327650+0000","last_undegraded":"2024-04-22T20:46:55.327650+0000","last_fullsized":"2024-04-22T20:46:55.327650+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"261'57","reported_seq":189,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.363453+0000","last_change":"2024-04-22T20:48:35.363498+0000","last_active":"2024-04-22T20:48:35.363453+0000","last_peered":"2024-04-22T20:48:35.363453+0000","last_clean":"2024-04-22T20:48:35.363453+0000","last_became_active":"2024-04-22T20:01:04.870253+0000","last_became_peered":"2024-04-22T20:01:04.870253+0000","last_unstale":"2024-04-22T20:48:35.363453+0000","last_undegraded":"2024-04-22T20:48:35.363453+0000","last_fullsized":"2024-04-22T20:48:35.363453+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'57","last_scrub_stamp":"2024-04-22T20:48:35.363335+0000","last_deep_scrub":"261'57","last_deep_scrub_stamp":"2024-04-22T20:48:35.363335+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.363335+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:44:05.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":145,"num_write":102,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"261'344","reported_seq":960,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.428579+0000","last_change":"2024-04-22T20:48:30.428623+0000","last_active":"2024-04-22T20:48:30.428579+0000","last_peered":"2024-04-22T20:48:30.428579+0000","last_clean":"2024-04-22T20:48:30.428579+0000","last_became_active":"2024-04-22T19:53:04.823351+0000","last_became_peered":"2024-04-22T19:53:04.823351+0000","last_unstale":"2024-04-22T20:48:30.428579+0000","last_undegraded":"2024-04-22T20:48:30.428579+0000","last_fullsized":"2024-04-22T20:48:30.428579+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'344","last_scrub_stamp":"2024-04-22T20:48:30.428472+0000","last_deep_scrub":"261'344","last_deep_scrub_stamp":"2024-04-22T20:48:30.428472+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.428472+0000","objects_scrubbed":5,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:45:58.154599+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":495,"num_write":1298,"num_write_kb":39497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"261'169","reported_seq":764,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.196673+0000","last_change":"2024-04-22T19:51:28.511862+0000","last_active":"2024-04-22T20:46:30.196673+0000","last_peered":"2024-04-22T20:46:30.196673+0000","last_clean":"2024-04-22T20:46:30.196673+0000","last_became_active":"2024-04-22T19:51:28.511595+0000","last_became_peered":"2024-04-22T19:51:28.511595+0000","last_unstale":"2024-04-22T20:46:30.196673+0000","last_undegraded":"2024-04-22T20:46:30.196673+0000","last_fullsized":"2024-04-22T20:46:30.196673+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":537,"num_write":262,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"261'316","reported_seq":968,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.927684+0000","last_change":"2024-04-22T19:51:26.496852+0000","last_active":"2024-04-22T20:48:17.927684+0000","last_peered":"2024-04-22T20:48:17.927684+0000","last_clean":"2024-04-22T20:48:17.927684+0000","last_became_active":"2024-04-22T19:51:26.496188+0000","last_became_peered":"2024-04-22T19:51:26.496188+0000","last_unstale":"2024-04-22T20:48:17.927684+0000","last_undegraded":"2024-04-22T20:48:17.927684+0000","last_fullsized":"2024-04-22T20:48:17.927684+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5719773,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":497,"num_read_kb":9758,"num_write":1112,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"261'314","reported_seq":887,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:28.116135+0000","last_change":"2024-04-22T19:51:28.519482+0000","last_active":"2024-04-22T20:47:28.116135+0000","last_peered":"2024-04-22T20:47:28.116135+0000","last_clean":"2024-04-22T20:47:28.116135+0000","last_became_active":"2024-04-22T19:51:28.519111+0000","last_became_peered":"2024-04-22T19:51:28.519111+0000","last_unstale":"2024-04-22T20:47:28.116135+0000","last_undegraded":"2024-04-22T20:47:28.116135+0000","last_fullsized":"2024-04-22T20:47:28.116135+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1190,"num_read_kb":913,"num_write":510,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"261'320","reported_seq":942,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.949162+0000","last_change":"2024-04-22T19:51:26.503143+0000","last_active":"2024-04-22T20:48:17.949162+0000","last_peered":"2024-04-22T20:48:17.949162+0000","last_clean":"2024-04-22T20:48:17.949162+0000","last_became_active":"2024-04-22T19:51:26.502744+0000","last_became_peered":"2024-04-22T19:51:26.502744+0000","last_unstale":"2024-04-22T20:48:17.949162+0000","last_undegraded":"2024-04-22T20:48:17.949162+0000","last_fullsized":"2024-04-22T20:48:17.949162+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437211,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":443,"num_read_kb":7116,"num_write":1035,"num_write_kb":21496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"261'150","reported_seq":727,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.188212+0000","last_change":"2024-04-22T19:51:28.518949+0000","last_active":"2024-04-22T20:46:30.188212+0000","last_peered":"2024-04-22T20:46:30.188212+0000","last_clean":"2024-04-22T20:46:30.188212+0000","last_became_active":"2024-04-22T19:51:28.518650+0000","last_became_peered":"2024-04-22T19:51:28.518650+0000","last_unstale":"2024-04-22T20:46:30.188212+0000","last_undegraded":"2024-04-22T20:46:30.188212+0000","last_fullsized":"2024-04-22T20:46:30.188212+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":572,"num_read_kb":453,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"261'324","reported_seq":940,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.941407+0000","last_change":"2024-04-22T19:51:26.504934+0000","last_active":"2024-04-22T20:48:17.941407+0000","last_peered":"2024-04-22T20:48:17.941407+0000","last_clean":"2024-04-22T20:48:17.941407+0000","last_became_active":"2024-04-22T19:51:26.504450+0000","last_became_peered":"2024-04-22T19:51:26.504450+0000","last_unstale":"2024-04-22T20:48:17.941407+0000","last_undegraded":"2024-04-22T20:48:17.941407+0000","last_fullsized":"2024-04-22T20:48:17.941407+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8563195,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":424,"num_read_kb":25508,"num_write":1190,"num_write_kb":37655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"261'81","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.546367+0000","last_change":"2024-04-22T20:48:32.546404+0000","last_active":"2024-04-22T20:48:32.546367+0000","last_peered":"2024-04-22T20:48:32.546367+0000","last_clean":"2024-04-22T20:48:32.546367+0000","last_became_active":"2024-04-22T19:52:04.619688+0000","last_became_peered":"2024-04-22T19:52:04.619688+0000","last_unstale":"2024-04-22T20:48:32.546367+0000","last_undegraded":"2024-04-22T20:48:32.546367+0000","last_fullsized":"2024-04-22T20:48:32.546367+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'81","last_scrub_stamp":"2024-04-22T20:48:32.546276+0000","last_deep_scrub":"261'81","last_deep_scrub_stamp":"2024-04-22T20:48:32.546276+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.546276+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:51:45.224120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":353,"num_read_kb":299,"num_write":120,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"261'281","reported_seq":970,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.945701+0000","last_change":"2024-04-22T19:51:26.509141+0000","last_active":"2024-04-22T20:48:17.945701+0000","last_peered":"2024-04-22T20:48:17.945701+0000","last_clean":"2024-04-22T20:48:17.945701+0000","last_became_active":"2024-04-22T19:51:26.507185+0000","last_became_peered":"2024-04-22T19:51:26.507185+0000","last_unstale":"2024-04-22T20:48:17.945701+0000","last_undegraded":"2024-04-22T20:48:17.945701+0000","last_fullsized":"2024-04-22T20:48:17.945701+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3508486,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":442,"num_read_kb":10200,"num_write":925,"num_write_kb":30630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"261'40","reported_seq":595,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.333461+0000","last_change":"2024-04-22T19:51:28.513981+0000","last_active":"2024-04-22T20:47:35.333461+0000","last_peered":"2024-04-22T20:47:35.333461+0000","last_clean":"2024-04-22T20:47:35.333461+0000","last_became_active":"2024-04-22T19:51:28.513693+0000","last_became_peered":"2024-04-22T19:51:28.513693+0000","last_unstale":"2024-04-22T20:47:35.333461+0000","last_undegraded":"2024-04-22T20:47:35.333461+0000","last_fullsized":"2024-04-22T20:47:35.333461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"261'444","reported_seq":1187,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.875369+0000","last_change":"2024-04-22T19:51:26.509287+0000","last_active":"2024-04-22T20:48:17.875369+0000","last_peered":"2024-04-22T20:48:17.875369+0000","last_clean":"2024-04-22T20:48:17.875369+0000","last_became_active":"2024-04-22T19:51:26.508716+0000","last_became_peered":"2024-04-22T19:51:26.508716+0000","last_unstale":"2024-04-22T20:48:17.875369+0000","last_undegraded":"2024-04-22T20:48:17.875369+0000","last_fullsized":"2024-04-22T20:48:17.875369+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":400,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661200,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1035,"num_read_kb":17244,"num_write":1523,"num_write_kb":25883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"261'144","reported_seq":743,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198578+0000","last_change":"2024-04-22T19:51:28.519053+0000","last_active":"2024-04-22T20:47:45.198578+0000","last_peered":"2024-04-22T20:47:45.198578+0000","last_clean":"2024-04-22T20:47:45.198578+0000","last_became_active":"2024-04-22T19:51:28.518501+0000","last_became_peered":"2024-04-22T19:51:28.518501+0000","last_unstale":"2024-04-22T20:47:45.198578+0000","last_undegraded":"2024-04-22T20:47:45.198578+0000","last_fullsized":"2024-04-22T20:47:45.198578+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":597,"num_read_kb":478,"num_write":226,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"261'377","reported_seq":1039,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.940266+0000","last_change":"2024-04-22T19:51:26.508109+0000","last_active":"2024-04-22T20:48:17.940266+0000","last_peered":"2024-04-22T20:48:17.940266+0000","last_clean":"2024-04-22T20:48:17.940266+0000","last_became_active":"2024-04-22T19:51:26.507141+0000","last_became_peered":"2024-04-22T19:51:26.507141+0000","last_unstale":"2024-04-22T20:48:17.940266+0000","last_undegraded":"2024-04-22T20:48:17.940266+0000","last_fullsized":"2024-04-22T20:48:17.940266+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2367620,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":550,"num_read_kb":43743,"num_write":1389,"num_write_kb":32515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"261'151","reported_seq":737,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.208173+0000","last_change":"2024-04-22T19:51:28.516437+0000","last_active":"2024-04-22T20:47:45.208173+0000","last_peered":"2024-04-22T20:47:45.208173+0000","last_clean":"2024-04-22T20:47:45.208173+0000","last_became_active":"2024-04-22T19:51:28.516012+0000","last_became_peered":"2024-04-22T19:51:28.516012+0000","last_unstale":"2024-04-22T20:47:45.208173+0000","last_undegraded":"2024-04-22T20:47:45.208173+0000","last_fullsized":"2024-04-22T20:47:45.208173+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":470,"num_write":219,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"261'333","reported_seq":978,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.916221+0000","last_change":"2024-04-22T19:51:26.505832+0000","last_active":"2024-04-22T20:48:17.916221+0000","last_peered":"2024-04-22T20:48:17.916221+0000","last_clean":"2024-04-22T20:48:17.916221+0000","last_became_active":"2024-04-22T19:51:26.502928+0000","last_became_peered":"2024-04-22T19:51:26.502928+0000","last_unstale":"2024-04-22T20:48:17.916221+0000","last_undegraded":"2024-04-22T20:48:17.916221+0000","last_fullsized":"2024-04-22T20:48:17.916221+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5414404,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":443,"num_read_kb":51849,"num_write":1202,"num_write_kb":39313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"261'60","reported_seq":630,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.080631+0000","last_change":"2024-04-22T19:51:28.510044+0000","last_active":"2024-04-22T20:47:45.080631+0000","last_peered":"2024-04-22T20:47:45.080631+0000","last_clean":"2024-04-22T20:47:45.080631+0000","last_became_active":"2024-04-22T19:51:28.509774+0000","last_became_peered":"2024-04-22T19:51:28.509774+0000","last_unstale":"2024-04-22T20:47:45.080631+0000","last_undegraded":"2024-04-22T20:47:45.080631+0000","last_fullsized":"2024-04-22T20:47:45.080631+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":211,"num_write":81,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"261'125","reported_seq":735,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.196591+0000","last_change":"2024-04-22T19:51:28.505929+0000","last_active":"2024-04-22T20:47:30.196591+0000","last_peered":"2024-04-22T20:47:30.196591+0000","last_clean":"2024-04-22T20:47:30.196591+0000","last_became_active":"2024-04-22T19:51:28.505562+0000","last_became_peered":"2024-04-22T19:51:28.505562+0000","last_unstale":"2024-04-22T20:47:30.196591+0000","last_undegraded":"2024-04-22T20:47:30.196591+0000","last_fullsized":"2024-04-22T20:47:30.196591+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":540,"num_read_kb":443,"num_write":193,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"261'307","reported_seq":940,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.883624+0000","last_change":"2024-04-22T19:51:26.507324+0000","last_active":"2024-04-22T20:48:17.883624+0000","last_peered":"2024-04-22T20:48:17.883624+0000","last_clean":"2024-04-22T20:48:17.883624+0000","last_became_active":"2024-04-22T19:51:26.504717+0000","last_became_peered":"2024-04-22T19:51:26.504717+0000","last_unstale":"2024-04-22T20:48:17.883624+0000","last_undegraded":"2024-04-22T20:48:17.883624+0000","last_fullsized":"2024-04-22T20:48:17.883624+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10266,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":28705,"num_write":1110,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"261'105","reported_seq":699,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.081746+0000","last_change":"2024-04-22T19:51:28.519503+0000","last_active":"2024-04-22T20:47:30.081746+0000","last_peered":"2024-04-22T20:47:30.081746+0000","last_clean":"2024-04-22T20:47:30.081746+0000","last_became_active":"2024-04-22T19:51:28.519230+0000","last_became_peered":"2024-04-22T19:51:28.519230+0000","last_unstale":"2024-04-22T20:47:30.081746+0000","last_undegraded":"2024-04-22T20:47:30.081746+0000","last_fullsized":"2024-04-22T20:47:30.081746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":465,"num_read_kb":383,"num_write":180,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"261'381","reported_seq":1039,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.881773+0000","last_change":"2024-04-22T19:51:26.496564+0000","last_active":"2024-04-22T20:48:17.881773+0000","last_peered":"2024-04-22T20:48:17.881773+0000","last_clean":"2024-04-22T20:48:17.881773+0000","last_became_active":"2024-04-22T19:51:26.496156+0000","last_became_peered":"2024-04-22T19:51:26.496156+0000","last_unstale":"2024-04-22T20:48:17.881773+0000","last_undegraded":"2024-04-22T20:48:17.881773+0000","last_fullsized":"2024-04-22T20:48:17.881773+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":300,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":611,"num_read_kb":792,"num_write":1516,"num_write_kb":25865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"261'172","reported_seq":765,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:40.207310+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:47:40.207310+0000","last_peered":"2024-04-22T20:47:40.207310+0000","last_clean":"2024-04-22T20:47:40.207310+0000","last_became_active":"2024-04-22T19:51:28.515673+0000","last_became_peered":"2024-04-22T19:51:28.515673+0000","last_unstale":"2024-04-22T20:47:40.207310+0000","last_undegraded":"2024-04-22T20:47:40.207310+0000","last_fullsized":"2024-04-22T20:47:40.207310+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":666,"num_read_kb":529,"num_write":258,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"261'360","reported_seq":1071,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.951096+0000","last_change":"2024-04-22T19:51:26.502589+0000","last_active":"2024-04-22T20:48:17.951096+0000","last_peered":"2024-04-22T20:48:17.951096+0000","last_clean":"2024-04-22T20:48:17.951096+0000","last_became_active":"2024-04-22T19:51:26.502156+0000","last_became_peered":"2024-04-22T19:51:26.502156+0000","last_unstale":"2024-04-22T20:48:17.951096+0000","last_undegraded":"2024-04-22T20:48:17.951096+0000","last_fullsized":"2024-04-22T20:48:17.951096+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1373470,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":619,"num_read_kb":68133,"num_write":1312,"num_write_kb":41095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"261'290","reported_seq":897,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:09.433607+0000","last_change":"2024-04-22T19:51:28.510666+0000","last_active":"2024-04-22T20:47:09.433607+0000","last_peered":"2024-04-22T20:47:09.433607+0000","last_clean":"2024-04-22T20:47:09.433607+0000","last_became_active":"2024-04-22T19:51:28.510415+0000","last_became_peered":"2024-04-22T19:51:28.510415+0000","last_unstale":"2024-04-22T20:47:09.433607+0000","last_undegraded":"2024-04-22T20:47:09.433607+0000","last_fullsized":"2024-04-22T20:47:09.433607+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1119,"num_read_kb":869,"num_write":475,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"261'320","reported_seq":966,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.904116+0000","last_change":"2024-04-22T19:51:26.507272+0000","last_active":"2024-04-22T20:48:17.904116+0000","last_peered":"2024-04-22T20:48:17.904116+0000","last_clean":"2024-04-22T20:48:17.904116+0000","last_became_active":"2024-04-22T19:51:26.506671+0000","last_became_peered":"2024-04-22T19:51:26.506671+0000","last_unstale":"2024-04-22T20:48:17.904116+0000","last_undegraded":"2024-04-22T20:48:17.904116+0000","last_fullsized":"2024-04-22T20:48:17.904116+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194361,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":6535,"num_write":1103,"num_write_kb":36003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"261'155","reported_seq":776,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.076624+0000","last_change":"2024-04-22T19:51:28.512453+0000","last_active":"2024-04-22T20:47:05.076624+0000","last_peered":"2024-04-22T20:47:05.076624+0000","last_clean":"2024-04-22T20:47:05.076624+0000","last_became_active":"2024-04-22T19:51:28.511889+0000","last_became_peered":"2024-04-22T19:51:28.511889+0000","last_unstale":"2024-04-22T20:47:05.076624+0000","last_undegraded":"2024-04-22T20:47:05.076624+0000","last_fullsized":"2024-04-22T20:47:05.076624+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":679,"num_read_kb":547,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"261'334","reported_seq":982,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.944581+0000","last_change":"2024-04-22T19:51:26.498087+0000","last_active":"2024-04-22T20:48:17.944581+0000","last_peered":"2024-04-22T20:48:17.944581+0000","last_clean":"2024-04-22T20:48:17.944581+0000","last_became_active":"2024-04-22T19:51:26.497718+0000","last_became_peered":"2024-04-22T19:51:26.497718+0000","last_unstale":"2024-04-22T20:48:17.944581+0000","last_undegraded":"2024-04-22T20:48:17.944581+0000","last_fullsized":"2024-04-22T20:48:17.944581+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050340,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":471,"num_read_kb":30112,"num_write":1163,"num_write_kb":27210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"261'93","reported_seq":700,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.193499+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:47:05.193499+0000","last_peered":"2024-04-22T20:47:05.193499+0000","last_clean":"2024-04-22T20:47:05.193499+0000","last_became_active":"2024-04-22T19:51:28.507673+0000","last_became_peered":"2024-04-22T19:51:28.507673+0000","last_unstale":"2024-04-22T20:47:05.193499+0000","last_undegraded":"2024-04-22T20:47:05.193499+0000","last_fullsized":"2024-04-22T20:47:05.193499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":404,"num_read_kb":350,"num_write":133,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"261'413","reported_seq":1125,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.927428+0000","last_change":"2024-04-22T19:51:26.511696+0000","last_active":"2024-04-22T20:48:17.927428+0000","last_peered":"2024-04-22T20:48:17.927428+0000","last_clean":"2024-04-22T20:48:17.927428+0000","last_became_active":"2024-04-22T19:51:26.511213+0000","last_became_peered":"2024-04-22T19:51:26.511213+0000","last_unstale":"2024-04-22T20:48:17.927428+0000","last_undegraded":"2024-04-22T20:48:17.927428+0000","last_fullsized":"2024-04-22T20:48:17.927428+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306877,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":730,"num_read_kb":36415,"num_write":1798,"num_write_kb":40524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"261'129","reported_seq":659,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.351652+0000","last_change":"2024-04-22T20:48:29.351716+0000","last_active":"2024-04-22T20:48:29.351652+0000","last_peered":"2024-04-22T20:48:29.351652+0000","last_clean":"2024-04-22T20:48:29.351652+0000","last_became_active":"2024-04-22T19:51:36.567561+0000","last_became_peered":"2024-04-22T19:51:36.567561+0000","last_unstale":"2024-04-22T20:48:29.351652+0000","last_undegraded":"2024-04-22T20:48:29.351652+0000","last_fullsized":"2024-04-22T20:48:29.351652+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'129","last_scrub_stamp":"2024-04-22T20:48:29.351556+0000","last_deep_scrub":"261'129","last_deep_scrub_stamp":"2024-04-22T20:48:29.351556+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.351556+0000","objects_scrubbed":4,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:20:31.079680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":168,"num_write":307,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"261'38","reported_seq":585,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171155+0000","last_change":"2024-04-22T19:51:28.520238+0000","last_active":"2024-04-22T20:47:35.171155+0000","last_peered":"2024-04-22T20:47:35.171155+0000","last_clean":"2024-04-22T20:47:35.171155+0000","last_became_active":"2024-04-22T19:51:28.519994+0000","last_became_peered":"2024-04-22T19:51:28.519994+0000","last_unstale":"2024-04-22T20:47:35.171155+0000","last_undegraded":"2024-04-22T20:47:35.171155+0000","last_fullsized":"2024-04-22T20:47:35.171155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":138,"num_write":52,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"261'317","reported_seq":1059,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.402857+0000","last_change":"2024-04-22T20:48:38.402920+0000","last_active":"2024-04-22T20:48:38.402857+0000","last_peered":"2024-04-22T20:48:38.402857+0000","last_clean":"2024-04-22T20:48:38.402857+0000","last_became_active":"2024-04-22T19:53:05.051272+0000","last_became_peered":"2024-04-22T19:53:05.051272+0000","last_unstale":"2024-04-22T20:48:38.402857+0000","last_undegraded":"2024-04-22T20:48:38.402857+0000","last_fullsized":"2024-04-22T20:48:38.402857+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:38.402757+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:38.402757+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.402757+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:31:08.678074+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263217,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":662,"num_read_kb":11069,"num_write":1270,"num_write_kb":33730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.202034+0000","last_change":"2024-04-22T19:51:30.521121+0000","last_active":"2024-04-22T20:47:05.202034+0000","last_peered":"2024-04-22T20:47:05.202034+0000","last_clean":"2024-04-22T20:47:05.202034+0000","last_became_active":"2024-04-22T19:51:30.520848+0000","last_became_peered":"2024-04-22T19:51:30.520848+0000","last_unstale":"2024-04-22T20:47:05.202034+0000","last_undegraded":"2024-04-22T20:47:05.202034+0000","last_fullsized":"2024-04-22T20:47:05.202034+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"261'329","reported_seq":934,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.477069+0000","last_change":"2024-04-22T19:51:32.537967+0000","last_active":"2024-04-22T20:48:16.477069+0000","last_peered":"2024-04-22T20:48:16.477069+0000","last_clean":"2024-04-22T20:48:16.477069+0000","last_became_active":"2024-04-22T19:51:32.537695+0000","last_became_peered":"2024-04-22T19:51:32.537695+0000","last_unstale":"2024-04-22T20:48:16.477069+0000","last_undegraded":"2024-04-22T20:48:16.477069+0000","last_fullsized":"2024-04-22T20:48:16.477069+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":300,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39872,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":550,"num_read_kb":1178,"num_write":438,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.078772+0000","last_change":"2024-04-22T19:51:34.557621+0000","last_active":"2024-04-22T20:47:10.078772+0000","last_peered":"2024-04-22T20:47:10.078772+0000","last_clean":"2024-04-22T20:47:10.078772+0000","last_became_active":"2024-04-22T19:51:34.557285+0000","last_became_peered":"2024-04-22T19:51:34.557285+0000","last_unstale":"2024-04-22T20:47:10.078772+0000","last_undegraded":"2024-04-22T20:47:10.078772+0000","last_fullsized":"2024-04-22T20:47:10.078772+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"261'2405","reported_seq":4273,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.005951+0000","last_change":"2024-04-22T20:48:34.006025+0000","last_active":"2024-04-22T20:48:34.005951+0000","last_peered":"2024-04-22T20:48:34.005951+0000","last_clean":"2024-04-22T20:48:34.005951+0000","last_became_active":"2024-04-22T19:51:36.567195+0000","last_became_peered":"2024-04-22T19:51:36.567195+0000","last_unstale":"2024-04-22T20:48:34.005951+0000","last_undegraded":"2024-04-22T20:48:34.005951+0000","last_fullsized":"2024-04-22T20:48:34.005951+0000","mapping_epoch":30,"log_start":"261'2300","ondisk_log_start":"261'2300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2405","last_scrub_stamp":"2024-04-22T20:48:34.005850+0000","last_deep_scrub":"261'2405","last_deep_scrub_stamp":"2024-04-22T20:48:34.005850+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.005850+0000","objects_scrubbed":6,"log_size":105,"log_dups_size":2300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:06.221017+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5986,"num_read_kb":5990,"num_write":4781,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"261'130","reported_seq":721,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.202009+0000","last_change":"2024-04-22T19:51:28.513076+0000","last_active":"2024-04-22T20:47:05.202009+0000","last_peered":"2024-04-22T20:47:05.202009+0000","last_clean":"2024-04-22T20:47:05.202009+0000","last_became_active":"2024-04-22T19:51:28.512786+0000","last_became_peered":"2024-04-22T19:51:28.512786+0000","last_unstale":"2024-04-22T20:47:05.202009+0000","last_undegraded":"2024-04-22T20:47:05.202009+0000","last_fullsized":"2024-04-22T20:47:05.202009+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":408,"num_write":191,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"261'290","reported_seq":894,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.948412+0000","last_change":"2024-04-22T19:51:26.500209+0000","last_active":"2024-04-22T20:48:17.948412+0000","last_peered":"2024-04-22T20:48:17.948412+0000","last_clean":"2024-04-22T20:48:17.948412+0000","last_became_active":"2024-04-22T19:51:26.499885+0000","last_became_peered":"2024-04-22T19:51:26.499885+0000","last_unstale":"2024-04-22T20:48:17.948412+0000","last_undegraded":"2024-04-22T20:48:17.948412+0000","last_fullsized":"2024-04-22T20:48:17.948412+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050193,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":16305,"num_write":962,"num_write_kb":23193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171171+0000","last_change":"2024-04-22T19:51:30.527682+0000","last_active":"2024-04-22T20:47:35.171171+0000","last_peered":"2024-04-22T20:47:35.171171+0000","last_clean":"2024-04-22T20:47:35.171171+0000","last_became_active":"2024-04-22T19:51:30.526622+0000","last_became_peered":"2024-04-22T19:51:30.526622+0000","last_unstale":"2024-04-22T20:47:35.171171+0000","last_undegraded":"2024-04-22T20:47:35.171171+0000","last_fullsized":"2024-04-22T20:47:35.171171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"261'89","reported_seq":623,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.323750+0000","last_change":"2024-04-22T20:48:39.323782+0000","last_active":"2024-04-22T20:48:39.323750+0000","last_peered":"2024-04-22T20:48:39.323750+0000","last_clean":"2024-04-22T20:48:39.323750+0000","last_became_active":"2024-04-22T19:51:32.536180+0000","last_became_peered":"2024-04-22T19:51:32.536180+0000","last_unstale":"2024-04-22T20:48:39.323750+0000","last_undegraded":"2024-04-22T20:48:39.323750+0000","last_fullsized":"2024-04-22T20:48:39.323750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'89","last_scrub_stamp":"2024-04-22T20:48:39.323672+0000","last_deep_scrub":"261'89","last_deep_scrub_stamp":"2024-04-22T20:48:39.323672+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.323672+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:12:25.808631+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":95,"num_write":64,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.367492+0000","last_change":"2024-04-22T20:48:34.367530+0000","last_active":"2024-04-22T20:48:34.367492+0000","last_peered":"2024-04-22T20:48:34.367492+0000","last_clean":"2024-04-22T20:48:34.367492+0000","last_became_active":"2024-04-22T19:51:34.549551+0000","last_became_peered":"2024-04-22T19:51:34.549551+0000","last_unstale":"2024-04-22T20:48:34.367492+0000","last_undegraded":"2024-04-22T20:48:34.367492+0000","last_fullsized":"2024-04-22T20:48:34.367492+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.367401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.367401+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.367401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:41.545716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"261'151","reported_seq":659,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.539635+0000","last_change":"2024-04-22T20:48:38.539676+0000","last_active":"2024-04-22T20:48:38.539635+0000","last_peered":"2024-04-22T20:48:38.539635+0000","last_clean":"2024-04-22T20:48:38.539635+0000","last_became_active":"2024-04-22T19:51:36.573189+0000","last_became_peered":"2024-04-22T19:51:36.573189+0000","last_unstale":"2024-04-22T20:48:38.539635+0000","last_undegraded":"2024-04-22T20:48:38.539635+0000","last_fullsized":"2024-04-22T20:48:38.539635+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'151","last_scrub_stamp":"2024-04-22T20:48:38.539539+0000","last_deep_scrub":"261'151","last_deep_scrub_stamp":"2024-04-22T20:48:38.539539+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.539539+0000","objects_scrubbed":5,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:40:24.145792+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":328,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"261'212","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.192825+0000","last_change":"2024-04-22T19:51:28.518334+0000","last_active":"2024-04-22T20:47:05.192825+0000","last_peered":"2024-04-22T20:47:05.192825+0000","last_clean":"2024-04-22T20:47:05.192825+0000","last_became_active":"2024-04-22T19:51:28.517737+0000","last_became_peered":"2024-04-22T19:51:28.517737+0000","last_unstale":"2024-04-22T20:47:05.192825+0000","last_undegraded":"2024-04-22T20:47:05.192825+0000","last_fullsized":"2024-04-22T20:47:05.192825+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":871,"num_read_kb":733,"num_write":338,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"261'298","reported_seq":943,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.944282+0000","last_change":"2024-04-22T19:51:26.509257+0000","last_active":"2024-04-22T20:48:17.944282+0000","last_peered":"2024-04-22T20:48:17.944282+0000","last_clean":"2024-04-22T20:48:17.944282+0000","last_became_active":"2024-04-22T19:51:26.508933+0000","last_became_peered":"2024-04-22T19:51:26.508933+0000","last_unstale":"2024-04-22T20:48:17.944282+0000","last_undegraded":"2024-04-22T20:48:17.944282+0000","last_fullsized":"2024-04-22T20:48:17.944282+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632139,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":513,"num_read_kb":15262,"num_write":1122,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.167813+0000","last_change":"2024-04-22T19:51:30.528152+0000","last_active":"2024-04-22T20:47:05.167813+0000","last_peered":"2024-04-22T20:47:05.167813+0000","last_clean":"2024-04-22T20:47:05.167813+0000","last_became_active":"2024-04-22T19:51:30.527712+0000","last_became_peered":"2024-04-22T19:51:30.527712+0000","last_unstale":"2024-04-22T20:47:05.167813+0000","last_undegraded":"2024-04-22T20:47:05.167813+0000","last_fullsized":"2024-04-22T20:47:05.167813+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"261'118","reported_seq":671,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.458436+0000","last_change":"2024-04-22T20:48:29.458484+0000","last_active":"2024-04-22T20:48:29.458436+0000","last_peered":"2024-04-22T20:48:29.458436+0000","last_clean":"2024-04-22T20:48:29.458436+0000","last_became_active":"2024-04-22T19:51:32.536177+0000","last_became_peered":"2024-04-22T19:51:32.536177+0000","last_unstale":"2024-04-22T20:48:29.458436+0000","last_undegraded":"2024-04-22T20:48:29.458436+0000","last_fullsized":"2024-04-22T20:48:29.458436+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'118","last_scrub_stamp":"2024-04-22T20:48:29.458333+0000","last_deep_scrub":"261'118","last_deep_scrub_stamp":"2024-04-22T20:48:29.458333+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.458333+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:20:42.045716+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"261'523","reported_seq":2084,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.958203+0000","last_change":"2024-04-22T19:51:34.562120+0000","last_active":"2024-04-22T20:48:17.958203+0000","last_peered":"2024-04-22T20:48:17.958203+0000","last_clean":"2024-04-22T20:48:17.958203+0000","last_became_active":"2024-04-22T19:51:34.561828+0000","last_became_peered":"2024-04-22T19:51:34.561828+0000","last_unstale":"2024-04-22T20:48:17.958203+0000","last_undegraded":"2024-04-22T20:48:17.958203+0000","last_fullsized":"2024-04-22T20:48:17.958203+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":500,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"261'176","reported_seq":997,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.374030+0000","last_change":"2024-04-22T20:48:30.374062+0000","last_active":"2024-04-22T20:48:30.374030+0000","last_peered":"2024-04-22T20:48:30.374030+0000","last_clean":"2024-04-22T20:48:30.374030+0000","last_became_active":"2024-04-22T19:51:36.565673+0000","last_became_peered":"2024-04-22T19:51:36.565673+0000","last_unstale":"2024-04-22T20:48:30.374030+0000","last_undegraded":"2024-04-22T20:48:30.374030+0000","last_fullsized":"2024-04-22T20:48:30.374030+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'176","last_scrub_stamp":"2024-04-22T20:48:30.373943+0000","last_deep_scrub":"261'176","last_deep_scrub_stamp":"2024-04-22T20:48:30.373943+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.373943+0000","objects_scrubbed":2,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:19.241756+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":579,"num_read_kb":567,"num_write":371,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"261'176","reported_seq":815,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.167801+0000","last_change":"2024-04-22T19:51:28.516658+0000","last_active":"2024-04-22T20:47:05.167801+0000","last_peered":"2024-04-22T20:47:05.167801+0000","last_clean":"2024-04-22T20:47:05.167801+0000","last_became_active":"2024-04-22T19:51:28.515670+0000","last_became_peered":"2024-04-22T19:51:28.515670+0000","last_unstale":"2024-04-22T20:47:05.167801+0000","last_undegraded":"2024-04-22T20:47:05.167801+0000","last_fullsized":"2024-04-22T20:47:05.167801+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":729,"num_read_kb":590,"num_write":271,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"261'317","reported_seq":996,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.935184+0000","last_change":"2024-04-22T19:51:26.503775+0000","last_active":"2024-04-22T20:48:17.935184+0000","last_peered":"2024-04-22T20:48:17.935184+0000","last_clean":"2024-04-22T20:48:17.935184+0000","last_became_active":"2024-04-22T19:51:26.503438+0000","last_became_peered":"2024-04-22T19:51:26.503438+0000","last_unstale":"2024-04-22T20:48:17.935184+0000","last_undegraded":"2024-04-22T20:48:17.935184+0000","last_fullsized":"2024-04-22T20:48:17.935184+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3601437,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":581,"num_read_kb":32833,"num_write":1303,"num_write_kb":25399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.202096+0000","last_change":"2024-04-22T19:51:30.523415+0000","last_active":"2024-04-22T20:47:05.202096+0000","last_peered":"2024-04-22T20:47:05.202096+0000","last_clean":"2024-04-22T20:47:05.202096+0000","last_became_active":"2024-04-22T19:51:30.523125+0000","last_became_peered":"2024-04-22T19:51:30.523125+0000","last_unstale":"2024-04-22T20:47:05.202096+0000","last_undegraded":"2024-04-22T20:47:05.202096+0000","last_fullsized":"2024-04-22T20:47:05.202096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"261'252","reported_seq":823,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.114246+0000","last_change":"2024-04-22T19:51:33.248486+0000","last_active":"2024-04-22T20:48:16.114246+0000","last_peered":"2024-04-22T20:48:16.114246+0000","last_clean":"2024-04-22T20:48:16.114246+0000","last_became_active":"2024-04-22T19:51:33.248160+0000","last_became_peered":"2024-04-22T19:51:33.248160+0000","last_unstale":"2024-04-22T20:48:16.114246+0000","last_undegraded":"2024-04-22T20:48:16.114246+0000","last_fullsized":"2024-04-22T20:48:16.114246+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":425,"num_read_kb":974,"num_write":344,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"261'536","reported_seq":4230,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.961901+0000","last_change":"2024-04-22T19:51:34.556255+0000","last_active":"2024-04-22T20:48:17.961901+0000","last_peered":"2024-04-22T20:48:17.961901+0000","last_clean":"2024-04-22T20:48:17.961901+0000","last_became_active":"2024-04-22T19:51:34.555720+0000","last_became_peered":"2024-04-22T19:51:34.555720+0000","last_unstale":"2024-04-22T20:48:17.961901+0000","last_undegraded":"2024-04-22T20:48:17.961901+0000","last_fullsized":"2024-04-22T20:48:17.961901+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"261'385","reported_seq":1058,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.208839+0000","last_change":"2024-04-22T19:51:28.513088+0000","last_active":"2024-04-22T20:47:50.208839+0000","last_peered":"2024-04-22T20:47:50.208839+0000","last_clean":"2024-04-22T20:47:50.208839+0000","last_became_active":"2024-04-22T19:51:28.512125+0000","last_became_peered":"2024-04-22T19:51:28.512125+0000","last_unstale":"2024-04-22T20:47:50.208839+0000","last_undegraded":"2024-04-22T20:47:50.208839+0000","last_fullsized":"2024-04-22T20:47:50.208839+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1728,"num_read_kb":1430,"num_write":663,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"261'305","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.932529+0000","last_change":"2024-04-22T19:51:26.508318+0000","last_active":"2024-04-22T20:48:17.932529+0000","last_peered":"2024-04-22T20:48:17.932529+0000","last_clean":"2024-04-22T20:48:17.932529+0000","last_became_active":"2024-04-22T19:51:26.504654+0000","last_became_peered":"2024-04-22T19:51:26.504654+0000","last_unstale":"2024-04-22T20:48:17.932529+0000","last_undegraded":"2024-04-22T20:48:17.932529+0000","last_fullsized":"2024-04-22T20:48:17.932529+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10487408,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":445,"num_read_kb":80529,"num_write":1013,"num_write_kb":26179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.084840+0000","last_change":"2024-04-22T19:51:30.526634+0000","last_active":"2024-04-22T20:47:50.084840+0000","last_peered":"2024-04-22T20:47:50.084840+0000","last_clean":"2024-04-22T20:47:50.084840+0000","last_became_active":"2024-04-22T19:51:30.526306+0000","last_became_peered":"2024-04-22T19:51:30.526306+0000","last_unstale":"2024-04-22T20:47:50.084840+0000","last_undegraded":"2024-04-22T20:47:50.084840+0000","last_fullsized":"2024-04-22T20:47:50.084840+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"261'342","reported_seq":966,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.959517+0000","last_change":"2024-04-22T20:48:38.959567+0000","last_active":"2024-04-22T20:48:38.959517+0000","last_peered":"2024-04-22T20:48:38.959517+0000","last_clean":"2024-04-22T20:48:38.959517+0000","last_became_active":"2024-04-22T19:51:33.248059+0000","last_became_peered":"2024-04-22T19:51:33.248059+0000","last_unstale":"2024-04-22T20:48:38.959517+0000","last_undegraded":"2024-04-22T20:48:38.959517+0000","last_fullsized":"2024-04-22T20:48:38.959517+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'342","last_scrub_stamp":"2024-04-22T20:48:38.959416+0000","last_deep_scrub":"261'342","last_deep_scrub_stamp":"2024-04-22T20:48:38.959416+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.959416+0000","objects_scrubbed":9,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:10:05.470719+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45321,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":537,"num_read_kb":1022,"num_write":408,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.060168+0000","last_change":"2024-04-22T19:51:34.557926+0000","last_active":"2024-04-22T20:47:50.060168+0000","last_peered":"2024-04-22T20:47:50.060168+0000","last_clean":"2024-04-22T20:47:50.060168+0000","last_became_active":"2024-04-22T19:51:34.557662+0000","last_became_peered":"2024-04-22T19:51:34.557662+0000","last_unstale":"2024-04-22T20:47:50.060168+0000","last_undegraded":"2024-04-22T20:47:50.060168+0000","last_fullsized":"2024-04-22T20:47:50.060168+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"261'566","reported_seq":1383,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.924663+0000","last_change":"2024-04-22T20:48:30.924698+0000","last_active":"2024-04-22T20:48:30.924663+0000","last_peered":"2024-04-22T20:48:30.924663+0000","last_clean":"2024-04-22T20:48:30.924663+0000","last_became_active":"2024-04-22T19:51:36.570520+0000","last_became_peered":"2024-04-22T19:51:36.570520+0000","last_unstale":"2024-04-22T20:48:30.924663+0000","last_undegraded":"2024-04-22T20:48:30.924663+0000","last_fullsized":"2024-04-22T20:48:30.924663+0000","mapping_epoch":30,"log_start":"261'500","ondisk_log_start":"261'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'566","last_scrub_stamp":"2024-04-22T20:48:30.924566+0000","last_deep_scrub":"261'566","last_deep_scrub_stamp":"2024-04-22T20:48:30.924566+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.924566+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":500,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:46:13.753614+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1283,"num_read_kb":1280,"num_write":1166,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"261'311","reported_seq":960,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.199155+0000","last_change":"2024-04-22T19:51:28.512328+0000","last_active":"2024-04-22T20:47:50.199155+0000","last_peered":"2024-04-22T20:47:50.199155+0000","last_clean":"2024-04-22T20:47:50.199155+0000","last_became_active":"2024-04-22T19:51:28.511527+0000","last_became_peered":"2024-04-22T19:51:28.511527+0000","last_unstale":"2024-04-22T20:47:50.199155+0000","last_undegraded":"2024-04-22T20:47:50.199155+0000","last_fullsized":"2024-04-22T20:47:50.199155+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1221,"num_read_kb":977,"num_write":455,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"261'330","reported_seq":972,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.945450+0000","last_change":"2024-04-22T19:51:26.505351+0000","last_active":"2024-04-22T20:48:17.945450+0000","last_peered":"2024-04-22T20:48:17.945450+0000","last_clean":"2024-04-22T20:48:17.945450+0000","last_became_active":"2024-04-22T19:51:26.504472+0000","last_became_peered":"2024-04-22T19:51:26.504472+0000","last_unstale":"2024-04-22T20:48:17.945450+0000","last_undegraded":"2024-04-22T20:48:17.945450+0000","last_fullsized":"2024-04-22T20:48:17.945450+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158015,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":483,"num_read_kb":33343,"num_write":1137,"num_write_kb":49829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"261'6","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.511865+0000","last_change":"2024-04-22T20:48:37.511895+0000","last_active":"2024-04-22T20:48:37.511865+0000","last_peered":"2024-04-22T20:48:37.511865+0000","last_clean":"2024-04-22T20:48:37.511865+0000","last_became_active":"2024-04-22T19:51:30.524043+0000","last_became_peered":"2024-04-22T19:51:30.524043+0000","last_unstale":"2024-04-22T20:48:37.511865+0000","last_undegraded":"2024-04-22T20:48:37.511865+0000","last_fullsized":"2024-04-22T20:48:37.511865+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:37.511762+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:37.511762+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.511762+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:15.538566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":492,"num_read_kb":799,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"261'266","reported_seq":846,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.788252+0000","last_change":"2024-04-22T19:51:32.539254+0000","last_active":"2024-04-22T20:48:17.788252+0000","last_peered":"2024-04-22T20:48:17.788252+0000","last_clean":"2024-04-22T20:48:17.788252+0000","last_became_active":"2024-04-22T19:51:32.537968+0000","last_became_peered":"2024-04-22T19:51:32.537968+0000","last_unstale":"2024-04-22T20:48:17.788252+0000","last_undegraded":"2024-04-22T20:48:17.788252+0000","last_fullsized":"2024-04-22T20:48:17.788252+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43364,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":998,"num_write":350,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.174096+0000","last_change":"2024-04-22T19:51:34.556181+0000","last_active":"2024-04-22T20:48:00.174096+0000","last_peered":"2024-04-22T20:48:00.174096+0000","last_clean":"2024-04-22T20:48:00.174096+0000","last_became_active":"2024-04-22T19:51:34.555889+0000","last_became_peered":"2024-04-22T19:51:34.555889+0000","last_unstale":"2024-04-22T20:48:00.174096+0000","last_undegraded":"2024-04-22T20:48:00.174096+0000","last_fullsized":"2024-04-22T20:48:00.174096+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"261'80","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.463200+0000","last_change":"2024-04-22T20:48:31.463234+0000","last_active":"2024-04-22T20:48:31.463200+0000","last_peered":"2024-04-22T20:48:31.463200+0000","last_clean":"2024-04-22T20:48:31.463200+0000","last_became_active":"2024-04-22T19:51:36.574555+0000","last_became_peered":"2024-04-22T19:51:36.574555+0000","last_unstale":"2024-04-22T20:48:31.463200+0000","last_undegraded":"2024-04-22T20:48:31.463200+0000","last_fullsized":"2024-04-22T20:48:31.463200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'80","last_scrub_stamp":"2024-04-22T20:48:31.463110+0000","last_deep_scrub":"261'80","last_deep_scrub_stamp":"2024-04-22T20:48:31.463110+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.463110+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:09:05.164724+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":91,"num_write":194,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"261'189","reported_seq":793,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.643992+0000","last_change":"2024-04-22T20:48:39.644036+0000","last_active":"2024-04-22T20:48:39.643992+0000","last_peered":"2024-04-22T20:48:39.643992+0000","last_clean":"2024-04-22T20:48:39.643992+0000","last_became_active":"2024-04-22T19:51:28.519424+0000","last_became_peered":"2024-04-22T19:51:28.519424+0000","last_unstale":"2024-04-22T20:48:39.643992+0000","last_undegraded":"2024-04-22T20:48:39.643992+0000","last_fullsized":"2024-04-22T20:48:39.643992+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'189","last_scrub_stamp":"2024-04-22T20:48:39.643857+0000","last_deep_scrub":"261'189","last_deep_scrub_stamp":"2024-04-22T20:48:39.643857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.643857+0000","objects_scrubbed":1,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:30:41.321492+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":758,"num_read_kb":601,"num_write":297,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"261'289","reported_seq":912,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.926908+0000","last_change":"2024-04-22T19:51:26.511055+0000","last_active":"2024-04-22T20:48:17.926908+0000","last_peered":"2024-04-22T20:48:17.926908+0000","last_clean":"2024-04-22T20:48:17.926908+0000","last_became_active":"2024-04-22T19:51:26.510273+0000","last_became_peered":"2024-04-22T19:51:26.510273+0000","last_unstale":"2024-04-22T20:48:17.926908+0000","last_undegraded":"2024-04-22T20:48:17.926908+0000","last_fullsized":"2024-04-22T20:48:17.926908+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2115,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":380,"num_read_kb":65007,"num_write":1004,"num_write_kb":29343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.472320+0000","last_change":"2024-04-22T20:48:35.472348+0000","last_active":"2024-04-22T20:48:35.472320+0000","last_peered":"2024-04-22T20:48:35.472320+0000","last_clean":"2024-04-22T20:48:35.472320+0000","last_became_active":"2024-04-22T19:51:30.525018+0000","last_became_peered":"2024-04-22T19:51:30.525018+0000","last_unstale":"2024-04-22T20:48:35.472320+0000","last_undegraded":"2024-04-22T20:48:35.472320+0000","last_fullsized":"2024-04-22T20:48:35.472320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:35.472216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:35.472216+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.472216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:21.173339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"261'309","reported_seq":903,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.944979+0000","last_change":"2024-04-22T19:51:32.538989+0000","last_active":"2024-04-22T20:48:16.944979+0000","last_peered":"2024-04-22T20:48:16.944979+0000","last_clean":"2024-04-22T20:48:16.944979+0000","last_became_active":"2024-04-22T19:51:32.538656+0000","last_became_peered":"2024-04-22T19:51:32.538656+0000","last_unstale":"2024-04-22T20:48:16.944979+0000","last_undegraded":"2024-04-22T20:48:16.944979+0000","last_fullsized":"2024-04-22T20:48:16.944979+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47724,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":498,"num_read_kb":1086,"num_write":396,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"261'509","reported_seq":2199,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.962784+0000","last_change":"2024-04-22T19:51:34.561771+0000","last_active":"2024-04-22T20:48:17.962784+0000","last_peered":"2024-04-22T20:48:17.962784+0000","last_clean":"2024-04-22T20:48:17.962784+0000","last_became_active":"2024-04-22T19:51:34.561488+0000","last_became_peered":"2024-04-22T19:51:34.561488+0000","last_unstale":"2024-04-22T20:48:17.962784+0000","last_undegraded":"2024-04-22T20:48:17.962784+0000","last_fullsized":"2024-04-22T20:48:17.962784+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"261'136","reported_seq":634,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.548887+0000","last_change":"2024-04-22T20:48:39.548938+0000","last_active":"2024-04-22T20:48:39.548887+0000","last_peered":"2024-04-22T20:48:39.548887+0000","last_clean":"2024-04-22T20:48:39.548887+0000","last_became_active":"2024-04-22T19:51:36.573167+0000","last_became_peered":"2024-04-22T19:51:36.573167+0000","last_unstale":"2024-04-22T20:48:39.548887+0000","last_undegraded":"2024-04-22T20:48:39.548887+0000","last_fullsized":"2024-04-22T20:48:39.548887+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'136","last_scrub_stamp":"2024-04-22T20:48:39.548772+0000","last_deep_scrub":"261'136","last_deep_scrub_stamp":"2024-04-22T20:48:39.548772+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.548772+0000","objects_scrubbed":2,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:36:24.248525+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"261'150","reported_seq":716,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.197482+0000","last_change":"2024-04-22T19:51:28.518692+0000","last_active":"2024-04-22T20:47:45.197482+0000","last_peered":"2024-04-22T20:47:45.197482+0000","last_clean":"2024-04-22T20:47:45.197482+0000","last_became_active":"2024-04-22T19:51:28.518398+0000","last_became_peered":"2024-04-22T19:51:28.518398+0000","last_unstale":"2024-04-22T20:47:45.197482+0000","last_undegraded":"2024-04-22T20:47:45.197482+0000","last_fullsized":"2024-04-22T20:47:45.197482+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":596,"num_read_kb":465,"num_write":243,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"261'313","reported_seq":991,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.857532+0000","last_change":"2024-04-22T20:48:24.857599+0000","last_active":"2024-04-22T20:48:24.857532+0000","last_peered":"2024-04-22T20:48:24.857532+0000","last_clean":"2024-04-22T20:48:24.857532+0000","last_became_active":"2024-04-22T19:52:05.112910+0000","last_became_peered":"2024-04-22T19:52:05.112910+0000","last_unstale":"2024-04-22T20:48:24.857532+0000","last_undegraded":"2024-04-22T20:48:24.857532+0000","last_fullsized":"2024-04-22T20:48:24.857532+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:24.857439+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:24.857439+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.857439+0000","objects_scrubbed":12,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:24:03.346339+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":486619,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":517,"num_read_kb":39016,"num_write":1092,"num_write_kb":34073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":513,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.406179+0000","last_change":"2024-04-22T20:48:31.406203+0000","last_active":"2024-04-22T20:48:31.406179+0000","last_peered":"2024-04-22T20:48:31.406179+0000","last_clean":"2024-04-22T20:48:31.406179+0000","last_became_active":"2024-04-22T19:51:30.529011+0000","last_became_peered":"2024-04-22T19:51:30.529011+0000","last_unstale":"2024-04-22T20:48:31.406179+0000","last_undegraded":"2024-04-22T20:48:31.406179+0000","last_fullsized":"2024-04-22T20:48:31.406179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:31.406079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:31.406079+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.406079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:00:54.573347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"261'488","reported_seq":1123,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.953471+0000","last_change":"2024-04-22T19:51:32.537571+0000","last_active":"2024-04-22T20:48:17.953471+0000","last_peered":"2024-04-22T20:48:17.953471+0000","last_clean":"2024-04-22T20:48:17.953471+0000","last_became_active":"2024-04-22T19:51:32.537267+0000","last_became_peered":"2024-04-22T19:51:32.537267+0000","last_unstale":"2024-04-22T20:48:17.953471+0000","last_undegraded":"2024-04-22T20:48:17.953471+0000","last_fullsized":"2024-04-22T20:48:17.953471+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":400,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":856,"num_read_kb":2097,"num_write":704,"num_write_kb":611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.377549+0000","last_change":"2024-04-22T20:48:25.377584+0000","last_active":"2024-04-22T20:48:25.377549+0000","last_peered":"2024-04-22T20:48:25.377549+0000","last_clean":"2024-04-22T20:48:25.377549+0000","last_became_active":"2024-04-22T19:51:34.552446+0000","last_became_peered":"2024-04-22T19:51:34.552446+0000","last_unstale":"2024-04-22T20:48:25.377549+0000","last_undegraded":"2024-04-22T20:48:25.377549+0000","last_fullsized":"2024-04-22T20:48:25.377549+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:25.377456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:25.377456+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.377456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:06:04.391078+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"261'106","reported_seq":609,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.061533+0000","last_change":"2024-04-22T19:51:36.570982+0000","last_active":"2024-04-22T20:48:00.061533+0000","last_peered":"2024-04-22T20:48:00.061533+0000","last_clean":"2024-04-22T20:48:00.061533+0000","last_became_active":"2024-04-22T19:51:36.570666+0000","last_became_peered":"2024-04-22T19:51:36.570666+0000","last_unstale":"2024-04-22T20:48:00.061533+0000","last_undegraded":"2024-04-22T20:48:00.061533+0000","last_fullsized":"2024-04-22T20:48:00.061533+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":242,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"261'168","reported_seq":796,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198538+0000","last_change":"2024-04-22T19:51:28.508639+0000","last_active":"2024-04-22T20:47:45.198538+0000","last_peered":"2024-04-22T20:47:45.198538+0000","last_clean":"2024-04-22T20:47:45.198538+0000","last_became_active":"2024-04-22T19:51:28.508288+0000","last_became_peered":"2024-04-22T19:51:28.508288+0000","last_unstale":"2024-04-22T20:47:45.198538+0000","last_undegraded":"2024-04-22T20:47:45.198538+0000","last_fullsized":"2024-04-22T20:47:45.198538+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":576,"num_write":244,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"261'317","reported_seq":932,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.935157+0000","last_change":"2024-04-22T19:51:26.506251+0000","last_active":"2024-04-22T20:48:17.935157+0000","last_peered":"2024-04-22T20:48:17.935157+0000","last_clean":"2024-04-22T20:48:17.935157+0000","last_became_active":"2024-04-22T19:51:26.503669+0000","last_became_peered":"2024-04-22T19:51:26.503669+0000","last_unstale":"2024-04-22T20:48:17.935157+0000","last_undegraded":"2024-04-22T20:48:17.935157+0000","last_fullsized":"2024-04-22T20:48:17.935157+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194320,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":456,"num_read_kb":4498,"num_write":1097,"num_write_kb":33252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.451692+0000","last_change":"2024-04-22T20:48:33.451746+0000","last_active":"2024-04-22T20:48:33.451692+0000","last_peered":"2024-04-22T20:48:33.451692+0000","last_clean":"2024-04-22T20:48:33.451692+0000","last_became_active":"2024-04-22T19:51:30.526420+0000","last_became_peered":"2024-04-22T19:51:30.526420+0000","last_unstale":"2024-04-22T20:48:33.451692+0000","last_undegraded":"2024-04-22T20:48:33.451692+0000","last_fullsized":"2024-04-22T20:48:33.451692+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.451567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.451567+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.451567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:35.930208+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"261'274","reported_seq":862,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.374111+0000","last_change":"2024-04-22T20:48:38.374154+0000","last_active":"2024-04-22T20:48:38.374111+0000","last_peered":"2024-04-22T20:48:38.374111+0000","last_clean":"2024-04-22T20:48:38.374111+0000","last_became_active":"2024-04-22T19:51:32.536463+0000","last_became_peered":"2024-04-22T19:51:32.536463+0000","last_unstale":"2024-04-22T20:48:38.374111+0000","last_undegraded":"2024-04-22T20:48:38.374111+0000","last_fullsized":"2024-04-22T20:48:38.374111+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'274","last_scrub_stamp":"2024-04-22T20:48:38.374022+0000","last_deep_scrub":"261'274","last_deep_scrub_stamp":"2024-04-22T20:48:38.374022+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.374022+0000","objects_scrubbed":6,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:48:58.818076+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42355,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":457,"num_read_kb":963,"num_write":402,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":490,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.199264+0000","last_change":"2024-04-22T19:51:34.555066+0000","last_active":"2024-04-22T20:48:00.199264+0000","last_peered":"2024-04-22T20:48:00.199264+0000","last_clean":"2024-04-22T20:48:00.199264+0000","last_became_active":"2024-04-22T19:51:34.554737+0000","last_became_peered":"2024-04-22T19:51:34.554737+0000","last_unstale":"2024-04-22T20:48:00.199264+0000","last_undegraded":"2024-04-22T20:48:00.199264+0000","last_fullsized":"2024-04-22T20:48:00.199264+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"261'96","reported_seq":602,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.589774+0000","last_change":"2024-04-22T20:48:31.589814+0000","last_active":"2024-04-22T20:48:31.589774+0000","last_peered":"2024-04-22T20:48:31.589774+0000","last_clean":"2024-04-22T20:48:31.589774+0000","last_became_active":"2024-04-22T19:51:36.571086+0000","last_became_peered":"2024-04-22T19:51:36.571086+0000","last_unstale":"2024-04-22T20:48:31.589774+0000","last_undegraded":"2024-04-22T20:48:31.589774+0000","last_fullsized":"2024-04-22T20:48:31.589774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'96","last_scrub_stamp":"2024-04-22T20:48:31.589672+0000","last_deep_scrub":"261'96","last_deep_scrub_stamp":"2024-04-22T20:48:31.589672+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.589672+0000","objects_scrubbed":3,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:50.688646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":72,"num_write":227,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"261'253","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198566+0000","last_change":"2024-04-22T19:51:28.514713+0000","last_active":"2024-04-22T20:47:45.198566+0000","last_peered":"2024-04-22T20:47:45.198566+0000","last_clean":"2024-04-22T20:47:45.198566+0000","last_became_active":"2024-04-22T19:51:28.514445+0000","last_became_peered":"2024-04-22T19:51:28.514445+0000","last_unstale":"2024-04-22T20:47:45.198566+0000","last_undegraded":"2024-04-22T20:47:45.198566+0000","last_fullsized":"2024-04-22T20:47:45.198566+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":723,"num_write":394,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'395","reported_seq":1142,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.944457+0000","last_change":"2024-04-22T19:51:26.496789+0000","last_active":"2024-04-22T20:48:17.944457+0000","last_peered":"2024-04-22T20:48:17.944457+0000","last_clean":"2024-04-22T20:48:17.944457+0000","last_became_active":"2024-04-22T19:51:26.496408+0000","last_became_peered":"2024-04-22T19:51:26.496408+0000","last_unstale":"2024-04-22T20:48:17.944457+0000","last_undegraded":"2024-04-22T20:48:17.944457+0000","last_fullsized":"2024-04-22T20:48:17.944457+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":840,"num_read_kb":8023,"num_write":1479,"num_write_kb":27015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.499734+0000","last_change":"2024-04-22T20:48:33.499756+0000","last_active":"2024-04-22T20:48:33.499734+0000","last_peered":"2024-04-22T20:48:33.499734+0000","last_clean":"2024-04-22T20:48:33.499734+0000","last_became_active":"2024-04-22T19:51:30.527309+0000","last_became_peered":"2024-04-22T19:51:30.527309+0000","last_unstale":"2024-04-22T20:48:33.499734+0000","last_undegraded":"2024-04-22T20:48:33.499734+0000","last_fullsized":"2024-04-22T20:48:33.499734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.499634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.499634+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.499634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:37:13.088297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"261'314","reported_seq":919,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.436156+0000","last_change":"2024-04-22T20:48:27.436199+0000","last_active":"2024-04-22T20:48:27.436156+0000","last_peered":"2024-04-22T20:48:27.436156+0000","last_clean":"2024-04-22T20:48:27.436156+0000","last_became_active":"2024-04-22T19:51:32.539732+0000","last_became_peered":"2024-04-22T19:51:32.539732+0000","last_unstale":"2024-04-22T20:48:27.436156+0000","last_undegraded":"2024-04-22T20:48:27.436156+0000","last_fullsized":"2024-04-22T20:48:27.436156+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'314","last_scrub_stamp":"2024-04-22T20:48:27.436075+0000","last_deep_scrub":"261'314","last_deep_scrub_stamp":"2024-04-22T20:48:27.436075+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.436075+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:26:29.043824+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52364,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":512,"num_read_kb":1133,"num_write":408,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"261'490","reported_seq":2212,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.422541+0000","last_change":"2024-04-22T20:48:23.422593+0000","last_active":"2024-04-22T20:48:23.422541+0000","last_peered":"2024-04-22T20:48:23.422541+0000","last_clean":"2024-04-22T20:48:23.422541+0000","last_became_active":"2024-04-22T19:51:34.556124+0000","last_became_peered":"2024-04-22T19:51:34.556124+0000","last_unstale":"2024-04-22T20:48:23.422541+0000","last_undegraded":"2024-04-22T20:48:23.422541+0000","last_fullsized":"2024-04-22T20:48:23.422541+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'490","last_scrub_stamp":"2024-04-22T20:48:23.422457+0000","last_deep_scrub":"261'490","last_deep_scrub_stamp":"2024-04-22T20:48:23.422457+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.422457+0000","objects_scrubbed":1,"log_size":90,"log_dups_size":400,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:12.183058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"261'120","reported_seq":621,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.517134+0000","last_change":"2024-04-22T20:48:30.517192+0000","last_active":"2024-04-22T20:48:30.517134+0000","last_peered":"2024-04-22T20:48:30.517134+0000","last_clean":"2024-04-22T20:48:30.517134+0000","last_became_active":"2024-04-22T19:51:36.572913+0000","last_became_peered":"2024-04-22T19:51:36.572913+0000","last_unstale":"2024-04-22T20:48:30.517134+0000","last_undegraded":"2024-04-22T20:48:30.517134+0000","last_fullsized":"2024-04-22T20:48:30.517134+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'120","last_scrub_stamp":"2024-04-22T20:48:30.517024+0000","last_deep_scrub":"261'120","last_deep_scrub_stamp":"2024-04-22T20:48:30.517024+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.517024+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:42.774241+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":102,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"261'338","reported_seq":1081,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198592+0000","last_change":"2024-04-22T19:51:28.516099+0000","last_active":"2024-04-22T20:47:45.198592+0000","last_peered":"2024-04-22T20:47:45.198592+0000","last_clean":"2024-04-22T20:47:45.198592+0000","last_became_active":"2024-04-22T19:51:28.515809+0000","last_became_peered":"2024-04-22T19:51:28.515809+0000","last_unstale":"2024-04-22T20:47:45.198592+0000","last_undegraded":"2024-04-22T20:47:45.198592+0000","last_fullsized":"2024-04-22T20:47:45.198592+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1504,"num_read_kb":1227,"num_write":616,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"261'336","reported_seq":972,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.945307+0000","last_change":"2024-04-22T19:51:26.499730+0000","last_active":"2024-04-22T20:48:17.945307+0000","last_peered":"2024-04-22T20:48:17.945307+0000","last_clean":"2024-04-22T20:48:17.945307+0000","last_became_active":"2024-04-22T19:51:26.499362+0000","last_became_peered":"2024-04-22T19:51:26.499362+0000","last_unstale":"2024-04-22T20:48:17.945307+0000","last_undegraded":"2024-04-22T20:48:17.945307+0000","last_fullsized":"2024-04-22T20:48:17.945307+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":26712,"num_write":1252,"num_write_kb":36357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"25'2","reported_seq":1136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.534226+0000","last_change":"2024-04-22T19:51:30.527770+0000","last_active":"2024-04-22T20:48:15.534226+0000","last_peered":"2024-04-22T20:48:15.534226+0000","last_clean":"2024-04-22T20:48:15.534226+0000","last_became_active":"2024-04-22T19:51:30.527490+0000","last_became_peered":"2024-04-22T19:51:30.527490+0000","last_unstale":"2024-04-22T20:48:15.534226+0000","last_undegraded":"2024-04-22T20:48:15.534226+0000","last_fullsized":"2024-04-22T20:48:15.534226+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":640,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"261'90","reported_seq":621,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.173029+0000","last_change":"2024-04-22T19:51:33.248919+0000","last_active":"2024-04-22T20:47:50.173029+0000","last_peered":"2024-04-22T20:47:50.173029+0000","last_clean":"2024-04-22T20:47:50.173029+0000","last_became_active":"2024-04-22T19:51:33.248349+0000","last_became_peered":"2024-04-22T19:51:33.248349+0000","last_unstale":"2024-04-22T20:47:50.173029+0000","last_undegraded":"2024-04-22T20:47:50.173029+0000","last_fullsized":"2024-04-22T20:47:50.173029+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":60,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"261'518","reported_seq":2073,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.959983+0000","last_change":"2024-04-22T19:51:34.562158+0000","last_active":"2024-04-22T20:48:17.959983+0000","last_peered":"2024-04-22T20:48:17.959983+0000","last_clean":"2024-04-22T20:48:17.959983+0000","last_became_active":"2024-04-22T19:51:34.561825+0000","last_became_peered":"2024-04-22T19:51:34.561825+0000","last_unstale":"2024-04-22T20:48:17.959983+0000","last_undegraded":"2024-04-22T20:48:17.959983+0000","last_fullsized":"2024-04-22T20:48:17.959983+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"261'131","reported_seq":635,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.325984+0000","last_change":"2024-04-22T20:48:27.326015+0000","last_active":"2024-04-22T20:48:27.325984+0000","last_peered":"2024-04-22T20:48:27.325984+0000","last_clean":"2024-04-22T20:48:27.325984+0000","last_became_active":"2024-04-22T19:51:36.573026+0000","last_became_peered":"2024-04-22T19:51:36.573026+0000","last_unstale":"2024-04-22T20:48:27.325984+0000","last_undegraded":"2024-04-22T20:48:27.325984+0000","last_fullsized":"2024-04-22T20:48:27.325984+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:27.325897+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:27.325897+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.325897+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:17.870401+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":119,"num_write":309,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"261'207","reported_seq":819,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.060883+0000","last_change":"2024-04-22T19:51:28.512959+0000","last_active":"2024-04-22T20:47:55.060883+0000","last_peered":"2024-04-22T20:47:55.060883+0000","last_clean":"2024-04-22T20:47:55.060883+0000","last_became_active":"2024-04-22T19:51:28.512275+0000","last_became_peered":"2024-04-22T19:51:28.512275+0000","last_unstale":"2024-04-22T20:47:55.060883+0000","last_undegraded":"2024-04-22T20:47:55.060883+0000","last_fullsized":"2024-04-22T20:47:55.060883+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":832,"num_read_kb":677,"num_write":302,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"261'328","reported_seq":967,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.890163+0000","last_change":"2024-04-22T19:51:26.498630+0000","last_active":"2024-04-22T20:48:17.890163+0000","last_peered":"2024-04-22T20:48:17.890163+0000","last_clean":"2024-04-22T20:48:17.890163+0000","last_became_active":"2024-04-22T19:51:26.498276+0000","last_became_peered":"2024-04-22T19:51:26.498276+0000","last_unstale":"2024-04-22T20:48:17.890163+0000","last_undegraded":"2024-04-22T20:48:17.890163+0000","last_fullsized":"2024-04-22T20:48:17.890163+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1581644,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":468,"num_read_kb":30145,"num_write":1162,"num_write_kb":23135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"32'3","reported_seq":822,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.534600+0000","last_change":"2024-04-22T19:51:30.530726+0000","last_active":"2024-04-22T20:48:15.534600+0000","last_peered":"2024-04-22T20:48:15.534600+0000","last_clean":"2024-04-22T20:48:15.534600+0000","last_became_active":"2024-04-22T19:51:30.530363+0000","last_became_peered":"2024-04-22T19:51:30.530363+0000","last_unstale":"2024-04-22T20:48:15.534600+0000","last_undegraded":"2024-04-22T20:48:15.534600+0000","last_fullsized":"2024-04-22T20:48:15.534600+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":490,"num_read_kb":488,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"261'401","reported_seq":1010,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.559078+0000","last_change":"2024-04-22T19:51:32.540935+0000","last_active":"2024-04-22T20:48:16.559078+0000","last_peered":"2024-04-22T20:48:16.559078+0000","last_clean":"2024-04-22T20:48:16.559078+0000","last_became_active":"2024-04-22T19:51:32.540622+0000","last_became_peered":"2024-04-22T19:51:32.540622+0000","last_unstale":"2024-04-22T20:48:16.559078+0000","last_undegraded":"2024-04-22T20:48:16.559078+0000","last_fullsized":"2024-04-22T20:48:16.559078+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":300,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56095,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1229,"num_read_kb":1850,"num_write":1762,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.518288+0000","last_change":"2024-04-22T20:48:34.518332+0000","last_active":"2024-04-22T20:48:34.518288+0000","last_peered":"2024-04-22T20:48:34.518288+0000","last_clean":"2024-04-22T20:48:34.518288+0000","last_became_active":"2024-04-22T19:51:34.551009+0000","last_became_peered":"2024-04-22T19:51:34.551009+0000","last_unstale":"2024-04-22T20:48:34.518288+0000","last_undegraded":"2024-04-22T20:48:34.518288+0000","last_fullsized":"2024-04-22T20:48:34.518288+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.518165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.518165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.518165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:35:03.091773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"261'124","reported_seq":632,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.530433+0000","last_change":"2024-04-22T20:48:32.530468+0000","last_active":"2024-04-22T20:48:32.530433+0000","last_peered":"2024-04-22T20:48:32.530433+0000","last_clean":"2024-04-22T20:48:32.530433+0000","last_became_active":"2024-04-22T19:51:36.571517+0000","last_became_peered":"2024-04-22T19:51:36.571517+0000","last_unstale":"2024-04-22T20:48:32.530433+0000","last_undegraded":"2024-04-22T20:48:32.530433+0000","last_fullsized":"2024-04-22T20:48:32.530433+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:32.530326+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:32.530326+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.530326+0000","objects_scrubbed":3,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:41:58.642528+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":272,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"261'4186","reported_seq":4782,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.335611+0000","last_change":"2024-04-22T19:51:28.515172+0000","last_active":"2024-04-22T20:47:50.335611+0000","last_peered":"2024-04-22T20:47:50.335611+0000","last_clean":"2024-04-22T20:47:50.335611+0000","last_became_active":"2024-04-22T19:51:28.514869+0000","last_became_peered":"2024-04-22T19:51:28.514869+0000","last_unstale":"2024-04-22T20:47:50.335611+0000","last_undegraded":"2024-04-22T20:47:50.335611+0000","last_fullsized":"2024-04-22T20:47:50.335611+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":3000,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14767,"num_read_kb":11981,"num_write":6295,"num_write_kb":3142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":521,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.080937+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T20:47:25.080937+0000","last_peered":"2024-04-22T20:47:25.080937+0000","last_clean":"2024-04-22T20:47:25.080937+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T20:47:25.080937+0000","last_undegraded":"2024-04-22T20:47:25.080937+0000","last_fullsized":"2024-04-22T20:47:25.080937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"261'336","reported_seq":1032,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.912940+0000","last_change":"2024-04-22T19:51:26.506322+0000","last_active":"2024-04-22T20:48:17.912940+0000","last_peered":"2024-04-22T20:48:17.912940+0000","last_clean":"2024-04-22T20:48:17.912940+0000","last_became_active":"2024-04-22T19:51:26.505982+0000","last_became_peered":"2024-04-22T19:51:26.505982+0000","last_unstale":"2024-04-22T20:48:17.912940+0000","last_undegraded":"2024-04-22T20:48:17.912940+0000","last_fullsized":"2024-04-22T20:48:17.912940+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893637,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":542,"num_read_kb":107293,"num_write":1359,"num_write_kb":45616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.084060+0000","last_change":"2024-04-22T19:51:30.527554+0000","last_active":"2024-04-22T20:47:45.084060+0000","last_peered":"2024-04-22T20:47:45.084060+0000","last_clean":"2024-04-22T20:47:45.084060+0000","last_became_active":"2024-04-22T19:51:30.526939+0000","last_became_peered":"2024-04-22T19:51:30.526939+0000","last_unstale":"2024-04-22T20:47:45.084060+0000","last_undegraded":"2024-04-22T20:47:45.084060+0000","last_fullsized":"2024-04-22T20:47:45.084060+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"261'715","reported_seq":1417,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.303171+0000","last_change":"2024-04-22T19:51:32.543191+0000","last_active":"2024-04-22T20:48:17.303171+0000","last_peered":"2024-04-22T20:48:17.303171+0000","last_clean":"2024-04-22T20:48:17.303171+0000","last_became_active":"2024-04-22T19:51:32.542970+0000","last_became_peered":"2024-04-22T19:51:32.542970+0000","last_unstale":"2024-04-22T20:48:17.303171+0000","last_undegraded":"2024-04-22T20:48:17.303171+0000","last_fullsized":"2024-04-22T20:48:17.303171+0000","mapping_epoch":26,"log_start":"261'700","ondisk_log_start":"261'700","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":700,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142271,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1403,"num_read_kb":3194,"num_write":1318,"num_write_kb":1170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.198686+0000","last_change":"2024-04-22T19:51:34.554048+0000","last_active":"2024-04-22T20:47:55.198686+0000","last_peered":"2024-04-22T20:47:55.198686+0000","last_clean":"2024-04-22T20:47:55.198686+0000","last_became_active":"2024-04-22T19:51:34.553704+0000","last_became_peered":"2024-04-22T19:51:34.553704+0000","last_unstale":"2024-04-22T20:47:55.198686+0000","last_undegraded":"2024-04-22T20:47:55.198686+0000","last_fullsized":"2024-04-22T20:47:55.198686+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"261'266","reported_seq":1002,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.538415+0000","last_change":"2024-04-22T19:51:36.571003+0000","last_active":"2024-04-22T20:48:08.538415+0000","last_peered":"2024-04-22T20:48:08.538415+0000","last_clean":"2024-04-22T20:48:08.538415+0000","last_became_active":"2024-04-22T19:51:36.570717+0000","last_became_peered":"2024-04-22T19:51:36.570717+0000","last_unstale":"2024-04-22T20:48:08.538415+0000","last_undegraded":"2024-04-22T20:48:08.538415+0000","last_fullsized":"2024-04-22T20:48:08.538415+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":200,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":568,"num_write":495,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"261'50","reported_seq":588,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.059611+0000","last_change":"2024-04-22T19:51:28.506462+0000","last_active":"2024-04-22T20:47:45.059611+0000","last_peered":"2024-04-22T20:47:45.059611+0000","last_clean":"2024-04-22T20:47:45.059611+0000","last_became_active":"2024-04-22T19:51:28.505719+0000","last_became_peered":"2024-04-22T19:51:28.505719+0000","last_unstale":"2024-04-22T20:47:45.059611+0000","last_undegraded":"2024-04-22T20:47:45.059611+0000","last_fullsized":"2024-04-22T20:47:45.059611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":69,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":515,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.204930+0000","last_change":"2024-04-22T19:51:12.251085+0000","last_active":"2024-04-22T20:47:25.204930+0000","last_peered":"2024-04-22T20:47:25.204930+0000","last_clean":"2024-04-22T20:47:25.204930+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T20:47:25.204930+0000","last_undegraded":"2024-04-22T20:47:25.204930+0000","last_fullsized":"2024-04-22T20:47:25.204930+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"261'358","reported_seq":1035,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.935076+0000","last_change":"2024-04-22T19:51:26.501306+0000","last_active":"2024-04-22T20:48:17.935076+0000","last_peered":"2024-04-22T20:48:17.935076+0000","last_clean":"2024-04-22T20:48:17.935076+0000","last_became_active":"2024-04-22T19:51:26.500968+0000","last_became_peered":"2024-04-22T19:51:26.500968+0000","last_unstale":"2024-04-22T20:48:17.935076+0000","last_undegraded":"2024-04-22T20:48:17.935076+0000","last_fullsized":"2024-04-22T20:48:17.935076+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048617,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":597,"num_read_kb":30331,"num_write":1294,"num_write_kb":32709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.198663+0000","last_change":"2024-04-22T19:51:30.526621+0000","last_active":"2024-04-22T20:47:55.198663+0000","last_peered":"2024-04-22T20:47:55.198663+0000","last_clean":"2024-04-22T20:47:55.198663+0000","last_became_active":"2024-04-22T19:51:30.526328+0000","last_became_peered":"2024-04-22T19:51:30.526328+0000","last_unstale":"2024-04-22T20:47:55.198663+0000","last_undegraded":"2024-04-22T20:47:55.198663+0000","last_fullsized":"2024-04-22T20:47:55.198663+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"261'124","reported_seq":678,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.173632+0000","last_change":"2024-04-22T19:51:32.541073+0000","last_active":"2024-04-22T20:47:55.173632+0000","last_peered":"2024-04-22T20:47:55.173632+0000","last_clean":"2024-04-22T20:47:55.173632+0000","last_became_active":"2024-04-22T19:51:32.540805+0000","last_became_peered":"2024-04-22T19:51:32.540805+0000","last_unstale":"2024-04-22T20:47:55.173632+0000","last_undegraded":"2024-04-22T20:47:55.173632+0000","last_fullsized":"2024-04-22T20:47:55.173632+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":112,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.336303+0000","last_change":"2024-04-22T19:51:34.554535+0000","last_active":"2024-04-22T20:47:55.336303+0000","last_peered":"2024-04-22T20:47:55.336303+0000","last_clean":"2024-04-22T20:47:55.336303+0000","last_became_active":"2024-04-22T19:51:34.554307+0000","last_became_peered":"2024-04-22T19:51:34.554307+0000","last_unstale":"2024-04-22T20:47:55.336303+0000","last_undegraded":"2024-04-22T20:47:55.336303+0000","last_fullsized":"2024-04-22T20:47:55.336303+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"261'124","reported_seq":635,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.895261+0000","last_change":"2024-04-22T20:48:29.895308+0000","last_active":"2024-04-22T20:48:29.895261+0000","last_peered":"2024-04-22T20:48:29.895261+0000","last_clean":"2024-04-22T20:48:29.895261+0000","last_became_active":"2024-04-22T19:51:36.571130+0000","last_became_peered":"2024-04-22T19:51:36.571130+0000","last_unstale":"2024-04-22T20:48:29.895261+0000","last_undegraded":"2024-04-22T20:48:29.895261+0000","last_fullsized":"2024-04-22T20:48:29.895261+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:29.895174+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:29.895174+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.895174+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:20:29.352021+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":131,"num_write":305,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"261'90","reported_seq":659,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.060239+0000","last_change":"2024-04-22T19:51:28.514065+0000","last_active":"2024-04-22T20:47:50.060239+0000","last_peered":"2024-04-22T20:47:50.060239+0000","last_clean":"2024-04-22T20:47:50.060239+0000","last_became_active":"2024-04-22T19:51:28.513451+0000","last_became_peered":"2024-04-22T19:51:28.513451+0000","last_unstale":"2024-04-22T20:47:50.060239+0000","last_undegraded":"2024-04-22T20:47:50.060239+0000","last_fullsized":"2024-04-22T20:47:50.060239+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":305,"num_write":130,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":520,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.080993+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T20:47:25.080993+0000","last_peered":"2024-04-22T20:47:25.080993+0000","last_clean":"2024-04-22T20:47:25.080993+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T20:47:25.080993+0000","last_undegraded":"2024-04-22T20:47:25.080993+0000","last_fullsized":"2024-04-22T20:47:25.080993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"261'323","reported_seq":986,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.948838+0000","last_change":"2024-04-22T19:51:26.495730+0000","last_active":"2024-04-22T20:48:17.948838+0000","last_peered":"2024-04-22T20:48:17.948838+0000","last_clean":"2024-04-22T20:48:17.948838+0000","last_became_active":"2024-04-22T19:51:26.495405+0000","last_became_peered":"2024-04-22T19:51:26.495405+0000","last_unstale":"2024-04-22T20:48:17.948838+0000","last_undegraded":"2024-04-22T20:48:17.948838+0000","last_fullsized":"2024-04-22T20:48:17.948838+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264208,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":523,"num_read_kb":57840,"num_write":1265,"num_write_kb":50104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"25'1","reported_seq":506,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.081146+0000","last_change":"2024-04-22T19:51:30.521583+0000","last_active":"2024-04-22T20:47:50.081146+0000","last_peered":"2024-04-22T20:47:50.081146+0000","last_clean":"2024-04-22T20:47:50.081146+0000","last_became_active":"2024-04-22T19:51:30.521134+0000","last_became_peered":"2024-04-22T19:51:30.521134+0000","last_unstale":"2024-04-22T20:47:50.081146+0000","last_undegraded":"2024-04-22T20:47:50.081146+0000","last_fullsized":"2024-04-22T20:47:50.081146+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"261'221","reported_seq":1449,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:08.704004+0000","last_change":"2024-04-22T19:51:33.250383+0000","last_active":"2024-04-22T20:48:08.704004+0000","last_peered":"2024-04-22T20:48:08.704004+0000","last_clean":"2024-04-22T20:48:08.704004+0000","last_became_active":"2024-04-22T19:51:33.250082+0000","last_became_peered":"2024-04-22T19:51:33.250082+0000","last_unstale":"2024-04-22T20:48:08.704004+0000","last_undegraded":"2024-04-22T20:48:08.704004+0000","last_fullsized":"2024-04-22T20:48:08.704004+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":188,"num_read_kb":180,"num_write":143,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.198095+0000","last_change":"2024-04-22T19:51:34.548541+0000","last_active":"2024-04-22T20:47:50.198095+0000","last_peered":"2024-04-22T20:47:50.198095+0000","last_clean":"2024-04-22T20:47:50.198095+0000","last_became_active":"2024-04-22T19:51:34.546652+0000","last_became_peered":"2024-04-22T19:51:34.546652+0000","last_unstale":"2024-04-22T20:47:50.198095+0000","last_undegraded":"2024-04-22T20:47:50.198095+0000","last_fullsized":"2024-04-22T20:47:50.198095+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"261'258","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.494703+0000","last_change":"2024-04-22T20:48:34.494735+0000","last_active":"2024-04-22T20:48:34.494703+0000","last_peered":"2024-04-22T20:48:34.494703+0000","last_clean":"2024-04-22T20:48:34.494703+0000","last_became_active":"2024-04-22T19:51:36.568767+0000","last_became_peered":"2024-04-22T19:51:36.568767+0000","last_unstale":"2024-04-22T20:48:34.494703+0000","last_undegraded":"2024-04-22T20:48:34.494703+0000","last_fullsized":"2024-04-22T20:48:34.494703+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'258","last_scrub_stamp":"2024-04-22T20:48:34.494609+0000","last_deep_scrub":"261'258","last_deep_scrub_stamp":"2024-04-22T20:48:34.494609+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.494609+0000","objects_scrubbed":4,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:08:51.017317+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":625,"num_write":569,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"261'124","reported_seq":752,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.198136+0000","last_change":"2024-04-22T19:51:28.518594+0000","last_active":"2024-04-22T20:47:50.198136+0000","last_peered":"2024-04-22T20:47:50.198136+0000","last_clean":"2024-04-22T20:47:50.198136+0000","last_became_active":"2024-04-22T19:51:28.518084+0000","last_became_peered":"2024-04-22T19:51:28.518084+0000","last_unstale":"2024-04-22T20:47:50.198136+0000","last_undegraded":"2024-04-22T20:47:50.198136+0000","last_fullsized":"2024-04-22T20:47:50.198136+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":533,"num_read_kb":444,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":518,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.057885+0000","last_change":"2024-04-22T19:51:11.437739+0000","last_active":"2024-04-22T20:47:30.057885+0000","last_peered":"2024-04-22T20:47:30.057885+0000","last_clean":"2024-04-22T20:47:30.057885+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T20:47:30.057885+0000","last_undegraded":"2024-04-22T20:47:30.057885+0000","last_fullsized":"2024-04-22T20:47:30.057885+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":669,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.080951+0000","last_change":"2024-04-22T19:51:12.892628+0000","last_active":"2024-04-22T20:47:25.080951+0000","last_peered":"2024-04-22T20:47:25.080951+0000","last_clean":"2024-04-22T20:47:25.080951+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T20:47:25.080951+0000","last_undegraded":"2024-04-22T20:47:25.080951+0000","last_fullsized":"2024-04-22T20:47:25.080951+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"261'340","reported_seq":984,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.944151+0000","last_change":"2024-04-22T19:51:26.497957+0000","last_active":"2024-04-22T20:48:17.944151+0000","last_peered":"2024-04-22T20:48:17.944151+0000","last_clean":"2024-04-22T20:48:17.944151+0000","last_became_active":"2024-04-22T19:51:26.497608+0000","last_became_peered":"2024-04-22T19:51:26.497608+0000","last_unstale":"2024-04-22T20:48:17.944151+0000","last_undegraded":"2024-04-22T20:48:17.944151+0000","last_fullsized":"2024-04-22T20:48:17.944151+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4198438,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":523,"num_read_kb":42688,"num_write":1248,"num_write_kb":37697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.208989+0000","last_change":"2024-04-22T19:51:30.524749+0000","last_active":"2024-04-22T20:47:50.208989+0000","last_peered":"2024-04-22T20:47:50.208989+0000","last_clean":"2024-04-22T20:47:50.208989+0000","last_became_active":"2024-04-22T19:51:30.524437+0000","last_became_peered":"2024-04-22T19:51:30.524437+0000","last_unstale":"2024-04-22T20:47:50.208989+0000","last_undegraded":"2024-04-22T20:47:50.208989+0000","last_fullsized":"2024-04-22T20:47:50.208989+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"261'1111","reported_seq":2658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.570536+0000","last_change":"2024-04-22T20:48:35.570575+0000","last_active":"2024-04-22T20:48:35.570536+0000","last_peered":"2024-04-22T20:48:35.570536+0000","last_clean":"2024-04-22T20:48:35.570536+0000","last_became_active":"2024-04-22T19:51:32.538560+0000","last_became_peered":"2024-04-22T19:51:32.538560+0000","last_unstale":"2024-04-22T20:48:35.570536+0000","last_undegraded":"2024-04-22T20:48:35.570536+0000","last_fullsized":"2024-04-22T20:48:35.570536+0000","mapping_epoch":26,"log_start":"261'1100","ondisk_log_start":"261'1100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'1111","last_scrub_stamp":"2024-04-22T20:48:35.570456+0000","last_deep_scrub":"261'1111","last_deep_scrub_stamp":"2024-04-22T20:48:35.570456+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.570456+0000","objects_scrubbed":11,"log_size":11,"log_dups_size":1100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:51:22.759382+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148242,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1781,"num_read_kb":3545,"num_write":1100,"num_write_kb":929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5722,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.086412+0000","last_change":"2024-04-22T19:51:34.557341+0000","last_active":"2024-04-22T20:48:00.086412+0000","last_peered":"2024-04-22T20:48:00.086412+0000","last_clean":"2024-04-22T20:48:00.086412+0000","last_became_active":"2024-04-22T19:51:34.557022+0000","last_became_peered":"2024-04-22T19:51:34.557022+0000","last_unstale":"2024-04-22T20:48:00.086412+0000","last_undegraded":"2024-04-22T20:48:00.086412+0000","last_fullsized":"2024-04-22T20:48:00.086412+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"261'163","reported_seq":672,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:10.062782+0000","last_change":"2024-04-22T19:51:36.567874+0000","last_active":"2024-04-22T20:48:10.062782+0000","last_peered":"2024-04-22T20:48:10.062782+0000","last_clean":"2024-04-22T20:48:10.062782+0000","last_became_active":"2024-04-22T19:51:36.567603+0000","last_became_peered":"2024-04-22T19:51:36.567603+0000","last_unstale":"2024-04-22T20:48:10.062782+0000","last_undegraded":"2024-04-22T20:48:10.062782+0000","last_fullsized":"2024-04-22T20:48:10.062782+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":641,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":222,"num_read_kb":216,"num_write":373,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"261'364","reported_seq":999,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198456+0000","last_change":"2024-04-22T19:51:28.512889+0000","last_active":"2024-04-22T20:47:45.198456+0000","last_peered":"2024-04-22T20:47:45.198456+0000","last_clean":"2024-04-22T20:47:45.198456+0000","last_became_active":"2024-04-22T19:51:28.512561+0000","last_became_peered":"2024-04-22T19:51:28.512561+0000","last_unstale":"2024-04-22T20:47:45.198456+0000","last_undegraded":"2024-04-22T20:47:45.198456+0000","last_fullsized":"2024-04-22T20:47:45.198456+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1455,"num_read_kb":1128,"num_write":627,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"261'380","reported_seq":1098,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.857235+0000","last_change":"2024-04-22T19:51:26.500958+0000","last_active":"2024-04-22T20:48:17.857235+0000","last_peered":"2024-04-22T20:48:17.857235+0000","last_clean":"2024-04-22T20:48:17.857235+0000","last_became_active":"2024-04-22T19:51:26.498178+0000","last_became_peered":"2024-04-22T19:51:26.498178+0000","last_unstale":"2024-04-22T20:48:17.857235+0000","last_undegraded":"2024-04-22T20:48:17.857235+0000","last_fullsized":"2024-04-22T20:48:17.857235+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432486,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":709,"num_read_kb":47025,"num_write":1529,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":527,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.170453+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T20:47:30.170453+0000","last_peered":"2024-04-22T20:47:30.170453+0000","last_clean":"2024-04-22T20:47:30.170453+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T20:47:30.170453+0000","last_undegraded":"2024-04-22T20:47:30.170453+0000","last_fullsized":"2024-04-22T20:47:30.170453+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171021+0000","last_change":"2024-04-22T19:51:30.523537+0000","last_active":"2024-04-22T20:47:35.171021+0000","last_peered":"2024-04-22T20:47:35.171021+0000","last_clean":"2024-04-22T20:47:35.171021+0000","last_became_active":"2024-04-22T19:51:30.523242+0000","last_became_peered":"2024-04-22T19:51:30.523242+0000","last_unstale":"2024-04-22T20:47:35.171021+0000","last_undegraded":"2024-04-22T20:47:35.171021+0000","last_fullsized":"2024-04-22T20:47:35.171021+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"261'707","reported_seq":1433,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.894641+0000","last_change":"2024-04-22T19:51:33.248507+0000","last_active":"2024-04-22T20:48:17.894641+0000","last_peered":"2024-04-22T20:48:17.894641+0000","last_clean":"2024-04-22T20:48:17.894641+0000","last_became_active":"2024-04-22T19:51:33.248144+0000","last_became_peered":"2024-04-22T19:51:33.248144+0000","last_unstale":"2024-04-22T20:48:17.894641+0000","last_undegraded":"2024-04-22T20:48:17.894641+0000","last_fullsized":"2024-04-22T20:48:17.894641+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":600,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130335,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1263,"num_read_kb":2915,"num_write":1018,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.457004+0000","last_change":"2024-04-22T20:48:39.457028+0000","last_active":"2024-04-22T20:48:39.457004+0000","last_peered":"2024-04-22T20:48:39.457004+0000","last_clean":"2024-04-22T20:48:39.457004+0000","last_became_active":"2024-04-22T19:51:34.562001+0000","last_became_peered":"2024-04-22T19:51:34.562001+0000","last_unstale":"2024-04-22T20:48:39.457004+0000","last_undegraded":"2024-04-22T20:48:39.457004+0000","last_fullsized":"2024-04-22T20:48:39.457004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:39.456917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:39.456917+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.456917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:40.897612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"261'107","reported_seq":598,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.978305+0000","last_change":"2024-04-22T20:48:32.978335+0000","last_active":"2024-04-22T20:48:32.978305+0000","last_peered":"2024-04-22T20:48:32.978305+0000","last_clean":"2024-04-22T20:48:32.978305+0000","last_became_active":"2024-04-22T19:51:36.570134+0000","last_became_peered":"2024-04-22T19:51:36.570134+0000","last_unstale":"2024-04-22T20:48:32.978305+0000","last_undegraded":"2024-04-22T20:48:32.978305+0000","last_fullsized":"2024-04-22T20:48:32.978305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'107","last_scrub_stamp":"2024-04-22T20:48:32.978212+0000","last_deep_scrub":"261'107","last_deep_scrub_stamp":"2024-04-22T20:48:32.978212+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.978212+0000","objects_scrubbed":2,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:21.385230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":256,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"261'101","reported_seq":689,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171037+0000","last_change":"2024-04-22T19:51:28.516624+0000","last_active":"2024-04-22T20:47:35.171037+0000","last_peered":"2024-04-22T20:47:35.171037+0000","last_clean":"2024-04-22T20:47:35.171037+0000","last_became_active":"2024-04-22T19:51:28.516178+0000","last_became_peered":"2024-04-22T19:51:28.516178+0000","last_unstale":"2024-04-22T20:47:35.171037+0000","last_undegraded":"2024-04-22T20:47:35.171037+0000","last_fullsized":"2024-04-22T20:47:35.171037+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":337,"num_write":140,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"261'359","reported_seq":1055,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.868608+0000","last_change":"2024-04-22T19:51:26.503699+0000","last_active":"2024-04-22T20:48:17.868608+0000","last_peered":"2024-04-22T20:48:17.868608+0000","last_clean":"2024-04-22T20:48:17.868608+0000","last_became_active":"2024-04-22T19:51:26.502765+0000","last_became_peered":"2024-04-22T19:51:26.502765+0000","last_unstale":"2024-04-22T20:48:17.868608+0000","last_undegraded":"2024-04-22T20:48:17.868608+0000","last_fullsized":"2024-04-22T20:48:17.868608+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":300,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4728363,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":573,"num_read_kb":127053,"num_write":1383,"num_write_kb":34012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":519,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:40.197929+0000","last_change":"2024-04-22T19:51:11.438084+0000","last_active":"2024-04-22T20:47:40.197929+0000","last_peered":"2024-04-22T20:47:40.197929+0000","last_clean":"2024-04-22T20:47:40.197929+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T20:47:40.197929+0000","last_undegraded":"2024-04-22T20:47:40.197929+0000","last_fullsized":"2024-04-22T20:47:40.197929+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198443+0000","last_change":"2024-04-22T19:51:30.529790+0000","last_active":"2024-04-22T20:47:45.198443+0000","last_peered":"2024-04-22T20:47:45.198443+0000","last_clean":"2024-04-22T20:47:45.198443+0000","last_became_active":"2024-04-22T19:51:30.529437+0000","last_became_peered":"2024-04-22T19:51:30.529437+0000","last_unstale":"2024-04-22T20:47:45.198443+0000","last_undegraded":"2024-04-22T20:47:45.198443+0000","last_fullsized":"2024-04-22T20:47:45.198443+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"261'110","reported_seq":652,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.199181+0000","last_change":"2024-04-22T19:51:32.534640+0000","last_active":"2024-04-22T20:47:50.199181+0000","last_peered":"2024-04-22T20:47:50.199181+0000","last_clean":"2024-04-22T20:47:50.199181+0000","last_became_active":"2024-04-22T19:51:32.534280+0000","last_became_peered":"2024-04-22T19:51:32.534280+0000","last_unstale":"2024-04-22T20:47:50.199181+0000","last_undegraded":"2024-04-22T20:47:50.199181+0000","last_fullsized":"2024-04-22T20:47:50.199181+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"261'536","reported_seq":2157,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.960918+0000","last_change":"2024-04-22T19:51:34.557012+0000","last_active":"2024-04-22T20:48:17.960918+0000","last_peered":"2024-04-22T20:48:17.960918+0000","last_clean":"2024-04-22T20:48:17.960918+0000","last_became_active":"2024-04-22T19:51:34.556673+0000","last_became_peered":"2024-04-22T19:51:34.556673+0000","last_unstale":"2024-04-22T20:48:17.960918+0000","last_undegraded":"2024-04-22T20:48:17.960918+0000","last_fullsized":"2024-04-22T20:48:17.960918+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"261'152","reported_seq":651,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.460819+0000","last_change":"2024-04-22T20:48:34.460850+0000","last_active":"2024-04-22T20:48:34.460819+0000","last_peered":"2024-04-22T20:48:34.460819+0000","last_clean":"2024-04-22T20:48:34.460819+0000","last_became_active":"2024-04-22T19:51:36.575351+0000","last_became_peered":"2024-04-22T19:51:36.575351+0000","last_unstale":"2024-04-22T20:48:34.460819+0000","last_undegraded":"2024-04-22T20:48:34.460819+0000","last_fullsized":"2024-04-22T20:48:34.460819+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:34.460727+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:34.460727+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.460727+0000","objects_scrubbed":3,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:55:34.883011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":80,"num_write":319,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"261'146","reported_seq":772,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.197509+0000","last_change":"2024-04-22T19:51:28.514202+0000","last_active":"2024-04-22T20:47:45.197509+0000","last_peered":"2024-04-22T20:47:45.197509+0000","last_clean":"2024-04-22T20:47:45.197509+0000","last_became_active":"2024-04-22T19:51:28.513940+0000","last_became_peered":"2024-04-22T19:51:28.513940+0000","last_unstale":"2024-04-22T20:47:45.197509+0000","last_undegraded":"2024-04-22T20:47:45.197509+0000","last_fullsized":"2024-04-22T20:47:45.197509+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":550,"num_write":219,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"261'357","reported_seq":996,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.914517+0000","last_change":"2024-04-22T19:51:26.506454+0000","last_active":"2024-04-22T20:48:17.914517+0000","last_peered":"2024-04-22T20:48:17.914517+0000","last_clean":"2024-04-22T20:48:17.914517+0000","last_became_active":"2024-04-22T19:51:26.505842+0000","last_became_peered":"2024-04-22T19:51:26.505842+0000","last_unstale":"2024-04-22T20:48:17.914517+0000","last_undegraded":"2024-04-22T20:48:17.914517+0000","last_fullsized":"2024-04-22T20:48:17.914517+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308777,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":488,"num_read_kb":22191,"num_write":1115,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":518,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.332750+0000","last_change":"2024-04-22T19:51:12.251205+0000","last_active":"2024-04-22T20:47:30.332750+0000","last_peered":"2024-04-22T20:47:30.332750+0000","last_clean":"2024-04-22T20:47:30.332750+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T20:47:30.332750+0000","last_undegraded":"2024-04-22T20:47:30.332750+0000","last_fullsized":"2024-04-22T20:47:30.332750+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.523764+0000","last_change":"2024-04-22T20:48:35.523791+0000","last_active":"2024-04-22T20:48:35.523764+0000","last_peered":"2024-04-22T20:48:35.523764+0000","last_clean":"2024-04-22T20:48:35.523764+0000","last_became_active":"2024-04-22T19:51:30.525777+0000","last_became_peered":"2024-04-22T19:51:30.525777+0000","last_unstale":"2024-04-22T20:48:35.523764+0000","last_undegraded":"2024-04-22T20:48:35.523764+0000","last_fullsized":"2024-04-22T20:48:35.523764+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:35.523661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:35.523661+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.523661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:53:00.959964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"261'186","reported_seq":762,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.648519+0000","last_change":"2024-04-22T20:48:39.648562+0000","last_active":"2024-04-22T20:48:39.648519+0000","last_peered":"2024-04-22T20:48:39.648519+0000","last_clean":"2024-04-22T20:48:39.648519+0000","last_became_active":"2024-04-22T19:51:33.248354+0000","last_became_peered":"2024-04-22T19:51:33.248354+0000","last_unstale":"2024-04-22T20:48:39.648519+0000","last_undegraded":"2024-04-22T20:48:39.648519+0000","last_fullsized":"2024-04-22T20:48:39.648519+0000","mapping_epoch":26,"log_start":"258'100","ondisk_log_start":"258'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'186","last_scrub_stamp":"2024-04-22T20:48:39.648420+0000","last_deep_scrub":"261'186","last_deep_scrub_stamp":"2024-04-22T20:48:39.648420+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.648420+0000","objects_scrubbed":7,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:06:13.449126+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":222,"num_read_kb":211,"num_write":148,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"261'548","reported_seq":4432,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.947950+0000","last_change":"2024-04-22T20:48:36.948010+0000","last_active":"2024-04-22T20:48:36.947950+0000","last_peered":"2024-04-22T20:48:36.947950+0000","last_clean":"2024-04-22T20:48:36.947950+0000","last_became_active":"2024-04-22T19:51:34.561206+0000","last_became_peered":"2024-04-22T19:51:34.561206+0000","last_unstale":"2024-04-22T20:48:36.947950+0000","last_undegraded":"2024-04-22T20:48:36.947950+0000","last_fullsized":"2024-04-22T20:48:36.947950+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'548","last_scrub_stamp":"2024-04-22T20:48:36.947822+0000","last_deep_scrub":"261'548","last_deep_scrub_stamp":"2024-04-22T20:48:36.947822+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.947822+0000","objects_scrubbed":1,"log_size":48,"log_dups_size":500,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:24:06.222760+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"261'185","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.541264+0000","last_change":"2024-04-22T20:48:33.541297+0000","last_active":"2024-04-22T20:48:33.541264+0000","last_peered":"2024-04-22T20:48:33.541264+0000","last_clean":"2024-04-22T20:48:33.541264+0000","last_became_active":"2024-04-22T19:51:36.569338+0000","last_became_peered":"2024-04-22T19:51:36.569338+0000","last_unstale":"2024-04-22T20:48:33.541264+0000","last_undegraded":"2024-04-22T20:48:33.541264+0000","last_fullsized":"2024-04-22T20:48:33.541264+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'185","last_scrub_stamp":"2024-04-22T20:48:33.541172+0000","last_deep_scrub":"261'185","last_deep_scrub_stamp":"2024-04-22T20:48:33.541172+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.541172+0000","objects_scrubbed":3,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:38:36.406576+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":109,"num_write":422,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"261'109","reported_seq":705,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.059572+0000","last_change":"2024-04-22T19:51:28.512614+0000","last_active":"2024-04-22T20:47:45.059572+0000","last_peered":"2024-04-22T20:47:45.059572+0000","last_clean":"2024-04-22T20:47:45.059572+0000","last_became_active":"2024-04-22T19:51:28.512337+0000","last_became_peered":"2024-04-22T19:51:28.512337+0000","last_unstale":"2024-04-22T20:47:45.059572+0000","last_undegraded":"2024-04-22T20:47:45.059572+0000","last_fullsized":"2024-04-22T20:47:45.059572+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":434,"num_read_kb":362,"num_write":158,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'319","reported_seq":969,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.949257+0000","last_change":"2024-04-22T19:51:26.508311+0000","last_active":"2024-04-22T20:48:17.949257+0000","last_peered":"2024-04-22T20:48:17.949257+0000","last_clean":"2024-04-22T20:48:17.949257+0000","last_became_active":"2024-04-22T19:51:26.507804+0000","last_became_peered":"2024-04-22T19:51:26.507804+0000","last_unstale":"2024-04-22T20:48:17.949257+0000","last_undegraded":"2024-04-22T20:48:17.949257+0000","last_fullsized":"2024-04-22T20:48:17.949257+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":511,"num_read_kb":11012,"num_write":1234,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":518,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.332765+0000","last_change":"2024-04-22T19:51:11.437977+0000","last_active":"2024-04-22T20:47:30.332765+0000","last_peered":"2024-04-22T20:47:30.332765+0000","last_clean":"2024-04-22T20:47:30.332765+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T20:47:30.332765+0000","last_undegraded":"2024-04-22T20:47:30.332765+0000","last_fullsized":"2024-04-22T20:47:30.332765+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.197522+0000","last_change":"2024-04-22T19:51:30.527109+0000","last_active":"2024-04-22T20:47:45.197522+0000","last_peered":"2024-04-22T20:47:45.197522+0000","last_clean":"2024-04-22T20:47:45.197522+0000","last_became_active":"2024-04-22T19:51:30.526837+0000","last_became_peered":"2024-04-22T19:51:30.526837+0000","last_unstale":"2024-04-22T20:47:45.197522+0000","last_undegraded":"2024-04-22T20:47:45.197522+0000","last_fullsized":"2024-04-22T20:47:45.197522+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"261'542","reported_seq":1213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.352519+0000","last_change":"2024-04-22T19:51:32.542607+0000","last_active":"2024-04-22T20:48:17.352519+0000","last_peered":"2024-04-22T20:48:17.352519+0000","last_clean":"2024-04-22T20:48:17.352519+0000","last_became_active":"2024-04-22T19:51:32.542344+0000","last_became_peered":"2024-04-22T19:51:32.542344+0000","last_unstale":"2024-04-22T20:48:17.352519+0000","last_undegraded":"2024-04-22T20:48:17.352519+0000","last_fullsized":"2024-04-22T20:48:17.352519+0000","mapping_epoch":26,"log_start":"261'500","ondisk_log_start":"261'500","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":500,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97214,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":946,"num_read_kb":2097,"num_write":792,"num_write_kb":634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.386032+0000","last_change":"2024-04-22T20:48:33.386077+0000","last_active":"2024-04-22T20:48:33.386032+0000","last_peered":"2024-04-22T20:48:33.386032+0000","last_clean":"2024-04-22T20:48:33.386032+0000","last_became_active":"2024-04-22T19:51:34.550951+0000","last_became_peered":"2024-04-22T19:51:34.550951+0000","last_unstale":"2024-04-22T20:48:33.386032+0000","last_undegraded":"2024-04-22T20:48:33.386032+0000","last_fullsized":"2024-04-22T20:48:33.386032+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.385902+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.385902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:42.635210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"261'112","reported_seq":618,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.557761+0000","last_change":"2024-04-22T19:51:36.573230+0000","last_active":"2024-04-22T20:48:00.557761+0000","last_peered":"2024-04-22T20:48:00.557761+0000","last_clean":"2024-04-22T20:48:00.557761+0000","last_became_active":"2024-04-22T19:51:36.572976+0000","last_became_peered":"2024-04-22T19:51:36.572976+0000","last_unstale":"2024-04-22T20:48:00.557761+0000","last_undegraded":"2024-04-22T20:48:00.557761+0000","last_fullsized":"2024-04-22T20:48:00.557761+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":120,"num_write":270,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"261'131","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.497001+0000","last_change":"2024-04-22T20:48:35.497051+0000","last_active":"2024-04-22T20:48:35.497001+0000","last_peered":"2024-04-22T20:48:35.497001+0000","last_clean":"2024-04-22T20:48:35.497001+0000","last_became_active":"2024-04-22T19:51:36.574238+0000","last_became_peered":"2024-04-22T19:51:36.574238+0000","last_unstale":"2024-04-22T20:48:35.497001+0000","last_undegraded":"2024-04-22T20:48:35.497001+0000","last_fullsized":"2024-04-22T20:48:35.497001+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:35.496901+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:35.496901+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.496901+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:05:26.775249+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":193,"num_read_kb":192,"num_write":291,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"261'123","reported_seq":740,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.196668+0000","last_change":"2024-04-22T19:51:28.515646+0000","last_active":"2024-04-22T20:47:30.196668+0000","last_peered":"2024-04-22T20:47:30.196668+0000","last_clean":"2024-04-22T20:47:30.196668+0000","last_became_active":"2024-04-22T19:51:28.515334+0000","last_became_peered":"2024-04-22T19:51:28.515334+0000","last_unstale":"2024-04-22T20:47:30.196668+0000","last_undegraded":"2024-04-22T20:47:30.196668+0000","last_fullsized":"2024-04-22T20:47:30.196668+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":424,"num_write":184,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"261'324","reported_seq":1038,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.891699+0000","last_change":"2024-04-22T19:51:26.506527+0000","last_active":"2024-04-22T20:48:17.891699+0000","last_peered":"2024-04-22T20:48:17.891699+0000","last_clean":"2024-04-22T20:48:17.891699+0000","last_became_active":"2024-04-22T19:51:26.506150+0000","last_became_peered":"2024-04-22T19:51:26.506150+0000","last_unstale":"2024-04-22T20:48:17.891699+0000","last_undegraded":"2024-04-22T20:48:17.891699+0000","last_fullsized":"2024-04-22T20:48:17.891699+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097285,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":716,"num_read_kb":35402,"num_write":1112,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.602028+0000","last_change":"2024-04-22T20:48:36.602053+0000","last_active":"2024-04-22T20:48:36.602028+0000","last_peered":"2024-04-22T20:48:36.602028+0000","last_clean":"2024-04-22T20:48:36.602028+0000","last_became_active":"2024-04-22T19:51:30.522903+0000","last_became_peered":"2024-04-22T19:51:30.522903+0000","last_unstale":"2024-04-22T20:48:36.602028+0000","last_undegraded":"2024-04-22T20:48:36.602028+0000","last_fullsized":"2024-04-22T20:48:36.602028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:36.601934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:36.601934+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.601934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:58.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"261'296","reported_seq":881,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.871248+0000","last_change":"2024-04-22T19:51:32.539884+0000","last_active":"2024-04-22T20:48:16.871248+0000","last_peered":"2024-04-22T20:48:16.871248+0000","last_clean":"2024-04-22T20:48:16.871248+0000","last_became_active":"2024-04-22T19:51:32.539631+0000","last_became_peered":"2024-04-22T19:51:32.539631+0000","last_unstale":"2024-04-22T20:48:16.871248+0000","last_undegraded":"2024-04-22T20:48:16.871248+0000","last_fullsized":"2024-04-22T20:48:16.871248+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":200,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44228,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":495,"num_read_kb":1138,"num_write":404,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.170989+0000","last_change":"2024-04-22T19:51:34.556940+0000","last_active":"2024-04-22T20:47:35.170989+0000","last_peered":"2024-04-22T20:47:35.170989+0000","last_clean":"2024-04-22T20:47:35.170989+0000","last_became_active":"2024-04-22T19:51:34.556650+0000","last_became_peered":"2024-04-22T19:51:34.556650+0000","last_unstale":"2024-04-22T20:47:35.170989+0000","last_undegraded":"2024-04-22T20:47:35.170989+0000","last_fullsized":"2024-04-22T20:47:35.170989+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"261'99","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.450094+0000","last_change":"2024-04-22T20:48:38.450136+0000","last_active":"2024-04-22T20:48:38.450094+0000","last_peered":"2024-04-22T20:48:38.450094+0000","last_clean":"2024-04-22T20:48:38.450094+0000","last_became_active":"2024-04-22T19:51:36.566980+0000","last_became_peered":"2024-04-22T19:51:36.566980+0000","last_unstale":"2024-04-22T20:48:38.450094+0000","last_undegraded":"2024-04-22T20:48:38.450094+0000","last_fullsized":"2024-04-22T20:48:38.450094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'99","last_scrub_stamp":"2024-04-22T20:48:38.449980+0000","last_deep_scrub":"261'99","last_deep_scrub_stamp":"2024-04-22T20:48:38.449980+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.449980+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:31:45.431617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":235,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"261'130","reported_seq":735,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.057867+0000","last_change":"2024-04-22T19:51:28.517914+0000","last_active":"2024-04-22T20:47:30.057867+0000","last_peered":"2024-04-22T20:47:30.057867+0000","last_clean":"2024-04-22T20:47:30.057867+0000","last_became_active":"2024-04-22T19:51:28.517675+0000","last_became_peered":"2024-04-22T19:51:28.517675+0000","last_unstale":"2024-04-22T20:47:30.057867+0000","last_undegraded":"2024-04-22T20:47:30.057867+0000","last_fullsized":"2024-04-22T20:47:30.057867+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":543,"num_read_kb":446,"num_write":195,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"261'341","reported_seq":1042,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.922424+0000","last_change":"2024-04-22T19:51:26.502622+0000","last_active":"2024-04-22T20:48:17.922424+0000","last_peered":"2024-04-22T20:48:17.922424+0000","last_clean":"2024-04-22T20:48:17.922424+0000","last_became_active":"2024-04-22T19:51:26.501897+0000","last_became_peered":"2024-04-22T19:51:26.501897+0000","last_unstale":"2024-04-22T20:48:17.922424+0000","last_undegraded":"2024-04-22T20:48:17.922424+0000","last_fullsized":"2024-04-22T20:48:17.922424+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3673455,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":613,"num_read_kb":41436,"num_write":1146,"num_write_kb":29344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.170436+0000","last_change":"2024-04-22T19:51:30.524665+0000","last_active":"2024-04-22T20:47:30.170436+0000","last_peered":"2024-04-22T20:47:30.170436+0000","last_clean":"2024-04-22T20:47:30.170436+0000","last_became_active":"2024-04-22T19:51:30.524388+0000","last_became_peered":"2024-04-22T19:51:30.524388+0000","last_unstale":"2024-04-22T20:47:30.170436+0000","last_undegraded":"2024-04-22T20:47:30.170436+0000","last_fullsized":"2024-04-22T20:47:30.170436+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"261'49","reported_seq":554,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.196264+0000","last_change":"2024-04-22T19:51:32.539546+0000","last_active":"2024-04-22T20:47:35.196264+0000","last_peered":"2024-04-22T20:47:35.196264+0000","last_clean":"2024-04-22T20:47:35.196264+0000","last_became_active":"2024-04-22T19:51:32.539260+0000","last_became_peered":"2024-04-22T19:51:32.539260+0000","last_unstale":"2024-04-22T20:47:35.196264+0000","last_undegraded":"2024-04-22T20:47:35.196264+0000","last_fullsized":"2024-04-22T20:47:35.196264+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":36,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.082473+0000","last_change":"2024-04-22T19:51:34.551584+0000","last_active":"2024-04-22T20:47:35.082473+0000","last_peered":"2024-04-22T20:47:35.082473+0000","last_clean":"2024-04-22T20:47:35.082473+0000","last_became_active":"2024-04-22T19:51:34.551294+0000","last_became_peered":"2024-04-22T19:51:34.551294+0000","last_unstale":"2024-04-22T20:47:35.082473+0000","last_undegraded":"2024-04-22T20:47:35.082473+0000","last_fullsized":"2024-04-22T20:47:35.082473+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"261'202","reported_seq":728,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.470360+0000","last_change":"2024-04-22T20:48:36.470421+0000","last_active":"2024-04-22T20:48:36.470360+0000","last_peered":"2024-04-22T20:48:36.470360+0000","last_clean":"2024-04-22T20:48:36.470360+0000","last_became_active":"2024-04-22T19:51:36.573936+0000","last_became_peered":"2024-04-22T19:51:36.573936+0000","last_unstale":"2024-04-22T20:48:36.470360+0000","last_undegraded":"2024-04-22T20:48:36.470360+0000","last_fullsized":"2024-04-22T20:48:36.470360+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'202","last_scrub_stamp":"2024-04-22T20:48:36.470271+0000","last_deep_scrub":"261'202","last_deep_scrub_stamp":"2024-04-22T20:48:36.470271+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.470271+0000","objects_scrubbed":1,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:05:08.510777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":284,"num_write":451,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"261'231","reported_seq":918,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.170399+0000","last_change":"2024-04-22T19:51:28.508566+0000","last_active":"2024-04-22T20:47:30.170399+0000","last_peered":"2024-04-22T20:47:30.170399+0000","last_clean":"2024-04-22T20:47:30.170399+0000","last_became_active":"2024-04-22T19:51:28.508307+0000","last_became_peered":"2024-04-22T19:51:28.508307+0000","last_unstale":"2024-04-22T20:47:30.170399+0000","last_undegraded":"2024-04-22T20:47:30.170399+0000","last_fullsized":"2024-04-22T20:47:30.170399+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":959,"num_read_kb":784,"num_write":348,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"261'323","reported_seq":955,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.946121+0000","last_change":"2024-04-22T19:51:26.501920+0000","last_active":"2024-04-22T20:48:17.946121+0000","last_peered":"2024-04-22T20:48:17.946121+0000","last_clean":"2024-04-22T20:48:17.946121+0000","last_became_active":"2024-04-22T19:51:26.501477+0000","last_became_peered":"2024-04-22T19:51:26.501477+0000","last_unstale":"2024-04-22T20:48:17.946121+0000","last_undegraded":"2024-04-22T20:48:17.946121+0000","last_fullsized":"2024-04-22T20:48:17.946121+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":46522,"num_write":1280,"num_write_kb":35021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.081689+0000","last_change":"2024-04-22T19:51:30.527405+0000","last_active":"2024-04-22T20:47:30.081689+0000","last_peered":"2024-04-22T20:47:30.081689+0000","last_clean":"2024-04-22T20:47:30.081689+0000","last_became_active":"2024-04-22T19:51:30.527076+0000","last_became_peered":"2024-04-22T19:51:30.527076+0000","last_unstale":"2024-04-22T20:47:30.081689+0000","last_undegraded":"2024-04-22T20:47:30.081689+0000","last_fullsized":"2024-04-22T20:47:30.081689+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"261'302","reported_seq":903,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.040715+0000","last_change":"2024-04-22T19:51:32.542586+0000","last_active":"2024-04-22T20:48:16.040715+0000","last_peered":"2024-04-22T20:48:16.040715+0000","last_clean":"2024-04-22T20:48:16.040715+0000","last_became_active":"2024-04-22T19:51:32.542262+0000","last_became_peered":"2024-04-22T19:51:32.542262+0000","last_unstale":"2024-04-22T20:48:16.040715+0000","last_undegraded":"2024-04-22T20:48:16.040715+0000","last_fullsized":"2024-04-22T20:48:16.040715+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46510,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":491,"num_read_kb":1038,"num_write":384,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.651253+0000","last_change":"2024-04-22T20:48:38.651275+0000","last_active":"2024-04-22T20:48:38.651253+0000","last_peered":"2024-04-22T20:48:38.651253+0000","last_clean":"2024-04-22T20:48:38.651253+0000","last_became_active":"2024-04-22T19:51:34.553691+0000","last_became_peered":"2024-04-22T19:51:34.553691+0000","last_unstale":"2024-04-22T20:48:38.651253+0000","last_undegraded":"2024-04-22T20:48:38.651253+0000","last_fullsized":"2024-04-22T20:48:38.651253+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:38.651145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:38.651145+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.651145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:39:15.734170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"261'137","reported_seq":627,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.859857+0000","last_change":"2024-04-22T20:48:28.859930+0000","last_active":"2024-04-22T20:48:28.859857+0000","last_peered":"2024-04-22T20:48:28.859857+0000","last_clean":"2024-04-22T20:48:28.859857+0000","last_became_active":"2024-04-22T19:51:36.571290+0000","last_became_peered":"2024-04-22T19:51:36.571290+0000","last_unstale":"2024-04-22T20:48:28.859857+0000","last_undegraded":"2024-04-22T20:48:28.859857+0000","last_fullsized":"2024-04-22T20:48:28.859857+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'137","last_scrub_stamp":"2024-04-22T20:48:28.859747+0000","last_deep_scrub":"261'137","last_deep_scrub_stamp":"2024-04-22T20:48:28.859747+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.859747+0000","objects_scrubbed":2,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:02:28.549243+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":111,"num_write":334,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"261'185","reported_seq":855,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.170359+0000","last_change":"2024-04-22T19:51:28.507938+0000","last_active":"2024-04-22T20:47:30.170359+0000","last_peered":"2024-04-22T20:47:30.170359+0000","last_clean":"2024-04-22T20:47:30.170359+0000","last_became_active":"2024-04-22T19:51:28.506615+0000","last_became_peered":"2024-04-22T19:51:28.506615+0000","last_unstale":"2024-04-22T20:47:30.170359+0000","last_undegraded":"2024-04-22T20:47:30.170359+0000","last_fullsized":"2024-04-22T20:47:30.170359+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":757,"num_read_kb":623,"num_write":269,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'338","reported_seq":980,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.941579+0000","last_change":"2024-04-22T19:51:26.503480+0000","last_active":"2024-04-22T20:48:17.941579+0000","last_peered":"2024-04-22T20:48:17.941579+0000","last_clean":"2024-04-22T20:48:17.941579+0000","last_became_active":"2024-04-22T19:51:26.502938+0000","last_became_peered":"2024-04-22T19:51:26.502938+0000","last_unstale":"2024-04-22T20:48:17.941579+0000","last_undegraded":"2024-04-22T20:48:17.941579+0000","last_fullsized":"2024-04-22T20:48:17.941579+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194317,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":478,"num_read_kb":28738,"num_write":1269,"num_write_kb":45425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.504136+0000","last_change":"2024-04-22T20:48:32.504160+0000","last_active":"2024-04-22T20:48:32.504136+0000","last_peered":"2024-04-22T20:48:32.504136+0000","last_clean":"2024-04-22T20:48:32.504136+0000","last_became_active":"2024-04-22T19:51:30.526994+0000","last_became_peered":"2024-04-22T19:51:30.526994+0000","last_unstale":"2024-04-22T20:48:32.504136+0000","last_undegraded":"2024-04-22T20:48:32.504136+0000","last_fullsized":"2024-04-22T20:48:32.504136+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:32.504047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:32.504047+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.504047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:39.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"261'251","reported_seq":823,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.793738+0000","last_change":"2024-04-22T19:51:32.538193+0000","last_active":"2024-04-22T20:48:15.793738+0000","last_peered":"2024-04-22T20:48:15.793738+0000","last_clean":"2024-04-22T20:48:15.793738+0000","last_became_active":"2024-04-22T19:51:32.537875+0000","last_became_peered":"2024-04-22T19:51:32.537875+0000","last_unstale":"2024-04-22T20:48:15.793738+0000","last_undegraded":"2024-04-22T20:48:15.793738+0000","last_fullsized":"2024-04-22T20:48:15.793738+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":422,"num_read_kb":961,"num_write":338,"num_write_kb":268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":489,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.196310+0000","last_change":"2024-04-22T19:51:34.554251+0000","last_active":"2024-04-22T20:47:35.196310+0000","last_peered":"2024-04-22T20:47:35.196310+0000","last_clean":"2024-04-22T20:47:35.196310+0000","last_became_active":"2024-04-22T19:51:34.553954+0000","last_became_peered":"2024-04-22T19:51:34.553954+0000","last_unstale":"2024-04-22T20:47:35.196310+0000","last_undegraded":"2024-04-22T20:47:35.196310+0000","last_fullsized":"2024-04-22T20:47:35.196310+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"261'192","reported_seq":831,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.334979+0000","last_change":"2024-04-22T19:51:28.514880+0000","last_active":"2024-04-22T20:47:45.334979+0000","last_peered":"2024-04-22T20:47:45.334979+0000","last_clean":"2024-04-22T20:47:45.334979+0000","last_became_active":"2024-04-22T19:51:28.514593+0000","last_became_peered":"2024-04-22T19:51:28.514593+0000","last_unstale":"2024-04-22T20:47:45.334979+0000","last_undegraded":"2024-04-22T20:47:45.334979+0000","last_fullsized":"2024-04-22T20:47:45.334979+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":788,"num_read_kb":638,"num_write":274,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"261'342","reported_seq":1005,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.914032+0000","last_change":"2024-04-22T19:51:26.502507+0000","last_active":"2024-04-22T20:48:17.914032+0000","last_peered":"2024-04-22T20:48:17.914032+0000","last_clean":"2024-04-22T20:48:17.914032+0000","last_became_active":"2024-04-22T19:51:26.501812+0000","last_became_peered":"2024-04-22T19:51:26.501812+0000","last_unstale":"2024-04-22T20:48:17.914032+0000","last_undegraded":"2024-04-22T20:48:17.914032+0000","last_fullsized":"2024-04-22T20:48:17.914032+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857304,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":552,"num_read_kb":55834,"num_write":1162,"num_write_kb":35884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":501,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:40.196896+0000","last_change":"2024-04-22T19:51:30.525750+0000","last_active":"2024-04-22T20:47:40.196896+0000","last_peered":"2024-04-22T20:47:40.196896+0000","last_clean":"2024-04-22T20:47:40.196896+0000","last_became_active":"2024-04-22T19:51:30.525450+0000","last_became_peered":"2024-04-22T19:51:30.525450+0000","last_unstale":"2024-04-22T20:47:40.196896+0000","last_undegraded":"2024-04-22T20:47:40.196896+0000","last_fullsized":"2024-04-22T20:47:40.196896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"261'489","reported_seq":1126,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.643131+0000","last_change":"2024-04-22T20:48:37.643184+0000","last_active":"2024-04-22T20:48:37.643131+0000","last_peered":"2024-04-22T20:48:37.643131+0000","last_clean":"2024-04-22T20:48:37.643131+0000","last_became_active":"2024-04-22T19:51:32.540755+0000","last_became_peered":"2024-04-22T19:51:32.540755+0000","last_unstale":"2024-04-22T20:48:37.643131+0000","last_undegraded":"2024-04-22T20:48:37.643131+0000","last_fullsized":"2024-04-22T20:48:37.643131+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'489","last_scrub_stamp":"2024-04-22T20:48:37.643033+0000","last_deep_scrub":"261'489","last_deep_scrub_stamp":"2024-04-22T20:48:37.643033+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.643033+0000","objects_scrubbed":7,"log_size":89,"log_dups_size":400,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:20:59.082062+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93788,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":875,"num_read_kb":2058,"num_write":756,"num_write_kb":671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23904,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.952835+0000","last_change":"2024-04-22T20:48:37.952860+0000","last_active":"2024-04-22T20:48:37.952835+0000","last_peered":"2024-04-22T20:48:37.952835+0000","last_clean":"2024-04-22T20:48:37.952835+0000","last_became_active":"2024-04-22T19:51:34.553363+0000","last_became_peered":"2024-04-22T19:51:34.553363+0000","last_unstale":"2024-04-22T20:48:37.952835+0000","last_undegraded":"2024-04-22T20:48:37.952835+0000","last_fullsized":"2024-04-22T20:48:37.952835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:37.952749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:37.952749+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.952749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:14.410865+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"261'162","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.484277+0000","last_change":"2024-04-22T20:48:35.484310+0000","last_active":"2024-04-22T20:48:35.484277+0000","last_peered":"2024-04-22T20:48:35.484277+0000","last_clean":"2024-04-22T20:48:35.484277+0000","last_became_active":"2024-04-22T19:51:36.568373+0000","last_became_peered":"2024-04-22T19:51:36.568373+0000","last_unstale":"2024-04-22T20:48:35.484277+0000","last_undegraded":"2024-04-22T20:48:35.484277+0000","last_fullsized":"2024-04-22T20:48:35.484277+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'162","last_scrub_stamp":"2024-04-22T20:48:35.484193+0000","last_deep_scrub":"261'162","last_deep_scrub_stamp":"2024-04-22T20:48:35.484193+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.484193+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:59:33.641598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":213,"num_write":373,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"261'100","reported_seq":686,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:40.196882+0000","last_change":"2024-04-22T19:51:28.510922+0000","last_active":"2024-04-22T20:47:40.196882+0000","last_peered":"2024-04-22T20:47:40.196882+0000","last_clean":"2024-04-22T20:47:40.196882+0000","last_became_active":"2024-04-22T19:51:28.510629+0000","last_became_peered":"2024-04-22T19:51:28.510629+0000","last_unstale":"2024-04-22T20:47:40.196882+0000","last_undegraded":"2024-04-22T20:47:40.196882+0000","last_fullsized":"2024-04-22T20:47:40.196882+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":435,"num_read_kb":357,"num_write":161,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"261'305","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.906421+0000","last_change":"2024-04-22T19:51:26.497497+0000","last_active":"2024-04-22T20:48:17.906421+0000","last_peered":"2024-04-22T20:48:17.906421+0000","last_clean":"2024-04-22T20:48:17.906421+0000","last_became_active":"2024-04-22T19:51:26.497165+0000","last_became_peered":"2024-04-22T19:51:26.497165+0000","last_unstale":"2024-04-22T20:48:17.906421+0000","last_undegraded":"2024-04-22T20:48:17.906421+0000","last_fullsized":"2024-04-22T20:48:17.906421+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":662323,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":493,"num_read_kb":39207,"num_write":1122,"num_write_kb":42297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.479719+0000","last_change":"2024-04-22T20:48:33.479741+0000","last_active":"2024-04-22T20:48:33.479719+0000","last_peered":"2024-04-22T20:48:33.479719+0000","last_clean":"2024-04-22T20:48:33.479719+0000","last_became_active":"2024-04-22T19:51:30.526597+0000","last_became_peered":"2024-04-22T19:51:30.526597+0000","last_unstale":"2024-04-22T20:48:33.479719+0000","last_undegraded":"2024-04-22T20:48:33.479719+0000","last_fullsized":"2024-04-22T20:48:33.479719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.479622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.479622+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.479622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:35:51.805769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"261'311","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.881178+0000","last_change":"2024-04-22T19:51:32.537791+0000","last_active":"2024-04-22T20:48:15.881178+0000","last_peered":"2024-04-22T20:48:15.881178+0000","last_clean":"2024-04-22T20:48:15.881178+0000","last_became_active":"2024-04-22T19:51:32.537416+0000","last_became_peered":"2024-04-22T19:51:32.537416+0000","last_unstale":"2024-04-22T20:48:15.881178+0000","last_undegraded":"2024-04-22T20:48:15.881178+0000","last_fullsized":"2024-04-22T20:48:15.881178+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43451,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":521,"num_read_kb":1119,"num_write":436,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":485,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.051983+0000","last_change":"2024-04-22T19:51:34.561345+0000","last_active":"2024-04-22T20:46:35.051983+0000","last_peered":"2024-04-22T20:46:35.051983+0000","last_clean":"2024-04-22T20:46:35.051983+0000","last_became_active":"2024-04-22T19:51:34.561096+0000","last_became_peered":"2024-04-22T19:51:34.561096+0000","last_unstale":"2024-04-22T20:46:35.051983+0000","last_undegraded":"2024-04-22T20:46:35.051983+0000","last_fullsized":"2024-04-22T20:46:35.051983+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"261'161","reported_seq":662,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:40.334084+0000","last_change":"2024-04-22T19:51:36.571812+0000","last_active":"2024-04-22T20:47:40.334084+0000","last_peered":"2024-04-22T20:47:40.334084+0000","last_clean":"2024-04-22T20:47:40.334084+0000","last_became_active":"2024-04-22T19:51:36.571565+0000","last_became_peered":"2024-04-22T19:51:36.571565+0000","last_unstale":"2024-04-22T20:47:40.334084+0000","last_undegraded":"2024-04-22T20:47:40.334084+0000","last_fullsized":"2024-04-22T20:47:40.334084+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":100,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":180,"num_write":473,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"261'44","reported_seq":610,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.163690+0000","last_change":"2024-04-22T19:51:28.507235+0000","last_active":"2024-04-22T20:46:30.163690+0000","last_peered":"2024-04-22T20:46:30.163690+0000","last_clean":"2024-04-22T20:46:30.163690+0000","last_became_active":"2024-04-22T19:51:28.506835+0000","last_became_peered":"2024-04-22T19:51:28.506835+0000","last_unstale":"2024-04-22T20:46:30.163690+0000","last_undegraded":"2024-04-22T20:46:30.163690+0000","last_fullsized":"2024-04-22T20:46:30.163690+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":202,"num_read_kb":174,"num_write":58,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"261'377","reported_seq":1013,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.442985+0000","last_change":"2024-04-22T20:48:24.443059+0000","last_active":"2024-04-22T20:48:24.442985+0000","last_peered":"2024-04-22T20:48:24.442985+0000","last_clean":"2024-04-22T20:48:24.442985+0000","last_became_active":"2024-04-22T19:52:04.617820+0000","last_became_peered":"2024-04-22T19:52:04.617820+0000","last_unstale":"2024-04-22T20:48:24.442985+0000","last_undegraded":"2024-04-22T20:48:24.442985+0000","last_fullsized":"2024-04-22T20:48:24.442985+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'377","last_scrub_stamp":"2024-04-22T20:48:24.442876+0000","last_deep_scrub":"261'377","last_deep_scrub_stamp":"2024-04-22T20:48:24.442876+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.442876+0000","objects_scrubbed":13,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:42:38.642906+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14463569,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":535,"num_read_kb":31540,"num_write":1473,"num_write_kb":43247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.188260+0000","last_change":"2024-04-22T19:51:30.529632+0000","last_active":"2024-04-22T20:46:30.188260+0000","last_peered":"2024-04-22T20:46:30.188260+0000","last_clean":"2024-04-22T20:46:30.188260+0000","last_became_active":"2024-04-22T19:51:30.529322+0000","last_became_peered":"2024-04-22T19:51:30.529322+0000","last_unstale":"2024-04-22T20:46:30.188260+0000","last_undegraded":"2024-04-22T20:46:30.188260+0000","last_fullsized":"2024-04-22T20:46:30.188260+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"261'271","reported_seq":859,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.119577+0000","last_change":"2024-04-22T19:51:33.248631+0000","last_active":"2024-04-22T20:48:17.119577+0000","last_peered":"2024-04-22T20:48:17.119577+0000","last_clean":"2024-04-22T20:48:17.119577+0000","last_became_active":"2024-04-22T19:51:33.248293+0000","last_became_peered":"2024-04-22T19:51:33.248293+0000","last_unstale":"2024-04-22T20:48:17.119577+0000","last_undegraded":"2024-04-22T20:48:17.119577+0000","last_fullsized":"2024-04-22T20:48:17.119577+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56069,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1056,"num_write":364,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.059650+0000","last_change":"2024-04-22T19:51:34.561273+0000","last_active":"2024-04-22T20:47:45.059650+0000","last_peered":"2024-04-22T20:47:45.059650+0000","last_clean":"2024-04-22T20:47:45.059650+0000","last_became_active":"2024-04-22T19:51:34.560955+0000","last_became_peered":"2024-04-22T19:51:34.560955+0000","last_unstale":"2024-04-22T20:47:45.059650+0000","last_undegraded":"2024-04-22T20:47:45.059650+0000","last_fullsized":"2024-04-22T20:47:45.059650+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"261'122","reported_seq":614,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.995578+0000","last_change":"2024-04-22T20:48:34.995676+0000","last_active":"2024-04-22T20:48:34.995578+0000","last_peered":"2024-04-22T20:48:34.995578+0000","last_clean":"2024-04-22T20:48:34.995578+0000","last_became_active":"2024-04-22T19:51:36.567139+0000","last_became_peered":"2024-04-22T19:51:36.567139+0000","last_unstale":"2024-04-22T20:48:34.995578+0000","last_undegraded":"2024-04-22T20:48:34.995578+0000","last_fullsized":"2024-04-22T20:48:34.995578+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'122","last_scrub_stamp":"2024-04-22T20:48:34.995457+0000","last_deep_scrub":"261'122","last_deep_scrub_stamp":"2024-04-22T20:48:34.995457+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.995457+0000","objects_scrubbed":3,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:31:14.860340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":92,"num_write":268,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"261'106","reported_seq":689,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.073243+0000","last_change":"2024-04-22T19:51:28.512939+0000","last_active":"2024-04-22T20:46:30.073243+0000","last_peered":"2024-04-22T20:46:30.073243+0000","last_clean":"2024-04-22T20:46:30.073243+0000","last_became_active":"2024-04-22T19:51:28.512642+0000","last_became_peered":"2024-04-22T19:51:28.512642+0000","last_unstale":"2024-04-22T20:46:30.073243+0000","last_undegraded":"2024-04-22T20:46:30.073243+0000","last_fullsized":"2024-04-22T20:46:30.073243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":349,"num_write":169,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"261'328","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.949077+0000","last_change":"2024-04-22T19:51:26.512313+0000","last_active":"2024-04-22T20:48:17.949077+0000","last_peered":"2024-04-22T20:48:17.949077+0000","last_clean":"2024-04-22T20:48:17.949077+0000","last_became_active":"2024-04-22T19:51:26.511966+0000","last_became_peered":"2024-04-22T19:51:26.511966+0000","last_unstale":"2024-04-22T20:48:17.949077+0000","last_undegraded":"2024-04-22T20:48:17.949077+0000","last_fullsized":"2024-04-22T20:48:17.949077+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262244,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":436,"num_read_kb":38275,"num_write":1102,"num_write_kb":51509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":501,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.538950+0000","last_change":"2024-04-22T20:48:34.538975+0000","last_active":"2024-04-22T20:48:34.538950+0000","last_peered":"2024-04-22T20:48:34.538950+0000","last_clean":"2024-04-22T20:48:34.538950+0000","last_became_active":"2024-04-22T19:51:30.526949+0000","last_became_peered":"2024-04-22T19:51:30.526949+0000","last_unstale":"2024-04-22T20:48:34.538950+0000","last_undegraded":"2024-04-22T20:48:34.538950+0000","last_fullsized":"2024-04-22T20:48:34.538950+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.538867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.538867+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.538867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:45:12.857278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"261'111","reported_seq":646,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.397743+0000","last_change":"2024-04-22T20:48:35.397789+0000","last_active":"2024-04-22T20:48:35.397743+0000","last_peered":"2024-04-22T20:48:35.397743+0000","last_clean":"2024-04-22T20:48:35.397743+0000","last_became_active":"2024-04-22T19:51:33.248113+0000","last_became_peered":"2024-04-22T19:51:33.248113+0000","last_unstale":"2024-04-22T20:48:35.397743+0000","last_undegraded":"2024-04-22T20:48:35.397743+0000","last_fullsized":"2024-04-22T20:48:35.397743+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'111","last_scrub_stamp":"2024-04-22T20:48:35.397661+0000","last_deep_scrub":"261'111","last_deep_scrub_stamp":"2024-04-22T20:48:35.397661+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.397661+0000","objects_scrubbed":8,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:59:22.081071+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":119,"num_read_kb":113,"num_write":116,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11222,"num_omap_keys":30,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"261'518","reported_seq":2298,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.956117+0000","last_change":"2024-04-22T19:51:34.557105+0000","last_active":"2024-04-22T20:48:17.956117+0000","last_peered":"2024-04-22T20:48:17.956117+0000","last_clean":"2024-04-22T20:48:17.956117+0000","last_became_active":"2024-04-22T19:51:34.556789+0000","last_became_peered":"2024-04-22T19:51:34.556789+0000","last_unstale":"2024-04-22T20:48:17.956117+0000","last_undegraded":"2024-04-22T20:48:17.956117+0000","last_fullsized":"2024-04-22T20:48:17.956117+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"261'96","reported_seq":595,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:40.083084+0000","last_change":"2024-04-22T19:51:36.571445+0000","last_active":"2024-04-22T20:47:40.083084+0000","last_peered":"2024-04-22T20:47:40.083084+0000","last_clean":"2024-04-22T20:47:40.083084+0000","last_became_active":"2024-04-22T19:51:36.571175+0000","last_became_peered":"2024-04-22T19:51:36.571175+0000","last_unstale":"2024-04-22T20:47:40.083084+0000","last_undegraded":"2024-04-22T20:47:40.083084+0000","last_fullsized":"2024-04-22T20:47:40.083084+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":221,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"261'131","reported_seq":709,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.188239+0000","last_change":"2024-04-22T19:51:28.509477+0000","last_active":"2024-04-22T20:46:30.188239+0000","last_peered":"2024-04-22T20:46:30.188239+0000","last_clean":"2024-04-22T20:46:30.188239+0000","last_became_active":"2024-04-22T19:51:28.509110+0000","last_became_peered":"2024-04-22T19:51:28.509110+0000","last_unstale":"2024-04-22T20:46:30.188239+0000","last_undegraded":"2024-04-22T20:46:30.188239+0000","last_fullsized":"2024-04-22T20:46:30.188239+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":519,"num_read_kb":420,"num_write":188,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"261'402","reported_seq":1037,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.849997+0000","last_change":"2024-04-22T20:48:25.850089+0000","last_active":"2024-04-22T20:48:25.849997+0000","last_peered":"2024-04-22T20:48:25.849997+0000","last_clean":"2024-04-22T20:48:25.849997+0000","last_became_active":"2024-04-22T19:52:04.619385+0000","last_became_peered":"2024-04-22T19:52:04.619385+0000","last_unstale":"2024-04-22T20:48:25.849997+0000","last_undegraded":"2024-04-22T20:48:25.849997+0000","last_fullsized":"2024-04-22T20:48:25.849997+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'402","last_scrub_stamp":"2024-04-22T20:48:25.849900+0000","last_deep_scrub":"261'402","last_deep_scrub_stamp":"2024-04-22T20:48:25.849900+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.849900+0000","objects_scrubbed":10,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:07.464040+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682319,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":541,"num_read_kb":21001,"num_write":1452,"num_write_kb":33571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.073195+0000","last_change":"2024-04-22T19:51:30.527064+0000","last_active":"2024-04-22T20:46:30.073195+0000","last_peered":"2024-04-22T20:46:30.073195+0000","last_clean":"2024-04-22T20:46:30.073195+0000","last_became_active":"2024-04-22T19:51:30.526469+0000","last_became_peered":"2024-04-22T19:51:30.526469+0000","last_unstale":"2024-04-22T20:46:30.073195+0000","last_undegraded":"2024-04-22T20:46:30.073195+0000","last_fullsized":"2024-04-22T20:46:30.073195+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"261'306","reported_seq":891,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.484892+0000","last_change":"2024-04-22T19:51:32.540830+0000","last_active":"2024-04-22T20:48:17.484892+0000","last_peered":"2024-04-22T20:48:17.484892+0000","last_clean":"2024-04-22T20:48:17.484892+0000","last_became_active":"2024-04-22T19:51:32.540573+0000","last_became_peered":"2024-04-22T19:51:32.540573+0000","last_unstale":"2024-04-22T20:48:17.484892+0000","last_undegraded":"2024-04-22T20:48:17.484892+0000","last_fullsized":"2024-04-22T20:48:17.484892+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48433,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":496,"num_read_kb":1091,"num_write":394,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":488,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.193314+0000","last_change":"2024-04-22T19:51:34.554124+0000","last_active":"2024-04-22T20:47:10.193314+0000","last_peered":"2024-04-22T20:47:10.193314+0000","last_clean":"2024-04-22T20:47:10.193314+0000","last_became_active":"2024-04-22T19:51:34.553857+0000","last_became_peered":"2024-04-22T19:51:34.553857+0000","last_unstale":"2024-04-22T20:47:10.193314+0000","last_undegraded":"2024-04-22T20:47:10.193314+0000","last_fullsized":"2024-04-22T20:47:10.193314+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"261'157","reported_seq":667,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.377887+0000","last_change":"2024-04-22T20:48:39.377921+0000","last_active":"2024-04-22T20:48:39.377887+0000","last_peered":"2024-04-22T20:48:39.377887+0000","last_clean":"2024-04-22T20:48:39.377887+0000","last_became_active":"2024-04-22T19:51:36.575211+0000","last_became_peered":"2024-04-22T19:51:36.575211+0000","last_unstale":"2024-04-22T20:48:39.377887+0000","last_undegraded":"2024-04-22T20:48:39.377887+0000","last_fullsized":"2024-04-22T20:48:39.377887+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'157","last_scrub_stamp":"2024-04-22T20:48:39.377810+0000","last_deep_scrub":"261'157","last_deep_scrub_stamp":"2024-04-22T20:48:39.377810+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.377810+0000","objects_scrubbed":4,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:18:41.159874+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":158,"num_write":375,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"261'85","reported_seq":681,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:04.040297+0000","last_change":"2024-04-22T19:51:28.508354+0000","last_active":"2024-04-22T20:48:04.040297+0000","last_peered":"2024-04-22T20:48:04.040297+0000","last_clean":"2024-04-22T20:48:04.040297+0000","last_became_active":"2024-04-22T19:51:28.508145+0000","last_became_peered":"2024-04-22T19:51:28.508145+0000","last_unstale":"2024-04-22T20:48:04.040297+0000","last_undegraded":"2024-04-22T20:48:04.040297+0000","last_fullsized":"2024-04-22T20:48:04.040297+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":376,"num_read_kb":320,"num_write":115,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"261'271","reported_seq":890,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.940444+0000","last_change":"2024-04-22T19:51:26.504671+0000","last_active":"2024-04-22T20:48:17.940444+0000","last_peered":"2024-04-22T20:48:17.940444+0000","last_clean":"2024-04-22T20:48:17.940444+0000","last_became_active":"2024-04-22T19:51:26.504301+0000","last_became_peered":"2024-04-22T19:51:26.504301+0000","last_unstale":"2024-04-22T20:48:17.940444+0000","last_undegraded":"2024-04-22T20:48:17.940444+0000","last_fullsized":"2024-04-22T20:48:17.940444+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":4555,"num_write":1089,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":491,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:30.196771+0000","last_change":"2024-04-22T19:51:30.524895+0000","last_active":"2024-04-22T20:46:30.196771+0000","last_peered":"2024-04-22T20:46:30.196771+0000","last_clean":"2024-04-22T20:46:30.196771+0000","last_became_active":"2024-04-22T19:51:30.524417+0000","last_became_peered":"2024-04-22T19:51:30.524417+0000","last_unstale":"2024-04-22T20:46:30.196771+0000","last_undegraded":"2024-04-22T20:46:30.196771+0000","last_fullsized":"2024-04-22T20:46:30.196771+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"261'229","reported_seq":799,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.731038+0000","last_change":"2024-04-22T19:51:32.540923+0000","last_active":"2024-04-22T20:48:17.731038+0000","last_peered":"2024-04-22T20:48:17.731038+0000","last_clean":"2024-04-22T20:48:17.731038+0000","last_became_active":"2024-04-22T19:51:32.540659+0000","last_became_peered":"2024-04-22T19:51:32.540659+0000","last_unstale":"2024-04-22T20:48:17.731038+0000","last_undegraded":"2024-04-22T20:48:17.731038+0000","last_fullsized":"2024-04-22T20:48:17.731038+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16866,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":857,"num_write":302,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.574005+0000","last_change":"2024-04-22T20:48:36.574034+0000","last_active":"2024-04-22T20:48:36.574005+0000","last_peered":"2024-04-22T20:48:36.574005+0000","last_clean":"2024-04-22T20:48:36.574005+0000","last_became_active":"2024-04-22T19:51:34.552344+0000","last_became_peered":"2024-04-22T19:51:34.552344+0000","last_unstale":"2024-04-22T20:48:36.574005+0000","last_undegraded":"2024-04-22T20:48:36.574005+0000","last_fullsized":"2024-04-22T20:48:36.574005+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:36.573910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:36.573910+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.573910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:39:04.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"261'138","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.476371+0000","last_change":"2024-04-22T20:48:37.476415+0000","last_active":"2024-04-22T20:48:37.476371+0000","last_peered":"2024-04-22T20:48:37.476371+0000","last_clean":"2024-04-22T20:48:37.476371+0000","last_became_active":"2024-04-22T19:51:36.567594+0000","last_became_peered":"2024-04-22T19:51:36.567594+0000","last_unstale":"2024-04-22T20:48:37.476371+0000","last_undegraded":"2024-04-22T20:48:37.476371+0000","last_fullsized":"2024-04-22T20:48:37.476371+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'138","last_scrub_stamp":"2024-04-22T20:48:37.476271+0000","last_deep_scrub":"261'138","last_deep_scrub_stamp":"2024-04-22T20:48:37.476271+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.476271+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:27:27.543918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":214,"num_write":335,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"261'87","reported_seq":649,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:10.168259+0000","last_change":"2024-04-22T19:51:28.519780+0000","last_active":"2024-04-22T20:47:10.168259+0000","last_peered":"2024-04-22T20:47:10.168259+0000","last_clean":"2024-04-22T20:47:10.168259+0000","last_became_active":"2024-04-22T19:51:28.519542+0000","last_became_peered":"2024-04-22T19:51:28.519542+0000","last_unstale":"2024-04-22T20:47:10.168259+0000","last_undegraded":"2024-04-22T20:47:10.168259+0000","last_fullsized":"2024-04-22T20:47:10.168259+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":347,"num_read_kb":277,"num_write":137,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"261'398","reported_seq":1243,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.890036+0000","last_change":"2024-04-22T19:51:26.500077+0000","last_active":"2024-04-22T20:48:17.890036+0000","last_peered":"2024-04-22T20:48:17.890036+0000","last_clean":"2024-04-22T20:48:17.890036+0000","last_became_active":"2024-04-22T19:51:26.499390+0000","last_became_peered":"2024-04-22T19:51:26.499390+0000","last_unstale":"2024-04-22T20:48:17.890036+0000","last_undegraded":"2024-04-22T20:48:17.890036+0000","last_fullsized":"2024-04-22T20:48:17.890036+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4204544,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1043,"num_read_kb":88369,"num_write":1413,"num_write_kb":41583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.193374+0000","last_change":"2024-04-22T19:51:30.526678+0000","last_active":"2024-04-22T20:47:05.193374+0000","last_peered":"2024-04-22T20:47:05.193374+0000","last_clean":"2024-04-22T20:47:05.193374+0000","last_became_active":"2024-04-22T19:51:30.526183+0000","last_became_peered":"2024-04-22T19:51:30.526183+0000","last_unstale":"2024-04-22T20:47:05.193374+0000","last_undegraded":"2024-04-22T20:47:05.193374+0000","last_fullsized":"2024-04-22T20:47:05.193374+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"261'248","reported_seq":813,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.999897+0000","last_change":"2024-04-22T19:51:32.538965+0000","last_active":"2024-04-22T20:48:16.999897+0000","last_peered":"2024-04-22T20:48:16.999897+0000","last_clean":"2024-04-22T20:48:16.999897+0000","last_became_active":"2024-04-22T19:51:32.538644+0000","last_became_peered":"2024-04-22T19:51:32.538644+0000","last_unstale":"2024-04-22T20:48:16.999897+0000","last_undegraded":"2024-04-22T20:48:16.999897+0000","last_fullsized":"2024-04-22T20:48:16.999897+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45694,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":402,"num_read_kb":955,"num_write":332,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":495,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.405082+0000","last_change":"2024-04-22T20:48:31.405108+0000","last_active":"2024-04-22T20:48:31.405082+0000","last_peered":"2024-04-22T20:48:31.405082+0000","last_clean":"2024-04-22T20:48:31.405082+0000","last_became_active":"2024-04-22T19:51:34.551085+0000","last_became_peered":"2024-04-22T19:51:34.551085+0000","last_unstale":"2024-04-22T20:48:31.405082+0000","last_undegraded":"2024-04-22T20:48:31.405082+0000","last_fullsized":"2024-04-22T20:48:31.405082+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:31.404990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:31.404990+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.404990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:00.257782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"261'179","reported_seq":904,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.460845+0000","last_change":"2024-04-22T19:51:36.569537+0000","last_active":"2024-04-22T20:48:00.460845+0000","last_peered":"2024-04-22T20:48:00.460845+0000","last_clean":"2024-04-22T20:48:00.460845+0000","last_became_active":"2024-04-22T19:51:36.569234+0000","last_became_peered":"2024-04-22T19:51:36.569234+0000","last_unstale":"2024-04-22T20:48:00.460845+0000","last_undegraded":"2024-04-22T20:48:00.460845+0000","last_fullsized":"2024-04-22T20:48:00.460845+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:35.544909+0000","last_clean_scrub_stamp":"2024-04-22T19:51:35.544909+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":100,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":419,"num_read_kb":454,"num_write":429,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"261'180","reported_seq":751,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.193408+0000","last_change":"2024-04-22T19:51:28.516276+0000","last_active":"2024-04-22T20:47:05.193408+0000","last_peered":"2024-04-22T20:47:05.193408+0000","last_clean":"2024-04-22T20:47:05.193408+0000","last_became_active":"2024-04-22T19:51:28.515506+0000","last_became_peered":"2024-04-22T19:51:28.515506+0000","last_unstale":"2024-04-22T20:47:05.193408+0000","last_undegraded":"2024-04-22T20:47:05.193408+0000","last_fullsized":"2024-04-22T20:47:05.193408+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":678,"num_read_kb":529,"num_write":279,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"261'297","reported_seq":910,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.948862+0000","last_change":"2024-04-22T19:51:26.504948+0000","last_active":"2024-04-22T20:48:17.948862+0000","last_peered":"2024-04-22T20:48:17.948862+0000","last_clean":"2024-04-22T20:48:17.948862+0000","last_became_active":"2024-04-22T19:51:26.504604+0000","last_became_peered":"2024-04-22T19:51:26.504604+0000","last_unstale":"2024-04-22T20:48:17.948862+0000","last_undegraded":"2024-04-22T20:48:17.948862+0000","last_fullsized":"2024-04-22T20:48:17.948862+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242937,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":372,"num_read_kb":9007,"num_write":952,"num_write_kb":46630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"261'2","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.636032+0000","last_change":"2024-04-22T20:48:38.636081+0000","last_active":"2024-04-22T20:48:38.636032+0000","last_peered":"2024-04-22T20:48:38.636032+0000","last_clean":"2024-04-22T20:48:38.636032+0000","last_became_active":"2024-04-22T19:51:30.523338+0000","last_became_peered":"2024-04-22T19:51:30.523338+0000","last_unstale":"2024-04-22T20:48:38.636032+0000","last_undegraded":"2024-04-22T20:48:38.636032+0000","last_fullsized":"2024-04-22T20:48:38.636032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:38.635935+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:38.635935+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.635935+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:28:44.233266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"261'313","reported_seq":907,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.337535+0000","last_change":"2024-04-22T20:48:37.337600+0000","last_active":"2024-04-22T20:48:37.337535+0000","last_peered":"2024-04-22T20:48:37.337535+0000","last_clean":"2024-04-22T20:48:37.337535+0000","last_became_active":"2024-04-22T19:51:32.537373+0000","last_became_peered":"2024-04-22T19:51:32.537373+0000","last_unstale":"2024-04-22T20:48:37.337535+0000","last_undegraded":"2024-04-22T20:48:37.337535+0000","last_fullsized":"2024-04-22T20:48:37.337535+0000","mapping_epoch":26,"log_start":"261'301","ondisk_log_start":"261'301","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:37.337420+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:37.337420+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.337420+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":301,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:00.481534+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49488,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":1080,"num_write":394,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":487,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:35.189264+0000","last_change":"2024-04-22T19:51:34.555960+0000","last_active":"2024-04-22T20:46:35.189264+0000","last_peered":"2024-04-22T20:46:35.189264+0000","last_clean":"2024-04-22T20:46:35.189264+0000","last_became_active":"2024-04-22T19:51:34.555709+0000","last_became_peered":"2024-04-22T19:51:34.555709+0000","last_unstale":"2024-04-22T20:46:35.189264+0000","last_undegraded":"2024-04-22T20:46:35.189264+0000","last_fullsized":"2024-04-22T20:46:35.189264+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:33.531091+0000","last_clean_scrub_stamp":"2024-04-22T19:51:33.531091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"261'146","reported_seq":639,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.318417+0000","last_change":"2024-04-22T20:48:26.318450+0000","last_active":"2024-04-22T20:48:26.318417+0000","last_peered":"2024-04-22T20:48:26.318417+0000","last_clean":"2024-04-22T20:48:26.318417+0000","last_became_active":"2024-04-22T19:51:36.574012+0000","last_became_peered":"2024-04-22T19:51:36.574012+0000","last_unstale":"2024-04-22T20:48:26.318417+0000","last_undegraded":"2024-04-22T20:48:26.318417+0000","last_fullsized":"2024-04-22T20:48:26.318417+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'146","last_scrub_stamp":"2024-04-22T20:48:26.318331+0000","last_deep_scrub":"261'146","last_deep_scrub_stamp":"2024-04-22T20:48:26.318331+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.318331+0000","objects_scrubbed":2,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:20.701708+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":109,"num_write":348,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"261'372","reported_seq":1038,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.060227+0000","last_change":"2024-04-22T19:51:28.518589+0000","last_active":"2024-04-22T20:47:50.060227+0000","last_peered":"2024-04-22T20:47:50.060227+0000","last_clean":"2024-04-22T20:47:50.060227+0000","last_became_active":"2024-04-22T19:51:28.518339+0000","last_became_peered":"2024-04-22T19:51:28.518339+0000","last_unstale":"2024-04-22T20:47:50.060227+0000","last_undegraded":"2024-04-22T20:47:50.060227+0000","last_fullsized":"2024-04-22T20:47:50.060227+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1526,"num_read_kb":1211,"num_write":619,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"261'324","reported_seq":1024,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.940444+0000","last_change":"2024-04-22T19:51:26.503151+0000","last_active":"2024-04-22T20:48:17.940444+0000","last_peered":"2024-04-22T20:48:17.940444+0000","last_clean":"2024-04-22T20:48:17.940444+0000","last_became_active":"2024-04-22T19:51:26.502773+0000","last_became_peered":"2024-04-22T19:51:26.502773+0000","last_unstale":"2024-04-22T20:48:17.940444+0000","last_undegraded":"2024-04-22T20:48:17.940444+0000","last_fullsized":"2024-04-22T20:48:17.940444+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4371014,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":594,"num_read_kb":14386,"num_write":1094,"num_write_kb":17858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"261'524","reported_seq":1296,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.877481+0000","last_change":"2024-04-22T20:48:26.877525+0000","last_active":"2024-04-22T20:48:26.877481+0000","last_peered":"2024-04-22T20:48:26.877481+0000","last_clean":"2024-04-22T20:48:26.877481+0000","last_became_active":"2024-04-22T19:52:04.617036+0000","last_became_peered":"2024-04-22T19:52:04.617036+0000","last_unstale":"2024-04-22T20:48:26.877481+0000","last_undegraded":"2024-04-22T20:48:26.877481+0000","last_fullsized":"2024-04-22T20:48:26.877481+0000","mapping_epoch":33,"log_start":"261'500","ondisk_log_start":"261'500","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'524","last_scrub_stamp":"2024-04-22T20:48:26.877358+0000","last_deep_scrub":"261'524","last_deep_scrub_stamp":"2024-04-22T20:48:26.877358+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.877358+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":500,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:34:00.586382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2230,"num_read_kb":1847,"num_write":935,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"261'286","reported_seq":934,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.553956+0000","last_change":"2024-04-22T20:48:38.553990+0000","last_active":"2024-04-22T20:48:38.553956+0000","last_peered":"2024-04-22T20:48:38.553956+0000","last_clean":"2024-04-22T20:48:38.553956+0000","last_became_active":"2024-04-22T19:51:26.495948+0000","last_became_peered":"2024-04-22T19:51:26.495948+0000","last_unstale":"2024-04-22T20:48:38.553956+0000","last_undegraded":"2024-04-22T20:48:38.553956+0000","last_fullsized":"2024-04-22T20:48:38.553956+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'286","last_scrub_stamp":"2024-04-22T20:48:38.553878+0000","last_deep_scrub":"261'286","last_deep_scrub_stamp":"2024-04-22T20:48:38.553878+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.553878+0000","objects_scrubbed":6,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:12.594083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180512,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":454,"num_read_kb":10117,"num_write":1004,"num_write_kb":22835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"261'208","reported_seq":842,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.208952+0000","last_change":"2024-04-22T19:51:28.513290+0000","last_active":"2024-04-22T20:47:50.208952+0000","last_peered":"2024-04-22T20:47:50.208952+0000","last_clean":"2024-04-22T20:47:50.208952+0000","last_became_active":"2024-04-22T19:51:28.513026+0000","last_became_peered":"2024-04-22T19:51:28.513026+0000","last_unstale":"2024-04-22T20:47:50.208952+0000","last_undegraded":"2024-04-22T20:47:50.208952+0000","last_fullsized":"2024-04-22T20:47:50.208952+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":100,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":827,"num_read_kb":665,"num_write":342,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"261'313","reported_seq":933,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.948510+0000","last_change":"2024-04-22T19:51:26.507266+0000","last_active":"2024-04-22T20:48:17.948510+0000","last_peered":"2024-04-22T20:48:17.948510+0000","last_clean":"2024-04-22T20:48:17.948510+0000","last_became_active":"2024-04-22T19:51:26.506748+0000","last_became_peered":"2024-04-22T19:51:26.506748+0000","last_unstale":"2024-04-22T20:48:17.948510+0000","last_undegraded":"2024-04-22T20:48:17.948510+0000","last_fullsized":"2024-04-22T20:48:17.948510+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":416,"num_read_kb":12678,"num_write":1099,"num_write_kb":41347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"261'4175","reported_seq":4792,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.060184+0000","last_change":"2024-04-22T19:51:28.511560+0000","last_active":"2024-04-22T20:47:50.060184+0000","last_peered":"2024-04-22T20:47:50.060184+0000","last_clean":"2024-04-22T20:47:50.060184+0000","last_became_active":"2024-04-22T19:51:28.511312+0000","last_became_peered":"2024-04-22T19:51:28.511312+0000","last_unstale":"2024-04-22T20:47:50.060184+0000","last_undegraded":"2024-04-22T20:47:50.060184+0000","last_fullsized":"2024-04-22T20:47:50.060184+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":3000,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14718,"num_read_kb":11919,"num_write":6278,"num_write_kb":3133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"261'312","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.452280+0000","last_change":"2024-04-22T20:48:28.452280+0000","last_active":"2024-04-22T20:48:28.452280+0000","last_peered":"2024-04-22T20:48:28.452280+0000","last_clean":"2024-04-22T20:48:28.452280+0000","last_became_active":"2024-04-22T19:52:05.114734+0000","last_became_peered":"2024-04-22T19:52:05.114734+0000","last_unstale":"2024-04-22T20:48:28.452280+0000","last_undegraded":"2024-04-22T20:48:28.452280+0000","last_fullsized":"2024-04-22T20:48:28.452280+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'312","last_scrub_stamp":"2024-04-22T20:48:28.452109+0000","last_deep_scrub":"261'312","last_deep_scrub_stamp":"2024-04-22T20:48:28.452109+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.452109+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:41:21.599729+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":5518,"num_write":1088,"num_write_kb":21667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.23","version":"261'170","reported_seq":852,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:03.021789+0000","last_change":"2024-04-22T19:51:28.516445+0000","last_active":"2024-04-22T20:48:03.021789+0000","last_peered":"2024-04-22T20:48:03.021789+0000","last_clean":"2024-04-22T20:48:03.021789+0000","last_became_active":"2024-04-22T19:51:28.516125+0000","last_became_peered":"2024-04-22T19:51:28.516125+0000","last_unstale":"2024-04-22T20:48:03.021789+0000","last_undegraded":"2024-04-22T20:48:03.021789+0000","last_fullsized":"2024-04-22T20:48:03.021789+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":741,"num_read_kb":616,"num_write":243,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"261'368","reported_seq":1024,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.429047+0000","last_change":"2024-04-22T20:48:26.429162+0000","last_active":"2024-04-22T20:48:26.429047+0000","last_peered":"2024-04-22T20:48:26.429047+0000","last_clean":"2024-04-22T20:48:26.429047+0000","last_became_active":"2024-04-22T19:52:04.619286+0000","last_became_peered":"2024-04-22T19:52:04.619286+0000","last_unstale":"2024-04-22T20:48:26.429047+0000","last_undegraded":"2024-04-22T20:48:26.429047+0000","last_fullsized":"2024-04-22T20:48:26.429047+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'368","last_scrub_stamp":"2024-04-22T20:48:26.428925+0000","last_deep_scrub":"261'368","last_deep_scrub_stamp":"2024-04-22T20:48:26.428925+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.428925+0000","objects_scrubbed":9,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:38:16.414530+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4719706,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":588,"num_read_kb":69897,"num_write":1236,"num_write_kb":36151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"261'122","reported_seq":727,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198494+0000","last_change":"2024-04-22T19:51:28.516690+0000","last_active":"2024-04-22T20:47:45.198494+0000","last_peered":"2024-04-22T20:47:45.198494+0000","last_clean":"2024-04-22T20:47:45.198494+0000","last_became_active":"2024-04-22T19:51:28.516437+0000","last_became_peered":"2024-04-22T19:51:28.516437+0000","last_unstale":"2024-04-22T20:47:45.198494+0000","last_undegraded":"2024-04-22T20:47:45.198494+0000","last_fullsized":"2024-04-22T20:47:45.198494+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":423,"num_write":178,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"261'317","reported_seq":1052,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.868098+0000","last_change":"2024-04-22T19:51:26.498349+0000","last_active":"2024-04-22T20:48:17.868098+0000","last_peered":"2024-04-22T20:48:17.868098+0000","last_clean":"2024-04-22T20:48:17.868098+0000","last_became_active":"2024-04-22T19:51:26.498039+0000","last_became_peered":"2024-04-22T19:51:26.498039+0000","last_unstale":"2024-04-22T20:48:17.868098+0000","last_undegraded":"2024-04-22T20:48:17.868098+0000","last_fullsized":"2024-04-22T20:48:17.868098+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194319,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":766,"num_read_kb":22382,"num_write":1253,"num_write_kb":18139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"261'199","reported_seq":849,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.172341+0000","last_change":"2024-04-22T19:51:28.520853+0000","last_active":"2024-04-22T20:47:45.172341+0000","last_peered":"2024-04-22T20:47:45.172341+0000","last_clean":"2024-04-22T20:47:45.172341+0000","last_became_active":"2024-04-22T19:51:28.520612+0000","last_became_peered":"2024-04-22T19:51:28.520612+0000","last_unstale":"2024-04-22T20:47:45.172341+0000","last_undegraded":"2024-04-22T20:47:45.172341+0000","last_fullsized":"2024-04-22T20:47:45.172341+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":735,"num_read_kb":593,"num_write":262,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"261'365","reported_seq":1014,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.914335+0000","last_change":"2024-04-22T19:51:26.499154+0000","last_active":"2024-04-22T20:48:17.914335+0000","last_peered":"2024-04-22T20:48:17.914335+0000","last_clean":"2024-04-22T20:48:17.914335+0000","last_became_active":"2024-04-22T19:51:26.498845+0000","last_became_peered":"2024-04-22T19:51:26.498845+0000","last_unstale":"2024-04-22T20:48:17.914335+0000","last_undegraded":"2024-04-22T20:48:17.914335+0000","last_fullsized":"2024-04-22T20:48:17.914335+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050721,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":506,"num_read_kb":33263,"num_write":1241,"num_write_kb":30860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"261'133","reported_seq":775,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.610209+0000","last_change":"2024-04-22T20:48:37.610234+0000","last_active":"2024-04-22T20:48:37.610209+0000","last_peered":"2024-04-22T20:48:37.610209+0000","last_clean":"2024-04-22T20:48:37.610209+0000","last_became_active":"2024-04-22T19:51:28.519669+0000","last_became_peered":"2024-04-22T19:51:28.519669+0000","last_unstale":"2024-04-22T20:48:37.610209+0000","last_undegraded":"2024-04-22T20:48:37.610209+0000","last_fullsized":"2024-04-22T20:48:37.610209+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'133","last_scrub_stamp":"2024-04-22T20:48:37.610118+0000","last_deep_scrub":"261'133","last_deep_scrub_stamp":"2024-04-22T20:48:37.610118+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.610118+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:03:13.662918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":560,"num_read_kb":466,"num_write":196,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"261'330","reported_seq":935,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.490447+0000","last_change":"2024-04-22T20:48:29.490501+0000","last_active":"2024-04-22T20:48:29.490447+0000","last_peered":"2024-04-22T20:48:29.490447+0000","last_clean":"2024-04-22T20:48:29.490447+0000","last_became_active":"2024-04-22T19:52:04.617649+0000","last_became_peered":"2024-04-22T19:52:04.617649+0000","last_unstale":"2024-04-22T20:48:29.490447+0000","last_undegraded":"2024-04-22T20:48:29.490447+0000","last_fullsized":"2024-04-22T20:48:29.490447+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'330","last_scrub_stamp":"2024-04-22T20:48:29.490328+0000","last_deep_scrub":"261'330","last_deep_scrub_stamp":"2024-04-22T20:48:29.490328+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.490328+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:55:48.835393+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050919,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":29074,"num_write":1030,"num_write_kb":22627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"261'128","reported_seq":710,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.195821+0000","last_change":"2024-04-22T19:51:28.521764+0000","last_active":"2024-04-22T20:47:30.195821+0000","last_peered":"2024-04-22T20:47:30.195821+0000","last_clean":"2024-04-22T20:47:30.195821+0000","last_became_active":"2024-04-22T19:51:28.521516+0000","last_became_peered":"2024-04-22T19:51:28.521516+0000","last_unstale":"2024-04-22T20:47:30.195821+0000","last_undegraded":"2024-04-22T20:47:30.195821+0000","last_fullsized":"2024-04-22T20:47:30.195821+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":542,"num_read_kb":442,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"261'338","reported_seq":1019,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.910603+0000","last_change":"2024-04-22T19:51:26.510292+0000","last_active":"2024-04-22T20:48:17.910603+0000","last_peered":"2024-04-22T20:48:17.910603+0000","last_clean":"2024-04-22T20:48:17.910603+0000","last_became_active":"2024-04-22T19:51:26.507843+0000","last_became_peered":"2024-04-22T19:51:26.507843+0000","last_unstale":"2024-04-22T20:48:17.910603+0000","last_undegraded":"2024-04-22T20:48:17.910603+0000","last_fullsized":"2024-04-22T20:48:17.910603+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5531396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":136369,"num_write":1233,"num_write_kb":55788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"261'235","reported_seq":861,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.196708+0000","last_change":"2024-04-22T19:51:28.515473+0000","last_active":"2024-04-22T20:47:30.196708+0000","last_peered":"2024-04-22T20:47:30.196708+0000","last_clean":"2024-04-22T20:47:30.196708+0000","last_became_active":"2024-04-22T19:51:28.515164+0000","last_became_peered":"2024-04-22T19:51:28.515164+0000","last_unstale":"2024-04-22T20:47:30.196708+0000","last_undegraded":"2024-04-22T20:47:30.196708+0000","last_fullsized":"2024-04-22T20:47:30.196708+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":950,"num_read_kb":750,"num_write":364,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"261'502","reported_seq":1203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.940696+0000","last_change":"2024-04-22T19:51:26.494217+0000","last_active":"2024-04-22T20:48:17.940696+0000","last_peered":"2024-04-22T20:48:17.940696+0000","last_clean":"2024-04-22T20:48:17.940696+0000","last_became_active":"2024-04-22T19:51:26.493860+0000","last_became_peered":"2024-04-22T19:51:26.493860+0000","last_unstale":"2024-04-22T20:48:17.940696+0000","last_undegraded":"2024-04-22T20:48:17.940696+0000","last_fullsized":"2024-04-22T20:48:17.940696+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":400,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6292618,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":951,"num_read_kb":13872,"num_write":1439,"num_write_kb":37693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"261'251","reported_seq":930,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:55.081653+0000","last_change":"2024-04-22T19:51:28.515833+0000","last_active":"2024-04-22T20:47:55.081653+0000","last_peered":"2024-04-22T20:47:55.081653+0000","last_clean":"2024-04-22T20:47:55.081653+0000","last_became_active":"2024-04-22T19:51:28.515547+0000","last_became_peered":"2024-04-22T19:51:28.515547+0000","last_unstale":"2024-04-22T20:47:55.081653+0000","last_undegraded":"2024-04-22T20:47:55.081653+0000","last_fullsized":"2024-04-22T20:47:55.081653+0000","mapping_epoch":22,"log_start":"247'200","ondisk_log_start":"247'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1120,"num_read_kb":973,"num_write":365,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"261'324","reported_seq":955,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.900703+0000","last_change":"2024-04-22T19:51:26.500937+0000","last_active":"2024-04-22T20:48:17.900703+0000","last_peered":"2024-04-22T20:48:17.900703+0000","last_clean":"2024-04-22T20:48:17.900703+0000","last_became_active":"2024-04-22T19:51:26.500566+0000","last_became_peered":"2024-04-22T19:51:26.500566+0000","last_unstale":"2024-04-22T20:48:17.900703+0000","last_undegraded":"2024-04-22T20:48:17.900703+0000","last_fullsized":"2024-04-22T20:48:17.900703+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283749,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":424,"num_read_kb":45844,"num_write":1120,"num_write_kb":42819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"261'152","reported_seq":751,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.198527+0000","last_change":"2024-04-22T19:51:28.516044+0000","last_active":"2024-04-22T20:47:45.198527+0000","last_peered":"2024-04-22T20:47:45.198527+0000","last_clean":"2024-04-22T20:47:45.198527+0000","last_became_active":"2024-04-22T19:51:28.515764+0000","last_became_peered":"2024-04-22T19:51:28.515764+0000","last_unstale":"2024-04-22T20:47:45.198527+0000","last_undegraded":"2024-04-22T20:47:45.198527+0000","last_fullsized":"2024-04-22T20:47:45.198527+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":633,"num_read_kb":515,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"261'261","reported_seq":871,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.822172+0000","last_change":"2024-04-22T19:51:26.512810+0000","last_active":"2024-04-22T20:48:17.822172+0000","last_peered":"2024-04-22T20:48:17.822172+0000","last_clean":"2024-04-22T20:48:17.822172+0000","last_became_active":"2024-04-22T19:51:26.512467+0000","last_became_peered":"2024-04-22T19:51:26.512467+0000","last_unstale":"2024-04-22T20:48:17.822172+0000","last_undegraded":"2024-04-22T20:48:17.822172+0000","last_fullsized":"2024-04-22T20:48:17.822172+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11536625,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":377,"num_read_kb":17908,"num_write":933,"num_write_kb":43703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"261'27","reported_seq":555,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.084038+0000","last_change":"2024-04-22T19:51:28.510362+0000","last_active":"2024-04-22T20:47:45.084038+0000","last_peered":"2024-04-22T20:47:45.084038+0000","last_clean":"2024-04-22T20:47:45.084038+0000","last_became_active":"2024-04-22T19:51:28.510091+0000","last_became_peered":"2024-04-22T19:51:28.510091+0000","last_unstale":"2024-04-22T20:47:45.084038+0000","last_undegraded":"2024-04-22T20:47:45.084038+0000","last_fullsized":"2024-04-22T20:47:45.084038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":95,"num_write":39,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"261'340","reported_seq":988,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.940495+0000","last_change":"2024-04-22T19:51:26.502070+0000","last_active":"2024-04-22T20:48:17.940495+0000","last_peered":"2024-04-22T20:48:17.940495+0000","last_clean":"2024-04-22T20:48:17.940495+0000","last_became_active":"2024-04-22T19:51:26.501700+0000","last_became_peered":"2024-04-22T19:51:26.501700+0000","last_unstale":"2024-04-22T20:48:17.940495+0000","last_undegraded":"2024-04-22T20:48:17.940495+0000","last_fullsized":"2024-04-22T20:48:17.940495+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231939,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":506,"num_read_kb":6050,"num_write":1241,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"261'54","reported_seq":604,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.197445+0000","last_change":"2024-04-22T19:51:28.509002+0000","last_active":"2024-04-22T20:47:45.197445+0000","last_peered":"2024-04-22T20:47:45.197445+0000","last_clean":"2024-04-22T20:47:45.197445+0000","last_became_active":"2024-04-22T19:51:28.508768+0000","last_became_peered":"2024-04-22T19:51:28.508768+0000","last_unstale":"2024-04-22T20:47:45.197445+0000","last_undegraded":"2024-04-22T20:47:45.197445+0000","last_fullsized":"2024-04-22T20:47:45.197445+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":238,"num_read_kb":196,"num_write":80,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"261'355","reported_seq":995,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.932058+0000","last_change":"2024-04-22T19:51:26.497503+0000","last_active":"2024-04-22T20:48:17.932058+0000","last_peered":"2024-04-22T20:48:17.932058+0000","last_clean":"2024-04-22T20:48:17.932058+0000","last_became_active":"2024-04-22T19:51:26.497156+0000","last_became_peered":"2024-04-22T19:51:26.497156+0000","last_unstale":"2024-04-22T20:48:17.932058+0000","last_undegraded":"2024-04-22T20:48:17.932058+0000","last_fullsized":"2024-04-22T20:48:17.932058+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437245,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":35895,"num_write":1244,"num_write_kb":35286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"261'4050","reported_seq":4613,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:45.172350+0000","last_change":"2024-04-22T19:51:28.515076+0000","last_active":"2024-04-22T20:47:45.172350+0000","last_peered":"2024-04-22T20:47:45.172350+0000","last_clean":"2024-04-22T20:47:45.172350+0000","last_became_active":"2024-04-22T19:51:28.514780+0000","last_became_peered":"2024-04-22T19:51:28.514780+0000","last_unstale":"2024-04-22T20:47:45.172350+0000","last_undegraded":"2024-04-22T20:47:45.172350+0000","last_fullsized":"2024-04-22T20:47:45.172350+0000","mapping_epoch":22,"log_start":"261'4000","ondisk_log_start":"261'4000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":3000,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14224,"num_read_kb":11535,"num_write":6070,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"261'302","reported_seq":1070,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.563705+0000","last_change":"2024-04-22T20:48:39.563748+0000","last_active":"2024-04-22T20:48:39.563705+0000","last_peered":"2024-04-22T20:48:39.563705+0000","last_clean":"2024-04-22T20:48:39.563705+0000","last_became_active":"2024-04-22T19:51:26.496214+0000","last_became_peered":"2024-04-22T19:51:26.496214+0000","last_unstale":"2024-04-22T20:48:39.563705+0000","last_undegraded":"2024-04-22T20:48:39.563705+0000","last_fullsized":"2024-04-22T20:48:39.563705+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'302","last_scrub_stamp":"2024-04-22T20:48:39.563612+0000","last_deep_scrub":"261'302","last_deep_scrub_stamp":"2024-04-22T20:48:39.563612+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.563612+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:41:49.634270+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1338117,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":584,"num_read_kb":13796,"num_write":1181,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"261'298","reported_seq":980,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:00.086282+0000","last_change":"2024-04-22T19:51:28.508937+0000","last_active":"2024-04-22T20:48:00.086282+0000","last_peered":"2024-04-22T20:48:00.086282+0000","last_clean":"2024-04-22T20:48:00.086282+0000","last_became_active":"2024-04-22T19:51:28.508681+0000","last_became_peered":"2024-04-22T19:51:28.508681+0000","last_unstale":"2024-04-22T20:48:00.086282+0000","last_undegraded":"2024-04-22T20:48:00.086282+0000","last_fullsized":"2024-04-22T20:48:00.086282+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1231,"num_read_kb":1000,"num_write":489,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"261'282","reported_seq":892,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.949293+0000","last_change":"2024-04-22T19:51:26.505139+0000","last_active":"2024-04-22T20:48:17.949293+0000","last_peered":"2024-04-22T20:48:17.949293+0000","last_clean":"2024-04-22T20:48:17.949293+0000","last_became_active":"2024-04-22T19:51:26.502438+0000","last_became_peered":"2024-04-22T19:51:26.502438+0000","last_unstale":"2024-04-22T20:48:17.949293+0000","last_undegraded":"2024-04-22T20:48:17.949293+0000","last_fullsized":"2024-04-22T20:48:17.949293+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194332,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":386,"num_read_kb":36638,"num_write":936,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"261'374","reported_seq":1254,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.933037+0000","last_change":"2024-04-22T19:51:26.505871+0000","last_active":"2024-04-22T20:48:17.933037+0000","last_peered":"2024-04-22T20:48:17.933037+0000","last_clean":"2024-04-22T20:48:17.933037+0000","last_became_active":"2024-04-22T19:51:26.504878+0000","last_became_peered":"2024-04-22T19:51:26.504878+0000","last_unstale":"2024-04-22T20:48:17.933037+0000","last_undegraded":"2024-04-22T20:48:17.933037+0000","last_fullsized":"2024-04-22T20:48:17.933037+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5353839,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1018,"num_read_kb":55238,"num_write":1407,"num_write_kb":52838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"261'127","reported_seq":729,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:15.193968+0000","last_change":"2024-04-22T19:51:28.521641+0000","last_active":"2024-04-22T20:47:15.193968+0000","last_peered":"2024-04-22T20:47:15.193968+0000","last_clean":"2024-04-22T20:47:15.193968+0000","last_became_active":"2024-04-22T19:51:28.521372+0000","last_became_peered":"2024-04-22T19:51:28.521372+0000","last_unstale":"2024-04-22T20:47:15.193968+0000","last_undegraded":"2024-04-22T20:47:15.193968+0000","last_fullsized":"2024-04-22T20:47:15.193968+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":525,"num_read_kb":437,"num_write":198,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"261'287","reported_seq":950,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.495078+0000","last_change":"2024-04-22T20:48:28.495119+0000","last_active":"2024-04-22T20:48:28.495078+0000","last_peered":"2024-04-22T20:48:28.495078+0000","last_clean":"2024-04-22T20:48:28.495078+0000","last_became_active":"2024-04-22T19:52:04.618273+0000","last_became_peered":"2024-04-22T19:52:04.618273+0000","last_unstale":"2024-04-22T20:48:28.495078+0000","last_undegraded":"2024-04-22T20:48:28.495078+0000","last_fullsized":"2024-04-22T20:48:28.495078+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'287","last_scrub_stamp":"2024-04-22T20:48:28.494984+0000","last_deep_scrub":"261'287","last_deep_scrub_stamp":"2024-04-22T20:48:28.494984+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.494984+0000","objects_scrubbed":7,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:31:35.950376+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1439002,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":73466,"num_write":989,"num_write_kb":40708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"261'95","reported_seq":649,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.202078+0000","last_change":"2024-04-22T19:51:28.513718+0000","last_active":"2024-04-22T20:47:05.202078+0000","last_peered":"2024-04-22T20:47:05.202078+0000","last_clean":"2024-04-22T20:47:05.202078+0000","last_became_active":"2024-04-22T19:51:28.513450+0000","last_became_peered":"2024-04-22T19:51:28.513450+0000","last_unstale":"2024-04-22T20:47:05.202078+0000","last_undegraded":"2024-04-22T20:47:05.202078+0000","last_fullsized":"2024-04-22T20:47:05.202078+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":382,"num_read_kb":302,"num_write":152,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5385,"num_read_kb":3813,"num_write":2124,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7203,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19082,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":13196,"num_read_kb":13004,"num_write":15956,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2753,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":38164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1917,"ondisk_log_size":1917,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":378,"ondisk_log_size":378,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1476154,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":17118,"num_read_kb":35541,"num_write":14403,"num_write_kb":11173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":56812,"num_omap_keys":128,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1863,"ondisk_log_size":1863,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1622,"num_read_kb":1927,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":85960,"num_read_kb":69763,"num_write":34673,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3890,"ondisk_log_size":3890,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":231670890,"num_objects":469,"num_object_clones":0,"num_object_copies":1407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":469,"num_whiteouts":0,"num_read":35099,"num_read_kb":2217057,"num_write":77336,"num_write_kb":2138644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":349753344,"data_stored":349753344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3387,"ondisk_log_size":3387,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":169,"num_read_kb":196064,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":185784,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132,"ondisk_log_size":132,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575541,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":339124,"kb_used_data":51116,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":94032716,"statfs":{"total":96636764160,"available":96289501184,"internally_reserved":0,"allocated":52342784,"data_stored":50369134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":294910410},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575543,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316900,"kb_used_data":46940,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":94054940,"statfs":{"total":96636764160,"available":96312258560,"internally_reserved":0,"allocated":48066560,"data_stored":46141552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":276429254},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575542,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175548,"kb_used_data":54196,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94196292,"statfs":{"total":96636764160,"available":96457003008,"internally_reserved":0,"allocated":55496704,"data_stored":53487509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575543,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145528,"kb_used_data":51888,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94226312,"statfs":{"total":96636764160,"available":96487743488,"internally_reserved":0,"allocated":53133312,"data_stored":51218557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575543,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":336520,"kb_used_data":62464,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94035320,"statfs":{"total":96636764160,"available":96292167680,"internally_reserved":0,"allocated":63963136,"data_stored":62012545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":280623560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575544,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144372,"kb_used_data":59116,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94227468,"statfs":{"total":96636764160,"available":96488927232,"internally_reserved":0,"allocated":60534784,"data_stored":58629075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575541,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":188908,"kb_used_data":53668,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94182932,"statfs":{"total":96636764160,"available":96443322368,"internally_reserved":0,"allocated":54956032,"data_stored":53018011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575543,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305392,"kb_used_data":61800,"kb_used_omap":1,"kb_used_meta":243582,"kb_avail":94066448,"statfs":{"total":96636764160,"available":96324042752,"internally_reserved":0,"allocated":63283200,"data_stored":61318787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":249428422},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":48562176,"data_stored":48562176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44916736,"data_stored":44916736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37486592,"data_stored":37486592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50307072,"data_stored":50307072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36368384,"data_stored":36368384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44924928,"data_stored":44924928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":43872256,"data_stored":43872256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":43315200,"data_stored":43315200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":5948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":4195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":3595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":6129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":4904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":3145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13631488,"data_stored":13631488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":9437184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T20:48:46.077 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.077 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.078 INFO:tasks.ceph:pgid 68.1b last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.078 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.078 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.078 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.079 INFO:tasks.ceph:pgid 66.1e last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.079 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.079 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.079 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.079 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.080 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.080 INFO:tasks.ceph:pgid 67.18 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.080 INFO:tasks.ceph:pgid 66.19 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.080 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.081 INFO:tasks.ceph:pgid 67.1b last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.081 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.081 INFO:tasks.ceph:pgid 67.1a last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.081 INFO:tasks.ceph:pgid 66.1b last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.081 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.082 INFO:tasks.ceph:pgid 67.15 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.082 INFO:tasks.ceph:pgid 68.13 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.082 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.083 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.083 INFO:tasks.ceph:pgid 67.17 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.083 INFO:tasks.ceph:pgid 66.16 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.083 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.083 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.084 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.084 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.084 INFO:tasks.ceph:pgid 66.10 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.084 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.084 INFO:tasks.ceph:pgid 67.10 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.085 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.085 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.085 INFO:tasks.ceph:pgid 67.12 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.085 INFO:tasks.ceph:pgid 68.a last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.085 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.086 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.086 INFO:tasks.ceph:pgid 66.d last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.086 INFO:tasks.ceph:pgid 68.8 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.086 INFO:tasks.ceph:pgid 67.f last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.086 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.087 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.087 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.087 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.087 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.087 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.088 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.088 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.088 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.088 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.089 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.089 INFO:tasks.ceph:pgid 66.5 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.089 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.089 INFO:tasks.ceph:pgid 67.7 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.089 INFO:tasks.ceph:pgid 66.6 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.090 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.090 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.090 INFO:tasks.ceph:pgid 68.6 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.090 INFO:tasks.ceph:pgid 67.1 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.090 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.091 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.091 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.091 INFO:tasks.ceph:pgid 67.3 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.091 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.091 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.092 INFO:tasks.ceph:pgid 66.3 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.092 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.092 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.092 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.092 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.093 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.093 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.093 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.093 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.093 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.093 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.094 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.094 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.094 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.094 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.094 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.095 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.095 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.095 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.095 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.095 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.095 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.096 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.096 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.096 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.096 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.096 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.096 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.097 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.097 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.097 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.097 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.097 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.098 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.098 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.098 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.098 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.098 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.098 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.099 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.099 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.099 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.099 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.099 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.100 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.100 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.100 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.100 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.100 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.100 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.101 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.101 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.101 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.101 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.101 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.102 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.102 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.102 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.102 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.102 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.103 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.103 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.103 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.103 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.103 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.103 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.104 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.104 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.104 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.104 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.104 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.105 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.105 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.105 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.105 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.105 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.105 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.106 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.106 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.106 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.106 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.106 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.106 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.107 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.107 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.107 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.107 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.107 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.107 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.108 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.108 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.108 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.108 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-22T19:51:11.423705+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=11, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.108 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.109 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.109 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.109 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.109 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.109 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.110 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.110 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.110 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.110 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.110 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.110 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.111 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.111 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.111 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.111 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.111 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.112 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.112 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.112 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.112 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.112 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.113 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.113 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.113 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.113 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.113 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.114 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.114 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.114 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.114 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.114 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.115 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.115 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.115 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.115 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.116 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.116 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.116 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.116 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.116 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.116 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.117 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.117 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.117 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.117 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.117 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.117 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.118 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.118 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.118 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.118 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.118 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.119 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.119 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.119 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.119 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.119 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.120 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.120 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.120 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.120 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.120 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.120 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.121 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.121 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.121 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.121 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.121 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.122 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-22T19:51:35.544909+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=35, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.122 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.122 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.122 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-22T19:51:33.531091+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=33, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.122 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.123 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.123 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.123 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.123 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.123 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.124 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.124 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.124 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.124 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.124 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.124 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.125 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.125 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.125 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.125 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.125 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.125 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.126 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.126 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.126 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.126 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.126 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.127 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.127 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.127 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.127 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.127 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:48:46.127 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-22T20:49:06.128 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T20:49:06.468 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:49:06.480 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T20:49:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":1992,"stamp":"2024-04-22T20:49:06.348002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":267312719,"num_objects":825,"num_object_clones":0,"num_object_copies":2119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":158844,"num_read_kb":2723065,"num_write":145049,"num_write_kb":2429940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219792,"num_omap_keys":610,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12829,"ondisk_log_size":12829,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1956452,"kb_used_data":441188,"kb_used_omap":12,"kb_used_meta":1515187,"kb_avail":753018268,"statfs":{"total":773094113280,"available":771090706432,"internally_reserved":0,"allocated":451776512,"data_stored":436195170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1551552050},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":18263,"num_omap_keys":55,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005020"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:50.199771+0000","last_change":"2024-04-22T20:19:16.796805+0000","last_active":"2024-04-22T20:46:50.199771+0000","last_peered":"2024-04-22T20:46:50.199771+0000","last_clean":"2024-04-22T20:46:50.199771+0000","last_became_active":"2024-04-22T20:19:16.796520+0000","last_became_peered":"2024-04-22T20:19:16.796520+0000","last_unstale":"2024-04-22T20:46:50.199771+0000","last_undegraded":"2024-04-22T20:46:50.199771+0000","last_fullsized":"2024-04-22T20:46:50.199771+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"261'5","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.124617+0000","last_change":"2024-04-22T20:05:17.900640+0000","last_active":"2024-04-22T20:48:17.124617+0000","last_peered":"2024-04-22T20:48:17.124617+0000","last_clean":"2024-04-22T20:48:17.124617+0000","last_became_active":"2024-04-22T20:05:17.900325+0000","last_became_peered":"2024-04-22T20:05:17.900325+0000","last_unstale":"2024-04-22T20:48:17.124617+0000","last_undegraded":"2024-04-22T20:48:17.124617+0000","last_fullsized":"2024-04-22T20:48:17.124617+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.497585+0000","last_change":"2024-04-22T20:48:24.497620+0000","last_active":"2024-04-22T20:48:24.497585+0000","last_peered":"2024-04-22T20:48:24.497585+0000","last_clean":"2024-04-22T20:48:24.497585+0000","last_became_active":"2024-04-22T20:05:13.812933+0000","last_became_peered":"2024-04-22T20:05:13.812933+0000","last_unstale":"2024-04-22T20:48:24.497585+0000","last_undegraded":"2024-04-22T20:48:24.497585+0000","last_fullsized":"2024-04-22T20:48:24.497585+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:24.497472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:24.497472+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.497472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:19.340438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"261'20","reported_seq":110,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.612368+0000","last_change":"2024-04-22T20:48:24.612368+0000","last_active":"2024-04-22T20:48:24.612368+0000","last_peered":"2024-04-22T20:48:24.612368+0000","last_clean":"2024-04-22T20:48:24.612368+0000","last_became_active":"2024-04-22T20:01:04.875981+0000","last_became_peered":"2024-04-22T20:01:04.875981+0000","last_unstale":"2024-04-22T20:48:24.612368+0000","last_undegraded":"2024-04-22T20:48:24.612368+0000","last_fullsized":"2024-04-22T20:48:24.612368+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'20","last_scrub_stamp":"2024-04-22T20:48:24.612214+0000","last_deep_scrub":"261'20","last_deep_scrub_stamp":"2024-04-22T20:48:24.612214+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.612214+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:22:32.284247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":32,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.544945+0000","last_change":"2024-04-22T20:48:50.544978+0000","last_active":"2024-04-22T20:48:50.544945+0000","last_peered":"2024-04-22T20:48:50.544945+0000","last_clean":"2024-04-22T20:48:50.544945+0000","last_became_active":"2024-04-22T20:19:16.790377+0000","last_became_peered":"2024-04-22T20:19:16.790377+0000","last_unstale":"2024-04-22T20:48:50.544945+0000","last_undegraded":"2024-04-22T20:48:50.544945+0000","last_fullsized":"2024-04-22T20:48:50.544945+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.544857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.544857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.544857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:18:08.988881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":40,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:50.199235+0000","last_change":"2024-04-22T20:05:17.899675+0000","last_active":"2024-04-22T20:47:50.199235+0000","last_peered":"2024-04-22T20:47:50.199235+0000","last_clean":"2024-04-22T20:47:50.199235+0000","last_became_active":"2024-04-22T20:05:17.899338+0000","last_became_peered":"2024-04-22T20:05:17.899338+0000","last_unstale":"2024-04-22T20:47:50.199235+0000","last_undegraded":"2024-04-22T20:47:50.199235+0000","last_fullsized":"2024-04-22T20:47:50.199235+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"258'4","reported_seq":62,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.584270+0000","last_change":"2024-04-22T20:48:23.584307+0000","last_active":"2024-04-22T20:48:23.584270+0000","last_peered":"2024-04-22T20:48:23.584270+0000","last_clean":"2024-04-22T20:48:23.584270+0000","last_became_active":"2024-04-22T20:05:13.807231+0000","last_became_peered":"2024-04-22T20:05:13.807231+0000","last_unstale":"2024-04-22T20:48:23.584270+0000","last_undegraded":"2024-04-22T20:48:23.584270+0000","last_fullsized":"2024-04-22T20:48:23.584270+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:23.584182+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:23.584182+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.584182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:34.394463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"261'22","reported_seq":110,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.458354+0000","last_change":"2024-04-22T20:48:34.458388+0000","last_active":"2024-04-22T20:48:34.458354+0000","last_peered":"2024-04-22T20:48:34.458354+0000","last_clean":"2024-04-22T20:48:34.458354+0000","last_became_active":"2024-04-22T20:01:04.874575+0000","last_became_peered":"2024-04-22T20:01:04.874575+0000","last_unstale":"2024-04-22T20:48:34.458354+0000","last_undegraded":"2024-04-22T20:48:34.458354+0000","last_fullsized":"2024-04-22T20:48:34.458354+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'22","last_scrub_stamp":"2024-04-22T20:48:34.458264+0000","last_deep_scrub":"261'22","last_deep_scrub_stamp":"2024-04-22T20:48:34.458264+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.458264+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:46:15.278433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":48,"num_write":47,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.725001+0000","last_change":"2024-04-22T20:49:03.725044+0000","last_active":"2024-04-22T20:49:03.725001+0000","last_peered":"2024-04-22T20:49:03.725001+0000","last_clean":"2024-04-22T20:49:03.725001+0000","last_became_active":"2024-04-22T20:19:16.794700+0000","last_became_peered":"2024-04-22T20:19:16.794700+0000","last_unstale":"2024-04-22T20:49:03.725001+0000","last_undegraded":"2024-04-22T20:49:03.725001+0000","last_fullsized":"2024-04-22T20:49:03.725001+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:03.724898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:03.724898+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.724898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:21:17.669058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":44,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.034570+0000","last_change":"2024-04-22T20:05:18.211663+0000","last_active":"2024-04-22T20:48:22.034570+0000","last_peered":"2024-04-22T20:48:22.034570+0000","last_clean":"2024-04-22T20:48:22.034570+0000","last_became_active":"2024-04-22T20:05:18.211400+0000","last_became_peered":"2024-04-22T20:05:18.211400+0000","last_unstale":"2024-04-22T20:48:22.034570+0000","last_undegraded":"2024-04-22T20:48:22.034570+0000","last_fullsized":"2024-04-22T20:48:22.034570+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"261'2","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.570825+0000","last_change":"2024-04-22T20:48:46.570845+0000","last_active":"2024-04-22T20:48:46.570825+0000","last_peered":"2024-04-22T20:48:46.570825+0000","last_clean":"2024-04-22T20:48:46.570825+0000","last_became_active":"2024-04-22T20:05:13.809349+0000","last_became_peered":"2024-04-22T20:05:13.809349+0000","last_unstale":"2024-04-22T20:48:46.570825+0000","last_undegraded":"2024-04-22T20:48:46.570825+0000","last_fullsized":"2024-04-22T20:48:46.570825+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:46.570725+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:46.570725+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.570725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:00.895789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"261'24","reported_seq":124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.511512+0000","last_change":"2024-04-22T20:48:31.511512+0000","last_active":"2024-04-22T20:48:31.511512+0000","last_peered":"2024-04-22T20:48:31.511512+0000","last_clean":"2024-04-22T20:48:31.511512+0000","last_became_active":"2024-04-22T20:01:04.869785+0000","last_became_peered":"2024-04-22T20:01:04.869785+0000","last_unstale":"2024-04-22T20:48:31.511512+0000","last_undegraded":"2024-04-22T20:48:31.511512+0000","last_fullsized":"2024-04-22T20:48:31.511512+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'24","last_scrub_stamp":"2024-04-22T20:48:31.511334+0000","last_deep_scrub":"261'24","last_deep_scrub_stamp":"2024-04-22T20:48:31.511334+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.511334+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:43:54.338193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":82,"num_write":36,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:20.176404+0000","last_change":"2024-04-22T20:19:16.795441+0000","last_active":"2024-04-22T20:48:20.176404+0000","last_peered":"2024-04-22T20:48:20.176404+0000","last_clean":"2024-04-22T20:48:20.176404+0000","last_became_active":"2024-04-22T20:19:16.793667+0000","last_became_peered":"2024-04-22T20:19:16.793667+0000","last_unstale":"2024-04-22T20:48:20.176404+0000","last_undegraded":"2024-04-22T20:48:20.176404+0000","last_fullsized":"2024-04-22T20:48:20.176404+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"261'10","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.491093+0000","last_change":"2024-04-22T20:06:05.349958+0000","last_active":"2024-04-22T20:48:17.491093+0000","last_peered":"2024-04-22T20:48:17.491093+0000","last_clean":"2024-04-22T20:48:17.491093+0000","last_became_active":"2024-04-22T20:06:05.345554+0000","last_became_peered":"2024-04-22T20:06:05.345554+0000","last_unstale":"2024-04-22T20:48:17.491093+0000","last_undegraded":"2024-04-22T20:48:17.491093+0000","last_fullsized":"2024-04-22T20:48:17.491093+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":20,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1f","version":"261'10","reported_seq":78,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.367896+0000","last_change":"2024-04-22T20:48:36.367896+0000","last_active":"2024-04-22T20:48:36.367896+0000","last_peered":"2024-04-22T20:48:36.367896+0000","last_clean":"2024-04-22T20:48:36.367896+0000","last_became_active":"2024-04-22T20:05:13.806246+0000","last_became_peered":"2024-04-22T20:05:13.806246+0000","last_unstale":"2024-04-22T20:48:36.367896+0000","last_undegraded":"2024-04-22T20:48:36.367896+0000","last_fullsized":"2024-04-22T20:48:36.367896+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:48:36.367701+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:48:36.367701+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.367701+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:06:18.240490+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":16720,"num_write":13,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"261'59","reported_seq":208,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.583472+0000","last_change":"2024-04-22T20:48:30.583472+0000","last_active":"2024-04-22T20:48:30.583472+0000","last_peered":"2024-04-22T20:48:30.583472+0000","last_clean":"2024-04-22T20:48:30.583472+0000","last_became_active":"2024-04-22T20:01:04.873899+0000","last_became_peered":"2024-04-22T20:01:04.873899+0000","last_unstale":"2024-04-22T20:48:30.583472+0000","last_undegraded":"2024-04-22T20:48:30.583472+0000","last_fullsized":"2024-04-22T20:48:30.583472+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'59","last_scrub_stamp":"2024-04-22T20:48:30.583311+0000","last_deep_scrub":"261'59","last_deep_scrub_stamp":"2024-04-22T20:48:30.583311+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.583311+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:58:29.692843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":183,"num_write":99,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.169419+0000","last_change":"2024-04-22T20:19:16.794646+0000","last_active":"2024-04-22T20:47:20.169419+0000","last_peered":"2024-04-22T20:47:20.169419+0000","last_clean":"2024-04-22T20:47:20.169419+0000","last_became_active":"2024-04-22T20:19:16.794110+0000","last_became_peered":"2024-04-22T20:19:16.794110+0000","last_unstale":"2024-04-22T20:47:20.169419+0000","last_undegraded":"2024-04-22T20:47:20.169419+0000","last_fullsized":"2024-04-22T20:47:20.169419+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"261'2","reported_seq":45,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.335373+0000","last_change":"2024-04-22T20:05:17.898277+0000","last_active":"2024-04-22T20:48:24.335373+0000","last_peered":"2024-04-22T20:48:24.335373+0000","last_clean":"2024-04-22T20:48:24.335373+0000","last_became_active":"2024-04-22T20:05:17.897771+0000","last_became_peered":"2024-04-22T20:05:17.897771+0000","last_unstale":"2024-04-22T20:48:24.335373+0000","last_undegraded":"2024-04-22T20:48:24.335373+0000","last_fullsized":"2024-04-22T20:48:24.335373+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"261'6","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.520623+0000","last_change":"2024-04-22T20:48:29.520623+0000","last_active":"2024-04-22T20:48:29.520623+0000","last_peered":"2024-04-22T20:48:29.520623+0000","last_clean":"2024-04-22T20:48:29.520623+0000","last_became_active":"2024-04-22T20:05:13.812308+0000","last_became_peered":"2024-04-22T20:05:13.812308+0000","last_unstale":"2024-04-22T20:48:29.520623+0000","last_undegraded":"2024-04-22T20:48:29.520623+0000","last_fullsized":"2024-04-22T20:48:29.520623+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:29.520502+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:29.520502+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.520502+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:11:42.190343+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":9,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"261'8","reported_seq":82,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.526061+0000","last_change":"2024-04-22T20:48:25.526061+0000","last_active":"2024-04-22T20:48:25.526061+0000","last_peered":"2024-04-22T20:48:25.526061+0000","last_clean":"2024-04-22T20:48:25.526061+0000","last_became_active":"2024-04-22T20:01:04.877209+0000","last_became_peered":"2024-04-22T20:01:04.877209+0000","last_unstale":"2024-04-22T20:48:25.526061+0000","last_undegraded":"2024-04-22T20:48:25.526061+0000","last_fullsized":"2024-04-22T20:48:25.526061+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'8","last_scrub_stamp":"2024-04-22T20:48:25.525971+0000","last_deep_scrub":"261'8","last_deep_scrub_stamp":"2024-04-22T20:48:25.525971+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.525971+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:37.255572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.626708+0000","last_change":"2024-04-22T20:49:02.626730+0000","last_active":"2024-04-22T20:49:02.626708+0000","last_peered":"2024-04-22T20:49:02.626708+0000","last_clean":"2024-04-22T20:49:02.626708+0000","last_became_active":"2024-04-22T20:19:16.789599+0000","last_became_peered":"2024-04-22T20:19:16.789599+0000","last_unstale":"2024-04-22T20:49:02.626708+0000","last_undegraded":"2024-04-22T20:49:02.626708+0000","last_fullsized":"2024-04-22T20:49:02.626708+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.626625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.626625+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.626625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:06:20.379636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"261'2","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.611806+0000","last_change":"2024-04-22T20:48:59.611832+0000","last_active":"2024-04-22T20:48:59.611806+0000","last_peered":"2024-04-22T20:48:59.611806+0000","last_clean":"2024-04-22T20:48:59.611806+0000","last_became_active":"2024-04-22T20:05:17.890841+0000","last_became_peered":"2024-04-22T20:05:17.890841+0000","last_unstale":"2024-04-22T20:48:59.611806+0000","last_undegraded":"2024-04-22T20:48:59.611806+0000","last_fullsized":"2024-04-22T20:48:59.611806+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:59.611713+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:59.611713+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.611713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:39:18.164914+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.299460+0000","last_change":"2024-04-22T20:48:40.299460+0000","last_active":"2024-04-22T20:48:40.299460+0000","last_peered":"2024-04-22T20:48:40.299460+0000","last_clean":"2024-04-22T20:48:40.299460+0000","last_became_active":"2024-04-22T20:05:13.807516+0000","last_became_peered":"2024-04-22T20:05:13.807516+0000","last_unstale":"2024-04-22T20:48:40.299460+0000","last_undegraded":"2024-04-22T20:48:40.299460+0000","last_fullsized":"2024-04-22T20:48:40.299460+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.299341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.299341+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.299341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:32:01.048139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"261'77","reported_seq":249,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.378570+0000","last_change":"2024-04-22T20:48:26.378570+0000","last_active":"2024-04-22T20:48:26.378570+0000","last_peered":"2024-04-22T20:48:26.378570+0000","last_clean":"2024-04-22T20:48:26.378570+0000","last_became_active":"2024-04-22T20:01:04.873015+0000","last_became_peered":"2024-04-22T20:01:04.873015+0000","last_unstale":"2024-04-22T20:48:26.378570+0000","last_undegraded":"2024-04-22T20:48:26.378570+0000","last_fullsized":"2024-04-22T20:48:26.378570+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'77","last_scrub_stamp":"2024-04-22T20:48:26.378447+0000","last_deep_scrub":"261'77","last_deep_scrub_stamp":"2024-04-22T20:48:26.378447+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.378447+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:22.636845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":236,"num_write":125,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"261'2","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.451419+0000","last_change":"2024-04-22T20:48:48.451468+0000","last_active":"2024-04-22T20:48:48.451419+0000","last_peered":"2024-04-22T20:48:48.451419+0000","last_clean":"2024-04-22T20:48:48.451419+0000","last_became_active":"2024-04-22T20:19:16.794564+0000","last_became_peered":"2024-04-22T20:19:16.794564+0000","last_unstale":"2024-04-22T20:48:48.451419+0000","last_undegraded":"2024-04-22T20:48:48.451419+0000","last_fullsized":"2024-04-22T20:48:48.451419+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:48.451302+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:48.451302+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.451302+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:04:33.139707+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"261'5","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.663979+0000","last_change":"2024-04-22T20:49:01.664006+0000","last_active":"2024-04-22T20:49:01.663979+0000","last_peered":"2024-04-22T20:49:01.663979+0000","last_clean":"2024-04-22T20:49:01.663979+0000","last_became_active":"2024-04-22T20:05:17.901446+0000","last_became_peered":"2024-04-22T20:05:17.901446+0000","last_unstale":"2024-04-22T20:49:01.663979+0000","last_undegraded":"2024-04-22T20:49:01.663979+0000","last_fullsized":"2024-04-22T20:49:01.663979+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:01.663883+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:01.663883+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.663883+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:07:50.105267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":11264,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.351494+0000","last_change":"2024-04-22T20:48:28.351557+0000","last_active":"2024-04-22T20:48:28.351494+0000","last_peered":"2024-04-22T20:48:28.351494+0000","last_clean":"2024-04-22T20:48:28.351494+0000","last_became_active":"2024-04-22T20:05:13.798311+0000","last_became_peered":"2024-04-22T20:05:13.798311+0000","last_unstale":"2024-04-22T20:48:28.351494+0000","last_undegraded":"2024-04-22T20:48:28.351494+0000","last_fullsized":"2024-04-22T20:48:28.351494+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:28.351390+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:28.351390+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.351390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:49:42.653069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'27","reported_seq":125,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.545417+0000","last_change":"2024-04-22T20:48:28.545440+0000","last_active":"2024-04-22T20:48:28.545417+0000","last_peered":"2024-04-22T20:48:28.545417+0000","last_clean":"2024-04-22T20:48:28.545417+0000","last_became_active":"2024-04-22T20:01:04.873426+0000","last_became_peered":"2024-04-22T20:01:04.873426+0000","last_unstale":"2024-04-22T20:48:28.545417+0000","last_undegraded":"2024-04-22T20:48:28.545417+0000","last_fullsized":"2024-04-22T20:48:28.545417+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'27","last_scrub_stamp":"2024-04-22T20:48:28.545320+0000","last_deep_scrub":"261'27","last_deep_scrub_stamp":"2024-04-22T20:48:28.545320+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.545320+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:51:52.257563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":80,"num_write":54,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":24,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:46:50.076263+0000","last_change":"2024-04-22T20:19:16.800874+0000","last_active":"2024-04-22T20:46:50.076263+0000","last_peered":"2024-04-22T20:46:50.076263+0000","last_clean":"2024-04-22T20:46:50.076263+0000","last_became_active":"2024-04-22T20:19:16.800611+0000","last_became_peered":"2024-04-22T20:19:16.800611+0000","last_unstale":"2024-04-22T20:46:50.076263+0000","last_undegraded":"2024-04-22T20:46:50.076263+0000","last_fullsized":"2024-04-22T20:46:50.076263+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"258'4","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.512936+0000","last_change":"2024-04-22T20:48:44.512963+0000","last_active":"2024-04-22T20:48:44.512936+0000","last_peered":"2024-04-22T20:48:44.512936+0000","last_clean":"2024-04-22T20:48:44.512936+0000","last_became_active":"2024-04-22T20:05:17.892624+0000","last_became_peered":"2024-04-22T20:05:17.892624+0000","last_unstale":"2024-04-22T20:48:44.512936+0000","last_undegraded":"2024-04-22T20:48:44.512936+0000","last_fullsized":"2024-04-22T20:48:44.512936+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:44.512843+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:44.512843+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.512843+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:13:48.032126+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3072,"num_write":4,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.574707+0000","last_change":"2024-04-22T20:48:41.574733+0000","last_active":"2024-04-22T20:48:41.574707+0000","last_peered":"2024-04-22T20:48:41.574707+0000","last_clean":"2024-04-22T20:48:41.574707+0000","last_became_active":"2024-04-22T20:05:13.816011+0000","last_became_peered":"2024-04-22T20:05:13.816011+0000","last_unstale":"2024-04-22T20:48:41.574707+0000","last_undegraded":"2024-04-22T20:48:41.574707+0000","last_fullsized":"2024-04-22T20:48:41.574707+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:41.574619+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:41.574619+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.574619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:10:05.304886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"261'43","reported_seq":159,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.562937+0000","last_change":"2024-04-22T20:48:25.562960+0000","last_active":"2024-04-22T20:48:25.562937+0000","last_peered":"2024-04-22T20:48:25.562937+0000","last_clean":"2024-04-22T20:48:25.562937+0000","last_became_active":"2024-04-22T20:01:04.872356+0000","last_became_peered":"2024-04-22T20:01:04.872356+0000","last_unstale":"2024-04-22T20:48:25.562937+0000","last_undegraded":"2024-04-22T20:48:25.562937+0000","last_fullsized":"2024-04-22T20:48:25.562937+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'43","last_scrub_stamp":"2024-04-22T20:48:25.562858+0000","last_deep_scrub":"261'43","last_deep_scrub_stamp":"2024-04-22T20:48:25.562858+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.562858+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:39:26.408917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":119,"num_write":72,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.688311+0000","last_change":"2024-04-22T20:49:04.688337+0000","last_active":"2024-04-22T20:49:04.688311+0000","last_peered":"2024-04-22T20:49:04.688311+0000","last_clean":"2024-04-22T20:49:04.688311+0000","last_became_active":"2024-04-22T20:19:16.796155+0000","last_became_peered":"2024-04-22T20:19:16.796155+0000","last_unstale":"2024-04-22T20:49:04.688311+0000","last_undegraded":"2024-04-22T20:49:04.688311+0000","last_fullsized":"2024-04-22T20:49:04.688311+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:04.688168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:04.688168+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.688168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:59.083750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.425810+0000","last_change":"2024-04-22T20:48:52.425835+0000","last_active":"2024-04-22T20:48:52.425810+0000","last_peered":"2024-04-22T20:48:52.425810+0000","last_clean":"2024-04-22T20:48:52.425810+0000","last_became_active":"2024-04-22T20:05:18.209023+0000","last_became_peered":"2024-04-22T20:05:18.209023+0000","last_unstale":"2024-04-22T20:48:52.425810+0000","last_undegraded":"2024-04-22T20:48:52.425810+0000","last_fullsized":"2024-04-22T20:48:52.425810+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.425722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.425722+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.425722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:32:32.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"261'7","reported_seq":70,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.910170+0000","last_change":"2024-04-22T20:48:39.910210+0000","last_active":"2024-04-22T20:48:39.910170+0000","last_peered":"2024-04-22T20:48:39.910170+0000","last_clean":"2024-04-22T20:48:39.910170+0000","last_became_active":"2024-04-22T20:05:13.809548+0000","last_became_peered":"2024-04-22T20:05:13.809548+0000","last_unstale":"2024-04-22T20:48:39.910170+0000","last_undegraded":"2024-04-22T20:48:39.910170+0000","last_fullsized":"2024-04-22T20:48:39.910170+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:39.910080+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:39.910080+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.910080+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:39:16.656357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":6146,"num_write":5,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'64","reported_seq":251,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.544212+0000","last_change":"2024-04-22T20:48:30.544266+0000","last_active":"2024-04-22T20:48:30.544212+0000","last_peered":"2024-04-22T20:48:30.544212+0000","last_clean":"2024-04-22T20:48:30.544212+0000","last_became_active":"2024-04-22T20:01:04.873236+0000","last_became_peered":"2024-04-22T20:01:04.873236+0000","last_unstale":"2024-04-22T20:48:30.544212+0000","last_undegraded":"2024-04-22T20:48:30.544212+0000","last_fullsized":"2024-04-22T20:48:30.544212+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'64","last_scrub_stamp":"2024-04-22T20:48:30.544111+0000","last_deep_scrub":"261'64","last_deep_scrub_stamp":"2024-04-22T20:48:30.544111+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.544111+0000","objects_scrubbed":1,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:51.536623+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":250,"num_write":111,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3950,"num_omap_keys":8,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"261'2","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.590705+0000","last_change":"2024-04-22T20:48:51.590731+0000","last_active":"2024-04-22T20:48:51.590705+0000","last_peered":"2024-04-22T20:48:51.590705+0000","last_clean":"2024-04-22T20:48:51.590705+0000","last_became_active":"2024-04-22T20:19:16.791597+0000","last_became_peered":"2024-04-22T20:19:16.791597+0000","last_unstale":"2024-04-22T20:48:51.590705+0000","last_undegraded":"2024-04-22T20:48:51.590705+0000","last_fullsized":"2024-04-22T20:48:51.590705+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:51.590624+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:51.590624+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.590624+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:08.152952+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.169871+0000","last_change":"2024-04-22T20:05:18.211042+0000","last_active":"2024-04-22T20:47:25.169871+0000","last_peered":"2024-04-22T20:47:25.169871+0000","last_clean":"2024-04-22T20:47:25.169871+0000","last_became_active":"2024-04-22T20:05:18.210704+0000","last_became_peered":"2024-04-22T20:05:18.210704+0000","last_unstale":"2024-04-22T20:47:25.169871+0000","last_undegraded":"2024-04-22T20:47:25.169871+0000","last_fullsized":"2024-04-22T20:47:25.169871+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.609480+0000","last_change":"2024-04-22T20:48:27.609480+0000","last_active":"2024-04-22T20:48:27.609480+0000","last_peered":"2024-04-22T20:48:27.609480+0000","last_clean":"2024-04-22T20:48:27.609480+0000","last_became_active":"2024-04-22T20:05:13.810902+0000","last_became_peered":"2024-04-22T20:05:13.810902+0000","last_unstale":"2024-04-22T20:48:27.609480+0000","last_undegraded":"2024-04-22T20:48:27.609480+0000","last_fullsized":"2024-04-22T20:48:27.609480+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:27.609361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:27.609361+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.609361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:08.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"250'9","reported_seq":85,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.596475+0000","last_change":"2024-04-22T20:48:27.596526+0000","last_active":"2024-04-22T20:48:27.596475+0000","last_peered":"2024-04-22T20:48:27.596475+0000","last_clean":"2024-04-22T20:48:27.596475+0000","last_became_active":"2024-04-22T20:01:04.874230+0000","last_became_peered":"2024-04-22T20:01:04.874230+0000","last_unstale":"2024-04-22T20:48:27.596475+0000","last_undegraded":"2024-04-22T20:48:27.596475+0000","last_fullsized":"2024-04-22T20:48:27.596475+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"250'9","last_scrub_stamp":"2024-04-22T20:48:27.596354+0000","last_deep_scrub":"250'9","last_deep_scrub_stamp":"2024-04-22T20:48:27.596354+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.596354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:12:20.356156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.080141+0000","last_change":"2024-04-22T20:19:16.795626+0000","last_active":"2024-04-22T20:47:20.080141+0000","last_peered":"2024-04-22T20:47:20.080141+0000","last_clean":"2024-04-22T20:47:20.080141+0000","last_became_active":"2024-04-22T20:19:16.795139+0000","last_became_peered":"2024-04-22T20:19:16.795139+0000","last_unstale":"2024-04-22T20:47:20.080141+0000","last_undegraded":"2024-04-22T20:47:20.080141+0000","last_fullsized":"2024-04-22T20:47:20.080141+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"261'16","reported_seq":77,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.415657+0000","last_change":"2024-04-22T20:05:18.212823+0000","last_active":"2024-04-22T20:48:23.415657+0000","last_peered":"2024-04-22T20:48:23.415657+0000","last_clean":"2024-04-22T20:48:23.415657+0000","last_became_active":"2024-04-22T20:05:18.212336+0000","last_became_peered":"2024-04-22T20:05:18.212336+0000","last_unstale":"2024-04-22T20:48:23.415657+0000","last_undegraded":"2024-04-22T20:48:23.415657+0000","last_fullsized":"2024-04-22T20:48:23.415657+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437184,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":38918,"num_write":23,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.569695+0000","last_change":"2024-04-22T20:48:49.569733+0000","last_active":"2024-04-22T20:48:49.569695+0000","last_peered":"2024-04-22T20:48:49.569695+0000","last_clean":"2024-04-22T20:48:49.569695+0000","last_became_active":"2024-04-22T20:05:13.816036+0000","last_became_peered":"2024-04-22T20:05:13.816036+0000","last_unstale":"2024-04-22T20:48:49.569695+0000","last_undegraded":"2024-04-22T20:48:49.569695+0000","last_fullsized":"2024-04-22T20:48:49.569695+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:49.569601+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:49.569601+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.569601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:07:18.107853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'53","reported_seq":185,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.516499+0000","last_change":"2024-04-22T20:48:36.516524+0000","last_active":"2024-04-22T20:48:36.516499+0000","last_peered":"2024-04-22T20:48:36.516499+0000","last_clean":"2024-04-22T20:48:36.516499+0000","last_became_active":"2024-04-22T20:01:04.878067+0000","last_became_peered":"2024-04-22T20:01:04.878067+0000","last_unstale":"2024-04-22T20:48:36.516499+0000","last_undegraded":"2024-04-22T20:48:36.516499+0000","last_fullsized":"2024-04-22T20:48:36.516499+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'53","last_scrub_stamp":"2024-04-22T20:48:36.516407+0000","last_deep_scrub":"261'53","last_deep_scrub_stamp":"2024-04-22T20:48:36.516407+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.516407+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:26.653699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":150,"num_write":99,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:20.204124+0000","last_change":"2024-04-22T20:19:16.796141+0000","last_active":"2024-04-22T20:47:20.204124+0000","last_peered":"2024-04-22T20:47:20.204124+0000","last_clean":"2024-04-22T20:47:20.204124+0000","last_became_active":"2024-04-22T20:19:16.795855+0000","last_became_peered":"2024-04-22T20:19:16.795855+0000","last_unstale":"2024-04-22T20:47:20.204124+0000","last_undegraded":"2024-04-22T20:47:20.204124+0000","last_fullsized":"2024-04-22T20:47:20.204124+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"0'0","reported_seq":41,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875815+0000","last_change":"2024-04-22T20:05:17.902267+0000","last_active":"2024-04-22T20:48:23.875815+0000","last_peered":"2024-04-22T20:48:23.875815+0000","last_clean":"2024-04-22T20:48:23.875815+0000","last_became_active":"2024-04-22T20:05:17.901954+0000","last_became_peered":"2024-04-22T20:05:17.901954+0000","last_unstale":"2024-04-22T20:48:23.875815+0000","last_undegraded":"2024-04-22T20:48:23.875815+0000","last_fullsized":"2024-04-22T20:48:23.875815+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'4","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.504218+0000","last_change":"2024-04-22T20:48:33.504218+0000","last_active":"2024-04-22T20:48:33.504218+0000","last_peered":"2024-04-22T20:48:33.504218+0000","last_clean":"2024-04-22T20:48:33.504218+0000","last_became_active":"2024-04-22T20:05:13.813256+0000","last_became_peered":"2024-04-22T20:05:13.813256+0000","last_unstale":"2024-04-22T20:48:33.504218+0000","last_undegraded":"2024-04-22T20:48:33.504218+0000","last_fullsized":"2024-04-22T20:48:33.504218+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:33.504106+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:33.504106+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.504106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:41:49.887327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":44,"num_write":4,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"261'46","reported_seq":173,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.448464+0000","last_change":"2024-04-22T20:48:33.448487+0000","last_active":"2024-04-22T20:48:33.448464+0000","last_peered":"2024-04-22T20:48:33.448464+0000","last_clean":"2024-04-22T20:48:33.448464+0000","last_became_active":"2024-04-22T20:01:04.873533+0000","last_became_peered":"2024-04-22T20:01:04.873533+0000","last_unstale":"2024-04-22T20:48:33.448464+0000","last_undegraded":"2024-04-22T20:48:33.448464+0000","last_fullsized":"2024-04-22T20:48:33.448464+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'46","last_scrub_stamp":"2024-04-22T20:48:33.448376+0000","last_deep_scrub":"261'46","last_deep_scrub_stamp":"2024-04-22T20:48:33.448376+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.448376+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:02:25.135711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":192,"num_read_kb":137,"num_write":82,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.175890+0000","last_change":"2024-04-22T20:19:16.793148+0000","last_active":"2024-04-22T20:48:15.175890+0000","last_peered":"2024-04-22T20:48:15.175890+0000","last_clean":"2024-04-22T20:48:15.175890+0000","last_became_active":"2024-04-22T20:19:16.792856+0000","last_became_peered":"2024-04-22T20:19:16.792856+0000","last_unstale":"2024-04-22T20:48:15.175890+0000","last_undegraded":"2024-04-22T20:48:15.175890+0000","last_fullsized":"2024-04-22T20:48:15.175890+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.654837+0000","last_change":"2024-04-22T20:48:59.654866+0000","last_active":"2024-04-22T20:48:59.654837+0000","last_peered":"2024-04-22T20:48:59.654837+0000","last_clean":"2024-04-22T20:48:59.654837+0000","last_became_active":"2024-04-22T20:05:17.897040+0000","last_became_peered":"2024-04-22T20:05:17.897040+0000","last_unstale":"2024-04-22T20:48:59.654837+0000","last_undegraded":"2024-04-22T20:48:59.654837+0000","last_fullsized":"2024-04-22T20:48:59.654837+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-22T20:48:59.654751+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-22T20:48:59.654751+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.654751+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:11.365352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":12290,"num_write":7,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"261'4","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.949605+0000","last_change":"2024-04-22T20:05:13.814475+0000","last_active":"2024-04-22T20:48:16.949605+0000","last_peered":"2024-04-22T20:48:16.949605+0000","last_clean":"2024-04-22T20:48:16.949605+0000","last_became_active":"2024-04-22T20:05:13.814222+0000","last_became_peered":"2024-04-22T20:05:13.814222+0000","last_unstale":"2024-04-22T20:48:16.949605+0000","last_undegraded":"2024-04-22T20:48:16.949605+0000","last_fullsized":"2024-04-22T20:48:16.949605+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"261'54","reported_seq":199,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.410439+0000","last_change":"2024-04-22T20:48:34.410468+0000","last_active":"2024-04-22T20:48:34.410439+0000","last_peered":"2024-04-22T20:48:34.410439+0000","last_clean":"2024-04-22T20:48:34.410439+0000","last_became_active":"2024-04-22T20:01:04.873422+0000","last_became_peered":"2024-04-22T20:01:04.873422+0000","last_unstale":"2024-04-22T20:48:34.410439+0000","last_undegraded":"2024-04-22T20:48:34.410439+0000","last_fullsized":"2024-04-22T20:48:34.410439+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'54","last_scrub_stamp":"2024-04-22T20:48:34.410341+0000","last_deep_scrub":"261'54","last_deep_scrub_stamp":"2024-04-22T20:48:34.410341+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.410341+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:47:24.001212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":184,"num_write":82,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:20.201608+0000","last_change":"2024-04-22T20:19:16.795990+0000","last_active":"2024-04-22T20:48:20.201608+0000","last_peered":"2024-04-22T20:48:20.201608+0000","last_clean":"2024-04-22T20:48:20.201608+0000","last_became_active":"2024-04-22T20:19:16.795633+0000","last_became_peered":"2024-04-22T20:19:16.795633+0000","last_unstale":"2024-04-22T20:48:20.201608+0000","last_undegraded":"2024-04-22T20:48:20.201608+0000","last_fullsized":"2024-04-22T20:48:20.201608+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"261'10","reported_seq":68,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.653814+0000","last_change":"2024-04-22T20:49:00.653851+0000","last_active":"2024-04-22T20:49:00.653814+0000","last_peered":"2024-04-22T20:49:00.653814+0000","last_clean":"2024-04-22T20:49:00.653814+0000","last_became_active":"2024-04-22T20:05:17.899682+0000","last_became_peered":"2024-04-22T20:05:17.899682+0000","last_unstale":"2024-04-22T20:49:00.653814+0000","last_undegraded":"2024-04-22T20:49:00.653814+0000","last_fullsized":"2024-04-22T20:49:00.653814+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:49:00.653727+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:49:00.653727+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.653727+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:51:18.897192+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":8210,"num_write":13,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"261'6","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.625913+0000","last_change":"2024-04-22T20:48:28.625974+0000","last_active":"2024-04-22T20:48:28.625913+0000","last_peered":"2024-04-22T20:48:28.625913+0000","last_clean":"2024-04-22T20:48:28.625913+0000","last_became_active":"2024-04-22T20:05:13.812893+0000","last_became_peered":"2024-04-22T20:05:13.812893+0000","last_unstale":"2024-04-22T20:48:28.625913+0000","last_undegraded":"2024-04-22T20:48:28.625913+0000","last_fullsized":"2024-04-22T20:48:28.625913+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:28.625814+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:28.625814+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.625814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:06.391872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4096,"num_write":16,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"261'47","reported_seq":172,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.628532+0000","last_change":"2024-04-22T20:48:23.628571+0000","last_active":"2024-04-22T20:48:23.628532+0000","last_peered":"2024-04-22T20:48:23.628532+0000","last_clean":"2024-04-22T20:48:23.628532+0000","last_became_active":"2024-04-22T20:01:04.877755+0000","last_became_peered":"2024-04-22T20:01:04.877755+0000","last_unstale":"2024-04-22T20:48:23.628532+0000","last_undegraded":"2024-04-22T20:48:23.628532+0000","last_fullsized":"2024-04-22T20:48:23.628532+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'47","last_scrub_stamp":"2024-04-22T20:48:23.628425+0000","last_deep_scrub":"261'47","last_deep_scrub_stamp":"2024-04-22T20:48:23.628425+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.628425+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:22.945025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":135,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"260'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.195183+0000","last_change":"2024-04-22T20:19:16.793874+0000","last_active":"2024-04-22T20:47:25.195183+0000","last_peered":"2024-04-22T20:47:25.195183+0000","last_clean":"2024-04-22T20:47:25.195183+0000","last_became_active":"2024-04-22T20:19:16.793555+0000","last_became_peered":"2024-04-22T20:19:16.793555+0000","last_unstale":"2024-04-22T20:47:25.195183+0000","last_undegraded":"2024-04-22T20:47:25.195183+0000","last_fullsized":"2024-04-22T20:47:25.195183+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"258'2","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.475045+0000","last_change":"2024-04-22T20:48:36.475071+0000","last_active":"2024-04-22T20:48:36.475045+0000","last_peered":"2024-04-22T20:48:36.475045+0000","last_clean":"2024-04-22T20:48:36.475045+0000","last_became_active":"2024-04-22T20:05:17.890070+0000","last_became_peered":"2024-04-22T20:05:17.890070+0000","last_unstale":"2024-04-22T20:48:36.475045+0000","last_undegraded":"2024-04-22T20:48:36.475045+0000","last_fullsized":"2024-04-22T20:48:36.475045+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'2","last_scrub_stamp":"2024-04-22T20:48:36.474953+0000","last_deep_scrub":"258'2","last_deep_scrub_stamp":"2024-04-22T20:48:36.474953+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.474953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:22:16.222599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"261'5","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.415179+0000","last_change":"2024-04-22T20:48:30.415205+0000","last_active":"2024-04-22T20:48:30.415179+0000","last_peered":"2024-04-22T20:48:30.415179+0000","last_clean":"2024-04-22T20:48:30.415179+0000","last_became_active":"2024-04-22T20:05:13.806882+0000","last_became_peered":"2024-04-22T20:05:13.806882+0000","last_unstale":"2024-04-22T20:48:30.415179+0000","last_undegraded":"2024-04-22T20:48:30.415179+0000","last_fullsized":"2024-04-22T20:48:30.415179+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:30.415089+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:30.415089+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.415089+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:39.469597+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"261'11","reported_seq":89,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.276700+0000","last_change":"2024-04-22T20:48:25.276728+0000","last_active":"2024-04-22T20:48:25.276700+0000","last_peered":"2024-04-22T20:48:25.276700+0000","last_clean":"2024-04-22T20:48:25.276700+0000","last_became_active":"2024-04-22T20:01:04.860991+0000","last_became_peered":"2024-04-22T20:01:04.860991+0000","last_unstale":"2024-04-22T20:48:25.276700+0000","last_undegraded":"2024-04-22T20:48:25.276700+0000","last_fullsized":"2024-04-22T20:48:25.276700+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'11","last_scrub_stamp":"2024-04-22T20:48:25.276600+0000","last_deep_scrub":"261'11","last_deep_scrub_stamp":"2024-04-22T20:48:25.276600+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.276600+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:39:08.585319+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":31,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"261'1","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.175863+0000","last_change":"2024-04-22T20:19:16.794789+0000","last_active":"2024-04-22T20:48:15.175863+0000","last_peered":"2024-04-22T20:48:15.175863+0000","last_clean":"2024-04-22T20:48:15.175863+0000","last_became_active":"2024-04-22T20:19:16.791995+0000","last_became_peered":"2024-04-22T20:19:16.791995+0000","last_unstale":"2024-04-22T20:48:15.175863+0000","last_undegraded":"2024-04-22T20:48:15.175863+0000","last_fullsized":"2024-04-22T20:48:15.175863+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"258'4","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.641951+0000","last_change":"2024-04-22T20:48:58.641997+0000","last_active":"2024-04-22T20:48:58.641951+0000","last_peered":"2024-04-22T20:48:58.641951+0000","last_clean":"2024-04-22T20:48:58.641951+0000","last_became_active":"2024-04-22T20:05:17.890408+0000","last_became_peered":"2024-04-22T20:05:17.890408+0000","last_unstale":"2024-04-22T20:48:58.641951+0000","last_undegraded":"2024-04-22T20:48:58.641951+0000","last_fullsized":"2024-04-22T20:48:58.641951+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:58.641819+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:58.641819+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.641819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:00.757652+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1342,"num_write":4,"num_write_kb":1342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"258'4","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.956463+0000","last_change":"2024-04-22T20:48:35.956522+0000","last_active":"2024-04-22T20:48:35.956463+0000","last_peered":"2024-04-22T20:48:35.956463+0000","last_clean":"2024-04-22T20:48:35.956463+0000","last_became_active":"2024-04-22T20:05:13.808473+0000","last_became_peered":"2024-04-22T20:05:13.808473+0000","last_unstale":"2024-04-22T20:48:35.956463+0000","last_undegraded":"2024-04-22T20:48:35.956463+0000","last_fullsized":"2024-04-22T20:48:35.956463+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:35.956357+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:35.956357+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.956357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:07:38.200109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"261'47","reported_seq":213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.554713+0000","last_change":"2024-04-22T20:48:26.554763+0000","last_active":"2024-04-22T20:48:26.554713+0000","last_peered":"2024-04-22T20:48:26.554713+0000","last_clean":"2024-04-22T20:48:26.554713+0000","last_became_active":"2024-04-22T20:01:04.874816+0000","last_became_peered":"2024-04-22T20:01:04.874816+0000","last_unstale":"2024-04-22T20:48:26.554713+0000","last_undegraded":"2024-04-22T20:48:26.554713+0000","last_fullsized":"2024-04-22T20:48:26.554713+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'47","last_scrub_stamp":"2024-04-22T20:48:26.554614+0000","last_deep_scrub":"261'47","last_deep_scrub_stamp":"2024-04-22T20:48:26.554614+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.554614+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:35:39.583851+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":291,"num_read_kb":200,"num_write":62,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3253,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.555923+0000","last_change":"2024-04-22T20:48:53.555978+0000","last_active":"2024-04-22T20:48:53.555923+0000","last_peered":"2024-04-22T20:48:53.555923+0000","last_clean":"2024-04-22T20:48:53.555923+0000","last_became_active":"2024-04-22T20:19:16.794954+0000","last_became_peered":"2024-04-22T20:19:16.794954+0000","last_unstale":"2024-04-22T20:48:53.555923+0000","last_undegraded":"2024-04-22T20:48:53.555923+0000","last_fullsized":"2024-04-22T20:48:53.555923+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:53.555824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:53.555824+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.555824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:08:41.681280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"261'6","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.542755+0000","last_change":"2024-04-22T20:48:32.542810+0000","last_active":"2024-04-22T20:48:32.542755+0000","last_peered":"2024-04-22T20:48:32.542755+0000","last_clean":"2024-04-22T20:48:32.542755+0000","last_became_active":"2024-04-22T20:05:17.889370+0000","last_became_peered":"2024-04-22T20:05:17.889370+0000","last_unstale":"2024-04-22T20:48:32.542755+0000","last_undegraded":"2024-04-22T20:48:32.542755+0000","last_fullsized":"2024-04-22T20:48:32.542755+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:32.542653+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:32.542653+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.542653+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:05:37.965129+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":6,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"261'2","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.599057+0000","last_change":"2024-04-22T20:48:25.599082+0000","last_active":"2024-04-22T20:48:25.599057+0000","last_peered":"2024-04-22T20:48:25.599057+0000","last_clean":"2024-04-22T20:48:25.599057+0000","last_became_active":"2024-04-22T20:05:13.815317+0000","last_became_peered":"2024-04-22T20:05:13.815317+0000","last_unstale":"2024-04-22T20:48:25.599057+0000","last_undegraded":"2024-04-22T20:48:25.599057+0000","last_fullsized":"2024-04-22T20:48:25.599057+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:25.598973+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:25.598973+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.598973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:21:44.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"261'28","reported_seq":128,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.478567+0000","last_change":"2024-04-22T20:48:23.478567+0000","last_active":"2024-04-22T20:48:23.478567+0000","last_peered":"2024-04-22T20:48:23.478567+0000","last_clean":"2024-04-22T20:48:23.478567+0000","last_became_active":"2024-04-22T20:01:04.878599+0000","last_became_peered":"2024-04-22T20:01:04.878599+0000","last_unstale":"2024-04-22T20:48:23.478567+0000","last_undegraded":"2024-04-22T20:48:23.478567+0000","last_fullsized":"2024-04-22T20:48:23.478567+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:23.478425+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:23.478425+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.478425+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:33.610274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":77,"num_write":54,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.257006+0000","last_change":"2024-04-22T20:19:16.794996+0000","last_active":"2024-04-22T20:48:25.257006+0000","last_peered":"2024-04-22T20:48:25.257006+0000","last_clean":"2024-04-22T20:48:25.257006+0000","last_became_active":"2024-04-22T20:19:16.793678+0000","last_became_peered":"2024-04-22T20:19:16.793678+0000","last_unstale":"2024-04-22T20:48:25.257006+0000","last_undegraded":"2024-04-22T20:48:25.257006+0000","last_fullsized":"2024-04-22T20:48:25.257006+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"0'0","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.495293+0000","last_change":"2024-04-22T20:48:40.495316+0000","last_active":"2024-04-22T20:48:40.495293+0000","last_peered":"2024-04-22T20:48:40.495293+0000","last_clean":"2024-04-22T20:48:40.495293+0000","last_became_active":"2024-04-22T20:05:17.892604+0000","last_became_peered":"2024-04-22T20:05:17.892604+0000","last_unstale":"2024-04-22T20:48:40.495293+0000","last_undegraded":"2024-04-22T20:48:40.495293+0000","last_fullsized":"2024-04-22T20:48:40.495293+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.495210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.495210+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.495210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:53:52.170315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"261'11","reported_seq":78,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.602185+0000","last_change":"2024-04-22T20:48:48.602213+0000","last_active":"2024-04-22T20:48:48.602185+0000","last_peered":"2024-04-22T20:48:48.602185+0000","last_clean":"2024-04-22T20:48:48.602185+0000","last_became_active":"2024-04-22T20:05:13.813946+0000","last_became_peered":"2024-04-22T20:05:13.813946+0000","last_unstale":"2024-04-22T20:48:48.602185+0000","last_undegraded":"2024-04-22T20:48:48.602185+0000","last_fullsized":"2024-04-22T20:48:48.602185+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'11","last_scrub_stamp":"2024-04-22T20:48:48.602091+0000","last_deep_scrub":"261'11","last_deep_scrub_stamp":"2024-04-22T20:48:48.602091+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.602091+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:32.170316+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":15362,"num_write":9,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"261'28","reported_seq":136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.953873+0000","last_change":"2024-04-22T20:48:31.953898+0000","last_active":"2024-04-22T20:48:31.953873+0000","last_peered":"2024-04-22T20:48:31.953873+0000","last_clean":"2024-04-22T20:48:31.953873+0000","last_became_active":"2024-04-22T20:01:04.872834+0000","last_became_peered":"2024-04-22T20:01:04.872834+0000","last_unstale":"2024-04-22T20:48:31.953873+0000","last_undegraded":"2024-04-22T20:48:31.953873+0000","last_fullsized":"2024-04-22T20:48:31.953873+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:31.953787+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:31.953787+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.953787+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:05:33.171326+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":90,"num_write":50,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.712975+0000","last_change":"2024-04-22T20:49:02.713000+0000","last_active":"2024-04-22T20:49:02.712975+0000","last_peered":"2024-04-22T20:49:02.712975+0000","last_clean":"2024-04-22T20:49:02.712975+0000","last_became_active":"2024-04-22T20:19:16.790917+0000","last_became_peered":"2024-04-22T20:19:16.790917+0000","last_unstale":"2024-04-22T20:49:02.712975+0000","last_undegraded":"2024-04-22T20:49:02.712975+0000","last_fullsized":"2024-04-22T20:49:02.712975+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.712882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.712882+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.712882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:02.266687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.612835+0000","last_change":"2024-04-22T20:49:01.612858+0000","last_active":"2024-04-22T20:49:01.612835+0000","last_peered":"2024-04-22T20:49:01.612835+0000","last_clean":"2024-04-22T20:49:01.612835+0000","last_became_active":"2024-04-22T20:05:18.211445+0000","last_became_peered":"2024-04-22T20:05:18.211445+0000","last_unstale":"2024-04-22T20:49:01.612835+0000","last_undegraded":"2024-04-22T20:49:01.612835+0000","last_fullsized":"2024-04-22T20:49:01.612835+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:01.612734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:01.612734+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.612734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:00:33.333955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"258'2","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.661659+0000","last_change":"2024-04-22T20:48:22.661681+0000","last_active":"2024-04-22T20:48:22.661659+0000","last_peered":"2024-04-22T20:48:22.661659+0000","last_clean":"2024-04-22T20:48:22.661659+0000","last_became_active":"2024-04-22T20:05:13.806928+0000","last_became_peered":"2024-04-22T20:05:13.806928+0000","last_unstale":"2024-04-22T20:48:22.661659+0000","last_undegraded":"2024-04-22T20:48:22.661659+0000","last_fullsized":"2024-04-22T20:48:22.661659+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'2","last_scrub_stamp":"2024-04-22T20:48:22.661579+0000","last_deep_scrub":"258'2","last_deep_scrub_stamp":"2024-04-22T20:48:22.661579+0000","last_clean_scrub_stamp":"2024-04-22T20:48:22.661579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:42.533500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"261'28","reported_seq":127,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.409573+0000","last_change":"2024-04-22T20:48:27.409602+0000","last_active":"2024-04-22T20:48:27.409573+0000","last_peered":"2024-04-22T20:48:27.409573+0000","last_clean":"2024-04-22T20:48:27.409573+0000","last_became_active":"2024-04-22T20:01:04.869226+0000","last_became_peered":"2024-04-22T20:01:04.869226+0000","last_unstale":"2024-04-22T20:48:27.409573+0000","last_undegraded":"2024-04-22T20:48:27.409573+0000","last_fullsized":"2024-04-22T20:48:27.409573+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:27.409466+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:27.409466+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.409466+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:07.279566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":80,"num_write":56,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.081712+0000","last_change":"2024-04-22T20:19:16.796398+0000","last_active":"2024-04-22T20:47:30.081712+0000","last_peered":"2024-04-22T20:47:30.081712+0000","last_clean":"2024-04-22T20:47:30.081712+0000","last_became_active":"2024-04-22T20:19:16.795524+0000","last_became_peered":"2024-04-22T20:19:16.795524+0000","last_unstale":"2024-04-22T20:47:30.081712+0000","last_undegraded":"2024-04-22T20:47:30.081712+0000","last_fullsized":"2024-04-22T20:47:30.081712+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'4","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.484819+0000","last_change":"2024-04-22T20:48:48.484845+0000","last_active":"2024-04-22T20:48:48.484819+0000","last_peered":"2024-04-22T20:48:48.484819+0000","last_clean":"2024-04-22T20:48:48.484819+0000","last_became_active":"2024-04-22T20:05:17.898858+0000","last_became_peered":"2024-04-22T20:05:17.898858+0000","last_unstale":"2024-04-22T20:48:48.484819+0000","last_undegraded":"2024-04-22T20:48:48.484819+0000","last_fullsized":"2024-04-22T20:48:48.484819+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:48.484715+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:48.484715+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.484715+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:42:15.414201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"258'3","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.510708+0000","last_change":"2024-04-22T20:48:27.510738+0000","last_active":"2024-04-22T20:48:27.510708+0000","last_peered":"2024-04-22T20:48:27.510708+0000","last_clean":"2024-04-22T20:48:27.510708+0000","last_became_active":"2024-04-22T20:05:13.807945+0000","last_became_peered":"2024-04-22T20:05:13.807945+0000","last_unstale":"2024-04-22T20:48:27.510708+0000","last_undegraded":"2024-04-22T20:48:27.510708+0000","last_fullsized":"2024-04-22T20:48:27.510708+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'3","last_scrub_stamp":"2024-04-22T20:48:27.510574+0000","last_deep_scrub":"258'3","last_deep_scrub_stamp":"2024-04-22T20:48:27.510574+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.510574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:46.475027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":8,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"261'24","reported_seq":115,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.452597+0000","last_change":"2024-04-22T20:48:32.452634+0000","last_active":"2024-04-22T20:48:32.452597+0000","last_peered":"2024-04-22T20:48:32.452597+0000","last_clean":"2024-04-22T20:48:32.452597+0000","last_became_active":"2024-04-22T20:01:04.871829+0000","last_became_peered":"2024-04-22T20:01:04.871829+0000","last_unstale":"2024-04-22T20:48:32.452597+0000","last_undegraded":"2024-04-22T20:48:32.452597+0000","last_fullsized":"2024-04-22T20:48:32.452597+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'24","last_scrub_stamp":"2024-04-22T20:48:32.452510+0000","last_deep_scrub":"261'24","last_deep_scrub_stamp":"2024-04-22T20:48:32.452510+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.452510+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:57.685562+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":61,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.415669+0000","last_change":"2024-04-22T20:19:16.794419+0000","last_active":"2024-04-22T20:48:23.415669+0000","last_peered":"2024-04-22T20:48:23.415669+0000","last_clean":"2024-04-22T20:48:23.415669+0000","last_became_active":"2024-04-22T20:19:16.793727+0000","last_became_peered":"2024-04-22T20:19:16.793727+0000","last_unstale":"2024-04-22T20:48:23.415669+0000","last_undegraded":"2024-04-22T20:48:23.415669+0000","last_fullsized":"2024-04-22T20:48:23.415669+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"261'5","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875813+0000","last_change":"2024-04-22T20:05:17.899691+0000","last_active":"2024-04-22T20:48:23.875813+0000","last_peered":"2024-04-22T20:48:23.875813+0000","last_clean":"2024-04-22T20:48:23.875813+0000","last_became_active":"2024-04-22T20:05:17.899328+0000","last_became_peered":"2024-04-22T20:05:17.899328+0000","last_unstale":"2024-04-22T20:48:23.875813+0000","last_undegraded":"2024-04-22T20:48:23.875813+0000","last_fullsized":"2024-04-22T20:48:23.875813+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'7","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.630926+0000","last_change":"2024-04-22T20:48:25.630926+0000","last_active":"2024-04-22T20:48:25.630926+0000","last_peered":"2024-04-22T20:48:25.630926+0000","last_clean":"2024-04-22T20:48:25.630926+0000","last_became_active":"2024-04-22T20:05:13.811986+0000","last_became_peered":"2024-04-22T20:05:13.811986+0000","last_unstale":"2024-04-22T20:48:25.630926+0000","last_undegraded":"2024-04-22T20:48:25.630926+0000","last_fullsized":"2024-04-22T20:48:25.630926+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:25.630769+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:25.630769+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.630769+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:31:51.985237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8240,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"261'66","reported_seq":221,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.587356+0000","last_change":"2024-04-22T20:48:26.587419+0000","last_active":"2024-04-22T20:48:26.587356+0000","last_peered":"2024-04-22T20:48:26.587356+0000","last_clean":"2024-04-22T20:48:26.587356+0000","last_became_active":"2024-04-22T20:01:04.874827+0000","last_became_peered":"2024-04-22T20:01:04.874827+0000","last_unstale":"2024-04-22T20:48:26.587356+0000","last_undegraded":"2024-04-22T20:48:26.587356+0000","last_fullsized":"2024-04-22T20:48:26.587356+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'66","last_scrub_stamp":"2024-04-22T20:48:26.587233+0000","last_deep_scrub":"261'66","last_deep_scrub_stamp":"2024-04-22T20:48:26.587233+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.587233+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:32.347942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":281,"num_read_kb":200,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.423664+0000","last_change":"2024-04-22T20:48:44.423664+0000","last_active":"2024-04-22T20:48:44.423664+0000","last_peered":"2024-04-22T20:48:44.423664+0000","last_clean":"2024-04-22T20:48:44.423664+0000","last_became_active":"2024-04-22T20:19:16.794434+0000","last_became_peered":"2024-04-22T20:19:16.794434+0000","last_unstale":"2024-04-22T20:48:44.423664+0000","last_undegraded":"2024-04-22T20:48:44.423664+0000","last_fullsized":"2024-04-22T20:48:44.423664+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:44.423537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:44.423537+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.423537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:22:23.617146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"261'9","reported_seq":65,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.412462+0000","last_change":"2024-04-22T20:48:36.412462+0000","last_active":"2024-04-22T20:48:36.412462+0000","last_peered":"2024-04-22T20:48:36.412462+0000","last_clean":"2024-04-22T20:48:36.412462+0000","last_became_active":"2024-04-22T20:05:18.209025+0000","last_became_peered":"2024-04-22T20:05:18.209025+0000","last_unstale":"2024-04-22T20:48:36.412462+0000","last_undegraded":"2024-04-22T20:48:36.412462+0000","last_fullsized":"2024-04-22T20:48:36.412462+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-22T20:48:36.412356+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-22T20:48:36.412356+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.412356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:02.300050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":13312,"num_write":14,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"261'4","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.619549+0000","last_change":"2024-04-22T20:48:22.619549+0000","last_active":"2024-04-22T20:48:22.619549+0000","last_peered":"2024-04-22T20:48:22.619549+0000","last_clean":"2024-04-22T20:48:22.619549+0000","last_became_active":"2024-04-22T20:05:13.807603+0000","last_became_peered":"2024-04-22T20:05:13.807603+0000","last_unstale":"2024-04-22T20:48:22.619549+0000","last_undegraded":"2024-04-22T20:48:22.619549+0000","last_fullsized":"2024-04-22T20:48:22.619549+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:22.619402+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:22.619402+0000","last_clean_scrub_stamp":"2024-04-22T20:48:22.619402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:32:09.200890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"258'23","reported_seq":122,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.404488+0000","last_change":"2024-04-22T20:48:37.404488+0000","last_active":"2024-04-22T20:48:37.404488+0000","last_peered":"2024-04-22T20:48:37.404488+0000","last_clean":"2024-04-22T20:48:37.404488+0000","last_became_active":"2024-04-22T20:01:04.868847+0000","last_became_peered":"2024-04-22T20:01:04.868847+0000","last_unstale":"2024-04-22T20:48:37.404488+0000","last_undegraded":"2024-04-22T20:48:37.404488+0000","last_fullsized":"2024-04-22T20:48:37.404488+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'23","last_scrub_stamp":"2024-04-22T20:48:37.404364+0000","last_deep_scrub":"258'23","last_deep_scrub_stamp":"2024-04-22T20:48:37.404364+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.404364+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:31.027402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":76,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"261'1","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.257034+0000","last_change":"2024-04-22T20:19:16.796622+0000","last_active":"2024-04-22T20:48:25.257034+0000","last_peered":"2024-04-22T20:48:25.257034+0000","last_clean":"2024-04-22T20:48:25.257034+0000","last_became_active":"2024-04-22T20:19:16.795613+0000","last_became_peered":"2024-04-22T20:19:16.795613+0000","last_unstale":"2024-04-22T20:48:25.257034+0000","last_undegraded":"2024-04-22T20:48:25.257034+0000","last_fullsized":"2024-04-22T20:48:25.257034+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"261'6","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.538494+0000","last_change":"2024-04-22T20:48:31.538524+0000","last_active":"2024-04-22T20:48:31.538494+0000","last_peered":"2024-04-22T20:48:31.538494+0000","last_clean":"2024-04-22T20:48:31.538494+0000","last_became_active":"2024-04-22T20:05:17.891331+0000","last_became_peered":"2024-04-22T20:05:17.891331+0000","last_unstale":"2024-04-22T20:48:31.538494+0000","last_undegraded":"2024-04-22T20:48:31.538494+0000","last_fullsized":"2024-04-22T20:48:31.538494+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:31.538340+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:31.538340+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.538340+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:36:13.794601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.504525+0000","last_change":"2024-04-22T20:48:37.504525+0000","last_active":"2024-04-22T20:48:37.504525+0000","last_peered":"2024-04-22T20:48:37.504525+0000","last_clean":"2024-04-22T20:48:37.504525+0000","last_became_active":"2024-04-22T20:05:13.813811+0000","last_became_peered":"2024-04-22T20:05:13.813811+0000","last_unstale":"2024-04-22T20:48:37.504525+0000","last_undegraded":"2024-04-22T20:48:37.504525+0000","last_fullsized":"2024-04-22T20:48:37.504525+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:37.504369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:37.504369+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.504369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:22.962064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"258'49","reported_seq":180,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.613719+0000","last_change":"2024-04-22T20:48:29.613719+0000","last_active":"2024-04-22T20:48:29.613719+0000","last_peered":"2024-04-22T20:48:29.613719+0000","last_clean":"2024-04-22T20:48:29.613719+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:48:29.613719+0000","last_undegraded":"2024-04-22T20:48:29.613719+0000","last_fullsized":"2024-04-22T20:48:29.613719+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'49","last_scrub_stamp":"2024-04-22T20:48:29.613592+0000","last_deep_scrub":"258'49","last_deep_scrub_stamp":"2024-04-22T20:48:29.613592+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.613592+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:50:57.860950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":145,"num_write":78,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.034593+0000","last_change":"2024-04-22T20:19:16.793813+0000","last_active":"2024-04-22T20:48:22.034593+0000","last_peered":"2024-04-22T20:48:22.034593+0000","last_clean":"2024-04-22T20:48:22.034593+0000","last_became_active":"2024-04-22T20:19:16.793486+0000","last_became_peered":"2024-04-22T20:19:16.793486+0000","last_unstale":"2024-04-22T20:48:22.034593+0000","last_undegraded":"2024-04-22T20:48:22.034593+0000","last_fullsized":"2024-04-22T20:48:22.034593+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.489207+0000","last_change":"2024-04-22T20:48:32.489207+0000","last_active":"2024-04-22T20:48:32.489207+0000","last_peered":"2024-04-22T20:48:32.489207+0000","last_clean":"2024-04-22T20:48:32.489207+0000","last_became_active":"2024-04-22T20:05:17.897008+0000","last_became_peered":"2024-04-22T20:05:17.897008+0000","last_unstale":"2024-04-22T20:48:32.489207+0000","last_undegraded":"2024-04-22T20:48:32.489207+0000","last_fullsized":"2024-04-22T20:48:32.489207+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:32.489059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:32.489059+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.489059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:11.685871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"261'15","reported_seq":85,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.378030+0000","last_change":"2024-04-22T20:48:32.378065+0000","last_active":"2024-04-22T20:48:32.378030+0000","last_peered":"2024-04-22T20:48:32.378030+0000","last_clean":"2024-04-22T20:48:32.378030+0000","last_became_active":"2024-04-22T20:05:13.798319+0000","last_became_peered":"2024-04-22T20:05:13.798319+0000","last_unstale":"2024-04-22T20:48:32.378030+0000","last_undegraded":"2024-04-22T20:48:32.378030+0000","last_fullsized":"2024-04-22T20:48:32.378030+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'15","last_scrub_stamp":"2024-04-22T20:48:32.377942+0000","last_deep_scrub":"261'15","last_deep_scrub_stamp":"2024-04-22T20:48:32.377942+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.377942+0000","objects_scrubbed":2,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:58.634704+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":16388,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"261'37","reported_seq":148,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.458538+0000","last_change":"2024-04-22T20:48:28.458538+0000","last_active":"2024-04-22T20:48:28.458538+0000","last_peered":"2024-04-22T20:48:28.458538+0000","last_clean":"2024-04-22T20:48:28.458538+0000","last_became_active":"2024-04-22T20:01:04.869079+0000","last_became_peered":"2024-04-22T20:01:04.869079+0000","last_unstale":"2024-04-22T20:48:28.458538+0000","last_undegraded":"2024-04-22T20:48:28.458538+0000","last_fullsized":"2024-04-22T20:48:28.458538+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'37","last_scrub_stamp":"2024-04-22T20:48:28.458400+0000","last_deep_scrub":"261'37","last_deep_scrub_stamp":"2024-04-22T20:48:28.458400+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.458400+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:24:11.187493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":61,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.256974+0000","last_change":"2024-04-22T20:19:16.796317+0000","last_active":"2024-04-22T20:48:25.256974+0000","last_peered":"2024-04-22T20:48:25.256974+0000","last_clean":"2024-04-22T20:48:25.256974+0000","last_became_active":"2024-04-22T20:19:16.795462+0000","last_became_peered":"2024-04-22T20:19:16.795462+0000","last_unstale":"2024-04-22T20:48:25.256974+0000","last_undegraded":"2024-04-22T20:48:25.256974+0000","last_fullsized":"2024-04-22T20:48:25.256974+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'7","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.601140+0000","last_change":"2024-04-22T20:48:45.601169+0000","last_active":"2024-04-22T20:48:45.601140+0000","last_peered":"2024-04-22T20:48:45.601140+0000","last_clean":"2024-04-22T20:48:45.601140+0000","last_became_active":"2024-04-22T20:05:17.892044+0000","last_became_peered":"2024-04-22T20:05:17.892044+0000","last_unstale":"2024-04-22T20:48:45.601140+0000","last_undegraded":"2024-04-22T20:48:45.601140+0000","last_fullsized":"2024-04-22T20:48:45.601140+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:45.601054+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:45.601054+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.601054+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:42:15.237225+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"261'5","reported_seq":69,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.541577+0000","last_change":"2024-04-22T20:48:29.541577+0000","last_active":"2024-04-22T20:48:29.541577+0000","last_peered":"2024-04-22T20:48:29.541577+0000","last_clean":"2024-04-22T20:48:29.541577+0000","last_became_active":"2024-04-22T20:05:13.809368+0000","last_became_peered":"2024-04-22T20:05:13.809368+0000","last_unstale":"2024-04-22T20:48:29.541577+0000","last_undegraded":"2024-04-22T20:48:29.541577+0000","last_fullsized":"2024-04-22T20:48:29.541577+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:29.541434+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:29.541434+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.541434+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:38:26.695883+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"261'35","reported_seq":148,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.558111+0000","last_change":"2024-04-22T20:48:27.558111+0000","last_active":"2024-04-22T20:48:27.558111+0000","last_peered":"2024-04-22T20:48:27.558111+0000","last_clean":"2024-04-22T20:48:27.558111+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:48:27.558111+0000","last_undegraded":"2024-04-22T20:48:27.558111+0000","last_fullsized":"2024-04-22T20:48:27.558111+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'35","last_scrub_stamp":"2024-04-22T20:48:27.558001+0000","last_deep_scrub":"261'35","last_deep_scrub_stamp":"2024-04-22T20:48:27.558001+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.558001+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:44:57.759608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":59,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.335414+0000","last_change":"2024-04-22T20:19:16.791940+0000","last_active":"2024-04-22T20:48:24.335414+0000","last_peered":"2024-04-22T20:48:24.335414+0000","last_clean":"2024-04-22T20:48:24.335414+0000","last_became_active":"2024-04-22T20:19:16.791663+0000","last_became_peered":"2024-04-22T20:19:16.791663+0000","last_unstale":"2024-04-22T20:48:24.335414+0000","last_undegraded":"2024-04-22T20:48:24.335414+0000","last_fullsized":"2024-04-22T20:48:24.335414+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"261'7","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.669783+0000","last_change":"2024-04-22T20:48:49.669783+0000","last_active":"2024-04-22T20:48:49.669783+0000","last_peered":"2024-04-22T20:48:49.669783+0000","last_clean":"2024-04-22T20:48:49.669783+0000","last_became_active":"2024-04-22T20:05:18.210736+0000","last_became_peered":"2024-04-22T20:05:18.210736+0000","last_unstale":"2024-04-22T20:48:49.669783+0000","last_undegraded":"2024-04-22T20:48:49.669783+0000","last_fullsized":"2024-04-22T20:48:49.669783+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:49.669616+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:49.669616+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.669616+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:27:27.520540+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20482,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"261'6","reported_seq":65,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.576912+0000","last_change":"2024-04-22T20:48:47.576941+0000","last_active":"2024-04-22T20:48:47.576912+0000","last_peered":"2024-04-22T20:48:47.576912+0000","last_clean":"2024-04-22T20:48:47.576912+0000","last_became_active":"2024-04-22T20:05:13.813277+0000","last_became_peered":"2024-04-22T20:05:13.813277+0000","last_unstale":"2024-04-22T20:48:47.576912+0000","last_undegraded":"2024-04-22T20:48:47.576912+0000","last_fullsized":"2024-04-22T20:48:47.576912+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:47.576814+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:47.576814+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.576814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:43:34.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1024,"num_write":16,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"261'61","reported_seq":204,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.363698+0000","last_change":"2024-04-22T20:48:25.363698+0000","last_active":"2024-04-22T20:48:25.363698+0000","last_peered":"2024-04-22T20:48:25.363698+0000","last_clean":"2024-04-22T20:48:25.363698+0000","last_became_active":"2024-04-22T20:01:04.871859+0000","last_became_peered":"2024-04-22T20:01:04.871859+0000","last_unstale":"2024-04-22T20:48:25.363698+0000","last_undegraded":"2024-04-22T20:48:25.363698+0000","last_fullsized":"2024-04-22T20:48:25.363698+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'61","last_scrub_stamp":"2024-04-22T20:48:25.363585+0000","last_deep_scrub":"261'61","last_deep_scrub_stamp":"2024-04-22T20:48:25.363585+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.363585+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:50:52.645407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":172,"num_write":109,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.659621+0000","last_change":"2024-04-22T20:49:03.659643+0000","last_active":"2024-04-22T20:49:03.659621+0000","last_peered":"2024-04-22T20:49:03.659621+0000","last_clean":"2024-04-22T20:49:03.659621+0000","last_became_active":"2024-04-22T20:19:16.789688+0000","last_became_peered":"2024-04-22T20:19:16.789688+0000","last_unstale":"2024-04-22T20:49:03.659621+0000","last_undegraded":"2024-04-22T20:49:03.659621+0000","last_fullsized":"2024-04-22T20:49:03.659621+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:03.659497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:03.659497+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.659497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:02.426808+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"258'4","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.411626+0000","last_change":"2024-04-22T20:48:40.411671+0000","last_active":"2024-04-22T20:48:40.411626+0000","last_peered":"2024-04-22T20:48:40.411626+0000","last_clean":"2024-04-22T20:48:40.411626+0000","last_became_active":"2024-04-22T20:05:18.211478+0000","last_became_peered":"2024-04-22T20:05:18.211478+0000","last_unstale":"2024-04-22T20:48:40.411626+0000","last_undegraded":"2024-04-22T20:48:40.411626+0000","last_fullsized":"2024-04-22T20:48:40.411626+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:40.411479+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:40.411479+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.411479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:39:01.313336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'7","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.335294+0000","last_change":"2024-04-22T20:05:13.811784+0000","last_active":"2024-04-22T20:48:24.335294+0000","last_peered":"2024-04-22T20:48:24.335294+0000","last_clean":"2024-04-22T20:48:24.335294+0000","last_became_active":"2024-04-22T20:05:13.811561+0000","last_became_peered":"2024-04-22T20:05:13.811561+0000","last_unstale":"2024-04-22T20:48:24.335294+0000","last_undegraded":"2024-04-22T20:48:24.335294+0000","last_fullsized":"2024-04-22T20:48:24.335294+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:12.785876+0000","last_clean_scrub_stamp":"2024-04-22T20:05:12.785876+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":12,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"261'33","reported_seq":144,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.528994+0000","last_change":"2024-04-22T20:48:31.529020+0000","last_active":"2024-04-22T20:48:31.528994+0000","last_peered":"2024-04-22T20:48:31.528994+0000","last_clean":"2024-04-22T20:48:31.528994+0000","last_became_active":"2024-04-22T20:01:04.874790+0000","last_became_peered":"2024-04-22T20:01:04.874790+0000","last_unstale":"2024-04-22T20:48:31.528994+0000","last_undegraded":"2024-04-22T20:48:31.528994+0000","last_fullsized":"2024-04-22T20:48:31.528994+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'33","last_scrub_stamp":"2024-04-22T20:48:31.528870+0000","last_deep_scrub":"261'33","last_deep_scrub_stamp":"2024-04-22T20:48:31.528870+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.528870+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:43.937257+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":100,"num_write":57,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875852+0000","last_change":"2024-04-22T20:19:16.794065+0000","last_active":"2024-04-22T20:48:23.875852+0000","last_peered":"2024-04-22T20:48:23.875852+0000","last_clean":"2024-04-22T20:48:23.875852+0000","last_became_active":"2024-04-22T20:19:16.793188+0000","last_became_peered":"2024-04-22T20:19:16.793188+0000","last_unstale":"2024-04-22T20:48:23.875852+0000","last_undegraded":"2024-04-22T20:48:23.875852+0000","last_fullsized":"2024-04-22T20:48:23.875852+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"261'5","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875856+0000","last_change":"2024-04-22T20:05:17.901392+0000","last_active":"2024-04-22T20:48:23.875856+0000","last_peered":"2024-04-22T20:48:23.875856+0000","last_clean":"2024-04-22T20:48:23.875856+0000","last_became_active":"2024-04-22T20:05:17.901128+0000","last_became_peered":"2024-04-22T20:05:17.901128+0000","last_unstale":"2024-04-22T20:48:23.875856+0000","last_undegraded":"2024-04-22T20:48:23.875856+0000","last_fullsized":"2024-04-22T20:48:23.875856+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":8192,"num_write":10,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"261'5","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.568012+0000","last_change":"2024-04-22T20:48:28.568012+0000","last_active":"2024-04-22T20:48:28.568012+0000","last_peered":"2024-04-22T20:48:28.568012+0000","last_clean":"2024-04-22T20:48:28.568012+0000","last_became_active":"2024-04-22T20:05:13.815332+0000","last_became_peered":"2024-04-22T20:05:13.815332+0000","last_unstale":"2024-04-22T20:48:28.568012+0000","last_undegraded":"2024-04-22T20:48:28.568012+0000","last_fullsized":"2024-04-22T20:48:28.568012+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:28.567863+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:28.567863+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.567863+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:52:21.493631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":9216,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'28","reported_seq":134,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.541277+0000","last_change":"2024-04-22T20:48:26.541277+0000","last_active":"2024-04-22T20:48:26.541277+0000","last_peered":"2024-04-22T20:48:26.541277+0000","last_clean":"2024-04-22T20:48:26.541277+0000","last_became_active":"2024-04-22T20:01:04.871840+0000","last_became_peered":"2024-04-22T20:01:04.871840+0000","last_unstale":"2024-04-22T20:48:26.541277+0000","last_undegraded":"2024-04-22T20:48:26.541277+0000","last_fullsized":"2024-04-22T20:48:26.541277+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:26.541161+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:26.541161+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.541161+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:03:51.347595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":89,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"261'2","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.585427+0000","last_change":"2024-04-22T20:48:52.585467+0000","last_active":"2024-04-22T20:48:52.585427+0000","last_peered":"2024-04-22T20:48:52.585427+0000","last_clean":"2024-04-22T20:48:52.585427+0000","last_became_active":"2024-04-22T20:19:16.794123+0000","last_became_peered":"2024-04-22T20:19:16.794123+0000","last_unstale":"2024-04-22T20:48:52.585427+0000","last_undegraded":"2024-04-22T20:48:52.585427+0000","last_fullsized":"2024-04-22T20:48:52.585427+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:52.585332+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:52.585332+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.585332+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:48.233250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"261'4","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.624822+0000","last_change":"2024-04-22T20:48:58.624822+0000","last_active":"2024-04-22T20:48:58.624822+0000","last_peered":"2024-04-22T20:48:58.624822+0000","last_clean":"2024-04-22T20:48:58.624822+0000","last_became_active":"2024-04-22T20:05:17.895177+0000","last_became_peered":"2024-04-22T20:05:17.895177+0000","last_unstale":"2024-04-22T20:48:58.624822+0000","last_undegraded":"2024-04-22T20:48:58.624822+0000","last_fullsized":"2024-04-22T20:48:58.624822+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:58.624700+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:58.624700+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.624700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:42:11.281730+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":4,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"261'4","reported_seq":65,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.553051+0000","last_change":"2024-04-22T20:48:30.553078+0000","last_active":"2024-04-22T20:48:30.553051+0000","last_peered":"2024-04-22T20:48:30.553051+0000","last_clean":"2024-04-22T20:48:30.553051+0000","last_became_active":"2024-04-22T20:05:13.809047+0000","last_became_peered":"2024-04-22T20:05:13.809047+0000","last_unstale":"2024-04-22T20:48:30.553051+0000","last_undegraded":"2024-04-22T20:48:30.553051+0000","last_fullsized":"2024-04-22T20:48:30.553051+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:30.552959+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:30.552959+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.552959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:44:03.327794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"261'33","reported_seq":140,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.902332+0000","last_change":"2024-04-22T20:48:27.902332+0000","last_active":"2024-04-22T20:48:27.902332+0000","last_peered":"2024-04-22T20:48:27.902332+0000","last_clean":"2024-04-22T20:48:27.902332+0000","last_became_active":"2024-04-22T20:01:04.872370+0000","last_became_peered":"2024-04-22T20:01:04.872370+0000","last_unstale":"2024-04-22T20:48:27.902332+0000","last_undegraded":"2024-04-22T20:48:27.902332+0000","last_fullsized":"2024-04-22T20:48:27.902332+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'33","last_scrub_stamp":"2024-04-22T20:48:27.902204+0000","last_deep_scrub":"261'33","last_deep_scrub_stamp":"2024-04-22T20:48:27.902204+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.902204+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:57:41.905060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":57,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":26,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.195802+0000","last_change":"2024-04-22T20:19:16.793728+0000","last_active":"2024-04-22T20:47:30.195802+0000","last_peered":"2024-04-22T20:47:30.195802+0000","last_clean":"2024-04-22T20:47:30.195802+0000","last_became_active":"2024-04-22T20:19:16.793429+0000","last_became_peered":"2024-04-22T20:19:16.793429+0000","last_unstale":"2024-04-22T20:47:30.195802+0000","last_undegraded":"2024-04-22T20:47:30.195802+0000","last_fullsized":"2024-04-22T20:47:30.195802+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"261'2","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.489775+0000","last_change":"2024-04-22T20:05:17.901476+0000","last_active":"2024-04-22T20:48:16.489775+0000","last_peered":"2024-04-22T20:48:16.489775+0000","last_clean":"2024-04-22T20:48:16.489775+0000","last_became_active":"2024-04-22T20:05:17.900874+0000","last_became_peered":"2024-04-22T20:05:17.900874+0000","last_unstale":"2024-04-22T20:48:16.489775+0000","last_undegraded":"2024-04-22T20:48:16.489775+0000","last_fullsized":"2024-04-22T20:48:16.489775+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"261'2","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.612637+0000","last_change":"2024-04-22T20:48:26.612637+0000","last_active":"2024-04-22T20:48:26.612637+0000","last_peered":"2024-04-22T20:48:26.612637+0000","last_clean":"2024-04-22T20:48:26.612637+0000","last_became_active":"2024-04-22T20:05:13.813934+0000","last_became_peered":"2024-04-22T20:05:13.813934+0000","last_unstale":"2024-04-22T20:48:26.612637+0000","last_undegraded":"2024-04-22T20:48:26.612637+0000","last_fullsized":"2024-04-22T20:48:26.612637+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:26.612508+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:26.612508+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.612508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:58:57.908355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1024,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"261'37","reported_seq":156,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.240514+0000","last_change":"2024-04-22T20:48:24.240514+0000","last_active":"2024-04-22T20:48:24.240514+0000","last_peered":"2024-04-22T20:48:24.240514+0000","last_clean":"2024-04-22T20:48:24.240514+0000","last_became_active":"2024-04-22T20:01:04.869660+0000","last_became_peered":"2024-04-22T20:01:04.869660+0000","last_unstale":"2024-04-22T20:48:24.240514+0000","last_undegraded":"2024-04-22T20:48:24.240514+0000","last_fullsized":"2024-04-22T20:48:24.240514+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'37","last_scrub_stamp":"2024-04-22T20:48:24.240383+0000","last_deep_scrub":"261'37","last_deep_scrub_stamp":"2024-04-22T20:48:24.240383+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.240383+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:40:22.419745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":113,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.034566+0000","last_change":"2024-04-22T20:19:16.795977+0000","last_active":"2024-04-22T20:48:22.034566+0000","last_peered":"2024-04-22T20:48:22.034566+0000","last_clean":"2024-04-22T20:48:22.034566+0000","last_became_active":"2024-04-22T20:19:16.795509+0000","last_became_peered":"2024-04-22T20:19:16.795509+0000","last_unstale":"2024-04-22T20:48:22.034566+0000","last_undegraded":"2024-04-22T20:48:22.034566+0000","last_fullsized":"2024-04-22T20:48:22.034566+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.628465+0000","last_change":"2024-04-22T20:49:00.628465+0000","last_active":"2024-04-22T20:49:00.628465+0000","last_peered":"2024-04-22T20:49:00.628465+0000","last_clean":"2024-04-22T20:49:00.628465+0000","last_became_active":"2024-04-22T20:05:17.891523+0000","last_became_peered":"2024-04-22T20:05:17.891523+0000","last_unstale":"2024-04-22T20:49:00.628465+0000","last_undegraded":"2024-04-22T20:49:00.628465+0000","last_fullsized":"2024-04-22T20:49:00.628465+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:00.628331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:00.628331+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.628331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:34:52.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"261'2","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.553971+0000","last_change":"2024-04-22T20:48:24.553971+0000","last_active":"2024-04-22T20:48:24.553971+0000","last_peered":"2024-04-22T20:48:24.553971+0000","last_clean":"2024-04-22T20:48:24.553971+0000","last_became_active":"2024-04-22T20:05:13.805704+0000","last_became_peered":"2024-04-22T20:05:13.805704+0000","last_unstale":"2024-04-22T20:48:24.553971+0000","last_undegraded":"2024-04-22T20:48:24.553971+0000","last_fullsized":"2024-04-22T20:48:24.553971+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:24.553871+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:24.553871+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.553871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:02.108209+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"261'65","reported_seq":234,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.443246+0000","last_change":"2024-04-22T20:48:29.443246+0000","last_active":"2024-04-22T20:48:29.443246+0000","last_peered":"2024-04-22T20:48:29.443246+0000","last_clean":"2024-04-22T20:48:29.443246+0000","last_became_active":"2024-04-22T20:01:04.865981+0000","last_became_peered":"2024-04-22T20:01:04.865981+0000","last_unstale":"2024-04-22T20:48:29.443246+0000","last_undegraded":"2024-04-22T20:48:29.443246+0000","last_fullsized":"2024-04-22T20:48:29.443246+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'65","last_scrub_stamp":"2024-04-22T20:48:29.443124+0000","last_deep_scrub":"261'65","last_deep_scrub_stamp":"2024-04-22T20:48:29.443124+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.443124+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:12:31.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":226,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":27,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.256956+0000","last_change":"2024-04-22T20:19:16.799325+0000","last_active":"2024-04-22T20:48:25.256956+0000","last_peered":"2024-04-22T20:48:25.256956+0000","last_clean":"2024-04-22T20:48:25.256956+0000","last_became_active":"2024-04-22T20:19:16.799063+0000","last_became_peered":"2024-04-22T20:19:16.799063+0000","last_unstale":"2024-04-22T20:48:25.256956+0000","last_undegraded":"2024-04-22T20:48:25.256956+0000","last_fullsized":"2024-04-22T20:48:25.256956+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"261'2","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875776+0000","last_change":"2024-04-22T20:05:17.901995+0000","last_active":"2024-04-22T20:48:23.875776+0000","last_peered":"2024-04-22T20:48:23.875776+0000","last_clean":"2024-04-22T20:48:23.875776+0000","last_became_active":"2024-04-22T20:05:17.901680+0000","last_became_peered":"2024-04-22T20:05:17.901680+0000","last_unstale":"2024-04-22T20:48:23.875776+0000","last_undegraded":"2024-04-22T20:48:23.875776+0000","last_fullsized":"2024-04-22T20:48:23.875776+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:05:16.866416+0000","last_clean_scrub_stamp":"2024-04-22T20:05:16.866416+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":80,"num_write":2,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.618610+0000","last_change":"2024-04-22T20:48:57.618610+0000","last_active":"2024-04-22T20:48:57.618610+0000","last_peered":"2024-04-22T20:48:57.618610+0000","last_clean":"2024-04-22T20:48:57.618610+0000","last_became_active":"2024-04-22T20:05:13.810865+0000","last_became_peered":"2024-04-22T20:05:13.810865+0000","last_unstale":"2024-04-22T20:48:57.618610+0000","last_undegraded":"2024-04-22T20:48:57.618610+0000","last_fullsized":"2024-04-22T20:48:57.618610+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.618439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.618439+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.618439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:51:28.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"261'57","reported_seq":189,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.363453+0000","last_change":"2024-04-22T20:48:35.363498+0000","last_active":"2024-04-22T20:48:35.363453+0000","last_peered":"2024-04-22T20:48:35.363453+0000","last_clean":"2024-04-22T20:48:35.363453+0000","last_became_active":"2024-04-22T20:01:04.870253+0000","last_became_peered":"2024-04-22T20:01:04.870253+0000","last_unstale":"2024-04-22T20:48:35.363453+0000","last_undegraded":"2024-04-22T20:48:35.363453+0000","last_fullsized":"2024-04-22T20:48:35.363453+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'57","last_scrub_stamp":"2024-04-22T20:48:35.363335+0000","last_deep_scrub":"261'57","last_deep_scrub_stamp":"2024-04-22T20:48:35.363335+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.363335+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:44:05.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":145,"num_write":102,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"261'344","reported_seq":960,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.428579+0000","last_change":"2024-04-22T20:48:30.428623+0000","last_active":"2024-04-22T20:48:30.428579+0000","last_peered":"2024-04-22T20:48:30.428579+0000","last_clean":"2024-04-22T20:48:30.428579+0000","last_became_active":"2024-04-22T19:53:04.823351+0000","last_became_peered":"2024-04-22T19:53:04.823351+0000","last_unstale":"2024-04-22T20:48:30.428579+0000","last_undegraded":"2024-04-22T20:48:30.428579+0000","last_fullsized":"2024-04-22T20:48:30.428579+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'344","last_scrub_stamp":"2024-04-22T20:48:30.428472+0000","last_deep_scrub":"261'344","last_deep_scrub_stamp":"2024-04-22T20:48:30.428472+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.428472+0000","objects_scrubbed":5,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:45:58.154599+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":495,"num_write":1298,"num_write_kb":39497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"261'169","reported_seq":773,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.947499+0000","last_change":"2024-04-22T20:48:50.947499+0000","last_active":"2024-04-22T20:48:50.947499+0000","last_peered":"2024-04-22T20:48:50.947499+0000","last_clean":"2024-04-22T20:48:50.947499+0000","last_became_active":"2024-04-22T19:51:28.511595+0000","last_became_peered":"2024-04-22T19:51:28.511595+0000","last_unstale":"2024-04-22T20:48:50.947499+0000","last_undegraded":"2024-04-22T20:48:50.947499+0000","last_fullsized":"2024-04-22T20:48:50.947499+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'169","last_scrub_stamp":"2024-04-22T20:48:50.947377+0000","last_deep_scrub":"261'169","last_deep_scrub_stamp":"2024-04-22T20:48:50.947377+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.947377+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:25:29.288345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":537,"num_write":262,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"261'316","reported_seq":976,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.575341+0000","last_change":"2024-04-22T20:48:51.575445+0000","last_active":"2024-04-22T20:48:51.575341+0000","last_peered":"2024-04-22T20:48:51.575341+0000","last_clean":"2024-04-22T20:48:51.575341+0000","last_became_active":"2024-04-22T19:51:26.496188+0000","last_became_peered":"2024-04-22T19:51:26.496188+0000","last_unstale":"2024-04-22T20:48:51.575341+0000","last_undegraded":"2024-04-22T20:48:51.575341+0000","last_fullsized":"2024-04-22T20:48:51.575341+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'316","last_scrub_stamp":"2024-04-22T20:48:51.575227+0000","last_deep_scrub":"261'316","last_deep_scrub_stamp":"2024-04-22T20:48:51.575227+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.575227+0000","objects_scrubbed":11,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:05:45.518799+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5719773,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":497,"num_read_kb":9758,"num_write":1112,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"261'315","reported_seq":897,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.386962+0000","last_change":"2024-04-22T20:48:53.386962+0000","last_active":"2024-04-22T20:48:53.386962+0000","last_peered":"2024-04-22T20:48:53.386962+0000","last_clean":"2024-04-22T20:48:53.386962+0000","last_became_active":"2024-04-22T19:51:28.519111+0000","last_became_peered":"2024-04-22T19:51:28.519111+0000","last_unstale":"2024-04-22T20:48:53.386962+0000","last_undegraded":"2024-04-22T20:48:53.386962+0000","last_fullsized":"2024-04-22T20:48:53.386962+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'315","last_scrub_stamp":"2024-04-22T20:48:53.386841+0000","last_deep_scrub":"261'315","last_deep_scrub_stamp":"2024-04-22T20:48:53.386841+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.386841+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:13.253246+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1190,"num_read_kb":913,"num_write":510,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"261'320","reported_seq":942,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.949162+0000","last_change":"2024-04-22T19:51:26.503143+0000","last_active":"2024-04-22T20:48:17.949162+0000","last_peered":"2024-04-22T20:48:17.949162+0000","last_clean":"2024-04-22T20:48:17.949162+0000","last_became_active":"2024-04-22T19:51:26.502744+0000","last_became_peered":"2024-04-22T19:51:26.502744+0000","last_unstale":"2024-04-22T20:48:17.949162+0000","last_undegraded":"2024-04-22T20:48:17.949162+0000","last_fullsized":"2024-04-22T20:48:17.949162+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437211,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":443,"num_read_kb":7116,"num_write":1035,"num_write_kb":21496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"261'151","reported_seq":737,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.271385+0000","last_change":"2024-04-22T20:48:48.271385+0000","last_active":"2024-04-22T20:48:48.271385+0000","last_peered":"2024-04-22T20:48:48.271385+0000","last_clean":"2024-04-22T20:48:48.271385+0000","last_became_active":"2024-04-22T19:51:28.518650+0000","last_became_peered":"2024-04-22T19:51:28.518650+0000","last_unstale":"2024-04-22T20:48:48.271385+0000","last_undegraded":"2024-04-22T20:48:48.271385+0000","last_fullsized":"2024-04-22T20:48:48.271385+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'151","last_scrub_stamp":"2024-04-22T20:48:48.271264+0000","last_deep_scrub":"261'151","last_deep_scrub_stamp":"2024-04-22T20:48:48.271264+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.271264+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:14.794941+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":572,"num_read_kb":453,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"261'324","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.858153+0000","last_change":"2024-04-22T20:49:01.858233+0000","last_active":"2024-04-22T20:49:01.858153+0000","last_peered":"2024-04-22T20:49:01.858153+0000","last_clean":"2024-04-22T20:49:01.858153+0000","last_became_active":"2024-04-22T19:51:26.504450+0000","last_became_peered":"2024-04-22T19:51:26.504450+0000","last_unstale":"2024-04-22T20:49:01.858153+0000","last_undegraded":"2024-04-22T20:49:01.858153+0000","last_fullsized":"2024-04-22T20:49:01.858153+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:49:01.858049+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:49:01.858049+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.858049+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:13.565120+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8563195,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":424,"num_read_kb":25508,"num_write":1190,"num_write_kb":37655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"261'81","reported_seq":686,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.546404+0000","last_change":"2024-04-22T20:48:32.546404+0000","last_active":"2024-04-22T20:48:32.546404+0000","last_peered":"2024-04-22T20:48:32.546404+0000","last_clean":"2024-04-22T20:48:32.546404+0000","last_became_active":"2024-04-22T19:52:04.619688+0000","last_became_peered":"2024-04-22T19:52:04.619688+0000","last_unstale":"2024-04-22T20:48:32.546404+0000","last_undegraded":"2024-04-22T20:48:32.546404+0000","last_fullsized":"2024-04-22T20:48:32.546404+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'81","last_scrub_stamp":"2024-04-22T20:48:32.546276+0000","last_deep_scrub":"261'81","last_deep_scrub_stamp":"2024-04-22T20:48:32.546276+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.546276+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:51:45.224120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":353,"num_read_kb":299,"num_write":120,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"261'281","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.591663+0000","last_change":"2024-04-22T20:49:03.591744+0000","last_active":"2024-04-22T20:49:03.591663+0000","last_peered":"2024-04-22T20:49:03.591663+0000","last_clean":"2024-04-22T20:49:03.591663+0000","last_became_active":"2024-04-22T19:51:26.507185+0000","last_became_peered":"2024-04-22T19:51:26.507185+0000","last_unstale":"2024-04-22T20:49:03.591663+0000","last_undegraded":"2024-04-22T20:49:03.591663+0000","last_fullsized":"2024-04-22T20:49:03.591663+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'281","last_scrub_stamp":"2024-04-22T20:49:03.591510+0000","last_deep_scrub":"261'281","last_deep_scrub_stamp":"2024-04-22T20:49:03.591510+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.591510+0000","objects_scrubbed":13,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:35.348173+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3508486,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":442,"num_read_kb":10200,"num_write":925,"num_write_kb":30630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"261'40","reported_seq":603,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.557811+0000","last_change":"2024-04-22T20:48:45.557842+0000","last_active":"2024-04-22T20:48:45.557811+0000","last_peered":"2024-04-22T20:48:45.557811+0000","last_clean":"2024-04-22T20:48:45.557811+0000","last_became_active":"2024-04-22T19:51:28.513693+0000","last_became_peered":"2024-04-22T19:51:28.513693+0000","last_unstale":"2024-04-22T20:48:45.557811+0000","last_undegraded":"2024-04-22T20:48:45.557811+0000","last_fullsized":"2024-04-22T20:48:45.557811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'40","last_scrub_stamp":"2024-04-22T20:48:45.557708+0000","last_deep_scrub":"261'40","last_deep_scrub_stamp":"2024-04-22T20:48:45.557708+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.557708+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:39:08.485749+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"261'444","reported_seq":1196,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.909652+0000","last_change":"2024-04-22T20:48:54.909652+0000","last_active":"2024-04-22T20:48:54.909652+0000","last_peered":"2024-04-22T20:48:54.909652+0000","last_clean":"2024-04-22T20:48:54.909652+0000","last_became_active":"2024-04-22T19:51:26.508716+0000","last_became_peered":"2024-04-22T19:51:26.508716+0000","last_unstale":"2024-04-22T20:48:54.909652+0000","last_undegraded":"2024-04-22T20:48:54.909652+0000","last_fullsized":"2024-04-22T20:48:54.909652+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'444","last_scrub_stamp":"2024-04-22T20:48:54.909489+0000","last_deep_scrub":"261'444","last_deep_scrub_stamp":"2024-04-22T20:48:54.909489+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.909489+0000","objects_scrubbed":4,"log_size":44,"log_dups_size":400,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:47:48.964740+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661200,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1035,"num_read_kb":17244,"num_write":1523,"num_write_kb":25883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"261'145","reported_seq":752,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.279346+0000","last_change":"2024-04-22T20:48:50.279372+0000","last_active":"2024-04-22T20:48:50.279346+0000","last_peered":"2024-04-22T20:48:50.279346+0000","last_clean":"2024-04-22T20:48:50.279346+0000","last_became_active":"2024-04-22T19:51:28.518501+0000","last_became_peered":"2024-04-22T19:51:28.518501+0000","last_unstale":"2024-04-22T20:48:50.279346+0000","last_undegraded":"2024-04-22T20:48:50.279346+0000","last_fullsized":"2024-04-22T20:48:50.279346+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'145","last_scrub_stamp":"2024-04-22T20:48:50.279249+0000","last_deep_scrub":"261'145","last_deep_scrub_stamp":"2024-04-22T20:48:50.279249+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.279249+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:46:54.700622+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":597,"num_read_kb":478,"num_write":226,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"261'377","reported_seq":1040,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.256282+0000","last_change":"2024-04-22T19:51:26.508109+0000","last_active":"2024-04-22T20:48:25.256282+0000","last_peered":"2024-04-22T20:48:25.256282+0000","last_clean":"2024-04-22T20:48:25.256282+0000","last_became_active":"2024-04-22T19:51:26.507141+0000","last_became_peered":"2024-04-22T19:51:26.507141+0000","last_unstale":"2024-04-22T20:48:25.256282+0000","last_undegraded":"2024-04-22T20:48:25.256282+0000","last_fullsized":"2024-04-22T20:48:25.256282+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2367620,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":550,"num_read_kb":43743,"num_write":1389,"num_write_kb":32515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"261'152","reported_seq":746,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.990255+0000","last_change":"2024-04-22T20:48:43.990284+0000","last_active":"2024-04-22T20:48:43.990255+0000","last_peered":"2024-04-22T20:48:43.990255+0000","last_clean":"2024-04-22T20:48:43.990255+0000","last_became_active":"2024-04-22T19:51:28.516012+0000","last_became_peered":"2024-04-22T19:51:28.516012+0000","last_unstale":"2024-04-22T20:48:43.990255+0000","last_undegraded":"2024-04-22T20:48:43.990255+0000","last_fullsized":"2024-04-22T20:48:43.990255+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:43.990165+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:43.990165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.990165+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:23.460100+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":470,"num_write":219,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7560,"num_omap_keys":32,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"261'333","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.415000+0000","last_change":"2024-04-22T19:51:26.505832+0000","last_active":"2024-04-22T20:48:23.415000+0000","last_peered":"2024-04-22T20:48:23.415000+0000","last_clean":"2024-04-22T20:48:23.415000+0000","last_became_active":"2024-04-22T19:51:26.502928+0000","last_became_peered":"2024-04-22T19:51:26.502928+0000","last_unstale":"2024-04-22T20:48:23.415000+0000","last_undegraded":"2024-04-22T20:48:23.415000+0000","last_fullsized":"2024-04-22T20:48:23.415000+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5414404,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":443,"num_read_kb":51849,"num_write":1202,"num_write_kb":39313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"261'60","reported_seq":638,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.590400+0000","last_change":"2024-04-22T20:48:40.590452+0000","last_active":"2024-04-22T20:48:40.590400+0000","last_peered":"2024-04-22T20:48:40.590400+0000","last_clean":"2024-04-22T20:48:40.590400+0000","last_became_active":"2024-04-22T19:51:28.509774+0000","last_became_peered":"2024-04-22T19:51:28.509774+0000","last_unstale":"2024-04-22T20:48:40.590400+0000","last_undegraded":"2024-04-22T20:48:40.590400+0000","last_fullsized":"2024-04-22T20:48:40.590400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'60","last_scrub_stamp":"2024-04-22T20:48:40.590291+0000","last_deep_scrub":"261'60","last_deep_scrub_stamp":"2024-04-22T20:48:40.590291+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.590291+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:30:33.030111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":211,"num_write":81,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"261'126","reported_seq":744,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.246253+0000","last_change":"2024-04-22T20:49:03.246307+0000","last_active":"2024-04-22T20:49:03.246253+0000","last_peered":"2024-04-22T20:49:03.246253+0000","last_clean":"2024-04-22T20:49:03.246253+0000","last_became_active":"2024-04-22T19:51:28.505562+0000","last_became_peered":"2024-04-22T19:51:28.505562+0000","last_unstale":"2024-04-22T20:49:03.246253+0000","last_undegraded":"2024-04-22T20:49:03.246253+0000","last_fullsized":"2024-04-22T20:49:03.246253+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'126","last_scrub_stamp":"2024-04-22T20:49:03.246160+0000","last_deep_scrub":"261'126","last_deep_scrub_stamp":"2024-04-22T20:49:03.246160+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.246160+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:08.512189+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":540,"num_read_kb":443,"num_write":193,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"261'307","reported_seq":940,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.883624+0000","last_change":"2024-04-22T19:51:26.507324+0000","last_active":"2024-04-22T20:48:17.883624+0000","last_peered":"2024-04-22T20:48:17.883624+0000","last_clean":"2024-04-22T20:48:17.883624+0000","last_became_active":"2024-04-22T19:51:26.504717+0000","last_became_peered":"2024-04-22T19:51:26.504717+0000","last_unstale":"2024-04-22T20:48:17.883624+0000","last_undegraded":"2024-04-22T20:48:17.883624+0000","last_fullsized":"2024-04-22T20:48:17.883624+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10266,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":28705,"num_write":1110,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"261'105","reported_seq":707,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.395682+0000","last_change":"2024-04-22T20:48:51.395709+0000","last_active":"2024-04-22T20:48:51.395682+0000","last_peered":"2024-04-22T20:48:51.395682+0000","last_clean":"2024-04-22T20:48:51.395682+0000","last_became_active":"2024-04-22T19:51:28.519230+0000","last_became_peered":"2024-04-22T19:51:28.519230+0000","last_unstale":"2024-04-22T20:48:51.395682+0000","last_undegraded":"2024-04-22T20:48:51.395682+0000","last_fullsized":"2024-04-22T20:48:51.395682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'105","last_scrub_stamp":"2024-04-22T20:48:51.395575+0000","last_deep_scrub":"261'105","last_deep_scrub_stamp":"2024-04-22T20:48:51.395575+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.395575+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:48:07.398102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":465,"num_read_kb":383,"num_write":180,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"261'381","reported_seq":1047,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.558613+0000","last_change":"2024-04-22T20:48:50.558671+0000","last_active":"2024-04-22T20:48:50.558613+0000","last_peered":"2024-04-22T20:48:50.558613+0000","last_clean":"2024-04-22T20:48:50.558613+0000","last_became_active":"2024-04-22T19:51:26.496156+0000","last_became_peered":"2024-04-22T19:51:26.496156+0000","last_unstale":"2024-04-22T20:48:50.558613+0000","last_undegraded":"2024-04-22T20:48:50.558613+0000","last_fullsized":"2024-04-22T20:48:50.558613+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'381","last_scrub_stamp":"2024-04-22T20:48:50.558509+0000","last_deep_scrub":"261'381","last_deep_scrub_stamp":"2024-04-22T20:48:50.558509+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.558509+0000","objects_scrubbed":6,"log_size":81,"log_dups_size":300,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:54:45.615667+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":611,"num_read_kb":792,"num_write":1516,"num_write_kb":25865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"261'172","reported_seq":773,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.926538+0000","last_change":"2024-04-22T20:48:41.926573+0000","last_active":"2024-04-22T20:48:41.926538+0000","last_peered":"2024-04-22T20:48:41.926538+0000","last_clean":"2024-04-22T20:48:41.926538+0000","last_became_active":"2024-04-22T19:51:28.515673+0000","last_became_peered":"2024-04-22T19:51:28.515673+0000","last_unstale":"2024-04-22T20:48:41.926538+0000","last_undegraded":"2024-04-22T20:48:41.926538+0000","last_fullsized":"2024-04-22T20:48:41.926538+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'172","last_scrub_stamp":"2024-04-22T20:48:41.926446+0000","last_deep_scrub":"261'172","last_deep_scrub_stamp":"2024-04-22T20:48:41.926446+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.926446+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:29:59.234763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":666,"num_read_kb":529,"num_write":258,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"261'360","reported_seq":1079,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.586412+0000","last_change":"2024-04-22T20:48:54.586516+0000","last_active":"2024-04-22T20:48:54.586412+0000","last_peered":"2024-04-22T20:48:54.586412+0000","last_clean":"2024-04-22T20:48:54.586412+0000","last_became_active":"2024-04-22T19:51:26.502156+0000","last_became_peered":"2024-04-22T19:51:26.502156+0000","last_unstale":"2024-04-22T20:48:54.586412+0000","last_undegraded":"2024-04-22T20:48:54.586412+0000","last_fullsized":"2024-04-22T20:48:54.586412+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'360","last_scrub_stamp":"2024-04-22T20:48:54.586311+0000","last_deep_scrub":"261'360","last_deep_scrub_stamp":"2024-04-22T20:48:54.586311+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.586311+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:01.200779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1373470,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":619,"num_read_kb":68133,"num_write":1312,"num_write_kb":41095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"261'290","reported_seq":905,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.550263+0000","last_change":"2024-04-22T20:48:41.550305+0000","last_active":"2024-04-22T20:48:41.550263+0000","last_peered":"2024-04-22T20:48:41.550263+0000","last_clean":"2024-04-22T20:48:41.550263+0000","last_became_active":"2024-04-22T19:51:28.510415+0000","last_became_peered":"2024-04-22T19:51:28.510415+0000","last_unstale":"2024-04-22T20:48:41.550263+0000","last_undegraded":"2024-04-22T20:48:41.550263+0000","last_fullsized":"2024-04-22T20:48:41.550263+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'290","last_scrub_stamp":"2024-04-22T20:48:41.550169+0000","last_deep_scrub":"261'290","last_deep_scrub_stamp":"2024-04-22T20:48:41.550169+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.550169+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:30:48.444121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1119,"num_read_kb":869,"num_write":475,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"261'320","reported_seq":974,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.247648+0000","last_change":"2024-04-22T20:49:02.247714+0000","last_active":"2024-04-22T20:49:02.247648+0000","last_peered":"2024-04-22T20:49:02.247648+0000","last_clean":"2024-04-22T20:49:02.247648+0000","last_became_active":"2024-04-22T19:51:26.506671+0000","last_became_peered":"2024-04-22T19:51:26.506671+0000","last_unstale":"2024-04-22T20:49:02.247648+0000","last_undegraded":"2024-04-22T20:49:02.247648+0000","last_fullsized":"2024-04-22T20:49:02.247648+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'320","last_scrub_stamp":"2024-04-22T20:49:02.247508+0000","last_deep_scrub":"261'320","last_deep_scrub_stamp":"2024-04-22T20:49:02.247508+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.247508+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:15.451052+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194361,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":6535,"num_write":1103,"num_write_kb":36003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"261'157","reported_seq":786,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.600988+0000","last_change":"2024-04-22T20:48:42.601043+0000","last_active":"2024-04-22T20:48:42.600988+0000","last_peered":"2024-04-22T20:48:42.600988+0000","last_clean":"2024-04-22T20:48:42.600988+0000","last_became_active":"2024-04-22T19:51:28.511889+0000","last_became_peered":"2024-04-22T19:51:28.511889+0000","last_unstale":"2024-04-22T20:48:42.600988+0000","last_undegraded":"2024-04-22T20:48:42.600988+0000","last_fullsized":"2024-04-22T20:48:42.600988+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'157","last_scrub_stamp":"2024-04-22T20:48:42.600881+0000","last_deep_scrub":"261'157","last_deep_scrub_stamp":"2024-04-22T20:48:42.600881+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.600881+0000","objects_scrubbed":2,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:30.539100+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":679,"num_read_kb":547,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3316,"num_omap_keys":19,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"261'334","reported_seq":982,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.944581+0000","last_change":"2024-04-22T19:51:26.498087+0000","last_active":"2024-04-22T20:48:17.944581+0000","last_peered":"2024-04-22T20:48:17.944581+0000","last_clean":"2024-04-22T20:48:17.944581+0000","last_became_active":"2024-04-22T19:51:26.497718+0000","last_became_peered":"2024-04-22T19:51:26.497718+0000","last_unstale":"2024-04-22T20:48:17.944581+0000","last_undegraded":"2024-04-22T20:48:17.944581+0000","last_fullsized":"2024-04-22T20:48:17.944581+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050340,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":471,"num_read_kb":30112,"num_write":1163,"num_write_kb":27210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"261'94","reported_seq":709,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.252306+0000","last_change":"2024-04-22T20:49:04.252334+0000","last_active":"2024-04-22T20:49:04.252306+0000","last_peered":"2024-04-22T20:49:04.252306+0000","last_clean":"2024-04-22T20:49:04.252306+0000","last_became_active":"2024-04-22T19:51:28.507673+0000","last_became_peered":"2024-04-22T19:51:28.507673+0000","last_unstale":"2024-04-22T20:49:04.252306+0000","last_undegraded":"2024-04-22T20:49:04.252306+0000","last_fullsized":"2024-04-22T20:49:04.252306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'94","last_scrub_stamp":"2024-04-22T20:49:04.252218+0000","last_deep_scrub":"261'94","last_deep_scrub_stamp":"2024-04-22T20:49:04.252218+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.252218+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:55:27.370264+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":404,"num_read_kb":350,"num_write":133,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"261'413","reported_seq":1125,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.927428+0000","last_change":"2024-04-22T19:51:26.511696+0000","last_active":"2024-04-22T20:48:17.927428+0000","last_peered":"2024-04-22T20:48:17.927428+0000","last_clean":"2024-04-22T20:48:17.927428+0000","last_became_active":"2024-04-22T19:51:26.511213+0000","last_became_peered":"2024-04-22T19:51:26.511213+0000","last_unstale":"2024-04-22T20:48:17.927428+0000","last_undegraded":"2024-04-22T20:48:17.927428+0000","last_fullsized":"2024-04-22T20:48:17.927428+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306877,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":730,"num_read_kb":36415,"num_write":1798,"num_write_kb":40524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"261'129","reported_seq":659,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.351652+0000","last_change":"2024-04-22T20:48:29.351716+0000","last_active":"2024-04-22T20:48:29.351652+0000","last_peered":"2024-04-22T20:48:29.351652+0000","last_clean":"2024-04-22T20:48:29.351652+0000","last_became_active":"2024-04-22T19:51:36.567561+0000","last_became_peered":"2024-04-22T19:51:36.567561+0000","last_unstale":"2024-04-22T20:48:29.351652+0000","last_undegraded":"2024-04-22T20:48:29.351652+0000","last_fullsized":"2024-04-22T20:48:29.351652+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'129","last_scrub_stamp":"2024-04-22T20:48:29.351556+0000","last_deep_scrub":"261'129","last_deep_scrub_stamp":"2024-04-22T20:48:29.351556+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.351556+0000","objects_scrubbed":4,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:20:31.079680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":168,"num_write":307,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"261'38","reported_seq":593,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.462844+0000","last_change":"2024-04-22T20:48:58.462869+0000","last_active":"2024-04-22T20:48:58.462844+0000","last_peered":"2024-04-22T20:48:58.462844+0000","last_clean":"2024-04-22T20:48:58.462844+0000","last_became_active":"2024-04-22T19:51:28.519994+0000","last_became_peered":"2024-04-22T19:51:28.519994+0000","last_unstale":"2024-04-22T20:48:58.462844+0000","last_undegraded":"2024-04-22T20:48:58.462844+0000","last_fullsized":"2024-04-22T20:48:58.462844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'38","last_scrub_stamp":"2024-04-22T20:48:58.462753+0000","last_deep_scrub":"261'38","last_deep_scrub_stamp":"2024-04-22T20:48:58.462753+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.462753+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:57:15.046577+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":138,"num_write":52,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"261'317","reported_seq":1059,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.402857+0000","last_change":"2024-04-22T20:48:38.402920+0000","last_active":"2024-04-22T20:48:38.402857+0000","last_peered":"2024-04-22T20:48:38.402857+0000","last_clean":"2024-04-22T20:48:38.402857+0000","last_became_active":"2024-04-22T19:53:05.051272+0000","last_became_peered":"2024-04-22T19:53:05.051272+0000","last_unstale":"2024-04-22T20:48:38.402857+0000","last_undegraded":"2024-04-22T20:48:38.402857+0000","last_fullsized":"2024-04-22T20:48:38.402857+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:38.402757+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:38.402757+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.402757+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:31:08.678074+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263217,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":662,"num_read_kb":11069,"num_write":1270,"num_write_kb":33730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.985630+0000","last_change":"2024-04-22T20:48:45.985652+0000","last_active":"2024-04-22T20:48:45.985630+0000","last_peered":"2024-04-22T20:48:45.985630+0000","last_clean":"2024-04-22T20:48:45.985630+0000","last_became_active":"2024-04-22T19:51:30.520848+0000","last_became_peered":"2024-04-22T19:51:30.520848+0000","last_unstale":"2024-04-22T20:48:45.985630+0000","last_undegraded":"2024-04-22T20:48:45.985630+0000","last_fullsized":"2024-04-22T20:48:45.985630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.985531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.985531+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.985531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:16:52.817893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"261'334","reported_seq":947,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.658874+0000","last_change":"2024-04-22T20:49:01.658905+0000","last_active":"2024-04-22T20:49:01.658874+0000","last_peered":"2024-04-22T20:49:01.658874+0000","last_clean":"2024-04-22T20:49:01.658874+0000","last_became_active":"2024-04-22T19:51:32.537695+0000","last_became_peered":"2024-04-22T19:51:32.537695+0000","last_unstale":"2024-04-22T20:49:01.658874+0000","last_undegraded":"2024-04-22T20:49:01.658874+0000","last_fullsized":"2024-04-22T20:49:01.658874+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'334","last_scrub_stamp":"2024-04-22T20:49:01.658785+0000","last_deep_scrub":"261'334","last_deep_scrub_stamp":"2024-04-22T20:49:01.658785+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.658785+0000","objects_scrubbed":5,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:50:34.689896+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39872,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":550,"num_read_kb":1178,"num_write":438,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.404496+0000","last_change":"2024-04-22T20:48:45.404533+0000","last_active":"2024-04-22T20:48:45.404496+0000","last_peered":"2024-04-22T20:48:45.404496+0000","last_clean":"2024-04-22T20:48:45.404496+0000","last_became_active":"2024-04-22T19:51:34.557285+0000","last_became_peered":"2024-04-22T19:51:34.557285+0000","last_unstale":"2024-04-22T20:48:45.404496+0000","last_undegraded":"2024-04-22T20:48:45.404496+0000","last_fullsized":"2024-04-22T20:48:45.404496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.404404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.404404+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.404404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:14:28.347705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"261'2405","reported_seq":4273,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.005951+0000","last_change":"2024-04-22T20:48:34.006025+0000","last_active":"2024-04-22T20:48:34.005951+0000","last_peered":"2024-04-22T20:48:34.005951+0000","last_clean":"2024-04-22T20:48:34.005951+0000","last_became_active":"2024-04-22T19:51:36.567195+0000","last_became_peered":"2024-04-22T19:51:36.567195+0000","last_unstale":"2024-04-22T20:48:34.005951+0000","last_undegraded":"2024-04-22T20:48:34.005951+0000","last_fullsized":"2024-04-22T20:48:34.005951+0000","mapping_epoch":30,"log_start":"261'2300","ondisk_log_start":"261'2300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2405","last_scrub_stamp":"2024-04-22T20:48:34.005850+0000","last_deep_scrub":"261'2405","last_deep_scrub_stamp":"2024-04-22T20:48:34.005850+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.005850+0000","objects_scrubbed":6,"log_size":105,"log_dups_size":2300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:06.221017+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5986,"num_read_kb":5990,"num_write":4781,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"261'131","reported_seq":730,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.955621+0000","last_change":"2024-04-22T20:48:48.955649+0000","last_active":"2024-04-22T20:48:48.955621+0000","last_peered":"2024-04-22T20:48:48.955621+0000","last_clean":"2024-04-22T20:48:48.955621+0000","last_became_active":"2024-04-22T19:51:28.512786+0000","last_became_peered":"2024-04-22T19:51:28.512786+0000","last_unstale":"2024-04-22T20:48:48.955621+0000","last_undegraded":"2024-04-22T20:48:48.955621+0000","last_fullsized":"2024-04-22T20:48:48.955621+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:48.955527+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:48.955527+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.955527+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:47:43.304224+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":408,"num_write":191,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"261'290","reported_seq":894,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.948412+0000","last_change":"2024-04-22T19:51:26.500209+0000","last_active":"2024-04-22T20:48:17.948412+0000","last_peered":"2024-04-22T20:48:17.948412+0000","last_clean":"2024-04-22T20:48:17.948412+0000","last_became_active":"2024-04-22T19:51:26.499885+0000","last_became_peered":"2024-04-22T19:51:26.499885+0000","last_unstale":"2024-04-22T20:48:17.948412+0000","last_undegraded":"2024-04-22T20:48:17.948412+0000","last_fullsized":"2024-04-22T20:48:17.948412+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050193,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":16305,"num_write":962,"num_write_kb":23193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:35.171171+0000","last_change":"2024-04-22T19:51:30.527682+0000","last_active":"2024-04-22T20:47:35.171171+0000","last_peered":"2024-04-22T20:47:35.171171+0000","last_clean":"2024-04-22T20:47:35.171171+0000","last_became_active":"2024-04-22T19:51:30.526622+0000","last_became_peered":"2024-04-22T19:51:30.526622+0000","last_unstale":"2024-04-22T20:47:35.171171+0000","last_undegraded":"2024-04-22T20:47:35.171171+0000","last_fullsized":"2024-04-22T20:47:35.171171+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"261'89","reported_seq":623,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.323750+0000","last_change":"2024-04-22T20:48:39.323782+0000","last_active":"2024-04-22T20:48:39.323750+0000","last_peered":"2024-04-22T20:48:39.323750+0000","last_clean":"2024-04-22T20:48:39.323750+0000","last_became_active":"2024-04-22T19:51:32.536180+0000","last_became_peered":"2024-04-22T19:51:32.536180+0000","last_unstale":"2024-04-22T20:48:39.323750+0000","last_undegraded":"2024-04-22T20:48:39.323750+0000","last_fullsized":"2024-04-22T20:48:39.323750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'89","last_scrub_stamp":"2024-04-22T20:48:39.323672+0000","last_deep_scrub":"261'89","last_deep_scrub_stamp":"2024-04-22T20:48:39.323672+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.323672+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:12:25.808631+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":95,"num_write":64,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.367492+0000","last_change":"2024-04-22T20:48:34.367530+0000","last_active":"2024-04-22T20:48:34.367492+0000","last_peered":"2024-04-22T20:48:34.367492+0000","last_clean":"2024-04-22T20:48:34.367492+0000","last_became_active":"2024-04-22T19:51:34.549551+0000","last_became_peered":"2024-04-22T19:51:34.549551+0000","last_unstale":"2024-04-22T20:48:34.367492+0000","last_undegraded":"2024-04-22T20:48:34.367492+0000","last_fullsized":"2024-04-22T20:48:34.367492+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.367401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.367401+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.367401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:41.545716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"261'151","reported_seq":659,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.539635+0000","last_change":"2024-04-22T20:48:38.539676+0000","last_active":"2024-04-22T20:48:38.539635+0000","last_peered":"2024-04-22T20:48:38.539635+0000","last_clean":"2024-04-22T20:48:38.539635+0000","last_became_active":"2024-04-22T19:51:36.573189+0000","last_became_peered":"2024-04-22T19:51:36.573189+0000","last_unstale":"2024-04-22T20:48:38.539635+0000","last_undegraded":"2024-04-22T20:48:38.539635+0000","last_fullsized":"2024-04-22T20:48:38.539635+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'151","last_scrub_stamp":"2024-04-22T20:48:38.539539+0000","last_deep_scrub":"261'151","last_deep_scrub_stamp":"2024-04-22T20:48:38.539539+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.539539+0000","objects_scrubbed":5,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:40:24.145792+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":328,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"261'212","reported_seq":846,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.682759+0000","last_change":"2024-04-22T20:48:58.682785+0000","last_active":"2024-04-22T20:48:58.682759+0000","last_peered":"2024-04-22T20:48:58.682759+0000","last_clean":"2024-04-22T20:48:58.682759+0000","last_became_active":"2024-04-22T19:51:28.517737+0000","last_became_peered":"2024-04-22T19:51:28.517737+0000","last_unstale":"2024-04-22T20:48:58.682759+0000","last_undegraded":"2024-04-22T20:48:58.682759+0000","last_fullsized":"2024-04-22T20:48:58.682759+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'212","last_scrub_stamp":"2024-04-22T20:48:58.682670+0000","last_deep_scrub":"261'212","last_deep_scrub_stamp":"2024-04-22T20:48:58.682670+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.682670+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:37:06.432832+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":871,"num_read_kb":733,"num_write":338,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"261'298","reported_seq":951,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.509069+0000","last_change":"2024-04-22T20:49:01.509118+0000","last_active":"2024-04-22T20:49:01.509069+0000","last_peered":"2024-04-22T20:49:01.509069+0000","last_clean":"2024-04-22T20:49:01.509069+0000","last_became_active":"2024-04-22T19:51:26.508933+0000","last_became_peered":"2024-04-22T19:51:26.508933+0000","last_unstale":"2024-04-22T20:49:01.509069+0000","last_undegraded":"2024-04-22T20:49:01.509069+0000","last_fullsized":"2024-04-22T20:49:01.509069+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'298","last_scrub_stamp":"2024-04-22T20:49:01.508963+0000","last_deep_scrub":"261'298","last_deep_scrub_stamp":"2024-04-22T20:49:01.508963+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.508963+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:48:40.562892+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632139,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":513,"num_read_kb":15262,"num_write":1122,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:05.167813+0000","last_change":"2024-04-22T19:51:30.528152+0000","last_active":"2024-04-22T20:47:05.167813+0000","last_peered":"2024-04-22T20:47:05.167813+0000","last_clean":"2024-04-22T20:47:05.167813+0000","last_became_active":"2024-04-22T19:51:30.527712+0000","last_became_peered":"2024-04-22T19:51:30.527712+0000","last_unstale":"2024-04-22T20:47:05.167813+0000","last_undegraded":"2024-04-22T20:47:05.167813+0000","last_fullsized":"2024-04-22T20:47:05.167813+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"261'118","reported_seq":671,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.458436+0000","last_change":"2024-04-22T20:48:29.458484+0000","last_active":"2024-04-22T20:48:29.458436+0000","last_peered":"2024-04-22T20:48:29.458436+0000","last_clean":"2024-04-22T20:48:29.458436+0000","last_became_active":"2024-04-22T19:51:32.536177+0000","last_became_peered":"2024-04-22T19:51:32.536177+0000","last_unstale":"2024-04-22T20:48:29.458436+0000","last_undegraded":"2024-04-22T20:48:29.458436+0000","last_fullsized":"2024-04-22T20:48:29.458436+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'118","last_scrub_stamp":"2024-04-22T20:48:29.458333+0000","last_deep_scrub":"261'118","last_deep_scrub_stamp":"2024-04-22T20:48:29.458333+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.458333+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:20:42.045716+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"261'523","reported_seq":2092,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.422107+0000","last_change":"2024-04-22T20:48:43.422135+0000","last_active":"2024-04-22T20:48:43.422107+0000","last_peered":"2024-04-22T20:48:43.422107+0000","last_clean":"2024-04-22T20:48:43.422107+0000","last_became_active":"2024-04-22T19:51:34.561828+0000","last_became_peered":"2024-04-22T19:51:34.561828+0000","last_unstale":"2024-04-22T20:48:43.422107+0000","last_undegraded":"2024-04-22T20:48:43.422107+0000","last_fullsized":"2024-04-22T20:48:43.422107+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'523","last_scrub_stamp":"2024-04-22T20:48:43.422006+0000","last_deep_scrub":"261'523","last_deep_scrub_stamp":"2024-04-22T20:48:43.422006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.422006+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":500,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:09:15.020119+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"261'176","reported_seq":997,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.374030+0000","last_change":"2024-04-22T20:48:30.374062+0000","last_active":"2024-04-22T20:48:30.374030+0000","last_peered":"2024-04-22T20:48:30.374030+0000","last_clean":"2024-04-22T20:48:30.374030+0000","last_became_active":"2024-04-22T19:51:36.565673+0000","last_became_peered":"2024-04-22T19:51:36.565673+0000","last_unstale":"2024-04-22T20:48:30.374030+0000","last_undegraded":"2024-04-22T20:48:30.374030+0000","last_fullsized":"2024-04-22T20:48:30.374030+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'176","last_scrub_stamp":"2024-04-22T20:48:30.373943+0000","last_deep_scrub":"261'176","last_deep_scrub_stamp":"2024-04-22T20:48:30.373943+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.373943+0000","objects_scrubbed":2,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:19.241756+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":579,"num_read_kb":567,"num_write":371,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"261'176","reported_seq":823,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.454747+0000","last_change":"2024-04-22T20:48:54.454777+0000","last_active":"2024-04-22T20:48:54.454747+0000","last_peered":"2024-04-22T20:48:54.454747+0000","last_clean":"2024-04-22T20:48:54.454747+0000","last_became_active":"2024-04-22T19:51:28.515670+0000","last_became_peered":"2024-04-22T19:51:28.515670+0000","last_unstale":"2024-04-22T20:48:54.454747+0000","last_undegraded":"2024-04-22T20:48:54.454747+0000","last_fullsized":"2024-04-22T20:48:54.454747+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'176","last_scrub_stamp":"2024-04-22T20:48:54.454610+0000","last_deep_scrub":"261'176","last_deep_scrub_stamp":"2024-04-22T20:48:54.454610+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.454610+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:57:21.493070+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":729,"num_read_kb":590,"num_write":271,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"261'317","reported_seq":1004,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.627131+0000","last_change":"2024-04-22T20:48:47.627177+0000","last_active":"2024-04-22T20:48:47.627131+0000","last_peered":"2024-04-22T20:48:47.627131+0000","last_clean":"2024-04-22T20:48:47.627131+0000","last_became_active":"2024-04-22T19:51:26.503438+0000","last_became_peered":"2024-04-22T19:51:26.503438+0000","last_unstale":"2024-04-22T20:48:47.627131+0000","last_undegraded":"2024-04-22T20:48:47.627131+0000","last_fullsized":"2024-04-22T20:48:47.627131+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:47.627031+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:47.627031+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.627031+0000","objects_scrubbed":7,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:03:27.258395+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3601437,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":581,"num_read_kb":32833,"num_write":1303,"num_write_kb":25399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.961966+0000","last_change":"2024-04-22T20:48:47.962021+0000","last_active":"2024-04-22T20:48:47.961966+0000","last_peered":"2024-04-22T20:48:47.961966+0000","last_clean":"2024-04-22T20:48:47.961966+0000","last_became_active":"2024-04-22T19:51:30.523125+0000","last_became_peered":"2024-04-22T19:51:30.523125+0000","last_unstale":"2024-04-22T20:48:47.961966+0000","last_undegraded":"2024-04-22T20:48:47.961966+0000","last_fullsized":"2024-04-22T20:48:47.961966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.961820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.961820+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.961820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:36:15.603182+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"261'257","reported_seq":836,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.599089+0000","last_change":"2024-04-22T20:49:04.599129+0000","last_active":"2024-04-22T20:49:04.599089+0000","last_peered":"2024-04-22T20:49:04.599089+0000","last_clean":"2024-04-22T20:49:04.599089+0000","last_became_active":"2024-04-22T19:51:33.248160+0000","last_became_peered":"2024-04-22T19:51:33.248160+0000","last_unstale":"2024-04-22T20:49:04.599089+0000","last_undegraded":"2024-04-22T20:49:04.599089+0000","last_fullsized":"2024-04-22T20:49:04.599089+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'257","last_scrub_stamp":"2024-04-22T20:49:04.598996+0000","last_deep_scrub":"261'257","last_deep_scrub_stamp":"2024-04-22T20:49:04.598996+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.598996+0000","objects_scrubbed":5,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:45:50.161224+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":425,"num_read_kb":974,"num_write":344,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"261'536","reported_seq":4238,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.560508+0000","last_change":"2024-04-22T20:48:43.560537+0000","last_active":"2024-04-22T20:48:43.560508+0000","last_peered":"2024-04-22T20:48:43.560508+0000","last_clean":"2024-04-22T20:48:43.560508+0000","last_became_active":"2024-04-22T19:51:34.555720+0000","last_became_peered":"2024-04-22T19:51:34.555720+0000","last_unstale":"2024-04-22T20:48:43.560508+0000","last_undegraded":"2024-04-22T20:48:43.560508+0000","last_fullsized":"2024-04-22T20:48:43.560508+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'536","last_scrub_stamp":"2024-04-22T20:48:43.560412+0000","last_deep_scrub":"261'536","last_deep_scrub_stamp":"2024-04-22T20:48:43.560412+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.560412+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:28.072035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"261'385","reported_seq":1066,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.949946+0000","last_change":"2024-04-22T20:48:46.949973+0000","last_active":"2024-04-22T20:48:46.949946+0000","last_peered":"2024-04-22T20:48:46.949946+0000","last_clean":"2024-04-22T20:48:46.949946+0000","last_became_active":"2024-04-22T19:51:28.512125+0000","last_became_peered":"2024-04-22T19:51:28.512125+0000","last_unstale":"2024-04-22T20:48:46.949946+0000","last_undegraded":"2024-04-22T20:48:46.949946+0000","last_fullsized":"2024-04-22T20:48:46.949946+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'385","last_scrub_stamp":"2024-04-22T20:48:46.949859+0000","last_deep_scrub":"261'385","last_deep_scrub_stamp":"2024-04-22T20:48:46.949859+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.949859+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:49.034926+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1728,"num_read_kb":1430,"num_write":663,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"261'305","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.932529+0000","last_change":"2024-04-22T19:51:26.508318+0000","last_active":"2024-04-22T20:48:17.932529+0000","last_peered":"2024-04-22T20:48:17.932529+0000","last_clean":"2024-04-22T20:48:17.932529+0000","last_became_active":"2024-04-22T19:51:26.504654+0000","last_became_peered":"2024-04-22T19:51:26.504654+0000","last_unstale":"2024-04-22T20:48:17.932529+0000","last_undegraded":"2024-04-22T20:48:17.932529+0000","last_fullsized":"2024-04-22T20:48:17.932529+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10487408,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":445,"num_read_kb":80529,"num_write":1013,"num_write_kb":26179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.400985+0000","last_change":"2024-04-22T20:48:50.401013+0000","last_active":"2024-04-22T20:48:50.400985+0000","last_peered":"2024-04-22T20:48:50.400985+0000","last_clean":"2024-04-22T20:48:50.400985+0000","last_became_active":"2024-04-22T19:51:30.526306+0000","last_became_peered":"2024-04-22T19:51:30.526306+0000","last_unstale":"2024-04-22T20:48:50.400985+0000","last_undegraded":"2024-04-22T20:48:50.400985+0000","last_fullsized":"2024-04-22T20:48:50.400985+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.400891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.400891+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.400891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:05:17.361036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"261'342","reported_seq":966,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.959517+0000","last_change":"2024-04-22T20:48:38.959567+0000","last_active":"2024-04-22T20:48:38.959517+0000","last_peered":"2024-04-22T20:48:38.959517+0000","last_clean":"2024-04-22T20:48:38.959517+0000","last_became_active":"2024-04-22T19:51:33.248059+0000","last_became_peered":"2024-04-22T19:51:33.248059+0000","last_unstale":"2024-04-22T20:48:38.959517+0000","last_undegraded":"2024-04-22T20:48:38.959517+0000","last_fullsized":"2024-04-22T20:48:38.959517+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'342","last_scrub_stamp":"2024-04-22T20:48:38.959416+0000","last_deep_scrub":"261'342","last_deep_scrub_stamp":"2024-04-22T20:48:38.959416+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.959416+0000","objects_scrubbed":9,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:10:05.470719+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45321,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":537,"num_read_kb":1022,"num_write":408,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.728684+0000","last_change":"2024-04-22T20:48:41.728710+0000","last_active":"2024-04-22T20:48:41.728684+0000","last_peered":"2024-04-22T20:48:41.728684+0000","last_clean":"2024-04-22T20:48:41.728684+0000","last_became_active":"2024-04-22T19:51:34.557662+0000","last_became_peered":"2024-04-22T19:51:34.557662+0000","last_unstale":"2024-04-22T20:48:41.728684+0000","last_undegraded":"2024-04-22T20:48:41.728684+0000","last_fullsized":"2024-04-22T20:48:41.728684+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:41.728597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:41.728597+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.728597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:59:34.359524+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"261'566","reported_seq":1383,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.924663+0000","last_change":"2024-04-22T20:48:30.924698+0000","last_active":"2024-04-22T20:48:30.924663+0000","last_peered":"2024-04-22T20:48:30.924663+0000","last_clean":"2024-04-22T20:48:30.924663+0000","last_became_active":"2024-04-22T19:51:36.570520+0000","last_became_peered":"2024-04-22T19:51:36.570520+0000","last_unstale":"2024-04-22T20:48:30.924663+0000","last_undegraded":"2024-04-22T20:48:30.924663+0000","last_fullsized":"2024-04-22T20:48:30.924663+0000","mapping_epoch":30,"log_start":"261'500","ondisk_log_start":"261'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'566","last_scrub_stamp":"2024-04-22T20:48:30.924566+0000","last_deep_scrub":"261'566","last_deep_scrub_stamp":"2024-04-22T20:48:30.924566+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.924566+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":500,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:46:13.753614+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1283,"num_read_kb":1280,"num_write":1166,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"261'311","reported_seq":968,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.300802+0000","last_change":"2024-04-22T20:48:53.300849+0000","last_active":"2024-04-22T20:48:53.300802+0000","last_peered":"2024-04-22T20:48:53.300802+0000","last_clean":"2024-04-22T20:48:53.300802+0000","last_became_active":"2024-04-22T19:51:28.511527+0000","last_became_peered":"2024-04-22T19:51:28.511527+0000","last_unstale":"2024-04-22T20:48:53.300802+0000","last_undegraded":"2024-04-22T20:48:53.300802+0000","last_fullsized":"2024-04-22T20:48:53.300802+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'311","last_scrub_stamp":"2024-04-22T20:48:53.300705+0000","last_deep_scrub":"261'311","last_deep_scrub_stamp":"2024-04-22T20:48:53.300705+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.300705+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:10.014981+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1221,"num_read_kb":977,"num_write":455,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"261'330","reported_seq":972,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.945450+0000","last_change":"2024-04-22T19:51:26.505351+0000","last_active":"2024-04-22T20:48:17.945450+0000","last_peered":"2024-04-22T20:48:17.945450+0000","last_clean":"2024-04-22T20:48:17.945450+0000","last_became_active":"2024-04-22T19:51:26.504472+0000","last_became_peered":"2024-04-22T19:51:26.504472+0000","last_unstale":"2024-04-22T20:48:17.945450+0000","last_undegraded":"2024-04-22T20:48:17.945450+0000","last_fullsized":"2024-04-22T20:48:17.945450+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158015,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":483,"num_read_kb":33343,"num_write":1137,"num_write_kb":49829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"261'6","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.511865+0000","last_change":"2024-04-22T20:48:37.511895+0000","last_active":"2024-04-22T20:48:37.511865+0000","last_peered":"2024-04-22T20:48:37.511865+0000","last_clean":"2024-04-22T20:48:37.511865+0000","last_became_active":"2024-04-22T19:51:30.524043+0000","last_became_peered":"2024-04-22T19:51:30.524043+0000","last_unstale":"2024-04-22T20:48:37.511865+0000","last_undegraded":"2024-04-22T20:48:37.511865+0000","last_fullsized":"2024-04-22T20:48:37.511865+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:37.511762+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:37.511762+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.511762+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:15.538566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":492,"num_read_kb":799,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"261'270","reported_seq":858,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.457615+0000","last_change":"2024-04-22T20:48:53.457678+0000","last_active":"2024-04-22T20:48:53.457615+0000","last_peered":"2024-04-22T20:48:53.457615+0000","last_clean":"2024-04-22T20:48:53.457615+0000","last_became_active":"2024-04-22T19:51:32.537968+0000","last_became_peered":"2024-04-22T19:51:32.537968+0000","last_unstale":"2024-04-22T20:48:53.457615+0000","last_undegraded":"2024-04-22T20:48:53.457615+0000","last_fullsized":"2024-04-22T20:48:53.457615+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'270","last_scrub_stamp":"2024-04-22T20:48:53.457514+0000","last_deep_scrub":"261'270","last_deep_scrub_stamp":"2024-04-22T20:48:53.457514+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.457514+0000","objects_scrubbed":4,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:35.634800+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43364,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":998,"num_write":350,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":506,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.498235+0000","last_change":"2024-04-22T20:48:45.498260+0000","last_active":"2024-04-22T20:48:45.498235+0000","last_peered":"2024-04-22T20:48:45.498235+0000","last_clean":"2024-04-22T20:48:45.498235+0000","last_became_active":"2024-04-22T19:51:34.555889+0000","last_became_peered":"2024-04-22T19:51:34.555889+0000","last_unstale":"2024-04-22T20:48:45.498235+0000","last_undegraded":"2024-04-22T20:48:45.498235+0000","last_fullsized":"2024-04-22T20:48:45.498235+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.498142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.498142+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.498142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:09:57.323691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"261'80","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.463200+0000","last_change":"2024-04-22T20:48:31.463234+0000","last_active":"2024-04-22T20:48:31.463200+0000","last_peered":"2024-04-22T20:48:31.463200+0000","last_clean":"2024-04-22T20:48:31.463200+0000","last_became_active":"2024-04-22T19:51:36.574555+0000","last_became_peered":"2024-04-22T19:51:36.574555+0000","last_unstale":"2024-04-22T20:48:31.463200+0000","last_undegraded":"2024-04-22T20:48:31.463200+0000","last_fullsized":"2024-04-22T20:48:31.463200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'80","last_scrub_stamp":"2024-04-22T20:48:31.463110+0000","last_deep_scrub":"261'80","last_deep_scrub_stamp":"2024-04-22T20:48:31.463110+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.463110+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:09:05.164724+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":91,"num_write":194,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"261'189","reported_seq":793,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.643992+0000","last_change":"2024-04-22T20:48:39.644036+0000","last_active":"2024-04-22T20:48:39.643992+0000","last_peered":"2024-04-22T20:48:39.643992+0000","last_clean":"2024-04-22T20:48:39.643992+0000","last_became_active":"2024-04-22T19:51:28.519424+0000","last_became_peered":"2024-04-22T19:51:28.519424+0000","last_unstale":"2024-04-22T20:48:39.643992+0000","last_undegraded":"2024-04-22T20:48:39.643992+0000","last_fullsized":"2024-04-22T20:48:39.643992+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'189","last_scrub_stamp":"2024-04-22T20:48:39.643857+0000","last_deep_scrub":"261'189","last_deep_scrub_stamp":"2024-04-22T20:48:39.643857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.643857+0000","objects_scrubbed":1,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:30:41.321492+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":758,"num_read_kb":601,"num_write":297,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"261'289","reported_seq":920,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.419906+0000","last_change":"2024-04-22T20:48:59.420014+0000","last_active":"2024-04-22T20:48:59.419906+0000","last_peered":"2024-04-22T20:48:59.419906+0000","last_clean":"2024-04-22T20:48:59.419906+0000","last_became_active":"2024-04-22T19:51:26.510273+0000","last_became_peered":"2024-04-22T19:51:26.510273+0000","last_unstale":"2024-04-22T20:48:59.419906+0000","last_undegraded":"2024-04-22T20:48:59.419906+0000","last_fullsized":"2024-04-22T20:48:59.419906+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'289","last_scrub_stamp":"2024-04-22T20:48:59.419798+0000","last_deep_scrub":"261'289","last_deep_scrub_stamp":"2024-04-22T20:48:59.419798+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.419798+0000","objects_scrubbed":9,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:16.853038+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2115,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":380,"num_read_kb":65007,"num_write":1004,"num_write_kb":29343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.472320+0000","last_change":"2024-04-22T20:48:35.472348+0000","last_active":"2024-04-22T20:48:35.472320+0000","last_peered":"2024-04-22T20:48:35.472320+0000","last_clean":"2024-04-22T20:48:35.472320+0000","last_became_active":"2024-04-22T19:51:30.525018+0000","last_became_peered":"2024-04-22T19:51:30.525018+0000","last_unstale":"2024-04-22T20:48:35.472320+0000","last_undegraded":"2024-04-22T20:48:35.472320+0000","last_fullsized":"2024-04-22T20:48:35.472320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:35.472216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:35.472216+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.472216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:21.173339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"261'317","reported_seq":919,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.539311+0000","last_change":"2024-04-22T20:49:02.539351+0000","last_active":"2024-04-22T20:49:02.539311+0000","last_peered":"2024-04-22T20:49:02.539311+0000","last_clean":"2024-04-22T20:49:02.539311+0000","last_became_active":"2024-04-22T19:51:32.538656+0000","last_became_peered":"2024-04-22T19:51:32.538656+0000","last_unstale":"2024-04-22T20:49:02.539311+0000","last_undegraded":"2024-04-22T20:49:02.539311+0000","last_fullsized":"2024-04-22T20:49:02.539311+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:49:02.539217+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:49:02.539217+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.539217+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:34:48.236169+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47724,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":498,"num_read_kb":1086,"num_write":396,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"261'509","reported_seq":2207,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.561625+0000","last_change":"2024-04-22T20:48:42.561655+0000","last_active":"2024-04-22T20:48:42.561625+0000","last_peered":"2024-04-22T20:48:42.561625+0000","last_clean":"2024-04-22T20:48:42.561625+0000","last_became_active":"2024-04-22T19:51:34.561488+0000","last_became_peered":"2024-04-22T19:51:34.561488+0000","last_unstale":"2024-04-22T20:48:42.561625+0000","last_undegraded":"2024-04-22T20:48:42.561625+0000","last_fullsized":"2024-04-22T20:48:42.561625+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'509","last_scrub_stamp":"2024-04-22T20:48:42.561535+0000","last_deep_scrub":"261'509","last_deep_scrub_stamp":"2024-04-22T20:48:42.561535+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.561535+0000","objects_scrubbed":1,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.827544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"261'136","reported_seq":634,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.548887+0000","last_change":"2024-04-22T20:48:39.548938+0000","last_active":"2024-04-22T20:48:39.548887+0000","last_peered":"2024-04-22T20:48:39.548887+0000","last_clean":"2024-04-22T20:48:39.548887+0000","last_became_active":"2024-04-22T19:51:36.573167+0000","last_became_peered":"2024-04-22T19:51:36.573167+0000","last_unstale":"2024-04-22T20:48:39.548887+0000","last_undegraded":"2024-04-22T20:48:39.548887+0000","last_fullsized":"2024-04-22T20:48:39.548887+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'136","last_scrub_stamp":"2024-04-22T20:48:39.548772+0000","last_deep_scrub":"261'136","last_deep_scrub_stamp":"2024-04-22T20:48:39.548772+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.548772+0000","objects_scrubbed":2,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:36:24.248525+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"261'150","reported_seq":724,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.595935+0000","last_change":"2024-04-22T20:48:54.595961+0000","last_active":"2024-04-22T20:48:54.595935+0000","last_peered":"2024-04-22T20:48:54.595935+0000","last_clean":"2024-04-22T20:48:54.595935+0000","last_became_active":"2024-04-22T19:51:28.518398+0000","last_became_peered":"2024-04-22T19:51:28.518398+0000","last_unstale":"2024-04-22T20:48:54.595935+0000","last_undegraded":"2024-04-22T20:48:54.595935+0000","last_fullsized":"2024-04-22T20:48:54.595935+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'150","last_scrub_stamp":"2024-04-22T20:48:54.595847+0000","last_deep_scrub":"261'150","last_deep_scrub_stamp":"2024-04-22T20:48:54.595847+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.595847+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:35:56.696922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":596,"num_read_kb":465,"num_write":243,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"261'313","reported_seq":991,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.857532+0000","last_change":"2024-04-22T20:48:24.857599+0000","last_active":"2024-04-22T20:48:24.857532+0000","last_peered":"2024-04-22T20:48:24.857532+0000","last_clean":"2024-04-22T20:48:24.857532+0000","last_became_active":"2024-04-22T19:52:05.112910+0000","last_became_peered":"2024-04-22T19:52:05.112910+0000","last_unstale":"2024-04-22T20:48:24.857532+0000","last_undegraded":"2024-04-22T20:48:24.857532+0000","last_fullsized":"2024-04-22T20:48:24.857532+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:24.857439+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:24.857439+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.857439+0000","objects_scrubbed":12,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:24:03.346339+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":486619,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":517,"num_read_kb":39016,"num_write":1092,"num_write_kb":34073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":513,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.406179+0000","last_change":"2024-04-22T20:48:31.406203+0000","last_active":"2024-04-22T20:48:31.406179+0000","last_peered":"2024-04-22T20:48:31.406179+0000","last_clean":"2024-04-22T20:48:31.406179+0000","last_became_active":"2024-04-22T19:51:30.529011+0000","last_became_peered":"2024-04-22T19:51:30.529011+0000","last_unstale":"2024-04-22T20:48:31.406179+0000","last_undegraded":"2024-04-22T20:48:31.406179+0000","last_fullsized":"2024-04-22T20:48:31.406179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:31.406079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:31.406079+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.406079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:00:54.573347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"261'493","reported_seq":1136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.712908+0000","last_change":"2024-04-22T20:48:59.712954+0000","last_active":"2024-04-22T20:48:59.712908+0000","last_peered":"2024-04-22T20:48:59.712908+0000","last_clean":"2024-04-22T20:48:59.712908+0000","last_became_active":"2024-04-22T19:51:32.537267+0000","last_became_peered":"2024-04-22T19:51:32.537267+0000","last_unstale":"2024-04-22T20:48:59.712908+0000","last_undegraded":"2024-04-22T20:48:59.712908+0000","last_fullsized":"2024-04-22T20:48:59.712908+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'493","last_scrub_stamp":"2024-04-22T20:48:59.712822+0000","last_deep_scrub":"261'493","last_deep_scrub_stamp":"2024-04-22T20:48:59.712822+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.712822+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":400,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:05.889509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":856,"num_read_kb":2097,"num_write":704,"num_write_kb":611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.377549+0000","last_change":"2024-04-22T20:48:25.377584+0000","last_active":"2024-04-22T20:48:25.377549+0000","last_peered":"2024-04-22T20:48:25.377549+0000","last_clean":"2024-04-22T20:48:25.377549+0000","last_became_active":"2024-04-22T19:51:34.552446+0000","last_became_peered":"2024-04-22T19:51:34.552446+0000","last_unstale":"2024-04-22T20:48:25.377549+0000","last_undegraded":"2024-04-22T20:48:25.377549+0000","last_fullsized":"2024-04-22T20:48:25.377549+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:25.377456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:25.377456+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.377456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:06:04.391078+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"261'107","reported_seq":618,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.636250+0000","last_change":"2024-04-22T20:48:57.636279+0000","last_active":"2024-04-22T20:48:57.636250+0000","last_peered":"2024-04-22T20:48:57.636250+0000","last_clean":"2024-04-22T20:48:57.636250+0000","last_became_active":"2024-04-22T19:51:36.570666+0000","last_became_peered":"2024-04-22T19:51:36.570666+0000","last_unstale":"2024-04-22T20:48:57.636250+0000","last_undegraded":"2024-04-22T20:48:57.636250+0000","last_fullsized":"2024-04-22T20:48:57.636250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'107","last_scrub_stamp":"2024-04-22T20:48:57.636158+0000","last_deep_scrub":"261'107","last_deep_scrub_stamp":"2024-04-22T20:48:57.636158+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.636158+0000","objects_scrubbed":1,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:11:43.927733+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":242,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"261'168","reported_seq":804,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.326095+0000","last_change":"2024-04-22T20:48:57.326118+0000","last_active":"2024-04-22T20:48:57.326095+0000","last_peered":"2024-04-22T20:48:57.326095+0000","last_clean":"2024-04-22T20:48:57.326095+0000","last_became_active":"2024-04-22T19:51:28.508288+0000","last_became_peered":"2024-04-22T19:51:28.508288+0000","last_unstale":"2024-04-22T20:48:57.326095+0000","last_undegraded":"2024-04-22T20:48:57.326095+0000","last_fullsized":"2024-04-22T20:48:57.326095+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'168","last_scrub_stamp":"2024-04-22T20:48:57.326006+0000","last_deep_scrub":"261'168","last_deep_scrub_stamp":"2024-04-22T20:48:57.326006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.326006+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:08.291741+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":576,"num_write":244,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"261'317","reported_seq":932,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.935157+0000","last_change":"2024-04-22T19:51:26.506251+0000","last_active":"2024-04-22T20:48:17.935157+0000","last_peered":"2024-04-22T20:48:17.935157+0000","last_clean":"2024-04-22T20:48:17.935157+0000","last_became_active":"2024-04-22T19:51:26.503669+0000","last_became_peered":"2024-04-22T19:51:26.503669+0000","last_unstale":"2024-04-22T20:48:17.935157+0000","last_undegraded":"2024-04-22T20:48:17.935157+0000","last_fullsized":"2024-04-22T20:48:17.935157+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194320,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":456,"num_read_kb":4498,"num_write":1097,"num_write_kb":33252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.451692+0000","last_change":"2024-04-22T20:48:33.451746+0000","last_active":"2024-04-22T20:48:33.451692+0000","last_peered":"2024-04-22T20:48:33.451692+0000","last_clean":"2024-04-22T20:48:33.451692+0000","last_became_active":"2024-04-22T19:51:30.526420+0000","last_became_peered":"2024-04-22T19:51:30.526420+0000","last_unstale":"2024-04-22T20:48:33.451692+0000","last_undegraded":"2024-04-22T20:48:33.451692+0000","last_fullsized":"2024-04-22T20:48:33.451692+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.451567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.451567+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.451567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:35.930208+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"261'274","reported_seq":862,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.374111+0000","last_change":"2024-04-22T20:48:38.374154+0000","last_active":"2024-04-22T20:48:38.374111+0000","last_peered":"2024-04-22T20:48:38.374111+0000","last_clean":"2024-04-22T20:48:38.374111+0000","last_became_active":"2024-04-22T19:51:32.536463+0000","last_became_peered":"2024-04-22T19:51:32.536463+0000","last_unstale":"2024-04-22T20:48:38.374111+0000","last_undegraded":"2024-04-22T20:48:38.374111+0000","last_fullsized":"2024-04-22T20:48:38.374111+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'274","last_scrub_stamp":"2024-04-22T20:48:38.374022+0000","last_deep_scrub":"261'274","last_deep_scrub_stamp":"2024-04-22T20:48:38.374022+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.374022+0000","objects_scrubbed":6,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:48:58.818076+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42355,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":457,"num_read_kb":963,"num_write":402,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.563374+0000","last_change":"2024-04-22T20:48:44.563409+0000","last_active":"2024-04-22T20:48:44.563374+0000","last_peered":"2024-04-22T20:48:44.563374+0000","last_clean":"2024-04-22T20:48:44.563374+0000","last_became_active":"2024-04-22T19:51:34.554737+0000","last_became_peered":"2024-04-22T19:51:34.554737+0000","last_unstale":"2024-04-22T20:48:44.563374+0000","last_undegraded":"2024-04-22T20:48:44.563374+0000","last_fullsized":"2024-04-22T20:48:44.563374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:44.563281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:44.563281+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.563281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:04:54.592040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"261'96","reported_seq":602,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.589774+0000","last_change":"2024-04-22T20:48:31.589814+0000","last_active":"2024-04-22T20:48:31.589774+0000","last_peered":"2024-04-22T20:48:31.589774+0000","last_clean":"2024-04-22T20:48:31.589774+0000","last_became_active":"2024-04-22T19:51:36.571086+0000","last_became_peered":"2024-04-22T19:51:36.571086+0000","last_unstale":"2024-04-22T20:48:31.589774+0000","last_undegraded":"2024-04-22T20:48:31.589774+0000","last_fullsized":"2024-04-22T20:48:31.589774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'96","last_scrub_stamp":"2024-04-22T20:48:31.589672+0000","last_deep_scrub":"261'96","last_deep_scrub_stamp":"2024-04-22T20:48:31.589672+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.589672+0000","objects_scrubbed":3,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:50.688646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":72,"num_write":227,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"261'254","reported_seq":847,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.244259+0000","last_change":"2024-04-22T20:48:49.244288+0000","last_active":"2024-04-22T20:48:49.244259+0000","last_peered":"2024-04-22T20:48:49.244259+0000","last_clean":"2024-04-22T20:48:49.244259+0000","last_became_active":"2024-04-22T19:51:28.514445+0000","last_became_peered":"2024-04-22T19:51:28.514445+0000","last_unstale":"2024-04-22T20:48:49.244259+0000","last_undegraded":"2024-04-22T20:48:49.244259+0000","last_fullsized":"2024-04-22T20:48:49.244259+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'254","last_scrub_stamp":"2024-04-22T20:48:49.244171+0000","last_deep_scrub":"261'254","last_deep_scrub_stamp":"2024-04-22T20:48:49.244171+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.244171+0000","objects_scrubbed":1,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:02:41.401583+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":723,"num_write":394,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":474,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'395","reported_seq":1150,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.572198+0000","last_change":"2024-04-22T20:48:51.572259+0000","last_active":"2024-04-22T20:48:51.572198+0000","last_peered":"2024-04-22T20:48:51.572198+0000","last_clean":"2024-04-22T20:48:51.572198+0000","last_became_active":"2024-04-22T19:51:26.496408+0000","last_became_peered":"2024-04-22T19:51:26.496408+0000","last_unstale":"2024-04-22T20:48:51.572198+0000","last_undegraded":"2024-04-22T20:48:51.572198+0000","last_fullsized":"2024-04-22T20:48:51.572198+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'395","last_scrub_stamp":"2024-04-22T20:48:51.572069+0000","last_deep_scrub":"261'395","last_deep_scrub_stamp":"2024-04-22T20:48:51.572069+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.572069+0000","objects_scrubbed":4,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:05:51.329345+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":840,"num_read_kb":8023,"num_write":1479,"num_write_kb":27015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.499734+0000","last_change":"2024-04-22T20:48:33.499756+0000","last_active":"2024-04-22T20:48:33.499734+0000","last_peered":"2024-04-22T20:48:33.499734+0000","last_clean":"2024-04-22T20:48:33.499734+0000","last_became_active":"2024-04-22T19:51:30.527309+0000","last_became_peered":"2024-04-22T19:51:30.527309+0000","last_unstale":"2024-04-22T20:48:33.499734+0000","last_undegraded":"2024-04-22T20:48:33.499734+0000","last_fullsized":"2024-04-22T20:48:33.499734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.499634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.499634+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.499634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:37:13.088297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"261'314","reported_seq":919,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.436156+0000","last_change":"2024-04-22T20:48:27.436199+0000","last_active":"2024-04-22T20:48:27.436156+0000","last_peered":"2024-04-22T20:48:27.436156+0000","last_clean":"2024-04-22T20:48:27.436156+0000","last_became_active":"2024-04-22T19:51:32.539732+0000","last_became_peered":"2024-04-22T19:51:32.539732+0000","last_unstale":"2024-04-22T20:48:27.436156+0000","last_undegraded":"2024-04-22T20:48:27.436156+0000","last_fullsized":"2024-04-22T20:48:27.436156+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'314","last_scrub_stamp":"2024-04-22T20:48:27.436075+0000","last_deep_scrub":"261'314","last_deep_scrub_stamp":"2024-04-22T20:48:27.436075+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.436075+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:26:29.043824+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52364,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":512,"num_read_kb":1133,"num_write":408,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"261'490","reported_seq":2212,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.422541+0000","last_change":"2024-04-22T20:48:23.422593+0000","last_active":"2024-04-22T20:48:23.422541+0000","last_peered":"2024-04-22T20:48:23.422541+0000","last_clean":"2024-04-22T20:48:23.422541+0000","last_became_active":"2024-04-22T19:51:34.556124+0000","last_became_peered":"2024-04-22T19:51:34.556124+0000","last_unstale":"2024-04-22T20:48:23.422541+0000","last_undegraded":"2024-04-22T20:48:23.422541+0000","last_fullsized":"2024-04-22T20:48:23.422541+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'490","last_scrub_stamp":"2024-04-22T20:48:23.422457+0000","last_deep_scrub":"261'490","last_deep_scrub_stamp":"2024-04-22T20:48:23.422457+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.422457+0000","objects_scrubbed":1,"log_size":90,"log_dups_size":400,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:12.183058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"261'120","reported_seq":621,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.517134+0000","last_change":"2024-04-22T20:48:30.517192+0000","last_active":"2024-04-22T20:48:30.517134+0000","last_peered":"2024-04-22T20:48:30.517134+0000","last_clean":"2024-04-22T20:48:30.517134+0000","last_became_active":"2024-04-22T19:51:36.572913+0000","last_became_peered":"2024-04-22T19:51:36.572913+0000","last_unstale":"2024-04-22T20:48:30.517134+0000","last_undegraded":"2024-04-22T20:48:30.517134+0000","last_fullsized":"2024-04-22T20:48:30.517134+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'120","last_scrub_stamp":"2024-04-22T20:48:30.517024+0000","last_deep_scrub":"261'120","last_deep_scrub_stamp":"2024-04-22T20:48:30.517024+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.517024+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:42.774241+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":102,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"261'338","reported_seq":1089,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.299517+0000","last_change":"2024-04-22T20:48:42.299554+0000","last_active":"2024-04-22T20:48:42.299517+0000","last_peered":"2024-04-22T20:48:42.299517+0000","last_clean":"2024-04-22T20:48:42.299517+0000","last_became_active":"2024-04-22T19:51:28.515809+0000","last_became_peered":"2024-04-22T19:51:28.515809+0000","last_unstale":"2024-04-22T20:48:42.299517+0000","last_undegraded":"2024-04-22T20:48:42.299517+0000","last_fullsized":"2024-04-22T20:48:42.299517+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'338","last_scrub_stamp":"2024-04-22T20:48:42.299436+0000","last_deep_scrub":"261'338","last_deep_scrub_stamp":"2024-04-22T20:48:42.299436+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.299436+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:24:36.610927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1504,"num_read_kb":1227,"num_write":616,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"261'336","reported_seq":980,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.290731+0000","last_change":"2024-04-22T20:48:59.290826+0000","last_active":"2024-04-22T20:48:59.290731+0000","last_peered":"2024-04-22T20:48:59.290731+0000","last_clean":"2024-04-22T20:48:59.290731+0000","last_became_active":"2024-04-22T19:51:26.499362+0000","last_became_peered":"2024-04-22T19:51:26.499362+0000","last_unstale":"2024-04-22T20:48:59.290731+0000","last_undegraded":"2024-04-22T20:48:59.290731+0000","last_fullsized":"2024-04-22T20:48:59.290731+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'336","last_scrub_stamp":"2024-04-22T20:48:59.290601+0000","last_deep_scrub":"261'336","last_deep_scrub_stamp":"2024-04-22T20:48:59.290601+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.290601+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:58:38.439661+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":26712,"num_write":1252,"num_write_kb":36357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"261'4","reported_seq":1146,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.941246+0000","last_change":"2024-04-22T20:48:53.941288+0000","last_active":"2024-04-22T20:48:53.941246+0000","last_peered":"2024-04-22T20:48:53.941246+0000","last_clean":"2024-04-22T20:48:53.941246+0000","last_became_active":"2024-04-22T19:51:30.527490+0000","last_became_peered":"2024-04-22T19:51:30.527490+0000","last_unstale":"2024-04-22T20:48:53.941246+0000","last_undegraded":"2024-04-22T20:48:53.941246+0000","last_fullsized":"2024-04-22T20:48:53.941246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:53.941159+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:53.941159+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.941159+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:57:29.554201+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":640,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"261'95","reported_seq":634,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.456406+0000","last_change":"2024-04-22T20:48:46.456437+0000","last_active":"2024-04-22T20:48:46.456406+0000","last_peered":"2024-04-22T20:48:46.456406+0000","last_clean":"2024-04-22T20:48:46.456406+0000","last_became_active":"2024-04-22T19:51:33.248349+0000","last_became_peered":"2024-04-22T19:51:33.248349+0000","last_unstale":"2024-04-22T20:48:46.456406+0000","last_undegraded":"2024-04-22T20:48:46.456406+0000","last_fullsized":"2024-04-22T20:48:46.456406+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'95","last_scrub_stamp":"2024-04-22T20:48:46.456325+0000","last_deep_scrub":"261'95","last_deep_scrub_stamp":"2024-04-22T20:48:46.456325+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.456325+0000","objects_scrubbed":5,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:50:06.808730+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":60,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"261'518","reported_seq":2081,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.510605+0000","last_change":"2024-04-22T20:48:44.510682+0000","last_active":"2024-04-22T20:48:44.510605+0000","last_peered":"2024-04-22T20:48:44.510605+0000","last_clean":"2024-04-22T20:48:44.510605+0000","last_became_active":"2024-04-22T19:51:34.561825+0000","last_became_peered":"2024-04-22T19:51:34.561825+0000","last_unstale":"2024-04-22T20:48:44.510605+0000","last_undegraded":"2024-04-22T20:48:44.510605+0000","last_fullsized":"2024-04-22T20:48:44.510605+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'518","last_scrub_stamp":"2024-04-22T20:48:44.510498+0000","last_deep_scrub":"261'518","last_deep_scrub_stamp":"2024-04-22T20:48:44.510498+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.510498+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:12:43.249410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"261'131","reported_seq":635,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.325984+0000","last_change":"2024-04-22T20:48:27.326015+0000","last_active":"2024-04-22T20:48:27.325984+0000","last_peered":"2024-04-22T20:48:27.325984+0000","last_clean":"2024-04-22T20:48:27.325984+0000","last_became_active":"2024-04-22T19:51:36.573026+0000","last_became_peered":"2024-04-22T19:51:36.573026+0000","last_unstale":"2024-04-22T20:48:27.325984+0000","last_undegraded":"2024-04-22T20:48:27.325984+0000","last_fullsized":"2024-04-22T20:48:27.325984+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:27.325897+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:27.325897+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.325897+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:17.870401+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":119,"num_write":309,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"261'207","reported_seq":827,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.659024+0000","last_change":"2024-04-22T20:48:46.659046+0000","last_active":"2024-04-22T20:48:46.659024+0000","last_peered":"2024-04-22T20:48:46.659024+0000","last_clean":"2024-04-22T20:48:46.659024+0000","last_became_active":"2024-04-22T19:51:28.512275+0000","last_became_peered":"2024-04-22T19:51:28.512275+0000","last_unstale":"2024-04-22T20:48:46.659024+0000","last_undegraded":"2024-04-22T20:48:46.659024+0000","last_fullsized":"2024-04-22T20:48:46.659024+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'207","last_scrub_stamp":"2024-04-22T20:48:46.658952+0000","last_deep_scrub":"261'207","last_deep_scrub_stamp":"2024-04-22T20:48:46.658952+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.658952+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:19:16.667922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":832,"num_read_kb":677,"num_write":302,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"261'328","reported_seq":967,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.890163+0000","last_change":"2024-04-22T19:51:26.498630+0000","last_active":"2024-04-22T20:48:17.890163+0000","last_peered":"2024-04-22T20:48:17.890163+0000","last_clean":"2024-04-22T20:48:17.890163+0000","last_became_active":"2024-04-22T19:51:26.498276+0000","last_became_peered":"2024-04-22T19:51:26.498276+0000","last_unstale":"2024-04-22T20:48:17.890163+0000","last_undegraded":"2024-04-22T20:48:17.890163+0000","last_fullsized":"2024-04-22T20:48:17.890163+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1581644,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":468,"num_read_kb":30145,"num_write":1162,"num_write_kb":23135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"32'3","reported_seq":822,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:15.534600+0000","last_change":"2024-04-22T19:51:30.530726+0000","last_active":"2024-04-22T20:48:15.534600+0000","last_peered":"2024-04-22T20:48:15.534600+0000","last_clean":"2024-04-22T20:48:15.534600+0000","last_became_active":"2024-04-22T19:51:30.530363+0000","last_became_peered":"2024-04-22T19:51:30.530363+0000","last_unstale":"2024-04-22T20:48:15.534600+0000","last_undegraded":"2024-04-22T20:48:15.534600+0000","last_fullsized":"2024-04-22T20:48:15.534600+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:29.503528+0000","last_clean_scrub_stamp":"2024-04-22T19:51:29.503528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":490,"num_read_kb":488,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"261'401","reported_seq":1010,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:16.559078+0000","last_change":"2024-04-22T19:51:32.540935+0000","last_active":"2024-04-22T20:48:16.559078+0000","last_peered":"2024-04-22T20:48:16.559078+0000","last_clean":"2024-04-22T20:48:16.559078+0000","last_became_active":"2024-04-22T19:51:32.540622+0000","last_became_peered":"2024-04-22T19:51:32.540622+0000","last_unstale":"2024-04-22T20:48:16.559078+0000","last_undegraded":"2024-04-22T20:48:16.559078+0000","last_fullsized":"2024-04-22T20:48:16.559078+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":300,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56095,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1229,"num_read_kb":1850,"num_write":1762,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.518288+0000","last_change":"2024-04-22T20:48:34.518332+0000","last_active":"2024-04-22T20:48:34.518288+0000","last_peered":"2024-04-22T20:48:34.518288+0000","last_clean":"2024-04-22T20:48:34.518288+0000","last_became_active":"2024-04-22T19:51:34.551009+0000","last_became_peered":"2024-04-22T19:51:34.551009+0000","last_unstale":"2024-04-22T20:48:34.518288+0000","last_undegraded":"2024-04-22T20:48:34.518288+0000","last_fullsized":"2024-04-22T20:48:34.518288+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.518165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.518165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.518165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:35:03.091773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"261'124","reported_seq":632,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.530433+0000","last_change":"2024-04-22T20:48:32.530468+0000","last_active":"2024-04-22T20:48:32.530433+0000","last_peered":"2024-04-22T20:48:32.530433+0000","last_clean":"2024-04-22T20:48:32.530433+0000","last_became_active":"2024-04-22T19:51:36.571517+0000","last_became_peered":"2024-04-22T19:51:36.571517+0000","last_unstale":"2024-04-22T20:48:32.530433+0000","last_undegraded":"2024-04-22T20:48:32.530433+0000","last_fullsized":"2024-04-22T20:48:32.530433+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:32.530326+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:32.530326+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.530326+0000","objects_scrubbed":3,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:41:58.642528+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":272,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"261'4187","reported_seq":4791,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.592564+0000","last_change":"2024-04-22T20:48:41.592601+0000","last_active":"2024-04-22T20:48:41.592564+0000","last_peered":"2024-04-22T20:48:41.592564+0000","last_clean":"2024-04-22T20:48:41.592564+0000","last_became_active":"2024-04-22T19:51:28.514869+0000","last_became_peered":"2024-04-22T19:51:28.514869+0000","last_unstale":"2024-04-22T20:48:41.592564+0000","last_undegraded":"2024-04-22T20:48:41.592564+0000","last_fullsized":"2024-04-22T20:48:41.592564+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4187","last_scrub_stamp":"2024-04-22T20:48:41.592448+0000","last_deep_scrub":"261'4187","last_deep_scrub_stamp":"2024-04-22T20:48:41.592448+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.592448+0000","objects_scrubbed":1,"log_size":87,"log_dups_size":3000,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:17:00.234094+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14767,"num_read_kb":11981,"num_write":6295,"num_write_kb":3142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":521,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.080937+0000","last_change":"2024-04-22T19:51:12.251217+0000","last_active":"2024-04-22T20:47:25.080937+0000","last_peered":"2024-04-22T20:47:25.080937+0000","last_clean":"2024-04-22T20:47:25.080937+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T20:47:25.080937+0000","last_undegraded":"2024-04-22T20:47:25.080937+0000","last_fullsized":"2024-04-22T20:47:25.080937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"261'336","reported_seq":1032,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.912940+0000","last_change":"2024-04-22T19:51:26.506322+0000","last_active":"2024-04-22T20:48:17.912940+0000","last_peered":"2024-04-22T20:48:17.912940+0000","last_clean":"2024-04-22T20:48:17.912940+0000","last_became_active":"2024-04-22T19:51:26.505982+0000","last_became_peered":"2024-04-22T19:51:26.505982+0000","last_unstale":"2024-04-22T20:48:17.912940+0000","last_undegraded":"2024-04-22T20:48:17.912940+0000","last_fullsized":"2024-04-22T20:48:17.912940+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893637,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":542,"num_read_kb":107293,"num_write":1359,"num_write_kb":45616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.405165+0000","last_change":"2024-04-22T20:48:56.405202+0000","last_active":"2024-04-22T20:48:56.405165+0000","last_peered":"2024-04-22T20:48:56.405165+0000","last_clean":"2024-04-22T20:48:56.405165+0000","last_became_active":"2024-04-22T19:51:30.526939+0000","last_became_peered":"2024-04-22T19:51:30.526939+0000","last_unstale":"2024-04-22T20:48:56.405165+0000","last_undegraded":"2024-04-22T20:48:56.405165+0000","last_fullsized":"2024-04-22T20:48:56.405165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:56.405070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:56.405070+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.405070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:17.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"261'718","reported_seq":1428,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.940154+0000","last_change":"2024-04-22T20:48:40.940188+0000","last_active":"2024-04-22T20:48:40.940154+0000","last_peered":"2024-04-22T20:48:40.940154+0000","last_clean":"2024-04-22T20:48:40.940154+0000","last_became_active":"2024-04-22T19:51:32.542970+0000","last_became_peered":"2024-04-22T19:51:32.542970+0000","last_unstale":"2024-04-22T20:48:40.940154+0000","last_undegraded":"2024-04-22T20:48:40.940154+0000","last_fullsized":"2024-04-22T20:48:40.940154+0000","mapping_epoch":26,"log_start":"261'700","ondisk_log_start":"261'700","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'718","last_scrub_stamp":"2024-04-22T20:48:40.940067+0000","last_deep_scrub":"261'718","last_deep_scrub_stamp":"2024-04-22T20:48:40.940067+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.940067+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":700,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:32:36.401897+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142271,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1403,"num_read_kb":3194,"num_write":1318,"num_write_kb":1170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":89366,"num_omap_keys":214,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.618892+0000","last_change":"2024-04-22T20:48:48.618914+0000","last_active":"2024-04-22T20:48:48.618892+0000","last_peered":"2024-04-22T20:48:48.618892+0000","last_clean":"2024-04-22T20:48:48.618892+0000","last_became_active":"2024-04-22T19:51:34.553704+0000","last_became_peered":"2024-04-22T19:51:34.553704+0000","last_unstale":"2024-04-22T20:48:48.618892+0000","last_undegraded":"2024-04-22T20:48:48.618892+0000","last_fullsized":"2024-04-22T20:48:48.618892+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:48.618804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:48.618804+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.618804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:35:39.328500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"261'272","reported_seq":1016,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.596549+0000","last_change":"2024-04-22T20:48:46.596598+0000","last_active":"2024-04-22T20:48:46.596549+0000","last_peered":"2024-04-22T20:48:46.596549+0000","last_clean":"2024-04-22T20:48:46.596549+0000","last_became_active":"2024-04-22T19:51:36.570717+0000","last_became_peered":"2024-04-22T19:51:36.570717+0000","last_unstale":"2024-04-22T20:48:46.596549+0000","last_undegraded":"2024-04-22T20:48:46.596549+0000","last_fullsized":"2024-04-22T20:48:46.596549+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'272","last_scrub_stamp":"2024-04-22T20:48:46.596449+0000","last_deep_scrub":"261'272","last_deep_scrub_stamp":"2024-04-22T20:48:46.596449+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.596449+0000","objects_scrubbed":6,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:32:49.047679+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":568,"num_write":495,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"261'50","reported_seq":596,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.539678+0000","last_change":"2024-04-22T20:48:55.539707+0000","last_active":"2024-04-22T20:48:55.539678+0000","last_peered":"2024-04-22T20:48:55.539678+0000","last_clean":"2024-04-22T20:48:55.539678+0000","last_became_active":"2024-04-22T19:51:28.505719+0000","last_became_peered":"2024-04-22T19:51:28.505719+0000","last_unstale":"2024-04-22T20:48:55.539678+0000","last_undegraded":"2024-04-22T20:48:55.539678+0000","last_fullsized":"2024-04-22T20:48:55.539678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'50","last_scrub_stamp":"2024-04-22T20:48:55.539583+0000","last_deep_scrub":"261'50","last_deep_scrub_stamp":"2024-04-22T20:48:55.539583+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.539583+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:12.051486+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":69,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":523,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.792100+0000","last_change":"2024-04-22T20:48:57.792144+0000","last_active":"2024-04-22T20:48:57.792100+0000","last_peered":"2024-04-22T20:48:57.792100+0000","last_clean":"2024-04-22T20:48:57.792100+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T20:48:57.792100+0000","last_undegraded":"2024-04-22T20:48:57.792100+0000","last_fullsized":"2024-04-22T20:48:57.792100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.791981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.791981+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.791981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:13:35.794791+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"261'358","reported_seq":1043,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.259784+0000","last_change":"2024-04-22T20:49:00.259858+0000","last_active":"2024-04-22T20:49:00.259784+0000","last_peered":"2024-04-22T20:49:00.259784+0000","last_clean":"2024-04-22T20:49:00.259784+0000","last_became_active":"2024-04-22T19:51:26.500968+0000","last_became_peered":"2024-04-22T19:51:26.500968+0000","last_unstale":"2024-04-22T20:49:00.259784+0000","last_undegraded":"2024-04-22T20:49:00.259784+0000","last_fullsized":"2024-04-22T20:49:00.259784+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'358","last_scrub_stamp":"2024-04-22T20:49:00.259692+0000","last_deep_scrub":"261'358","last_deep_scrub_stamp":"2024-04-22T20:49:00.259692+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.259692+0000","objects_scrubbed":7,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:46:46.910363+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048617,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":597,"num_read_kb":30331,"num_write":1294,"num_write_kb":32709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.639975+0000","last_change":"2024-04-22T20:48:55.639998+0000","last_active":"2024-04-22T20:48:55.639975+0000","last_peered":"2024-04-22T20:48:55.639975+0000","last_clean":"2024-04-22T20:48:55.639975+0000","last_became_active":"2024-04-22T19:51:30.526328+0000","last_became_peered":"2024-04-22T19:51:30.526328+0000","last_unstale":"2024-04-22T20:48:55.639975+0000","last_undegraded":"2024-04-22T20:48:55.639975+0000","last_fullsized":"2024-04-22T20:48:55.639975+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:55.639878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:55.639878+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.639878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:22:06.953672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"261'131","reported_seq":693,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.513766+0000","last_change":"2024-04-22T20:48:50.513809+0000","last_active":"2024-04-22T20:48:50.513766+0000","last_peered":"2024-04-22T20:48:50.513766+0000","last_clean":"2024-04-22T20:48:50.513766+0000","last_became_active":"2024-04-22T19:51:32.540805+0000","last_became_peered":"2024-04-22T19:51:32.540805+0000","last_unstale":"2024-04-22T20:48:50.513766+0000","last_undegraded":"2024-04-22T20:48:50.513766+0000","last_fullsized":"2024-04-22T20:48:50.513766+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:50.513677+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:50.513677+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.513677+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:46.102770+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":112,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1504,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.594752+0000","last_change":"2024-04-22T20:48:40.594815+0000","last_active":"2024-04-22T20:48:40.594752+0000","last_peered":"2024-04-22T20:48:40.594752+0000","last_clean":"2024-04-22T20:48:40.594752+0000","last_became_active":"2024-04-22T19:51:34.554307+0000","last_became_peered":"2024-04-22T19:51:34.554307+0000","last_unstale":"2024-04-22T20:48:40.594752+0000","last_undegraded":"2024-04-22T20:48:40.594752+0000","last_fullsized":"2024-04-22T20:48:40.594752+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.594644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.594644+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.594644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:32:53.799541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"261'124","reported_seq":635,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.895261+0000","last_change":"2024-04-22T20:48:29.895308+0000","last_active":"2024-04-22T20:48:29.895261+0000","last_peered":"2024-04-22T20:48:29.895261+0000","last_clean":"2024-04-22T20:48:29.895261+0000","last_became_active":"2024-04-22T19:51:36.571130+0000","last_became_peered":"2024-04-22T19:51:36.571130+0000","last_unstale":"2024-04-22T20:48:29.895261+0000","last_undegraded":"2024-04-22T20:48:29.895261+0000","last_fullsized":"2024-04-22T20:48:29.895261+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:29.895174+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:29.895174+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.895174+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:20:29.352021+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":131,"num_write":305,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"261'90","reported_seq":667,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.696575+0000","last_change":"2024-04-22T20:48:44.696603+0000","last_active":"2024-04-22T20:48:44.696575+0000","last_peered":"2024-04-22T20:48:44.696575+0000","last_clean":"2024-04-22T20:48:44.696575+0000","last_became_active":"2024-04-22T19:51:28.513451+0000","last_became_peered":"2024-04-22T19:51:28.513451+0000","last_unstale":"2024-04-22T20:48:44.696575+0000","last_undegraded":"2024-04-22T20:48:44.696575+0000","last_fullsized":"2024-04-22T20:48:44.696575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'90","last_scrub_stamp":"2024-04-22T20:48:44.696479+0000","last_deep_scrub":"261'90","last_deep_scrub_stamp":"2024-04-22T20:48:44.696479+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.696479+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:49.307610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":305,"num_write":130,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":520,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.080993+0000","last_change":"2024-04-22T19:51:12.251086+0000","last_active":"2024-04-22T20:47:25.080993+0000","last_peered":"2024-04-22T20:47:25.080993+0000","last_clean":"2024-04-22T20:47:25.080993+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T20:47:25.080993+0000","last_undegraded":"2024-04-22T20:47:25.080993+0000","last_fullsized":"2024-04-22T20:47:25.080993+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"261'323","reported_seq":994,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.609782+0000","last_change":"2024-04-22T20:48:50.609839+0000","last_active":"2024-04-22T20:48:50.609782+0000","last_peered":"2024-04-22T20:48:50.609782+0000","last_clean":"2024-04-22T20:48:50.609782+0000","last_became_active":"2024-04-22T19:51:26.495405+0000","last_became_peered":"2024-04-22T19:51:26.495405+0000","last_unstale":"2024-04-22T20:48:50.609782+0000","last_undegraded":"2024-04-22T20:48:50.609782+0000","last_fullsized":"2024-04-22T20:48:50.609782+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'323","last_scrub_stamp":"2024-04-22T20:48:50.609693+0000","last_deep_scrub":"261'323","last_deep_scrub_stamp":"2024-04-22T20:48:50.609693+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.609693+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:11:03.984643+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264208,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":523,"num_read_kb":57840,"num_write":1265,"num_write_kb":50104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"261'2","reported_seq":515,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.599212+0000","last_change":"2024-04-22T20:48:45.599259+0000","last_active":"2024-04-22T20:48:45.599212+0000","last_peered":"2024-04-22T20:48:45.599212+0000","last_clean":"2024-04-22T20:48:45.599212+0000","last_became_active":"2024-04-22T19:51:30.521134+0000","last_became_peered":"2024-04-22T19:51:30.521134+0000","last_unstale":"2024-04-22T20:48:45.599212+0000","last_undegraded":"2024-04-22T20:48:45.599212+0000","last_fullsized":"2024-04-22T20:48:45.599212+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:45.599098+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:45.599098+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.599098+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:03:06.489339+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"261'230","reported_seq":1466,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.455374+0000","last_change":"2024-04-22T20:48:49.455413+0000","last_active":"2024-04-22T20:48:49.455374+0000","last_peered":"2024-04-22T20:48:49.455374+0000","last_clean":"2024-04-22T20:48:49.455374+0000","last_became_active":"2024-04-22T19:51:33.250082+0000","last_became_peered":"2024-04-22T19:51:33.250082+0000","last_unstale":"2024-04-22T20:48:49.455374+0000","last_undegraded":"2024-04-22T20:48:49.455374+0000","last_fullsized":"2024-04-22T20:48:49.455374+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'230","last_scrub_stamp":"2024-04-22T20:48:49.455289+0000","last_deep_scrub":"261'230","last_deep_scrub_stamp":"2024-04-22T20:48:49.455289+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.455289+0000","objects_scrubbed":9,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:11.492021+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":188,"num_read_kb":180,"num_write":143,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5400,"num_omap_keys":12,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.690582+0000","last_change":"2024-04-22T20:48:50.690632+0000","last_active":"2024-04-22T20:48:50.690582+0000","last_peered":"2024-04-22T20:48:50.690582+0000","last_clean":"2024-04-22T20:48:50.690582+0000","last_became_active":"2024-04-22T19:51:34.546652+0000","last_became_peered":"2024-04-22T19:51:34.546652+0000","last_unstale":"2024-04-22T20:48:50.690582+0000","last_undegraded":"2024-04-22T20:48:50.690582+0000","last_fullsized":"2024-04-22T20:48:50.690582+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.690480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.690480+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.690480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:59:48.164650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"261'258","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.494703+0000","last_change":"2024-04-22T20:48:34.494735+0000","last_active":"2024-04-22T20:48:34.494703+0000","last_peered":"2024-04-22T20:48:34.494703+0000","last_clean":"2024-04-22T20:48:34.494703+0000","last_became_active":"2024-04-22T19:51:36.568767+0000","last_became_peered":"2024-04-22T19:51:36.568767+0000","last_unstale":"2024-04-22T20:48:34.494703+0000","last_undegraded":"2024-04-22T20:48:34.494703+0000","last_fullsized":"2024-04-22T20:48:34.494703+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'258","last_scrub_stamp":"2024-04-22T20:48:34.494609+0000","last_deep_scrub":"261'258","last_deep_scrub_stamp":"2024-04-22T20:48:34.494609+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.494609+0000","objects_scrubbed":4,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:08:51.017317+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":625,"num_write":569,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"261'125","reported_seq":761,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.657888+0000","last_change":"2024-04-22T20:48:56.657915+0000","last_active":"2024-04-22T20:48:56.657888+0000","last_peered":"2024-04-22T20:48:56.657888+0000","last_clean":"2024-04-22T20:48:56.657888+0000","last_became_active":"2024-04-22T19:51:28.518084+0000","last_became_peered":"2024-04-22T19:51:28.518084+0000","last_unstale":"2024-04-22T20:48:56.657888+0000","last_undegraded":"2024-04-22T20:48:56.657888+0000","last_fullsized":"2024-04-22T20:48:56.657888+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'125","last_scrub_stamp":"2024-04-22T20:48:56.657791+0000","last_deep_scrub":"261'125","last_deep_scrub_stamp":"2024-04-22T20:48:56.657791+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.657791+0000","objects_scrubbed":1,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:58.131958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":533,"num_read_kb":444,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":546,"num_omap_keys":3,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.596104+0000","last_change":"2024-04-22T20:48:49.596128+0000","last_active":"2024-04-22T20:48:49.596104+0000","last_peered":"2024-04-22T20:48:49.596104+0000","last_clean":"2024-04-22T20:48:49.596104+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T20:48:49.596104+0000","last_undegraded":"2024-04-22T20:48:49.596104+0000","last_fullsized":"2024-04-22T20:48:49.596104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.596018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.596018+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.596018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:43:28.175145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":669,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:25.080951+0000","last_change":"2024-04-22T19:51:12.892628+0000","last_active":"2024-04-22T20:47:25.080951+0000","last_peered":"2024-04-22T20:47:25.080951+0000","last_clean":"2024-04-22T20:47:25.080951+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T20:47:25.080951+0000","last_undegraded":"2024-04-22T20:47:25.080951+0000","last_fullsized":"2024-04-22T20:47:25.080951+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:11.423705+0000","last_clean_scrub_stamp":"2024-04-22T19:51:11.423705+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"261'340","reported_seq":984,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.944151+0000","last_change":"2024-04-22T19:51:26.497957+0000","last_active":"2024-04-22T20:48:17.944151+0000","last_peered":"2024-04-22T20:48:17.944151+0000","last_clean":"2024-04-22T20:48:17.944151+0000","last_became_active":"2024-04-22T19:51:26.497608+0000","last_became_peered":"2024-04-22T19:51:26.497608+0000","last_unstale":"2024-04-22T20:48:17.944151+0000","last_undegraded":"2024-04-22T20:48:17.944151+0000","last_fullsized":"2024-04-22T20:48:17.944151+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4198438,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":523,"num_read_kb":42688,"num_write":1248,"num_write_kb":37697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.993233+0000","last_change":"2024-04-22T20:48:49.993255+0000","last_active":"2024-04-22T20:48:49.993233+0000","last_peered":"2024-04-22T20:48:49.993233+0000","last_clean":"2024-04-22T20:48:49.993233+0000","last_became_active":"2024-04-22T19:51:30.524437+0000","last_became_peered":"2024-04-22T19:51:30.524437+0000","last_unstale":"2024-04-22T20:48:49.993233+0000","last_undegraded":"2024-04-22T20:48:49.993233+0000","last_fullsized":"2024-04-22T20:48:49.993233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.993107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.993107+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.993107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:03:04.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"261'1111","reported_seq":2658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.570536+0000","last_change":"2024-04-22T20:48:35.570575+0000","last_active":"2024-04-22T20:48:35.570536+0000","last_peered":"2024-04-22T20:48:35.570536+0000","last_clean":"2024-04-22T20:48:35.570536+0000","last_became_active":"2024-04-22T19:51:32.538560+0000","last_became_peered":"2024-04-22T19:51:32.538560+0000","last_unstale":"2024-04-22T20:48:35.570536+0000","last_undegraded":"2024-04-22T20:48:35.570536+0000","last_fullsized":"2024-04-22T20:48:35.570536+0000","mapping_epoch":26,"log_start":"261'1100","ondisk_log_start":"261'1100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'1111","last_scrub_stamp":"2024-04-22T20:48:35.570456+0000","last_deep_scrub":"261'1111","last_deep_scrub_stamp":"2024-04-22T20:48:35.570456+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.570456+0000","objects_scrubbed":11,"log_size":11,"log_dups_size":1100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:51:22.759382+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148242,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1781,"num_read_kb":3545,"num_write":1100,"num_write_kb":929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5722,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.453095+0000","last_change":"2024-04-22T20:48:46.453118+0000","last_active":"2024-04-22T20:48:46.453095+0000","last_peered":"2024-04-22T20:48:46.453095+0000","last_clean":"2024-04-22T20:48:46.453095+0000","last_became_active":"2024-04-22T19:51:34.557022+0000","last_became_peered":"2024-04-22T19:51:34.557022+0000","last_unstale":"2024-04-22T20:48:46.453095+0000","last_undegraded":"2024-04-22T20:48:46.453095+0000","last_fullsized":"2024-04-22T20:48:46.453095+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:46.453005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:46.453005+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.453005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:30:22.992979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"261'167","reported_seq":684,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.647984+0000","last_change":"2024-04-22T20:48:47.648022+0000","last_active":"2024-04-22T20:48:47.647984+0000","last_peered":"2024-04-22T20:48:47.647984+0000","last_clean":"2024-04-22T20:48:47.647984+0000","last_became_active":"2024-04-22T19:51:36.567603+0000","last_became_peered":"2024-04-22T19:51:36.567603+0000","last_unstale":"2024-04-22T20:48:47.647984+0000","last_undegraded":"2024-04-22T20:48:47.647984+0000","last_fullsized":"2024-04-22T20:48:47.647984+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'167","last_scrub_stamp":"2024-04-22T20:48:47.647894+0000","last_deep_scrub":"261'167","last_deep_scrub_stamp":"2024-04-22T20:48:47.647894+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.647894+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:03:44.321582+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":641,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":222,"num_read_kb":216,"num_write":373,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"261'364","reported_seq":1007,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.287319+0000","last_change":"2024-04-22T20:48:51.287344+0000","last_active":"2024-04-22T20:48:51.287319+0000","last_peered":"2024-04-22T20:48:51.287319+0000","last_clean":"2024-04-22T20:48:51.287319+0000","last_became_active":"2024-04-22T19:51:28.512561+0000","last_became_peered":"2024-04-22T19:51:28.512561+0000","last_unstale":"2024-04-22T20:48:51.287319+0000","last_undegraded":"2024-04-22T20:48:51.287319+0000","last_fullsized":"2024-04-22T20:48:51.287319+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'364","last_scrub_stamp":"2024-04-22T20:48:51.287228+0000","last_deep_scrub":"261'364","last_deep_scrub_stamp":"2024-04-22T20:48:51.287228+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.287228+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:36:20.582606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1455,"num_read_kb":1128,"num_write":627,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"261'380","reported_seq":1098,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.857235+0000","last_change":"2024-04-22T19:51:26.500958+0000","last_active":"2024-04-22T20:48:17.857235+0000","last_peered":"2024-04-22T20:48:17.857235+0000","last_clean":"2024-04-22T20:48:17.857235+0000","last_became_active":"2024-04-22T19:51:26.498178+0000","last_became_peered":"2024-04-22T19:51:26.498178+0000","last_unstale":"2024-04-22T20:48:17.857235+0000","last_undegraded":"2024-04-22T20:48:17.857235+0000","last_fullsized":"2024-04-22T20:48:17.857235+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432486,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":709,"num_read_kb":47025,"num_write":1529,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":527,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:47:30.170453+0000","last_change":"2024-04-22T19:51:11.434834+0000","last_active":"2024-04-22T20:47:30.170453+0000","last_peered":"2024-04-22T20:47:30.170453+0000","last_clean":"2024-04-22T20:47:30.170453+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T20:47:30.170453+0000","last_undegraded":"2024-04-22T20:47:30.170453+0000","last_fullsized":"2024-04-22T20:47:30.170453+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:10.422172+0000","last_clean_scrub_stamp":"2024-04-22T19:51:10.422172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":516,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.407400+0000","last_change":"2024-04-22T20:49:02.407429+0000","last_active":"2024-04-22T20:49:02.407400+0000","last_peered":"2024-04-22T20:49:02.407400+0000","last_clean":"2024-04-22T20:49:02.407400+0000","last_became_active":"2024-04-22T19:51:30.523242+0000","last_became_peered":"2024-04-22T19:51:30.523242+0000","last_unstale":"2024-04-22T20:49:02.407400+0000","last_undegraded":"2024-04-22T20:49:02.407400+0000","last_fullsized":"2024-04-22T20:49:02.407400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.407287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.407287+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.407287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:26.442506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"261'707","reported_seq":1433,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.894641+0000","last_change":"2024-04-22T19:51:33.248507+0000","last_active":"2024-04-22T20:48:17.894641+0000","last_peered":"2024-04-22T20:48:17.894641+0000","last_clean":"2024-04-22T20:48:17.894641+0000","last_became_active":"2024-04-22T19:51:33.248144+0000","last_became_peered":"2024-04-22T19:51:33.248144+0000","last_unstale":"2024-04-22T20:48:17.894641+0000","last_undegraded":"2024-04-22T20:48:17.894641+0000","last_fullsized":"2024-04-22T20:48:17.894641+0000","mapping_epoch":26,"log_start":"261'600","ondisk_log_start":"261'600","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":600,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130335,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1263,"num_read_kb":2915,"num_write":1018,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.457004+0000","last_change":"2024-04-22T20:48:39.457028+0000","last_active":"2024-04-22T20:48:39.457004+0000","last_peered":"2024-04-22T20:48:39.457004+0000","last_clean":"2024-04-22T20:48:39.457004+0000","last_became_active":"2024-04-22T19:51:34.562001+0000","last_became_peered":"2024-04-22T19:51:34.562001+0000","last_unstale":"2024-04-22T20:48:39.457004+0000","last_undegraded":"2024-04-22T20:48:39.457004+0000","last_fullsized":"2024-04-22T20:48:39.457004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:39.456917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:39.456917+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.456917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:40.897612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"261'107","reported_seq":598,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.978305+0000","last_change":"2024-04-22T20:48:32.978335+0000","last_active":"2024-04-22T20:48:32.978305+0000","last_peered":"2024-04-22T20:48:32.978305+0000","last_clean":"2024-04-22T20:48:32.978305+0000","last_became_active":"2024-04-22T19:51:36.570134+0000","last_became_peered":"2024-04-22T19:51:36.570134+0000","last_unstale":"2024-04-22T20:48:32.978305+0000","last_undegraded":"2024-04-22T20:48:32.978305+0000","last_fullsized":"2024-04-22T20:48:32.978305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'107","last_scrub_stamp":"2024-04-22T20:48:32.978212+0000","last_deep_scrub":"261'107","last_deep_scrub_stamp":"2024-04-22T20:48:32.978212+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.978212+0000","objects_scrubbed":2,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:21.385230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":256,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"261'102","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.482552+0000","last_change":"2024-04-22T20:48:55.482598+0000","last_active":"2024-04-22T20:48:55.482552+0000","last_peered":"2024-04-22T20:48:55.482552+0000","last_clean":"2024-04-22T20:48:55.482552+0000","last_became_active":"2024-04-22T19:51:28.516178+0000","last_became_peered":"2024-04-22T19:51:28.516178+0000","last_unstale":"2024-04-22T20:48:55.482552+0000","last_undegraded":"2024-04-22T20:48:55.482552+0000","last_fullsized":"2024-04-22T20:48:55.482552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'102","last_scrub_stamp":"2024-04-22T20:48:55.482441+0000","last_deep_scrub":"261'102","last_deep_scrub_stamp":"2024-04-22T20:48:55.482441+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.482441+0000","objects_scrubbed":1,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:57:52.451471+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":337,"num_write":140,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"261'359","reported_seq":1055,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.868608+0000","last_change":"2024-04-22T19:51:26.503699+0000","last_active":"2024-04-22T20:48:17.868608+0000","last_peered":"2024-04-22T20:48:17.868608+0000","last_clean":"2024-04-22T20:48:17.868608+0000","last_became_active":"2024-04-22T19:51:26.502765+0000","last_became_peered":"2024-04-22T19:51:26.502765+0000","last_unstale":"2024-04-22T20:48:17.868608+0000","last_undegraded":"2024-04-22T20:48:17.868608+0000","last_fullsized":"2024-04-22T20:48:17.868608+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":300,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4728363,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":573,"num_read_kb":127053,"num_write":1383,"num_write_kb":34012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":527,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.304810+0000","last_change":"2024-04-22T20:48:58.304837+0000","last_active":"2024-04-22T20:48:58.304810+0000","last_peered":"2024-04-22T20:48:58.304810+0000","last_clean":"2024-04-22T20:48:58.304810+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T20:48:58.304810+0000","last_undegraded":"2024-04-22T20:48:58.304810+0000","last_fullsized":"2024-04-22T20:48:58.304810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:58.304721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:58.304721+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.304721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:12:13.781455+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.303397+0000","last_change":"2024-04-22T20:48:56.303445+0000","last_active":"2024-04-22T20:48:56.303397+0000","last_peered":"2024-04-22T20:48:56.303397+0000","last_clean":"2024-04-22T20:48:56.303397+0000","last_became_active":"2024-04-22T19:51:30.529437+0000","last_became_peered":"2024-04-22T19:51:30.529437+0000","last_unstale":"2024-04-22T20:48:56.303397+0000","last_undegraded":"2024-04-22T20:48:56.303397+0000","last_fullsized":"2024-04-22T20:48:56.303397+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:56.303278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:56.303278+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.303278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:53:40.592791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"261'116","reported_seq":666,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.284844+0000","last_change":"2024-04-22T20:48:41.284912+0000","last_active":"2024-04-22T20:48:41.284844+0000","last_peered":"2024-04-22T20:48:41.284844+0000","last_clean":"2024-04-22T20:48:41.284844+0000","last_became_active":"2024-04-22T19:51:32.534280+0000","last_became_peered":"2024-04-22T19:51:32.534280+0000","last_unstale":"2024-04-22T20:48:41.284844+0000","last_undegraded":"2024-04-22T20:48:41.284844+0000","last_fullsized":"2024-04-22T20:48:41.284844+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'116","last_scrub_stamp":"2024-04-22T20:48:41.284749+0000","last_deep_scrub":"261'116","last_deep_scrub_stamp":"2024-04-22T20:48:41.284749+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.284749+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:15:26.736573+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"261'536","reported_seq":2165,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.509077+0000","last_change":"2024-04-22T20:48:41.509108+0000","last_active":"2024-04-22T20:48:41.509077+0000","last_peered":"2024-04-22T20:48:41.509077+0000","last_clean":"2024-04-22T20:48:41.509077+0000","last_became_active":"2024-04-22T19:51:34.556673+0000","last_became_peered":"2024-04-22T19:51:34.556673+0000","last_unstale":"2024-04-22T20:48:41.509077+0000","last_undegraded":"2024-04-22T20:48:41.509077+0000","last_fullsized":"2024-04-22T20:48:41.509077+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'536","last_scrub_stamp":"2024-04-22T20:48:41.508985+0000","last_deep_scrub":"261'536","last_deep_scrub_stamp":"2024-04-22T20:48:41.508985+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.508985+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:10:21.553002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"261'152","reported_seq":651,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.460819+0000","last_change":"2024-04-22T20:48:34.460850+0000","last_active":"2024-04-22T20:48:34.460819+0000","last_peered":"2024-04-22T20:48:34.460819+0000","last_clean":"2024-04-22T20:48:34.460819+0000","last_became_active":"2024-04-22T19:51:36.575351+0000","last_became_peered":"2024-04-22T19:51:36.575351+0000","last_unstale":"2024-04-22T20:48:34.460819+0000","last_undegraded":"2024-04-22T20:48:34.460819+0000","last_fullsized":"2024-04-22T20:48:34.460819+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:34.460727+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:34.460727+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.460727+0000","objects_scrubbed":3,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:55:34.883011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":80,"num_write":319,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"261'148","reported_seq":782,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.690078+0000","last_change":"2024-04-22T20:49:00.690116+0000","last_active":"2024-04-22T20:49:00.690078+0000","last_peered":"2024-04-22T20:49:00.690078+0000","last_clean":"2024-04-22T20:49:00.690078+0000","last_became_active":"2024-04-22T19:51:28.513940+0000","last_became_peered":"2024-04-22T19:51:28.513940+0000","last_unstale":"2024-04-22T20:49:00.690078+0000","last_undegraded":"2024-04-22T20:49:00.690078+0000","last_fullsized":"2024-04-22T20:49:00.690078+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'148","last_scrub_stamp":"2024-04-22T20:49:00.689987+0000","last_deep_scrub":"261'148","last_deep_scrub_stamp":"2024-04-22T20:49:00.689987+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.689987+0000","objects_scrubbed":2,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:59.928019+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":550,"num_write":219,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3271,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"261'357","reported_seq":1004,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.580271+0000","last_change":"2024-04-22T20:48:54.580328+0000","last_active":"2024-04-22T20:48:54.580271+0000","last_peered":"2024-04-22T20:48:54.580271+0000","last_clean":"2024-04-22T20:48:54.580271+0000","last_became_active":"2024-04-22T19:51:26.505842+0000","last_became_peered":"2024-04-22T19:51:26.505842+0000","last_unstale":"2024-04-22T20:48:54.580271+0000","last_undegraded":"2024-04-22T20:48:54.580271+0000","last_fullsized":"2024-04-22T20:48:54.580271+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'357","last_scrub_stamp":"2024-04-22T20:48:54.580170+0000","last_deep_scrub":"261'357","last_deep_scrub_stamp":"2024-04-22T20:48:54.580170+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.580170+0000","objects_scrubbed":9,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:58:46.289725+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308777,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":488,"num_read_kb":22191,"num_write":1115,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.583659+0000","last_change":"2024-04-22T20:48:49.583695+0000","last_active":"2024-04-22T20:48:49.583659+0000","last_peered":"2024-04-22T20:48:49.583659+0000","last_clean":"2024-04-22T20:48:49.583659+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T20:48:49.583659+0000","last_undegraded":"2024-04-22T20:48:49.583659+0000","last_fullsized":"2024-04-22T20:48:49.583659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.583572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.583572+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.583572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:08.812143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.523764+0000","last_change":"2024-04-22T20:48:35.523791+0000","last_active":"2024-04-22T20:48:35.523764+0000","last_peered":"2024-04-22T20:48:35.523764+0000","last_clean":"2024-04-22T20:48:35.523764+0000","last_became_active":"2024-04-22T19:51:30.525777+0000","last_became_peered":"2024-04-22T19:51:30.525777+0000","last_unstale":"2024-04-22T20:48:35.523764+0000","last_undegraded":"2024-04-22T20:48:35.523764+0000","last_fullsized":"2024-04-22T20:48:35.523764+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:35.523661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:35.523661+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.523661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:53:00.959964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"261'186","reported_seq":762,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.648519+0000","last_change":"2024-04-22T20:48:39.648562+0000","last_active":"2024-04-22T20:48:39.648519+0000","last_peered":"2024-04-22T20:48:39.648519+0000","last_clean":"2024-04-22T20:48:39.648519+0000","last_became_active":"2024-04-22T19:51:33.248354+0000","last_became_peered":"2024-04-22T19:51:33.248354+0000","last_unstale":"2024-04-22T20:48:39.648519+0000","last_undegraded":"2024-04-22T20:48:39.648519+0000","last_fullsized":"2024-04-22T20:48:39.648519+0000","mapping_epoch":26,"log_start":"258'100","ondisk_log_start":"258'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'186","last_scrub_stamp":"2024-04-22T20:48:39.648420+0000","last_deep_scrub":"261'186","last_deep_scrub_stamp":"2024-04-22T20:48:39.648420+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.648420+0000","objects_scrubbed":7,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:06:13.449126+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":222,"num_read_kb":211,"num_write":148,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"261'548","reported_seq":4432,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.947950+0000","last_change":"2024-04-22T20:48:36.948010+0000","last_active":"2024-04-22T20:48:36.947950+0000","last_peered":"2024-04-22T20:48:36.947950+0000","last_clean":"2024-04-22T20:48:36.947950+0000","last_became_active":"2024-04-22T19:51:34.561206+0000","last_became_peered":"2024-04-22T19:51:34.561206+0000","last_unstale":"2024-04-22T20:48:36.947950+0000","last_undegraded":"2024-04-22T20:48:36.947950+0000","last_fullsized":"2024-04-22T20:48:36.947950+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'548","last_scrub_stamp":"2024-04-22T20:48:36.947822+0000","last_deep_scrub":"261'548","last_deep_scrub_stamp":"2024-04-22T20:48:36.947822+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.947822+0000","objects_scrubbed":1,"log_size":48,"log_dups_size":500,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:24:06.222760+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"261'185","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.541264+0000","last_change":"2024-04-22T20:48:33.541297+0000","last_active":"2024-04-22T20:48:33.541264+0000","last_peered":"2024-04-22T20:48:33.541264+0000","last_clean":"2024-04-22T20:48:33.541264+0000","last_became_active":"2024-04-22T19:51:36.569338+0000","last_became_peered":"2024-04-22T19:51:36.569338+0000","last_unstale":"2024-04-22T20:48:33.541264+0000","last_undegraded":"2024-04-22T20:48:33.541264+0000","last_fullsized":"2024-04-22T20:48:33.541264+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'185","last_scrub_stamp":"2024-04-22T20:48:33.541172+0000","last_deep_scrub":"261'185","last_deep_scrub_stamp":"2024-04-22T20:48:33.541172+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.541172+0000","objects_scrubbed":3,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:38:36.406576+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":109,"num_write":422,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"261'109","reported_seq":713,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.629698+0000","last_change":"2024-04-22T20:48:48.629746+0000","last_active":"2024-04-22T20:48:48.629698+0000","last_peered":"2024-04-22T20:48:48.629698+0000","last_clean":"2024-04-22T20:48:48.629698+0000","last_became_active":"2024-04-22T19:51:28.512337+0000","last_became_peered":"2024-04-22T19:51:28.512337+0000","last_unstale":"2024-04-22T20:48:48.629698+0000","last_undegraded":"2024-04-22T20:48:48.629698+0000","last_fullsized":"2024-04-22T20:48:48.629698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'109","last_scrub_stamp":"2024-04-22T20:48:48.629602+0000","last_deep_scrub":"261'109","last_deep_scrub_stamp":"2024-04-22T20:48:48.629602+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.629602+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:59:28.312973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":434,"num_read_kb":362,"num_write":158,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'319","reported_seq":977,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.835821+0000","last_change":"2024-04-22T20:48:56.835865+0000","last_active":"2024-04-22T20:48:56.835821+0000","last_peered":"2024-04-22T20:48:56.835821+0000","last_clean":"2024-04-22T20:48:56.835821+0000","last_became_active":"2024-04-22T19:51:26.507804+0000","last_became_peered":"2024-04-22T19:51:26.507804+0000","last_unstale":"2024-04-22T20:48:56.835821+0000","last_undegraded":"2024-04-22T20:48:56.835821+0000","last_fullsized":"2024-04-22T20:48:56.835821+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'319","last_scrub_stamp":"2024-04-22T20:48:56.835711+0000","last_deep_scrub":"261'319","last_deep_scrub_stamp":"2024-04-22T20:48:56.835711+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.835711+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:07.605731+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":511,"num_read_kb":11012,"num_write":1234,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.628800+0000","last_change":"2024-04-22T20:48:48.628832+0000","last_active":"2024-04-22T20:48:48.628800+0000","last_peered":"2024-04-22T20:48:48.628800+0000","last_clean":"2024-04-22T20:48:48.628800+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T20:48:48.628800+0000","last_undegraded":"2024-04-22T20:48:48.628800+0000","last_fullsized":"2024-04-22T20:48:48.628800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:48.628712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:48.628712+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.628712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:37:25.666135+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.639531+0000","last_change":"2024-04-22T20:48:57.639557+0000","last_active":"2024-04-22T20:48:57.639531+0000","last_peered":"2024-04-22T20:48:57.639531+0000","last_clean":"2024-04-22T20:48:57.639531+0000","last_became_active":"2024-04-22T19:51:30.526837+0000","last_became_peered":"2024-04-22T19:51:30.526837+0000","last_unstale":"2024-04-22T20:48:57.639531+0000","last_undegraded":"2024-04-22T20:48:57.639531+0000","last_fullsized":"2024-04-22T20:48:57.639531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.639444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.639444+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.639444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:33:25.769033+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"261'542","reported_seq":1213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.352519+0000","last_change":"2024-04-22T19:51:32.542607+0000","last_active":"2024-04-22T20:48:17.352519+0000","last_peered":"2024-04-22T20:48:17.352519+0000","last_clean":"2024-04-22T20:48:17.352519+0000","last_became_active":"2024-04-22T19:51:32.542344+0000","last_became_peered":"2024-04-22T19:51:32.542344+0000","last_unstale":"2024-04-22T20:48:17.352519+0000","last_undegraded":"2024-04-22T20:48:17.352519+0000","last_fullsized":"2024-04-22T20:48:17.352519+0000","mapping_epoch":26,"log_start":"261'500","ondisk_log_start":"261'500","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":500,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97214,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":946,"num_read_kb":2097,"num_write":792,"num_write_kb":634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.386032+0000","last_change":"2024-04-22T20:48:33.386077+0000","last_active":"2024-04-22T20:48:33.386032+0000","last_peered":"2024-04-22T20:48:33.386032+0000","last_clean":"2024-04-22T20:48:33.386032+0000","last_became_active":"2024-04-22T19:51:34.550951+0000","last_became_peered":"2024-04-22T19:51:34.550951+0000","last_unstale":"2024-04-22T20:48:33.386032+0000","last_undegraded":"2024-04-22T20:48:33.386032+0000","last_fullsized":"2024-04-22T20:48:33.386032+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.385902+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.385902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:42.635210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"261'118","reported_seq":632,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.587158+0000","last_change":"2024-04-22T20:48:40.587201+0000","last_active":"2024-04-22T20:48:40.587158+0000","last_peered":"2024-04-22T20:48:40.587158+0000","last_clean":"2024-04-22T20:48:40.587158+0000","last_became_active":"2024-04-22T19:51:36.572976+0000","last_became_peered":"2024-04-22T19:51:36.572976+0000","last_unstale":"2024-04-22T20:48:40.587158+0000","last_undegraded":"2024-04-22T20:48:40.587158+0000","last_fullsized":"2024-04-22T20:48:40.587158+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'118","last_scrub_stamp":"2024-04-22T20:48:40.587061+0000","last_deep_scrub":"261'118","last_deep_scrub_stamp":"2024-04-22T20:48:40.587061+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.587061+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:35:06.423672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":120,"num_write":270,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"261'131","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.497001+0000","last_change":"2024-04-22T20:48:35.497051+0000","last_active":"2024-04-22T20:48:35.497001+0000","last_peered":"2024-04-22T20:48:35.497001+0000","last_clean":"2024-04-22T20:48:35.497001+0000","last_became_active":"2024-04-22T19:51:36.574238+0000","last_became_peered":"2024-04-22T19:51:36.574238+0000","last_unstale":"2024-04-22T20:48:35.497001+0000","last_undegraded":"2024-04-22T20:48:35.497001+0000","last_fullsized":"2024-04-22T20:48:35.497001+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:35.496901+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:35.496901+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.496901+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:05:26.775249+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":193,"num_read_kb":192,"num_write":291,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"261'124","reported_seq":749,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.322836+0000","last_change":"2024-04-22T20:48:45.322877+0000","last_active":"2024-04-22T20:48:45.322836+0000","last_peered":"2024-04-22T20:48:45.322836+0000","last_clean":"2024-04-22T20:48:45.322836+0000","last_became_active":"2024-04-22T19:51:28.515334+0000","last_became_peered":"2024-04-22T19:51:28.515334+0000","last_unstale":"2024-04-22T20:48:45.322836+0000","last_undegraded":"2024-04-22T20:48:45.322836+0000","last_fullsized":"2024-04-22T20:48:45.322836+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:45.322739+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:45.322739+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.322739+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:20.200394+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":424,"num_write":184,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"261'324","reported_seq":1046,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.259253+0000","last_change":"2024-04-22T20:49:01.259314+0000","last_active":"2024-04-22T20:49:01.259253+0000","last_peered":"2024-04-22T20:49:01.259253+0000","last_clean":"2024-04-22T20:49:01.259253+0000","last_became_active":"2024-04-22T19:51:26.506150+0000","last_became_peered":"2024-04-22T19:51:26.506150+0000","last_unstale":"2024-04-22T20:49:01.259253+0000","last_undegraded":"2024-04-22T20:49:01.259253+0000","last_fullsized":"2024-04-22T20:49:01.259253+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:49:01.259157+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:49:01.259157+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.259157+0000","objects_scrubbed":11,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:51:26.309359+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097285,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":716,"num_read_kb":35402,"num_write":1112,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.602028+0000","last_change":"2024-04-22T20:48:36.602053+0000","last_active":"2024-04-22T20:48:36.602028+0000","last_peered":"2024-04-22T20:48:36.602028+0000","last_clean":"2024-04-22T20:48:36.602028+0000","last_became_active":"2024-04-22T19:51:30.522903+0000","last_became_peered":"2024-04-22T19:51:30.522903+0000","last_unstale":"2024-04-22T20:48:36.602028+0000","last_undegraded":"2024-04-22T20:48:36.602028+0000","last_fullsized":"2024-04-22T20:48:36.602028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:36.601934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:36.601934+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.601934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:58.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"261'301","reported_seq":894,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.465902+0000","last_change":"2024-04-22T20:48:51.465949+0000","last_active":"2024-04-22T20:48:51.465902+0000","last_peered":"2024-04-22T20:48:51.465902+0000","last_clean":"2024-04-22T20:48:51.465902+0000","last_became_active":"2024-04-22T19:51:32.539631+0000","last_became_peered":"2024-04-22T19:51:32.539631+0000","last_unstale":"2024-04-22T20:48:51.465902+0000","last_undegraded":"2024-04-22T20:48:51.465902+0000","last_fullsized":"2024-04-22T20:48:51.465902+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'301","last_scrub_stamp":"2024-04-22T20:48:51.465803+0000","last_deep_scrub":"261'301","last_deep_scrub_stamp":"2024-04-22T20:48:51.465803+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.465803+0000","objects_scrubbed":5,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:40.094839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44228,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":495,"num_read_kb":1138,"num_write":404,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.536890+0000","last_change":"2024-04-22T20:48:42.536915+0000","last_active":"2024-04-22T20:48:42.536890+0000","last_peered":"2024-04-22T20:48:42.536890+0000","last_clean":"2024-04-22T20:48:42.536890+0000","last_became_active":"2024-04-22T19:51:34.556650+0000","last_became_peered":"2024-04-22T19:51:34.556650+0000","last_unstale":"2024-04-22T20:48:42.536890+0000","last_undegraded":"2024-04-22T20:48:42.536890+0000","last_fullsized":"2024-04-22T20:48:42.536890+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:42.536759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:42.536759+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.536759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:46:57.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"261'99","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.450094+0000","last_change":"2024-04-22T20:48:38.450136+0000","last_active":"2024-04-22T20:48:38.450094+0000","last_peered":"2024-04-22T20:48:38.450094+0000","last_clean":"2024-04-22T20:48:38.450094+0000","last_became_active":"2024-04-22T19:51:36.566980+0000","last_became_peered":"2024-04-22T19:51:36.566980+0000","last_unstale":"2024-04-22T20:48:38.450094+0000","last_undegraded":"2024-04-22T20:48:38.450094+0000","last_fullsized":"2024-04-22T20:48:38.450094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'99","last_scrub_stamp":"2024-04-22T20:48:38.449980+0000","last_deep_scrub":"261'99","last_deep_scrub_stamp":"2024-04-22T20:48:38.449980+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.449980+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:31:45.431617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":235,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"261'130","reported_seq":743,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.716009+0000","last_change":"2024-04-22T20:48:42.716036+0000","last_active":"2024-04-22T20:48:42.716009+0000","last_peered":"2024-04-22T20:48:42.716009+0000","last_clean":"2024-04-22T20:48:42.716009+0000","last_became_active":"2024-04-22T19:51:28.517675+0000","last_became_peered":"2024-04-22T19:51:28.517675+0000","last_unstale":"2024-04-22T20:48:42.716009+0000","last_undegraded":"2024-04-22T20:48:42.716009+0000","last_fullsized":"2024-04-22T20:48:42.716009+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'130","last_scrub_stamp":"2024-04-22T20:48:42.715887+0000","last_deep_scrub":"261'130","last_deep_scrub_stamp":"2024-04-22T20:48:42.715887+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.715887+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:12:53.780947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":543,"num_read_kb":446,"num_write":195,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"261'341","reported_seq":1042,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.922424+0000","last_change":"2024-04-22T19:51:26.502622+0000","last_active":"2024-04-22T20:48:17.922424+0000","last_peered":"2024-04-22T20:48:17.922424+0000","last_clean":"2024-04-22T20:48:17.922424+0000","last_became_active":"2024-04-22T19:51:26.501897+0000","last_became_peered":"2024-04-22T19:51:26.501897+0000","last_unstale":"2024-04-22T20:48:17.922424+0000","last_undegraded":"2024-04-22T20:48:17.922424+0000","last_fullsized":"2024-04-22T20:48:17.922424+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3673455,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":613,"num_read_kb":41436,"num_write":1146,"num_write_kb":29344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.413044+0000","last_change":"2024-04-22T20:49:04.413074+0000","last_active":"2024-04-22T20:49:04.413044+0000","last_peered":"2024-04-22T20:49:04.413044+0000","last_clean":"2024-04-22T20:49:04.413044+0000","last_became_active":"2024-04-22T19:51:30.524388+0000","last_became_peered":"2024-04-22T19:51:30.524388+0000","last_unstale":"2024-04-22T20:49:04.413044+0000","last_undegraded":"2024-04-22T20:49:04.413044+0000","last_fullsized":"2024-04-22T20:49:04.413044+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:04.412948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:04.412948+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.412948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:33:44.545552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"261'52","reported_seq":565,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.671890+0000","last_change":"2024-04-22T20:49:03.671919+0000","last_active":"2024-04-22T20:49:03.671890+0000","last_peered":"2024-04-22T20:49:03.671890+0000","last_clean":"2024-04-22T20:49:03.671890+0000","last_became_active":"2024-04-22T19:51:32.539260+0000","last_became_peered":"2024-04-22T19:51:32.539260+0000","last_unstale":"2024-04-22T20:49:03.671890+0000","last_undegraded":"2024-04-22T20:49:03.671890+0000","last_fullsized":"2024-04-22T20:49:03.671890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'52","last_scrub_stamp":"2024-04-22T20:49:03.671816+0000","last_deep_scrub":"261'52","last_deep_scrub_stamp":"2024-04-22T20:49:03.671816+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.671816+0000","objects_scrubbed":3,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:59.506345+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":36,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":588,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.488848+0000","last_change":"2024-04-22T20:48:47.488871+0000","last_active":"2024-04-22T20:48:47.488848+0000","last_peered":"2024-04-22T20:48:47.488848+0000","last_clean":"2024-04-22T20:48:47.488848+0000","last_became_active":"2024-04-22T19:51:34.551294+0000","last_became_peered":"2024-04-22T19:51:34.551294+0000","last_unstale":"2024-04-22T20:48:47.488848+0000","last_undegraded":"2024-04-22T20:48:47.488848+0000","last_fullsized":"2024-04-22T20:48:47.488848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.488766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.488766+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.488766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:13:45.930564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"261'202","reported_seq":728,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.470360+0000","last_change":"2024-04-22T20:48:36.470421+0000","last_active":"2024-04-22T20:48:36.470360+0000","last_peered":"2024-04-22T20:48:36.470360+0000","last_clean":"2024-04-22T20:48:36.470360+0000","last_became_active":"2024-04-22T19:51:36.573936+0000","last_became_peered":"2024-04-22T19:51:36.573936+0000","last_unstale":"2024-04-22T20:48:36.470360+0000","last_undegraded":"2024-04-22T20:48:36.470360+0000","last_fullsized":"2024-04-22T20:48:36.470360+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'202","last_scrub_stamp":"2024-04-22T20:48:36.470271+0000","last_deep_scrub":"261'202","last_deep_scrub_stamp":"2024-04-22T20:48:36.470271+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.470271+0000","objects_scrubbed":1,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:05:08.510777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":284,"num_write":451,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"261'231","reported_seq":926,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.411158+0000","last_change":"2024-04-22T20:49:01.411188+0000","last_active":"2024-04-22T20:49:01.411158+0000","last_peered":"2024-04-22T20:49:01.411158+0000","last_clean":"2024-04-22T20:49:01.411158+0000","last_became_active":"2024-04-22T19:51:28.508307+0000","last_became_peered":"2024-04-22T19:51:28.508307+0000","last_unstale":"2024-04-22T20:49:01.411158+0000","last_undegraded":"2024-04-22T20:49:01.411158+0000","last_fullsized":"2024-04-22T20:49:01.411158+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'231","last_scrub_stamp":"2024-04-22T20:49:01.411072+0000","last_deep_scrub":"261'231","last_deep_scrub_stamp":"2024-04-22T20:49:01.411072+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.411072+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:33.901303+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":959,"num_read_kb":784,"num_write":348,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"261'323","reported_seq":963,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.793039+0000","last_change":"2024-04-22T20:48:59.793110+0000","last_active":"2024-04-22T20:48:59.793039+0000","last_peered":"2024-04-22T20:48:59.793039+0000","last_clean":"2024-04-22T20:48:59.793039+0000","last_became_active":"2024-04-22T19:51:26.501477+0000","last_became_peered":"2024-04-22T19:51:26.501477+0000","last_unstale":"2024-04-22T20:48:59.793039+0000","last_undegraded":"2024-04-22T20:48:59.793039+0000","last_fullsized":"2024-04-22T20:48:59.793039+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'323","last_scrub_stamp":"2024-04-22T20:48:59.792943+0000","last_deep_scrub":"261'323","last_deep_scrub_stamp":"2024-04-22T20:48:59.792943+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.792943+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:21.956398+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":46522,"num_write":1280,"num_write_kb":35021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.361909+0000","last_change":"2024-04-22T20:48:54.361936+0000","last_active":"2024-04-22T20:48:54.361909+0000","last_peered":"2024-04-22T20:48:54.361909+0000","last_clean":"2024-04-22T20:48:54.361909+0000","last_became_active":"2024-04-22T19:51:30.527076+0000","last_became_peered":"2024-04-22T19:51:30.527076+0000","last_unstale":"2024-04-22T20:48:54.361909+0000","last_undegraded":"2024-04-22T20:48:54.361909+0000","last_fullsized":"2024-04-22T20:48:54.361909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:54.361806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:54.361806+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.361806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:55.484727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"261'307","reported_seq":916,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.485477+0000","last_change":"2024-04-22T20:48:49.485517+0000","last_active":"2024-04-22T20:48:49.485477+0000","last_peered":"2024-04-22T20:48:49.485477+0000","last_clean":"2024-04-22T20:48:49.485477+0000","last_became_active":"2024-04-22T19:51:32.542262+0000","last_became_peered":"2024-04-22T19:51:32.542262+0000","last_unstale":"2024-04-22T20:48:49.485477+0000","last_undegraded":"2024-04-22T20:48:49.485477+0000","last_fullsized":"2024-04-22T20:48:49.485477+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'307","last_scrub_stamp":"2024-04-22T20:48:49.485363+0000","last_deep_scrub":"261'307","last_deep_scrub_stamp":"2024-04-22T20:48:49.485363+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.485363+0000","objects_scrubbed":5,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:26:58.383965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46510,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":491,"num_read_kb":1038,"num_write":384,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.651253+0000","last_change":"2024-04-22T20:48:38.651275+0000","last_active":"2024-04-22T20:48:38.651253+0000","last_peered":"2024-04-22T20:48:38.651253+0000","last_clean":"2024-04-22T20:48:38.651253+0000","last_became_active":"2024-04-22T19:51:34.553691+0000","last_became_peered":"2024-04-22T19:51:34.553691+0000","last_unstale":"2024-04-22T20:48:38.651253+0000","last_undegraded":"2024-04-22T20:48:38.651253+0000","last_fullsized":"2024-04-22T20:48:38.651253+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:38.651145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:38.651145+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.651145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:39:15.734170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"261'137","reported_seq":627,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.859857+0000","last_change":"2024-04-22T20:48:28.859930+0000","last_active":"2024-04-22T20:48:28.859857+0000","last_peered":"2024-04-22T20:48:28.859857+0000","last_clean":"2024-04-22T20:48:28.859857+0000","last_became_active":"2024-04-22T19:51:36.571290+0000","last_became_peered":"2024-04-22T19:51:36.571290+0000","last_unstale":"2024-04-22T20:48:28.859857+0000","last_undegraded":"2024-04-22T20:48:28.859857+0000","last_fullsized":"2024-04-22T20:48:28.859857+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'137","last_scrub_stamp":"2024-04-22T20:48:28.859747+0000","last_deep_scrub":"261'137","last_deep_scrub_stamp":"2024-04-22T20:48:28.859747+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.859747+0000","objects_scrubbed":2,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:02:28.549243+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":111,"num_write":334,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"261'185","reported_seq":863,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.384703+0000","last_change":"2024-04-22T20:49:03.384728+0000","last_active":"2024-04-22T20:49:03.384703+0000","last_peered":"2024-04-22T20:49:03.384703+0000","last_clean":"2024-04-22T20:49:03.384703+0000","last_became_active":"2024-04-22T19:51:28.506615+0000","last_became_peered":"2024-04-22T19:51:28.506615+0000","last_unstale":"2024-04-22T20:49:03.384703+0000","last_undegraded":"2024-04-22T20:49:03.384703+0000","last_fullsized":"2024-04-22T20:49:03.384703+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'185","last_scrub_stamp":"2024-04-22T20:49:03.384610+0000","last_deep_scrub":"261'185","last_deep_scrub_stamp":"2024-04-22T20:49:03.384610+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.384610+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:20:25.431536+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":757,"num_read_kb":623,"num_write":269,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'338","reported_seq":980,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.941579+0000","last_change":"2024-04-22T19:51:26.503480+0000","last_active":"2024-04-22T20:48:17.941579+0000","last_peered":"2024-04-22T20:48:17.941579+0000","last_clean":"2024-04-22T20:48:17.941579+0000","last_became_active":"2024-04-22T19:51:26.502938+0000","last_became_peered":"2024-04-22T19:51:26.502938+0000","last_unstale":"2024-04-22T20:48:17.941579+0000","last_undegraded":"2024-04-22T20:48:17.941579+0000","last_fullsized":"2024-04-22T20:48:17.941579+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194317,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":478,"num_read_kb":28738,"num_write":1269,"num_write_kb":45425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.504136+0000","last_change":"2024-04-22T20:48:32.504160+0000","last_active":"2024-04-22T20:48:32.504136+0000","last_peered":"2024-04-22T20:48:32.504136+0000","last_clean":"2024-04-22T20:48:32.504136+0000","last_became_active":"2024-04-22T19:51:30.526994+0000","last_became_peered":"2024-04-22T19:51:30.526994+0000","last_unstale":"2024-04-22T20:48:32.504136+0000","last_undegraded":"2024-04-22T20:48:32.504136+0000","last_fullsized":"2024-04-22T20:48:32.504136+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:32.504047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:32.504047+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.504047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:39.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"261'255","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.460418+0000","last_change":"2024-04-22T20:49:00.460454+0000","last_active":"2024-04-22T20:49:00.460418+0000","last_peered":"2024-04-22T20:49:00.460418+0000","last_clean":"2024-04-22T20:49:00.460418+0000","last_became_active":"2024-04-22T19:51:32.537875+0000","last_became_peered":"2024-04-22T19:51:32.537875+0000","last_unstale":"2024-04-22T20:49:00.460418+0000","last_undegraded":"2024-04-22T20:49:00.460418+0000","last_fullsized":"2024-04-22T20:49:00.460418+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'255","last_scrub_stamp":"2024-04-22T20:49:00.460334+0000","last_deep_scrub":"261'255","last_deep_scrub_stamp":"2024-04-22T20:49:00.460334+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.460334+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:02.585609+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":422,"num_read_kb":961,"num_write":338,"num_write_kb":268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.635537+0000","last_change":"2024-04-22T20:48:46.635559+0000","last_active":"2024-04-22T20:48:46.635537+0000","last_peered":"2024-04-22T20:48:46.635537+0000","last_clean":"2024-04-22T20:48:46.635537+0000","last_became_active":"2024-04-22T19:51:34.553954+0000","last_became_peered":"2024-04-22T19:51:34.553954+0000","last_unstale":"2024-04-22T20:48:46.635537+0000","last_undegraded":"2024-04-22T20:48:46.635537+0000","last_fullsized":"2024-04-22T20:48:46.635537+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:46.635459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:46.635459+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.635459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:54:27.052935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"261'192","reported_seq":839,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.551497+0000","last_change":"2024-04-22T20:48:43.551521+0000","last_active":"2024-04-22T20:48:43.551497+0000","last_peered":"2024-04-22T20:48:43.551497+0000","last_clean":"2024-04-22T20:48:43.551497+0000","last_became_active":"2024-04-22T19:51:28.514593+0000","last_became_peered":"2024-04-22T19:51:28.514593+0000","last_unstale":"2024-04-22T20:48:43.551497+0000","last_undegraded":"2024-04-22T20:48:43.551497+0000","last_fullsized":"2024-04-22T20:48:43.551497+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'192","last_scrub_stamp":"2024-04-22T20:48:43.551394+0000","last_deep_scrub":"261'192","last_deep_scrub_stamp":"2024-04-22T20:48:43.551394+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.551394+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:06:00.844687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":788,"num_read_kb":638,"num_write":274,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"261'342","reported_seq":1005,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.914032+0000","last_change":"2024-04-22T19:51:26.502507+0000","last_active":"2024-04-22T20:48:17.914032+0000","last_peered":"2024-04-22T20:48:17.914032+0000","last_clean":"2024-04-22T20:48:17.914032+0000","last_became_active":"2024-04-22T19:51:26.501812+0000","last_became_peered":"2024-04-22T19:51:26.501812+0000","last_unstale":"2024-04-22T20:48:17.914032+0000","last_undegraded":"2024-04-22T20:48:17.914032+0000","last_fullsized":"2024-04-22T20:48:17.914032+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857304,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":552,"num_read_kb":55834,"num_write":1162,"num_write_kb":35884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":509,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.630664+0000","last_change":"2024-04-22T20:48:53.630690+0000","last_active":"2024-04-22T20:48:53.630664+0000","last_peered":"2024-04-22T20:48:53.630664+0000","last_clean":"2024-04-22T20:48:53.630664+0000","last_became_active":"2024-04-22T19:51:30.525450+0000","last_became_peered":"2024-04-22T19:51:30.525450+0000","last_unstale":"2024-04-22T20:48:53.630664+0000","last_undegraded":"2024-04-22T20:48:53.630664+0000","last_fullsized":"2024-04-22T20:48:53.630664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:53.630576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:53.630576+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.630576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:07:46.156214+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"261'489","reported_seq":1127,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.643184+0000","last_change":"2024-04-22T20:48:37.643184+0000","last_active":"2024-04-22T20:48:37.643184+0000","last_peered":"2024-04-22T20:48:37.643184+0000","last_clean":"2024-04-22T20:48:37.643184+0000","last_became_active":"2024-04-22T19:51:32.540755+0000","last_became_peered":"2024-04-22T19:51:32.540755+0000","last_unstale":"2024-04-22T20:48:37.643184+0000","last_undegraded":"2024-04-22T20:48:37.643184+0000","last_fullsized":"2024-04-22T20:48:37.643184+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'489","last_scrub_stamp":"2024-04-22T20:48:37.643033+0000","last_deep_scrub":"261'489","last_deep_scrub_stamp":"2024-04-22T20:48:37.643033+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.643033+0000","objects_scrubbed":7,"log_size":89,"log_dups_size":400,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:20:59.082062+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93788,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":875,"num_read_kb":2058,"num_write":756,"num_write_kb":671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23904,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.952860+0000","last_change":"2024-04-22T20:48:37.952860+0000","last_active":"2024-04-22T20:48:37.952860+0000","last_peered":"2024-04-22T20:48:37.952860+0000","last_clean":"2024-04-22T20:48:37.952860+0000","last_became_active":"2024-04-22T19:51:34.553363+0000","last_became_peered":"2024-04-22T19:51:34.553363+0000","last_unstale":"2024-04-22T20:48:37.952860+0000","last_undegraded":"2024-04-22T20:48:37.952860+0000","last_fullsized":"2024-04-22T20:48:37.952860+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:37.952749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:37.952749+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.952749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:14.410865+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"261'162","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.484277+0000","last_change":"2024-04-22T20:48:35.484310+0000","last_active":"2024-04-22T20:48:35.484277+0000","last_peered":"2024-04-22T20:48:35.484277+0000","last_clean":"2024-04-22T20:48:35.484277+0000","last_became_active":"2024-04-22T19:51:36.568373+0000","last_became_peered":"2024-04-22T19:51:36.568373+0000","last_unstale":"2024-04-22T20:48:35.484277+0000","last_undegraded":"2024-04-22T20:48:35.484277+0000","last_fullsized":"2024-04-22T20:48:35.484277+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'162","last_scrub_stamp":"2024-04-22T20:48:35.484193+0000","last_deep_scrub":"261'162","last_deep_scrub_stamp":"2024-04-22T20:48:35.484193+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.484193+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:59:33.641598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":213,"num_write":373,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"261'100","reported_seq":694,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.619802+0000","last_change":"2024-04-22T20:49:02.619828+0000","last_active":"2024-04-22T20:49:02.619802+0000","last_peered":"2024-04-22T20:49:02.619802+0000","last_clean":"2024-04-22T20:49:02.619802+0000","last_became_active":"2024-04-22T19:51:28.510629+0000","last_became_peered":"2024-04-22T19:51:28.510629+0000","last_unstale":"2024-04-22T20:49:02.619802+0000","last_undegraded":"2024-04-22T20:49:02.619802+0000","last_fullsized":"2024-04-22T20:49:02.619802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'100","last_scrub_stamp":"2024-04-22T20:49:02.619712+0000","last_deep_scrub":"261'100","last_deep_scrub_stamp":"2024-04-22T20:49:02.619712+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.619712+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:26.099759+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":435,"num_read_kb":357,"num_write":161,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"261'305","reported_seq":947,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.605247+0000","last_change":"2024-04-22T20:48:52.605390+0000","last_active":"2024-04-22T20:48:52.605247+0000","last_peered":"2024-04-22T20:48:52.605247+0000","last_clean":"2024-04-22T20:48:52.605247+0000","last_became_active":"2024-04-22T19:51:26.497165+0000","last_became_peered":"2024-04-22T19:51:26.497165+0000","last_unstale":"2024-04-22T20:48:52.605247+0000","last_undegraded":"2024-04-22T20:48:52.605247+0000","last_fullsized":"2024-04-22T20:48:52.605247+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'305","last_scrub_stamp":"2024-04-22T20:48:52.605144+0000","last_deep_scrub":"261'305","last_deep_scrub_stamp":"2024-04-22T20:48:52.605144+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.605144+0000","objects_scrubbed":9,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:37.401989+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":662323,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":493,"num_read_kb":39207,"num_write":1122,"num_write_kb":42297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.479719+0000","last_change":"2024-04-22T20:48:33.479741+0000","last_active":"2024-04-22T20:48:33.479719+0000","last_peered":"2024-04-22T20:48:33.479719+0000","last_clean":"2024-04-22T20:48:33.479719+0000","last_became_active":"2024-04-22T19:51:30.526597+0000","last_became_peered":"2024-04-22T19:51:30.526597+0000","last_unstale":"2024-04-22T20:48:33.479719+0000","last_undegraded":"2024-04-22T20:48:33.479719+0000","last_fullsized":"2024-04-22T20:48:33.479719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.479622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.479622+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.479622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:35:51.805769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"261'319","reported_seq":916,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.463525+0000","last_change":"2024-04-22T20:48:58.463525+0000","last_active":"2024-04-22T20:48:58.463525+0000","last_peered":"2024-04-22T20:48:58.463525+0000","last_clean":"2024-04-22T20:48:58.463525+0000","last_became_active":"2024-04-22T19:51:32.537416+0000","last_became_peered":"2024-04-22T19:51:32.537416+0000","last_unstale":"2024-04-22T20:48:58.463525+0000","last_undegraded":"2024-04-22T20:48:58.463525+0000","last_fullsized":"2024-04-22T20:48:58.463525+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'319","last_scrub_stamp":"2024-04-22T20:48:58.463340+0000","last_deep_scrub":"261'319","last_deep_scrub_stamp":"2024-04-22T20:48:58.463340+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.463340+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:11:45.125481+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43451,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":521,"num_read_kb":1119,"num_write":436,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.694176+0000","last_change":"2024-04-22T20:48:45.694176+0000","last_active":"2024-04-22T20:48:45.694176+0000","last_peered":"2024-04-22T20:48:45.694176+0000","last_clean":"2024-04-22T20:48:45.694176+0000","last_became_active":"2024-04-22T19:51:34.561096+0000","last_became_peered":"2024-04-22T19:51:34.561096+0000","last_unstale":"2024-04-22T20:48:45.694176+0000","last_undegraded":"2024-04-22T20:48:45.694176+0000","last_fullsized":"2024-04-22T20:48:45.694176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.694065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.694065+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.694065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:07:56.257157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"261'166","reported_seq":675,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.608939+0000","last_change":"2024-04-22T20:48:56.608983+0000","last_active":"2024-04-22T20:48:56.608939+0000","last_peered":"2024-04-22T20:48:56.608939+0000","last_clean":"2024-04-22T20:48:56.608939+0000","last_became_active":"2024-04-22T19:51:36.571565+0000","last_became_peered":"2024-04-22T19:51:36.571565+0000","last_unstale":"2024-04-22T20:48:56.608939+0000","last_undegraded":"2024-04-22T20:48:56.608939+0000","last_fullsized":"2024-04-22T20:48:56.608939+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'166","last_scrub_stamp":"2024-04-22T20:48:56.608839+0000","last_deep_scrub":"261'166","last_deep_scrub_stamp":"2024-04-22T20:48:56.608839+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.608839+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:31:59.748806+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":180,"num_write":473,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"261'44","reported_seq":611,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.414925+0000","last_change":"2024-04-22T19:51:28.507235+0000","last_active":"2024-04-22T20:48:23.414925+0000","last_peered":"2024-04-22T20:48:23.414925+0000","last_clean":"2024-04-22T20:48:23.414925+0000","last_became_active":"2024-04-22T19:51:28.506835+0000","last_became_peered":"2024-04-22T19:51:28.506835+0000","last_unstale":"2024-04-22T20:48:23.414925+0000","last_undegraded":"2024-04-22T20:48:23.414925+0000","last_fullsized":"2024-04-22T20:48:23.414925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":202,"num_read_kb":174,"num_write":58,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"261'377","reported_seq":1013,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.442985+0000","last_change":"2024-04-22T20:48:24.443059+0000","last_active":"2024-04-22T20:48:24.442985+0000","last_peered":"2024-04-22T20:48:24.442985+0000","last_clean":"2024-04-22T20:48:24.442985+0000","last_became_active":"2024-04-22T19:52:04.617820+0000","last_became_peered":"2024-04-22T19:52:04.617820+0000","last_unstale":"2024-04-22T20:48:24.442985+0000","last_undegraded":"2024-04-22T20:48:24.442985+0000","last_fullsized":"2024-04-22T20:48:24.442985+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'377","last_scrub_stamp":"2024-04-22T20:48:24.442876+0000","last_deep_scrub":"261'377","last_deep_scrub_stamp":"2024-04-22T20:48:24.442876+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.442876+0000","objects_scrubbed":13,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:42:38.642906+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14463569,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":535,"num_read_kb":31540,"num_write":1473,"num_write_kb":43247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.299779+0000","last_change":"2024-04-22T20:48:54.299779+0000","last_active":"2024-04-22T20:48:54.299779+0000","last_peered":"2024-04-22T20:48:54.299779+0000","last_clean":"2024-04-22T20:48:54.299779+0000","last_became_active":"2024-04-22T19:51:30.529322+0000","last_became_peered":"2024-04-22T19:51:30.529322+0000","last_unstale":"2024-04-22T20:48:54.299779+0000","last_undegraded":"2024-04-22T20:48:54.299779+0000","last_fullsized":"2024-04-22T20:48:54.299779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:54.299664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:54.299664+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.299664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:25.063326+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"261'276","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.466258+0000","last_change":"2024-04-22T20:48:47.466295+0000","last_active":"2024-04-22T20:48:47.466258+0000","last_peered":"2024-04-22T20:48:47.466258+0000","last_clean":"2024-04-22T20:48:47.466258+0000","last_became_active":"2024-04-22T19:51:33.248293+0000","last_became_peered":"2024-04-22T19:51:33.248293+0000","last_unstale":"2024-04-22T20:48:47.466258+0000","last_undegraded":"2024-04-22T20:48:47.466258+0000","last_fullsized":"2024-04-22T20:48:47.466258+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'276","last_scrub_stamp":"2024-04-22T20:48:47.466165+0000","last_deep_scrub":"261'276","last_deep_scrub_stamp":"2024-04-22T20:48:47.466165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.466165+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:12:37.324192+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56069,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1056,"num_write":364,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.684260+0000","last_change":"2024-04-22T20:48:43.684299+0000","last_active":"2024-04-22T20:48:43.684260+0000","last_peered":"2024-04-22T20:48:43.684260+0000","last_clean":"2024-04-22T20:48:43.684260+0000","last_became_active":"2024-04-22T19:51:34.560955+0000","last_became_peered":"2024-04-22T19:51:34.560955+0000","last_unstale":"2024-04-22T20:48:43.684260+0000","last_undegraded":"2024-04-22T20:48:43.684260+0000","last_fullsized":"2024-04-22T20:48:43.684260+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:43.684147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:43.684147+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.684147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:05:24.733318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"261'122","reported_seq":614,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.995578+0000","last_change":"2024-04-22T20:48:34.995676+0000","last_active":"2024-04-22T20:48:34.995578+0000","last_peered":"2024-04-22T20:48:34.995578+0000","last_clean":"2024-04-22T20:48:34.995578+0000","last_became_active":"2024-04-22T19:51:36.567139+0000","last_became_peered":"2024-04-22T19:51:36.567139+0000","last_unstale":"2024-04-22T20:48:34.995578+0000","last_undegraded":"2024-04-22T20:48:34.995578+0000","last_fullsized":"2024-04-22T20:48:34.995578+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'122","last_scrub_stamp":"2024-04-22T20:48:34.995457+0000","last_deep_scrub":"261'122","last_deep_scrub_stamp":"2024-04-22T20:48:34.995457+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.995457+0000","objects_scrubbed":3,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:31:14.860340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":92,"num_write":268,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"261'106","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.592002+0000","last_change":"2024-04-22T20:48:43.592002+0000","last_active":"2024-04-22T20:48:43.592002+0000","last_peered":"2024-04-22T20:48:43.592002+0000","last_clean":"2024-04-22T20:48:43.592002+0000","last_became_active":"2024-04-22T19:51:28.512642+0000","last_became_peered":"2024-04-22T19:51:28.512642+0000","last_unstale":"2024-04-22T20:48:43.592002+0000","last_undegraded":"2024-04-22T20:48:43.592002+0000","last_fullsized":"2024-04-22T20:48:43.592002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'106","last_scrub_stamp":"2024-04-22T20:48:43.591857+0000","last_deep_scrub":"261'106","last_deep_scrub_stamp":"2024-04-22T20:48:43.591857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.591857+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:12:33.205148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":349,"num_write":169,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"261'328","reported_seq":940,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.033489+0000","last_change":"2024-04-22T19:51:26.512313+0000","last_active":"2024-04-22T20:48:22.033489+0000","last_peered":"2024-04-22T20:48:22.033489+0000","last_clean":"2024-04-22T20:48:22.033489+0000","last_became_active":"2024-04-22T19:51:26.511966+0000","last_became_peered":"2024-04-22T19:51:26.511966+0000","last_unstale":"2024-04-22T20:48:22.033489+0000","last_undegraded":"2024-04-22T20:48:22.033489+0000","last_fullsized":"2024-04-22T20:48:22.033489+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262244,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":436,"num_read_kb":38275,"num_write":1102,"num_write_kb":51509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":502,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.538975+0000","last_change":"2024-04-22T20:48:34.538975+0000","last_active":"2024-04-22T20:48:34.538975+0000","last_peered":"2024-04-22T20:48:34.538975+0000","last_clean":"2024-04-22T20:48:34.538975+0000","last_became_active":"2024-04-22T19:51:30.526949+0000","last_became_peered":"2024-04-22T19:51:30.526949+0000","last_unstale":"2024-04-22T20:48:34.538975+0000","last_undegraded":"2024-04-22T20:48:34.538975+0000","last_fullsized":"2024-04-22T20:48:34.538975+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.538867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.538867+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.538867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:45:12.857278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"261'111","reported_seq":646,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.397743+0000","last_change":"2024-04-22T20:48:35.397789+0000","last_active":"2024-04-22T20:48:35.397743+0000","last_peered":"2024-04-22T20:48:35.397743+0000","last_clean":"2024-04-22T20:48:35.397743+0000","last_became_active":"2024-04-22T19:51:33.248113+0000","last_became_peered":"2024-04-22T19:51:33.248113+0000","last_unstale":"2024-04-22T20:48:35.397743+0000","last_undegraded":"2024-04-22T20:48:35.397743+0000","last_fullsized":"2024-04-22T20:48:35.397743+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'111","last_scrub_stamp":"2024-04-22T20:48:35.397661+0000","last_deep_scrub":"261'111","last_deep_scrub_stamp":"2024-04-22T20:48:35.397661+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.397661+0000","objects_scrubbed":8,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:59:22.081071+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":119,"num_read_kb":113,"num_write":116,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11222,"num_omap_keys":30,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"261'518","reported_seq":2306,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.597254+0000","last_change":"2024-04-22T20:48:42.597282+0000","last_active":"2024-04-22T20:48:42.597254+0000","last_peered":"2024-04-22T20:48:42.597254+0000","last_clean":"2024-04-22T20:48:42.597254+0000","last_became_active":"2024-04-22T19:51:34.556789+0000","last_became_peered":"2024-04-22T19:51:34.556789+0000","last_unstale":"2024-04-22T20:48:42.597254+0000","last_undegraded":"2024-04-22T20:48:42.597254+0000","last_fullsized":"2024-04-22T20:48:42.597254+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'518","last_scrub_stamp":"2024-04-22T20:48:42.597160+0000","last_deep_scrub":"261'518","last_deep_scrub_stamp":"2024-04-22T20:48:42.597160+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.597160+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:48:25.653530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"261'98","reported_seq":605,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.461752+0000","last_change":"2024-04-22T20:48:41.461786+0000","last_active":"2024-04-22T20:48:41.461752+0000","last_peered":"2024-04-22T20:48:41.461752+0000","last_clean":"2024-04-22T20:48:41.461752+0000","last_became_active":"2024-04-22T19:51:36.571175+0000","last_became_peered":"2024-04-22T19:51:36.571175+0000","last_unstale":"2024-04-22T20:48:41.461752+0000","last_undegraded":"2024-04-22T20:48:41.461752+0000","last_fullsized":"2024-04-22T20:48:41.461752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'98","last_scrub_stamp":"2024-04-22T20:48:41.461655+0000","last_deep_scrub":"261'98","last_deep_scrub_stamp":"2024-04-22T20:48:41.461655+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.461655+0000","objects_scrubbed":2,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:03.950310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":221,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"261'131","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.334715+0000","last_change":"2024-04-22T20:48:55.334715+0000","last_active":"2024-04-22T20:48:55.334715+0000","last_peered":"2024-04-22T20:48:55.334715+0000","last_clean":"2024-04-22T20:48:55.334715+0000","last_became_active":"2024-04-22T19:51:28.509110+0000","last_became_peered":"2024-04-22T19:51:28.509110+0000","last_unstale":"2024-04-22T20:48:55.334715+0000","last_undegraded":"2024-04-22T20:48:55.334715+0000","last_fullsized":"2024-04-22T20:48:55.334715+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:55.334593+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:55.334593+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.334593+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:37.391655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":519,"num_read_kb":420,"num_write":188,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"261'402","reported_seq":1038,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.850089+0000","last_change":"2024-04-22T20:48:25.850089+0000","last_active":"2024-04-22T20:48:25.850089+0000","last_peered":"2024-04-22T20:48:25.850089+0000","last_clean":"2024-04-22T20:48:25.850089+0000","last_became_active":"2024-04-22T19:52:04.619385+0000","last_became_peered":"2024-04-22T19:52:04.619385+0000","last_unstale":"2024-04-22T20:48:25.850089+0000","last_undegraded":"2024-04-22T20:48:25.850089+0000","last_fullsized":"2024-04-22T20:48:25.850089+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'402","last_scrub_stamp":"2024-04-22T20:48:25.849900+0000","last_deep_scrub":"261'402","last_deep_scrub_stamp":"2024-04-22T20:48:25.849900+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.849900+0000","objects_scrubbed":10,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:07.464040+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682319,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":541,"num_read_kb":21001,"num_write":1452,"num_write_kb":33571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.396156+0000","last_change":"2024-04-22T20:48:52.396156+0000","last_active":"2024-04-22T20:48:52.396156+0000","last_peered":"2024-04-22T20:48:52.396156+0000","last_clean":"2024-04-22T20:48:52.396156+0000","last_became_active":"2024-04-22T19:51:30.526469+0000","last_became_peered":"2024-04-22T19:51:30.526469+0000","last_unstale":"2024-04-22T20:48:52.396156+0000","last_undegraded":"2024-04-22T20:48:52.396156+0000","last_fullsized":"2024-04-22T20:48:52.396156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.396006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.396006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.396006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:26:59.620393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"261'313","reported_seq":906,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.871978+0000","last_change":"2024-04-22T20:48:55.872020+0000","last_active":"2024-04-22T20:48:55.871978+0000","last_peered":"2024-04-22T20:48:55.871978+0000","last_clean":"2024-04-22T20:48:55.871978+0000","last_became_active":"2024-04-22T19:51:32.540573+0000","last_became_peered":"2024-04-22T19:51:32.540573+0000","last_unstale":"2024-04-22T20:48:55.871978+0000","last_undegraded":"2024-04-22T20:48:55.871978+0000","last_fullsized":"2024-04-22T20:48:55.871978+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:55.871874+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:55.871874+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.871874+0000","objects_scrubbed":7,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:02:42.734441+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48433,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":496,"num_read_kb":1091,"num_write":394,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.665319+0000","last_change":"2024-04-22T20:48:47.665362+0000","last_active":"2024-04-22T20:48:47.665319+0000","last_peered":"2024-04-22T20:48:47.665319+0000","last_clean":"2024-04-22T20:48:47.665319+0000","last_became_active":"2024-04-22T19:51:34.553857+0000","last_became_peered":"2024-04-22T19:51:34.553857+0000","last_unstale":"2024-04-22T20:48:47.665319+0000","last_undegraded":"2024-04-22T20:48:47.665319+0000","last_fullsized":"2024-04-22T20:48:47.665319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.665221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.665221+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.665221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:00:40.709377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"261'157","reported_seq":667,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.377887+0000","last_change":"2024-04-22T20:48:39.377921+0000","last_active":"2024-04-22T20:48:39.377887+0000","last_peered":"2024-04-22T20:48:39.377887+0000","last_clean":"2024-04-22T20:48:39.377887+0000","last_became_active":"2024-04-22T19:51:36.575211+0000","last_became_peered":"2024-04-22T19:51:36.575211+0000","last_unstale":"2024-04-22T20:48:39.377887+0000","last_undegraded":"2024-04-22T20:48:39.377887+0000","last_fullsized":"2024-04-22T20:48:39.377887+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'157","last_scrub_stamp":"2024-04-22T20:48:39.377810+0000","last_deep_scrub":"261'157","last_deep_scrub_stamp":"2024-04-22T20:48:39.377810+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.377810+0000","objects_scrubbed":4,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:18:41.159874+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":158,"num_write":375,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"261'85","reported_seq":682,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.875149+0000","last_change":"2024-04-22T19:51:28.508354+0000","last_active":"2024-04-22T20:48:23.875149+0000","last_peered":"2024-04-22T20:48:23.875149+0000","last_clean":"2024-04-22T20:48:23.875149+0000","last_became_active":"2024-04-22T19:51:28.508145+0000","last_became_peered":"2024-04-22T19:51:28.508145+0000","last_unstale":"2024-04-22T20:48:23.875149+0000","last_undegraded":"2024-04-22T20:48:23.875149+0000","last_fullsized":"2024-04-22T20:48:23.875149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:27.487444+0000","last_clean_scrub_stamp":"2024-04-22T19:51:27.487444+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":376,"num_read_kb":320,"num_write":115,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"261'271","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.817431+0000","last_change":"2024-04-22T20:49:00.817466+0000","last_active":"2024-04-22T20:49:00.817431+0000","last_peered":"2024-04-22T20:49:00.817431+0000","last_clean":"2024-04-22T20:49:00.817431+0000","last_became_active":"2024-04-22T19:51:26.504301+0000","last_became_peered":"2024-04-22T19:51:26.504301+0000","last_unstale":"2024-04-22T20:49:00.817431+0000","last_undegraded":"2024-04-22T20:49:00.817431+0000","last_fullsized":"2024-04-22T20:49:00.817431+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'271","last_scrub_stamp":"2024-04-22T20:49:00.817344+0000","last_deep_scrub":"261'271","last_deep_scrub_stamp":"2024-04-22T20:49:00.817344+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.817344+0000","objects_scrubbed":4,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:03:08.734664+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":4555,"num_write":1089,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.927324+0000","last_change":"2024-04-22T20:48:51.927324+0000","last_active":"2024-04-22T20:48:51.927324+0000","last_peered":"2024-04-22T20:48:51.927324+0000","last_clean":"2024-04-22T20:48:51.927324+0000","last_became_active":"2024-04-22T19:51:30.524417+0000","last_became_peered":"2024-04-22T19:51:30.524417+0000","last_unstale":"2024-04-22T20:48:51.927324+0000","last_undegraded":"2024-04-22T20:48:51.927324+0000","last_fullsized":"2024-04-22T20:48:51.927324+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:51.927209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:51.927209+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.927209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:31:17.320634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"261'229","reported_seq":799,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.731038+0000","last_change":"2024-04-22T19:51:32.540923+0000","last_active":"2024-04-22T20:48:17.731038+0000","last_peered":"2024-04-22T20:48:17.731038+0000","last_clean":"2024-04-22T20:48:17.731038+0000","last_became_active":"2024-04-22T19:51:32.540659+0000","last_became_peered":"2024-04-22T19:51:32.540659+0000","last_unstale":"2024-04-22T20:48:17.731038+0000","last_undegraded":"2024-04-22T20:48:17.731038+0000","last_fullsized":"2024-04-22T20:48:17.731038+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":200,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16866,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":857,"num_write":302,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.574005+0000","last_change":"2024-04-22T20:48:36.574034+0000","last_active":"2024-04-22T20:48:36.574005+0000","last_peered":"2024-04-22T20:48:36.574005+0000","last_clean":"2024-04-22T20:48:36.574005+0000","last_became_active":"2024-04-22T19:51:34.552344+0000","last_became_peered":"2024-04-22T19:51:34.552344+0000","last_unstale":"2024-04-22T20:48:36.574005+0000","last_undegraded":"2024-04-22T20:48:36.574005+0000","last_fullsized":"2024-04-22T20:48:36.574005+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:36.573910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:36.573910+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.573910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:39:04.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"261'138","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.476371+0000","last_change":"2024-04-22T20:48:37.476415+0000","last_active":"2024-04-22T20:48:37.476371+0000","last_peered":"2024-04-22T20:48:37.476371+0000","last_clean":"2024-04-22T20:48:37.476371+0000","last_became_active":"2024-04-22T19:51:36.567594+0000","last_became_peered":"2024-04-22T19:51:36.567594+0000","last_unstale":"2024-04-22T20:48:37.476371+0000","last_undegraded":"2024-04-22T20:48:37.476371+0000","last_fullsized":"2024-04-22T20:48:37.476371+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'138","last_scrub_stamp":"2024-04-22T20:48:37.476271+0000","last_deep_scrub":"261'138","last_deep_scrub_stamp":"2024-04-22T20:48:37.476271+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.476271+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:27:27.543918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":214,"num_write":335,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"261'87","reported_seq":657,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.451797+0000","last_change":"2024-04-22T20:48:56.451837+0000","last_active":"2024-04-22T20:48:56.451797+0000","last_peered":"2024-04-22T20:48:56.451797+0000","last_clean":"2024-04-22T20:48:56.451797+0000","last_became_active":"2024-04-22T19:51:28.519542+0000","last_became_peered":"2024-04-22T19:51:28.519542+0000","last_unstale":"2024-04-22T20:48:56.451797+0000","last_undegraded":"2024-04-22T20:48:56.451797+0000","last_fullsized":"2024-04-22T20:48:56.451797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'87","last_scrub_stamp":"2024-04-22T20:48:56.451708+0000","last_deep_scrub":"261'87","last_deep_scrub_stamp":"2024-04-22T20:48:56.451708+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.451708+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:39.301084+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":347,"num_read_kb":277,"num_write":137,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"261'398","reported_seq":1243,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.890036+0000","last_change":"2024-04-22T19:51:26.500077+0000","last_active":"2024-04-22T20:48:17.890036+0000","last_peered":"2024-04-22T20:48:17.890036+0000","last_clean":"2024-04-22T20:48:17.890036+0000","last_became_active":"2024-04-22T19:51:26.499390+0000","last_became_peered":"2024-04-22T19:51:26.499390+0000","last_unstale":"2024-04-22T20:48:17.890036+0000","last_undegraded":"2024-04-22T20:48:17.890036+0000","last_fullsized":"2024-04-22T20:48:17.890036+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4204544,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1043,"num_read_kb":88369,"num_write":1413,"num_write_kb":41583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.330279+0000","last_change":"2024-04-22T20:48:52.330304+0000","last_active":"2024-04-22T20:48:52.330279+0000","last_peered":"2024-04-22T20:48:52.330279+0000","last_clean":"2024-04-22T20:48:52.330279+0000","last_became_active":"2024-04-22T19:51:30.526183+0000","last_became_peered":"2024-04-22T19:51:30.526183+0000","last_unstale":"2024-04-22T20:48:52.330279+0000","last_undegraded":"2024-04-22T20:48:52.330279+0000","last_fullsized":"2024-04-22T20:48:52.330279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.330187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.330187+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.330187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:30.153067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"261'248","reported_seq":814,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.415263+0000","last_change":"2024-04-22T19:51:32.538965+0000","last_active":"2024-04-22T20:48:23.415263+0000","last_peered":"2024-04-22T20:48:23.415263+0000","last_clean":"2024-04-22T20:48:23.415263+0000","last_became_active":"2024-04-22T19:51:32.538644+0000","last_became_peered":"2024-04-22T19:51:32.538644+0000","last_unstale":"2024-04-22T20:48:23.415263+0000","last_undegraded":"2024-04-22T20:48:23.415263+0000","last_fullsized":"2024-04-22T20:48:23.415263+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:31.517761+0000","last_clean_scrub_stamp":"2024-04-22T19:51:31.517761+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":200,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45694,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":402,"num_read_kb":955,"num_write":332,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.405108+0000","last_change":"2024-04-22T20:48:31.405108+0000","last_active":"2024-04-22T20:48:31.405108+0000","last_peered":"2024-04-22T20:48:31.405108+0000","last_clean":"2024-04-22T20:48:31.405108+0000","last_became_active":"2024-04-22T19:51:34.551085+0000","last_became_peered":"2024-04-22T19:51:34.551085+0000","last_unstale":"2024-04-22T20:48:31.405108+0000","last_undegraded":"2024-04-22T20:48:31.405108+0000","last_fullsized":"2024-04-22T20:48:31.405108+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:31.404990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:31.404990+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.404990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:00.257782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"261'187","reported_seq":920,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.442559+0000","last_change":"2024-04-22T20:48:42.442612+0000","last_active":"2024-04-22T20:48:42.442559+0000","last_peered":"2024-04-22T20:48:42.442559+0000","last_clean":"2024-04-22T20:48:42.442559+0000","last_became_active":"2024-04-22T19:51:36.569234+0000","last_became_peered":"2024-04-22T19:51:36.569234+0000","last_unstale":"2024-04-22T20:48:42.442559+0000","last_undegraded":"2024-04-22T20:48:42.442559+0000","last_fullsized":"2024-04-22T20:48:42.442559+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'187","last_scrub_stamp":"2024-04-22T20:48:42.442437+0000","last_deep_scrub":"261'187","last_deep_scrub_stamp":"2024-04-22T20:48:42.442437+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.442437+0000","objects_scrubbed":8,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:21:18.797025+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":419,"num_read_kb":454,"num_write":429,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"261'180","reported_seq":759,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.345583+0000","last_change":"2024-04-22T20:48:44.345608+0000","last_active":"2024-04-22T20:48:44.345583+0000","last_peered":"2024-04-22T20:48:44.345583+0000","last_clean":"2024-04-22T20:48:44.345583+0000","last_became_active":"2024-04-22T19:51:28.515506+0000","last_became_peered":"2024-04-22T19:51:28.515506+0000","last_unstale":"2024-04-22T20:48:44.345583+0000","last_undegraded":"2024-04-22T20:48:44.345583+0000","last_fullsized":"2024-04-22T20:48:44.345583+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'180","last_scrub_stamp":"2024-04-22T20:48:44.345489+0000","last_deep_scrub":"261'180","last_deep_scrub_stamp":"2024-04-22T20:48:44.345489+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.345489+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:48.784252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":678,"num_read_kb":529,"num_write":279,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"261'297","reported_seq":910,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.948862+0000","last_change":"2024-04-22T19:51:26.504948+0000","last_active":"2024-04-22T20:48:17.948862+0000","last_peered":"2024-04-22T20:48:17.948862+0000","last_clean":"2024-04-22T20:48:17.948862+0000","last_became_active":"2024-04-22T19:51:26.504604+0000","last_became_peered":"2024-04-22T19:51:26.504604+0000","last_unstale":"2024-04-22T20:48:17.948862+0000","last_undegraded":"2024-04-22T20:48:17.948862+0000","last_fullsized":"2024-04-22T20:48:17.948862+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242937,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":372,"num_read_kb":9007,"num_write":952,"num_write_kb":46630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"261'2","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.636032+0000","last_change":"2024-04-22T20:48:38.636081+0000","last_active":"2024-04-22T20:48:38.636032+0000","last_peered":"2024-04-22T20:48:38.636032+0000","last_clean":"2024-04-22T20:48:38.636032+0000","last_became_active":"2024-04-22T19:51:30.523338+0000","last_became_peered":"2024-04-22T19:51:30.523338+0000","last_unstale":"2024-04-22T20:48:38.636032+0000","last_undegraded":"2024-04-22T20:48:38.636032+0000","last_fullsized":"2024-04-22T20:48:38.636032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:38.635935+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:38.635935+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.635935+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:28:44.233266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"261'313","reported_seq":908,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.337600+0000","last_change":"2024-04-22T20:48:37.337600+0000","last_active":"2024-04-22T20:48:37.337600+0000","last_peered":"2024-04-22T20:48:37.337600+0000","last_clean":"2024-04-22T20:48:37.337600+0000","last_became_active":"2024-04-22T19:51:32.537373+0000","last_became_peered":"2024-04-22T19:51:32.537373+0000","last_unstale":"2024-04-22T20:48:37.337600+0000","last_undegraded":"2024-04-22T20:48:37.337600+0000","last_fullsized":"2024-04-22T20:48:37.337600+0000","mapping_epoch":26,"log_start":"261'301","ondisk_log_start":"261'301","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:37.337420+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:37.337420+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.337420+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":301,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:00.481534+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49488,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":1080,"num_write":394,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.571825+0000","last_change":"2024-04-22T20:48:43.571825+0000","last_active":"2024-04-22T20:48:43.571825+0000","last_peered":"2024-04-22T20:48:43.571825+0000","last_clean":"2024-04-22T20:48:43.571825+0000","last_became_active":"2024-04-22T19:51:34.555709+0000","last_became_peered":"2024-04-22T19:51:34.555709+0000","last_unstale":"2024-04-22T20:48:43.571825+0000","last_undegraded":"2024-04-22T20:48:43.571825+0000","last_fullsized":"2024-04-22T20:48:43.571825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:43.571712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:43.571712+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.571712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:04:20.739611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"261'146","reported_seq":639,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.318417+0000","last_change":"2024-04-22T20:48:26.318450+0000","last_active":"2024-04-22T20:48:26.318417+0000","last_peered":"2024-04-22T20:48:26.318417+0000","last_clean":"2024-04-22T20:48:26.318417+0000","last_became_active":"2024-04-22T19:51:36.574012+0000","last_became_peered":"2024-04-22T19:51:36.574012+0000","last_unstale":"2024-04-22T20:48:26.318417+0000","last_undegraded":"2024-04-22T20:48:26.318417+0000","last_fullsized":"2024-04-22T20:48:26.318417+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'146","last_scrub_stamp":"2024-04-22T20:48:26.318331+0000","last_deep_scrub":"261'146","last_deep_scrub_stamp":"2024-04-22T20:48:26.318331+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.318331+0000","objects_scrubbed":2,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:20.701708+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":109,"num_write":348,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"261'372","reported_seq":1046,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.694389+0000","last_change":"2024-04-22T20:48:40.694419+0000","last_active":"2024-04-22T20:48:40.694389+0000","last_peered":"2024-04-22T20:48:40.694389+0000","last_clean":"2024-04-22T20:48:40.694389+0000","last_became_active":"2024-04-22T19:51:28.518339+0000","last_became_peered":"2024-04-22T19:51:28.518339+0000","last_unstale":"2024-04-22T20:48:40.694389+0000","last_undegraded":"2024-04-22T20:48:40.694389+0000","last_fullsized":"2024-04-22T20:48:40.694389+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'372","last_scrub_stamp":"2024-04-22T20:48:40.694294+0000","last_deep_scrub":"261'372","last_deep_scrub_stamp":"2024-04-22T20:48:40.694294+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.694294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:07.075387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1526,"num_read_kb":1211,"num_write":619,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"261'324","reported_seq":1032,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.592130+0000","last_change":"2024-04-22T20:48:55.592179+0000","last_active":"2024-04-22T20:48:55.592130+0000","last_peered":"2024-04-22T20:48:55.592130+0000","last_clean":"2024-04-22T20:48:55.592130+0000","last_became_active":"2024-04-22T19:51:26.502773+0000","last_became_peered":"2024-04-22T19:51:26.502773+0000","last_unstale":"2024-04-22T20:48:55.592130+0000","last_undegraded":"2024-04-22T20:48:55.592130+0000","last_fullsized":"2024-04-22T20:48:55.592130+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:48:55.592027+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:48:55.592027+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.592027+0000","objects_scrubbed":7,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:38.053119+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4371014,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":594,"num_read_kb":14386,"num_write":1094,"num_write_kb":17858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"261'524","reported_seq":1296,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.877481+0000","last_change":"2024-04-22T20:48:26.877525+0000","last_active":"2024-04-22T20:48:26.877481+0000","last_peered":"2024-04-22T20:48:26.877481+0000","last_clean":"2024-04-22T20:48:26.877481+0000","last_became_active":"2024-04-22T19:52:04.617036+0000","last_became_peered":"2024-04-22T19:52:04.617036+0000","last_unstale":"2024-04-22T20:48:26.877481+0000","last_undegraded":"2024-04-22T20:48:26.877481+0000","last_fullsized":"2024-04-22T20:48:26.877481+0000","mapping_epoch":33,"log_start":"261'500","ondisk_log_start":"261'500","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'524","last_scrub_stamp":"2024-04-22T20:48:26.877358+0000","last_deep_scrub":"261'524","last_deep_scrub_stamp":"2024-04-22T20:48:26.877358+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.877358+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":500,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:34:00.586382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2230,"num_read_kb":1847,"num_write":935,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"261'286","reported_seq":934,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.553956+0000","last_change":"2024-04-22T20:48:38.553990+0000","last_active":"2024-04-22T20:48:38.553956+0000","last_peered":"2024-04-22T20:48:38.553956+0000","last_clean":"2024-04-22T20:48:38.553956+0000","last_became_active":"2024-04-22T19:51:26.495948+0000","last_became_peered":"2024-04-22T19:51:26.495948+0000","last_unstale":"2024-04-22T20:48:38.553956+0000","last_undegraded":"2024-04-22T20:48:38.553956+0000","last_fullsized":"2024-04-22T20:48:38.553956+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'286","last_scrub_stamp":"2024-04-22T20:48:38.553878+0000","last_deep_scrub":"261'286","last_deep_scrub_stamp":"2024-04-22T20:48:38.553878+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.553878+0000","objects_scrubbed":6,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:12.594083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180512,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":454,"num_read_kb":10117,"num_write":1004,"num_write_kb":22835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"261'209","reported_seq":851,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.973501+0000","last_change":"2024-04-22T20:48:44.973554+0000","last_active":"2024-04-22T20:48:44.973501+0000","last_peered":"2024-04-22T20:48:44.973501+0000","last_clean":"2024-04-22T20:48:44.973501+0000","last_became_active":"2024-04-22T19:51:28.513026+0000","last_became_peered":"2024-04-22T19:51:28.513026+0000","last_unstale":"2024-04-22T20:48:44.973501+0000","last_undegraded":"2024-04-22T20:48:44.973501+0000","last_fullsized":"2024-04-22T20:48:44.973501+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'209","last_scrub_stamp":"2024-04-22T20:48:44.973375+0000","last_deep_scrub":"261'209","last_deep_scrub_stamp":"2024-04-22T20:48:44.973375+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.973375+0000","objects_scrubbed":1,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:59.805690+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":827,"num_read_kb":665,"num_write":342,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"261'313","reported_seq":941,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.835329+0000","last_change":"2024-04-22T20:49:04.835404+0000","last_active":"2024-04-22T20:49:04.835329+0000","last_peered":"2024-04-22T20:49:04.835329+0000","last_clean":"2024-04-22T20:49:04.835329+0000","last_became_active":"2024-04-22T19:51:26.506748+0000","last_became_peered":"2024-04-22T19:51:26.506748+0000","last_unstale":"2024-04-22T20:49:04.835329+0000","last_undegraded":"2024-04-22T20:49:04.835329+0000","last_fullsized":"2024-04-22T20:49:04.835329+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:49:04.835221+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:49:04.835221+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.835221+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:46.453041+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":416,"num_read_kb":12678,"num_write":1099,"num_write_kb":41347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"261'4175","reported_seq":4800,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.586185+0000","last_change":"2024-04-22T20:48:56.586215+0000","last_active":"2024-04-22T20:48:56.586185+0000","last_peered":"2024-04-22T20:48:56.586185+0000","last_clean":"2024-04-22T20:48:56.586185+0000","last_became_active":"2024-04-22T19:51:28.511312+0000","last_became_peered":"2024-04-22T19:51:28.511312+0000","last_unstale":"2024-04-22T20:48:56.586185+0000","last_undegraded":"2024-04-22T20:48:56.586185+0000","last_fullsized":"2024-04-22T20:48:56.586185+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4175","last_scrub_stamp":"2024-04-22T20:48:56.586085+0000","last_deep_scrub":"261'4175","last_deep_scrub_stamp":"2024-04-22T20:48:56.586085+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.586085+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":3000,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:15.045023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14718,"num_read_kb":11919,"num_write":6278,"num_write_kb":3133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"261'312","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.452280+0000","last_change":"2024-04-22T20:48:28.452280+0000","last_active":"2024-04-22T20:48:28.452280+0000","last_peered":"2024-04-22T20:48:28.452280+0000","last_clean":"2024-04-22T20:48:28.452280+0000","last_became_active":"2024-04-22T19:52:05.114734+0000","last_became_peered":"2024-04-22T19:52:05.114734+0000","last_unstale":"2024-04-22T20:48:28.452280+0000","last_undegraded":"2024-04-22T20:48:28.452280+0000","last_fullsized":"2024-04-22T20:48:28.452280+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'312","last_scrub_stamp":"2024-04-22T20:48:28.452109+0000","last_deep_scrub":"261'312","last_deep_scrub_stamp":"2024-04-22T20:48:28.452109+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.452109+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:41:21.599729+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":5518,"num_write":1088,"num_write_kb":21667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.23","version":"261'171","reported_seq":861,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.466280+0000","last_change":"2024-04-22T20:48:57.466312+0000","last_active":"2024-04-22T20:48:57.466280+0000","last_peered":"2024-04-22T20:48:57.466280+0000","last_clean":"2024-04-22T20:48:57.466280+0000","last_became_active":"2024-04-22T19:51:28.516125+0000","last_became_peered":"2024-04-22T19:51:28.516125+0000","last_unstale":"2024-04-22T20:48:57.466280+0000","last_undegraded":"2024-04-22T20:48:57.466280+0000","last_fullsized":"2024-04-22T20:48:57.466280+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'171","last_scrub_stamp":"2024-04-22T20:48:57.466192+0000","last_deep_scrub":"261'171","last_deep_scrub_stamp":"2024-04-22T20:48:57.466192+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.466192+0000","objects_scrubbed":1,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:22:48.965827+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":741,"num_read_kb":616,"num_write":243,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"261'368","reported_seq":1024,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.429047+0000","last_change":"2024-04-22T20:48:26.429162+0000","last_active":"2024-04-22T20:48:26.429047+0000","last_peered":"2024-04-22T20:48:26.429047+0000","last_clean":"2024-04-22T20:48:26.429047+0000","last_became_active":"2024-04-22T19:52:04.619286+0000","last_became_peered":"2024-04-22T19:52:04.619286+0000","last_unstale":"2024-04-22T20:48:26.429047+0000","last_undegraded":"2024-04-22T20:48:26.429047+0000","last_fullsized":"2024-04-22T20:48:26.429047+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'368","last_scrub_stamp":"2024-04-22T20:48:26.428925+0000","last_deep_scrub":"261'368","last_deep_scrub_stamp":"2024-04-22T20:48:26.428925+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.428925+0000","objects_scrubbed":9,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:38:16.414530+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4719706,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":588,"num_read_kb":69897,"num_write":1236,"num_write_kb":36151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"261'122","reported_seq":735,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.272617+0000","last_change":"2024-04-22T20:48:46.272648+0000","last_active":"2024-04-22T20:48:46.272617+0000","last_peered":"2024-04-22T20:48:46.272617+0000","last_clean":"2024-04-22T20:48:46.272617+0000","last_became_active":"2024-04-22T19:51:28.516437+0000","last_became_peered":"2024-04-22T19:51:28.516437+0000","last_unstale":"2024-04-22T20:48:46.272617+0000","last_undegraded":"2024-04-22T20:48:46.272617+0000","last_fullsized":"2024-04-22T20:48:46.272617+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'122","last_scrub_stamp":"2024-04-22T20:48:46.272512+0000","last_deep_scrub":"261'122","last_deep_scrub_stamp":"2024-04-22T20:48:46.272512+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.272512+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:54:33.342867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":423,"num_write":178,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"261'317","reported_seq":1060,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.585793+0000","last_change":"2024-04-22T20:48:53.585833+0000","last_active":"2024-04-22T20:48:53.585793+0000","last_peered":"2024-04-22T20:48:53.585793+0000","last_clean":"2024-04-22T20:48:53.585793+0000","last_became_active":"2024-04-22T19:51:26.498039+0000","last_became_peered":"2024-04-22T19:51:26.498039+0000","last_unstale":"2024-04-22T20:48:53.585793+0000","last_undegraded":"2024-04-22T20:48:53.585793+0000","last_fullsized":"2024-04-22T20:48:53.585793+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:53.585698+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:53.585698+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.585698+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:05:32.063217+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194319,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":766,"num_read_kb":22382,"num_write":1253,"num_write_kb":18139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"261'199","reported_seq":857,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.407111+0000","last_change":"2024-04-22T20:49:00.407135+0000","last_active":"2024-04-22T20:49:00.407111+0000","last_peered":"2024-04-22T20:49:00.407111+0000","last_clean":"2024-04-22T20:49:00.407111+0000","last_became_active":"2024-04-22T19:51:28.520612+0000","last_became_peered":"2024-04-22T19:51:28.520612+0000","last_unstale":"2024-04-22T20:49:00.407111+0000","last_undegraded":"2024-04-22T20:49:00.407111+0000","last_fullsized":"2024-04-22T20:49:00.407111+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'199","last_scrub_stamp":"2024-04-22T20:49:00.407015+0000","last_deep_scrub":"261'199","last_deep_scrub_stamp":"2024-04-22T20:49:00.407015+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.407015+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:11:38.305157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":735,"num_read_kb":593,"num_write":262,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"261'365","reported_seq":1022,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.558974+0000","last_change":"2024-04-22T20:48:52.559103+0000","last_active":"2024-04-22T20:48:52.558974+0000","last_peered":"2024-04-22T20:48:52.558974+0000","last_clean":"2024-04-22T20:48:52.558974+0000","last_became_active":"2024-04-22T19:51:26.498845+0000","last_became_peered":"2024-04-22T19:51:26.498845+0000","last_unstale":"2024-04-22T20:48:52.558974+0000","last_undegraded":"2024-04-22T20:48:52.558974+0000","last_fullsized":"2024-04-22T20:48:52.558974+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'365","last_scrub_stamp":"2024-04-22T20:48:52.558851+0000","last_deep_scrub":"261'365","last_deep_scrub_stamp":"2024-04-22T20:48:52.558851+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.558851+0000","objects_scrubbed":9,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:38:15.742114+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050721,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":506,"num_read_kb":33263,"num_write":1241,"num_write_kb":30860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"261'133","reported_seq":775,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.610209+0000","last_change":"2024-04-22T20:48:37.610234+0000","last_active":"2024-04-22T20:48:37.610209+0000","last_peered":"2024-04-22T20:48:37.610209+0000","last_clean":"2024-04-22T20:48:37.610209+0000","last_became_active":"2024-04-22T19:51:28.519669+0000","last_became_peered":"2024-04-22T19:51:28.519669+0000","last_unstale":"2024-04-22T20:48:37.610209+0000","last_undegraded":"2024-04-22T20:48:37.610209+0000","last_fullsized":"2024-04-22T20:48:37.610209+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'133","last_scrub_stamp":"2024-04-22T20:48:37.610118+0000","last_deep_scrub":"261'133","last_deep_scrub_stamp":"2024-04-22T20:48:37.610118+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.610118+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:03:13.662918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":560,"num_read_kb":466,"num_write":196,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"261'330","reported_seq":935,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.490447+0000","last_change":"2024-04-22T20:48:29.490501+0000","last_active":"2024-04-22T20:48:29.490447+0000","last_peered":"2024-04-22T20:48:29.490447+0000","last_clean":"2024-04-22T20:48:29.490447+0000","last_became_active":"2024-04-22T19:52:04.617649+0000","last_became_peered":"2024-04-22T19:52:04.617649+0000","last_unstale":"2024-04-22T20:48:29.490447+0000","last_undegraded":"2024-04-22T20:48:29.490447+0000","last_fullsized":"2024-04-22T20:48:29.490447+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'330","last_scrub_stamp":"2024-04-22T20:48:29.490328+0000","last_deep_scrub":"261'330","last_deep_scrub_stamp":"2024-04-22T20:48:29.490328+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.490328+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:55:48.835393+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050919,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":29074,"num_write":1030,"num_write_kb":22627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"261'128","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.677849+0000","last_change":"2024-04-22T20:48:51.677899+0000","last_active":"2024-04-22T20:48:51.677849+0000","last_peered":"2024-04-22T20:48:51.677849+0000","last_clean":"2024-04-22T20:48:51.677849+0000","last_became_active":"2024-04-22T19:51:28.521516+0000","last_became_peered":"2024-04-22T19:51:28.521516+0000","last_unstale":"2024-04-22T20:48:51.677849+0000","last_undegraded":"2024-04-22T20:48:51.677849+0000","last_fullsized":"2024-04-22T20:48:51.677849+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'128","last_scrub_stamp":"2024-04-22T20:48:51.677734+0000","last_deep_scrub":"261'128","last_deep_scrub_stamp":"2024-04-22T20:48:51.677734+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.677734+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:24.647580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":542,"num_read_kb":442,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"261'338","reported_seq":1027,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.972384+0000","last_change":"2024-04-22T20:48:52.972424+0000","last_active":"2024-04-22T20:48:52.972384+0000","last_peered":"2024-04-22T20:48:52.972384+0000","last_clean":"2024-04-22T20:48:52.972384+0000","last_became_active":"2024-04-22T19:51:26.507843+0000","last_became_peered":"2024-04-22T19:51:26.507843+0000","last_unstale":"2024-04-22T20:48:52.972384+0000","last_undegraded":"2024-04-22T20:48:52.972384+0000","last_fullsized":"2024-04-22T20:48:52.972384+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'338","last_scrub_stamp":"2024-04-22T20:48:52.972288+0000","last_deep_scrub":"261'338","last_deep_scrub_stamp":"2024-04-22T20:48:52.972288+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.972288+0000","objects_scrubbed":6,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:36.857751+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5531396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":136369,"num_write":1233,"num_write_kb":55788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"261'235","reported_seq":869,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.241423+0000","last_change":"2024-04-22T20:48:47.241450+0000","last_active":"2024-04-22T20:48:47.241423+0000","last_peered":"2024-04-22T20:48:47.241423+0000","last_clean":"2024-04-22T20:48:47.241423+0000","last_became_active":"2024-04-22T19:51:28.515164+0000","last_became_peered":"2024-04-22T19:51:28.515164+0000","last_unstale":"2024-04-22T20:48:47.241423+0000","last_undegraded":"2024-04-22T20:48:47.241423+0000","last_fullsized":"2024-04-22T20:48:47.241423+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'235","last_scrub_stamp":"2024-04-22T20:48:47.241303+0000","last_deep_scrub":"261'235","last_deep_scrub_stamp":"2024-04-22T20:48:47.241303+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.241303+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:24.721788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":950,"num_read_kb":750,"num_write":364,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"261'502","reported_seq":1203,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.940696+0000","last_change":"2024-04-22T19:51:26.494217+0000","last_active":"2024-04-22T20:48:17.940696+0000","last_peered":"2024-04-22T20:48:17.940696+0000","last_clean":"2024-04-22T20:48:17.940696+0000","last_became_active":"2024-04-22T19:51:26.493860+0000","last_became_peered":"2024-04-22T19:51:26.493860+0000","last_unstale":"2024-04-22T20:48:17.940696+0000","last_undegraded":"2024-04-22T20:48:17.940696+0000","last_fullsized":"2024-04-22T20:48:17.940696+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":400,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6292618,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":951,"num_read_kb":13872,"num_write":1439,"num_write_kb":37693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"261'252","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.584503+0000","last_change":"2024-04-22T20:48:44.584554+0000","last_active":"2024-04-22T20:48:44.584503+0000","last_peered":"2024-04-22T20:48:44.584503+0000","last_clean":"2024-04-22T20:48:44.584503+0000","last_became_active":"2024-04-22T19:51:28.515547+0000","last_became_peered":"2024-04-22T19:51:28.515547+0000","last_unstale":"2024-04-22T20:48:44.584503+0000","last_undegraded":"2024-04-22T20:48:44.584503+0000","last_fullsized":"2024-04-22T20:48:44.584503+0000","mapping_epoch":22,"log_start":"247'200","ondisk_log_start":"247'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'252","last_scrub_stamp":"2024-04-22T20:48:44.584399+0000","last_deep_scrub":"261'252","last_deep_scrub_stamp":"2024-04-22T20:48:44.584399+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.584399+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:23:23.642516+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1120,"num_read_kb":973,"num_write":365,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25681,"num_omap_keys":101,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"261'324","reported_seq":963,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.831693+0000","last_change":"2024-04-22T20:48:58.831759+0000","last_active":"2024-04-22T20:48:58.831693+0000","last_peered":"2024-04-22T20:48:58.831693+0000","last_clean":"2024-04-22T20:48:58.831693+0000","last_became_active":"2024-04-22T19:51:26.500566+0000","last_became_peered":"2024-04-22T19:51:26.500566+0000","last_unstale":"2024-04-22T20:48:58.831693+0000","last_undegraded":"2024-04-22T20:48:58.831693+0000","last_fullsized":"2024-04-22T20:48:58.831693+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:48:58.831593+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:48:58.831593+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.831593+0000","objects_scrubbed":7,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:41:02.913836+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283749,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":424,"num_read_kb":45844,"num_write":1120,"num_write_kb":42819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"261'152","reported_seq":759,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.297232+0000","last_change":"2024-04-22T20:48:43.297263+0000","last_active":"2024-04-22T20:48:43.297232+0000","last_peered":"2024-04-22T20:48:43.297232+0000","last_clean":"2024-04-22T20:48:43.297232+0000","last_became_active":"2024-04-22T19:51:28.515764+0000","last_became_peered":"2024-04-22T19:51:28.515764+0000","last_unstale":"2024-04-22T20:48:43.297232+0000","last_undegraded":"2024-04-22T20:48:43.297232+0000","last_fullsized":"2024-04-22T20:48:43.297232+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:43.297146+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:43.297146+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.297146+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:48.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":633,"num_read_kb":515,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"261'261","reported_seq":871,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.822172+0000","last_change":"2024-04-22T19:51:26.512810+0000","last_active":"2024-04-22T20:48:17.822172+0000","last_peered":"2024-04-22T20:48:17.822172+0000","last_clean":"2024-04-22T20:48:17.822172+0000","last_became_active":"2024-04-22T19:51:26.512467+0000","last_became_peered":"2024-04-22T19:51:26.512467+0000","last_unstale":"2024-04-22T20:48:17.822172+0000","last_undegraded":"2024-04-22T20:48:17.822172+0000","last_fullsized":"2024-04-22T20:48:17.822172+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11536625,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":377,"num_read_kb":17908,"num_write":933,"num_write_kb":43703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"261'27","reported_seq":563,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.366682+0000","last_change":"2024-04-22T20:48:55.366707+0000","last_active":"2024-04-22T20:48:55.366682+0000","last_peered":"2024-04-22T20:48:55.366682+0000","last_clean":"2024-04-22T20:48:55.366682+0000","last_became_active":"2024-04-22T19:51:28.510091+0000","last_became_peered":"2024-04-22T19:51:28.510091+0000","last_unstale":"2024-04-22T20:48:55.366682+0000","last_undegraded":"2024-04-22T20:48:55.366682+0000","last_fullsized":"2024-04-22T20:48:55.366682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'27","last_scrub_stamp":"2024-04-22T20:48:55.366594+0000","last_deep_scrub":"261'27","last_deep_scrub_stamp":"2024-04-22T20:48:55.366594+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.366594+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:48:05.352437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":95,"num_write":39,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"261'340","reported_seq":996,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.582045+0000","last_change":"2024-04-22T20:48:53.582088+0000","last_active":"2024-04-22T20:48:53.582045+0000","last_peered":"2024-04-22T20:48:53.582045+0000","last_clean":"2024-04-22T20:48:53.582045+0000","last_became_active":"2024-04-22T19:51:26.501700+0000","last_became_peered":"2024-04-22T19:51:26.501700+0000","last_unstale":"2024-04-22T20:48:53.582045+0000","last_undegraded":"2024-04-22T20:48:53.582045+0000","last_fullsized":"2024-04-22T20:48:53.582045+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'340","last_scrub_stamp":"2024-04-22T20:48:53.581939+0000","last_deep_scrub":"261'340","last_deep_scrub_stamp":"2024-04-22T20:48:53.581939+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.581939+0000","objects_scrubbed":7,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:58.999952+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231939,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":506,"num_read_kb":6050,"num_write":1241,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"261'54","reported_seq":612,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.710469+0000","last_change":"2024-04-22T20:49:04.710497+0000","last_active":"2024-04-22T20:49:04.710469+0000","last_peered":"2024-04-22T20:49:04.710469+0000","last_clean":"2024-04-22T20:49:04.710469+0000","last_became_active":"2024-04-22T19:51:28.508768+0000","last_became_peered":"2024-04-22T19:51:28.508768+0000","last_unstale":"2024-04-22T20:49:04.710469+0000","last_undegraded":"2024-04-22T20:49:04.710469+0000","last_fullsized":"2024-04-22T20:49:04.710469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'54","last_scrub_stamp":"2024-04-22T20:49:04.710365+0000","last_deep_scrub":"261'54","last_deep_scrub_stamp":"2024-04-22T20:49:04.710365+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.710365+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:28:08.981902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":238,"num_read_kb":196,"num_write":80,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"261'355","reported_seq":995,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:17.932058+0000","last_change":"2024-04-22T19:51:26.497503+0000","last_active":"2024-04-22T20:48:17.932058+0000","last_peered":"2024-04-22T20:48:17.932058+0000","last_clean":"2024-04-22T20:48:17.932058+0000","last_became_active":"2024-04-22T19:51:26.497156+0000","last_became_peered":"2024-04-22T19:51:26.497156+0000","last_unstale":"2024-04-22T20:48:17.932058+0000","last_undegraded":"2024-04-22T20:48:17.932058+0000","last_fullsized":"2024-04-22T20:48:17.932058+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T19:51:25.466756+0000","last_clean_scrub_stamp":"2024-04-22T19:51:25.466756+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437245,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":35895,"num_write":1244,"num_write_kb":35286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"261'4050","reported_seq":4621,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.446481+0000","last_change":"2024-04-22T20:48:59.446504+0000","last_active":"2024-04-22T20:48:59.446481+0000","last_peered":"2024-04-22T20:48:59.446481+0000","last_clean":"2024-04-22T20:48:59.446481+0000","last_became_active":"2024-04-22T19:51:28.514780+0000","last_became_peered":"2024-04-22T19:51:28.514780+0000","last_unstale":"2024-04-22T20:48:59.446481+0000","last_undegraded":"2024-04-22T20:48:59.446481+0000","last_fullsized":"2024-04-22T20:48:59.446481+0000","mapping_epoch":22,"log_start":"261'4000","ondisk_log_start":"261'4000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4050","last_scrub_stamp":"2024-04-22T20:48:59.446394+0000","last_deep_scrub":"261'4050","last_deep_scrub_stamp":"2024-04-22T20:48:59.446394+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.446394+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":3000,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:24:00.775234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14224,"num_read_kb":11535,"num_write":6070,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"261'302","reported_seq":1070,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.563705+0000","last_change":"2024-04-22T20:48:39.563748+0000","last_active":"2024-04-22T20:48:39.563705+0000","last_peered":"2024-04-22T20:48:39.563705+0000","last_clean":"2024-04-22T20:48:39.563705+0000","last_became_active":"2024-04-22T19:51:26.496214+0000","last_became_peered":"2024-04-22T19:51:26.496214+0000","last_unstale":"2024-04-22T20:48:39.563705+0000","last_undegraded":"2024-04-22T20:48:39.563705+0000","last_fullsized":"2024-04-22T20:48:39.563705+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'302","last_scrub_stamp":"2024-04-22T20:48:39.563612+0000","last_deep_scrub":"261'302","last_deep_scrub_stamp":"2024-04-22T20:48:39.563612+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.563612+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:41:49.634270+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1338117,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":584,"num_read_kb":13796,"num_write":1181,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"261'298","reported_seq":988,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.430136+0000","last_change":"2024-04-22T20:48:57.430161+0000","last_active":"2024-04-22T20:48:57.430136+0000","last_peered":"2024-04-22T20:48:57.430136+0000","last_clean":"2024-04-22T20:48:57.430136+0000","last_became_active":"2024-04-22T19:51:28.508681+0000","last_became_peered":"2024-04-22T19:51:28.508681+0000","last_unstale":"2024-04-22T20:48:57.430136+0000","last_undegraded":"2024-04-22T20:48:57.430136+0000","last_fullsized":"2024-04-22T20:48:57.430136+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'298","last_scrub_stamp":"2024-04-22T20:48:57.430053+0000","last_deep_scrub":"261'298","last_deep_scrub_stamp":"2024-04-22T20:48:57.430053+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.430053+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:35.213640+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1231,"num_read_kb":1000,"num_write":489,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"261'282","reported_seq":900,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.830889+0000","last_change":"2024-04-22T20:49:02.831005+0000","last_active":"2024-04-22T20:49:02.830889+0000","last_peered":"2024-04-22T20:49:02.830889+0000","last_clean":"2024-04-22T20:49:02.830889+0000","last_became_active":"2024-04-22T19:51:26.502438+0000","last_became_peered":"2024-04-22T19:51:26.502438+0000","last_unstale":"2024-04-22T20:49:02.830889+0000","last_undegraded":"2024-04-22T20:49:02.830889+0000","last_fullsized":"2024-04-22T20:49:02.830889+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'282","last_scrub_stamp":"2024-04-22T20:49:02.830773+0000","last_deep_scrub":"261'282","last_deep_scrub_stamp":"2024-04-22T20:49:02.830773+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.830773+0000","objects_scrubbed":7,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:43:57.450915+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194332,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":386,"num_read_kb":36638,"num_write":936,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"261'374","reported_seq":1262,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.841410+0000","last_change":"2024-04-22T20:49:03.841470+0000","last_active":"2024-04-22T20:49:03.841410+0000","last_peered":"2024-04-22T20:49:03.841410+0000","last_clean":"2024-04-22T20:49:03.841410+0000","last_became_active":"2024-04-22T19:51:26.504878+0000","last_became_peered":"2024-04-22T19:51:26.504878+0000","last_unstale":"2024-04-22T20:49:03.841410+0000","last_undegraded":"2024-04-22T20:49:03.841410+0000","last_fullsized":"2024-04-22T20:49:03.841410+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'374","last_scrub_stamp":"2024-04-22T20:49:03.841311+0000","last_deep_scrub":"261'374","last_deep_scrub_stamp":"2024-04-22T20:49:03.841311+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.841311+0000","objects_scrubbed":8,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:34:22.018993+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5353839,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1018,"num_read_kb":55238,"num_write":1407,"num_write_kb":52838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"261'127","reported_seq":737,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.665636+0000","last_change":"2024-04-22T20:48:52.665674+0000","last_active":"2024-04-22T20:48:52.665636+0000","last_peered":"2024-04-22T20:48:52.665636+0000","last_clean":"2024-04-22T20:48:52.665636+0000","last_became_active":"2024-04-22T19:51:28.521372+0000","last_became_peered":"2024-04-22T19:51:28.521372+0000","last_unstale":"2024-04-22T20:48:52.665636+0000","last_undegraded":"2024-04-22T20:48:52.665636+0000","last_fullsized":"2024-04-22T20:48:52.665636+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'127","last_scrub_stamp":"2024-04-22T20:48:52.665538+0000","last_deep_scrub":"261'127","last_deep_scrub_stamp":"2024-04-22T20:48:52.665538+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.665538+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:58:05.220984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":525,"num_read_kb":437,"num_write":198,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"261'287","reported_seq":950,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.495078+0000","last_change":"2024-04-22T20:48:28.495119+0000","last_active":"2024-04-22T20:48:28.495078+0000","last_peered":"2024-04-22T20:48:28.495078+0000","last_clean":"2024-04-22T20:48:28.495078+0000","last_became_active":"2024-04-22T19:52:04.618273+0000","last_became_peered":"2024-04-22T19:52:04.618273+0000","last_unstale":"2024-04-22T20:48:28.495078+0000","last_undegraded":"2024-04-22T20:48:28.495078+0000","last_fullsized":"2024-04-22T20:48:28.495078+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'287","last_scrub_stamp":"2024-04-22T20:48:28.494984+0000","last_deep_scrub":"261'287","last_deep_scrub_stamp":"2024-04-22T20:48:28.494984+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.494984+0000","objects_scrubbed":7,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:31:35.950376+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1439002,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":73466,"num_write":989,"num_write_kb":40708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"261'95","reported_seq":657,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.938718+0000","last_change":"2024-04-22T20:48:42.938763+0000","last_active":"2024-04-22T20:48:42.938718+0000","last_peered":"2024-04-22T20:48:42.938718+0000","last_clean":"2024-04-22T20:48:42.938718+0000","last_became_active":"2024-04-22T19:51:28.513450+0000","last_became_peered":"2024-04-22T19:51:28.513450+0000","last_unstale":"2024-04-22T20:48:42.938718+0000","last_undegraded":"2024-04-22T20:48:42.938718+0000","last_fullsized":"2024-04-22T20:48:42.938718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'95","last_scrub_stamp":"2024-04-22T20:48:42.938629+0000","last_deep_scrub":"261'95","last_deep_scrub_stamp":"2024-04-22T20:48:42.938629+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.938629+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:02:38.964219+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":382,"num_read_kb":302,"num_write":152,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5385,"num_read_kb":3813,"num_write":2124,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7203,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19082,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":13196,"num_read_kb":13004,"num_write":15956,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4092,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":38164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1949,"ondisk_log_size":1949,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":378,"ondisk_log_size":378,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1476154,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":17118,"num_read_kb":35541,"num_write":14403,"num_write_kb":11173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":165130,"num_omap_keys":386,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1657,"ondisk_log_size":1657,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1622,"num_read_kb":1927,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":85960,"num_read_kb":69763,"num_write":34673,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":42803,"num_omap_keys":179,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3909,"ondisk_log_size":3909,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":564,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":231670890,"num_objects":469,"num_object_clones":0,"num_object_copies":1407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":469,"num_whiteouts":0,"num_read":35099,"num_read_kb":2217057,"num_write":77336,"num_write_kb":2138644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":349753344,"data_stored":349753344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3387,"ondisk_log_size":3387,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":169,"num_read_kb":196064,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":185784,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575546,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":339124,"kb_used_data":51116,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":94032716,"statfs":{"total":96636764160,"available":96289501184,"internally_reserved":0,"allocated":52342784,"data_stored":50369134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":294910410},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575548,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316900,"kb_used_data":46940,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":94054940,"statfs":{"total":96636764160,"available":96312258560,"internally_reserved":0,"allocated":48066560,"data_stored":46141552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":276429254},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575547,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175548,"kb_used_data":54196,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94196292,"statfs":{"total":96636764160,"available":96457003008,"internally_reserved":0,"allocated":55496704,"data_stored":53487509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575548,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145528,"kb_used_data":51888,"kb_used_omap":1,"kb_used_meta":93630,"kb_avail":94226312,"statfs":{"total":96636764160,"available":96487743488,"internally_reserved":0,"allocated":53133312,"data_stored":51218557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":95877573},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575548,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":336520,"kb_used_data":62464,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94035320,"statfs":{"total":96636764160,"available":96292167680,"internally_reserved":0,"allocated":63963136,"data_stored":62012545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":280623560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575549,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144372,"kb_used_data":59116,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94227468,"statfs":{"total":96636764160,"available":96488927232,"internally_reserved":0,"allocated":60534784,"data_stored":58629075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575546,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":188908,"kb_used_data":53668,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94182932,"statfs":{"total":96636764160,"available":96443322368,"internally_reserved":0,"allocated":54956032,"data_stored":53018011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575548,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309552,"kb_used_data":61800,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":94062288,"statfs":{"total":96636764160,"available":96319782912,"internally_reserved":0,"allocated":63283200,"data_stored":61318787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":253688262},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":48562176,"data_stored":48562176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44916736,"data_stored":44916736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37486592,"data_stored":37486592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50307072,"data_stored":50307072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36368384,"data_stored":36368384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44924928,"data_stored":44924928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":43872256,"data_stored":43872256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":43315200,"data_stored":43315200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":5948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":4195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":3595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":6129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":4904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":3145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13631488,"data_stored":13631488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":9437184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T20:49:06.501 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.501 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.502 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.502 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.502 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.502 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.502 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.503 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.503 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.503 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.503 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.504 INFO:tasks.ceph:pgid 67.17 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.504 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.504 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.504 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.504 INFO:tasks.ceph:pgid 66.10 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.504 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.505 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.505 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.505 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.506 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.506 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.506 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.506 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.506 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.507 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.507 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.507 INFO:tasks.ceph:pgid 66.6 last_scrub_stamp 2024-04-22T20:05:12.785876+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=12, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.507 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.507 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.508 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.508 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.508 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.508 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.508 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-04-22T20:05:16.866416+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=5, tm_sec=16, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.509 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.509 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.509 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.509 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.510 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.510 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.510 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.510 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.510 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.511 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.511 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.511 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.512 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.512 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-22T19:51:29.503528+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=29, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.512 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.512 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.512 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.513 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.513 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-22T19:51:11.423705+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=11, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.513 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.513 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.514 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-22T19:51:10.422172+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=10, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.514 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.514 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.514 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.515 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.515 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.515 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.516 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.516 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.516 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-22T19:51:27.487444+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=27, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.516 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.516 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.517 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-22T19:51:31.517761+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=31, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.517 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.517 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.517 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.518 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-22T19:51:25.466756+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=19, tm_min=51, tm_sec=25, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:06.518 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-22T20:49:26.519 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T20:49:26.863 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:49:26.875 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T20:49:26.877 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":2002,"stamp":"2024-04-22T20:49:26.356249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":267312719,"num_objects":825,"num_object_clones":0,"num_object_copies":2119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":158844,"num_read_kb":2723065,"num_write":145049,"num_write_kb":2429940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706437,"num_omap_keys":1741,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12773,"ondisk_log_size":12773,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1974500,"kb_used_data":441188,"kb_used_omap":12,"kb_used_meta":1533235,"kb_avail":753000220,"statfs":{"total":773094113280,"available":771072225280,"internally_reserved":0,"allocated":451776512,"data_stored":436195170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1570033202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7928,"num_omap_keys":26,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004711"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.853054+0000","last_change":"2024-04-22T20:49:12.853054+0000","last_active":"2024-04-22T20:49:12.853054+0000","last_peered":"2024-04-22T20:49:12.853054+0000","last_clean":"2024-04-22T20:49:12.853054+0000","last_became_active":"2024-04-22T20:19:16.796520+0000","last_became_peered":"2024-04-22T20:19:16.796520+0000","last_unstale":"2024-04-22T20:49:12.853054+0000","last_undegraded":"2024-04-22T20:49:12.853054+0000","last_fullsized":"2024-04-22T20:49:12.853054+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:12.852862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:12.852862+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.852862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:09.323293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"261'5","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.873311+0000","last_change":"2024-04-22T20:49:08.873339+0000","last_active":"2024-04-22T20:49:08.873311+0000","last_peered":"2024-04-22T20:49:08.873311+0000","last_clean":"2024-04-22T20:49:08.873311+0000","last_became_active":"2024-04-22T20:05:17.900325+0000","last_became_peered":"2024-04-22T20:05:17.900325+0000","last_unstale":"2024-04-22T20:49:08.873311+0000","last_undegraded":"2024-04-22T20:49:08.873311+0000","last_fullsized":"2024-04-22T20:49:08.873311+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:08.873212+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:08.873212+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.873212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:57:06.368419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.497585+0000","last_change":"2024-04-22T20:48:24.497620+0000","last_active":"2024-04-22T20:48:24.497585+0000","last_peered":"2024-04-22T20:48:24.497585+0000","last_clean":"2024-04-22T20:48:24.497585+0000","last_became_active":"2024-04-22T20:05:13.812933+0000","last_became_peered":"2024-04-22T20:05:13.812933+0000","last_unstale":"2024-04-22T20:48:24.497585+0000","last_undegraded":"2024-04-22T20:48:24.497585+0000","last_fullsized":"2024-04-22T20:48:24.497585+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:24.497472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:24.497472+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.497472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:19.340438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"261'20","reported_seq":110,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.612368+0000","last_change":"2024-04-22T20:48:24.612368+0000","last_active":"2024-04-22T20:48:24.612368+0000","last_peered":"2024-04-22T20:48:24.612368+0000","last_clean":"2024-04-22T20:48:24.612368+0000","last_became_active":"2024-04-22T20:01:04.875981+0000","last_became_peered":"2024-04-22T20:01:04.875981+0000","last_unstale":"2024-04-22T20:48:24.612368+0000","last_undegraded":"2024-04-22T20:48:24.612368+0000","last_fullsized":"2024-04-22T20:48:24.612368+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'20","last_scrub_stamp":"2024-04-22T20:48:24.612214+0000","last_deep_scrub":"261'20","last_deep_scrub_stamp":"2024-04-22T20:48:24.612214+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.612214+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:22:32.284247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.544978+0000","last_change":"2024-04-22T20:48:50.544978+0000","last_active":"2024-04-22T20:48:50.544978+0000","last_peered":"2024-04-22T20:48:50.544978+0000","last_clean":"2024-04-22T20:48:50.544978+0000","last_became_active":"2024-04-22T20:19:16.790377+0000","last_became_peered":"2024-04-22T20:19:16.790377+0000","last_unstale":"2024-04-22T20:48:50.544978+0000","last_undegraded":"2024-04-22T20:48:50.544978+0000","last_fullsized":"2024-04-22T20:48:50.544978+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.544857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.544857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.544857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:18:08.988881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.236997+0000","last_change":"2024-04-22T20:49:09.237038+0000","last_active":"2024-04-22T20:49:09.236997+0000","last_peered":"2024-04-22T20:49:09.236997+0000","last_clean":"2024-04-22T20:49:09.236997+0000","last_became_active":"2024-04-22T20:05:17.899338+0000","last_became_peered":"2024-04-22T20:05:17.899338+0000","last_unstale":"2024-04-22T20:49:09.236997+0000","last_undegraded":"2024-04-22T20:49:09.236997+0000","last_fullsized":"2024-04-22T20:49:09.236997+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:09.236883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:09.236883+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.236883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:49.186676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"258'4","reported_seq":62,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.584270+0000","last_change":"2024-04-22T20:48:23.584307+0000","last_active":"2024-04-22T20:48:23.584270+0000","last_peered":"2024-04-22T20:48:23.584270+0000","last_clean":"2024-04-22T20:48:23.584270+0000","last_became_active":"2024-04-22T20:05:13.807231+0000","last_became_peered":"2024-04-22T20:05:13.807231+0000","last_unstale":"2024-04-22T20:48:23.584270+0000","last_undegraded":"2024-04-22T20:48:23.584270+0000","last_fullsized":"2024-04-22T20:48:23.584270+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:23.584182+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:23.584182+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.584182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:34.394463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"261'22","reported_seq":111,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.458388+0000","last_change":"2024-04-22T20:48:34.458388+0000","last_active":"2024-04-22T20:48:34.458388+0000","last_peered":"2024-04-22T20:48:34.458388+0000","last_clean":"2024-04-22T20:48:34.458388+0000","last_became_active":"2024-04-22T20:01:04.874575+0000","last_became_peered":"2024-04-22T20:01:04.874575+0000","last_unstale":"2024-04-22T20:48:34.458388+0000","last_undegraded":"2024-04-22T20:48:34.458388+0000","last_fullsized":"2024-04-22T20:48:34.458388+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'22","last_scrub_stamp":"2024-04-22T20:48:34.458264+0000","last_deep_scrub":"261'22","last_deep_scrub_stamp":"2024-04-22T20:48:34.458264+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.458264+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:46:15.278433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":48,"num_write":47,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.725001+0000","last_change":"2024-04-22T20:49:03.725044+0000","last_active":"2024-04-22T20:49:03.725001+0000","last_peered":"2024-04-22T20:49:03.725001+0000","last_clean":"2024-04-22T20:49:03.725001+0000","last_became_active":"2024-04-22T20:19:16.794700+0000","last_became_peered":"2024-04-22T20:19:16.794700+0000","last_unstale":"2024-04-22T20:49:03.725001+0000","last_undegraded":"2024-04-22T20:49:03.725001+0000","last_fullsized":"2024-04-22T20:49:03.725001+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:03.724898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:03.724898+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.724898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:21:17.669058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:16.689969+0000","last_change":"2024-04-22T20:49:16.690009+0000","last_active":"2024-04-22T20:49:16.689969+0000","last_peered":"2024-04-22T20:49:16.689969+0000","last_clean":"2024-04-22T20:49:16.689969+0000","last_became_active":"2024-04-22T20:05:18.211400+0000","last_became_peered":"2024-04-22T20:05:18.211400+0000","last_unstale":"2024-04-22T20:49:16.689969+0000","last_undegraded":"2024-04-22T20:49:16.689969+0000","last_fullsized":"2024-04-22T20:49:16.689969+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:16.689857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:16.689857+0000","last_clean_scrub_stamp":"2024-04-22T20:49:16.689857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:20:14.133518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"261'2","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.570845+0000","last_change":"2024-04-22T20:48:46.570845+0000","last_active":"2024-04-22T20:48:46.570845+0000","last_peered":"2024-04-22T20:48:46.570845+0000","last_clean":"2024-04-22T20:48:46.570845+0000","last_became_active":"2024-04-22T20:05:13.809349+0000","last_became_peered":"2024-04-22T20:05:13.809349+0000","last_unstale":"2024-04-22T20:48:46.570845+0000","last_undegraded":"2024-04-22T20:48:46.570845+0000","last_fullsized":"2024-04-22T20:48:46.570845+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:46.570725+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:46.570725+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.570725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:00.895789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"261'24","reported_seq":124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.511512+0000","last_change":"2024-04-22T20:48:31.511512+0000","last_active":"2024-04-22T20:48:31.511512+0000","last_peered":"2024-04-22T20:48:31.511512+0000","last_clean":"2024-04-22T20:48:31.511512+0000","last_became_active":"2024-04-22T20:01:04.869785+0000","last_became_peered":"2024-04-22T20:01:04.869785+0000","last_unstale":"2024-04-22T20:48:31.511512+0000","last_undegraded":"2024-04-22T20:48:31.511512+0000","last_fullsized":"2024-04-22T20:48:31.511512+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'24","last_scrub_stamp":"2024-04-22T20:48:31.511334+0000","last_deep_scrub":"261'24","last_deep_scrub_stamp":"2024-04-22T20:48:31.511334+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.511334+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:43:54.338193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":82,"num_write":36,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":25,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:20.176404+0000","last_change":"2024-04-22T20:19:16.795441+0000","last_active":"2024-04-22T20:48:20.176404+0000","last_peered":"2024-04-22T20:48:20.176404+0000","last_clean":"2024-04-22T20:48:20.176404+0000","last_became_active":"2024-04-22T20:19:16.793667+0000","last_became_peered":"2024-04-22T20:19:16.793667+0000","last_unstale":"2024-04-22T20:48:20.176404+0000","last_undegraded":"2024-04-22T20:48:20.176404+0000","last_fullsized":"2024-04-22T20:48:20.176404+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:19:15.767262+0000","last_clean_scrub_stamp":"2024-04-22T20:19:15.767262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"261'10","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:17.704150+0000","last_change":"2024-04-22T20:49:17.704183+0000","last_active":"2024-04-22T20:49:17.704150+0000","last_peered":"2024-04-22T20:49:17.704150+0000","last_clean":"2024-04-22T20:49:17.704150+0000","last_became_active":"2024-04-22T20:06:05.345554+0000","last_became_peered":"2024-04-22T20:06:05.345554+0000","last_unstale":"2024-04-22T20:49:17.704150+0000","last_undegraded":"2024-04-22T20:49:17.704150+0000","last_fullsized":"2024-04-22T20:49:17.704150+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:49:17.704058+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:49:17.704058+0000","last_clean_scrub_stamp":"2024-04-22T20:49:17.704058+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:58.147978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":20,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1f","version":"261'10","reported_seq":78,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.367896+0000","last_change":"2024-04-22T20:48:36.367896+0000","last_active":"2024-04-22T20:48:36.367896+0000","last_peered":"2024-04-22T20:48:36.367896+0000","last_clean":"2024-04-22T20:48:36.367896+0000","last_became_active":"2024-04-22T20:05:13.806246+0000","last_became_peered":"2024-04-22T20:05:13.806246+0000","last_unstale":"2024-04-22T20:48:36.367896+0000","last_undegraded":"2024-04-22T20:48:36.367896+0000","last_fullsized":"2024-04-22T20:48:36.367896+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:48:36.367701+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:48:36.367701+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.367701+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:06:18.240490+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":16720,"num_write":13,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"261'59","reported_seq":208,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.583472+0000","last_change":"2024-04-22T20:48:30.583472+0000","last_active":"2024-04-22T20:48:30.583472+0000","last_peered":"2024-04-22T20:48:30.583472+0000","last_clean":"2024-04-22T20:48:30.583472+0000","last_became_active":"2024-04-22T20:01:04.873899+0000","last_became_peered":"2024-04-22T20:01:04.873899+0000","last_unstale":"2024-04-22T20:48:30.583472+0000","last_undegraded":"2024-04-22T20:48:30.583472+0000","last_fullsized":"2024-04-22T20:48:30.583472+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'59","last_scrub_stamp":"2024-04-22T20:48:30.583311+0000","last_deep_scrub":"261'59","last_deep_scrub_stamp":"2024-04-22T20:48:30.583311+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.583311+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:58:29.692843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":183,"num_write":99,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:23.243084+0000","last_change":"2024-04-22T20:49:23.243107+0000","last_active":"2024-04-22T20:49:23.243084+0000","last_peered":"2024-04-22T20:49:23.243084+0000","last_clean":"2024-04-22T20:49:23.243084+0000","last_became_active":"2024-04-22T20:19:16.794110+0000","last_became_peered":"2024-04-22T20:19:16.794110+0000","last_unstale":"2024-04-22T20:49:23.243084+0000","last_undegraded":"2024-04-22T20:49:23.243084+0000","last_fullsized":"2024-04-22T20:49:23.243084+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:23.242988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:23.242988+0000","last_clean_scrub_stamp":"2024-04-22T20:49:23.242988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:42:20.330984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"261'2","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.865180+0000","last_change":"2024-04-22T20:49:07.865227+0000","last_active":"2024-04-22T20:49:07.865180+0000","last_peered":"2024-04-22T20:49:07.865180+0000","last_clean":"2024-04-22T20:49:07.865180+0000","last_became_active":"2024-04-22T20:05:17.897771+0000","last_became_peered":"2024-04-22T20:05:17.897771+0000","last_unstale":"2024-04-22T20:49:07.865180+0000","last_undegraded":"2024-04-22T20:49:07.865180+0000","last_fullsized":"2024-04-22T20:49:07.865180+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:07.865075+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:07.865075+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.865075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:05.559298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"261'6","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.520623+0000","last_change":"2024-04-22T20:48:29.520623+0000","last_active":"2024-04-22T20:48:29.520623+0000","last_peered":"2024-04-22T20:48:29.520623+0000","last_clean":"2024-04-22T20:48:29.520623+0000","last_became_active":"2024-04-22T20:05:13.812308+0000","last_became_peered":"2024-04-22T20:05:13.812308+0000","last_unstale":"2024-04-22T20:48:29.520623+0000","last_undegraded":"2024-04-22T20:48:29.520623+0000","last_fullsized":"2024-04-22T20:48:29.520623+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:29.520502+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:29.520502+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.520502+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:11:42.190343+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":9,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"261'8","reported_seq":82,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.526061+0000","last_change":"2024-04-22T20:48:25.526061+0000","last_active":"2024-04-22T20:48:25.526061+0000","last_peered":"2024-04-22T20:48:25.526061+0000","last_clean":"2024-04-22T20:48:25.526061+0000","last_became_active":"2024-04-22T20:01:04.877209+0000","last_became_peered":"2024-04-22T20:01:04.877209+0000","last_unstale":"2024-04-22T20:48:25.526061+0000","last_undegraded":"2024-04-22T20:48:25.526061+0000","last_fullsized":"2024-04-22T20:48:25.526061+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'8","last_scrub_stamp":"2024-04-22T20:48:25.525971+0000","last_deep_scrub":"261'8","last_deep_scrub_stamp":"2024-04-22T20:48:25.525971+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.525971+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:37.255572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.626708+0000","last_change":"2024-04-22T20:49:02.626730+0000","last_active":"2024-04-22T20:49:02.626708+0000","last_peered":"2024-04-22T20:49:02.626708+0000","last_clean":"2024-04-22T20:49:02.626708+0000","last_became_active":"2024-04-22T20:19:16.789599+0000","last_became_peered":"2024-04-22T20:19:16.789599+0000","last_unstale":"2024-04-22T20:49:02.626708+0000","last_undegraded":"2024-04-22T20:49:02.626708+0000","last_fullsized":"2024-04-22T20:49:02.626708+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.626625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.626625+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.626625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:06:20.379636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"261'2","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.611806+0000","last_change":"2024-04-22T20:48:59.611832+0000","last_active":"2024-04-22T20:48:59.611806+0000","last_peered":"2024-04-22T20:48:59.611806+0000","last_clean":"2024-04-22T20:48:59.611806+0000","last_became_active":"2024-04-22T20:05:17.890841+0000","last_became_peered":"2024-04-22T20:05:17.890841+0000","last_unstale":"2024-04-22T20:48:59.611806+0000","last_undegraded":"2024-04-22T20:48:59.611806+0000","last_fullsized":"2024-04-22T20:48:59.611806+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:59.611713+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:59.611713+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.611713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:39:18.164914+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.299460+0000","last_change":"2024-04-22T20:48:40.299460+0000","last_active":"2024-04-22T20:48:40.299460+0000","last_peered":"2024-04-22T20:48:40.299460+0000","last_clean":"2024-04-22T20:48:40.299460+0000","last_became_active":"2024-04-22T20:05:13.807516+0000","last_became_peered":"2024-04-22T20:05:13.807516+0000","last_unstale":"2024-04-22T20:48:40.299460+0000","last_undegraded":"2024-04-22T20:48:40.299460+0000","last_fullsized":"2024-04-22T20:48:40.299460+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.299341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.299341+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.299341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:32:01.048139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"261'77","reported_seq":249,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.378570+0000","last_change":"2024-04-22T20:48:26.378570+0000","last_active":"2024-04-22T20:48:26.378570+0000","last_peered":"2024-04-22T20:48:26.378570+0000","last_clean":"2024-04-22T20:48:26.378570+0000","last_became_active":"2024-04-22T20:01:04.873015+0000","last_became_peered":"2024-04-22T20:01:04.873015+0000","last_unstale":"2024-04-22T20:48:26.378570+0000","last_undegraded":"2024-04-22T20:48:26.378570+0000","last_fullsized":"2024-04-22T20:48:26.378570+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'77","last_scrub_stamp":"2024-04-22T20:48:26.378447+0000","last_deep_scrub":"261'77","last_deep_scrub_stamp":"2024-04-22T20:48:26.378447+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.378447+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:22.636845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":236,"num_write":125,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"261'2","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.451468+0000","last_change":"2024-04-22T20:48:48.451468+0000","last_active":"2024-04-22T20:48:48.451468+0000","last_peered":"2024-04-22T20:48:48.451468+0000","last_clean":"2024-04-22T20:48:48.451468+0000","last_became_active":"2024-04-22T20:19:16.794564+0000","last_became_peered":"2024-04-22T20:19:16.794564+0000","last_unstale":"2024-04-22T20:48:48.451468+0000","last_undegraded":"2024-04-22T20:48:48.451468+0000","last_fullsized":"2024-04-22T20:48:48.451468+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:48.451302+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:48.451302+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.451302+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:04:33.139707+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"261'5","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.663979+0000","last_change":"2024-04-22T20:49:01.664006+0000","last_active":"2024-04-22T20:49:01.663979+0000","last_peered":"2024-04-22T20:49:01.663979+0000","last_clean":"2024-04-22T20:49:01.663979+0000","last_became_active":"2024-04-22T20:05:17.901446+0000","last_became_peered":"2024-04-22T20:05:17.901446+0000","last_unstale":"2024-04-22T20:49:01.663979+0000","last_undegraded":"2024-04-22T20:49:01.663979+0000","last_fullsized":"2024-04-22T20:49:01.663979+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:01.663883+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:01.663883+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.663883+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:07:50.105267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":11264,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.351494+0000","last_change":"2024-04-22T20:48:28.351557+0000","last_active":"2024-04-22T20:48:28.351494+0000","last_peered":"2024-04-22T20:48:28.351494+0000","last_clean":"2024-04-22T20:48:28.351494+0000","last_became_active":"2024-04-22T20:05:13.798311+0000","last_became_peered":"2024-04-22T20:05:13.798311+0000","last_unstale":"2024-04-22T20:48:28.351494+0000","last_undegraded":"2024-04-22T20:48:28.351494+0000","last_fullsized":"2024-04-22T20:48:28.351494+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:28.351390+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:28.351390+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.351390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:49:42.653069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'27","reported_seq":125,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.545417+0000","last_change":"2024-04-22T20:48:28.545440+0000","last_active":"2024-04-22T20:48:28.545417+0000","last_peered":"2024-04-22T20:48:28.545417+0000","last_clean":"2024-04-22T20:48:28.545417+0000","last_became_active":"2024-04-22T20:01:04.873426+0000","last_became_peered":"2024-04-22T20:01:04.873426+0000","last_unstale":"2024-04-22T20:48:28.545417+0000","last_undegraded":"2024-04-22T20:48:28.545417+0000","last_fullsized":"2024-04-22T20:48:28.545417+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'27","last_scrub_stamp":"2024-04-22T20:48:28.545320+0000","last_deep_scrub":"261'27","last_deep_scrub_stamp":"2024-04-22T20:48:28.545320+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.545320+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:51:52.257563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":80,"num_write":54,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:21.601132+0000","last_change":"2024-04-22T20:49:21.601157+0000","last_active":"2024-04-22T20:49:21.601132+0000","last_peered":"2024-04-22T20:49:21.601132+0000","last_clean":"2024-04-22T20:49:21.601132+0000","last_became_active":"2024-04-22T20:19:16.800611+0000","last_became_peered":"2024-04-22T20:19:16.800611+0000","last_unstale":"2024-04-22T20:49:21.601132+0000","last_undegraded":"2024-04-22T20:49:21.601132+0000","last_fullsized":"2024-04-22T20:49:21.601132+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:21.601034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:21.601034+0000","last_clean_scrub_stamp":"2024-04-22T20:49:21.601034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:03.789616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"258'4","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.512936+0000","last_change":"2024-04-22T20:48:44.512963+0000","last_active":"2024-04-22T20:48:44.512936+0000","last_peered":"2024-04-22T20:48:44.512936+0000","last_clean":"2024-04-22T20:48:44.512936+0000","last_became_active":"2024-04-22T20:05:17.892624+0000","last_became_peered":"2024-04-22T20:05:17.892624+0000","last_unstale":"2024-04-22T20:48:44.512936+0000","last_undegraded":"2024-04-22T20:48:44.512936+0000","last_fullsized":"2024-04-22T20:48:44.512936+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:44.512843+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:44.512843+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.512843+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:13:48.032126+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3072,"num_write":4,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.574707+0000","last_change":"2024-04-22T20:48:41.574733+0000","last_active":"2024-04-22T20:48:41.574707+0000","last_peered":"2024-04-22T20:48:41.574707+0000","last_clean":"2024-04-22T20:48:41.574707+0000","last_became_active":"2024-04-22T20:05:13.816011+0000","last_became_peered":"2024-04-22T20:05:13.816011+0000","last_unstale":"2024-04-22T20:48:41.574707+0000","last_undegraded":"2024-04-22T20:48:41.574707+0000","last_fullsized":"2024-04-22T20:48:41.574707+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:41.574619+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:41.574619+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.574619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:10:05.304886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"261'43","reported_seq":160,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.562960+0000","last_change":"2024-04-22T20:48:25.562960+0000","last_active":"2024-04-22T20:48:25.562960+0000","last_peered":"2024-04-22T20:48:25.562960+0000","last_clean":"2024-04-22T20:48:25.562960+0000","last_became_active":"2024-04-22T20:01:04.872356+0000","last_became_peered":"2024-04-22T20:01:04.872356+0000","last_unstale":"2024-04-22T20:48:25.562960+0000","last_undegraded":"2024-04-22T20:48:25.562960+0000","last_fullsized":"2024-04-22T20:48:25.562960+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'43","last_scrub_stamp":"2024-04-22T20:48:25.562858+0000","last_deep_scrub":"261'43","last_deep_scrub_stamp":"2024-04-22T20:48:25.562858+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.562858+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:39:26.408917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":119,"num_write":72,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.688311+0000","last_change":"2024-04-22T20:49:04.688337+0000","last_active":"2024-04-22T20:49:04.688311+0000","last_peered":"2024-04-22T20:49:04.688311+0000","last_clean":"2024-04-22T20:49:04.688311+0000","last_became_active":"2024-04-22T20:19:16.796155+0000","last_became_peered":"2024-04-22T20:19:16.796155+0000","last_unstale":"2024-04-22T20:49:04.688311+0000","last_undegraded":"2024-04-22T20:49:04.688311+0000","last_fullsized":"2024-04-22T20:49:04.688311+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:04.688168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:04.688168+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.688168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:59.083750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.425810+0000","last_change":"2024-04-22T20:48:52.425835+0000","last_active":"2024-04-22T20:48:52.425810+0000","last_peered":"2024-04-22T20:48:52.425810+0000","last_clean":"2024-04-22T20:48:52.425810+0000","last_became_active":"2024-04-22T20:05:18.209023+0000","last_became_peered":"2024-04-22T20:05:18.209023+0000","last_unstale":"2024-04-22T20:48:52.425810+0000","last_undegraded":"2024-04-22T20:48:52.425810+0000","last_fullsized":"2024-04-22T20:48:52.425810+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.425722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.425722+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.425722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:32:32.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"261'7","reported_seq":70,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.910170+0000","last_change":"2024-04-22T20:48:39.910210+0000","last_active":"2024-04-22T20:48:39.910170+0000","last_peered":"2024-04-22T20:48:39.910170+0000","last_clean":"2024-04-22T20:48:39.910170+0000","last_became_active":"2024-04-22T20:05:13.809548+0000","last_became_peered":"2024-04-22T20:05:13.809548+0000","last_unstale":"2024-04-22T20:48:39.910170+0000","last_undegraded":"2024-04-22T20:48:39.910170+0000","last_fullsized":"2024-04-22T20:48:39.910170+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:39.910080+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:39.910080+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.910080+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:39:16.656357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":6146,"num_write":5,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'64","reported_seq":252,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.544266+0000","last_change":"2024-04-22T20:48:30.544266+0000","last_active":"2024-04-22T20:48:30.544266+0000","last_peered":"2024-04-22T20:48:30.544266+0000","last_clean":"2024-04-22T20:48:30.544266+0000","last_became_active":"2024-04-22T20:01:04.873236+0000","last_became_peered":"2024-04-22T20:01:04.873236+0000","last_unstale":"2024-04-22T20:48:30.544266+0000","last_undegraded":"2024-04-22T20:48:30.544266+0000","last_fullsized":"2024-04-22T20:48:30.544266+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'64","last_scrub_stamp":"2024-04-22T20:48:30.544111+0000","last_deep_scrub":"261'64","last_deep_scrub_stamp":"2024-04-22T20:48:30.544111+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.544111+0000","objects_scrubbed":1,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:51.536623+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":250,"num_write":111,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3950,"num_omap_keys":8,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"261'2","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.590705+0000","last_change":"2024-04-22T20:48:51.590731+0000","last_active":"2024-04-22T20:48:51.590705+0000","last_peered":"2024-04-22T20:48:51.590705+0000","last_clean":"2024-04-22T20:48:51.590705+0000","last_became_active":"2024-04-22T20:19:16.791597+0000","last_became_peered":"2024-04-22T20:19:16.791597+0000","last_unstale":"2024-04-22T20:48:51.590705+0000","last_undegraded":"2024-04-22T20:48:51.590705+0000","last_fullsized":"2024-04-22T20:48:51.590705+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:51.590624+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:51.590624+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.590624+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:08.152952+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:19.298007+0000","last_change":"2024-04-22T20:49:19.298045+0000","last_active":"2024-04-22T20:49:19.298007+0000","last_peered":"2024-04-22T20:49:19.298007+0000","last_clean":"2024-04-22T20:49:19.298007+0000","last_became_active":"2024-04-22T20:05:18.210704+0000","last_became_peered":"2024-04-22T20:05:18.210704+0000","last_unstale":"2024-04-22T20:49:19.298007+0000","last_undegraded":"2024-04-22T20:49:19.298007+0000","last_fullsized":"2024-04-22T20:49:19.298007+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:19.297858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:19.297858+0000","last_clean_scrub_stamp":"2024-04-22T20:49:19.297858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:48:21.613563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.609480+0000","last_change":"2024-04-22T20:48:27.609480+0000","last_active":"2024-04-22T20:48:27.609480+0000","last_peered":"2024-04-22T20:48:27.609480+0000","last_clean":"2024-04-22T20:48:27.609480+0000","last_became_active":"2024-04-22T20:05:13.810902+0000","last_became_peered":"2024-04-22T20:05:13.810902+0000","last_unstale":"2024-04-22T20:48:27.609480+0000","last_undegraded":"2024-04-22T20:48:27.609480+0000","last_fullsized":"2024-04-22T20:48:27.609480+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:27.609361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:27.609361+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.609361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:08.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"250'9","reported_seq":86,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.596526+0000","last_change":"2024-04-22T20:48:27.596526+0000","last_active":"2024-04-22T20:48:27.596526+0000","last_peered":"2024-04-22T20:48:27.596526+0000","last_clean":"2024-04-22T20:48:27.596526+0000","last_became_active":"2024-04-22T20:01:04.874230+0000","last_became_peered":"2024-04-22T20:01:04.874230+0000","last_unstale":"2024-04-22T20:48:27.596526+0000","last_undegraded":"2024-04-22T20:48:27.596526+0000","last_fullsized":"2024-04-22T20:48:27.596526+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"250'9","last_scrub_stamp":"2024-04-22T20:48:27.596354+0000","last_deep_scrub":"250'9","last_deep_scrub_stamp":"2024-04-22T20:48:27.596354+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.596354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:12:20.356156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:16.548914+0000","last_change":"2024-04-22T20:49:16.548937+0000","last_active":"2024-04-22T20:49:16.548914+0000","last_peered":"2024-04-22T20:49:16.548914+0000","last_clean":"2024-04-22T20:49:16.548914+0000","last_became_active":"2024-04-22T20:19:16.795139+0000","last_became_peered":"2024-04-22T20:19:16.795139+0000","last_unstale":"2024-04-22T20:49:16.548914+0000","last_undegraded":"2024-04-22T20:49:16.548914+0000","last_fullsized":"2024-04-22T20:49:16.548914+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:16.548816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:16.548816+0000","last_clean_scrub_stamp":"2024-04-22T20:49:16.548816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:52:47.710385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"261'19","reported_seq":88,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:20.283697+0000","last_change":"2024-04-22T20:49:20.283758+0000","last_active":"2024-04-22T20:49:20.283697+0000","last_peered":"2024-04-22T20:49:20.283697+0000","last_clean":"2024-04-22T20:49:20.283697+0000","last_became_active":"2024-04-22T20:05:18.212336+0000","last_became_peered":"2024-04-22T20:05:18.212336+0000","last_unstale":"2024-04-22T20:49:20.283697+0000","last_undegraded":"2024-04-22T20:49:20.283697+0000","last_fullsized":"2024-04-22T20:49:20.283697+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'19","last_scrub_stamp":"2024-04-22T20:49:20.283608+0000","last_deep_scrub":"261'19","last_deep_scrub_stamp":"2024-04-22T20:49:20.283608+0000","last_clean_scrub_stamp":"2024-04-22T20:49:20.283608+0000","objects_scrubbed":3,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:02.878763+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437184,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":38918,"num_write":23,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.569695+0000","last_change":"2024-04-22T20:48:49.569733+0000","last_active":"2024-04-22T20:48:49.569695+0000","last_peered":"2024-04-22T20:48:49.569695+0000","last_clean":"2024-04-22T20:48:49.569695+0000","last_became_active":"2024-04-22T20:05:13.816036+0000","last_became_peered":"2024-04-22T20:05:13.816036+0000","last_unstale":"2024-04-22T20:48:49.569695+0000","last_undegraded":"2024-04-22T20:48:49.569695+0000","last_fullsized":"2024-04-22T20:48:49.569695+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:49.569601+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:49.569601+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.569601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:07:18.107853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'53","reported_seq":186,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.516524+0000","last_change":"2024-04-22T20:48:36.516524+0000","last_active":"2024-04-22T20:48:36.516524+0000","last_peered":"2024-04-22T20:48:36.516524+0000","last_clean":"2024-04-22T20:48:36.516524+0000","last_became_active":"2024-04-22T20:01:04.878067+0000","last_became_peered":"2024-04-22T20:01:04.878067+0000","last_unstale":"2024-04-22T20:48:36.516524+0000","last_undegraded":"2024-04-22T20:48:36.516524+0000","last_fullsized":"2024-04-22T20:48:36.516524+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'53","last_scrub_stamp":"2024-04-22T20:48:36.516407+0000","last_deep_scrub":"261'53","last_deep_scrub_stamp":"2024-04-22T20:48:36.516407+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.516407+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:26.653699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":150,"num_write":99,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.849615+0000","last_change":"2024-04-22T20:49:11.849660+0000","last_active":"2024-04-22T20:49:11.849615+0000","last_peered":"2024-04-22T20:49:11.849615+0000","last_clean":"2024-04-22T20:49:11.849615+0000","last_became_active":"2024-04-22T20:19:16.795855+0000","last_became_peered":"2024-04-22T20:19:16.795855+0000","last_unstale":"2024-04-22T20:49:11.849615+0000","last_undegraded":"2024-04-22T20:49:11.849615+0000","last_fullsized":"2024-04-22T20:49:11.849615+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:11.849504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:11.849504+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.849504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:18:41.306522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"0'0","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.281165+0000","last_change":"2024-04-22T20:49:12.281192+0000","last_active":"2024-04-22T20:49:12.281165+0000","last_peered":"2024-04-22T20:49:12.281165+0000","last_clean":"2024-04-22T20:49:12.281165+0000","last_became_active":"2024-04-22T20:05:17.901954+0000","last_became_peered":"2024-04-22T20:05:17.901954+0000","last_unstale":"2024-04-22T20:49:12.281165+0000","last_undegraded":"2024-04-22T20:49:12.281165+0000","last_fullsized":"2024-04-22T20:49:12.281165+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:12.281076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:12.281076+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.281076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:13.094684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'4","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.504218+0000","last_change":"2024-04-22T20:48:33.504218+0000","last_active":"2024-04-22T20:48:33.504218+0000","last_peered":"2024-04-22T20:48:33.504218+0000","last_clean":"2024-04-22T20:48:33.504218+0000","last_became_active":"2024-04-22T20:05:13.813256+0000","last_became_peered":"2024-04-22T20:05:13.813256+0000","last_unstale":"2024-04-22T20:48:33.504218+0000","last_undegraded":"2024-04-22T20:48:33.504218+0000","last_fullsized":"2024-04-22T20:48:33.504218+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:33.504106+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:33.504106+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.504106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:41:49.887327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":44,"num_write":4,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"261'46","reported_seq":173,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.448464+0000","last_change":"2024-04-22T20:48:33.448487+0000","last_active":"2024-04-22T20:48:33.448464+0000","last_peered":"2024-04-22T20:48:33.448464+0000","last_clean":"2024-04-22T20:48:33.448464+0000","last_became_active":"2024-04-22T20:01:04.873533+0000","last_became_peered":"2024-04-22T20:01:04.873533+0000","last_unstale":"2024-04-22T20:48:33.448464+0000","last_undegraded":"2024-04-22T20:48:33.448464+0000","last_fullsized":"2024-04-22T20:48:33.448464+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'46","last_scrub_stamp":"2024-04-22T20:48:33.448376+0000","last_deep_scrub":"261'46","last_deep_scrub_stamp":"2024-04-22T20:48:33.448376+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.448376+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:02:25.135711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":192,"num_read_kb":137,"num_write":82,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:21.272514+0000","last_change":"2024-04-22T20:49:21.272536+0000","last_active":"2024-04-22T20:49:21.272514+0000","last_peered":"2024-04-22T20:49:21.272514+0000","last_clean":"2024-04-22T20:49:21.272514+0000","last_became_active":"2024-04-22T20:19:16.792856+0000","last_became_peered":"2024-04-22T20:19:16.792856+0000","last_unstale":"2024-04-22T20:49:21.272514+0000","last_undegraded":"2024-04-22T20:49:21.272514+0000","last_fullsized":"2024-04-22T20:49:21.272514+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:21.272423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:21.272423+0000","last_clean_scrub_stamp":"2024-04-22T20:49:21.272423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:12:40.395975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'9","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.654837+0000","last_change":"2024-04-22T20:48:59.654866+0000","last_active":"2024-04-22T20:48:59.654837+0000","last_peered":"2024-04-22T20:48:59.654837+0000","last_clean":"2024-04-22T20:48:59.654837+0000","last_became_active":"2024-04-22T20:05:17.897040+0000","last_became_peered":"2024-04-22T20:05:17.897040+0000","last_unstale":"2024-04-22T20:48:59.654837+0000","last_undegraded":"2024-04-22T20:48:59.654837+0000","last_fullsized":"2024-04-22T20:48:59.654837+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-22T20:48:59.654751+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-22T20:48:59.654751+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.654751+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:11.365352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":12290,"num_write":7,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"261'4","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.219130+0000","last_change":"2024-04-22T20:49:07.219154+0000","last_active":"2024-04-22T20:49:07.219130+0000","last_peered":"2024-04-22T20:49:07.219130+0000","last_clean":"2024-04-22T20:49:07.219130+0000","last_became_active":"2024-04-22T20:05:13.814222+0000","last_became_peered":"2024-04-22T20:05:13.814222+0000","last_unstale":"2024-04-22T20:49:07.219130+0000","last_undegraded":"2024-04-22T20:49:07.219130+0000","last_fullsized":"2024-04-22T20:49:07.219130+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:49:07.219038+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:49:07.219038+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.219038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:39:41.878224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"261'54","reported_seq":199,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.410439+0000","last_change":"2024-04-22T20:48:34.410468+0000","last_active":"2024-04-22T20:48:34.410439+0000","last_peered":"2024-04-22T20:48:34.410439+0000","last_clean":"2024-04-22T20:48:34.410439+0000","last_became_active":"2024-04-22T20:01:04.873422+0000","last_became_peered":"2024-04-22T20:01:04.873422+0000","last_unstale":"2024-04-22T20:48:34.410439+0000","last_undegraded":"2024-04-22T20:48:34.410439+0000","last_fullsized":"2024-04-22T20:48:34.410439+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'54","last_scrub_stamp":"2024-04-22T20:48:34.410341+0000","last_deep_scrub":"261'54","last_deep_scrub_stamp":"2024-04-22T20:48:34.410341+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.410341+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:47:24.001212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":184,"num_write":82,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:22.728082+0000","last_change":"2024-04-22T20:49:22.728111+0000","last_active":"2024-04-22T20:49:22.728082+0000","last_peered":"2024-04-22T20:49:22.728082+0000","last_clean":"2024-04-22T20:49:22.728082+0000","last_became_active":"2024-04-22T20:19:16.795633+0000","last_became_peered":"2024-04-22T20:19:16.795633+0000","last_unstale":"2024-04-22T20:49:22.728082+0000","last_undegraded":"2024-04-22T20:49:22.728082+0000","last_fullsized":"2024-04-22T20:49:22.728082+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:22.727960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:22.727960+0000","last_clean_scrub_stamp":"2024-04-22T20:49:22.727960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:23:44.276656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"261'10","reported_seq":68,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.653814+0000","last_change":"2024-04-22T20:49:00.653851+0000","last_active":"2024-04-22T20:49:00.653814+0000","last_peered":"2024-04-22T20:49:00.653814+0000","last_clean":"2024-04-22T20:49:00.653814+0000","last_became_active":"2024-04-22T20:05:17.899682+0000","last_became_peered":"2024-04-22T20:05:17.899682+0000","last_unstale":"2024-04-22T20:49:00.653814+0000","last_undegraded":"2024-04-22T20:49:00.653814+0000","last_fullsized":"2024-04-22T20:49:00.653814+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:49:00.653727+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:49:00.653727+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.653727+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:51:18.897192+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":8210,"num_write":13,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"261'6","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.625913+0000","last_change":"2024-04-22T20:48:28.625974+0000","last_active":"2024-04-22T20:48:28.625913+0000","last_peered":"2024-04-22T20:48:28.625913+0000","last_clean":"2024-04-22T20:48:28.625913+0000","last_became_active":"2024-04-22T20:05:13.812893+0000","last_became_peered":"2024-04-22T20:05:13.812893+0000","last_unstale":"2024-04-22T20:48:28.625913+0000","last_undegraded":"2024-04-22T20:48:28.625913+0000","last_fullsized":"2024-04-22T20:48:28.625913+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:28.625814+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:28.625814+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.625814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:06.391872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4096,"num_write":16,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"261'47","reported_seq":172,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.628532+0000","last_change":"2024-04-22T20:48:23.628571+0000","last_active":"2024-04-22T20:48:23.628532+0000","last_peered":"2024-04-22T20:48:23.628532+0000","last_clean":"2024-04-22T20:48:23.628532+0000","last_became_active":"2024-04-22T20:01:04.877755+0000","last_became_peered":"2024-04-22T20:01:04.877755+0000","last_unstale":"2024-04-22T20:48:23.628532+0000","last_undegraded":"2024-04-22T20:48:23.628532+0000","last_fullsized":"2024-04-22T20:48:23.628532+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'47","last_scrub_stamp":"2024-04-22T20:48:23.628425+0000","last_deep_scrub":"261'47","last_deep_scrub_stamp":"2024-04-22T20:48:23.628425+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.628425+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:22.945025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":135,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"261'2","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:20.736208+0000","last_change":"2024-04-22T20:49:20.736240+0000","last_active":"2024-04-22T20:49:20.736208+0000","last_peered":"2024-04-22T20:49:20.736208+0000","last_clean":"2024-04-22T20:49:20.736208+0000","last_became_active":"2024-04-22T20:19:16.793555+0000","last_became_peered":"2024-04-22T20:19:16.793555+0000","last_unstale":"2024-04-22T20:49:20.736208+0000","last_undegraded":"2024-04-22T20:49:20.736208+0000","last_fullsized":"2024-04-22T20:49:20.736208+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:20.736112+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:20.736112+0000","last_clean_scrub_stamp":"2024-04-22T20:49:20.736112+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:41:57.356363+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"258'2","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.475045+0000","last_change":"2024-04-22T20:48:36.475071+0000","last_active":"2024-04-22T20:48:36.475045+0000","last_peered":"2024-04-22T20:48:36.475045+0000","last_clean":"2024-04-22T20:48:36.475045+0000","last_became_active":"2024-04-22T20:05:17.890070+0000","last_became_peered":"2024-04-22T20:05:17.890070+0000","last_unstale":"2024-04-22T20:48:36.475045+0000","last_undegraded":"2024-04-22T20:48:36.475045+0000","last_fullsized":"2024-04-22T20:48:36.475045+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'2","last_scrub_stamp":"2024-04-22T20:48:36.474953+0000","last_deep_scrub":"258'2","last_deep_scrub_stamp":"2024-04-22T20:48:36.474953+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.474953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:22:16.222599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"261'5","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.415179+0000","last_change":"2024-04-22T20:48:30.415205+0000","last_active":"2024-04-22T20:48:30.415179+0000","last_peered":"2024-04-22T20:48:30.415179+0000","last_clean":"2024-04-22T20:48:30.415179+0000","last_became_active":"2024-04-22T20:05:13.806882+0000","last_became_peered":"2024-04-22T20:05:13.806882+0000","last_unstale":"2024-04-22T20:48:30.415179+0000","last_undegraded":"2024-04-22T20:48:30.415179+0000","last_fullsized":"2024-04-22T20:48:30.415179+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:30.415089+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:30.415089+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.415089+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:39.469597+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"261'11","reported_seq":89,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.276700+0000","last_change":"2024-04-22T20:48:25.276728+0000","last_active":"2024-04-22T20:48:25.276700+0000","last_peered":"2024-04-22T20:48:25.276700+0000","last_clean":"2024-04-22T20:48:25.276700+0000","last_became_active":"2024-04-22T20:01:04.860991+0000","last_became_peered":"2024-04-22T20:01:04.860991+0000","last_unstale":"2024-04-22T20:48:25.276700+0000","last_undegraded":"2024-04-22T20:48:25.276700+0000","last_fullsized":"2024-04-22T20:48:25.276700+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'11","last_scrub_stamp":"2024-04-22T20:48:25.276600+0000","last_deep_scrub":"261'11","last_deep_scrub_stamp":"2024-04-22T20:48:25.276600+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.276600+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:39:08.585319+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":31,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"261'2","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:24.240697+0000","last_change":"2024-04-22T20:49:24.240735+0000","last_active":"2024-04-22T20:49:24.240697+0000","last_peered":"2024-04-22T20:49:24.240697+0000","last_clean":"2024-04-22T20:49:24.240697+0000","last_became_active":"2024-04-22T20:19:16.791995+0000","last_became_peered":"2024-04-22T20:19:16.791995+0000","last_unstale":"2024-04-22T20:49:24.240697+0000","last_undegraded":"2024-04-22T20:49:24.240697+0000","last_fullsized":"2024-04-22T20:49:24.240697+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:24.240601+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:24.240601+0000","last_clean_scrub_stamp":"2024-04-22T20:49:24.240601+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:53:39.725161+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"258'4","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.641951+0000","last_change":"2024-04-22T20:48:58.641997+0000","last_active":"2024-04-22T20:48:58.641951+0000","last_peered":"2024-04-22T20:48:58.641951+0000","last_clean":"2024-04-22T20:48:58.641951+0000","last_became_active":"2024-04-22T20:05:17.890408+0000","last_became_peered":"2024-04-22T20:05:17.890408+0000","last_unstale":"2024-04-22T20:48:58.641951+0000","last_undegraded":"2024-04-22T20:48:58.641951+0000","last_fullsized":"2024-04-22T20:48:58.641951+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:58.641819+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:58.641819+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.641819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:00.757652+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1342,"num_write":4,"num_write_kb":1342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"258'4","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.956463+0000","last_change":"2024-04-22T20:48:35.956522+0000","last_active":"2024-04-22T20:48:35.956463+0000","last_peered":"2024-04-22T20:48:35.956463+0000","last_clean":"2024-04-22T20:48:35.956463+0000","last_became_active":"2024-04-22T20:05:13.808473+0000","last_became_peered":"2024-04-22T20:05:13.808473+0000","last_unstale":"2024-04-22T20:48:35.956463+0000","last_undegraded":"2024-04-22T20:48:35.956463+0000","last_fullsized":"2024-04-22T20:48:35.956463+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:35.956357+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:35.956357+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.956357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:07:38.200109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"261'47","reported_seq":213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.554713+0000","last_change":"2024-04-22T20:48:26.554763+0000","last_active":"2024-04-22T20:48:26.554713+0000","last_peered":"2024-04-22T20:48:26.554713+0000","last_clean":"2024-04-22T20:48:26.554713+0000","last_became_active":"2024-04-22T20:01:04.874816+0000","last_became_peered":"2024-04-22T20:01:04.874816+0000","last_unstale":"2024-04-22T20:48:26.554713+0000","last_undegraded":"2024-04-22T20:48:26.554713+0000","last_fullsized":"2024-04-22T20:48:26.554713+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'47","last_scrub_stamp":"2024-04-22T20:48:26.554614+0000","last_deep_scrub":"261'47","last_deep_scrub_stamp":"2024-04-22T20:48:26.554614+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.554614+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:35:39.583851+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":291,"num_read_kb":200,"num_write":62,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3253,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.555923+0000","last_change":"2024-04-22T20:48:53.555978+0000","last_active":"2024-04-22T20:48:53.555923+0000","last_peered":"2024-04-22T20:48:53.555923+0000","last_clean":"2024-04-22T20:48:53.555923+0000","last_became_active":"2024-04-22T20:19:16.794954+0000","last_became_peered":"2024-04-22T20:19:16.794954+0000","last_unstale":"2024-04-22T20:48:53.555923+0000","last_undegraded":"2024-04-22T20:48:53.555923+0000","last_fullsized":"2024-04-22T20:48:53.555923+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:53.555824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:53.555824+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.555824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:08:41.681280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"261'6","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.542755+0000","last_change":"2024-04-22T20:48:32.542810+0000","last_active":"2024-04-22T20:48:32.542755+0000","last_peered":"2024-04-22T20:48:32.542755+0000","last_clean":"2024-04-22T20:48:32.542755+0000","last_became_active":"2024-04-22T20:05:17.889370+0000","last_became_peered":"2024-04-22T20:05:17.889370+0000","last_unstale":"2024-04-22T20:48:32.542755+0000","last_undegraded":"2024-04-22T20:48:32.542755+0000","last_fullsized":"2024-04-22T20:48:32.542755+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:32.542653+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:32.542653+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.542653+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:05:37.965129+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":6,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"261'2","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.599057+0000","last_change":"2024-04-22T20:48:25.599082+0000","last_active":"2024-04-22T20:48:25.599057+0000","last_peered":"2024-04-22T20:48:25.599057+0000","last_clean":"2024-04-22T20:48:25.599057+0000","last_became_active":"2024-04-22T20:05:13.815317+0000","last_became_peered":"2024-04-22T20:05:13.815317+0000","last_unstale":"2024-04-22T20:48:25.599057+0000","last_undegraded":"2024-04-22T20:48:25.599057+0000","last_fullsized":"2024-04-22T20:48:25.599057+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:25.598973+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:25.598973+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.598973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:21:44.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"261'28","reported_seq":128,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.478567+0000","last_change":"2024-04-22T20:48:23.478567+0000","last_active":"2024-04-22T20:48:23.478567+0000","last_peered":"2024-04-22T20:48:23.478567+0000","last_clean":"2024-04-22T20:48:23.478567+0000","last_became_active":"2024-04-22T20:01:04.878599+0000","last_became_peered":"2024-04-22T20:01:04.878599+0000","last_unstale":"2024-04-22T20:48:23.478567+0000","last_undegraded":"2024-04-22T20:48:23.478567+0000","last_fullsized":"2024-04-22T20:48:23.478567+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:23.478425+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:23.478425+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.478425+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:33.610274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":77,"num_write":54,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:15.594051+0000","last_change":"2024-04-22T20:49:15.594073+0000","last_active":"2024-04-22T20:49:15.594051+0000","last_peered":"2024-04-22T20:49:15.594051+0000","last_clean":"2024-04-22T20:49:15.594051+0000","last_became_active":"2024-04-22T20:19:16.793678+0000","last_became_peered":"2024-04-22T20:19:16.793678+0000","last_unstale":"2024-04-22T20:49:15.594051+0000","last_undegraded":"2024-04-22T20:49:15.594051+0000","last_fullsized":"2024-04-22T20:49:15.594051+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:15.593942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:15.593942+0000","last_clean_scrub_stamp":"2024-04-22T20:49:15.593942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:08.495410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"0'0","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.495293+0000","last_change":"2024-04-22T20:48:40.495316+0000","last_active":"2024-04-22T20:48:40.495293+0000","last_peered":"2024-04-22T20:48:40.495293+0000","last_clean":"2024-04-22T20:48:40.495293+0000","last_became_active":"2024-04-22T20:05:17.892604+0000","last_became_peered":"2024-04-22T20:05:17.892604+0000","last_unstale":"2024-04-22T20:48:40.495293+0000","last_undegraded":"2024-04-22T20:48:40.495293+0000","last_fullsized":"2024-04-22T20:48:40.495293+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.495210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.495210+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.495210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:53:52.170315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"261'11","reported_seq":78,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.602185+0000","last_change":"2024-04-22T20:48:48.602213+0000","last_active":"2024-04-22T20:48:48.602185+0000","last_peered":"2024-04-22T20:48:48.602185+0000","last_clean":"2024-04-22T20:48:48.602185+0000","last_became_active":"2024-04-22T20:05:13.813946+0000","last_became_peered":"2024-04-22T20:05:13.813946+0000","last_unstale":"2024-04-22T20:48:48.602185+0000","last_undegraded":"2024-04-22T20:48:48.602185+0000","last_fullsized":"2024-04-22T20:48:48.602185+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'11","last_scrub_stamp":"2024-04-22T20:48:48.602091+0000","last_deep_scrub":"261'11","last_deep_scrub_stamp":"2024-04-22T20:48:48.602091+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.602091+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:32.170316+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":15362,"num_write":9,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"261'28","reported_seq":136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.953873+0000","last_change":"2024-04-22T20:48:31.953898+0000","last_active":"2024-04-22T20:48:31.953873+0000","last_peered":"2024-04-22T20:48:31.953873+0000","last_clean":"2024-04-22T20:48:31.953873+0000","last_became_active":"2024-04-22T20:01:04.872834+0000","last_became_peered":"2024-04-22T20:01:04.872834+0000","last_unstale":"2024-04-22T20:48:31.953873+0000","last_undegraded":"2024-04-22T20:48:31.953873+0000","last_fullsized":"2024-04-22T20:48:31.953873+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:31.953787+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:31.953787+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.953787+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:05:33.171326+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":90,"num_write":50,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.712975+0000","last_change":"2024-04-22T20:49:02.713000+0000","last_active":"2024-04-22T20:49:02.712975+0000","last_peered":"2024-04-22T20:49:02.712975+0000","last_clean":"2024-04-22T20:49:02.712975+0000","last_became_active":"2024-04-22T20:19:16.790917+0000","last_became_peered":"2024-04-22T20:19:16.790917+0000","last_unstale":"2024-04-22T20:49:02.712975+0000","last_undegraded":"2024-04-22T20:49:02.712975+0000","last_fullsized":"2024-04-22T20:49:02.712975+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.712882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.712882+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.712882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:02.266687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.612835+0000","last_change":"2024-04-22T20:49:01.612858+0000","last_active":"2024-04-22T20:49:01.612835+0000","last_peered":"2024-04-22T20:49:01.612835+0000","last_clean":"2024-04-22T20:49:01.612835+0000","last_became_active":"2024-04-22T20:05:18.211445+0000","last_became_peered":"2024-04-22T20:05:18.211445+0000","last_unstale":"2024-04-22T20:49:01.612835+0000","last_undegraded":"2024-04-22T20:49:01.612835+0000","last_fullsized":"2024-04-22T20:49:01.612835+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:01.612734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:01.612734+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.612734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:00:33.333955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"258'2","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.661659+0000","last_change":"2024-04-22T20:48:22.661681+0000","last_active":"2024-04-22T20:48:22.661659+0000","last_peered":"2024-04-22T20:48:22.661659+0000","last_clean":"2024-04-22T20:48:22.661659+0000","last_became_active":"2024-04-22T20:05:13.806928+0000","last_became_peered":"2024-04-22T20:05:13.806928+0000","last_unstale":"2024-04-22T20:48:22.661659+0000","last_undegraded":"2024-04-22T20:48:22.661659+0000","last_fullsized":"2024-04-22T20:48:22.661659+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'2","last_scrub_stamp":"2024-04-22T20:48:22.661579+0000","last_deep_scrub":"258'2","last_deep_scrub_stamp":"2024-04-22T20:48:22.661579+0000","last_clean_scrub_stamp":"2024-04-22T20:48:22.661579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:42.533500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"261'28","reported_seq":127,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.409573+0000","last_change":"2024-04-22T20:48:27.409602+0000","last_active":"2024-04-22T20:48:27.409573+0000","last_peered":"2024-04-22T20:48:27.409573+0000","last_clean":"2024-04-22T20:48:27.409573+0000","last_became_active":"2024-04-22T20:01:04.869226+0000","last_became_peered":"2024-04-22T20:01:04.869226+0000","last_unstale":"2024-04-22T20:48:27.409573+0000","last_undegraded":"2024-04-22T20:48:27.409573+0000","last_fullsized":"2024-04-22T20:48:27.409573+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:27.409466+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:27.409466+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.409466+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:07.279566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":80,"num_write":56,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:18.553856+0000","last_change":"2024-04-22T20:49:18.553879+0000","last_active":"2024-04-22T20:49:18.553856+0000","last_peered":"2024-04-22T20:49:18.553856+0000","last_clean":"2024-04-22T20:49:18.553856+0000","last_became_active":"2024-04-22T20:19:16.795524+0000","last_became_peered":"2024-04-22T20:19:16.795524+0000","last_unstale":"2024-04-22T20:49:18.553856+0000","last_undegraded":"2024-04-22T20:49:18.553856+0000","last_fullsized":"2024-04-22T20:49:18.553856+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:18.553763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:18.553763+0000","last_clean_scrub_stamp":"2024-04-22T20:49:18.553763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:07:42.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'4","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.484819+0000","last_change":"2024-04-22T20:48:48.484845+0000","last_active":"2024-04-22T20:48:48.484819+0000","last_peered":"2024-04-22T20:48:48.484819+0000","last_clean":"2024-04-22T20:48:48.484819+0000","last_became_active":"2024-04-22T20:05:17.898858+0000","last_became_peered":"2024-04-22T20:05:17.898858+0000","last_unstale":"2024-04-22T20:48:48.484819+0000","last_undegraded":"2024-04-22T20:48:48.484819+0000","last_fullsized":"2024-04-22T20:48:48.484819+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:48.484715+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:48.484715+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.484715+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:42:15.414201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"258'3","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.510708+0000","last_change":"2024-04-22T20:48:27.510738+0000","last_active":"2024-04-22T20:48:27.510708+0000","last_peered":"2024-04-22T20:48:27.510708+0000","last_clean":"2024-04-22T20:48:27.510708+0000","last_became_active":"2024-04-22T20:05:13.807945+0000","last_became_peered":"2024-04-22T20:05:13.807945+0000","last_unstale":"2024-04-22T20:48:27.510708+0000","last_undegraded":"2024-04-22T20:48:27.510708+0000","last_fullsized":"2024-04-22T20:48:27.510708+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'3","last_scrub_stamp":"2024-04-22T20:48:27.510574+0000","last_deep_scrub":"258'3","last_deep_scrub_stamp":"2024-04-22T20:48:27.510574+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.510574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:46.475027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":8,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"261'24","reported_seq":115,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.452597+0000","last_change":"2024-04-22T20:48:32.452634+0000","last_active":"2024-04-22T20:48:32.452597+0000","last_peered":"2024-04-22T20:48:32.452597+0000","last_clean":"2024-04-22T20:48:32.452597+0000","last_became_active":"2024-04-22T20:01:04.871829+0000","last_became_peered":"2024-04-22T20:01:04.871829+0000","last_unstale":"2024-04-22T20:48:32.452597+0000","last_undegraded":"2024-04-22T20:48:32.452597+0000","last_fullsized":"2024-04-22T20:48:32.452597+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'24","last_scrub_stamp":"2024-04-22T20:48:32.452510+0000","last_deep_scrub":"261'24","last_deep_scrub_stamp":"2024-04-22T20:48:32.452510+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.452510+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:57.685562+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":61,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:22.288257+0000","last_change":"2024-04-22T20:49:22.288282+0000","last_active":"2024-04-22T20:49:22.288257+0000","last_peered":"2024-04-22T20:49:22.288257+0000","last_clean":"2024-04-22T20:49:22.288257+0000","last_became_active":"2024-04-22T20:19:16.793727+0000","last_became_peered":"2024-04-22T20:19:16.793727+0000","last_unstale":"2024-04-22T20:49:22.288257+0000","last_undegraded":"2024-04-22T20:49:22.288257+0000","last_fullsized":"2024-04-22T20:49:22.288257+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:22.288151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:22.288151+0000","last_clean_scrub_stamp":"2024-04-22T20:49:22.288151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:45:38.361185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"261'5","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.253489+0000","last_change":"2024-04-22T20:49:08.253566+0000","last_active":"2024-04-22T20:49:08.253489+0000","last_peered":"2024-04-22T20:49:08.253489+0000","last_clean":"2024-04-22T20:49:08.253489+0000","last_became_active":"2024-04-22T20:05:17.899328+0000","last_became_peered":"2024-04-22T20:05:17.899328+0000","last_unstale":"2024-04-22T20:49:08.253489+0000","last_undegraded":"2024-04-22T20:49:08.253489+0000","last_fullsized":"2024-04-22T20:49:08.253489+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:08.253395+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:08.253395+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.253395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:22.668162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'7","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.630926+0000","last_change":"2024-04-22T20:48:25.630926+0000","last_active":"2024-04-22T20:48:25.630926+0000","last_peered":"2024-04-22T20:48:25.630926+0000","last_clean":"2024-04-22T20:48:25.630926+0000","last_became_active":"2024-04-22T20:05:13.811986+0000","last_became_peered":"2024-04-22T20:05:13.811986+0000","last_unstale":"2024-04-22T20:48:25.630926+0000","last_undegraded":"2024-04-22T20:48:25.630926+0000","last_fullsized":"2024-04-22T20:48:25.630926+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:25.630769+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:25.630769+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.630769+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:31:51.985237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8240,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"261'66","reported_seq":221,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.587356+0000","last_change":"2024-04-22T20:48:26.587419+0000","last_active":"2024-04-22T20:48:26.587356+0000","last_peered":"2024-04-22T20:48:26.587356+0000","last_clean":"2024-04-22T20:48:26.587356+0000","last_became_active":"2024-04-22T20:01:04.874827+0000","last_became_peered":"2024-04-22T20:01:04.874827+0000","last_unstale":"2024-04-22T20:48:26.587356+0000","last_undegraded":"2024-04-22T20:48:26.587356+0000","last_fullsized":"2024-04-22T20:48:26.587356+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'66","last_scrub_stamp":"2024-04-22T20:48:26.587233+0000","last_deep_scrub":"261'66","last_deep_scrub_stamp":"2024-04-22T20:48:26.587233+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.587233+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:32.347942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":281,"num_read_kb":200,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.423664+0000","last_change":"2024-04-22T20:48:44.423664+0000","last_active":"2024-04-22T20:48:44.423664+0000","last_peered":"2024-04-22T20:48:44.423664+0000","last_clean":"2024-04-22T20:48:44.423664+0000","last_became_active":"2024-04-22T20:19:16.794434+0000","last_became_peered":"2024-04-22T20:19:16.794434+0000","last_unstale":"2024-04-22T20:48:44.423664+0000","last_undegraded":"2024-04-22T20:48:44.423664+0000","last_fullsized":"2024-04-22T20:48:44.423664+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:44.423537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:44.423537+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.423537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:22:23.617146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"261'9","reported_seq":65,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.412462+0000","last_change":"2024-04-22T20:48:36.412462+0000","last_active":"2024-04-22T20:48:36.412462+0000","last_peered":"2024-04-22T20:48:36.412462+0000","last_clean":"2024-04-22T20:48:36.412462+0000","last_became_active":"2024-04-22T20:05:18.209025+0000","last_became_peered":"2024-04-22T20:05:18.209025+0000","last_unstale":"2024-04-22T20:48:36.412462+0000","last_undegraded":"2024-04-22T20:48:36.412462+0000","last_fullsized":"2024-04-22T20:48:36.412462+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-22T20:48:36.412356+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-22T20:48:36.412356+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.412356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:02.300050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":13312,"num_write":14,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"261'4","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.619549+0000","last_change":"2024-04-22T20:48:22.619549+0000","last_active":"2024-04-22T20:48:22.619549+0000","last_peered":"2024-04-22T20:48:22.619549+0000","last_clean":"2024-04-22T20:48:22.619549+0000","last_became_active":"2024-04-22T20:05:13.807603+0000","last_became_peered":"2024-04-22T20:05:13.807603+0000","last_unstale":"2024-04-22T20:48:22.619549+0000","last_undegraded":"2024-04-22T20:48:22.619549+0000","last_fullsized":"2024-04-22T20:48:22.619549+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:22.619402+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:22.619402+0000","last_clean_scrub_stamp":"2024-04-22T20:48:22.619402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:32:09.200890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"258'23","reported_seq":122,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.404488+0000","last_change":"2024-04-22T20:48:37.404488+0000","last_active":"2024-04-22T20:48:37.404488+0000","last_peered":"2024-04-22T20:48:37.404488+0000","last_clean":"2024-04-22T20:48:37.404488+0000","last_became_active":"2024-04-22T20:01:04.868847+0000","last_became_peered":"2024-04-22T20:01:04.868847+0000","last_unstale":"2024-04-22T20:48:37.404488+0000","last_undegraded":"2024-04-22T20:48:37.404488+0000","last_fullsized":"2024-04-22T20:48:37.404488+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'23","last_scrub_stamp":"2024-04-22T20:48:37.404364+0000","last_deep_scrub":"258'23","last_deep_scrub_stamp":"2024-04-22T20:48:37.404364+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.404364+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:31.027402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":76,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"261'2","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:19.587547+0000","last_change":"2024-04-22T20:49:19.587578+0000","last_active":"2024-04-22T20:49:19.587547+0000","last_peered":"2024-04-22T20:49:19.587547+0000","last_clean":"2024-04-22T20:49:19.587547+0000","last_became_active":"2024-04-22T20:19:16.795613+0000","last_became_peered":"2024-04-22T20:19:16.795613+0000","last_unstale":"2024-04-22T20:49:19.587547+0000","last_undegraded":"2024-04-22T20:49:19.587547+0000","last_fullsized":"2024-04-22T20:49:19.587547+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:19.587456+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:19.587456+0000","last_clean_scrub_stamp":"2024-04-22T20:49:19.587456+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:54:03.988371+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"261'6","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.538494+0000","last_change":"2024-04-22T20:48:31.538524+0000","last_active":"2024-04-22T20:48:31.538494+0000","last_peered":"2024-04-22T20:48:31.538494+0000","last_clean":"2024-04-22T20:48:31.538494+0000","last_became_active":"2024-04-22T20:05:17.891331+0000","last_became_peered":"2024-04-22T20:05:17.891331+0000","last_unstale":"2024-04-22T20:48:31.538494+0000","last_undegraded":"2024-04-22T20:48:31.538494+0000","last_fullsized":"2024-04-22T20:48:31.538494+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:31.538340+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:31.538340+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.538340+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:36:13.794601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.504525+0000","last_change":"2024-04-22T20:48:37.504525+0000","last_active":"2024-04-22T20:48:37.504525+0000","last_peered":"2024-04-22T20:48:37.504525+0000","last_clean":"2024-04-22T20:48:37.504525+0000","last_became_active":"2024-04-22T20:05:13.813811+0000","last_became_peered":"2024-04-22T20:05:13.813811+0000","last_unstale":"2024-04-22T20:48:37.504525+0000","last_undegraded":"2024-04-22T20:48:37.504525+0000","last_fullsized":"2024-04-22T20:48:37.504525+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:37.504369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:37.504369+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.504369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:22.962064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"258'49","reported_seq":180,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.613719+0000","last_change":"2024-04-22T20:48:29.613719+0000","last_active":"2024-04-22T20:48:29.613719+0000","last_peered":"2024-04-22T20:48:29.613719+0000","last_clean":"2024-04-22T20:48:29.613719+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:48:29.613719+0000","last_undegraded":"2024-04-22T20:48:29.613719+0000","last_fullsized":"2024-04-22T20:48:29.613719+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'49","last_scrub_stamp":"2024-04-22T20:48:29.613592+0000","last_deep_scrub":"258'49","last_deep_scrub_stamp":"2024-04-22T20:48:29.613592+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.613592+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:50:57.860950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":145,"num_write":78,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:19.684330+0000","last_change":"2024-04-22T20:49:19.684353+0000","last_active":"2024-04-22T20:49:19.684330+0000","last_peered":"2024-04-22T20:49:19.684330+0000","last_clean":"2024-04-22T20:49:19.684330+0000","last_became_active":"2024-04-22T20:19:16.793486+0000","last_became_peered":"2024-04-22T20:19:16.793486+0000","last_unstale":"2024-04-22T20:49:19.684330+0000","last_undegraded":"2024-04-22T20:49:19.684330+0000","last_fullsized":"2024-04-22T20:49:19.684330+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:19.684234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:19.684234+0000","last_clean_scrub_stamp":"2024-04-22T20:49:19.684234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:13:31.944544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.489207+0000","last_change":"2024-04-22T20:48:32.489207+0000","last_active":"2024-04-22T20:48:32.489207+0000","last_peered":"2024-04-22T20:48:32.489207+0000","last_clean":"2024-04-22T20:48:32.489207+0000","last_became_active":"2024-04-22T20:05:17.897008+0000","last_became_peered":"2024-04-22T20:05:17.897008+0000","last_unstale":"2024-04-22T20:48:32.489207+0000","last_undegraded":"2024-04-22T20:48:32.489207+0000","last_fullsized":"2024-04-22T20:48:32.489207+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:32.489059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:32.489059+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.489059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:11.685871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"261'15","reported_seq":85,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.378030+0000","last_change":"2024-04-22T20:48:32.378065+0000","last_active":"2024-04-22T20:48:32.378030+0000","last_peered":"2024-04-22T20:48:32.378030+0000","last_clean":"2024-04-22T20:48:32.378030+0000","last_became_active":"2024-04-22T20:05:13.798319+0000","last_became_peered":"2024-04-22T20:05:13.798319+0000","last_unstale":"2024-04-22T20:48:32.378030+0000","last_undegraded":"2024-04-22T20:48:32.378030+0000","last_fullsized":"2024-04-22T20:48:32.378030+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'15","last_scrub_stamp":"2024-04-22T20:48:32.377942+0000","last_deep_scrub":"261'15","last_deep_scrub_stamp":"2024-04-22T20:48:32.377942+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.377942+0000","objects_scrubbed":2,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:58.634704+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":16388,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"261'37","reported_seq":148,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.458538+0000","last_change":"2024-04-22T20:48:28.458538+0000","last_active":"2024-04-22T20:48:28.458538+0000","last_peered":"2024-04-22T20:48:28.458538+0000","last_clean":"2024-04-22T20:48:28.458538+0000","last_became_active":"2024-04-22T20:01:04.869079+0000","last_became_peered":"2024-04-22T20:01:04.869079+0000","last_unstale":"2024-04-22T20:48:28.458538+0000","last_undegraded":"2024-04-22T20:48:28.458538+0000","last_fullsized":"2024-04-22T20:48:28.458538+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'37","last_scrub_stamp":"2024-04-22T20:48:28.458400+0000","last_deep_scrub":"261'37","last_deep_scrub_stamp":"2024-04-22T20:48:28.458400+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.458400+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:24:11.187493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":61,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:17.574269+0000","last_change":"2024-04-22T20:49:17.574296+0000","last_active":"2024-04-22T20:49:17.574269+0000","last_peered":"2024-04-22T20:49:17.574269+0000","last_clean":"2024-04-22T20:49:17.574269+0000","last_became_active":"2024-04-22T20:19:16.795462+0000","last_became_peered":"2024-04-22T20:19:16.795462+0000","last_unstale":"2024-04-22T20:49:17.574269+0000","last_undegraded":"2024-04-22T20:49:17.574269+0000","last_fullsized":"2024-04-22T20:49:17.574269+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:17.574180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:17.574180+0000","last_clean_scrub_stamp":"2024-04-22T20:49:17.574180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:50:53.041281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'7","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.601169+0000","last_change":"2024-04-22T20:48:45.601169+0000","last_active":"2024-04-22T20:48:45.601169+0000","last_peered":"2024-04-22T20:48:45.601169+0000","last_clean":"2024-04-22T20:48:45.601169+0000","last_became_active":"2024-04-22T20:05:17.892044+0000","last_became_peered":"2024-04-22T20:05:17.892044+0000","last_unstale":"2024-04-22T20:48:45.601169+0000","last_undegraded":"2024-04-22T20:48:45.601169+0000","last_fullsized":"2024-04-22T20:48:45.601169+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:45.601054+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:45.601054+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.601054+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:42:15.237225+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"261'5","reported_seq":69,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.541577+0000","last_change":"2024-04-22T20:48:29.541577+0000","last_active":"2024-04-22T20:48:29.541577+0000","last_peered":"2024-04-22T20:48:29.541577+0000","last_clean":"2024-04-22T20:48:29.541577+0000","last_became_active":"2024-04-22T20:05:13.809368+0000","last_became_peered":"2024-04-22T20:05:13.809368+0000","last_unstale":"2024-04-22T20:48:29.541577+0000","last_undegraded":"2024-04-22T20:48:29.541577+0000","last_fullsized":"2024-04-22T20:48:29.541577+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:29.541434+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:29.541434+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.541434+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:38:26.695883+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"261'35","reported_seq":148,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.558111+0000","last_change":"2024-04-22T20:48:27.558111+0000","last_active":"2024-04-22T20:48:27.558111+0000","last_peered":"2024-04-22T20:48:27.558111+0000","last_clean":"2024-04-22T20:48:27.558111+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:48:27.558111+0000","last_undegraded":"2024-04-22T20:48:27.558111+0000","last_fullsized":"2024-04-22T20:48:27.558111+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'35","last_scrub_stamp":"2024-04-22T20:48:27.558001+0000","last_deep_scrub":"261'35","last_deep_scrub_stamp":"2024-04-22T20:48:27.558001+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.558001+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:44:57.759608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":59,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.823339+0000","last_change":"2024-04-22T20:49:10.823362+0000","last_active":"2024-04-22T20:49:10.823339+0000","last_peered":"2024-04-22T20:49:10.823339+0000","last_clean":"2024-04-22T20:49:10.823339+0000","last_became_active":"2024-04-22T20:19:16.791663+0000","last_became_peered":"2024-04-22T20:19:16.791663+0000","last_unstale":"2024-04-22T20:49:10.823339+0000","last_undegraded":"2024-04-22T20:49:10.823339+0000","last_fullsized":"2024-04-22T20:49:10.823339+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:10.823245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:10.823245+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.823245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:09:49.787690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"261'7","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.669783+0000","last_change":"2024-04-22T20:48:49.669783+0000","last_active":"2024-04-22T20:48:49.669783+0000","last_peered":"2024-04-22T20:48:49.669783+0000","last_clean":"2024-04-22T20:48:49.669783+0000","last_became_active":"2024-04-22T20:05:18.210736+0000","last_became_peered":"2024-04-22T20:05:18.210736+0000","last_unstale":"2024-04-22T20:48:49.669783+0000","last_undegraded":"2024-04-22T20:48:49.669783+0000","last_fullsized":"2024-04-22T20:48:49.669783+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:49.669616+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:49.669616+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.669616+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:27:27.520540+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20482,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"261'6","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.576941+0000","last_change":"2024-04-22T20:48:47.576941+0000","last_active":"2024-04-22T20:48:47.576941+0000","last_peered":"2024-04-22T20:48:47.576941+0000","last_clean":"2024-04-22T20:48:47.576941+0000","last_became_active":"2024-04-22T20:05:13.813277+0000","last_became_peered":"2024-04-22T20:05:13.813277+0000","last_unstale":"2024-04-22T20:48:47.576941+0000","last_undegraded":"2024-04-22T20:48:47.576941+0000","last_fullsized":"2024-04-22T20:48:47.576941+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:47.576814+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:47.576814+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.576814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:43:34.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1024,"num_write":16,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"261'61","reported_seq":204,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.363698+0000","last_change":"2024-04-22T20:48:25.363698+0000","last_active":"2024-04-22T20:48:25.363698+0000","last_peered":"2024-04-22T20:48:25.363698+0000","last_clean":"2024-04-22T20:48:25.363698+0000","last_became_active":"2024-04-22T20:01:04.871859+0000","last_became_peered":"2024-04-22T20:01:04.871859+0000","last_unstale":"2024-04-22T20:48:25.363698+0000","last_undegraded":"2024-04-22T20:48:25.363698+0000","last_fullsized":"2024-04-22T20:48:25.363698+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'61","last_scrub_stamp":"2024-04-22T20:48:25.363585+0000","last_deep_scrub":"261'61","last_deep_scrub_stamp":"2024-04-22T20:48:25.363585+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.363585+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:50:52.645407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":172,"num_write":109,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.659643+0000","last_change":"2024-04-22T20:49:03.659643+0000","last_active":"2024-04-22T20:49:03.659643+0000","last_peered":"2024-04-22T20:49:03.659643+0000","last_clean":"2024-04-22T20:49:03.659643+0000","last_became_active":"2024-04-22T20:19:16.789688+0000","last_became_peered":"2024-04-22T20:19:16.789688+0000","last_unstale":"2024-04-22T20:49:03.659643+0000","last_undegraded":"2024-04-22T20:49:03.659643+0000","last_fullsized":"2024-04-22T20:49:03.659643+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:03.659497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:03.659497+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.659497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:02.426808+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"258'4","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.411626+0000","last_change":"2024-04-22T20:48:40.411671+0000","last_active":"2024-04-22T20:48:40.411626+0000","last_peered":"2024-04-22T20:48:40.411626+0000","last_clean":"2024-04-22T20:48:40.411626+0000","last_became_active":"2024-04-22T20:05:18.211478+0000","last_became_peered":"2024-04-22T20:05:18.211478+0000","last_unstale":"2024-04-22T20:48:40.411626+0000","last_undegraded":"2024-04-22T20:48:40.411626+0000","last_fullsized":"2024-04-22T20:48:40.411626+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:40.411479+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:40.411479+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.411479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:39:01.313336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'7","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.885170+0000","last_change":"2024-04-22T20:49:06.885194+0000","last_active":"2024-04-22T20:49:06.885170+0000","last_peered":"2024-04-22T20:49:06.885170+0000","last_clean":"2024-04-22T20:49:06.885170+0000","last_became_active":"2024-04-22T20:05:13.811561+0000","last_became_peered":"2024-04-22T20:05:13.811561+0000","last_unstale":"2024-04-22T20:49:06.885170+0000","last_undegraded":"2024-04-22T20:49:06.885170+0000","last_fullsized":"2024-04-22T20:49:06.885170+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:49:06.885081+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:49:06.885081+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.885081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:02.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":12,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"261'33","reported_seq":145,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.529020+0000","last_change":"2024-04-22T20:48:31.529020+0000","last_active":"2024-04-22T20:48:31.529020+0000","last_peered":"2024-04-22T20:48:31.529020+0000","last_clean":"2024-04-22T20:48:31.529020+0000","last_became_active":"2024-04-22T20:01:04.874790+0000","last_became_peered":"2024-04-22T20:01:04.874790+0000","last_unstale":"2024-04-22T20:48:31.529020+0000","last_undegraded":"2024-04-22T20:48:31.529020+0000","last_fullsized":"2024-04-22T20:48:31.529020+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'33","last_scrub_stamp":"2024-04-22T20:48:31.528870+0000","last_deep_scrub":"261'33","last_deep_scrub_stamp":"2024-04-22T20:48:31.528870+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.528870+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:43.937257+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":100,"num_write":57,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:13.233179+0000","last_change":"2024-04-22T20:49:13.233204+0000","last_active":"2024-04-22T20:49:13.233179+0000","last_peered":"2024-04-22T20:49:13.233179+0000","last_clean":"2024-04-22T20:49:13.233179+0000","last_became_active":"2024-04-22T20:19:16.793188+0000","last_became_peered":"2024-04-22T20:19:16.793188+0000","last_unstale":"2024-04-22T20:49:13.233179+0000","last_undegraded":"2024-04-22T20:49:13.233179+0000","last_fullsized":"2024-04-22T20:49:13.233179+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:13.233055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:13.233055+0000","last_clean_scrub_stamp":"2024-04-22T20:49:13.233055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:19:48.659861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"261'5","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.199142+0000","last_change":"2024-04-22T20:49:10.199194+0000","last_active":"2024-04-22T20:49:10.199142+0000","last_peered":"2024-04-22T20:49:10.199142+0000","last_clean":"2024-04-22T20:49:10.199142+0000","last_became_active":"2024-04-22T20:05:17.901128+0000","last_became_peered":"2024-04-22T20:05:17.901128+0000","last_unstale":"2024-04-22T20:49:10.199142+0000","last_undegraded":"2024-04-22T20:49:10.199142+0000","last_fullsized":"2024-04-22T20:49:10.199142+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:10.199016+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:10.199016+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.199016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:10.213103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":8192,"num_write":10,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"261'5","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.568012+0000","last_change":"2024-04-22T20:48:28.568012+0000","last_active":"2024-04-22T20:48:28.568012+0000","last_peered":"2024-04-22T20:48:28.568012+0000","last_clean":"2024-04-22T20:48:28.568012+0000","last_became_active":"2024-04-22T20:05:13.815332+0000","last_became_peered":"2024-04-22T20:05:13.815332+0000","last_unstale":"2024-04-22T20:48:28.568012+0000","last_undegraded":"2024-04-22T20:48:28.568012+0000","last_fullsized":"2024-04-22T20:48:28.568012+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:28.567863+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:28.567863+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.567863+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:52:21.493631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":9216,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'28","reported_seq":134,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.541277+0000","last_change":"2024-04-22T20:48:26.541277+0000","last_active":"2024-04-22T20:48:26.541277+0000","last_peered":"2024-04-22T20:48:26.541277+0000","last_clean":"2024-04-22T20:48:26.541277+0000","last_became_active":"2024-04-22T20:01:04.871840+0000","last_became_peered":"2024-04-22T20:01:04.871840+0000","last_unstale":"2024-04-22T20:48:26.541277+0000","last_undegraded":"2024-04-22T20:48:26.541277+0000","last_fullsized":"2024-04-22T20:48:26.541277+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:26.541161+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:26.541161+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.541161+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:03:51.347595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":89,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"261'2","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.585427+0000","last_change":"2024-04-22T20:48:52.585467+0000","last_active":"2024-04-22T20:48:52.585427+0000","last_peered":"2024-04-22T20:48:52.585427+0000","last_clean":"2024-04-22T20:48:52.585427+0000","last_became_active":"2024-04-22T20:19:16.794123+0000","last_became_peered":"2024-04-22T20:19:16.794123+0000","last_unstale":"2024-04-22T20:48:52.585427+0000","last_undegraded":"2024-04-22T20:48:52.585427+0000","last_fullsized":"2024-04-22T20:48:52.585427+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:52.585332+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:52.585332+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.585332+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:48.233250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"261'4","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.624822+0000","last_change":"2024-04-22T20:48:58.624822+0000","last_active":"2024-04-22T20:48:58.624822+0000","last_peered":"2024-04-22T20:48:58.624822+0000","last_clean":"2024-04-22T20:48:58.624822+0000","last_became_active":"2024-04-22T20:05:17.895177+0000","last_became_peered":"2024-04-22T20:05:17.895177+0000","last_unstale":"2024-04-22T20:48:58.624822+0000","last_undegraded":"2024-04-22T20:48:58.624822+0000","last_fullsized":"2024-04-22T20:48:58.624822+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:58.624700+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:58.624700+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.624700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:42:11.281730+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":4,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"261'4","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.553078+0000","last_change":"2024-04-22T20:48:30.553078+0000","last_active":"2024-04-22T20:48:30.553078+0000","last_peered":"2024-04-22T20:48:30.553078+0000","last_clean":"2024-04-22T20:48:30.553078+0000","last_became_active":"2024-04-22T20:05:13.809047+0000","last_became_peered":"2024-04-22T20:05:13.809047+0000","last_unstale":"2024-04-22T20:48:30.553078+0000","last_undegraded":"2024-04-22T20:48:30.553078+0000","last_fullsized":"2024-04-22T20:48:30.553078+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:30.552959+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:30.552959+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.552959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:44:03.327794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"261'33","reported_seq":140,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.902332+0000","last_change":"2024-04-22T20:48:27.902332+0000","last_active":"2024-04-22T20:48:27.902332+0000","last_peered":"2024-04-22T20:48:27.902332+0000","last_clean":"2024-04-22T20:48:27.902332+0000","last_became_active":"2024-04-22T20:01:04.872370+0000","last_became_peered":"2024-04-22T20:01:04.872370+0000","last_unstale":"2024-04-22T20:48:27.902332+0000","last_undegraded":"2024-04-22T20:48:27.902332+0000","last_fullsized":"2024-04-22T20:48:27.902332+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'33","last_scrub_stamp":"2024-04-22T20:48:27.902204+0000","last_deep_scrub":"261'33","last_deep_scrub_stamp":"2024-04-22T20:48:27.902204+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.902204+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:57:41.905060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":57,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:18.668593+0000","last_change":"2024-04-22T20:49:18.668649+0000","last_active":"2024-04-22T20:49:18.668593+0000","last_peered":"2024-04-22T20:49:18.668593+0000","last_clean":"2024-04-22T20:49:18.668593+0000","last_became_active":"2024-04-22T20:19:16.793429+0000","last_became_peered":"2024-04-22T20:19:16.793429+0000","last_unstale":"2024-04-22T20:49:18.668593+0000","last_undegraded":"2024-04-22T20:49:18.668593+0000","last_fullsized":"2024-04-22T20:49:18.668593+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:18.668468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:18.668468+0000","last_clean_scrub_stamp":"2024-04-22T20:49:18.668468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:12:31.861365+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"261'2","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.823905+0000","last_change":"2024-04-22T20:49:09.823962+0000","last_active":"2024-04-22T20:49:09.823905+0000","last_peered":"2024-04-22T20:49:09.823905+0000","last_clean":"2024-04-22T20:49:09.823905+0000","last_became_active":"2024-04-22T20:05:17.900874+0000","last_became_peered":"2024-04-22T20:05:17.900874+0000","last_unstale":"2024-04-22T20:49:09.823905+0000","last_undegraded":"2024-04-22T20:49:09.823905+0000","last_fullsized":"2024-04-22T20:49:09.823905+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:09.823786+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:09.823786+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.823786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:18:49.290238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"261'2","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.612637+0000","last_change":"2024-04-22T20:48:26.612637+0000","last_active":"2024-04-22T20:48:26.612637+0000","last_peered":"2024-04-22T20:48:26.612637+0000","last_clean":"2024-04-22T20:48:26.612637+0000","last_became_active":"2024-04-22T20:05:13.813934+0000","last_became_peered":"2024-04-22T20:05:13.813934+0000","last_unstale":"2024-04-22T20:48:26.612637+0000","last_undegraded":"2024-04-22T20:48:26.612637+0000","last_fullsized":"2024-04-22T20:48:26.612637+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:26.612508+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:26.612508+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.612508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:58:57.908355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1024,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"261'37","reported_seq":156,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.240514+0000","last_change":"2024-04-22T20:48:24.240514+0000","last_active":"2024-04-22T20:48:24.240514+0000","last_peered":"2024-04-22T20:48:24.240514+0000","last_clean":"2024-04-22T20:48:24.240514+0000","last_became_active":"2024-04-22T20:01:04.869660+0000","last_became_peered":"2024-04-22T20:01:04.869660+0000","last_unstale":"2024-04-22T20:48:24.240514+0000","last_undegraded":"2024-04-22T20:48:24.240514+0000","last_fullsized":"2024-04-22T20:48:24.240514+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'37","last_scrub_stamp":"2024-04-22T20:48:24.240383+0000","last_deep_scrub":"261'37","last_deep_scrub_stamp":"2024-04-22T20:48:24.240383+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.240383+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:40:22.419745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":113,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:21.775347+0000","last_change":"2024-04-22T20:49:21.775372+0000","last_active":"2024-04-22T20:49:21.775347+0000","last_peered":"2024-04-22T20:49:21.775347+0000","last_clean":"2024-04-22T20:49:21.775347+0000","last_became_active":"2024-04-22T20:19:16.795509+0000","last_became_peered":"2024-04-22T20:19:16.795509+0000","last_unstale":"2024-04-22T20:49:21.775347+0000","last_undegraded":"2024-04-22T20:49:21.775347+0000","last_fullsized":"2024-04-22T20:49:21.775347+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:21.775250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:21.775250+0000","last_clean_scrub_stamp":"2024-04-22T20:49:21.775250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:56:46.381539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.628465+0000","last_change":"2024-04-22T20:49:00.628465+0000","last_active":"2024-04-22T20:49:00.628465+0000","last_peered":"2024-04-22T20:49:00.628465+0000","last_clean":"2024-04-22T20:49:00.628465+0000","last_became_active":"2024-04-22T20:05:17.891523+0000","last_became_peered":"2024-04-22T20:05:17.891523+0000","last_unstale":"2024-04-22T20:49:00.628465+0000","last_undegraded":"2024-04-22T20:49:00.628465+0000","last_fullsized":"2024-04-22T20:49:00.628465+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:00.628331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:00.628331+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.628331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:34:52.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"261'2","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.553971+0000","last_change":"2024-04-22T20:48:24.553971+0000","last_active":"2024-04-22T20:48:24.553971+0000","last_peered":"2024-04-22T20:48:24.553971+0000","last_clean":"2024-04-22T20:48:24.553971+0000","last_became_active":"2024-04-22T20:05:13.805704+0000","last_became_peered":"2024-04-22T20:05:13.805704+0000","last_unstale":"2024-04-22T20:48:24.553971+0000","last_undegraded":"2024-04-22T20:48:24.553971+0000","last_fullsized":"2024-04-22T20:48:24.553971+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:24.553871+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:24.553871+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.553871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:02.108209+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"261'65","reported_seq":234,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.443246+0000","last_change":"2024-04-22T20:48:29.443246+0000","last_active":"2024-04-22T20:48:29.443246+0000","last_peered":"2024-04-22T20:48:29.443246+0000","last_clean":"2024-04-22T20:48:29.443246+0000","last_became_active":"2024-04-22T20:01:04.865981+0000","last_became_peered":"2024-04-22T20:01:04.865981+0000","last_unstale":"2024-04-22T20:48:29.443246+0000","last_undegraded":"2024-04-22T20:48:29.443246+0000","last_fullsized":"2024-04-22T20:48:29.443246+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'65","last_scrub_stamp":"2024-04-22T20:48:29.443124+0000","last_deep_scrub":"261'65","last_deep_scrub_stamp":"2024-04-22T20:48:29.443124+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.443124+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:12:31.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":226,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:20.556690+0000","last_change":"2024-04-22T20:49:20.556729+0000","last_active":"2024-04-22T20:49:20.556690+0000","last_peered":"2024-04-22T20:49:20.556690+0000","last_clean":"2024-04-22T20:49:20.556690+0000","last_became_active":"2024-04-22T20:19:16.799063+0000","last_became_peered":"2024-04-22T20:19:16.799063+0000","last_unstale":"2024-04-22T20:49:20.556690+0000","last_undegraded":"2024-04-22T20:49:20.556690+0000","last_fullsized":"2024-04-22T20:49:20.556690+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:20.556572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:20.556572+0000","last_clean_scrub_stamp":"2024-04-22T20:49:20.556572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:59:47.285896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"261'2","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.247215+0000","last_change":"2024-04-22T20:49:11.247260+0000","last_active":"2024-04-22T20:49:11.247215+0000","last_peered":"2024-04-22T20:49:11.247215+0000","last_clean":"2024-04-22T20:49:11.247215+0000","last_became_active":"2024-04-22T20:05:17.901680+0000","last_became_peered":"2024-04-22T20:05:17.901680+0000","last_unstale":"2024-04-22T20:49:11.247215+0000","last_undegraded":"2024-04-22T20:49:11.247215+0000","last_fullsized":"2024-04-22T20:49:11.247215+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:11.247100+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:11.247100+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.247100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:16:47.754643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":80,"num_write":2,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.618610+0000","last_change":"2024-04-22T20:48:57.618610+0000","last_active":"2024-04-22T20:48:57.618610+0000","last_peered":"2024-04-22T20:48:57.618610+0000","last_clean":"2024-04-22T20:48:57.618610+0000","last_became_active":"2024-04-22T20:05:13.810865+0000","last_became_peered":"2024-04-22T20:05:13.810865+0000","last_unstale":"2024-04-22T20:48:57.618610+0000","last_undegraded":"2024-04-22T20:48:57.618610+0000","last_fullsized":"2024-04-22T20:48:57.618610+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.618439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.618439+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.618439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:51:28.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"261'57","reported_seq":190,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.363498+0000","last_change":"2024-04-22T20:48:35.363498+0000","last_active":"2024-04-22T20:48:35.363498+0000","last_peered":"2024-04-22T20:48:35.363498+0000","last_clean":"2024-04-22T20:48:35.363498+0000","last_became_active":"2024-04-22T20:01:04.870253+0000","last_became_peered":"2024-04-22T20:01:04.870253+0000","last_unstale":"2024-04-22T20:48:35.363498+0000","last_undegraded":"2024-04-22T20:48:35.363498+0000","last_fullsized":"2024-04-22T20:48:35.363498+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'57","last_scrub_stamp":"2024-04-22T20:48:35.363335+0000","last_deep_scrub":"261'57","last_deep_scrub_stamp":"2024-04-22T20:48:35.363335+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.363335+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:44:05.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":145,"num_write":102,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"261'344","reported_seq":961,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.428623+0000","last_change":"2024-04-22T20:48:30.428623+0000","last_active":"2024-04-22T20:48:30.428623+0000","last_peered":"2024-04-22T20:48:30.428623+0000","last_clean":"2024-04-22T20:48:30.428623+0000","last_became_active":"2024-04-22T19:53:04.823351+0000","last_became_peered":"2024-04-22T19:53:04.823351+0000","last_unstale":"2024-04-22T20:48:30.428623+0000","last_undegraded":"2024-04-22T20:48:30.428623+0000","last_fullsized":"2024-04-22T20:48:30.428623+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'344","last_scrub_stamp":"2024-04-22T20:48:30.428472+0000","last_deep_scrub":"261'344","last_deep_scrub_stamp":"2024-04-22T20:48:30.428472+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.428472+0000","objects_scrubbed":5,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:45:58.154599+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":495,"num_write":1298,"num_write_kb":39497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"261'169","reported_seq":773,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.947499+0000","last_change":"2024-04-22T20:48:50.947499+0000","last_active":"2024-04-22T20:48:50.947499+0000","last_peered":"2024-04-22T20:48:50.947499+0000","last_clean":"2024-04-22T20:48:50.947499+0000","last_became_active":"2024-04-22T19:51:28.511595+0000","last_became_peered":"2024-04-22T19:51:28.511595+0000","last_unstale":"2024-04-22T20:48:50.947499+0000","last_undegraded":"2024-04-22T20:48:50.947499+0000","last_fullsized":"2024-04-22T20:48:50.947499+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'169","last_scrub_stamp":"2024-04-22T20:48:50.947377+0000","last_deep_scrub":"261'169","last_deep_scrub_stamp":"2024-04-22T20:48:50.947377+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.947377+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:25:29.288345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":537,"num_write":262,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"261'316","reported_seq":976,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.575341+0000","last_change":"2024-04-22T20:48:51.575445+0000","last_active":"2024-04-22T20:48:51.575341+0000","last_peered":"2024-04-22T20:48:51.575341+0000","last_clean":"2024-04-22T20:48:51.575341+0000","last_became_active":"2024-04-22T19:51:26.496188+0000","last_became_peered":"2024-04-22T19:51:26.496188+0000","last_unstale":"2024-04-22T20:48:51.575341+0000","last_undegraded":"2024-04-22T20:48:51.575341+0000","last_fullsized":"2024-04-22T20:48:51.575341+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'316","last_scrub_stamp":"2024-04-22T20:48:51.575227+0000","last_deep_scrub":"261'316","last_deep_scrub_stamp":"2024-04-22T20:48:51.575227+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.575227+0000","objects_scrubbed":11,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:05:45.518799+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5719773,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":497,"num_read_kb":9758,"num_write":1112,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"261'315","reported_seq":897,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.386962+0000","last_change":"2024-04-22T20:48:53.386962+0000","last_active":"2024-04-22T20:48:53.386962+0000","last_peered":"2024-04-22T20:48:53.386962+0000","last_clean":"2024-04-22T20:48:53.386962+0000","last_became_active":"2024-04-22T19:51:28.519111+0000","last_became_peered":"2024-04-22T19:51:28.519111+0000","last_unstale":"2024-04-22T20:48:53.386962+0000","last_undegraded":"2024-04-22T20:48:53.386962+0000","last_fullsized":"2024-04-22T20:48:53.386962+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'315","last_scrub_stamp":"2024-04-22T20:48:53.386841+0000","last_deep_scrub":"261'315","last_deep_scrub_stamp":"2024-04-22T20:48:53.386841+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.386841+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:13.253246+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1190,"num_read_kb":913,"num_write":510,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"261'320","reported_seq":950,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.733306+0000","last_change":"2024-04-22T20:49:10.733421+0000","last_active":"2024-04-22T20:49:10.733306+0000","last_peered":"2024-04-22T20:49:10.733306+0000","last_clean":"2024-04-22T20:49:10.733306+0000","last_became_active":"2024-04-22T19:51:26.502744+0000","last_became_peered":"2024-04-22T19:51:26.502744+0000","last_unstale":"2024-04-22T20:49:10.733306+0000","last_undegraded":"2024-04-22T20:49:10.733306+0000","last_fullsized":"2024-04-22T20:49:10.733306+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'320","last_scrub_stamp":"2024-04-22T20:49:10.733127+0000","last_deep_scrub":"261'320","last_deep_scrub_stamp":"2024-04-22T20:49:10.733127+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.733127+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:43.010904+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437211,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":443,"num_read_kb":7116,"num_write":1035,"num_write_kb":21496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"261'151","reported_seq":737,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.271385+0000","last_change":"2024-04-22T20:48:48.271385+0000","last_active":"2024-04-22T20:48:48.271385+0000","last_peered":"2024-04-22T20:48:48.271385+0000","last_clean":"2024-04-22T20:48:48.271385+0000","last_became_active":"2024-04-22T19:51:28.518650+0000","last_became_peered":"2024-04-22T19:51:28.518650+0000","last_unstale":"2024-04-22T20:48:48.271385+0000","last_undegraded":"2024-04-22T20:48:48.271385+0000","last_fullsized":"2024-04-22T20:48:48.271385+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'151","last_scrub_stamp":"2024-04-22T20:48:48.271264+0000","last_deep_scrub":"261'151","last_deep_scrub_stamp":"2024-04-22T20:48:48.271264+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.271264+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:14.794941+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":572,"num_read_kb":453,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"261'324","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.858153+0000","last_change":"2024-04-22T20:49:01.858233+0000","last_active":"2024-04-22T20:49:01.858153+0000","last_peered":"2024-04-22T20:49:01.858153+0000","last_clean":"2024-04-22T20:49:01.858153+0000","last_became_active":"2024-04-22T19:51:26.504450+0000","last_became_peered":"2024-04-22T19:51:26.504450+0000","last_unstale":"2024-04-22T20:49:01.858153+0000","last_undegraded":"2024-04-22T20:49:01.858153+0000","last_fullsized":"2024-04-22T20:49:01.858153+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:49:01.858049+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:49:01.858049+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.858049+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:13.565120+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8563195,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":424,"num_read_kb":25508,"num_write":1190,"num_write_kb":37655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"261'81","reported_seq":686,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.546404+0000","last_change":"2024-04-22T20:48:32.546404+0000","last_active":"2024-04-22T20:48:32.546404+0000","last_peered":"2024-04-22T20:48:32.546404+0000","last_clean":"2024-04-22T20:48:32.546404+0000","last_became_active":"2024-04-22T19:52:04.619688+0000","last_became_peered":"2024-04-22T19:52:04.619688+0000","last_unstale":"2024-04-22T20:48:32.546404+0000","last_undegraded":"2024-04-22T20:48:32.546404+0000","last_fullsized":"2024-04-22T20:48:32.546404+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'81","last_scrub_stamp":"2024-04-22T20:48:32.546276+0000","last_deep_scrub":"261'81","last_deep_scrub_stamp":"2024-04-22T20:48:32.546276+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.546276+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:51:45.224120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":353,"num_read_kb":299,"num_write":120,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"261'281","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.591663+0000","last_change":"2024-04-22T20:49:03.591744+0000","last_active":"2024-04-22T20:49:03.591663+0000","last_peered":"2024-04-22T20:49:03.591663+0000","last_clean":"2024-04-22T20:49:03.591663+0000","last_became_active":"2024-04-22T19:51:26.507185+0000","last_became_peered":"2024-04-22T19:51:26.507185+0000","last_unstale":"2024-04-22T20:49:03.591663+0000","last_undegraded":"2024-04-22T20:49:03.591663+0000","last_fullsized":"2024-04-22T20:49:03.591663+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'281","last_scrub_stamp":"2024-04-22T20:49:03.591510+0000","last_deep_scrub":"261'281","last_deep_scrub_stamp":"2024-04-22T20:49:03.591510+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.591510+0000","objects_scrubbed":13,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:35.348173+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3508486,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":442,"num_read_kb":10200,"num_write":925,"num_write_kb":30630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"261'40","reported_seq":603,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.557811+0000","last_change":"2024-04-22T20:48:45.557842+0000","last_active":"2024-04-22T20:48:45.557811+0000","last_peered":"2024-04-22T20:48:45.557811+0000","last_clean":"2024-04-22T20:48:45.557811+0000","last_became_active":"2024-04-22T19:51:28.513693+0000","last_became_peered":"2024-04-22T19:51:28.513693+0000","last_unstale":"2024-04-22T20:48:45.557811+0000","last_undegraded":"2024-04-22T20:48:45.557811+0000","last_fullsized":"2024-04-22T20:48:45.557811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'40","last_scrub_stamp":"2024-04-22T20:48:45.557708+0000","last_deep_scrub":"261'40","last_deep_scrub_stamp":"2024-04-22T20:48:45.557708+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.557708+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:39:08.485749+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"261'444","reported_seq":1196,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.909652+0000","last_change":"2024-04-22T20:48:54.909652+0000","last_active":"2024-04-22T20:48:54.909652+0000","last_peered":"2024-04-22T20:48:54.909652+0000","last_clean":"2024-04-22T20:48:54.909652+0000","last_became_active":"2024-04-22T19:51:26.508716+0000","last_became_peered":"2024-04-22T19:51:26.508716+0000","last_unstale":"2024-04-22T20:48:54.909652+0000","last_undegraded":"2024-04-22T20:48:54.909652+0000","last_fullsized":"2024-04-22T20:48:54.909652+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'444","last_scrub_stamp":"2024-04-22T20:48:54.909489+0000","last_deep_scrub":"261'444","last_deep_scrub_stamp":"2024-04-22T20:48:54.909489+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.909489+0000","objects_scrubbed":4,"log_size":44,"log_dups_size":400,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:47:48.964740+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661200,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1035,"num_read_kb":17244,"num_write":1523,"num_write_kb":25883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"261'145","reported_seq":752,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.279346+0000","last_change":"2024-04-22T20:48:50.279372+0000","last_active":"2024-04-22T20:48:50.279346+0000","last_peered":"2024-04-22T20:48:50.279346+0000","last_clean":"2024-04-22T20:48:50.279346+0000","last_became_active":"2024-04-22T19:51:28.518501+0000","last_became_peered":"2024-04-22T19:51:28.518501+0000","last_unstale":"2024-04-22T20:48:50.279346+0000","last_undegraded":"2024-04-22T20:48:50.279346+0000","last_fullsized":"2024-04-22T20:48:50.279346+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'145","last_scrub_stamp":"2024-04-22T20:48:50.279249+0000","last_deep_scrub":"261'145","last_deep_scrub_stamp":"2024-04-22T20:48:50.279249+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.279249+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:46:54.700622+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":597,"num_read_kb":478,"num_write":226,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"261'377","reported_seq":1048,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.546712+0000","last_change":"2024-04-22T20:49:12.546764+0000","last_active":"2024-04-22T20:49:12.546712+0000","last_peered":"2024-04-22T20:49:12.546712+0000","last_clean":"2024-04-22T20:49:12.546712+0000","last_became_active":"2024-04-22T19:51:26.507141+0000","last_became_peered":"2024-04-22T19:51:26.507141+0000","last_unstale":"2024-04-22T20:49:12.546712+0000","last_undegraded":"2024-04-22T20:49:12.546712+0000","last_fullsized":"2024-04-22T20:49:12.546712+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'377","last_scrub_stamp":"2024-04-22T20:49:12.546615+0000","last_deep_scrub":"261'377","last_deep_scrub_stamp":"2024-04-22T20:49:12.546615+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.546615+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:20:29.531515+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2367620,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":550,"num_read_kb":43743,"num_write":1389,"num_write_kb":32515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"261'152","reported_seq":746,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.990255+0000","last_change":"2024-04-22T20:48:43.990284+0000","last_active":"2024-04-22T20:48:43.990255+0000","last_peered":"2024-04-22T20:48:43.990255+0000","last_clean":"2024-04-22T20:48:43.990255+0000","last_became_active":"2024-04-22T19:51:28.516012+0000","last_became_peered":"2024-04-22T19:51:28.516012+0000","last_unstale":"2024-04-22T20:48:43.990255+0000","last_undegraded":"2024-04-22T20:48:43.990255+0000","last_fullsized":"2024-04-22T20:48:43.990255+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:43.990165+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:43.990165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.990165+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:23.460100+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":470,"num_write":219,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7560,"num_omap_keys":32,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"261'333","reported_seq":987,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:17.330968+0000","last_change":"2024-04-22T20:49:17.331060+0000","last_active":"2024-04-22T20:49:17.330968+0000","last_peered":"2024-04-22T20:49:17.330968+0000","last_clean":"2024-04-22T20:49:17.330968+0000","last_became_active":"2024-04-22T19:51:26.502928+0000","last_became_peered":"2024-04-22T19:51:26.502928+0000","last_unstale":"2024-04-22T20:49:17.330968+0000","last_undegraded":"2024-04-22T20:49:17.330968+0000","last_fullsized":"2024-04-22T20:49:17.330968+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'333","last_scrub_stamp":"2024-04-22T20:49:17.330844+0000","last_deep_scrub":"261'333","last_deep_scrub_stamp":"2024-04-22T20:49:17.330844+0000","last_clean_scrub_stamp":"2024-04-22T20:49:17.330844+0000","objects_scrubbed":8,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:54.551981+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5414404,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":443,"num_read_kb":51849,"num_write":1202,"num_write_kb":39313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"261'60","reported_seq":638,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.590400+0000","last_change":"2024-04-22T20:48:40.590452+0000","last_active":"2024-04-22T20:48:40.590400+0000","last_peered":"2024-04-22T20:48:40.590400+0000","last_clean":"2024-04-22T20:48:40.590400+0000","last_became_active":"2024-04-22T19:51:28.509774+0000","last_became_peered":"2024-04-22T19:51:28.509774+0000","last_unstale":"2024-04-22T20:48:40.590400+0000","last_undegraded":"2024-04-22T20:48:40.590400+0000","last_fullsized":"2024-04-22T20:48:40.590400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'60","last_scrub_stamp":"2024-04-22T20:48:40.590291+0000","last_deep_scrub":"261'60","last_deep_scrub_stamp":"2024-04-22T20:48:40.590291+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.590291+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:30:33.030111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":211,"num_write":81,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"261'126","reported_seq":744,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.246253+0000","last_change":"2024-04-22T20:49:03.246307+0000","last_active":"2024-04-22T20:49:03.246253+0000","last_peered":"2024-04-22T20:49:03.246253+0000","last_clean":"2024-04-22T20:49:03.246253+0000","last_became_active":"2024-04-22T19:51:28.505562+0000","last_became_peered":"2024-04-22T19:51:28.505562+0000","last_unstale":"2024-04-22T20:49:03.246253+0000","last_undegraded":"2024-04-22T20:49:03.246253+0000","last_fullsized":"2024-04-22T20:49:03.246253+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'126","last_scrub_stamp":"2024-04-22T20:49:03.246160+0000","last_deep_scrub":"261'126","last_deep_scrub_stamp":"2024-04-22T20:49:03.246160+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.246160+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:08.512189+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":540,"num_read_kb":443,"num_write":193,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"261'307","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.575172+0000","last_change":"2024-04-22T20:49:05.575247+0000","last_active":"2024-04-22T20:49:05.575172+0000","last_peered":"2024-04-22T20:49:05.575172+0000","last_clean":"2024-04-22T20:49:05.575172+0000","last_became_active":"2024-04-22T19:51:26.504717+0000","last_became_peered":"2024-04-22T19:51:26.504717+0000","last_unstale":"2024-04-22T20:49:05.575172+0000","last_undegraded":"2024-04-22T20:49:05.575172+0000","last_fullsized":"2024-04-22T20:49:05.575172+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'307","last_scrub_stamp":"2024-04-22T20:49:05.575076+0000","last_deep_scrub":"261'307","last_deep_scrub_stamp":"2024-04-22T20:49:05.575076+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.575076+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:55:01.561551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10266,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":28705,"num_write":1110,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"261'105","reported_seq":707,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.395682+0000","last_change":"2024-04-22T20:48:51.395709+0000","last_active":"2024-04-22T20:48:51.395682+0000","last_peered":"2024-04-22T20:48:51.395682+0000","last_clean":"2024-04-22T20:48:51.395682+0000","last_became_active":"2024-04-22T19:51:28.519230+0000","last_became_peered":"2024-04-22T19:51:28.519230+0000","last_unstale":"2024-04-22T20:48:51.395682+0000","last_undegraded":"2024-04-22T20:48:51.395682+0000","last_fullsized":"2024-04-22T20:48:51.395682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'105","last_scrub_stamp":"2024-04-22T20:48:51.395575+0000","last_deep_scrub":"261'105","last_deep_scrub_stamp":"2024-04-22T20:48:51.395575+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.395575+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:48:07.398102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":465,"num_read_kb":383,"num_write":180,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"261'381","reported_seq":1047,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.558613+0000","last_change":"2024-04-22T20:48:50.558671+0000","last_active":"2024-04-22T20:48:50.558613+0000","last_peered":"2024-04-22T20:48:50.558613+0000","last_clean":"2024-04-22T20:48:50.558613+0000","last_became_active":"2024-04-22T19:51:26.496156+0000","last_became_peered":"2024-04-22T19:51:26.496156+0000","last_unstale":"2024-04-22T20:48:50.558613+0000","last_undegraded":"2024-04-22T20:48:50.558613+0000","last_fullsized":"2024-04-22T20:48:50.558613+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'381","last_scrub_stamp":"2024-04-22T20:48:50.558509+0000","last_deep_scrub":"261'381","last_deep_scrub_stamp":"2024-04-22T20:48:50.558509+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.558509+0000","objects_scrubbed":6,"log_size":81,"log_dups_size":300,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:54:45.615667+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":611,"num_read_kb":792,"num_write":1516,"num_write_kb":25865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"261'172","reported_seq":773,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.926538+0000","last_change":"2024-04-22T20:48:41.926573+0000","last_active":"2024-04-22T20:48:41.926538+0000","last_peered":"2024-04-22T20:48:41.926538+0000","last_clean":"2024-04-22T20:48:41.926538+0000","last_became_active":"2024-04-22T19:51:28.515673+0000","last_became_peered":"2024-04-22T19:51:28.515673+0000","last_unstale":"2024-04-22T20:48:41.926538+0000","last_undegraded":"2024-04-22T20:48:41.926538+0000","last_fullsized":"2024-04-22T20:48:41.926538+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'172","last_scrub_stamp":"2024-04-22T20:48:41.926446+0000","last_deep_scrub":"261'172","last_deep_scrub_stamp":"2024-04-22T20:48:41.926446+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.926446+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:29:59.234763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":666,"num_read_kb":529,"num_write":258,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"261'360","reported_seq":1079,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.586412+0000","last_change":"2024-04-22T20:48:54.586516+0000","last_active":"2024-04-22T20:48:54.586412+0000","last_peered":"2024-04-22T20:48:54.586412+0000","last_clean":"2024-04-22T20:48:54.586412+0000","last_became_active":"2024-04-22T19:51:26.502156+0000","last_became_peered":"2024-04-22T19:51:26.502156+0000","last_unstale":"2024-04-22T20:48:54.586412+0000","last_undegraded":"2024-04-22T20:48:54.586412+0000","last_fullsized":"2024-04-22T20:48:54.586412+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'360","last_scrub_stamp":"2024-04-22T20:48:54.586311+0000","last_deep_scrub":"261'360","last_deep_scrub_stamp":"2024-04-22T20:48:54.586311+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.586311+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:01.200779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1373470,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":619,"num_read_kb":68133,"num_write":1312,"num_write_kb":41095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"261'290","reported_seq":905,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.550263+0000","last_change":"2024-04-22T20:48:41.550305+0000","last_active":"2024-04-22T20:48:41.550263+0000","last_peered":"2024-04-22T20:48:41.550263+0000","last_clean":"2024-04-22T20:48:41.550263+0000","last_became_active":"2024-04-22T19:51:28.510415+0000","last_became_peered":"2024-04-22T19:51:28.510415+0000","last_unstale":"2024-04-22T20:48:41.550263+0000","last_undegraded":"2024-04-22T20:48:41.550263+0000","last_fullsized":"2024-04-22T20:48:41.550263+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'290","last_scrub_stamp":"2024-04-22T20:48:41.550169+0000","last_deep_scrub":"261'290","last_deep_scrub_stamp":"2024-04-22T20:48:41.550169+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.550169+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:30:48.444121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1119,"num_read_kb":869,"num_write":475,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"261'320","reported_seq":974,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.247648+0000","last_change":"2024-04-22T20:49:02.247714+0000","last_active":"2024-04-22T20:49:02.247648+0000","last_peered":"2024-04-22T20:49:02.247648+0000","last_clean":"2024-04-22T20:49:02.247648+0000","last_became_active":"2024-04-22T19:51:26.506671+0000","last_became_peered":"2024-04-22T19:51:26.506671+0000","last_unstale":"2024-04-22T20:49:02.247648+0000","last_undegraded":"2024-04-22T20:49:02.247648+0000","last_fullsized":"2024-04-22T20:49:02.247648+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'320","last_scrub_stamp":"2024-04-22T20:49:02.247508+0000","last_deep_scrub":"261'320","last_deep_scrub_stamp":"2024-04-22T20:49:02.247508+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.247508+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:15.451052+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194361,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":6535,"num_write":1103,"num_write_kb":36003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"261'157","reported_seq":786,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.600988+0000","last_change":"2024-04-22T20:48:42.601043+0000","last_active":"2024-04-22T20:48:42.600988+0000","last_peered":"2024-04-22T20:48:42.600988+0000","last_clean":"2024-04-22T20:48:42.600988+0000","last_became_active":"2024-04-22T19:51:28.511889+0000","last_became_peered":"2024-04-22T19:51:28.511889+0000","last_unstale":"2024-04-22T20:48:42.600988+0000","last_undegraded":"2024-04-22T20:48:42.600988+0000","last_fullsized":"2024-04-22T20:48:42.600988+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'157","last_scrub_stamp":"2024-04-22T20:48:42.600881+0000","last_deep_scrub":"261'157","last_deep_scrub_stamp":"2024-04-22T20:48:42.600881+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.600881+0000","objects_scrubbed":2,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:30.539100+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":679,"num_read_kb":547,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3316,"num_omap_keys":19,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"261'334","reported_seq":990,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.339896+0000","last_change":"2024-04-22T20:49:12.339976+0000","last_active":"2024-04-22T20:49:12.339896+0000","last_peered":"2024-04-22T20:49:12.339896+0000","last_clean":"2024-04-22T20:49:12.339896+0000","last_became_active":"2024-04-22T19:51:26.497718+0000","last_became_peered":"2024-04-22T19:51:26.497718+0000","last_unstale":"2024-04-22T20:49:12.339896+0000","last_undegraded":"2024-04-22T20:49:12.339896+0000","last_fullsized":"2024-04-22T20:49:12.339896+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'334","last_scrub_stamp":"2024-04-22T20:49:12.339764+0000","last_deep_scrub":"261'334","last_deep_scrub_stamp":"2024-04-22T20:49:12.339764+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.339764+0000","objects_scrubbed":10,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:51.305007+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050340,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":471,"num_read_kb":30112,"num_write":1163,"num_write_kb":27210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"261'94","reported_seq":709,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.252306+0000","last_change":"2024-04-22T20:49:04.252334+0000","last_active":"2024-04-22T20:49:04.252306+0000","last_peered":"2024-04-22T20:49:04.252306+0000","last_clean":"2024-04-22T20:49:04.252306+0000","last_became_active":"2024-04-22T19:51:28.507673+0000","last_became_peered":"2024-04-22T19:51:28.507673+0000","last_unstale":"2024-04-22T20:49:04.252306+0000","last_undegraded":"2024-04-22T20:49:04.252306+0000","last_fullsized":"2024-04-22T20:49:04.252306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'94","last_scrub_stamp":"2024-04-22T20:49:04.252218+0000","last_deep_scrub":"261'94","last_deep_scrub_stamp":"2024-04-22T20:49:04.252218+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.252218+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:55:27.370264+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":404,"num_read_kb":350,"num_write":133,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"261'413","reported_seq":1133,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:13.703814+0000","last_change":"2024-04-22T20:49:13.703918+0000","last_active":"2024-04-22T20:49:13.703814+0000","last_peered":"2024-04-22T20:49:13.703814+0000","last_clean":"2024-04-22T20:49:13.703814+0000","last_became_active":"2024-04-22T19:51:26.511213+0000","last_became_peered":"2024-04-22T19:51:26.511213+0000","last_unstale":"2024-04-22T20:49:13.703814+0000","last_undegraded":"2024-04-22T20:49:13.703814+0000","last_fullsized":"2024-04-22T20:49:13.703814+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'413","last_scrub_stamp":"2024-04-22T20:49:13.703681+0000","last_deep_scrub":"261'413","last_deep_scrub_stamp":"2024-04-22T20:49:13.703681+0000","last_clean_scrub_stamp":"2024-04-22T20:49:13.703681+0000","objects_scrubbed":8,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:02:28.872480+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306877,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":730,"num_read_kb":36415,"num_write":1798,"num_write_kb":40524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"261'129","reported_seq":659,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.351652+0000","last_change":"2024-04-22T20:48:29.351716+0000","last_active":"2024-04-22T20:48:29.351652+0000","last_peered":"2024-04-22T20:48:29.351652+0000","last_clean":"2024-04-22T20:48:29.351652+0000","last_became_active":"2024-04-22T19:51:36.567561+0000","last_became_peered":"2024-04-22T19:51:36.567561+0000","last_unstale":"2024-04-22T20:48:29.351652+0000","last_undegraded":"2024-04-22T20:48:29.351652+0000","last_fullsized":"2024-04-22T20:48:29.351652+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'129","last_scrub_stamp":"2024-04-22T20:48:29.351556+0000","last_deep_scrub":"261'129","last_deep_scrub_stamp":"2024-04-22T20:48:29.351556+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.351556+0000","objects_scrubbed":4,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:20:31.079680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":168,"num_write":307,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"261'38","reported_seq":593,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.462844+0000","last_change":"2024-04-22T20:48:58.462869+0000","last_active":"2024-04-22T20:48:58.462844+0000","last_peered":"2024-04-22T20:48:58.462844+0000","last_clean":"2024-04-22T20:48:58.462844+0000","last_became_active":"2024-04-22T19:51:28.519994+0000","last_became_peered":"2024-04-22T19:51:28.519994+0000","last_unstale":"2024-04-22T20:48:58.462844+0000","last_undegraded":"2024-04-22T20:48:58.462844+0000","last_fullsized":"2024-04-22T20:48:58.462844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'38","last_scrub_stamp":"2024-04-22T20:48:58.462753+0000","last_deep_scrub":"261'38","last_deep_scrub_stamp":"2024-04-22T20:48:58.462753+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.462753+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:57:15.046577+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":138,"num_write":52,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"261'317","reported_seq":1059,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.402857+0000","last_change":"2024-04-22T20:48:38.402920+0000","last_active":"2024-04-22T20:48:38.402857+0000","last_peered":"2024-04-22T20:48:38.402857+0000","last_clean":"2024-04-22T20:48:38.402857+0000","last_became_active":"2024-04-22T19:53:05.051272+0000","last_became_peered":"2024-04-22T19:53:05.051272+0000","last_unstale":"2024-04-22T20:48:38.402857+0000","last_undegraded":"2024-04-22T20:48:38.402857+0000","last_fullsized":"2024-04-22T20:48:38.402857+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:38.402757+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:38.402757+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.402757+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:31:08.678074+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263217,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":662,"num_read_kb":11069,"num_write":1270,"num_write_kb":33730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.985630+0000","last_change":"2024-04-22T20:48:45.985652+0000","last_active":"2024-04-22T20:48:45.985630+0000","last_peered":"2024-04-22T20:48:45.985630+0000","last_clean":"2024-04-22T20:48:45.985630+0000","last_became_active":"2024-04-22T19:51:30.520848+0000","last_became_peered":"2024-04-22T19:51:30.520848+0000","last_unstale":"2024-04-22T20:48:45.985630+0000","last_undegraded":"2024-04-22T20:48:45.985630+0000","last_fullsized":"2024-04-22T20:48:45.985630+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.985531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.985531+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.985531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:16:52.817893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"261'334","reported_seq":947,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.658874+0000","last_change":"2024-04-22T20:49:01.658905+0000","last_active":"2024-04-22T20:49:01.658874+0000","last_peered":"2024-04-22T20:49:01.658874+0000","last_clean":"2024-04-22T20:49:01.658874+0000","last_became_active":"2024-04-22T19:51:32.537695+0000","last_became_peered":"2024-04-22T19:51:32.537695+0000","last_unstale":"2024-04-22T20:49:01.658874+0000","last_undegraded":"2024-04-22T20:49:01.658874+0000","last_fullsized":"2024-04-22T20:49:01.658874+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'334","last_scrub_stamp":"2024-04-22T20:49:01.658785+0000","last_deep_scrub":"261'334","last_deep_scrub_stamp":"2024-04-22T20:49:01.658785+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.658785+0000","objects_scrubbed":5,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:50:34.689896+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39872,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":550,"num_read_kb":1178,"num_write":438,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.404496+0000","last_change":"2024-04-22T20:48:45.404533+0000","last_active":"2024-04-22T20:48:45.404496+0000","last_peered":"2024-04-22T20:48:45.404496+0000","last_clean":"2024-04-22T20:48:45.404496+0000","last_became_active":"2024-04-22T19:51:34.557285+0000","last_became_peered":"2024-04-22T19:51:34.557285+0000","last_unstale":"2024-04-22T20:48:45.404496+0000","last_undegraded":"2024-04-22T20:48:45.404496+0000","last_fullsized":"2024-04-22T20:48:45.404496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.404404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.404404+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.404404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:14:28.347705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"261'2405","reported_seq":4273,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.005951+0000","last_change":"2024-04-22T20:48:34.006025+0000","last_active":"2024-04-22T20:48:34.005951+0000","last_peered":"2024-04-22T20:48:34.005951+0000","last_clean":"2024-04-22T20:48:34.005951+0000","last_became_active":"2024-04-22T19:51:36.567195+0000","last_became_peered":"2024-04-22T19:51:36.567195+0000","last_unstale":"2024-04-22T20:48:34.005951+0000","last_undegraded":"2024-04-22T20:48:34.005951+0000","last_fullsized":"2024-04-22T20:48:34.005951+0000","mapping_epoch":30,"log_start":"261'2300","ondisk_log_start":"261'2300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2405","last_scrub_stamp":"2024-04-22T20:48:34.005850+0000","last_deep_scrub":"261'2405","last_deep_scrub_stamp":"2024-04-22T20:48:34.005850+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.005850+0000","objects_scrubbed":6,"log_size":105,"log_dups_size":2300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:06.221017+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5986,"num_read_kb":5990,"num_write":4781,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"261'131","reported_seq":730,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.955621+0000","last_change":"2024-04-22T20:48:48.955649+0000","last_active":"2024-04-22T20:48:48.955621+0000","last_peered":"2024-04-22T20:48:48.955621+0000","last_clean":"2024-04-22T20:48:48.955621+0000","last_became_active":"2024-04-22T19:51:28.512786+0000","last_became_peered":"2024-04-22T19:51:28.512786+0000","last_unstale":"2024-04-22T20:48:48.955621+0000","last_undegraded":"2024-04-22T20:48:48.955621+0000","last_fullsized":"2024-04-22T20:48:48.955621+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:48.955527+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:48.955527+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.955527+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:47:43.304224+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":408,"num_write":191,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"261'290","reported_seq":902,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.689230+0000","last_change":"2024-04-22T20:49:08.689283+0000","last_active":"2024-04-22T20:49:08.689230+0000","last_peered":"2024-04-22T20:49:08.689230+0000","last_clean":"2024-04-22T20:49:08.689230+0000","last_became_active":"2024-04-22T19:51:26.499885+0000","last_became_peered":"2024-04-22T19:51:26.499885+0000","last_unstale":"2024-04-22T20:49:08.689230+0000","last_undegraded":"2024-04-22T20:49:08.689230+0000","last_fullsized":"2024-04-22T20:49:08.689230+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'290","last_scrub_stamp":"2024-04-22T20:49:08.689129+0000","last_deep_scrub":"261'290","last_deep_scrub_stamp":"2024-04-22T20:49:08.689129+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.689129+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:09.853456+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050193,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":16305,"num_write":962,"num_write_kb":23193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.330826+0000","last_change":"2024-04-22T20:49:06.330846+0000","last_active":"2024-04-22T20:49:06.330826+0000","last_peered":"2024-04-22T20:49:06.330826+0000","last_clean":"2024-04-22T20:49:06.330826+0000","last_became_active":"2024-04-22T19:51:30.526622+0000","last_became_peered":"2024-04-22T19:51:30.526622+0000","last_unstale":"2024-04-22T20:49:06.330826+0000","last_undegraded":"2024-04-22T20:49:06.330826+0000","last_fullsized":"2024-04-22T20:49:06.330826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:06.330744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:06.330744+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.330744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:58:15.948210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"261'89","reported_seq":623,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.323750+0000","last_change":"2024-04-22T20:48:39.323782+0000","last_active":"2024-04-22T20:48:39.323750+0000","last_peered":"2024-04-22T20:48:39.323750+0000","last_clean":"2024-04-22T20:48:39.323750+0000","last_became_active":"2024-04-22T19:51:32.536180+0000","last_became_peered":"2024-04-22T19:51:32.536180+0000","last_unstale":"2024-04-22T20:48:39.323750+0000","last_undegraded":"2024-04-22T20:48:39.323750+0000","last_fullsized":"2024-04-22T20:48:39.323750+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'89","last_scrub_stamp":"2024-04-22T20:48:39.323672+0000","last_deep_scrub":"261'89","last_deep_scrub_stamp":"2024-04-22T20:48:39.323672+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.323672+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:12:25.808631+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":95,"num_write":64,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.367492+0000","last_change":"2024-04-22T20:48:34.367530+0000","last_active":"2024-04-22T20:48:34.367492+0000","last_peered":"2024-04-22T20:48:34.367492+0000","last_clean":"2024-04-22T20:48:34.367492+0000","last_became_active":"2024-04-22T19:51:34.549551+0000","last_became_peered":"2024-04-22T19:51:34.549551+0000","last_unstale":"2024-04-22T20:48:34.367492+0000","last_undegraded":"2024-04-22T20:48:34.367492+0000","last_fullsized":"2024-04-22T20:48:34.367492+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.367401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.367401+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.367401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:41.545716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"261'151","reported_seq":660,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.539676+0000","last_change":"2024-04-22T20:48:38.539676+0000","last_active":"2024-04-22T20:48:38.539676+0000","last_peered":"2024-04-22T20:48:38.539676+0000","last_clean":"2024-04-22T20:48:38.539676+0000","last_became_active":"2024-04-22T19:51:36.573189+0000","last_became_peered":"2024-04-22T19:51:36.573189+0000","last_unstale":"2024-04-22T20:48:38.539676+0000","last_undegraded":"2024-04-22T20:48:38.539676+0000","last_fullsized":"2024-04-22T20:48:38.539676+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'151","last_scrub_stamp":"2024-04-22T20:48:38.539539+0000","last_deep_scrub":"261'151","last_deep_scrub_stamp":"2024-04-22T20:48:38.539539+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.539539+0000","objects_scrubbed":5,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:40:24.145792+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":328,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"261'212","reported_seq":846,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.682759+0000","last_change":"2024-04-22T20:48:58.682785+0000","last_active":"2024-04-22T20:48:58.682759+0000","last_peered":"2024-04-22T20:48:58.682759+0000","last_clean":"2024-04-22T20:48:58.682759+0000","last_became_active":"2024-04-22T19:51:28.517737+0000","last_became_peered":"2024-04-22T19:51:28.517737+0000","last_unstale":"2024-04-22T20:48:58.682759+0000","last_undegraded":"2024-04-22T20:48:58.682759+0000","last_fullsized":"2024-04-22T20:48:58.682759+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'212","last_scrub_stamp":"2024-04-22T20:48:58.682670+0000","last_deep_scrub":"261'212","last_deep_scrub_stamp":"2024-04-22T20:48:58.682670+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.682670+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:37:06.432832+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":871,"num_read_kb":733,"num_write":338,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"261'298","reported_seq":951,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.509069+0000","last_change":"2024-04-22T20:49:01.509118+0000","last_active":"2024-04-22T20:49:01.509069+0000","last_peered":"2024-04-22T20:49:01.509069+0000","last_clean":"2024-04-22T20:49:01.509069+0000","last_became_active":"2024-04-22T19:51:26.508933+0000","last_became_peered":"2024-04-22T19:51:26.508933+0000","last_unstale":"2024-04-22T20:49:01.509069+0000","last_undegraded":"2024-04-22T20:49:01.509069+0000","last_fullsized":"2024-04-22T20:49:01.509069+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'298","last_scrub_stamp":"2024-04-22T20:49:01.508963+0000","last_deep_scrub":"261'298","last_deep_scrub_stamp":"2024-04-22T20:49:01.508963+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.508963+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:48:40.562892+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632139,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":513,"num_read_kb":15262,"num_write":1122,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":511,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.319343+0000","last_change":"2024-04-22T20:49:08.319368+0000","last_active":"2024-04-22T20:49:08.319343+0000","last_peered":"2024-04-22T20:49:08.319343+0000","last_clean":"2024-04-22T20:49:08.319343+0000","last_became_active":"2024-04-22T19:51:30.527712+0000","last_became_peered":"2024-04-22T19:51:30.527712+0000","last_unstale":"2024-04-22T20:49:08.319343+0000","last_undegraded":"2024-04-22T20:49:08.319343+0000","last_fullsized":"2024-04-22T20:49:08.319343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:08.319242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:08.319242+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.319242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:55:18.192677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"261'118","reported_seq":671,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.458436+0000","last_change":"2024-04-22T20:48:29.458484+0000","last_active":"2024-04-22T20:48:29.458436+0000","last_peered":"2024-04-22T20:48:29.458436+0000","last_clean":"2024-04-22T20:48:29.458436+0000","last_became_active":"2024-04-22T19:51:32.536177+0000","last_became_peered":"2024-04-22T19:51:32.536177+0000","last_unstale":"2024-04-22T20:48:29.458436+0000","last_undegraded":"2024-04-22T20:48:29.458436+0000","last_fullsized":"2024-04-22T20:48:29.458436+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'118","last_scrub_stamp":"2024-04-22T20:48:29.458333+0000","last_deep_scrub":"261'118","last_deep_scrub_stamp":"2024-04-22T20:48:29.458333+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.458333+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:20:42.045716+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"261'523","reported_seq":2092,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.422107+0000","last_change":"2024-04-22T20:48:43.422135+0000","last_active":"2024-04-22T20:48:43.422107+0000","last_peered":"2024-04-22T20:48:43.422107+0000","last_clean":"2024-04-22T20:48:43.422107+0000","last_became_active":"2024-04-22T19:51:34.561828+0000","last_became_peered":"2024-04-22T19:51:34.561828+0000","last_unstale":"2024-04-22T20:48:43.422107+0000","last_undegraded":"2024-04-22T20:48:43.422107+0000","last_fullsized":"2024-04-22T20:48:43.422107+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'523","last_scrub_stamp":"2024-04-22T20:48:43.422006+0000","last_deep_scrub":"261'523","last_deep_scrub_stamp":"2024-04-22T20:48:43.422006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.422006+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":500,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:09:15.020119+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"261'176","reported_seq":998,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.374062+0000","last_change":"2024-04-22T20:48:30.374062+0000","last_active":"2024-04-22T20:48:30.374062+0000","last_peered":"2024-04-22T20:48:30.374062+0000","last_clean":"2024-04-22T20:48:30.374062+0000","last_became_active":"2024-04-22T19:51:36.565673+0000","last_became_peered":"2024-04-22T19:51:36.565673+0000","last_unstale":"2024-04-22T20:48:30.374062+0000","last_undegraded":"2024-04-22T20:48:30.374062+0000","last_fullsized":"2024-04-22T20:48:30.374062+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'176","last_scrub_stamp":"2024-04-22T20:48:30.373943+0000","last_deep_scrub":"261'176","last_deep_scrub_stamp":"2024-04-22T20:48:30.373943+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.373943+0000","objects_scrubbed":2,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:19.241756+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":579,"num_read_kb":567,"num_write":371,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"261'176","reported_seq":823,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.454747+0000","last_change":"2024-04-22T20:48:54.454777+0000","last_active":"2024-04-22T20:48:54.454747+0000","last_peered":"2024-04-22T20:48:54.454747+0000","last_clean":"2024-04-22T20:48:54.454747+0000","last_became_active":"2024-04-22T19:51:28.515670+0000","last_became_peered":"2024-04-22T19:51:28.515670+0000","last_unstale":"2024-04-22T20:48:54.454747+0000","last_undegraded":"2024-04-22T20:48:54.454747+0000","last_fullsized":"2024-04-22T20:48:54.454747+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'176","last_scrub_stamp":"2024-04-22T20:48:54.454610+0000","last_deep_scrub":"261'176","last_deep_scrub_stamp":"2024-04-22T20:48:54.454610+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.454610+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:57:21.493070+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":729,"num_read_kb":590,"num_write":271,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"261'317","reported_seq":1004,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.627131+0000","last_change":"2024-04-22T20:48:47.627177+0000","last_active":"2024-04-22T20:48:47.627131+0000","last_peered":"2024-04-22T20:48:47.627131+0000","last_clean":"2024-04-22T20:48:47.627131+0000","last_became_active":"2024-04-22T19:51:26.503438+0000","last_became_peered":"2024-04-22T19:51:26.503438+0000","last_unstale":"2024-04-22T20:48:47.627131+0000","last_undegraded":"2024-04-22T20:48:47.627131+0000","last_fullsized":"2024-04-22T20:48:47.627131+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:47.627031+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:47.627031+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.627031+0000","objects_scrubbed":7,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:03:27.258395+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3601437,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":581,"num_read_kb":32833,"num_write":1303,"num_write_kb":25399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.961966+0000","last_change":"2024-04-22T20:48:47.962021+0000","last_active":"2024-04-22T20:48:47.961966+0000","last_peered":"2024-04-22T20:48:47.961966+0000","last_clean":"2024-04-22T20:48:47.961966+0000","last_became_active":"2024-04-22T19:51:30.523125+0000","last_became_peered":"2024-04-22T19:51:30.523125+0000","last_unstale":"2024-04-22T20:48:47.961966+0000","last_undegraded":"2024-04-22T20:48:47.961966+0000","last_fullsized":"2024-04-22T20:48:47.961966+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.961820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.961820+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.961820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:36:15.603182+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"261'257","reported_seq":836,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.599089+0000","last_change":"2024-04-22T20:49:04.599129+0000","last_active":"2024-04-22T20:49:04.599089+0000","last_peered":"2024-04-22T20:49:04.599089+0000","last_clean":"2024-04-22T20:49:04.599089+0000","last_became_active":"2024-04-22T19:51:33.248160+0000","last_became_peered":"2024-04-22T19:51:33.248160+0000","last_unstale":"2024-04-22T20:49:04.599089+0000","last_undegraded":"2024-04-22T20:49:04.599089+0000","last_fullsized":"2024-04-22T20:49:04.599089+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'257","last_scrub_stamp":"2024-04-22T20:49:04.598996+0000","last_deep_scrub":"261'257","last_deep_scrub_stamp":"2024-04-22T20:49:04.598996+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.598996+0000","objects_scrubbed":5,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:45:50.161224+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":425,"num_read_kb":974,"num_write":344,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"261'536","reported_seq":4238,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.560508+0000","last_change":"2024-04-22T20:48:43.560537+0000","last_active":"2024-04-22T20:48:43.560508+0000","last_peered":"2024-04-22T20:48:43.560508+0000","last_clean":"2024-04-22T20:48:43.560508+0000","last_became_active":"2024-04-22T19:51:34.555720+0000","last_became_peered":"2024-04-22T19:51:34.555720+0000","last_unstale":"2024-04-22T20:48:43.560508+0000","last_undegraded":"2024-04-22T20:48:43.560508+0000","last_fullsized":"2024-04-22T20:48:43.560508+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'536","last_scrub_stamp":"2024-04-22T20:48:43.560412+0000","last_deep_scrub":"261'536","last_deep_scrub_stamp":"2024-04-22T20:48:43.560412+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.560412+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:28.072035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"261'385","reported_seq":1066,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.949946+0000","last_change":"2024-04-22T20:48:46.949973+0000","last_active":"2024-04-22T20:48:46.949946+0000","last_peered":"2024-04-22T20:48:46.949946+0000","last_clean":"2024-04-22T20:48:46.949946+0000","last_became_active":"2024-04-22T19:51:28.512125+0000","last_became_peered":"2024-04-22T19:51:28.512125+0000","last_unstale":"2024-04-22T20:48:46.949946+0000","last_undegraded":"2024-04-22T20:48:46.949946+0000","last_fullsized":"2024-04-22T20:48:46.949946+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'385","last_scrub_stamp":"2024-04-22T20:48:46.949859+0000","last_deep_scrub":"261'385","last_deep_scrub_stamp":"2024-04-22T20:48:46.949859+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.949859+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:49.034926+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1728,"num_read_kb":1430,"num_write":663,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"261'305","reported_seq":956,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:13.548151+0000","last_change":"2024-04-22T20:49:13.548245+0000","last_active":"2024-04-22T20:49:13.548151+0000","last_peered":"2024-04-22T20:49:13.548151+0000","last_clean":"2024-04-22T20:49:13.548151+0000","last_became_active":"2024-04-22T19:51:26.504654+0000","last_became_peered":"2024-04-22T19:51:26.504654+0000","last_unstale":"2024-04-22T20:49:13.548151+0000","last_undegraded":"2024-04-22T20:49:13.548151+0000","last_fullsized":"2024-04-22T20:49:13.548151+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'305","last_scrub_stamp":"2024-04-22T20:49:13.548046+0000","last_deep_scrub":"261'305","last_deep_scrub_stamp":"2024-04-22T20:49:13.548046+0000","last_clean_scrub_stamp":"2024-04-22T20:49:13.548046+0000","objects_scrubbed":10,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:58.481055+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10487408,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":445,"num_read_kb":80529,"num_write":1013,"num_write_kb":26179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.400985+0000","last_change":"2024-04-22T20:48:50.401013+0000","last_active":"2024-04-22T20:48:50.400985+0000","last_peered":"2024-04-22T20:48:50.400985+0000","last_clean":"2024-04-22T20:48:50.400985+0000","last_became_active":"2024-04-22T19:51:30.526306+0000","last_became_peered":"2024-04-22T19:51:30.526306+0000","last_unstale":"2024-04-22T20:48:50.400985+0000","last_undegraded":"2024-04-22T20:48:50.400985+0000","last_fullsized":"2024-04-22T20:48:50.400985+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.400891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.400891+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.400891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:05:17.361036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"261'342","reported_seq":966,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.959517+0000","last_change":"2024-04-22T20:48:38.959567+0000","last_active":"2024-04-22T20:48:38.959517+0000","last_peered":"2024-04-22T20:48:38.959517+0000","last_clean":"2024-04-22T20:48:38.959517+0000","last_became_active":"2024-04-22T19:51:33.248059+0000","last_became_peered":"2024-04-22T19:51:33.248059+0000","last_unstale":"2024-04-22T20:48:38.959517+0000","last_undegraded":"2024-04-22T20:48:38.959517+0000","last_fullsized":"2024-04-22T20:48:38.959517+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'342","last_scrub_stamp":"2024-04-22T20:48:38.959416+0000","last_deep_scrub":"261'342","last_deep_scrub_stamp":"2024-04-22T20:48:38.959416+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.959416+0000","objects_scrubbed":9,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:10:05.470719+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45321,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":537,"num_read_kb":1022,"num_write":408,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.728684+0000","last_change":"2024-04-22T20:48:41.728710+0000","last_active":"2024-04-22T20:48:41.728684+0000","last_peered":"2024-04-22T20:48:41.728684+0000","last_clean":"2024-04-22T20:48:41.728684+0000","last_became_active":"2024-04-22T19:51:34.557662+0000","last_became_peered":"2024-04-22T19:51:34.557662+0000","last_unstale":"2024-04-22T20:48:41.728684+0000","last_undegraded":"2024-04-22T20:48:41.728684+0000","last_fullsized":"2024-04-22T20:48:41.728684+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:41.728597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:41.728597+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.728597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:59:34.359524+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"261'566","reported_seq":1383,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.924663+0000","last_change":"2024-04-22T20:48:30.924698+0000","last_active":"2024-04-22T20:48:30.924663+0000","last_peered":"2024-04-22T20:48:30.924663+0000","last_clean":"2024-04-22T20:48:30.924663+0000","last_became_active":"2024-04-22T19:51:36.570520+0000","last_became_peered":"2024-04-22T19:51:36.570520+0000","last_unstale":"2024-04-22T20:48:30.924663+0000","last_undegraded":"2024-04-22T20:48:30.924663+0000","last_fullsized":"2024-04-22T20:48:30.924663+0000","mapping_epoch":30,"log_start":"261'500","ondisk_log_start":"261'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'566","last_scrub_stamp":"2024-04-22T20:48:30.924566+0000","last_deep_scrub":"261'566","last_deep_scrub_stamp":"2024-04-22T20:48:30.924566+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.924566+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":500,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:46:13.753614+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1283,"num_read_kb":1280,"num_write":1166,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"261'311","reported_seq":968,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.300802+0000","last_change":"2024-04-22T20:48:53.300849+0000","last_active":"2024-04-22T20:48:53.300802+0000","last_peered":"2024-04-22T20:48:53.300802+0000","last_clean":"2024-04-22T20:48:53.300802+0000","last_became_active":"2024-04-22T19:51:28.511527+0000","last_became_peered":"2024-04-22T19:51:28.511527+0000","last_unstale":"2024-04-22T20:48:53.300802+0000","last_undegraded":"2024-04-22T20:48:53.300802+0000","last_fullsized":"2024-04-22T20:48:53.300802+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'311","last_scrub_stamp":"2024-04-22T20:48:53.300705+0000","last_deep_scrub":"261'311","last_deep_scrub_stamp":"2024-04-22T20:48:53.300705+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.300705+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:10.014981+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1221,"num_read_kb":977,"num_write":455,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"261'330","reported_seq":980,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.583635+0000","last_change":"2024-04-22T20:49:11.583686+0000","last_active":"2024-04-22T20:49:11.583635+0000","last_peered":"2024-04-22T20:49:11.583635+0000","last_clean":"2024-04-22T20:49:11.583635+0000","last_became_active":"2024-04-22T19:51:26.504472+0000","last_became_peered":"2024-04-22T19:51:26.504472+0000","last_unstale":"2024-04-22T20:49:11.583635+0000","last_undegraded":"2024-04-22T20:49:11.583635+0000","last_fullsized":"2024-04-22T20:49:11.583635+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'330","last_scrub_stamp":"2024-04-22T20:49:11.583522+0000","last_deep_scrub":"261'330","last_deep_scrub_stamp":"2024-04-22T20:49:11.583522+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.583522+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:51:00.824173+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158015,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":483,"num_read_kb":33343,"num_write":1137,"num_write_kb":49829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"261'6","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.511865+0000","last_change":"2024-04-22T20:48:37.511895+0000","last_active":"2024-04-22T20:48:37.511865+0000","last_peered":"2024-04-22T20:48:37.511865+0000","last_clean":"2024-04-22T20:48:37.511865+0000","last_became_active":"2024-04-22T19:51:30.524043+0000","last_became_peered":"2024-04-22T19:51:30.524043+0000","last_unstale":"2024-04-22T20:48:37.511865+0000","last_undegraded":"2024-04-22T20:48:37.511865+0000","last_fullsized":"2024-04-22T20:48:37.511865+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:37.511762+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:37.511762+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.511762+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:15.538566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":492,"num_read_kb":799,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"261'270","reported_seq":858,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.457615+0000","last_change":"2024-04-22T20:48:53.457678+0000","last_active":"2024-04-22T20:48:53.457615+0000","last_peered":"2024-04-22T20:48:53.457615+0000","last_clean":"2024-04-22T20:48:53.457615+0000","last_became_active":"2024-04-22T19:51:32.537968+0000","last_became_peered":"2024-04-22T19:51:32.537968+0000","last_unstale":"2024-04-22T20:48:53.457615+0000","last_undegraded":"2024-04-22T20:48:53.457615+0000","last_fullsized":"2024-04-22T20:48:53.457615+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'270","last_scrub_stamp":"2024-04-22T20:48:53.457514+0000","last_deep_scrub":"261'270","last_deep_scrub_stamp":"2024-04-22T20:48:53.457514+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.457514+0000","objects_scrubbed":4,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:35.634800+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43364,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":998,"num_write":350,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":506,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.498235+0000","last_change":"2024-04-22T20:48:45.498260+0000","last_active":"2024-04-22T20:48:45.498235+0000","last_peered":"2024-04-22T20:48:45.498235+0000","last_clean":"2024-04-22T20:48:45.498235+0000","last_became_active":"2024-04-22T19:51:34.555889+0000","last_became_peered":"2024-04-22T19:51:34.555889+0000","last_unstale":"2024-04-22T20:48:45.498235+0000","last_undegraded":"2024-04-22T20:48:45.498235+0000","last_fullsized":"2024-04-22T20:48:45.498235+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.498142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.498142+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.498142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:09:57.323691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"261'80","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.463200+0000","last_change":"2024-04-22T20:48:31.463234+0000","last_active":"2024-04-22T20:48:31.463200+0000","last_peered":"2024-04-22T20:48:31.463200+0000","last_clean":"2024-04-22T20:48:31.463200+0000","last_became_active":"2024-04-22T19:51:36.574555+0000","last_became_peered":"2024-04-22T19:51:36.574555+0000","last_unstale":"2024-04-22T20:48:31.463200+0000","last_undegraded":"2024-04-22T20:48:31.463200+0000","last_fullsized":"2024-04-22T20:48:31.463200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'80","last_scrub_stamp":"2024-04-22T20:48:31.463110+0000","last_deep_scrub":"261'80","last_deep_scrub_stamp":"2024-04-22T20:48:31.463110+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.463110+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:09:05.164724+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":91,"num_write":194,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"261'189","reported_seq":793,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.643992+0000","last_change":"2024-04-22T20:48:39.644036+0000","last_active":"2024-04-22T20:48:39.643992+0000","last_peered":"2024-04-22T20:48:39.643992+0000","last_clean":"2024-04-22T20:48:39.643992+0000","last_became_active":"2024-04-22T19:51:28.519424+0000","last_became_peered":"2024-04-22T19:51:28.519424+0000","last_unstale":"2024-04-22T20:48:39.643992+0000","last_undegraded":"2024-04-22T20:48:39.643992+0000","last_fullsized":"2024-04-22T20:48:39.643992+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'189","last_scrub_stamp":"2024-04-22T20:48:39.643857+0000","last_deep_scrub":"261'189","last_deep_scrub_stamp":"2024-04-22T20:48:39.643857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.643857+0000","objects_scrubbed":1,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:30:41.321492+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":758,"num_read_kb":601,"num_write":297,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"261'289","reported_seq":920,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.419906+0000","last_change":"2024-04-22T20:48:59.420014+0000","last_active":"2024-04-22T20:48:59.419906+0000","last_peered":"2024-04-22T20:48:59.419906+0000","last_clean":"2024-04-22T20:48:59.419906+0000","last_became_active":"2024-04-22T19:51:26.510273+0000","last_became_peered":"2024-04-22T19:51:26.510273+0000","last_unstale":"2024-04-22T20:48:59.419906+0000","last_undegraded":"2024-04-22T20:48:59.419906+0000","last_fullsized":"2024-04-22T20:48:59.419906+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'289","last_scrub_stamp":"2024-04-22T20:48:59.419798+0000","last_deep_scrub":"261'289","last_deep_scrub_stamp":"2024-04-22T20:48:59.419798+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.419798+0000","objects_scrubbed":9,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:16.853038+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2115,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":380,"num_read_kb":65007,"num_write":1004,"num_write_kb":29343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.472320+0000","last_change":"2024-04-22T20:48:35.472348+0000","last_active":"2024-04-22T20:48:35.472320+0000","last_peered":"2024-04-22T20:48:35.472320+0000","last_clean":"2024-04-22T20:48:35.472320+0000","last_became_active":"2024-04-22T19:51:30.525018+0000","last_became_peered":"2024-04-22T19:51:30.525018+0000","last_unstale":"2024-04-22T20:48:35.472320+0000","last_undegraded":"2024-04-22T20:48:35.472320+0000","last_fullsized":"2024-04-22T20:48:35.472320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:35.472216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:35.472216+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.472216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:21.173339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"261'317","reported_seq":919,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.539311+0000","last_change":"2024-04-22T20:49:02.539351+0000","last_active":"2024-04-22T20:49:02.539311+0000","last_peered":"2024-04-22T20:49:02.539311+0000","last_clean":"2024-04-22T20:49:02.539311+0000","last_became_active":"2024-04-22T19:51:32.538656+0000","last_became_peered":"2024-04-22T19:51:32.538656+0000","last_unstale":"2024-04-22T20:49:02.539311+0000","last_undegraded":"2024-04-22T20:49:02.539311+0000","last_fullsized":"2024-04-22T20:49:02.539311+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:49:02.539217+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:49:02.539217+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.539217+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:34:48.236169+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47724,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":498,"num_read_kb":1086,"num_write":396,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"261'509","reported_seq":2207,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.561625+0000","last_change":"2024-04-22T20:48:42.561655+0000","last_active":"2024-04-22T20:48:42.561625+0000","last_peered":"2024-04-22T20:48:42.561625+0000","last_clean":"2024-04-22T20:48:42.561625+0000","last_became_active":"2024-04-22T19:51:34.561488+0000","last_became_peered":"2024-04-22T19:51:34.561488+0000","last_unstale":"2024-04-22T20:48:42.561625+0000","last_undegraded":"2024-04-22T20:48:42.561625+0000","last_fullsized":"2024-04-22T20:48:42.561625+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'509","last_scrub_stamp":"2024-04-22T20:48:42.561535+0000","last_deep_scrub":"261'509","last_deep_scrub_stamp":"2024-04-22T20:48:42.561535+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.561535+0000","objects_scrubbed":1,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.827544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"261'136","reported_seq":634,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.548887+0000","last_change":"2024-04-22T20:48:39.548938+0000","last_active":"2024-04-22T20:48:39.548887+0000","last_peered":"2024-04-22T20:48:39.548887+0000","last_clean":"2024-04-22T20:48:39.548887+0000","last_became_active":"2024-04-22T19:51:36.573167+0000","last_became_peered":"2024-04-22T19:51:36.573167+0000","last_unstale":"2024-04-22T20:48:39.548887+0000","last_undegraded":"2024-04-22T20:48:39.548887+0000","last_fullsized":"2024-04-22T20:48:39.548887+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'136","last_scrub_stamp":"2024-04-22T20:48:39.548772+0000","last_deep_scrub":"261'136","last_deep_scrub_stamp":"2024-04-22T20:48:39.548772+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.548772+0000","objects_scrubbed":2,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:36:24.248525+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"261'150","reported_seq":724,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.595935+0000","last_change":"2024-04-22T20:48:54.595961+0000","last_active":"2024-04-22T20:48:54.595935+0000","last_peered":"2024-04-22T20:48:54.595935+0000","last_clean":"2024-04-22T20:48:54.595935+0000","last_became_active":"2024-04-22T19:51:28.518398+0000","last_became_peered":"2024-04-22T19:51:28.518398+0000","last_unstale":"2024-04-22T20:48:54.595935+0000","last_undegraded":"2024-04-22T20:48:54.595935+0000","last_fullsized":"2024-04-22T20:48:54.595935+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'150","last_scrub_stamp":"2024-04-22T20:48:54.595847+0000","last_deep_scrub":"261'150","last_deep_scrub_stamp":"2024-04-22T20:48:54.595847+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.595847+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:35:56.696922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":596,"num_read_kb":465,"num_write":243,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"261'313","reported_seq":991,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.857532+0000","last_change":"2024-04-22T20:48:24.857599+0000","last_active":"2024-04-22T20:48:24.857532+0000","last_peered":"2024-04-22T20:48:24.857532+0000","last_clean":"2024-04-22T20:48:24.857532+0000","last_became_active":"2024-04-22T19:52:05.112910+0000","last_became_peered":"2024-04-22T19:52:05.112910+0000","last_unstale":"2024-04-22T20:48:24.857532+0000","last_undegraded":"2024-04-22T20:48:24.857532+0000","last_fullsized":"2024-04-22T20:48:24.857532+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:24.857439+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:24.857439+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.857439+0000","objects_scrubbed":12,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:24:03.346339+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":486619,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":517,"num_read_kb":39016,"num_write":1092,"num_write_kb":34073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":513,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.406179+0000","last_change":"2024-04-22T20:48:31.406203+0000","last_active":"2024-04-22T20:48:31.406179+0000","last_peered":"2024-04-22T20:48:31.406179+0000","last_clean":"2024-04-22T20:48:31.406179+0000","last_became_active":"2024-04-22T19:51:30.529011+0000","last_became_peered":"2024-04-22T19:51:30.529011+0000","last_unstale":"2024-04-22T20:48:31.406179+0000","last_undegraded":"2024-04-22T20:48:31.406179+0000","last_fullsized":"2024-04-22T20:48:31.406179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:31.406079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:31.406079+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.406079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:00:54.573347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"261'493","reported_seq":1136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.712908+0000","last_change":"2024-04-22T20:48:59.712954+0000","last_active":"2024-04-22T20:48:59.712908+0000","last_peered":"2024-04-22T20:48:59.712908+0000","last_clean":"2024-04-22T20:48:59.712908+0000","last_became_active":"2024-04-22T19:51:32.537267+0000","last_became_peered":"2024-04-22T19:51:32.537267+0000","last_unstale":"2024-04-22T20:48:59.712908+0000","last_undegraded":"2024-04-22T20:48:59.712908+0000","last_fullsized":"2024-04-22T20:48:59.712908+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'493","last_scrub_stamp":"2024-04-22T20:48:59.712822+0000","last_deep_scrub":"261'493","last_deep_scrub_stamp":"2024-04-22T20:48:59.712822+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.712822+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":400,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:05.889509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":856,"num_read_kb":2097,"num_write":704,"num_write_kb":611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.377549+0000","last_change":"2024-04-22T20:48:25.377584+0000","last_active":"2024-04-22T20:48:25.377549+0000","last_peered":"2024-04-22T20:48:25.377549+0000","last_clean":"2024-04-22T20:48:25.377549+0000","last_became_active":"2024-04-22T19:51:34.552446+0000","last_became_peered":"2024-04-22T19:51:34.552446+0000","last_unstale":"2024-04-22T20:48:25.377549+0000","last_undegraded":"2024-04-22T20:48:25.377549+0000","last_fullsized":"2024-04-22T20:48:25.377549+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:25.377456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:25.377456+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.377456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:06:04.391078+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"261'107","reported_seq":618,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.636250+0000","last_change":"2024-04-22T20:48:57.636279+0000","last_active":"2024-04-22T20:48:57.636250+0000","last_peered":"2024-04-22T20:48:57.636250+0000","last_clean":"2024-04-22T20:48:57.636250+0000","last_became_active":"2024-04-22T19:51:36.570666+0000","last_became_peered":"2024-04-22T19:51:36.570666+0000","last_unstale":"2024-04-22T20:48:57.636250+0000","last_undegraded":"2024-04-22T20:48:57.636250+0000","last_fullsized":"2024-04-22T20:48:57.636250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'107","last_scrub_stamp":"2024-04-22T20:48:57.636158+0000","last_deep_scrub":"261'107","last_deep_scrub_stamp":"2024-04-22T20:48:57.636158+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.636158+0000","objects_scrubbed":1,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:11:43.927733+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":242,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"261'168","reported_seq":804,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.326095+0000","last_change":"2024-04-22T20:48:57.326118+0000","last_active":"2024-04-22T20:48:57.326095+0000","last_peered":"2024-04-22T20:48:57.326095+0000","last_clean":"2024-04-22T20:48:57.326095+0000","last_became_active":"2024-04-22T19:51:28.508288+0000","last_became_peered":"2024-04-22T19:51:28.508288+0000","last_unstale":"2024-04-22T20:48:57.326095+0000","last_undegraded":"2024-04-22T20:48:57.326095+0000","last_fullsized":"2024-04-22T20:48:57.326095+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'168","last_scrub_stamp":"2024-04-22T20:48:57.326006+0000","last_deep_scrub":"261'168","last_deep_scrub_stamp":"2024-04-22T20:48:57.326006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.326006+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:08.291741+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":576,"num_write":244,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"261'317","reported_seq":940,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.310216+0000","last_change":"2024-04-22T20:49:09.310267+0000","last_active":"2024-04-22T20:49:09.310216+0000","last_peered":"2024-04-22T20:49:09.310216+0000","last_clean":"2024-04-22T20:49:09.310216+0000","last_became_active":"2024-04-22T19:51:26.503669+0000","last_became_peered":"2024-04-22T19:51:26.503669+0000","last_unstale":"2024-04-22T20:49:09.310216+0000","last_undegraded":"2024-04-22T20:49:09.310216+0000","last_fullsized":"2024-04-22T20:49:09.310216+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:49:09.310094+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:49:09.310094+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.310094+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:52:43.733235+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194320,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":456,"num_read_kb":4498,"num_write":1097,"num_write_kb":33252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.451692+0000","last_change":"2024-04-22T20:48:33.451746+0000","last_active":"2024-04-22T20:48:33.451692+0000","last_peered":"2024-04-22T20:48:33.451692+0000","last_clean":"2024-04-22T20:48:33.451692+0000","last_became_active":"2024-04-22T19:51:30.526420+0000","last_became_peered":"2024-04-22T19:51:30.526420+0000","last_unstale":"2024-04-22T20:48:33.451692+0000","last_undegraded":"2024-04-22T20:48:33.451692+0000","last_fullsized":"2024-04-22T20:48:33.451692+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.451567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.451567+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.451567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:35.930208+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"261'274","reported_seq":862,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.374111+0000","last_change":"2024-04-22T20:48:38.374154+0000","last_active":"2024-04-22T20:48:38.374111+0000","last_peered":"2024-04-22T20:48:38.374111+0000","last_clean":"2024-04-22T20:48:38.374111+0000","last_became_active":"2024-04-22T19:51:32.536463+0000","last_became_peered":"2024-04-22T19:51:32.536463+0000","last_unstale":"2024-04-22T20:48:38.374111+0000","last_undegraded":"2024-04-22T20:48:38.374111+0000","last_fullsized":"2024-04-22T20:48:38.374111+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'274","last_scrub_stamp":"2024-04-22T20:48:38.374022+0000","last_deep_scrub":"261'274","last_deep_scrub_stamp":"2024-04-22T20:48:38.374022+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.374022+0000","objects_scrubbed":6,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:48:58.818076+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42355,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":457,"num_read_kb":963,"num_write":402,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.563374+0000","last_change":"2024-04-22T20:48:44.563409+0000","last_active":"2024-04-22T20:48:44.563374+0000","last_peered":"2024-04-22T20:48:44.563374+0000","last_clean":"2024-04-22T20:48:44.563374+0000","last_became_active":"2024-04-22T19:51:34.554737+0000","last_became_peered":"2024-04-22T19:51:34.554737+0000","last_unstale":"2024-04-22T20:48:44.563374+0000","last_undegraded":"2024-04-22T20:48:44.563374+0000","last_fullsized":"2024-04-22T20:48:44.563374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:44.563281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:44.563281+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.563281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:04:54.592040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"261'96","reported_seq":602,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.589774+0000","last_change":"2024-04-22T20:48:31.589814+0000","last_active":"2024-04-22T20:48:31.589774+0000","last_peered":"2024-04-22T20:48:31.589774+0000","last_clean":"2024-04-22T20:48:31.589774+0000","last_became_active":"2024-04-22T19:51:36.571086+0000","last_became_peered":"2024-04-22T19:51:36.571086+0000","last_unstale":"2024-04-22T20:48:31.589774+0000","last_undegraded":"2024-04-22T20:48:31.589774+0000","last_fullsized":"2024-04-22T20:48:31.589774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'96","last_scrub_stamp":"2024-04-22T20:48:31.589672+0000","last_deep_scrub":"261'96","last_deep_scrub_stamp":"2024-04-22T20:48:31.589672+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.589672+0000","objects_scrubbed":3,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:50.688646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":72,"num_write":227,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"261'254","reported_seq":847,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.244259+0000","last_change":"2024-04-22T20:48:49.244288+0000","last_active":"2024-04-22T20:48:49.244259+0000","last_peered":"2024-04-22T20:48:49.244259+0000","last_clean":"2024-04-22T20:48:49.244259+0000","last_became_active":"2024-04-22T19:51:28.514445+0000","last_became_peered":"2024-04-22T19:51:28.514445+0000","last_unstale":"2024-04-22T20:48:49.244259+0000","last_undegraded":"2024-04-22T20:48:49.244259+0000","last_fullsized":"2024-04-22T20:48:49.244259+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'254","last_scrub_stamp":"2024-04-22T20:48:49.244171+0000","last_deep_scrub":"261'254","last_deep_scrub_stamp":"2024-04-22T20:48:49.244171+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.244171+0000","objects_scrubbed":1,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:02:41.401583+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":723,"num_write":394,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":474,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'395","reported_seq":1150,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.572198+0000","last_change":"2024-04-22T20:48:51.572259+0000","last_active":"2024-04-22T20:48:51.572198+0000","last_peered":"2024-04-22T20:48:51.572198+0000","last_clean":"2024-04-22T20:48:51.572198+0000","last_became_active":"2024-04-22T19:51:26.496408+0000","last_became_peered":"2024-04-22T19:51:26.496408+0000","last_unstale":"2024-04-22T20:48:51.572198+0000","last_undegraded":"2024-04-22T20:48:51.572198+0000","last_fullsized":"2024-04-22T20:48:51.572198+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'395","last_scrub_stamp":"2024-04-22T20:48:51.572069+0000","last_deep_scrub":"261'395","last_deep_scrub_stamp":"2024-04-22T20:48:51.572069+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.572069+0000","objects_scrubbed":4,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:05:51.329345+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":840,"num_read_kb":8023,"num_write":1479,"num_write_kb":27015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.499734+0000","last_change":"2024-04-22T20:48:33.499756+0000","last_active":"2024-04-22T20:48:33.499734+0000","last_peered":"2024-04-22T20:48:33.499734+0000","last_clean":"2024-04-22T20:48:33.499734+0000","last_became_active":"2024-04-22T19:51:30.527309+0000","last_became_peered":"2024-04-22T19:51:30.527309+0000","last_unstale":"2024-04-22T20:48:33.499734+0000","last_undegraded":"2024-04-22T20:48:33.499734+0000","last_fullsized":"2024-04-22T20:48:33.499734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.499634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.499634+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.499634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:37:13.088297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"261'314","reported_seq":919,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.436156+0000","last_change":"2024-04-22T20:48:27.436199+0000","last_active":"2024-04-22T20:48:27.436156+0000","last_peered":"2024-04-22T20:48:27.436156+0000","last_clean":"2024-04-22T20:48:27.436156+0000","last_became_active":"2024-04-22T19:51:32.539732+0000","last_became_peered":"2024-04-22T19:51:32.539732+0000","last_unstale":"2024-04-22T20:48:27.436156+0000","last_undegraded":"2024-04-22T20:48:27.436156+0000","last_fullsized":"2024-04-22T20:48:27.436156+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'314","last_scrub_stamp":"2024-04-22T20:48:27.436075+0000","last_deep_scrub":"261'314","last_deep_scrub_stamp":"2024-04-22T20:48:27.436075+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.436075+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:26:29.043824+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52364,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":512,"num_read_kb":1133,"num_write":408,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"261'490","reported_seq":2212,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.422541+0000","last_change":"2024-04-22T20:48:23.422593+0000","last_active":"2024-04-22T20:48:23.422541+0000","last_peered":"2024-04-22T20:48:23.422541+0000","last_clean":"2024-04-22T20:48:23.422541+0000","last_became_active":"2024-04-22T19:51:34.556124+0000","last_became_peered":"2024-04-22T19:51:34.556124+0000","last_unstale":"2024-04-22T20:48:23.422541+0000","last_undegraded":"2024-04-22T20:48:23.422541+0000","last_fullsized":"2024-04-22T20:48:23.422541+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'490","last_scrub_stamp":"2024-04-22T20:48:23.422457+0000","last_deep_scrub":"261'490","last_deep_scrub_stamp":"2024-04-22T20:48:23.422457+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.422457+0000","objects_scrubbed":1,"log_size":90,"log_dups_size":400,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:12.183058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"261'120","reported_seq":621,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.517134+0000","last_change":"2024-04-22T20:48:30.517192+0000","last_active":"2024-04-22T20:48:30.517134+0000","last_peered":"2024-04-22T20:48:30.517134+0000","last_clean":"2024-04-22T20:48:30.517134+0000","last_became_active":"2024-04-22T19:51:36.572913+0000","last_became_peered":"2024-04-22T19:51:36.572913+0000","last_unstale":"2024-04-22T20:48:30.517134+0000","last_undegraded":"2024-04-22T20:48:30.517134+0000","last_fullsized":"2024-04-22T20:48:30.517134+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'120","last_scrub_stamp":"2024-04-22T20:48:30.517024+0000","last_deep_scrub":"261'120","last_deep_scrub_stamp":"2024-04-22T20:48:30.517024+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.517024+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:42.774241+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":102,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"261'338","reported_seq":1089,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.299517+0000","last_change":"2024-04-22T20:48:42.299554+0000","last_active":"2024-04-22T20:48:42.299517+0000","last_peered":"2024-04-22T20:48:42.299517+0000","last_clean":"2024-04-22T20:48:42.299517+0000","last_became_active":"2024-04-22T19:51:28.515809+0000","last_became_peered":"2024-04-22T19:51:28.515809+0000","last_unstale":"2024-04-22T20:48:42.299517+0000","last_undegraded":"2024-04-22T20:48:42.299517+0000","last_fullsized":"2024-04-22T20:48:42.299517+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'338","last_scrub_stamp":"2024-04-22T20:48:42.299436+0000","last_deep_scrub":"261'338","last_deep_scrub_stamp":"2024-04-22T20:48:42.299436+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.299436+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:24:36.610927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1504,"num_read_kb":1227,"num_write":616,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"261'336","reported_seq":980,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.290731+0000","last_change":"2024-04-22T20:48:59.290826+0000","last_active":"2024-04-22T20:48:59.290731+0000","last_peered":"2024-04-22T20:48:59.290731+0000","last_clean":"2024-04-22T20:48:59.290731+0000","last_became_active":"2024-04-22T19:51:26.499362+0000","last_became_peered":"2024-04-22T19:51:26.499362+0000","last_unstale":"2024-04-22T20:48:59.290731+0000","last_undegraded":"2024-04-22T20:48:59.290731+0000","last_fullsized":"2024-04-22T20:48:59.290731+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'336","last_scrub_stamp":"2024-04-22T20:48:59.290601+0000","last_deep_scrub":"261'336","last_deep_scrub_stamp":"2024-04-22T20:48:59.290601+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.290601+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:58:38.439661+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":26712,"num_write":1252,"num_write_kb":36357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"261'4","reported_seq":1146,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.941246+0000","last_change":"2024-04-22T20:48:53.941288+0000","last_active":"2024-04-22T20:48:53.941246+0000","last_peered":"2024-04-22T20:48:53.941246+0000","last_clean":"2024-04-22T20:48:53.941246+0000","last_became_active":"2024-04-22T19:51:30.527490+0000","last_became_peered":"2024-04-22T19:51:30.527490+0000","last_unstale":"2024-04-22T20:48:53.941246+0000","last_undegraded":"2024-04-22T20:48:53.941246+0000","last_fullsized":"2024-04-22T20:48:53.941246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:53.941159+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:53.941159+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.941159+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:57:29.554201+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":640,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"261'95","reported_seq":634,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.456406+0000","last_change":"2024-04-22T20:48:46.456437+0000","last_active":"2024-04-22T20:48:46.456406+0000","last_peered":"2024-04-22T20:48:46.456406+0000","last_clean":"2024-04-22T20:48:46.456406+0000","last_became_active":"2024-04-22T19:51:33.248349+0000","last_became_peered":"2024-04-22T19:51:33.248349+0000","last_unstale":"2024-04-22T20:48:46.456406+0000","last_undegraded":"2024-04-22T20:48:46.456406+0000","last_fullsized":"2024-04-22T20:48:46.456406+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'95","last_scrub_stamp":"2024-04-22T20:48:46.456325+0000","last_deep_scrub":"261'95","last_deep_scrub_stamp":"2024-04-22T20:48:46.456325+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.456325+0000","objects_scrubbed":5,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:50:06.808730+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":60,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"261'518","reported_seq":2081,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.510605+0000","last_change":"2024-04-22T20:48:44.510682+0000","last_active":"2024-04-22T20:48:44.510605+0000","last_peered":"2024-04-22T20:48:44.510605+0000","last_clean":"2024-04-22T20:48:44.510605+0000","last_became_active":"2024-04-22T19:51:34.561825+0000","last_became_peered":"2024-04-22T19:51:34.561825+0000","last_unstale":"2024-04-22T20:48:44.510605+0000","last_undegraded":"2024-04-22T20:48:44.510605+0000","last_fullsized":"2024-04-22T20:48:44.510605+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'518","last_scrub_stamp":"2024-04-22T20:48:44.510498+0000","last_deep_scrub":"261'518","last_deep_scrub_stamp":"2024-04-22T20:48:44.510498+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.510498+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:12:43.249410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"261'131","reported_seq":635,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.325984+0000","last_change":"2024-04-22T20:48:27.326015+0000","last_active":"2024-04-22T20:48:27.325984+0000","last_peered":"2024-04-22T20:48:27.325984+0000","last_clean":"2024-04-22T20:48:27.325984+0000","last_became_active":"2024-04-22T19:51:36.573026+0000","last_became_peered":"2024-04-22T19:51:36.573026+0000","last_unstale":"2024-04-22T20:48:27.325984+0000","last_undegraded":"2024-04-22T20:48:27.325984+0000","last_fullsized":"2024-04-22T20:48:27.325984+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:27.325897+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:27.325897+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.325897+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:17.870401+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":119,"num_write":309,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"261'207","reported_seq":827,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.659024+0000","last_change":"2024-04-22T20:48:46.659046+0000","last_active":"2024-04-22T20:48:46.659024+0000","last_peered":"2024-04-22T20:48:46.659024+0000","last_clean":"2024-04-22T20:48:46.659024+0000","last_became_active":"2024-04-22T19:51:28.512275+0000","last_became_peered":"2024-04-22T19:51:28.512275+0000","last_unstale":"2024-04-22T20:48:46.659024+0000","last_undegraded":"2024-04-22T20:48:46.659024+0000","last_fullsized":"2024-04-22T20:48:46.659024+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'207","last_scrub_stamp":"2024-04-22T20:48:46.658952+0000","last_deep_scrub":"261'207","last_deep_scrub_stamp":"2024-04-22T20:48:46.658952+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.658952+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:19:16.667922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":832,"num_read_kb":677,"num_write":302,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"261'328","reported_seq":975,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:13.328029+0000","last_change":"2024-04-22T20:49:13.328090+0000","last_active":"2024-04-22T20:49:13.328029+0000","last_peered":"2024-04-22T20:49:13.328029+0000","last_clean":"2024-04-22T20:49:13.328029+0000","last_became_active":"2024-04-22T19:51:26.498276+0000","last_became_peered":"2024-04-22T19:51:26.498276+0000","last_unstale":"2024-04-22T20:49:13.328029+0000","last_undegraded":"2024-04-22T20:49:13.328029+0000","last_fullsized":"2024-04-22T20:49:13.328029+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'328","last_scrub_stamp":"2024-04-22T20:49:13.327925+0000","last_deep_scrub":"261'328","last_deep_scrub_stamp":"2024-04-22T20:49:13.327925+0000","last_clean_scrub_stamp":"2024-04-22T20:49:13.327925+0000","objects_scrubbed":12,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:25:52.745613+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1581644,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":468,"num_read_kb":30145,"num_write":1162,"num_write_kb":23135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"261'4","reported_seq":831,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.215281+0000","last_change":"2024-04-22T20:49:06.215317+0000","last_active":"2024-04-22T20:49:06.215281+0000","last_peered":"2024-04-22T20:49:06.215281+0000","last_clean":"2024-04-22T20:49:06.215281+0000","last_became_active":"2024-04-22T19:51:30.530363+0000","last_became_peered":"2024-04-22T19:51:30.530363+0000","last_unstale":"2024-04-22T20:49:06.215281+0000","last_undegraded":"2024-04-22T20:49:06.215281+0000","last_fullsized":"2024-04-22T20:49:06.215281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:49:06.215191+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:49:06.215191+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.215191+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:09:00.109508+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":490,"num_read_kb":488,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"261'410","reported_seq":1027,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.882043+0000","last_change":"2024-04-22T20:49:05.882095+0000","last_active":"2024-04-22T20:49:05.882043+0000","last_peered":"2024-04-22T20:49:05.882043+0000","last_clean":"2024-04-22T20:49:05.882043+0000","last_became_active":"2024-04-22T19:51:32.540622+0000","last_became_peered":"2024-04-22T19:51:32.540622+0000","last_unstale":"2024-04-22T20:49:05.882043+0000","last_undegraded":"2024-04-22T20:49:05.882043+0000","last_fullsized":"2024-04-22T20:49:05.882043+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'410","last_scrub_stamp":"2024-04-22T20:49:05.881933+0000","last_deep_scrub":"261'410","last_deep_scrub_stamp":"2024-04-22T20:49:05.881933+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.881933+0000","objects_scrubbed":9,"log_size":110,"log_dups_size":300,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:14:09.519956+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56095,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1229,"num_read_kb":1850,"num_write":1762,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":478468,"num_omap_keys":1104,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.518288+0000","last_change":"2024-04-22T20:48:34.518332+0000","last_active":"2024-04-22T20:48:34.518288+0000","last_peered":"2024-04-22T20:48:34.518288+0000","last_clean":"2024-04-22T20:48:34.518288+0000","last_became_active":"2024-04-22T19:51:34.551009+0000","last_became_peered":"2024-04-22T19:51:34.551009+0000","last_unstale":"2024-04-22T20:48:34.518288+0000","last_undegraded":"2024-04-22T20:48:34.518288+0000","last_fullsized":"2024-04-22T20:48:34.518288+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.518165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.518165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.518165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:35:03.091773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"261'124","reported_seq":632,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.530433+0000","last_change":"2024-04-22T20:48:32.530468+0000","last_active":"2024-04-22T20:48:32.530433+0000","last_peered":"2024-04-22T20:48:32.530433+0000","last_clean":"2024-04-22T20:48:32.530433+0000","last_became_active":"2024-04-22T19:51:36.571517+0000","last_became_peered":"2024-04-22T19:51:36.571517+0000","last_unstale":"2024-04-22T20:48:32.530433+0000","last_undegraded":"2024-04-22T20:48:32.530433+0000","last_fullsized":"2024-04-22T20:48:32.530433+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:32.530326+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:32.530326+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.530326+0000","objects_scrubbed":3,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:41:58.642528+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":272,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"261'4187","reported_seq":4791,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.592564+0000","last_change":"2024-04-22T20:48:41.592601+0000","last_active":"2024-04-22T20:48:41.592564+0000","last_peered":"2024-04-22T20:48:41.592564+0000","last_clean":"2024-04-22T20:48:41.592564+0000","last_became_active":"2024-04-22T19:51:28.514869+0000","last_became_peered":"2024-04-22T19:51:28.514869+0000","last_unstale":"2024-04-22T20:48:41.592564+0000","last_undegraded":"2024-04-22T20:48:41.592564+0000","last_fullsized":"2024-04-22T20:48:41.592564+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4187","last_scrub_stamp":"2024-04-22T20:48:41.592448+0000","last_deep_scrub":"261'4187","last_deep_scrub_stamp":"2024-04-22T20:48:41.592448+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.592448+0000","objects_scrubbed":1,"log_size":87,"log_dups_size":3000,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:17:00.234094+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14767,"num_read_kb":11981,"num_write":6295,"num_write_kb":3142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":530,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.576811+0000","last_change":"2024-04-22T20:49:07.576811+0000","last_active":"2024-04-22T20:49:07.576811+0000","last_peered":"2024-04-22T20:49:07.576811+0000","last_clean":"2024-04-22T20:49:07.576811+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T20:49:07.576811+0000","last_undegraded":"2024-04-22T20:49:07.576811+0000","last_fullsized":"2024-04-22T20:49:07.576811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:07.576689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:07.576689+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.576689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:34:59.756037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"261'336","reported_seq":1040,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.341507+0000","last_change":"2024-04-22T20:49:07.341575+0000","last_active":"2024-04-22T20:49:07.341507+0000","last_peered":"2024-04-22T20:49:07.341507+0000","last_clean":"2024-04-22T20:49:07.341507+0000","last_became_active":"2024-04-22T19:51:26.505982+0000","last_became_peered":"2024-04-22T19:51:26.505982+0000","last_unstale":"2024-04-22T20:49:07.341507+0000","last_undegraded":"2024-04-22T20:49:07.341507+0000","last_fullsized":"2024-04-22T20:49:07.341507+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'336","last_scrub_stamp":"2024-04-22T20:49:07.341421+0000","last_deep_scrub":"261'336","last_deep_scrub_stamp":"2024-04-22T20:49:07.341421+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.341421+0000","objects_scrubbed":14,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:47.801140+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893637,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":542,"num_read_kb":107293,"num_write":1359,"num_write_kb":45616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.405165+0000","last_change":"2024-04-22T20:48:56.405202+0000","last_active":"2024-04-22T20:48:56.405165+0000","last_peered":"2024-04-22T20:48:56.405165+0000","last_clean":"2024-04-22T20:48:56.405165+0000","last_became_active":"2024-04-22T19:51:30.526939+0000","last_became_peered":"2024-04-22T19:51:30.526939+0000","last_unstale":"2024-04-22T20:48:56.405165+0000","last_undegraded":"2024-04-22T20:48:56.405165+0000","last_fullsized":"2024-04-22T20:48:56.405165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:56.405070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:56.405070+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.405070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:17.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"261'718","reported_seq":1428,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.940154+0000","last_change":"2024-04-22T20:48:40.940188+0000","last_active":"2024-04-22T20:48:40.940154+0000","last_peered":"2024-04-22T20:48:40.940154+0000","last_clean":"2024-04-22T20:48:40.940154+0000","last_became_active":"2024-04-22T19:51:32.542970+0000","last_became_peered":"2024-04-22T19:51:32.542970+0000","last_unstale":"2024-04-22T20:48:40.940154+0000","last_undegraded":"2024-04-22T20:48:40.940154+0000","last_fullsized":"2024-04-22T20:48:40.940154+0000","mapping_epoch":26,"log_start":"261'700","ondisk_log_start":"261'700","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'718","last_scrub_stamp":"2024-04-22T20:48:40.940067+0000","last_deep_scrub":"261'718","last_deep_scrub_stamp":"2024-04-22T20:48:40.940067+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.940067+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":700,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:32:36.401897+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142271,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1403,"num_read_kb":3194,"num_write":1318,"num_write_kb":1170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":89366,"num_omap_keys":214,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.618892+0000","last_change":"2024-04-22T20:48:48.618914+0000","last_active":"2024-04-22T20:48:48.618892+0000","last_peered":"2024-04-22T20:48:48.618892+0000","last_clean":"2024-04-22T20:48:48.618892+0000","last_became_active":"2024-04-22T19:51:34.553704+0000","last_became_peered":"2024-04-22T19:51:34.553704+0000","last_unstale":"2024-04-22T20:48:48.618892+0000","last_undegraded":"2024-04-22T20:48:48.618892+0000","last_fullsized":"2024-04-22T20:48:48.618892+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:48.618804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:48.618804+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.618804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:35:39.328500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"261'272","reported_seq":1016,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.596549+0000","last_change":"2024-04-22T20:48:46.596598+0000","last_active":"2024-04-22T20:48:46.596549+0000","last_peered":"2024-04-22T20:48:46.596549+0000","last_clean":"2024-04-22T20:48:46.596549+0000","last_became_active":"2024-04-22T19:51:36.570717+0000","last_became_peered":"2024-04-22T19:51:36.570717+0000","last_unstale":"2024-04-22T20:48:46.596549+0000","last_undegraded":"2024-04-22T20:48:46.596549+0000","last_fullsized":"2024-04-22T20:48:46.596549+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'272","last_scrub_stamp":"2024-04-22T20:48:46.596449+0000","last_deep_scrub":"261'272","last_deep_scrub_stamp":"2024-04-22T20:48:46.596449+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.596449+0000","objects_scrubbed":6,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:32:49.047679+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":568,"num_write":495,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"261'50","reported_seq":596,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.539678+0000","last_change":"2024-04-22T20:48:55.539707+0000","last_active":"2024-04-22T20:48:55.539678+0000","last_peered":"2024-04-22T20:48:55.539678+0000","last_clean":"2024-04-22T20:48:55.539678+0000","last_became_active":"2024-04-22T19:51:28.505719+0000","last_became_peered":"2024-04-22T19:51:28.505719+0000","last_unstale":"2024-04-22T20:48:55.539678+0000","last_undegraded":"2024-04-22T20:48:55.539678+0000","last_fullsized":"2024-04-22T20:48:55.539678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'50","last_scrub_stamp":"2024-04-22T20:48:55.539583+0000","last_deep_scrub":"261'50","last_deep_scrub_stamp":"2024-04-22T20:48:55.539583+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.539583+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:12.051486+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":69,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":523,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.792100+0000","last_change":"2024-04-22T20:48:57.792144+0000","last_active":"2024-04-22T20:48:57.792100+0000","last_peered":"2024-04-22T20:48:57.792100+0000","last_clean":"2024-04-22T20:48:57.792100+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T20:48:57.792100+0000","last_undegraded":"2024-04-22T20:48:57.792100+0000","last_fullsized":"2024-04-22T20:48:57.792100+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.791981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.791981+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.791981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:13:35.794791+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"261'358","reported_seq":1043,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.259784+0000","last_change":"2024-04-22T20:49:00.259858+0000","last_active":"2024-04-22T20:49:00.259784+0000","last_peered":"2024-04-22T20:49:00.259784+0000","last_clean":"2024-04-22T20:49:00.259784+0000","last_became_active":"2024-04-22T19:51:26.500968+0000","last_became_peered":"2024-04-22T19:51:26.500968+0000","last_unstale":"2024-04-22T20:49:00.259784+0000","last_undegraded":"2024-04-22T20:49:00.259784+0000","last_fullsized":"2024-04-22T20:49:00.259784+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'358","last_scrub_stamp":"2024-04-22T20:49:00.259692+0000","last_deep_scrub":"261'358","last_deep_scrub_stamp":"2024-04-22T20:49:00.259692+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.259692+0000","objects_scrubbed":7,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:46:46.910363+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048617,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":597,"num_read_kb":30331,"num_write":1294,"num_write_kb":32709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.639975+0000","last_change":"2024-04-22T20:48:55.639998+0000","last_active":"2024-04-22T20:48:55.639975+0000","last_peered":"2024-04-22T20:48:55.639975+0000","last_clean":"2024-04-22T20:48:55.639975+0000","last_became_active":"2024-04-22T19:51:30.526328+0000","last_became_peered":"2024-04-22T19:51:30.526328+0000","last_unstale":"2024-04-22T20:48:55.639975+0000","last_undegraded":"2024-04-22T20:48:55.639975+0000","last_fullsized":"2024-04-22T20:48:55.639975+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:55.639878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:55.639878+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.639878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:22:06.953672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"261'131","reported_seq":693,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.513766+0000","last_change":"2024-04-22T20:48:50.513809+0000","last_active":"2024-04-22T20:48:50.513766+0000","last_peered":"2024-04-22T20:48:50.513766+0000","last_clean":"2024-04-22T20:48:50.513766+0000","last_became_active":"2024-04-22T19:51:32.540805+0000","last_became_peered":"2024-04-22T19:51:32.540805+0000","last_unstale":"2024-04-22T20:48:50.513766+0000","last_undegraded":"2024-04-22T20:48:50.513766+0000","last_fullsized":"2024-04-22T20:48:50.513766+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:50.513677+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:50.513677+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.513677+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:46.102770+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":112,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1504,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.594752+0000","last_change":"2024-04-22T20:48:40.594815+0000","last_active":"2024-04-22T20:48:40.594752+0000","last_peered":"2024-04-22T20:48:40.594752+0000","last_clean":"2024-04-22T20:48:40.594752+0000","last_became_active":"2024-04-22T19:51:34.554307+0000","last_became_peered":"2024-04-22T19:51:34.554307+0000","last_unstale":"2024-04-22T20:48:40.594752+0000","last_undegraded":"2024-04-22T20:48:40.594752+0000","last_fullsized":"2024-04-22T20:48:40.594752+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.594644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.594644+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.594644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:32:53.799541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"261'124","reported_seq":635,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.895261+0000","last_change":"2024-04-22T20:48:29.895308+0000","last_active":"2024-04-22T20:48:29.895261+0000","last_peered":"2024-04-22T20:48:29.895261+0000","last_clean":"2024-04-22T20:48:29.895261+0000","last_became_active":"2024-04-22T19:51:36.571130+0000","last_became_peered":"2024-04-22T19:51:36.571130+0000","last_unstale":"2024-04-22T20:48:29.895261+0000","last_undegraded":"2024-04-22T20:48:29.895261+0000","last_fullsized":"2024-04-22T20:48:29.895261+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:29.895174+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:29.895174+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.895174+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:20:29.352021+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":131,"num_write":305,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"261'90","reported_seq":667,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.696575+0000","last_change":"2024-04-22T20:48:44.696603+0000","last_active":"2024-04-22T20:48:44.696575+0000","last_peered":"2024-04-22T20:48:44.696575+0000","last_clean":"2024-04-22T20:48:44.696575+0000","last_became_active":"2024-04-22T19:51:28.513451+0000","last_became_peered":"2024-04-22T19:51:28.513451+0000","last_unstale":"2024-04-22T20:48:44.696575+0000","last_undegraded":"2024-04-22T20:48:44.696575+0000","last_fullsized":"2024-04-22T20:48:44.696575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'90","last_scrub_stamp":"2024-04-22T20:48:44.696479+0000","last_deep_scrub":"261'90","last_deep_scrub_stamp":"2024-04-22T20:48:44.696479+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.696479+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:49.307610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":305,"num_write":130,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":528,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.613851+0000","last_change":"2024-04-22T20:49:06.613875+0000","last_active":"2024-04-22T20:49:06.613851+0000","last_peered":"2024-04-22T20:49:06.613851+0000","last_clean":"2024-04-22T20:49:06.613851+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T20:49:06.613851+0000","last_undegraded":"2024-04-22T20:49:06.613851+0000","last_fullsized":"2024-04-22T20:49:06.613851+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:06.613754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:06.613754+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.613754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:21:06.767402+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"261'323","reported_seq":994,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.609782+0000","last_change":"2024-04-22T20:48:50.609839+0000","last_active":"2024-04-22T20:48:50.609782+0000","last_peered":"2024-04-22T20:48:50.609782+0000","last_clean":"2024-04-22T20:48:50.609782+0000","last_became_active":"2024-04-22T19:51:26.495405+0000","last_became_peered":"2024-04-22T19:51:26.495405+0000","last_unstale":"2024-04-22T20:48:50.609782+0000","last_undegraded":"2024-04-22T20:48:50.609782+0000","last_fullsized":"2024-04-22T20:48:50.609782+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'323","last_scrub_stamp":"2024-04-22T20:48:50.609693+0000","last_deep_scrub":"261'323","last_deep_scrub_stamp":"2024-04-22T20:48:50.609693+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.609693+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:11:03.984643+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264208,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":523,"num_read_kb":57840,"num_write":1265,"num_write_kb":50104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"261'2","reported_seq":515,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.599212+0000","last_change":"2024-04-22T20:48:45.599259+0000","last_active":"2024-04-22T20:48:45.599212+0000","last_peered":"2024-04-22T20:48:45.599212+0000","last_clean":"2024-04-22T20:48:45.599212+0000","last_became_active":"2024-04-22T19:51:30.521134+0000","last_became_peered":"2024-04-22T19:51:30.521134+0000","last_unstale":"2024-04-22T20:48:45.599212+0000","last_undegraded":"2024-04-22T20:48:45.599212+0000","last_fullsized":"2024-04-22T20:48:45.599212+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:45.599098+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:45.599098+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.599098+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:03:06.489339+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"261'230","reported_seq":1466,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.455374+0000","last_change":"2024-04-22T20:48:49.455413+0000","last_active":"2024-04-22T20:48:49.455374+0000","last_peered":"2024-04-22T20:48:49.455374+0000","last_clean":"2024-04-22T20:48:49.455374+0000","last_became_active":"2024-04-22T19:51:33.250082+0000","last_became_peered":"2024-04-22T19:51:33.250082+0000","last_unstale":"2024-04-22T20:48:49.455374+0000","last_undegraded":"2024-04-22T20:48:49.455374+0000","last_fullsized":"2024-04-22T20:48:49.455374+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'230","last_scrub_stamp":"2024-04-22T20:48:49.455289+0000","last_deep_scrub":"261'230","last_deep_scrub_stamp":"2024-04-22T20:48:49.455289+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.455289+0000","objects_scrubbed":9,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:11.492021+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":188,"num_read_kb":180,"num_write":143,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5400,"num_omap_keys":12,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.690582+0000","last_change":"2024-04-22T20:48:50.690632+0000","last_active":"2024-04-22T20:48:50.690582+0000","last_peered":"2024-04-22T20:48:50.690582+0000","last_clean":"2024-04-22T20:48:50.690582+0000","last_became_active":"2024-04-22T19:51:34.546652+0000","last_became_peered":"2024-04-22T19:51:34.546652+0000","last_unstale":"2024-04-22T20:48:50.690582+0000","last_undegraded":"2024-04-22T20:48:50.690582+0000","last_fullsized":"2024-04-22T20:48:50.690582+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.690480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.690480+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.690480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:59:48.164650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"261'258","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.494703+0000","last_change":"2024-04-22T20:48:34.494735+0000","last_active":"2024-04-22T20:48:34.494703+0000","last_peered":"2024-04-22T20:48:34.494703+0000","last_clean":"2024-04-22T20:48:34.494703+0000","last_became_active":"2024-04-22T19:51:36.568767+0000","last_became_peered":"2024-04-22T19:51:36.568767+0000","last_unstale":"2024-04-22T20:48:34.494703+0000","last_undegraded":"2024-04-22T20:48:34.494703+0000","last_fullsized":"2024-04-22T20:48:34.494703+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'258","last_scrub_stamp":"2024-04-22T20:48:34.494609+0000","last_deep_scrub":"261'258","last_deep_scrub_stamp":"2024-04-22T20:48:34.494609+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.494609+0000","objects_scrubbed":4,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:08:51.017317+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":625,"num_write":569,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"261'125","reported_seq":761,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.657888+0000","last_change":"2024-04-22T20:48:56.657915+0000","last_active":"2024-04-22T20:48:56.657888+0000","last_peered":"2024-04-22T20:48:56.657888+0000","last_clean":"2024-04-22T20:48:56.657888+0000","last_became_active":"2024-04-22T19:51:28.518084+0000","last_became_peered":"2024-04-22T19:51:28.518084+0000","last_unstale":"2024-04-22T20:48:56.657888+0000","last_undegraded":"2024-04-22T20:48:56.657888+0000","last_fullsized":"2024-04-22T20:48:56.657888+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'125","last_scrub_stamp":"2024-04-22T20:48:56.657791+0000","last_deep_scrub":"261'125","last_deep_scrub_stamp":"2024-04-22T20:48:56.657791+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.657791+0000","objects_scrubbed":1,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:58.131958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":533,"num_read_kb":444,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":546,"num_omap_keys":3,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.596104+0000","last_change":"2024-04-22T20:48:49.596128+0000","last_active":"2024-04-22T20:48:49.596104+0000","last_peered":"2024-04-22T20:48:49.596104+0000","last_clean":"2024-04-22T20:48:49.596104+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T20:48:49.596104+0000","last_undegraded":"2024-04-22T20:48:49.596104+0000","last_fullsized":"2024-04-22T20:48:49.596104+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.596018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.596018+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.596018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:43:28.175145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"261'119","reported_seq":680,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.587820+0000","last_change":"2024-04-22T20:49:08.587820+0000","last_active":"2024-04-22T20:49:08.587820+0000","last_peered":"2024-04-22T20:49:08.587820+0000","last_clean":"2024-04-22T20:49:08.587820+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T20:49:08.587820+0000","last_undegraded":"2024-04-22T20:49:08.587820+0000","last_fullsized":"2024-04-22T20:49:08.587820+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'119","last_scrub_stamp":"2024-04-22T20:49:08.587682+0000","last_deep_scrub":"261'119","last_deep_scrub_stamp":"2024-04-22T20:49:08.587682+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.587682+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:12:58.890978+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"261'340","reported_seq":992,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.739859+0000","last_change":"2024-04-22T20:49:07.739920+0000","last_active":"2024-04-22T20:49:07.739859+0000","last_peered":"2024-04-22T20:49:07.739859+0000","last_clean":"2024-04-22T20:49:07.739859+0000","last_became_active":"2024-04-22T19:51:26.497608+0000","last_became_peered":"2024-04-22T19:51:26.497608+0000","last_unstale":"2024-04-22T20:49:07.739859+0000","last_undegraded":"2024-04-22T20:49:07.739859+0000","last_fullsized":"2024-04-22T20:49:07.739859+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'340","last_scrub_stamp":"2024-04-22T20:49:07.739756+0000","last_deep_scrub":"261'340","last_deep_scrub_stamp":"2024-04-22T20:49:07.739756+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.739756+0000","objects_scrubbed":4,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:25.144936+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4198438,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":523,"num_read_kb":42688,"num_write":1248,"num_write_kb":37697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.993233+0000","last_change":"2024-04-22T20:48:49.993255+0000","last_active":"2024-04-22T20:48:49.993233+0000","last_peered":"2024-04-22T20:48:49.993233+0000","last_clean":"2024-04-22T20:48:49.993233+0000","last_became_active":"2024-04-22T19:51:30.524437+0000","last_became_peered":"2024-04-22T19:51:30.524437+0000","last_unstale":"2024-04-22T20:48:49.993233+0000","last_undegraded":"2024-04-22T20:48:49.993233+0000","last_fullsized":"2024-04-22T20:48:49.993233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.993107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.993107+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.993107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:03:04.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"261'1111","reported_seq":2658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.570536+0000","last_change":"2024-04-22T20:48:35.570575+0000","last_active":"2024-04-22T20:48:35.570536+0000","last_peered":"2024-04-22T20:48:35.570536+0000","last_clean":"2024-04-22T20:48:35.570536+0000","last_became_active":"2024-04-22T19:51:32.538560+0000","last_became_peered":"2024-04-22T19:51:32.538560+0000","last_unstale":"2024-04-22T20:48:35.570536+0000","last_undegraded":"2024-04-22T20:48:35.570536+0000","last_fullsized":"2024-04-22T20:48:35.570536+0000","mapping_epoch":26,"log_start":"261'1100","ondisk_log_start":"261'1100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'1111","last_scrub_stamp":"2024-04-22T20:48:35.570456+0000","last_deep_scrub":"261'1111","last_deep_scrub_stamp":"2024-04-22T20:48:35.570456+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.570456+0000","objects_scrubbed":11,"log_size":11,"log_dups_size":1100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:51:22.759382+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148242,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1781,"num_read_kb":3545,"num_write":1100,"num_write_kb":929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5722,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.453095+0000","last_change":"2024-04-22T20:48:46.453118+0000","last_active":"2024-04-22T20:48:46.453095+0000","last_peered":"2024-04-22T20:48:46.453095+0000","last_clean":"2024-04-22T20:48:46.453095+0000","last_became_active":"2024-04-22T19:51:34.557022+0000","last_became_peered":"2024-04-22T19:51:34.557022+0000","last_unstale":"2024-04-22T20:48:46.453095+0000","last_undegraded":"2024-04-22T20:48:46.453095+0000","last_fullsized":"2024-04-22T20:48:46.453095+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:46.453005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:46.453005+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.453005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:30:22.992979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"261'167","reported_seq":684,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.647984+0000","last_change":"2024-04-22T20:48:47.648022+0000","last_active":"2024-04-22T20:48:47.647984+0000","last_peered":"2024-04-22T20:48:47.647984+0000","last_clean":"2024-04-22T20:48:47.647984+0000","last_became_active":"2024-04-22T19:51:36.567603+0000","last_became_peered":"2024-04-22T19:51:36.567603+0000","last_unstale":"2024-04-22T20:48:47.647984+0000","last_undegraded":"2024-04-22T20:48:47.647984+0000","last_fullsized":"2024-04-22T20:48:47.647984+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'167","last_scrub_stamp":"2024-04-22T20:48:47.647894+0000","last_deep_scrub":"261'167","last_deep_scrub_stamp":"2024-04-22T20:48:47.647894+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.647894+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:03:44.321582+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":641,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":222,"num_read_kb":216,"num_write":373,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"261'364","reported_seq":1007,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.287319+0000","last_change":"2024-04-22T20:48:51.287344+0000","last_active":"2024-04-22T20:48:51.287319+0000","last_peered":"2024-04-22T20:48:51.287319+0000","last_clean":"2024-04-22T20:48:51.287319+0000","last_became_active":"2024-04-22T19:51:28.512561+0000","last_became_peered":"2024-04-22T19:51:28.512561+0000","last_unstale":"2024-04-22T20:48:51.287319+0000","last_undegraded":"2024-04-22T20:48:51.287319+0000","last_fullsized":"2024-04-22T20:48:51.287319+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'364","last_scrub_stamp":"2024-04-22T20:48:51.287228+0000","last_deep_scrub":"261'364","last_deep_scrub_stamp":"2024-04-22T20:48:51.287228+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.287228+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:36:20.582606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1455,"num_read_kb":1128,"num_write":627,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"261'380","reported_seq":1106,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.729429+0000","last_change":"2024-04-22T20:49:09.729518+0000","last_active":"2024-04-22T20:49:09.729429+0000","last_peered":"2024-04-22T20:49:09.729429+0000","last_clean":"2024-04-22T20:49:09.729429+0000","last_became_active":"2024-04-22T19:51:26.498178+0000","last_became_peered":"2024-04-22T19:51:26.498178+0000","last_unstale":"2024-04-22T20:49:09.729429+0000","last_undegraded":"2024-04-22T20:49:09.729429+0000","last_fullsized":"2024-04-22T20:49:09.729429+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'380","last_scrub_stamp":"2024-04-22T20:49:09.729304+0000","last_deep_scrub":"261'380","last_deep_scrub_stamp":"2024-04-22T20:49:09.729304+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.729304+0000","objects_scrubbed":9,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:46:50.275561+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432486,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":709,"num_read_kb":47025,"num_write":1529,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":535,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.275955+0000","last_change":"2024-04-22T20:49:10.275977+0000","last_active":"2024-04-22T20:49:10.275955+0000","last_peered":"2024-04-22T20:49:10.275955+0000","last_clean":"2024-04-22T20:49:10.275955+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T20:49:10.275955+0000","last_undegraded":"2024-04-22T20:49:10.275955+0000","last_fullsized":"2024-04-22T20:49:10.275955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:10.275826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:10.275826+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.275826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:01:41.001016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":516,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.407400+0000","last_change":"2024-04-22T20:49:02.407429+0000","last_active":"2024-04-22T20:49:02.407400+0000","last_peered":"2024-04-22T20:49:02.407400+0000","last_clean":"2024-04-22T20:49:02.407400+0000","last_became_active":"2024-04-22T19:51:30.523242+0000","last_became_peered":"2024-04-22T19:51:30.523242+0000","last_unstale":"2024-04-22T20:49:02.407400+0000","last_undegraded":"2024-04-22T20:49:02.407400+0000","last_fullsized":"2024-04-22T20:49:02.407400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.407287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.407287+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.407287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:26.442506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"261'716","reported_seq":1450,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:14.566971+0000","last_change":"2024-04-22T20:49:14.567056+0000","last_active":"2024-04-22T20:49:14.566971+0000","last_peered":"2024-04-22T20:49:14.566971+0000","last_clean":"2024-04-22T20:49:14.566971+0000","last_became_active":"2024-04-22T19:51:33.248144+0000","last_became_peered":"2024-04-22T19:51:33.248144+0000","last_unstale":"2024-04-22T20:49:14.566971+0000","last_undegraded":"2024-04-22T20:49:14.566971+0000","last_fullsized":"2024-04-22T20:49:14.566971+0000","mapping_epoch":26,"log_start":"261'700","ondisk_log_start":"261'700","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'716","last_scrub_stamp":"2024-04-22T20:49:14.566857+0000","last_deep_scrub":"261'716","last_deep_scrub_stamp":"2024-04-22T20:49:14.566857+0000","last_clean_scrub_stamp":"2024-04-22T20:49:14.566857+0000","objects_scrubbed":9,"log_size":16,"log_dups_size":700,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:00.839694+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130335,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1263,"num_read_kb":2915,"num_write":1018,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.457004+0000","last_change":"2024-04-22T20:48:39.457028+0000","last_active":"2024-04-22T20:48:39.457004+0000","last_peered":"2024-04-22T20:48:39.457004+0000","last_clean":"2024-04-22T20:48:39.457004+0000","last_became_active":"2024-04-22T19:51:34.562001+0000","last_became_peered":"2024-04-22T19:51:34.562001+0000","last_unstale":"2024-04-22T20:48:39.457004+0000","last_undegraded":"2024-04-22T20:48:39.457004+0000","last_fullsized":"2024-04-22T20:48:39.457004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:39.456917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:39.456917+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.456917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:40.897612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"261'107","reported_seq":598,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.978305+0000","last_change":"2024-04-22T20:48:32.978335+0000","last_active":"2024-04-22T20:48:32.978305+0000","last_peered":"2024-04-22T20:48:32.978305+0000","last_clean":"2024-04-22T20:48:32.978305+0000","last_became_active":"2024-04-22T19:51:36.570134+0000","last_became_peered":"2024-04-22T19:51:36.570134+0000","last_unstale":"2024-04-22T20:48:32.978305+0000","last_undegraded":"2024-04-22T20:48:32.978305+0000","last_fullsized":"2024-04-22T20:48:32.978305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'107","last_scrub_stamp":"2024-04-22T20:48:32.978212+0000","last_deep_scrub":"261'107","last_deep_scrub_stamp":"2024-04-22T20:48:32.978212+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.978212+0000","objects_scrubbed":2,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:21.385230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":256,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"261'102","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.482552+0000","last_change":"2024-04-22T20:48:55.482598+0000","last_active":"2024-04-22T20:48:55.482552+0000","last_peered":"2024-04-22T20:48:55.482552+0000","last_clean":"2024-04-22T20:48:55.482552+0000","last_became_active":"2024-04-22T19:51:28.516178+0000","last_became_peered":"2024-04-22T19:51:28.516178+0000","last_unstale":"2024-04-22T20:48:55.482552+0000","last_undegraded":"2024-04-22T20:48:55.482552+0000","last_fullsized":"2024-04-22T20:48:55.482552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'102","last_scrub_stamp":"2024-04-22T20:48:55.482441+0000","last_deep_scrub":"261'102","last_deep_scrub_stamp":"2024-04-22T20:48:55.482441+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.482441+0000","objects_scrubbed":1,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:57:52.451471+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":337,"num_write":140,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"261'359","reported_seq":1063,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:16.322689+0000","last_change":"2024-04-22T20:49:16.322826+0000","last_active":"2024-04-22T20:49:16.322689+0000","last_peered":"2024-04-22T20:49:16.322689+0000","last_clean":"2024-04-22T20:49:16.322689+0000","last_became_active":"2024-04-22T19:51:26.502765+0000","last_became_peered":"2024-04-22T19:51:26.502765+0000","last_unstale":"2024-04-22T20:49:16.322689+0000","last_undegraded":"2024-04-22T20:49:16.322689+0000","last_fullsized":"2024-04-22T20:49:16.322689+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'359","last_scrub_stamp":"2024-04-22T20:49:16.322563+0000","last_deep_scrub":"261'359","last_deep_scrub_stamp":"2024-04-22T20:49:16.322563+0000","last_clean_scrub_stamp":"2024-04-22T20:49:16.322563+0000","objects_scrubbed":7,"log_size":59,"log_dups_size":300,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:07:24.118720+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4728363,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":573,"num_read_kb":127053,"num_write":1383,"num_write_kb":34012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":527,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.304810+0000","last_change":"2024-04-22T20:48:58.304837+0000","last_active":"2024-04-22T20:48:58.304810+0000","last_peered":"2024-04-22T20:48:58.304810+0000","last_clean":"2024-04-22T20:48:58.304810+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T20:48:58.304810+0000","last_undegraded":"2024-04-22T20:48:58.304810+0000","last_fullsized":"2024-04-22T20:48:58.304810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:58.304721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:58.304721+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.304721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:12:13.781455+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.303397+0000","last_change":"2024-04-22T20:48:56.303445+0000","last_active":"2024-04-22T20:48:56.303397+0000","last_peered":"2024-04-22T20:48:56.303397+0000","last_clean":"2024-04-22T20:48:56.303397+0000","last_became_active":"2024-04-22T19:51:30.529437+0000","last_became_peered":"2024-04-22T19:51:30.529437+0000","last_unstale":"2024-04-22T20:48:56.303397+0000","last_undegraded":"2024-04-22T20:48:56.303397+0000","last_fullsized":"2024-04-22T20:48:56.303397+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:56.303278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:56.303278+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.303278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:53:40.592791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"261'116","reported_seq":666,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.284844+0000","last_change":"2024-04-22T20:48:41.284912+0000","last_active":"2024-04-22T20:48:41.284844+0000","last_peered":"2024-04-22T20:48:41.284844+0000","last_clean":"2024-04-22T20:48:41.284844+0000","last_became_active":"2024-04-22T19:51:32.534280+0000","last_became_peered":"2024-04-22T19:51:32.534280+0000","last_unstale":"2024-04-22T20:48:41.284844+0000","last_undegraded":"2024-04-22T20:48:41.284844+0000","last_fullsized":"2024-04-22T20:48:41.284844+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'116","last_scrub_stamp":"2024-04-22T20:48:41.284749+0000","last_deep_scrub":"261'116","last_deep_scrub_stamp":"2024-04-22T20:48:41.284749+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.284749+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:15:26.736573+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"261'536","reported_seq":2165,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.509077+0000","last_change":"2024-04-22T20:48:41.509108+0000","last_active":"2024-04-22T20:48:41.509077+0000","last_peered":"2024-04-22T20:48:41.509077+0000","last_clean":"2024-04-22T20:48:41.509077+0000","last_became_active":"2024-04-22T19:51:34.556673+0000","last_became_peered":"2024-04-22T19:51:34.556673+0000","last_unstale":"2024-04-22T20:48:41.509077+0000","last_undegraded":"2024-04-22T20:48:41.509077+0000","last_fullsized":"2024-04-22T20:48:41.509077+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'536","last_scrub_stamp":"2024-04-22T20:48:41.508985+0000","last_deep_scrub":"261'536","last_deep_scrub_stamp":"2024-04-22T20:48:41.508985+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.508985+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:10:21.553002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"261'152","reported_seq":651,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.460819+0000","last_change":"2024-04-22T20:48:34.460850+0000","last_active":"2024-04-22T20:48:34.460819+0000","last_peered":"2024-04-22T20:48:34.460819+0000","last_clean":"2024-04-22T20:48:34.460819+0000","last_became_active":"2024-04-22T19:51:36.575351+0000","last_became_peered":"2024-04-22T19:51:36.575351+0000","last_unstale":"2024-04-22T20:48:34.460819+0000","last_undegraded":"2024-04-22T20:48:34.460819+0000","last_fullsized":"2024-04-22T20:48:34.460819+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:34.460727+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:34.460727+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.460727+0000","objects_scrubbed":3,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:55:34.883011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":80,"num_write":319,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"261'148","reported_seq":782,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.690078+0000","last_change":"2024-04-22T20:49:00.690116+0000","last_active":"2024-04-22T20:49:00.690078+0000","last_peered":"2024-04-22T20:49:00.690078+0000","last_clean":"2024-04-22T20:49:00.690078+0000","last_became_active":"2024-04-22T19:51:28.513940+0000","last_became_peered":"2024-04-22T19:51:28.513940+0000","last_unstale":"2024-04-22T20:49:00.690078+0000","last_undegraded":"2024-04-22T20:49:00.690078+0000","last_fullsized":"2024-04-22T20:49:00.690078+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'148","last_scrub_stamp":"2024-04-22T20:49:00.689987+0000","last_deep_scrub":"261'148","last_deep_scrub_stamp":"2024-04-22T20:49:00.689987+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.689987+0000","objects_scrubbed":2,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:59.928019+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":550,"num_write":219,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3271,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"261'357","reported_seq":1004,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.580271+0000","last_change":"2024-04-22T20:48:54.580328+0000","last_active":"2024-04-22T20:48:54.580271+0000","last_peered":"2024-04-22T20:48:54.580271+0000","last_clean":"2024-04-22T20:48:54.580271+0000","last_became_active":"2024-04-22T19:51:26.505842+0000","last_became_peered":"2024-04-22T19:51:26.505842+0000","last_unstale":"2024-04-22T20:48:54.580271+0000","last_undegraded":"2024-04-22T20:48:54.580271+0000","last_fullsized":"2024-04-22T20:48:54.580271+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'357","last_scrub_stamp":"2024-04-22T20:48:54.580170+0000","last_deep_scrub":"261'357","last_deep_scrub_stamp":"2024-04-22T20:48:54.580170+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.580170+0000","objects_scrubbed":9,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:58:46.289725+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308777,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":488,"num_read_kb":22191,"num_write":1115,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.583659+0000","last_change":"2024-04-22T20:48:49.583695+0000","last_active":"2024-04-22T20:48:49.583659+0000","last_peered":"2024-04-22T20:48:49.583659+0000","last_clean":"2024-04-22T20:48:49.583659+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T20:48:49.583659+0000","last_undegraded":"2024-04-22T20:48:49.583659+0000","last_fullsized":"2024-04-22T20:48:49.583659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.583572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.583572+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.583572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:08.812143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.523764+0000","last_change":"2024-04-22T20:48:35.523791+0000","last_active":"2024-04-22T20:48:35.523764+0000","last_peered":"2024-04-22T20:48:35.523764+0000","last_clean":"2024-04-22T20:48:35.523764+0000","last_became_active":"2024-04-22T19:51:30.525777+0000","last_became_peered":"2024-04-22T19:51:30.525777+0000","last_unstale":"2024-04-22T20:48:35.523764+0000","last_undegraded":"2024-04-22T20:48:35.523764+0000","last_fullsized":"2024-04-22T20:48:35.523764+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:35.523661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:35.523661+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.523661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:53:00.959964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"261'186","reported_seq":762,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.648519+0000","last_change":"2024-04-22T20:48:39.648562+0000","last_active":"2024-04-22T20:48:39.648519+0000","last_peered":"2024-04-22T20:48:39.648519+0000","last_clean":"2024-04-22T20:48:39.648519+0000","last_became_active":"2024-04-22T19:51:33.248354+0000","last_became_peered":"2024-04-22T19:51:33.248354+0000","last_unstale":"2024-04-22T20:48:39.648519+0000","last_undegraded":"2024-04-22T20:48:39.648519+0000","last_fullsized":"2024-04-22T20:48:39.648519+0000","mapping_epoch":26,"log_start":"258'100","ondisk_log_start":"258'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'186","last_scrub_stamp":"2024-04-22T20:48:39.648420+0000","last_deep_scrub":"261'186","last_deep_scrub_stamp":"2024-04-22T20:48:39.648420+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.648420+0000","objects_scrubbed":7,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:06:13.449126+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":222,"num_read_kb":211,"num_write":148,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"261'548","reported_seq":4432,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.947950+0000","last_change":"2024-04-22T20:48:36.948010+0000","last_active":"2024-04-22T20:48:36.947950+0000","last_peered":"2024-04-22T20:48:36.947950+0000","last_clean":"2024-04-22T20:48:36.947950+0000","last_became_active":"2024-04-22T19:51:34.561206+0000","last_became_peered":"2024-04-22T19:51:34.561206+0000","last_unstale":"2024-04-22T20:48:36.947950+0000","last_undegraded":"2024-04-22T20:48:36.947950+0000","last_fullsized":"2024-04-22T20:48:36.947950+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'548","last_scrub_stamp":"2024-04-22T20:48:36.947822+0000","last_deep_scrub":"261'548","last_deep_scrub_stamp":"2024-04-22T20:48:36.947822+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.947822+0000","objects_scrubbed":1,"log_size":48,"log_dups_size":500,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:24:06.222760+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"261'185","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.541264+0000","last_change":"2024-04-22T20:48:33.541297+0000","last_active":"2024-04-22T20:48:33.541264+0000","last_peered":"2024-04-22T20:48:33.541264+0000","last_clean":"2024-04-22T20:48:33.541264+0000","last_became_active":"2024-04-22T19:51:36.569338+0000","last_became_peered":"2024-04-22T19:51:36.569338+0000","last_unstale":"2024-04-22T20:48:33.541264+0000","last_undegraded":"2024-04-22T20:48:33.541264+0000","last_fullsized":"2024-04-22T20:48:33.541264+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'185","last_scrub_stamp":"2024-04-22T20:48:33.541172+0000","last_deep_scrub":"261'185","last_deep_scrub_stamp":"2024-04-22T20:48:33.541172+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.541172+0000","objects_scrubbed":3,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:38:36.406576+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":109,"num_write":422,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"261'109","reported_seq":713,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.629698+0000","last_change":"2024-04-22T20:48:48.629746+0000","last_active":"2024-04-22T20:48:48.629698+0000","last_peered":"2024-04-22T20:48:48.629698+0000","last_clean":"2024-04-22T20:48:48.629698+0000","last_became_active":"2024-04-22T19:51:28.512337+0000","last_became_peered":"2024-04-22T19:51:28.512337+0000","last_unstale":"2024-04-22T20:48:48.629698+0000","last_undegraded":"2024-04-22T20:48:48.629698+0000","last_fullsized":"2024-04-22T20:48:48.629698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'109","last_scrub_stamp":"2024-04-22T20:48:48.629602+0000","last_deep_scrub":"261'109","last_deep_scrub_stamp":"2024-04-22T20:48:48.629602+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.629602+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:59:28.312973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":434,"num_read_kb":362,"num_write":158,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'319","reported_seq":977,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.835821+0000","last_change":"2024-04-22T20:48:56.835865+0000","last_active":"2024-04-22T20:48:56.835821+0000","last_peered":"2024-04-22T20:48:56.835821+0000","last_clean":"2024-04-22T20:48:56.835821+0000","last_became_active":"2024-04-22T19:51:26.507804+0000","last_became_peered":"2024-04-22T19:51:26.507804+0000","last_unstale":"2024-04-22T20:48:56.835821+0000","last_undegraded":"2024-04-22T20:48:56.835821+0000","last_fullsized":"2024-04-22T20:48:56.835821+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'319","last_scrub_stamp":"2024-04-22T20:48:56.835711+0000","last_deep_scrub":"261'319","last_deep_scrub_stamp":"2024-04-22T20:48:56.835711+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.835711+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:07.605731+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":511,"num_read_kb":11012,"num_write":1234,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":526,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.628800+0000","last_change":"2024-04-22T20:48:48.628832+0000","last_active":"2024-04-22T20:48:48.628800+0000","last_peered":"2024-04-22T20:48:48.628800+0000","last_clean":"2024-04-22T20:48:48.628800+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T20:48:48.628800+0000","last_undegraded":"2024-04-22T20:48:48.628800+0000","last_fullsized":"2024-04-22T20:48:48.628800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:48.628712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:48.628712+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.628712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:37:25.666135+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.639531+0000","last_change":"2024-04-22T20:48:57.639557+0000","last_active":"2024-04-22T20:48:57.639531+0000","last_peered":"2024-04-22T20:48:57.639531+0000","last_clean":"2024-04-22T20:48:57.639531+0000","last_became_active":"2024-04-22T19:51:30.526837+0000","last_became_peered":"2024-04-22T19:51:30.526837+0000","last_unstale":"2024-04-22T20:48:57.639531+0000","last_undegraded":"2024-04-22T20:48:57.639531+0000","last_fullsized":"2024-04-22T20:48:57.639531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.639444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.639444+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.639444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:33:25.769033+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"261'551","reported_seq":1230,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:15.702359+0000","last_change":"2024-04-22T20:49:15.702420+0000","last_active":"2024-04-22T20:49:15.702359+0000","last_peered":"2024-04-22T20:49:15.702359+0000","last_clean":"2024-04-22T20:49:15.702359+0000","last_became_active":"2024-04-22T19:51:32.542344+0000","last_became_peered":"2024-04-22T19:51:32.542344+0000","last_unstale":"2024-04-22T20:49:15.702359+0000","last_undegraded":"2024-04-22T20:49:15.702359+0000","last_fullsized":"2024-04-22T20:49:15.702359+0000","mapping_epoch":26,"log_start":"261'500","ondisk_log_start":"261'500","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'551","last_scrub_stamp":"2024-04-22T20:49:15.702259+0000","last_deep_scrub":"261'551","last_deep_scrub_stamp":"2024-04-22T20:49:15.702259+0000","last_clean_scrub_stamp":"2024-04-22T20:49:15.702259+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":500,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:44:17.904007+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97214,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":946,"num_read_kb":2097,"num_write":792,"num_write_kb":634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7364,"num_omap_keys":20,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.386032+0000","last_change":"2024-04-22T20:48:33.386077+0000","last_active":"2024-04-22T20:48:33.386032+0000","last_peered":"2024-04-22T20:48:33.386032+0000","last_clean":"2024-04-22T20:48:33.386032+0000","last_became_active":"2024-04-22T19:51:34.550951+0000","last_became_peered":"2024-04-22T19:51:34.550951+0000","last_unstale":"2024-04-22T20:48:33.386032+0000","last_undegraded":"2024-04-22T20:48:33.386032+0000","last_fullsized":"2024-04-22T20:48:33.386032+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.385902+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.385902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:42.635210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"261'118","reported_seq":632,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.587158+0000","last_change":"2024-04-22T20:48:40.587201+0000","last_active":"2024-04-22T20:48:40.587158+0000","last_peered":"2024-04-22T20:48:40.587158+0000","last_clean":"2024-04-22T20:48:40.587158+0000","last_became_active":"2024-04-22T19:51:36.572976+0000","last_became_peered":"2024-04-22T19:51:36.572976+0000","last_unstale":"2024-04-22T20:48:40.587158+0000","last_undegraded":"2024-04-22T20:48:40.587158+0000","last_fullsized":"2024-04-22T20:48:40.587158+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'118","last_scrub_stamp":"2024-04-22T20:48:40.587061+0000","last_deep_scrub":"261'118","last_deep_scrub_stamp":"2024-04-22T20:48:40.587061+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.587061+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:35:06.423672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":120,"num_write":270,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"261'131","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.497001+0000","last_change":"2024-04-22T20:48:35.497051+0000","last_active":"2024-04-22T20:48:35.497001+0000","last_peered":"2024-04-22T20:48:35.497001+0000","last_clean":"2024-04-22T20:48:35.497001+0000","last_became_active":"2024-04-22T19:51:36.574238+0000","last_became_peered":"2024-04-22T19:51:36.574238+0000","last_unstale":"2024-04-22T20:48:35.497001+0000","last_undegraded":"2024-04-22T20:48:35.497001+0000","last_fullsized":"2024-04-22T20:48:35.497001+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:35.496901+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:35.496901+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.496901+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:05:26.775249+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":193,"num_read_kb":192,"num_write":291,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"261'124","reported_seq":749,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.322836+0000","last_change":"2024-04-22T20:48:45.322877+0000","last_active":"2024-04-22T20:48:45.322836+0000","last_peered":"2024-04-22T20:48:45.322836+0000","last_clean":"2024-04-22T20:48:45.322836+0000","last_became_active":"2024-04-22T19:51:28.515334+0000","last_became_peered":"2024-04-22T19:51:28.515334+0000","last_unstale":"2024-04-22T20:48:45.322836+0000","last_undegraded":"2024-04-22T20:48:45.322836+0000","last_fullsized":"2024-04-22T20:48:45.322836+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:45.322739+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:45.322739+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.322739+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:20.200394+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":424,"num_write":184,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"261'324","reported_seq":1046,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.259253+0000","last_change":"2024-04-22T20:49:01.259314+0000","last_active":"2024-04-22T20:49:01.259253+0000","last_peered":"2024-04-22T20:49:01.259253+0000","last_clean":"2024-04-22T20:49:01.259253+0000","last_became_active":"2024-04-22T19:51:26.506150+0000","last_became_peered":"2024-04-22T19:51:26.506150+0000","last_unstale":"2024-04-22T20:49:01.259253+0000","last_undegraded":"2024-04-22T20:49:01.259253+0000","last_fullsized":"2024-04-22T20:49:01.259253+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:49:01.259157+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:49:01.259157+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.259157+0000","objects_scrubbed":11,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:51:26.309359+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097285,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":716,"num_read_kb":35402,"num_write":1112,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.602028+0000","last_change":"2024-04-22T20:48:36.602053+0000","last_active":"2024-04-22T20:48:36.602028+0000","last_peered":"2024-04-22T20:48:36.602028+0000","last_clean":"2024-04-22T20:48:36.602028+0000","last_became_active":"2024-04-22T19:51:30.522903+0000","last_became_peered":"2024-04-22T19:51:30.522903+0000","last_unstale":"2024-04-22T20:48:36.602028+0000","last_undegraded":"2024-04-22T20:48:36.602028+0000","last_fullsized":"2024-04-22T20:48:36.602028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:36.601934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:36.601934+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.601934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:58.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"261'301","reported_seq":894,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.465902+0000","last_change":"2024-04-22T20:48:51.465949+0000","last_active":"2024-04-22T20:48:51.465902+0000","last_peered":"2024-04-22T20:48:51.465902+0000","last_clean":"2024-04-22T20:48:51.465902+0000","last_became_active":"2024-04-22T19:51:32.539631+0000","last_became_peered":"2024-04-22T19:51:32.539631+0000","last_unstale":"2024-04-22T20:48:51.465902+0000","last_undegraded":"2024-04-22T20:48:51.465902+0000","last_fullsized":"2024-04-22T20:48:51.465902+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'301","last_scrub_stamp":"2024-04-22T20:48:51.465803+0000","last_deep_scrub":"261'301","last_deep_scrub_stamp":"2024-04-22T20:48:51.465803+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.465803+0000","objects_scrubbed":5,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:40.094839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44228,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":495,"num_read_kb":1138,"num_write":404,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.536890+0000","last_change":"2024-04-22T20:48:42.536915+0000","last_active":"2024-04-22T20:48:42.536890+0000","last_peered":"2024-04-22T20:48:42.536890+0000","last_clean":"2024-04-22T20:48:42.536890+0000","last_became_active":"2024-04-22T19:51:34.556650+0000","last_became_peered":"2024-04-22T19:51:34.556650+0000","last_unstale":"2024-04-22T20:48:42.536890+0000","last_undegraded":"2024-04-22T20:48:42.536890+0000","last_fullsized":"2024-04-22T20:48:42.536890+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:42.536759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:42.536759+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.536759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:46:57.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"261'99","reported_seq":608,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.450094+0000","last_change":"2024-04-22T20:48:38.450136+0000","last_active":"2024-04-22T20:48:38.450094+0000","last_peered":"2024-04-22T20:48:38.450094+0000","last_clean":"2024-04-22T20:48:38.450094+0000","last_became_active":"2024-04-22T19:51:36.566980+0000","last_became_peered":"2024-04-22T19:51:36.566980+0000","last_unstale":"2024-04-22T20:48:38.450094+0000","last_undegraded":"2024-04-22T20:48:38.450094+0000","last_fullsized":"2024-04-22T20:48:38.450094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'99","last_scrub_stamp":"2024-04-22T20:48:38.449980+0000","last_deep_scrub":"261'99","last_deep_scrub_stamp":"2024-04-22T20:48:38.449980+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.449980+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:31:45.431617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":235,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"261'130","reported_seq":743,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.716009+0000","last_change":"2024-04-22T20:48:42.716036+0000","last_active":"2024-04-22T20:48:42.716009+0000","last_peered":"2024-04-22T20:48:42.716009+0000","last_clean":"2024-04-22T20:48:42.716009+0000","last_became_active":"2024-04-22T19:51:28.517675+0000","last_became_peered":"2024-04-22T19:51:28.517675+0000","last_unstale":"2024-04-22T20:48:42.716009+0000","last_undegraded":"2024-04-22T20:48:42.716009+0000","last_fullsized":"2024-04-22T20:48:42.716009+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'130","last_scrub_stamp":"2024-04-22T20:48:42.715887+0000","last_deep_scrub":"261'130","last_deep_scrub_stamp":"2024-04-22T20:48:42.715887+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.715887+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:12:53.780947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":543,"num_read_kb":446,"num_write":195,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"261'341","reported_seq":1050,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.564901+0000","last_change":"2024-04-22T20:49:10.565004+0000","last_active":"2024-04-22T20:49:10.564901+0000","last_peered":"2024-04-22T20:49:10.564901+0000","last_clean":"2024-04-22T20:49:10.564901+0000","last_became_active":"2024-04-22T19:51:26.501897+0000","last_became_peered":"2024-04-22T19:51:26.501897+0000","last_unstale":"2024-04-22T20:49:10.564901+0000","last_undegraded":"2024-04-22T20:49:10.564901+0000","last_fullsized":"2024-04-22T20:49:10.564901+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'341","last_scrub_stamp":"2024-04-22T20:49:10.564792+0000","last_deep_scrub":"261'341","last_deep_scrub_stamp":"2024-04-22T20:49:10.564792+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.564792+0000","objects_scrubbed":12,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:09.024556+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3673455,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":613,"num_read_kb":41436,"num_write":1146,"num_write_kb":29344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.413044+0000","last_change":"2024-04-22T20:49:04.413074+0000","last_active":"2024-04-22T20:49:04.413044+0000","last_peered":"2024-04-22T20:49:04.413044+0000","last_clean":"2024-04-22T20:49:04.413044+0000","last_became_active":"2024-04-22T19:51:30.524388+0000","last_became_peered":"2024-04-22T19:51:30.524388+0000","last_unstale":"2024-04-22T20:49:04.413044+0000","last_undegraded":"2024-04-22T20:49:04.413044+0000","last_fullsized":"2024-04-22T20:49:04.413044+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:04.412948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:04.412948+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.412948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:33:44.545552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"261'52","reported_seq":565,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.671890+0000","last_change":"2024-04-22T20:49:03.671919+0000","last_active":"2024-04-22T20:49:03.671890+0000","last_peered":"2024-04-22T20:49:03.671890+0000","last_clean":"2024-04-22T20:49:03.671890+0000","last_became_active":"2024-04-22T19:51:32.539260+0000","last_became_peered":"2024-04-22T19:51:32.539260+0000","last_unstale":"2024-04-22T20:49:03.671890+0000","last_undegraded":"2024-04-22T20:49:03.671890+0000","last_fullsized":"2024-04-22T20:49:03.671890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'52","last_scrub_stamp":"2024-04-22T20:49:03.671816+0000","last_deep_scrub":"261'52","last_deep_scrub_stamp":"2024-04-22T20:49:03.671816+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.671816+0000","objects_scrubbed":3,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:59.506345+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":36,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":588,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.488848+0000","last_change":"2024-04-22T20:48:47.488871+0000","last_active":"2024-04-22T20:48:47.488848+0000","last_peered":"2024-04-22T20:48:47.488848+0000","last_clean":"2024-04-22T20:48:47.488848+0000","last_became_active":"2024-04-22T19:51:34.551294+0000","last_became_peered":"2024-04-22T19:51:34.551294+0000","last_unstale":"2024-04-22T20:48:47.488848+0000","last_undegraded":"2024-04-22T20:48:47.488848+0000","last_fullsized":"2024-04-22T20:48:47.488848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.488766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.488766+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.488766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:13:45.930564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"261'202","reported_seq":729,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.470421+0000","last_change":"2024-04-22T20:48:36.470421+0000","last_active":"2024-04-22T20:48:36.470421+0000","last_peered":"2024-04-22T20:48:36.470421+0000","last_clean":"2024-04-22T20:48:36.470421+0000","last_became_active":"2024-04-22T19:51:36.573936+0000","last_became_peered":"2024-04-22T19:51:36.573936+0000","last_unstale":"2024-04-22T20:48:36.470421+0000","last_undegraded":"2024-04-22T20:48:36.470421+0000","last_fullsized":"2024-04-22T20:48:36.470421+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'202","last_scrub_stamp":"2024-04-22T20:48:36.470271+0000","last_deep_scrub":"261'202","last_deep_scrub_stamp":"2024-04-22T20:48:36.470271+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.470271+0000","objects_scrubbed":1,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:05:08.510777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":284,"num_write":451,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"261'231","reported_seq":926,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.411158+0000","last_change":"2024-04-22T20:49:01.411188+0000","last_active":"2024-04-22T20:49:01.411158+0000","last_peered":"2024-04-22T20:49:01.411158+0000","last_clean":"2024-04-22T20:49:01.411158+0000","last_became_active":"2024-04-22T19:51:28.508307+0000","last_became_peered":"2024-04-22T19:51:28.508307+0000","last_unstale":"2024-04-22T20:49:01.411158+0000","last_undegraded":"2024-04-22T20:49:01.411158+0000","last_fullsized":"2024-04-22T20:49:01.411158+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'231","last_scrub_stamp":"2024-04-22T20:49:01.411072+0000","last_deep_scrub":"261'231","last_deep_scrub_stamp":"2024-04-22T20:49:01.411072+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.411072+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:33.901303+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":959,"num_read_kb":784,"num_write":348,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"261'323","reported_seq":963,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.793039+0000","last_change":"2024-04-22T20:48:59.793110+0000","last_active":"2024-04-22T20:48:59.793039+0000","last_peered":"2024-04-22T20:48:59.793039+0000","last_clean":"2024-04-22T20:48:59.793039+0000","last_became_active":"2024-04-22T19:51:26.501477+0000","last_became_peered":"2024-04-22T19:51:26.501477+0000","last_unstale":"2024-04-22T20:48:59.793039+0000","last_undegraded":"2024-04-22T20:48:59.793039+0000","last_fullsized":"2024-04-22T20:48:59.793039+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'323","last_scrub_stamp":"2024-04-22T20:48:59.792943+0000","last_deep_scrub":"261'323","last_deep_scrub_stamp":"2024-04-22T20:48:59.792943+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.792943+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:21.956398+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":46522,"num_write":1280,"num_write_kb":35021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.361909+0000","last_change":"2024-04-22T20:48:54.361936+0000","last_active":"2024-04-22T20:48:54.361909+0000","last_peered":"2024-04-22T20:48:54.361909+0000","last_clean":"2024-04-22T20:48:54.361909+0000","last_became_active":"2024-04-22T19:51:30.527076+0000","last_became_peered":"2024-04-22T19:51:30.527076+0000","last_unstale":"2024-04-22T20:48:54.361909+0000","last_undegraded":"2024-04-22T20:48:54.361909+0000","last_fullsized":"2024-04-22T20:48:54.361909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:54.361806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:54.361806+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.361806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:55.484727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"261'307","reported_seq":916,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.485477+0000","last_change":"2024-04-22T20:48:49.485517+0000","last_active":"2024-04-22T20:48:49.485477+0000","last_peered":"2024-04-22T20:48:49.485477+0000","last_clean":"2024-04-22T20:48:49.485477+0000","last_became_active":"2024-04-22T19:51:32.542262+0000","last_became_peered":"2024-04-22T19:51:32.542262+0000","last_unstale":"2024-04-22T20:48:49.485477+0000","last_undegraded":"2024-04-22T20:48:49.485477+0000","last_fullsized":"2024-04-22T20:48:49.485477+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'307","last_scrub_stamp":"2024-04-22T20:48:49.485363+0000","last_deep_scrub":"261'307","last_deep_scrub_stamp":"2024-04-22T20:48:49.485363+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.485363+0000","objects_scrubbed":5,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:26:58.383965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46510,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":491,"num_read_kb":1038,"num_write":384,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.651253+0000","last_change":"2024-04-22T20:48:38.651275+0000","last_active":"2024-04-22T20:48:38.651253+0000","last_peered":"2024-04-22T20:48:38.651253+0000","last_clean":"2024-04-22T20:48:38.651253+0000","last_became_active":"2024-04-22T19:51:34.553691+0000","last_became_peered":"2024-04-22T19:51:34.553691+0000","last_unstale":"2024-04-22T20:48:38.651253+0000","last_undegraded":"2024-04-22T20:48:38.651253+0000","last_fullsized":"2024-04-22T20:48:38.651253+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:38.651145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:38.651145+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.651145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:39:15.734170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"261'137","reported_seq":628,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.859930+0000","last_change":"2024-04-22T20:48:28.859930+0000","last_active":"2024-04-22T20:48:28.859930+0000","last_peered":"2024-04-22T20:48:28.859930+0000","last_clean":"2024-04-22T20:48:28.859930+0000","last_became_active":"2024-04-22T19:51:36.571290+0000","last_became_peered":"2024-04-22T19:51:36.571290+0000","last_unstale":"2024-04-22T20:48:28.859930+0000","last_undegraded":"2024-04-22T20:48:28.859930+0000","last_fullsized":"2024-04-22T20:48:28.859930+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'137","last_scrub_stamp":"2024-04-22T20:48:28.859747+0000","last_deep_scrub":"261'137","last_deep_scrub_stamp":"2024-04-22T20:48:28.859747+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.859747+0000","objects_scrubbed":2,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:02:28.549243+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":111,"num_write":334,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"261'185","reported_seq":863,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.384703+0000","last_change":"2024-04-22T20:49:03.384728+0000","last_active":"2024-04-22T20:49:03.384703+0000","last_peered":"2024-04-22T20:49:03.384703+0000","last_clean":"2024-04-22T20:49:03.384703+0000","last_became_active":"2024-04-22T19:51:28.506615+0000","last_became_peered":"2024-04-22T19:51:28.506615+0000","last_unstale":"2024-04-22T20:49:03.384703+0000","last_undegraded":"2024-04-22T20:49:03.384703+0000","last_fullsized":"2024-04-22T20:49:03.384703+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'185","last_scrub_stamp":"2024-04-22T20:49:03.384610+0000","last_deep_scrub":"261'185","last_deep_scrub_stamp":"2024-04-22T20:49:03.384610+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.384610+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:20:25.431536+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":757,"num_read_kb":623,"num_write":269,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'338","reported_seq":988,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.686128+0000","last_change":"2024-04-22T20:49:11.686193+0000","last_active":"2024-04-22T20:49:11.686128+0000","last_peered":"2024-04-22T20:49:11.686128+0000","last_clean":"2024-04-22T20:49:11.686128+0000","last_became_active":"2024-04-22T19:51:26.502938+0000","last_became_peered":"2024-04-22T19:51:26.502938+0000","last_unstale":"2024-04-22T20:49:11.686128+0000","last_undegraded":"2024-04-22T20:49:11.686128+0000","last_fullsized":"2024-04-22T20:49:11.686128+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'338","last_scrub_stamp":"2024-04-22T20:49:11.686003+0000","last_deep_scrub":"261'338","last_deep_scrub_stamp":"2024-04-22T20:49:11.686003+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.686003+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:46.874873+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194317,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":478,"num_read_kb":28738,"num_write":1269,"num_write_kb":45425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.504136+0000","last_change":"2024-04-22T20:48:32.504160+0000","last_active":"2024-04-22T20:48:32.504136+0000","last_peered":"2024-04-22T20:48:32.504136+0000","last_clean":"2024-04-22T20:48:32.504136+0000","last_became_active":"2024-04-22T19:51:30.526994+0000","last_became_peered":"2024-04-22T19:51:30.526994+0000","last_unstale":"2024-04-22T20:48:32.504136+0000","last_undegraded":"2024-04-22T20:48:32.504136+0000","last_fullsized":"2024-04-22T20:48:32.504136+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:32.504047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:32.504047+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.504047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:39.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"261'255","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.460418+0000","last_change":"2024-04-22T20:49:00.460454+0000","last_active":"2024-04-22T20:49:00.460418+0000","last_peered":"2024-04-22T20:49:00.460418+0000","last_clean":"2024-04-22T20:49:00.460418+0000","last_became_active":"2024-04-22T19:51:32.537875+0000","last_became_peered":"2024-04-22T19:51:32.537875+0000","last_unstale":"2024-04-22T20:49:00.460418+0000","last_undegraded":"2024-04-22T20:49:00.460418+0000","last_fullsized":"2024-04-22T20:49:00.460418+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'255","last_scrub_stamp":"2024-04-22T20:49:00.460334+0000","last_deep_scrub":"261'255","last_deep_scrub_stamp":"2024-04-22T20:49:00.460334+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.460334+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:02.585609+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":422,"num_read_kb":961,"num_write":338,"num_write_kb":268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.635537+0000","last_change":"2024-04-22T20:48:46.635559+0000","last_active":"2024-04-22T20:48:46.635537+0000","last_peered":"2024-04-22T20:48:46.635537+0000","last_clean":"2024-04-22T20:48:46.635537+0000","last_became_active":"2024-04-22T19:51:34.553954+0000","last_became_peered":"2024-04-22T19:51:34.553954+0000","last_unstale":"2024-04-22T20:48:46.635537+0000","last_undegraded":"2024-04-22T20:48:46.635537+0000","last_fullsized":"2024-04-22T20:48:46.635537+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:46.635459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:46.635459+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.635459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:54:27.052935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"261'192","reported_seq":839,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.551497+0000","last_change":"2024-04-22T20:48:43.551521+0000","last_active":"2024-04-22T20:48:43.551497+0000","last_peered":"2024-04-22T20:48:43.551497+0000","last_clean":"2024-04-22T20:48:43.551497+0000","last_became_active":"2024-04-22T19:51:28.514593+0000","last_became_peered":"2024-04-22T19:51:28.514593+0000","last_unstale":"2024-04-22T20:48:43.551497+0000","last_undegraded":"2024-04-22T20:48:43.551497+0000","last_fullsized":"2024-04-22T20:48:43.551497+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'192","last_scrub_stamp":"2024-04-22T20:48:43.551394+0000","last_deep_scrub":"261'192","last_deep_scrub_stamp":"2024-04-22T20:48:43.551394+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.551394+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:06:00.844687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":788,"num_read_kb":638,"num_write":274,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"261'342","reported_seq":1013,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:15.359241+0000","last_change":"2024-04-22T20:49:15.359341+0000","last_active":"2024-04-22T20:49:15.359241+0000","last_peered":"2024-04-22T20:49:15.359241+0000","last_clean":"2024-04-22T20:49:15.359241+0000","last_became_active":"2024-04-22T19:51:26.501812+0000","last_became_peered":"2024-04-22T19:51:26.501812+0000","last_unstale":"2024-04-22T20:49:15.359241+0000","last_undegraded":"2024-04-22T20:49:15.359241+0000","last_fullsized":"2024-04-22T20:49:15.359241+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'342","last_scrub_stamp":"2024-04-22T20:49:15.359113+0000","last_deep_scrub":"261'342","last_deep_scrub_stamp":"2024-04-22T20:49:15.359113+0000","last_clean_scrub_stamp":"2024-04-22T20:49:15.359113+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:28.788196+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857304,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":552,"num_read_kb":55834,"num_write":1162,"num_write_kb":35884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":509,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.630664+0000","last_change":"2024-04-22T20:48:53.630690+0000","last_active":"2024-04-22T20:48:53.630664+0000","last_peered":"2024-04-22T20:48:53.630664+0000","last_clean":"2024-04-22T20:48:53.630664+0000","last_became_active":"2024-04-22T19:51:30.525450+0000","last_became_peered":"2024-04-22T19:51:30.525450+0000","last_unstale":"2024-04-22T20:48:53.630664+0000","last_undegraded":"2024-04-22T20:48:53.630664+0000","last_fullsized":"2024-04-22T20:48:53.630664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:53.630576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:53.630576+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.630576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:07:46.156214+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"261'489","reported_seq":1127,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.643184+0000","last_change":"2024-04-22T20:48:37.643184+0000","last_active":"2024-04-22T20:48:37.643184+0000","last_peered":"2024-04-22T20:48:37.643184+0000","last_clean":"2024-04-22T20:48:37.643184+0000","last_became_active":"2024-04-22T19:51:32.540755+0000","last_became_peered":"2024-04-22T19:51:32.540755+0000","last_unstale":"2024-04-22T20:48:37.643184+0000","last_undegraded":"2024-04-22T20:48:37.643184+0000","last_fullsized":"2024-04-22T20:48:37.643184+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'489","last_scrub_stamp":"2024-04-22T20:48:37.643033+0000","last_deep_scrub":"261'489","last_deep_scrub_stamp":"2024-04-22T20:48:37.643033+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.643033+0000","objects_scrubbed":7,"log_size":89,"log_dups_size":400,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:20:59.082062+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93788,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":875,"num_read_kb":2058,"num_write":756,"num_write_kb":671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23904,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.952860+0000","last_change":"2024-04-22T20:48:37.952860+0000","last_active":"2024-04-22T20:48:37.952860+0000","last_peered":"2024-04-22T20:48:37.952860+0000","last_clean":"2024-04-22T20:48:37.952860+0000","last_became_active":"2024-04-22T19:51:34.553363+0000","last_became_peered":"2024-04-22T19:51:34.553363+0000","last_unstale":"2024-04-22T20:48:37.952860+0000","last_undegraded":"2024-04-22T20:48:37.952860+0000","last_fullsized":"2024-04-22T20:48:37.952860+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:37.952749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:37.952749+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.952749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:14.410865+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"261'162","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.484277+0000","last_change":"2024-04-22T20:48:35.484310+0000","last_active":"2024-04-22T20:48:35.484277+0000","last_peered":"2024-04-22T20:48:35.484277+0000","last_clean":"2024-04-22T20:48:35.484277+0000","last_became_active":"2024-04-22T19:51:36.568373+0000","last_became_peered":"2024-04-22T19:51:36.568373+0000","last_unstale":"2024-04-22T20:48:35.484277+0000","last_undegraded":"2024-04-22T20:48:35.484277+0000","last_fullsized":"2024-04-22T20:48:35.484277+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'162","last_scrub_stamp":"2024-04-22T20:48:35.484193+0000","last_deep_scrub":"261'162","last_deep_scrub_stamp":"2024-04-22T20:48:35.484193+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.484193+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:59:33.641598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":213,"num_write":373,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"261'100","reported_seq":694,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.619802+0000","last_change":"2024-04-22T20:49:02.619828+0000","last_active":"2024-04-22T20:49:02.619802+0000","last_peered":"2024-04-22T20:49:02.619802+0000","last_clean":"2024-04-22T20:49:02.619802+0000","last_became_active":"2024-04-22T19:51:28.510629+0000","last_became_peered":"2024-04-22T19:51:28.510629+0000","last_unstale":"2024-04-22T20:49:02.619802+0000","last_undegraded":"2024-04-22T20:49:02.619802+0000","last_fullsized":"2024-04-22T20:49:02.619802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'100","last_scrub_stamp":"2024-04-22T20:49:02.619712+0000","last_deep_scrub":"261'100","last_deep_scrub_stamp":"2024-04-22T20:49:02.619712+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.619712+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:26.099759+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":435,"num_read_kb":357,"num_write":161,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"261'305","reported_seq":947,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.605247+0000","last_change":"2024-04-22T20:48:52.605390+0000","last_active":"2024-04-22T20:48:52.605247+0000","last_peered":"2024-04-22T20:48:52.605247+0000","last_clean":"2024-04-22T20:48:52.605247+0000","last_became_active":"2024-04-22T19:51:26.497165+0000","last_became_peered":"2024-04-22T19:51:26.497165+0000","last_unstale":"2024-04-22T20:48:52.605247+0000","last_undegraded":"2024-04-22T20:48:52.605247+0000","last_fullsized":"2024-04-22T20:48:52.605247+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'305","last_scrub_stamp":"2024-04-22T20:48:52.605144+0000","last_deep_scrub":"261'305","last_deep_scrub_stamp":"2024-04-22T20:48:52.605144+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.605144+0000","objects_scrubbed":9,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:37.401989+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":662323,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":493,"num_read_kb":39207,"num_write":1122,"num_write_kb":42297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.479719+0000","last_change":"2024-04-22T20:48:33.479741+0000","last_active":"2024-04-22T20:48:33.479719+0000","last_peered":"2024-04-22T20:48:33.479719+0000","last_clean":"2024-04-22T20:48:33.479719+0000","last_became_active":"2024-04-22T19:51:30.526597+0000","last_became_peered":"2024-04-22T19:51:30.526597+0000","last_unstale":"2024-04-22T20:48:33.479719+0000","last_undegraded":"2024-04-22T20:48:33.479719+0000","last_fullsized":"2024-04-22T20:48:33.479719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.479622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.479622+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.479622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:35:51.805769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"261'319","reported_seq":916,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.463525+0000","last_change":"2024-04-22T20:48:58.463525+0000","last_active":"2024-04-22T20:48:58.463525+0000","last_peered":"2024-04-22T20:48:58.463525+0000","last_clean":"2024-04-22T20:48:58.463525+0000","last_became_active":"2024-04-22T19:51:32.537416+0000","last_became_peered":"2024-04-22T19:51:32.537416+0000","last_unstale":"2024-04-22T20:48:58.463525+0000","last_undegraded":"2024-04-22T20:48:58.463525+0000","last_fullsized":"2024-04-22T20:48:58.463525+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'319","last_scrub_stamp":"2024-04-22T20:48:58.463340+0000","last_deep_scrub":"261'319","last_deep_scrub_stamp":"2024-04-22T20:48:58.463340+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.463340+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:11:45.125481+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43451,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":521,"num_read_kb":1119,"num_write":436,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.694176+0000","last_change":"2024-04-22T20:48:45.694176+0000","last_active":"2024-04-22T20:48:45.694176+0000","last_peered":"2024-04-22T20:48:45.694176+0000","last_clean":"2024-04-22T20:48:45.694176+0000","last_became_active":"2024-04-22T19:51:34.561096+0000","last_became_peered":"2024-04-22T19:51:34.561096+0000","last_unstale":"2024-04-22T20:48:45.694176+0000","last_undegraded":"2024-04-22T20:48:45.694176+0000","last_fullsized":"2024-04-22T20:48:45.694176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.694065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.694065+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.694065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:07:56.257157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"261'166","reported_seq":675,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.608939+0000","last_change":"2024-04-22T20:48:56.608983+0000","last_active":"2024-04-22T20:48:56.608939+0000","last_peered":"2024-04-22T20:48:56.608939+0000","last_clean":"2024-04-22T20:48:56.608939+0000","last_became_active":"2024-04-22T19:51:36.571565+0000","last_became_peered":"2024-04-22T19:51:36.571565+0000","last_unstale":"2024-04-22T20:48:56.608939+0000","last_undegraded":"2024-04-22T20:48:56.608939+0000","last_fullsized":"2024-04-22T20:48:56.608939+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'166","last_scrub_stamp":"2024-04-22T20:48:56.608839+0000","last_deep_scrub":"261'166","last_deep_scrub_stamp":"2024-04-22T20:48:56.608839+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.608839+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:31:59.748806+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":180,"num_write":473,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"261'44","reported_seq":619,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.377542+0000","last_change":"2024-04-22T20:49:05.377569+0000","last_active":"2024-04-22T20:49:05.377542+0000","last_peered":"2024-04-22T20:49:05.377542+0000","last_clean":"2024-04-22T20:49:05.377542+0000","last_became_active":"2024-04-22T19:51:28.506835+0000","last_became_peered":"2024-04-22T19:51:28.506835+0000","last_unstale":"2024-04-22T20:49:05.377542+0000","last_undegraded":"2024-04-22T20:49:05.377542+0000","last_fullsized":"2024-04-22T20:49:05.377542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'44","last_scrub_stamp":"2024-04-22T20:49:05.377442+0000","last_deep_scrub":"261'44","last_deep_scrub_stamp":"2024-04-22T20:49:05.377442+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.377442+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:54:43.348262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":202,"num_read_kb":174,"num_write":58,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"261'377","reported_seq":1013,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.442985+0000","last_change":"2024-04-22T20:48:24.443059+0000","last_active":"2024-04-22T20:48:24.442985+0000","last_peered":"2024-04-22T20:48:24.442985+0000","last_clean":"2024-04-22T20:48:24.442985+0000","last_became_active":"2024-04-22T19:52:04.617820+0000","last_became_peered":"2024-04-22T19:52:04.617820+0000","last_unstale":"2024-04-22T20:48:24.442985+0000","last_undegraded":"2024-04-22T20:48:24.442985+0000","last_fullsized":"2024-04-22T20:48:24.442985+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'377","last_scrub_stamp":"2024-04-22T20:48:24.442876+0000","last_deep_scrub":"261'377","last_deep_scrub_stamp":"2024-04-22T20:48:24.442876+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.442876+0000","objects_scrubbed":13,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:42:38.642906+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14463569,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":535,"num_read_kb":31540,"num_write":1473,"num_write_kb":43247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.299779+0000","last_change":"2024-04-22T20:48:54.299779+0000","last_active":"2024-04-22T20:48:54.299779+0000","last_peered":"2024-04-22T20:48:54.299779+0000","last_clean":"2024-04-22T20:48:54.299779+0000","last_became_active":"2024-04-22T19:51:30.529322+0000","last_became_peered":"2024-04-22T19:51:30.529322+0000","last_unstale":"2024-04-22T20:48:54.299779+0000","last_undegraded":"2024-04-22T20:48:54.299779+0000","last_fullsized":"2024-04-22T20:48:54.299779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:54.299664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:54.299664+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.299664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:25.063326+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"261'276","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.466258+0000","last_change":"2024-04-22T20:48:47.466295+0000","last_active":"2024-04-22T20:48:47.466258+0000","last_peered":"2024-04-22T20:48:47.466258+0000","last_clean":"2024-04-22T20:48:47.466258+0000","last_became_active":"2024-04-22T19:51:33.248293+0000","last_became_peered":"2024-04-22T19:51:33.248293+0000","last_unstale":"2024-04-22T20:48:47.466258+0000","last_undegraded":"2024-04-22T20:48:47.466258+0000","last_fullsized":"2024-04-22T20:48:47.466258+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'276","last_scrub_stamp":"2024-04-22T20:48:47.466165+0000","last_deep_scrub":"261'276","last_deep_scrub_stamp":"2024-04-22T20:48:47.466165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.466165+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:12:37.324192+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56069,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1056,"num_write":364,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.684260+0000","last_change":"2024-04-22T20:48:43.684299+0000","last_active":"2024-04-22T20:48:43.684260+0000","last_peered":"2024-04-22T20:48:43.684260+0000","last_clean":"2024-04-22T20:48:43.684260+0000","last_became_active":"2024-04-22T19:51:34.560955+0000","last_became_peered":"2024-04-22T19:51:34.560955+0000","last_unstale":"2024-04-22T20:48:43.684260+0000","last_undegraded":"2024-04-22T20:48:43.684260+0000","last_fullsized":"2024-04-22T20:48:43.684260+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:43.684147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:43.684147+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.684147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:05:24.733318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"261'122","reported_seq":614,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.995578+0000","last_change":"2024-04-22T20:48:34.995676+0000","last_active":"2024-04-22T20:48:34.995578+0000","last_peered":"2024-04-22T20:48:34.995578+0000","last_clean":"2024-04-22T20:48:34.995578+0000","last_became_active":"2024-04-22T19:51:36.567139+0000","last_became_peered":"2024-04-22T19:51:36.567139+0000","last_unstale":"2024-04-22T20:48:34.995578+0000","last_undegraded":"2024-04-22T20:48:34.995578+0000","last_fullsized":"2024-04-22T20:48:34.995578+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'122","last_scrub_stamp":"2024-04-22T20:48:34.995457+0000","last_deep_scrub":"261'122","last_deep_scrub_stamp":"2024-04-22T20:48:34.995457+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.995457+0000","objects_scrubbed":3,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:31:14.860340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":92,"num_write":268,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"261'106","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.592002+0000","last_change":"2024-04-22T20:48:43.592002+0000","last_active":"2024-04-22T20:48:43.592002+0000","last_peered":"2024-04-22T20:48:43.592002+0000","last_clean":"2024-04-22T20:48:43.592002+0000","last_became_active":"2024-04-22T19:51:28.512642+0000","last_became_peered":"2024-04-22T19:51:28.512642+0000","last_unstale":"2024-04-22T20:48:43.592002+0000","last_undegraded":"2024-04-22T20:48:43.592002+0000","last_fullsized":"2024-04-22T20:48:43.592002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'106","last_scrub_stamp":"2024-04-22T20:48:43.591857+0000","last_deep_scrub":"261'106","last_deep_scrub_stamp":"2024-04-22T20:48:43.591857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.591857+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:12:33.205148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":349,"num_write":169,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"261'328","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:14.740039+0000","last_change":"2024-04-22T20:49:14.740117+0000","last_active":"2024-04-22T20:49:14.740039+0000","last_peered":"2024-04-22T20:49:14.740039+0000","last_clean":"2024-04-22T20:49:14.740039+0000","last_became_active":"2024-04-22T19:51:26.511966+0000","last_became_peered":"2024-04-22T19:51:26.511966+0000","last_unstale":"2024-04-22T20:49:14.740039+0000","last_undegraded":"2024-04-22T20:49:14.740039+0000","last_fullsized":"2024-04-22T20:49:14.740039+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'328","last_scrub_stamp":"2024-04-22T20:49:14.739928+0000","last_deep_scrub":"261'328","last_deep_scrub_stamp":"2024-04-22T20:49:14.739928+0000","last_clean_scrub_stamp":"2024-04-22T20:49:14.739928+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:57:07.865559+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262244,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":436,"num_read_kb":38275,"num_write":1102,"num_write_kb":51509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":502,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.538975+0000","last_change":"2024-04-22T20:48:34.538975+0000","last_active":"2024-04-22T20:48:34.538975+0000","last_peered":"2024-04-22T20:48:34.538975+0000","last_clean":"2024-04-22T20:48:34.538975+0000","last_became_active":"2024-04-22T19:51:30.526949+0000","last_became_peered":"2024-04-22T19:51:30.526949+0000","last_unstale":"2024-04-22T20:48:34.538975+0000","last_undegraded":"2024-04-22T20:48:34.538975+0000","last_fullsized":"2024-04-22T20:48:34.538975+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.538867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.538867+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.538867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:45:12.857278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"261'111","reported_seq":646,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.397743+0000","last_change":"2024-04-22T20:48:35.397789+0000","last_active":"2024-04-22T20:48:35.397743+0000","last_peered":"2024-04-22T20:48:35.397743+0000","last_clean":"2024-04-22T20:48:35.397743+0000","last_became_active":"2024-04-22T19:51:33.248113+0000","last_became_peered":"2024-04-22T19:51:33.248113+0000","last_unstale":"2024-04-22T20:48:35.397743+0000","last_undegraded":"2024-04-22T20:48:35.397743+0000","last_fullsized":"2024-04-22T20:48:35.397743+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'111","last_scrub_stamp":"2024-04-22T20:48:35.397661+0000","last_deep_scrub":"261'111","last_deep_scrub_stamp":"2024-04-22T20:48:35.397661+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.397661+0000","objects_scrubbed":8,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:59:22.081071+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":119,"num_read_kb":113,"num_write":116,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11222,"num_omap_keys":30,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"261'518","reported_seq":2306,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.597254+0000","last_change":"2024-04-22T20:48:42.597282+0000","last_active":"2024-04-22T20:48:42.597254+0000","last_peered":"2024-04-22T20:48:42.597254+0000","last_clean":"2024-04-22T20:48:42.597254+0000","last_became_active":"2024-04-22T19:51:34.556789+0000","last_became_peered":"2024-04-22T19:51:34.556789+0000","last_unstale":"2024-04-22T20:48:42.597254+0000","last_undegraded":"2024-04-22T20:48:42.597254+0000","last_fullsized":"2024-04-22T20:48:42.597254+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'518","last_scrub_stamp":"2024-04-22T20:48:42.597160+0000","last_deep_scrub":"261'518","last_deep_scrub_stamp":"2024-04-22T20:48:42.597160+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.597160+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:48:25.653530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"261'98","reported_seq":605,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.461752+0000","last_change":"2024-04-22T20:48:41.461786+0000","last_active":"2024-04-22T20:48:41.461752+0000","last_peered":"2024-04-22T20:48:41.461752+0000","last_clean":"2024-04-22T20:48:41.461752+0000","last_became_active":"2024-04-22T19:51:36.571175+0000","last_became_peered":"2024-04-22T19:51:36.571175+0000","last_unstale":"2024-04-22T20:48:41.461752+0000","last_undegraded":"2024-04-22T20:48:41.461752+0000","last_fullsized":"2024-04-22T20:48:41.461752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'98","last_scrub_stamp":"2024-04-22T20:48:41.461655+0000","last_deep_scrub":"261'98","last_deep_scrub_stamp":"2024-04-22T20:48:41.461655+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.461655+0000","objects_scrubbed":2,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:03.950310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":221,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"261'131","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.334715+0000","last_change":"2024-04-22T20:48:55.334715+0000","last_active":"2024-04-22T20:48:55.334715+0000","last_peered":"2024-04-22T20:48:55.334715+0000","last_clean":"2024-04-22T20:48:55.334715+0000","last_became_active":"2024-04-22T19:51:28.509110+0000","last_became_peered":"2024-04-22T19:51:28.509110+0000","last_unstale":"2024-04-22T20:48:55.334715+0000","last_undegraded":"2024-04-22T20:48:55.334715+0000","last_fullsized":"2024-04-22T20:48:55.334715+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:55.334593+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:55.334593+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.334593+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:37.391655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":519,"num_read_kb":420,"num_write":188,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"261'402","reported_seq":1038,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.850089+0000","last_change":"2024-04-22T20:48:25.850089+0000","last_active":"2024-04-22T20:48:25.850089+0000","last_peered":"2024-04-22T20:48:25.850089+0000","last_clean":"2024-04-22T20:48:25.850089+0000","last_became_active":"2024-04-22T19:52:04.619385+0000","last_became_peered":"2024-04-22T19:52:04.619385+0000","last_unstale":"2024-04-22T20:48:25.850089+0000","last_undegraded":"2024-04-22T20:48:25.850089+0000","last_fullsized":"2024-04-22T20:48:25.850089+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'402","last_scrub_stamp":"2024-04-22T20:48:25.849900+0000","last_deep_scrub":"261'402","last_deep_scrub_stamp":"2024-04-22T20:48:25.849900+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.849900+0000","objects_scrubbed":10,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:07.464040+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682319,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":541,"num_read_kb":21001,"num_write":1452,"num_write_kb":33571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.396156+0000","last_change":"2024-04-22T20:48:52.396156+0000","last_active":"2024-04-22T20:48:52.396156+0000","last_peered":"2024-04-22T20:48:52.396156+0000","last_clean":"2024-04-22T20:48:52.396156+0000","last_became_active":"2024-04-22T19:51:30.526469+0000","last_became_peered":"2024-04-22T19:51:30.526469+0000","last_unstale":"2024-04-22T20:48:52.396156+0000","last_undegraded":"2024-04-22T20:48:52.396156+0000","last_fullsized":"2024-04-22T20:48:52.396156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.396006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.396006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.396006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:26:59.620393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"261'313","reported_seq":906,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.871978+0000","last_change":"2024-04-22T20:48:55.872020+0000","last_active":"2024-04-22T20:48:55.871978+0000","last_peered":"2024-04-22T20:48:55.871978+0000","last_clean":"2024-04-22T20:48:55.871978+0000","last_became_active":"2024-04-22T19:51:32.540573+0000","last_became_peered":"2024-04-22T19:51:32.540573+0000","last_unstale":"2024-04-22T20:48:55.871978+0000","last_undegraded":"2024-04-22T20:48:55.871978+0000","last_fullsized":"2024-04-22T20:48:55.871978+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:55.871874+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:55.871874+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.871874+0000","objects_scrubbed":7,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:02:42.734441+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48433,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":496,"num_read_kb":1091,"num_write":394,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.665319+0000","last_change":"2024-04-22T20:48:47.665362+0000","last_active":"2024-04-22T20:48:47.665319+0000","last_peered":"2024-04-22T20:48:47.665319+0000","last_clean":"2024-04-22T20:48:47.665319+0000","last_became_active":"2024-04-22T19:51:34.553857+0000","last_became_peered":"2024-04-22T19:51:34.553857+0000","last_unstale":"2024-04-22T20:48:47.665319+0000","last_undegraded":"2024-04-22T20:48:47.665319+0000","last_fullsized":"2024-04-22T20:48:47.665319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.665221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.665221+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.665221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:00:40.709377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"261'157","reported_seq":667,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.377887+0000","last_change":"2024-04-22T20:48:39.377921+0000","last_active":"2024-04-22T20:48:39.377887+0000","last_peered":"2024-04-22T20:48:39.377887+0000","last_clean":"2024-04-22T20:48:39.377887+0000","last_became_active":"2024-04-22T19:51:36.575211+0000","last_became_peered":"2024-04-22T19:51:36.575211+0000","last_unstale":"2024-04-22T20:48:39.377887+0000","last_undegraded":"2024-04-22T20:48:39.377887+0000","last_fullsized":"2024-04-22T20:48:39.377887+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'157","last_scrub_stamp":"2024-04-22T20:48:39.377810+0000","last_deep_scrub":"261'157","last_deep_scrub_stamp":"2024-04-22T20:48:39.377810+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.377810+0000","objects_scrubbed":4,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:18:41.159874+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":158,"num_write":375,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"261'86","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.240691+0000","last_change":"2024-04-22T20:49:05.240723+0000","last_active":"2024-04-22T20:49:05.240691+0000","last_peered":"2024-04-22T20:49:05.240691+0000","last_clean":"2024-04-22T20:49:05.240691+0000","last_became_active":"2024-04-22T19:51:28.508145+0000","last_became_peered":"2024-04-22T19:51:28.508145+0000","last_unstale":"2024-04-22T20:49:05.240691+0000","last_undegraded":"2024-04-22T20:49:05.240691+0000","last_fullsized":"2024-04-22T20:49:05.240691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'86","last_scrub_stamp":"2024-04-22T20:49:05.240588+0000","last_deep_scrub":"261'86","last_deep_scrub_stamp":"2024-04-22T20:49:05.240588+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.240588+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:51:41.287365+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":376,"num_read_kb":320,"num_write":115,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"261'271","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.817431+0000","last_change":"2024-04-22T20:49:00.817466+0000","last_active":"2024-04-22T20:49:00.817431+0000","last_peered":"2024-04-22T20:49:00.817431+0000","last_clean":"2024-04-22T20:49:00.817431+0000","last_became_active":"2024-04-22T19:51:26.504301+0000","last_became_peered":"2024-04-22T19:51:26.504301+0000","last_unstale":"2024-04-22T20:49:00.817431+0000","last_undegraded":"2024-04-22T20:49:00.817431+0000","last_fullsized":"2024-04-22T20:49:00.817431+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'271","last_scrub_stamp":"2024-04-22T20:49:00.817344+0000","last_deep_scrub":"261'271","last_deep_scrub_stamp":"2024-04-22T20:49:00.817344+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.817344+0000","objects_scrubbed":4,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:03:08.734664+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":4555,"num_write":1089,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.927324+0000","last_change":"2024-04-22T20:48:51.927324+0000","last_active":"2024-04-22T20:48:51.927324+0000","last_peered":"2024-04-22T20:48:51.927324+0000","last_clean":"2024-04-22T20:48:51.927324+0000","last_became_active":"2024-04-22T19:51:30.524417+0000","last_became_peered":"2024-04-22T19:51:30.524417+0000","last_unstale":"2024-04-22T20:48:51.927324+0000","last_undegraded":"2024-04-22T20:48:51.927324+0000","last_fullsized":"2024-04-22T20:48:51.927324+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:51.927209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:51.927209+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.927209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:31:17.320634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"261'233","reported_seq":811,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.680733+0000","last_change":"2024-04-22T20:49:05.680783+0000","last_active":"2024-04-22T20:49:05.680733+0000","last_peered":"2024-04-22T20:49:05.680733+0000","last_clean":"2024-04-22T20:49:05.680733+0000","last_became_active":"2024-04-22T19:51:32.540659+0000","last_became_peered":"2024-04-22T19:51:32.540659+0000","last_unstale":"2024-04-22T20:49:05.680733+0000","last_undegraded":"2024-04-22T20:49:05.680733+0000","last_fullsized":"2024-04-22T20:49:05.680733+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'233","last_scrub_stamp":"2024-04-22T20:49:05.680614+0000","last_deep_scrub":"261'233","last_deep_scrub_stamp":"2024-04-22T20:49:05.680614+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.680614+0000","objects_scrubbed":4,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:25:49.392535+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16866,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":857,"num_write":302,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.574005+0000","last_change":"2024-04-22T20:48:36.574034+0000","last_active":"2024-04-22T20:48:36.574005+0000","last_peered":"2024-04-22T20:48:36.574005+0000","last_clean":"2024-04-22T20:48:36.574005+0000","last_became_active":"2024-04-22T19:51:34.552344+0000","last_became_peered":"2024-04-22T19:51:34.552344+0000","last_unstale":"2024-04-22T20:48:36.574005+0000","last_undegraded":"2024-04-22T20:48:36.574005+0000","last_fullsized":"2024-04-22T20:48:36.574005+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:36.573910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:36.573910+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.573910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:39:04.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"261'138","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.476371+0000","last_change":"2024-04-22T20:48:37.476415+0000","last_active":"2024-04-22T20:48:37.476371+0000","last_peered":"2024-04-22T20:48:37.476371+0000","last_clean":"2024-04-22T20:48:37.476371+0000","last_became_active":"2024-04-22T19:51:36.567594+0000","last_became_peered":"2024-04-22T19:51:36.567594+0000","last_unstale":"2024-04-22T20:48:37.476371+0000","last_undegraded":"2024-04-22T20:48:37.476371+0000","last_fullsized":"2024-04-22T20:48:37.476371+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'138","last_scrub_stamp":"2024-04-22T20:48:37.476271+0000","last_deep_scrub":"261'138","last_deep_scrub_stamp":"2024-04-22T20:48:37.476271+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.476271+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:27:27.543918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":214,"num_write":335,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"261'87","reported_seq":657,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.451797+0000","last_change":"2024-04-22T20:48:56.451837+0000","last_active":"2024-04-22T20:48:56.451797+0000","last_peered":"2024-04-22T20:48:56.451797+0000","last_clean":"2024-04-22T20:48:56.451797+0000","last_became_active":"2024-04-22T19:51:28.519542+0000","last_became_peered":"2024-04-22T19:51:28.519542+0000","last_unstale":"2024-04-22T20:48:56.451797+0000","last_undegraded":"2024-04-22T20:48:56.451797+0000","last_fullsized":"2024-04-22T20:48:56.451797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'87","last_scrub_stamp":"2024-04-22T20:48:56.451708+0000","last_deep_scrub":"261'87","last_deep_scrub_stamp":"2024-04-22T20:48:56.451708+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.451708+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:39.301084+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":347,"num_read_kb":277,"num_write":137,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"261'398","reported_seq":1251,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:14.336980+0000","last_change":"2024-04-22T20:49:14.337033+0000","last_active":"2024-04-22T20:49:14.336980+0000","last_peered":"2024-04-22T20:49:14.336980+0000","last_clean":"2024-04-22T20:49:14.336980+0000","last_became_active":"2024-04-22T19:51:26.499390+0000","last_became_peered":"2024-04-22T19:51:26.499390+0000","last_unstale":"2024-04-22T20:49:14.336980+0000","last_undegraded":"2024-04-22T20:49:14.336980+0000","last_fullsized":"2024-04-22T20:49:14.336980+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'398","last_scrub_stamp":"2024-04-22T20:49:14.336876+0000","last_deep_scrub":"261'398","last_deep_scrub_stamp":"2024-04-22T20:49:14.336876+0000","last_clean_scrub_stamp":"2024-04-22T20:49:14.336876+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:04:13.353440+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4204544,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1043,"num_read_kb":88369,"num_write":1413,"num_write_kb":41583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.330279+0000","last_change":"2024-04-22T20:48:52.330304+0000","last_active":"2024-04-22T20:48:52.330279+0000","last_peered":"2024-04-22T20:48:52.330279+0000","last_clean":"2024-04-22T20:48:52.330279+0000","last_became_active":"2024-04-22T19:51:30.526183+0000","last_became_peered":"2024-04-22T19:51:30.526183+0000","last_unstale":"2024-04-22T20:48:52.330279+0000","last_undegraded":"2024-04-22T20:48:52.330279+0000","last_fullsized":"2024-04-22T20:48:52.330279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.330187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.330187+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.330187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:30.153067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"261'251","reported_seq":825,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:18.295560+0000","last_change":"2024-04-22T20:49:18.295597+0000","last_active":"2024-04-22T20:49:18.295560+0000","last_peered":"2024-04-22T20:49:18.295560+0000","last_clean":"2024-04-22T20:49:18.295560+0000","last_became_active":"2024-04-22T19:51:32.538644+0000","last_became_peered":"2024-04-22T19:51:32.538644+0000","last_unstale":"2024-04-22T20:49:18.295560+0000","last_undegraded":"2024-04-22T20:49:18.295560+0000","last_fullsized":"2024-04-22T20:49:18.295560+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'251","last_scrub_stamp":"2024-04-22T20:49:18.295471+0000","last_deep_scrub":"261'251","last_deep_scrub_stamp":"2024-04-22T20:49:18.295471+0000","last_clean_scrub_stamp":"2024-04-22T20:49:18.295471+0000","objects_scrubbed":3,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:54.905784+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45694,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":402,"num_read_kb":955,"num_write":332,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.405108+0000","last_change":"2024-04-22T20:48:31.405108+0000","last_active":"2024-04-22T20:48:31.405108+0000","last_peered":"2024-04-22T20:48:31.405108+0000","last_clean":"2024-04-22T20:48:31.405108+0000","last_became_active":"2024-04-22T19:51:34.551085+0000","last_became_peered":"2024-04-22T19:51:34.551085+0000","last_unstale":"2024-04-22T20:48:31.405108+0000","last_undegraded":"2024-04-22T20:48:31.405108+0000","last_fullsized":"2024-04-22T20:48:31.405108+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:31.404990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:31.404990+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.404990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:00.257782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"261'187","reported_seq":920,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.442559+0000","last_change":"2024-04-22T20:48:42.442612+0000","last_active":"2024-04-22T20:48:42.442559+0000","last_peered":"2024-04-22T20:48:42.442559+0000","last_clean":"2024-04-22T20:48:42.442559+0000","last_became_active":"2024-04-22T19:51:36.569234+0000","last_became_peered":"2024-04-22T19:51:36.569234+0000","last_unstale":"2024-04-22T20:48:42.442559+0000","last_undegraded":"2024-04-22T20:48:42.442559+0000","last_fullsized":"2024-04-22T20:48:42.442559+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'187","last_scrub_stamp":"2024-04-22T20:48:42.442437+0000","last_deep_scrub":"261'187","last_deep_scrub_stamp":"2024-04-22T20:48:42.442437+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.442437+0000","objects_scrubbed":8,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:21:18.797025+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":419,"num_read_kb":454,"num_write":429,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"261'180","reported_seq":759,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.345583+0000","last_change":"2024-04-22T20:48:44.345608+0000","last_active":"2024-04-22T20:48:44.345583+0000","last_peered":"2024-04-22T20:48:44.345583+0000","last_clean":"2024-04-22T20:48:44.345583+0000","last_became_active":"2024-04-22T19:51:28.515506+0000","last_became_peered":"2024-04-22T19:51:28.515506+0000","last_unstale":"2024-04-22T20:48:44.345583+0000","last_undegraded":"2024-04-22T20:48:44.345583+0000","last_fullsized":"2024-04-22T20:48:44.345583+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'180","last_scrub_stamp":"2024-04-22T20:48:44.345489+0000","last_deep_scrub":"261'180","last_deep_scrub_stamp":"2024-04-22T20:48:44.345489+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.345489+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:48.784252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":678,"num_read_kb":529,"num_write":279,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"261'297","reported_seq":918,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.691195+0000","last_change":"2024-04-22T20:49:12.691256+0000","last_active":"2024-04-22T20:49:12.691195+0000","last_peered":"2024-04-22T20:49:12.691195+0000","last_clean":"2024-04-22T20:49:12.691195+0000","last_became_active":"2024-04-22T19:51:26.504604+0000","last_became_peered":"2024-04-22T19:51:26.504604+0000","last_unstale":"2024-04-22T20:49:12.691195+0000","last_undegraded":"2024-04-22T20:49:12.691195+0000","last_fullsized":"2024-04-22T20:49:12.691195+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'297","last_scrub_stamp":"2024-04-22T20:49:12.691089+0000","last_deep_scrub":"261'297","last_deep_scrub_stamp":"2024-04-22T20:49:12.691089+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.691089+0000","objects_scrubbed":9,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:01:34.115179+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242937,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":372,"num_read_kb":9007,"num_write":952,"num_write_kb":46630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"261'2","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.636032+0000","last_change":"2024-04-22T20:48:38.636081+0000","last_active":"2024-04-22T20:48:38.636032+0000","last_peered":"2024-04-22T20:48:38.636032+0000","last_clean":"2024-04-22T20:48:38.636032+0000","last_became_active":"2024-04-22T19:51:30.523338+0000","last_became_peered":"2024-04-22T19:51:30.523338+0000","last_unstale":"2024-04-22T20:48:38.636032+0000","last_undegraded":"2024-04-22T20:48:38.636032+0000","last_fullsized":"2024-04-22T20:48:38.636032+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:38.635935+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:38.635935+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.635935+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:28:44.233266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"261'313","reported_seq":908,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.337600+0000","last_change":"2024-04-22T20:48:37.337600+0000","last_active":"2024-04-22T20:48:37.337600+0000","last_peered":"2024-04-22T20:48:37.337600+0000","last_clean":"2024-04-22T20:48:37.337600+0000","last_became_active":"2024-04-22T19:51:32.537373+0000","last_became_peered":"2024-04-22T19:51:32.537373+0000","last_unstale":"2024-04-22T20:48:37.337600+0000","last_undegraded":"2024-04-22T20:48:37.337600+0000","last_fullsized":"2024-04-22T20:48:37.337600+0000","mapping_epoch":26,"log_start":"261'301","ondisk_log_start":"261'301","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:37.337420+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:37.337420+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.337420+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":301,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:00.481534+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49488,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":1080,"num_write":394,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.571825+0000","last_change":"2024-04-22T20:48:43.571825+0000","last_active":"2024-04-22T20:48:43.571825+0000","last_peered":"2024-04-22T20:48:43.571825+0000","last_clean":"2024-04-22T20:48:43.571825+0000","last_became_active":"2024-04-22T19:51:34.555709+0000","last_became_peered":"2024-04-22T19:51:34.555709+0000","last_unstale":"2024-04-22T20:48:43.571825+0000","last_undegraded":"2024-04-22T20:48:43.571825+0000","last_fullsized":"2024-04-22T20:48:43.571825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:43.571712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:43.571712+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.571712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:04:20.739611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"261'146","reported_seq":639,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.318417+0000","last_change":"2024-04-22T20:48:26.318450+0000","last_active":"2024-04-22T20:48:26.318417+0000","last_peered":"2024-04-22T20:48:26.318417+0000","last_clean":"2024-04-22T20:48:26.318417+0000","last_became_active":"2024-04-22T19:51:36.574012+0000","last_became_peered":"2024-04-22T19:51:36.574012+0000","last_unstale":"2024-04-22T20:48:26.318417+0000","last_undegraded":"2024-04-22T20:48:26.318417+0000","last_fullsized":"2024-04-22T20:48:26.318417+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'146","last_scrub_stamp":"2024-04-22T20:48:26.318331+0000","last_deep_scrub":"261'146","last_deep_scrub_stamp":"2024-04-22T20:48:26.318331+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.318331+0000","objects_scrubbed":2,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:20.701708+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":109,"num_write":348,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"261'372","reported_seq":1046,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.694389+0000","last_change":"2024-04-22T20:48:40.694419+0000","last_active":"2024-04-22T20:48:40.694389+0000","last_peered":"2024-04-22T20:48:40.694389+0000","last_clean":"2024-04-22T20:48:40.694389+0000","last_became_active":"2024-04-22T19:51:28.518339+0000","last_became_peered":"2024-04-22T19:51:28.518339+0000","last_unstale":"2024-04-22T20:48:40.694389+0000","last_undegraded":"2024-04-22T20:48:40.694389+0000","last_fullsized":"2024-04-22T20:48:40.694389+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'372","last_scrub_stamp":"2024-04-22T20:48:40.694294+0000","last_deep_scrub":"261'372","last_deep_scrub_stamp":"2024-04-22T20:48:40.694294+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.694294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:07.075387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1526,"num_read_kb":1211,"num_write":619,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"261'324","reported_seq":1032,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.592130+0000","last_change":"2024-04-22T20:48:55.592179+0000","last_active":"2024-04-22T20:48:55.592130+0000","last_peered":"2024-04-22T20:48:55.592130+0000","last_clean":"2024-04-22T20:48:55.592130+0000","last_became_active":"2024-04-22T19:51:26.502773+0000","last_became_peered":"2024-04-22T19:51:26.502773+0000","last_unstale":"2024-04-22T20:48:55.592130+0000","last_undegraded":"2024-04-22T20:48:55.592130+0000","last_fullsized":"2024-04-22T20:48:55.592130+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:48:55.592027+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:48:55.592027+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.592027+0000","objects_scrubbed":7,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:38.053119+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4371014,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":594,"num_read_kb":14386,"num_write":1094,"num_write_kb":17858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"261'524","reported_seq":1296,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.877481+0000","last_change":"2024-04-22T20:48:26.877525+0000","last_active":"2024-04-22T20:48:26.877481+0000","last_peered":"2024-04-22T20:48:26.877481+0000","last_clean":"2024-04-22T20:48:26.877481+0000","last_became_active":"2024-04-22T19:52:04.617036+0000","last_became_peered":"2024-04-22T19:52:04.617036+0000","last_unstale":"2024-04-22T20:48:26.877481+0000","last_undegraded":"2024-04-22T20:48:26.877481+0000","last_fullsized":"2024-04-22T20:48:26.877481+0000","mapping_epoch":33,"log_start":"261'500","ondisk_log_start":"261'500","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'524","last_scrub_stamp":"2024-04-22T20:48:26.877358+0000","last_deep_scrub":"261'524","last_deep_scrub_stamp":"2024-04-22T20:48:26.877358+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.877358+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":500,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:34:00.586382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2230,"num_read_kb":1847,"num_write":935,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"261'286","reported_seq":934,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.553956+0000","last_change":"2024-04-22T20:48:38.553990+0000","last_active":"2024-04-22T20:48:38.553956+0000","last_peered":"2024-04-22T20:48:38.553956+0000","last_clean":"2024-04-22T20:48:38.553956+0000","last_became_active":"2024-04-22T19:51:26.495948+0000","last_became_peered":"2024-04-22T19:51:26.495948+0000","last_unstale":"2024-04-22T20:48:38.553956+0000","last_undegraded":"2024-04-22T20:48:38.553956+0000","last_fullsized":"2024-04-22T20:48:38.553956+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'286","last_scrub_stamp":"2024-04-22T20:48:38.553878+0000","last_deep_scrub":"261'286","last_deep_scrub_stamp":"2024-04-22T20:48:38.553878+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.553878+0000","objects_scrubbed":6,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:12.594083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180512,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":454,"num_read_kb":10117,"num_write":1004,"num_write_kb":22835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"261'209","reported_seq":851,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.973501+0000","last_change":"2024-04-22T20:48:44.973554+0000","last_active":"2024-04-22T20:48:44.973501+0000","last_peered":"2024-04-22T20:48:44.973501+0000","last_clean":"2024-04-22T20:48:44.973501+0000","last_became_active":"2024-04-22T19:51:28.513026+0000","last_became_peered":"2024-04-22T19:51:28.513026+0000","last_unstale":"2024-04-22T20:48:44.973501+0000","last_undegraded":"2024-04-22T20:48:44.973501+0000","last_fullsized":"2024-04-22T20:48:44.973501+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'209","last_scrub_stamp":"2024-04-22T20:48:44.973375+0000","last_deep_scrub":"261'209","last_deep_scrub_stamp":"2024-04-22T20:48:44.973375+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.973375+0000","objects_scrubbed":1,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:59.805690+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":827,"num_read_kb":665,"num_write":342,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"261'313","reported_seq":941,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.835329+0000","last_change":"2024-04-22T20:49:04.835404+0000","last_active":"2024-04-22T20:49:04.835329+0000","last_peered":"2024-04-22T20:49:04.835329+0000","last_clean":"2024-04-22T20:49:04.835329+0000","last_became_active":"2024-04-22T19:51:26.506748+0000","last_became_peered":"2024-04-22T19:51:26.506748+0000","last_unstale":"2024-04-22T20:49:04.835329+0000","last_undegraded":"2024-04-22T20:49:04.835329+0000","last_fullsized":"2024-04-22T20:49:04.835329+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:49:04.835221+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:49:04.835221+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.835221+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:46.453041+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":416,"num_read_kb":12678,"num_write":1099,"num_write_kb":41347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"261'4175","reported_seq":4800,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.586185+0000","last_change":"2024-04-22T20:48:56.586215+0000","last_active":"2024-04-22T20:48:56.586185+0000","last_peered":"2024-04-22T20:48:56.586185+0000","last_clean":"2024-04-22T20:48:56.586185+0000","last_became_active":"2024-04-22T19:51:28.511312+0000","last_became_peered":"2024-04-22T19:51:28.511312+0000","last_unstale":"2024-04-22T20:48:56.586185+0000","last_undegraded":"2024-04-22T20:48:56.586185+0000","last_fullsized":"2024-04-22T20:48:56.586185+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4175","last_scrub_stamp":"2024-04-22T20:48:56.586085+0000","last_deep_scrub":"261'4175","last_deep_scrub_stamp":"2024-04-22T20:48:56.586085+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.586085+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":3000,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:15.045023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14718,"num_read_kb":11919,"num_write":6278,"num_write_kb":3133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"261'312","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.452280+0000","last_change":"2024-04-22T20:48:28.452280+0000","last_active":"2024-04-22T20:48:28.452280+0000","last_peered":"2024-04-22T20:48:28.452280+0000","last_clean":"2024-04-22T20:48:28.452280+0000","last_became_active":"2024-04-22T19:52:05.114734+0000","last_became_peered":"2024-04-22T19:52:05.114734+0000","last_unstale":"2024-04-22T20:48:28.452280+0000","last_undegraded":"2024-04-22T20:48:28.452280+0000","last_fullsized":"2024-04-22T20:48:28.452280+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'312","last_scrub_stamp":"2024-04-22T20:48:28.452109+0000","last_deep_scrub":"261'312","last_deep_scrub_stamp":"2024-04-22T20:48:28.452109+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.452109+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:41:21.599729+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":5518,"num_write":1088,"num_write_kb":21667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.23","version":"261'171","reported_seq":861,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.466280+0000","last_change":"2024-04-22T20:48:57.466312+0000","last_active":"2024-04-22T20:48:57.466280+0000","last_peered":"2024-04-22T20:48:57.466280+0000","last_clean":"2024-04-22T20:48:57.466280+0000","last_became_active":"2024-04-22T19:51:28.516125+0000","last_became_peered":"2024-04-22T19:51:28.516125+0000","last_unstale":"2024-04-22T20:48:57.466280+0000","last_undegraded":"2024-04-22T20:48:57.466280+0000","last_fullsized":"2024-04-22T20:48:57.466280+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'171","last_scrub_stamp":"2024-04-22T20:48:57.466192+0000","last_deep_scrub":"261'171","last_deep_scrub_stamp":"2024-04-22T20:48:57.466192+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.466192+0000","objects_scrubbed":1,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:22:48.965827+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":741,"num_read_kb":616,"num_write":243,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"261'368","reported_seq":1024,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.429047+0000","last_change":"2024-04-22T20:48:26.429162+0000","last_active":"2024-04-22T20:48:26.429047+0000","last_peered":"2024-04-22T20:48:26.429047+0000","last_clean":"2024-04-22T20:48:26.429047+0000","last_became_active":"2024-04-22T19:52:04.619286+0000","last_became_peered":"2024-04-22T19:52:04.619286+0000","last_unstale":"2024-04-22T20:48:26.429047+0000","last_undegraded":"2024-04-22T20:48:26.429047+0000","last_fullsized":"2024-04-22T20:48:26.429047+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'368","last_scrub_stamp":"2024-04-22T20:48:26.428925+0000","last_deep_scrub":"261'368","last_deep_scrub_stamp":"2024-04-22T20:48:26.428925+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.428925+0000","objects_scrubbed":9,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:38:16.414530+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4719706,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":588,"num_read_kb":69897,"num_write":1236,"num_write_kb":36151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"261'122","reported_seq":735,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.272617+0000","last_change":"2024-04-22T20:48:46.272648+0000","last_active":"2024-04-22T20:48:46.272617+0000","last_peered":"2024-04-22T20:48:46.272617+0000","last_clean":"2024-04-22T20:48:46.272617+0000","last_became_active":"2024-04-22T19:51:28.516437+0000","last_became_peered":"2024-04-22T19:51:28.516437+0000","last_unstale":"2024-04-22T20:48:46.272617+0000","last_undegraded":"2024-04-22T20:48:46.272617+0000","last_fullsized":"2024-04-22T20:48:46.272617+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'122","last_scrub_stamp":"2024-04-22T20:48:46.272512+0000","last_deep_scrub":"261'122","last_deep_scrub_stamp":"2024-04-22T20:48:46.272512+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.272512+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:54:33.342867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":423,"num_write":178,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"261'317","reported_seq":1060,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.585793+0000","last_change":"2024-04-22T20:48:53.585833+0000","last_active":"2024-04-22T20:48:53.585793+0000","last_peered":"2024-04-22T20:48:53.585793+0000","last_clean":"2024-04-22T20:48:53.585793+0000","last_became_active":"2024-04-22T19:51:26.498039+0000","last_became_peered":"2024-04-22T19:51:26.498039+0000","last_unstale":"2024-04-22T20:48:53.585793+0000","last_undegraded":"2024-04-22T20:48:53.585793+0000","last_fullsized":"2024-04-22T20:48:53.585793+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:53.585698+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:53.585698+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.585698+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:05:32.063217+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194319,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":766,"num_read_kb":22382,"num_write":1253,"num_write_kb":18139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"261'199","reported_seq":857,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.407111+0000","last_change":"2024-04-22T20:49:00.407135+0000","last_active":"2024-04-22T20:49:00.407111+0000","last_peered":"2024-04-22T20:49:00.407111+0000","last_clean":"2024-04-22T20:49:00.407111+0000","last_became_active":"2024-04-22T19:51:28.520612+0000","last_became_peered":"2024-04-22T19:51:28.520612+0000","last_unstale":"2024-04-22T20:49:00.407111+0000","last_undegraded":"2024-04-22T20:49:00.407111+0000","last_fullsized":"2024-04-22T20:49:00.407111+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'199","last_scrub_stamp":"2024-04-22T20:49:00.407015+0000","last_deep_scrub":"261'199","last_deep_scrub_stamp":"2024-04-22T20:49:00.407015+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.407015+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:11:38.305157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":735,"num_read_kb":593,"num_write":262,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"261'365","reported_seq":1022,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.558974+0000","last_change":"2024-04-22T20:48:52.559103+0000","last_active":"2024-04-22T20:48:52.558974+0000","last_peered":"2024-04-22T20:48:52.558974+0000","last_clean":"2024-04-22T20:48:52.558974+0000","last_became_active":"2024-04-22T19:51:26.498845+0000","last_became_peered":"2024-04-22T19:51:26.498845+0000","last_unstale":"2024-04-22T20:48:52.558974+0000","last_undegraded":"2024-04-22T20:48:52.558974+0000","last_fullsized":"2024-04-22T20:48:52.558974+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'365","last_scrub_stamp":"2024-04-22T20:48:52.558851+0000","last_deep_scrub":"261'365","last_deep_scrub_stamp":"2024-04-22T20:48:52.558851+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.558851+0000","objects_scrubbed":9,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:38:15.742114+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050721,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":506,"num_read_kb":33263,"num_write":1241,"num_write_kb":30860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"261'133","reported_seq":775,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.610209+0000","last_change":"2024-04-22T20:48:37.610234+0000","last_active":"2024-04-22T20:48:37.610209+0000","last_peered":"2024-04-22T20:48:37.610209+0000","last_clean":"2024-04-22T20:48:37.610209+0000","last_became_active":"2024-04-22T19:51:28.519669+0000","last_became_peered":"2024-04-22T19:51:28.519669+0000","last_unstale":"2024-04-22T20:48:37.610209+0000","last_undegraded":"2024-04-22T20:48:37.610209+0000","last_fullsized":"2024-04-22T20:48:37.610209+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'133","last_scrub_stamp":"2024-04-22T20:48:37.610118+0000","last_deep_scrub":"261'133","last_deep_scrub_stamp":"2024-04-22T20:48:37.610118+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.610118+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:03:13.662918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":560,"num_read_kb":466,"num_write":196,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"261'330","reported_seq":935,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.490447+0000","last_change":"2024-04-22T20:48:29.490501+0000","last_active":"2024-04-22T20:48:29.490447+0000","last_peered":"2024-04-22T20:48:29.490447+0000","last_clean":"2024-04-22T20:48:29.490447+0000","last_became_active":"2024-04-22T19:52:04.617649+0000","last_became_peered":"2024-04-22T19:52:04.617649+0000","last_unstale":"2024-04-22T20:48:29.490447+0000","last_undegraded":"2024-04-22T20:48:29.490447+0000","last_fullsized":"2024-04-22T20:48:29.490447+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'330","last_scrub_stamp":"2024-04-22T20:48:29.490328+0000","last_deep_scrub":"261'330","last_deep_scrub_stamp":"2024-04-22T20:48:29.490328+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.490328+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:55:48.835393+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050919,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":29074,"num_write":1030,"num_write_kb":22627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"261'128","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.677849+0000","last_change":"2024-04-22T20:48:51.677899+0000","last_active":"2024-04-22T20:48:51.677849+0000","last_peered":"2024-04-22T20:48:51.677849+0000","last_clean":"2024-04-22T20:48:51.677849+0000","last_became_active":"2024-04-22T19:51:28.521516+0000","last_became_peered":"2024-04-22T19:51:28.521516+0000","last_unstale":"2024-04-22T20:48:51.677849+0000","last_undegraded":"2024-04-22T20:48:51.677849+0000","last_fullsized":"2024-04-22T20:48:51.677849+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'128","last_scrub_stamp":"2024-04-22T20:48:51.677734+0000","last_deep_scrub":"261'128","last_deep_scrub_stamp":"2024-04-22T20:48:51.677734+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.677734+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:24.647580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":542,"num_read_kb":442,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"261'338","reported_seq":1027,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.972384+0000","last_change":"2024-04-22T20:48:52.972424+0000","last_active":"2024-04-22T20:48:52.972384+0000","last_peered":"2024-04-22T20:48:52.972384+0000","last_clean":"2024-04-22T20:48:52.972384+0000","last_became_active":"2024-04-22T19:51:26.507843+0000","last_became_peered":"2024-04-22T19:51:26.507843+0000","last_unstale":"2024-04-22T20:48:52.972384+0000","last_undegraded":"2024-04-22T20:48:52.972384+0000","last_fullsized":"2024-04-22T20:48:52.972384+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'338","last_scrub_stamp":"2024-04-22T20:48:52.972288+0000","last_deep_scrub":"261'338","last_deep_scrub_stamp":"2024-04-22T20:48:52.972288+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.972288+0000","objects_scrubbed":6,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:36.857751+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5531396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":136369,"num_write":1233,"num_write_kb":55788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"261'235","reported_seq":869,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.241423+0000","last_change":"2024-04-22T20:48:47.241450+0000","last_active":"2024-04-22T20:48:47.241423+0000","last_peered":"2024-04-22T20:48:47.241423+0000","last_clean":"2024-04-22T20:48:47.241423+0000","last_became_active":"2024-04-22T19:51:28.515164+0000","last_became_peered":"2024-04-22T19:51:28.515164+0000","last_unstale":"2024-04-22T20:48:47.241423+0000","last_undegraded":"2024-04-22T20:48:47.241423+0000","last_fullsized":"2024-04-22T20:48:47.241423+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'235","last_scrub_stamp":"2024-04-22T20:48:47.241303+0000","last_deep_scrub":"261'235","last_deep_scrub_stamp":"2024-04-22T20:48:47.241303+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.241303+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:24.721788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":950,"num_read_kb":750,"num_write":364,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"261'502","reported_seq":1211,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.319386+0000","last_change":"2024-04-22T20:49:11.319443+0000","last_active":"2024-04-22T20:49:11.319386+0000","last_peered":"2024-04-22T20:49:11.319386+0000","last_clean":"2024-04-22T20:49:11.319386+0000","last_became_active":"2024-04-22T19:51:26.493860+0000","last_became_peered":"2024-04-22T19:51:26.493860+0000","last_unstale":"2024-04-22T20:49:11.319386+0000","last_undegraded":"2024-04-22T20:49:11.319386+0000","last_fullsized":"2024-04-22T20:49:11.319386+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'502","last_scrub_stamp":"2024-04-22T20:49:11.319268+0000","last_deep_scrub":"261'502","last_deep_scrub_stamp":"2024-04-22T20:49:11.319268+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.319268+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":400,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:40:00.636759+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6292618,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":951,"num_read_kb":13872,"num_write":1439,"num_write_kb":37693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"261'252","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.584503+0000","last_change":"2024-04-22T20:48:44.584554+0000","last_active":"2024-04-22T20:48:44.584503+0000","last_peered":"2024-04-22T20:48:44.584503+0000","last_clean":"2024-04-22T20:48:44.584503+0000","last_became_active":"2024-04-22T19:51:28.515547+0000","last_became_peered":"2024-04-22T19:51:28.515547+0000","last_unstale":"2024-04-22T20:48:44.584503+0000","last_undegraded":"2024-04-22T20:48:44.584503+0000","last_fullsized":"2024-04-22T20:48:44.584503+0000","mapping_epoch":22,"log_start":"247'200","ondisk_log_start":"247'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'252","last_scrub_stamp":"2024-04-22T20:48:44.584399+0000","last_deep_scrub":"261'252","last_deep_scrub_stamp":"2024-04-22T20:48:44.584399+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.584399+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:23:23.642516+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1120,"num_read_kb":973,"num_write":365,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25681,"num_omap_keys":101,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"261'324","reported_seq":963,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.831693+0000","last_change":"2024-04-22T20:48:58.831759+0000","last_active":"2024-04-22T20:48:58.831693+0000","last_peered":"2024-04-22T20:48:58.831693+0000","last_clean":"2024-04-22T20:48:58.831693+0000","last_became_active":"2024-04-22T19:51:26.500566+0000","last_became_peered":"2024-04-22T19:51:26.500566+0000","last_unstale":"2024-04-22T20:48:58.831693+0000","last_undegraded":"2024-04-22T20:48:58.831693+0000","last_fullsized":"2024-04-22T20:48:58.831693+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:48:58.831593+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:48:58.831593+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.831593+0000","objects_scrubbed":7,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:41:02.913836+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283749,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":424,"num_read_kb":45844,"num_write":1120,"num_write_kb":42819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"261'152","reported_seq":759,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.297232+0000","last_change":"2024-04-22T20:48:43.297263+0000","last_active":"2024-04-22T20:48:43.297232+0000","last_peered":"2024-04-22T20:48:43.297232+0000","last_clean":"2024-04-22T20:48:43.297232+0000","last_became_active":"2024-04-22T19:51:28.515764+0000","last_became_peered":"2024-04-22T19:51:28.515764+0000","last_unstale":"2024-04-22T20:48:43.297232+0000","last_undegraded":"2024-04-22T20:48:43.297232+0000","last_fullsized":"2024-04-22T20:48:43.297232+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:43.297146+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:43.297146+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.297146+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:48.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":633,"num_read_kb":515,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"261'261","reported_seq":879,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.735739+0000","last_change":"2024-04-22T20:49:06.735795+0000","last_active":"2024-04-22T20:49:06.735739+0000","last_peered":"2024-04-22T20:49:06.735739+0000","last_clean":"2024-04-22T20:49:06.735739+0000","last_became_active":"2024-04-22T19:51:26.512467+0000","last_became_peered":"2024-04-22T19:51:26.512467+0000","last_unstale":"2024-04-22T20:49:06.735739+0000","last_undegraded":"2024-04-22T20:49:06.735739+0000","last_fullsized":"2024-04-22T20:49:06.735739+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'261","last_scrub_stamp":"2024-04-22T20:49:06.735656+0000","last_deep_scrub":"261'261","last_deep_scrub_stamp":"2024-04-22T20:49:06.735656+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.735656+0000","objects_scrubbed":8,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:48:01.033769+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11536625,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":377,"num_read_kb":17908,"num_write":933,"num_write_kb":43703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"261'27","reported_seq":563,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.366682+0000","last_change":"2024-04-22T20:48:55.366707+0000","last_active":"2024-04-22T20:48:55.366682+0000","last_peered":"2024-04-22T20:48:55.366682+0000","last_clean":"2024-04-22T20:48:55.366682+0000","last_became_active":"2024-04-22T19:51:28.510091+0000","last_became_peered":"2024-04-22T19:51:28.510091+0000","last_unstale":"2024-04-22T20:48:55.366682+0000","last_undegraded":"2024-04-22T20:48:55.366682+0000","last_fullsized":"2024-04-22T20:48:55.366682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'27","last_scrub_stamp":"2024-04-22T20:48:55.366594+0000","last_deep_scrub":"261'27","last_deep_scrub_stamp":"2024-04-22T20:48:55.366594+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.366594+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:48:05.352437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":95,"num_write":39,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"261'340","reported_seq":996,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.582045+0000","last_change":"2024-04-22T20:48:53.582088+0000","last_active":"2024-04-22T20:48:53.582045+0000","last_peered":"2024-04-22T20:48:53.582045+0000","last_clean":"2024-04-22T20:48:53.582045+0000","last_became_active":"2024-04-22T19:51:26.501700+0000","last_became_peered":"2024-04-22T19:51:26.501700+0000","last_unstale":"2024-04-22T20:48:53.582045+0000","last_undegraded":"2024-04-22T20:48:53.582045+0000","last_fullsized":"2024-04-22T20:48:53.582045+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'340","last_scrub_stamp":"2024-04-22T20:48:53.581939+0000","last_deep_scrub":"261'340","last_deep_scrub_stamp":"2024-04-22T20:48:53.581939+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.581939+0000","objects_scrubbed":7,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:58.999952+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231939,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":506,"num_read_kb":6050,"num_write":1241,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"261'54","reported_seq":612,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.710469+0000","last_change":"2024-04-22T20:49:04.710497+0000","last_active":"2024-04-22T20:49:04.710469+0000","last_peered":"2024-04-22T20:49:04.710469+0000","last_clean":"2024-04-22T20:49:04.710469+0000","last_became_active":"2024-04-22T19:51:28.508768+0000","last_became_peered":"2024-04-22T19:51:28.508768+0000","last_unstale":"2024-04-22T20:49:04.710469+0000","last_undegraded":"2024-04-22T20:49:04.710469+0000","last_fullsized":"2024-04-22T20:49:04.710469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'54","last_scrub_stamp":"2024-04-22T20:49:04.710365+0000","last_deep_scrub":"261'54","last_deep_scrub_stamp":"2024-04-22T20:49:04.710365+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.710365+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:28:08.981902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":238,"num_read_kb":196,"num_write":80,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"261'355","reported_seq":1003,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.613696+0000","last_change":"2024-04-22T20:49:09.613768+0000","last_active":"2024-04-22T20:49:09.613696+0000","last_peered":"2024-04-22T20:49:09.613696+0000","last_clean":"2024-04-22T20:49:09.613696+0000","last_became_active":"2024-04-22T19:51:26.497156+0000","last_became_peered":"2024-04-22T19:51:26.497156+0000","last_unstale":"2024-04-22T20:49:09.613696+0000","last_undegraded":"2024-04-22T20:49:09.613696+0000","last_fullsized":"2024-04-22T20:49:09.613696+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'355","last_scrub_stamp":"2024-04-22T20:49:09.613581+0000","last_deep_scrub":"261'355","last_deep_scrub_stamp":"2024-04-22T20:49:09.613581+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.613581+0000","objects_scrubbed":8,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:28:49.999075+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437245,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":35895,"num_write":1244,"num_write_kb":35286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"261'4050","reported_seq":4621,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.446481+0000","last_change":"2024-04-22T20:48:59.446504+0000","last_active":"2024-04-22T20:48:59.446481+0000","last_peered":"2024-04-22T20:48:59.446481+0000","last_clean":"2024-04-22T20:48:59.446481+0000","last_became_active":"2024-04-22T19:51:28.514780+0000","last_became_peered":"2024-04-22T19:51:28.514780+0000","last_unstale":"2024-04-22T20:48:59.446481+0000","last_undegraded":"2024-04-22T20:48:59.446481+0000","last_fullsized":"2024-04-22T20:48:59.446481+0000","mapping_epoch":22,"log_start":"261'4000","ondisk_log_start":"261'4000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4050","last_scrub_stamp":"2024-04-22T20:48:59.446394+0000","last_deep_scrub":"261'4050","last_deep_scrub_stamp":"2024-04-22T20:48:59.446394+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.446394+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":3000,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:24:00.775234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14224,"num_read_kb":11535,"num_write":6070,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"261'302","reported_seq":1070,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.563705+0000","last_change":"2024-04-22T20:48:39.563748+0000","last_active":"2024-04-22T20:48:39.563705+0000","last_peered":"2024-04-22T20:48:39.563705+0000","last_clean":"2024-04-22T20:48:39.563705+0000","last_became_active":"2024-04-22T19:51:26.496214+0000","last_became_peered":"2024-04-22T19:51:26.496214+0000","last_unstale":"2024-04-22T20:48:39.563705+0000","last_undegraded":"2024-04-22T20:48:39.563705+0000","last_fullsized":"2024-04-22T20:48:39.563705+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'302","last_scrub_stamp":"2024-04-22T20:48:39.563612+0000","last_deep_scrub":"261'302","last_deep_scrub_stamp":"2024-04-22T20:48:39.563612+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.563612+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:41:49.634270+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1338117,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":584,"num_read_kb":13796,"num_write":1181,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"261'298","reported_seq":988,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.430136+0000","last_change":"2024-04-22T20:48:57.430161+0000","last_active":"2024-04-22T20:48:57.430136+0000","last_peered":"2024-04-22T20:48:57.430136+0000","last_clean":"2024-04-22T20:48:57.430136+0000","last_became_active":"2024-04-22T19:51:28.508681+0000","last_became_peered":"2024-04-22T19:51:28.508681+0000","last_unstale":"2024-04-22T20:48:57.430136+0000","last_undegraded":"2024-04-22T20:48:57.430136+0000","last_fullsized":"2024-04-22T20:48:57.430136+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'298","last_scrub_stamp":"2024-04-22T20:48:57.430053+0000","last_deep_scrub":"261'298","last_deep_scrub_stamp":"2024-04-22T20:48:57.430053+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.430053+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:35.213640+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1231,"num_read_kb":1000,"num_write":489,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"261'282","reported_seq":900,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.830889+0000","last_change":"2024-04-22T20:49:02.831005+0000","last_active":"2024-04-22T20:49:02.830889+0000","last_peered":"2024-04-22T20:49:02.830889+0000","last_clean":"2024-04-22T20:49:02.830889+0000","last_became_active":"2024-04-22T19:51:26.502438+0000","last_became_peered":"2024-04-22T19:51:26.502438+0000","last_unstale":"2024-04-22T20:49:02.830889+0000","last_undegraded":"2024-04-22T20:49:02.830889+0000","last_fullsized":"2024-04-22T20:49:02.830889+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'282","last_scrub_stamp":"2024-04-22T20:49:02.830773+0000","last_deep_scrub":"261'282","last_deep_scrub_stamp":"2024-04-22T20:49:02.830773+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.830773+0000","objects_scrubbed":7,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:43:57.450915+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194332,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":386,"num_read_kb":36638,"num_write":936,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"261'374","reported_seq":1262,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.841410+0000","last_change":"2024-04-22T20:49:03.841470+0000","last_active":"2024-04-22T20:49:03.841410+0000","last_peered":"2024-04-22T20:49:03.841410+0000","last_clean":"2024-04-22T20:49:03.841410+0000","last_became_active":"2024-04-22T19:51:26.504878+0000","last_became_peered":"2024-04-22T19:51:26.504878+0000","last_unstale":"2024-04-22T20:49:03.841410+0000","last_undegraded":"2024-04-22T20:49:03.841410+0000","last_fullsized":"2024-04-22T20:49:03.841410+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'374","last_scrub_stamp":"2024-04-22T20:49:03.841311+0000","last_deep_scrub":"261'374","last_deep_scrub_stamp":"2024-04-22T20:49:03.841311+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.841311+0000","objects_scrubbed":8,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:34:22.018993+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5353839,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1018,"num_read_kb":55238,"num_write":1407,"num_write_kb":52838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"261'127","reported_seq":737,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.665636+0000","last_change":"2024-04-22T20:48:52.665674+0000","last_active":"2024-04-22T20:48:52.665636+0000","last_peered":"2024-04-22T20:48:52.665636+0000","last_clean":"2024-04-22T20:48:52.665636+0000","last_became_active":"2024-04-22T19:51:28.521372+0000","last_became_peered":"2024-04-22T19:51:28.521372+0000","last_unstale":"2024-04-22T20:48:52.665636+0000","last_undegraded":"2024-04-22T20:48:52.665636+0000","last_fullsized":"2024-04-22T20:48:52.665636+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'127","last_scrub_stamp":"2024-04-22T20:48:52.665538+0000","last_deep_scrub":"261'127","last_deep_scrub_stamp":"2024-04-22T20:48:52.665538+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.665538+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:58:05.220984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":525,"num_read_kb":437,"num_write":198,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"261'287","reported_seq":950,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.495078+0000","last_change":"2024-04-22T20:48:28.495119+0000","last_active":"2024-04-22T20:48:28.495078+0000","last_peered":"2024-04-22T20:48:28.495078+0000","last_clean":"2024-04-22T20:48:28.495078+0000","last_became_active":"2024-04-22T19:52:04.618273+0000","last_became_peered":"2024-04-22T19:52:04.618273+0000","last_unstale":"2024-04-22T20:48:28.495078+0000","last_undegraded":"2024-04-22T20:48:28.495078+0000","last_fullsized":"2024-04-22T20:48:28.495078+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'287","last_scrub_stamp":"2024-04-22T20:48:28.494984+0000","last_deep_scrub":"261'287","last_deep_scrub_stamp":"2024-04-22T20:48:28.494984+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.494984+0000","objects_scrubbed":7,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:31:35.950376+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1439002,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":73466,"num_write":989,"num_write_kb":40708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"261'95","reported_seq":657,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.938718+0000","last_change":"2024-04-22T20:48:42.938763+0000","last_active":"2024-04-22T20:48:42.938718+0000","last_peered":"2024-04-22T20:48:42.938718+0000","last_clean":"2024-04-22T20:48:42.938718+0000","last_became_active":"2024-04-22T19:51:28.513450+0000","last_became_peered":"2024-04-22T19:51:28.513450+0000","last_unstale":"2024-04-22T20:48:42.938718+0000","last_undegraded":"2024-04-22T20:48:42.938718+0000","last_fullsized":"2024-04-22T20:48:42.938718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'95","last_scrub_stamp":"2024-04-22T20:48:42.938629+0000","last_deep_scrub":"261'95","last_deep_scrub_stamp":"2024-04-22T20:48:42.938629+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.938629+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:02:38.964219+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":382,"num_read_kb":302,"num_write":152,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5385,"num_read_kb":3813,"num_write":2124,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7203,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19082,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":13196,"num_read_kb":13004,"num_write":15956,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4092,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":38164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1949,"ondisk_log_size":1949,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":378,"ondisk_log_size":378,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1476154,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":17118,"num_read_kb":35541,"num_write":14403,"num_write_kb":11173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":650962,"num_omap_keys":1510,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1591,"ondisk_log_size":1591,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1622,"num_read_kb":1927,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":85960,"num_read_kb":69763,"num_write":34673,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43052,"num_omap_keys":180,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3910,"ondisk_log_size":3910,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":231670890,"num_objects":469,"num_object_clones":0,"num_object_copies":1407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":469,"num_whiteouts":0,"num_read":35099,"num_read_kb":2217057,"num_write":77336,"num_write_kb":2138644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":349753344,"data_stored":349753344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3387,"ondisk_log_size":3387,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":169,"num_read_kb":196064,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":2,"acting":2,"num_store_stats":2},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":185784,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575550,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":339124,"kb_used_data":51116,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":94032716,"statfs":{"total":96636764160,"available":96289501184,"internally_reserved":0,"allocated":52342784,"data_stored":50369134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":294910410},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575552,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316900,"kb_used_data":46940,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":94054940,"statfs":{"total":96636764160,"available":96312258560,"internally_reserved":0,"allocated":48066560,"data_stored":46141552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":276429254},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575551,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175548,"kb_used_data":54196,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94196292,"statfs":{"total":96636764160,"available":96457003008,"internally_reserved":0,"allocated":55496704,"data_stored":53487509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575552,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163576,"kb_used_data":51888,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94208264,"statfs":{"total":96636764160,"available":96469262336,"internally_reserved":0,"allocated":53133312,"data_stored":51218557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575552,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":336520,"kb_used_data":62464,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94035320,"statfs":{"total":96636764160,"available":96292167680,"internally_reserved":0,"allocated":63963136,"data_stored":62012545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":280623560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575553,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144372,"kb_used_data":59116,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94227468,"statfs":{"total":96636764160,"available":96488927232,"internally_reserved":0,"allocated":60534784,"data_stored":58629075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575550,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":188908,"kb_used_data":53668,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94182932,"statfs":{"total":96636764160,"available":96443322368,"internally_reserved":0,"allocated":54956032,"data_stored":53018011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575552,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309552,"kb_used_data":61800,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":94062288,"statfs":{"total":96636764160,"available":96319782912,"internally_reserved":0,"allocated":63283200,"data_stored":61318787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":253688262},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":48562176,"data_stored":48562176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44916736,"data_stored":44916736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37486592,"data_stored":37486592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50307072,"data_stored":50307072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36368384,"data_stored":36368384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44924928,"data_stored":44924928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":43872256,"data_stored":43872256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":43315200,"data_stored":43315200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":5948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":4195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":3595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":6129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":4904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":3145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13631488,"data_stored":13631488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":9437184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T20:49:26.893 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-04-22T20:19:15.767262+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=19, tm_sec=15, tm_wday=0, tm_yday=113, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=22, tm_hour=20, tm_min=48, tm_sec=20, tm_wday=0, tm_yday=113, tm_isdst=0) 2024-04-22T20:49:26.897 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-22T20:49:46.898 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-22T20:49:47.239 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-22T20:49:47.251 INFO:teuthology.orchestra.run.smithi136.stderr:dumped all 2024-04-22T20:49:47.254 INFO:teuthology.orchestra.run.smithi136.stdout:{"pg_ready":true,"pg_map":{"version":2012,"stamp":"2024-04-22T20:49:46.364062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":267312719,"num_objects":825,"num_object_clones":0,"num_object_copies":2119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":158844,"num_read_kb":2723065,"num_write":145049,"num_write_kb":2429940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706437,"num_omap_keys":1741,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12773,"ondisk_log_size":12773,"up":850,"acting":850,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":850,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1974500,"kb_used_data":441188,"kb_used_omap":12,"kb_used_meta":1533235,"kb_avail":753000220,"statfs":{"total":773094113280,"available":771072225280,"internally_reserved":0,"allocated":451776512,"data_stored":436195170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1570033202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004335"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.853054+0000","last_change":"2024-04-22T20:49:12.853054+0000","last_active":"2024-04-22T20:49:12.853054+0000","last_peered":"2024-04-22T20:49:12.853054+0000","last_clean":"2024-04-22T20:49:12.853054+0000","last_became_active":"2024-04-22T20:19:16.796520+0000","last_became_peered":"2024-04-22T20:19:16.796520+0000","last_unstale":"2024-04-22T20:49:12.853054+0000","last_undegraded":"2024-04-22T20:49:12.853054+0000","last_fullsized":"2024-04-22T20:49:12.853054+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:12.852862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:12.852862+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.852862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:09.323293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"261'5","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.873311+0000","last_change":"2024-04-22T20:49:08.873339+0000","last_active":"2024-04-22T20:49:08.873311+0000","last_peered":"2024-04-22T20:49:08.873311+0000","last_clean":"2024-04-22T20:49:08.873311+0000","last_became_active":"2024-04-22T20:05:17.900325+0000","last_became_peered":"2024-04-22T20:05:17.900325+0000","last_unstale":"2024-04-22T20:49:08.873311+0000","last_undegraded":"2024-04-22T20:49:08.873311+0000","last_fullsized":"2024-04-22T20:49:08.873311+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:08.873212+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:08.873212+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.873212+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:57:06.368419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"0'0","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.497585+0000","last_change":"2024-04-22T20:48:24.497620+0000","last_active":"2024-04-22T20:48:24.497585+0000","last_peered":"2024-04-22T20:48:24.497585+0000","last_clean":"2024-04-22T20:48:24.497585+0000","last_became_active":"2024-04-22T20:05:13.812933+0000","last_became_peered":"2024-04-22T20:05:13.812933+0000","last_unstale":"2024-04-22T20:48:24.497585+0000","last_undegraded":"2024-04-22T20:48:24.497585+0000","last_fullsized":"2024-04-22T20:48:24.497585+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:24.497472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:24.497472+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.497472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:19.340438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"261'20","reported_seq":110,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.612368+0000","last_change":"2024-04-22T20:48:24.612368+0000","last_active":"2024-04-22T20:48:24.612368+0000","last_peered":"2024-04-22T20:48:24.612368+0000","last_clean":"2024-04-22T20:48:24.612368+0000","last_became_active":"2024-04-22T20:01:04.875981+0000","last_became_peered":"2024-04-22T20:01:04.875981+0000","last_unstale":"2024-04-22T20:48:24.612368+0000","last_undegraded":"2024-04-22T20:48:24.612368+0000","last_fullsized":"2024-04-22T20:48:24.612368+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'20","last_scrub_stamp":"2024-04-22T20:48:24.612214+0000","last_deep_scrub":"261'20","last_deep_scrub_stamp":"2024-04-22T20:48:24.612214+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.612214+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:22:32.284247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":53,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.544978+0000","last_change":"2024-04-22T20:48:50.544978+0000","last_active":"2024-04-22T20:48:50.544978+0000","last_peered":"2024-04-22T20:48:50.544978+0000","last_clean":"2024-04-22T20:48:50.544978+0000","last_became_active":"2024-04-22T20:19:16.790377+0000","last_became_peered":"2024-04-22T20:19:16.790377+0000","last_unstale":"2024-04-22T20:48:50.544978+0000","last_undegraded":"2024-04-22T20:48:50.544978+0000","last_fullsized":"2024-04-22T20:48:50.544978+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.544857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.544857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.544857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:18:08.988881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"0'0","reported_seq":48,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.236997+0000","last_change":"2024-04-22T20:49:09.237038+0000","last_active":"2024-04-22T20:49:09.236997+0000","last_peered":"2024-04-22T20:49:09.236997+0000","last_clean":"2024-04-22T20:49:09.236997+0000","last_became_active":"2024-04-22T20:05:17.899338+0000","last_became_peered":"2024-04-22T20:05:17.899338+0000","last_unstale":"2024-04-22T20:49:09.236997+0000","last_undegraded":"2024-04-22T20:49:09.236997+0000","last_fullsized":"2024-04-22T20:49:09.236997+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:09.236883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:09.236883+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.236883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:49.186676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"258'4","reported_seq":62,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.584270+0000","last_change":"2024-04-22T20:48:23.584307+0000","last_active":"2024-04-22T20:48:23.584270+0000","last_peered":"2024-04-22T20:48:23.584270+0000","last_clean":"2024-04-22T20:48:23.584270+0000","last_became_active":"2024-04-22T20:05:13.807231+0000","last_became_peered":"2024-04-22T20:05:13.807231+0000","last_unstale":"2024-04-22T20:48:23.584270+0000","last_undegraded":"2024-04-22T20:48:23.584270+0000","last_fullsized":"2024-04-22T20:48:23.584270+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:23.584182+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:23.584182+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.584182+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:34.394463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"261'22","reported_seq":111,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.458388+0000","last_change":"2024-04-22T20:48:34.458388+0000","last_active":"2024-04-22T20:48:34.458388+0000","last_peered":"2024-04-22T20:48:34.458388+0000","last_clean":"2024-04-22T20:48:34.458388+0000","last_became_active":"2024-04-22T20:01:04.874575+0000","last_became_peered":"2024-04-22T20:01:04.874575+0000","last_unstale":"2024-04-22T20:48:34.458388+0000","last_undegraded":"2024-04-22T20:48:34.458388+0000","last_fullsized":"2024-04-22T20:48:34.458388+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'22","last_scrub_stamp":"2024-04-22T20:48:34.458264+0000","last_deep_scrub":"261'22","last_deep_scrub_stamp":"2024-04-22T20:48:34.458264+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.458264+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:46:15.278433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":48,"num_write":47,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.725001+0000","last_change":"2024-04-22T20:49:03.725044+0000","last_active":"2024-04-22T20:49:03.725001+0000","last_peered":"2024-04-22T20:49:03.725001+0000","last_clean":"2024-04-22T20:49:03.725001+0000","last_became_active":"2024-04-22T20:19:16.794700+0000","last_became_peered":"2024-04-22T20:19:16.794700+0000","last_unstale":"2024-04-22T20:49:03.725001+0000","last_undegraded":"2024-04-22T20:49:03.725001+0000","last_fullsized":"2024-04-22T20:49:03.725001+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:03.724898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:03.724898+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.724898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:21:17.669058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"0'0","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:16.689969+0000","last_change":"2024-04-22T20:49:16.690009+0000","last_active":"2024-04-22T20:49:16.689969+0000","last_peered":"2024-04-22T20:49:16.689969+0000","last_clean":"2024-04-22T20:49:16.689969+0000","last_became_active":"2024-04-22T20:05:18.211400+0000","last_became_peered":"2024-04-22T20:05:18.211400+0000","last_unstale":"2024-04-22T20:49:16.689969+0000","last_undegraded":"2024-04-22T20:49:16.689969+0000","last_fullsized":"2024-04-22T20:49:16.689969+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:16.689857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:16.689857+0000","last_clean_scrub_stamp":"2024-04-22T20:49:16.689857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:20:14.133518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"261'2","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.570845+0000","last_change":"2024-04-22T20:48:46.570845+0000","last_active":"2024-04-22T20:48:46.570845+0000","last_peered":"2024-04-22T20:48:46.570845+0000","last_clean":"2024-04-22T20:48:46.570845+0000","last_became_active":"2024-04-22T20:05:13.809349+0000","last_became_peered":"2024-04-22T20:05:13.809349+0000","last_unstale":"2024-04-22T20:48:46.570845+0000","last_undegraded":"2024-04-22T20:48:46.570845+0000","last_fullsized":"2024-04-22T20:48:46.570845+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:46.570725+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:46.570725+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.570725+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:45:00.895789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"261'24","reported_seq":124,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.511512+0000","last_change":"2024-04-22T20:48:31.511512+0000","last_active":"2024-04-22T20:48:31.511512+0000","last_peered":"2024-04-22T20:48:31.511512+0000","last_clean":"2024-04-22T20:48:31.511512+0000","last_became_active":"2024-04-22T20:01:04.869785+0000","last_became_peered":"2024-04-22T20:01:04.869785+0000","last_unstale":"2024-04-22T20:48:31.511512+0000","last_undegraded":"2024-04-22T20:48:31.511512+0000","last_fullsized":"2024-04-22T20:48:31.511512+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'24","last_scrub_stamp":"2024-04-22T20:48:31.511334+0000","last_deep_scrub":"261'24","last_deep_scrub_stamp":"2024-04-22T20:48:31.511334+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.511334+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:43:54.338193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":82,"num_write":36,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:25.244342+0000","last_change":"2024-04-22T20:49:25.244367+0000","last_active":"2024-04-22T20:49:25.244342+0000","last_peered":"2024-04-22T20:49:25.244342+0000","last_clean":"2024-04-22T20:49:25.244342+0000","last_became_active":"2024-04-22T20:19:16.793667+0000","last_became_peered":"2024-04-22T20:19:16.793667+0000","last_unstale":"2024-04-22T20:49:25.244342+0000","last_undegraded":"2024-04-22T20:49:25.244342+0000","last_fullsized":"2024-04-22T20:49:25.244342+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:25.244245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:25.244245+0000","last_clean_scrub_stamp":"2024-04-22T20:49:25.244245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:05:31.088960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"261'10","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:17.704150+0000","last_change":"2024-04-22T20:49:17.704183+0000","last_active":"2024-04-22T20:49:17.704150+0000","last_peered":"2024-04-22T20:49:17.704150+0000","last_clean":"2024-04-22T20:49:17.704150+0000","last_became_active":"2024-04-22T20:06:05.345554+0000","last_became_peered":"2024-04-22T20:06:05.345554+0000","last_unstale":"2024-04-22T20:49:17.704150+0000","last_undegraded":"2024-04-22T20:49:17.704150+0000","last_fullsized":"2024-04-22T20:49:17.704150+0000","mapping_epoch":257,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":258,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:49:17.704058+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:49:17.704058+0000","last_clean_scrub_stamp":"2024-04-22T20:49:17.704058+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:09:58.147978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":20,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1f","version":"261'10","reported_seq":78,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.367896+0000","last_change":"2024-04-22T20:48:36.367896+0000","last_active":"2024-04-22T20:48:36.367896+0000","last_peered":"2024-04-22T20:48:36.367896+0000","last_clean":"2024-04-22T20:48:36.367896+0000","last_became_active":"2024-04-22T20:05:13.806246+0000","last_became_peered":"2024-04-22T20:05:13.806246+0000","last_unstale":"2024-04-22T20:48:36.367896+0000","last_undegraded":"2024-04-22T20:48:36.367896+0000","last_fullsized":"2024-04-22T20:48:36.367896+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:48:36.367701+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:48:36.367701+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.367701+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:06:18.240490+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":16720,"num_write":13,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"261'59","reported_seq":208,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.583472+0000","last_change":"2024-04-22T20:48:30.583472+0000","last_active":"2024-04-22T20:48:30.583472+0000","last_peered":"2024-04-22T20:48:30.583472+0000","last_clean":"2024-04-22T20:48:30.583472+0000","last_became_active":"2024-04-22T20:01:04.873899+0000","last_became_peered":"2024-04-22T20:01:04.873899+0000","last_unstale":"2024-04-22T20:48:30.583472+0000","last_undegraded":"2024-04-22T20:48:30.583472+0000","last_fullsized":"2024-04-22T20:48:30.583472+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'59","last_scrub_stamp":"2024-04-22T20:48:30.583311+0000","last_deep_scrub":"261'59","last_deep_scrub_stamp":"2024-04-22T20:48:30.583311+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.583311+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:58:29.692843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":183,"num_write":99,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:23.243084+0000","last_change":"2024-04-22T20:49:23.243107+0000","last_active":"2024-04-22T20:49:23.243084+0000","last_peered":"2024-04-22T20:49:23.243084+0000","last_clean":"2024-04-22T20:49:23.243084+0000","last_became_active":"2024-04-22T20:19:16.794110+0000","last_became_peered":"2024-04-22T20:19:16.794110+0000","last_unstale":"2024-04-22T20:49:23.243084+0000","last_undegraded":"2024-04-22T20:49:23.243084+0000","last_fullsized":"2024-04-22T20:49:23.243084+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:23.242988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:23.242988+0000","last_clean_scrub_stamp":"2024-04-22T20:49:23.242988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:42:20.330984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"261'2","reported_seq":53,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.865180+0000","last_change":"2024-04-22T20:49:07.865227+0000","last_active":"2024-04-22T20:49:07.865180+0000","last_peered":"2024-04-22T20:49:07.865180+0000","last_clean":"2024-04-22T20:49:07.865180+0000","last_became_active":"2024-04-22T20:05:17.897771+0000","last_became_peered":"2024-04-22T20:05:17.897771+0000","last_unstale":"2024-04-22T20:49:07.865180+0000","last_undegraded":"2024-04-22T20:49:07.865180+0000","last_fullsized":"2024-04-22T20:49:07.865180+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:07.865075+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:07.865075+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.865075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:05.559298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"261'6","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.520623+0000","last_change":"2024-04-22T20:48:29.520623+0000","last_active":"2024-04-22T20:48:29.520623+0000","last_peered":"2024-04-22T20:48:29.520623+0000","last_clean":"2024-04-22T20:48:29.520623+0000","last_became_active":"2024-04-22T20:05:13.812308+0000","last_became_peered":"2024-04-22T20:05:13.812308+0000","last_unstale":"2024-04-22T20:48:29.520623+0000","last_undegraded":"2024-04-22T20:48:29.520623+0000","last_fullsized":"2024-04-22T20:48:29.520623+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:29.520502+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:29.520502+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.520502+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:11:42.190343+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":9,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"261'8","reported_seq":82,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.526061+0000","last_change":"2024-04-22T20:48:25.526061+0000","last_active":"2024-04-22T20:48:25.526061+0000","last_peered":"2024-04-22T20:48:25.526061+0000","last_clean":"2024-04-22T20:48:25.526061+0000","last_became_active":"2024-04-22T20:01:04.877209+0000","last_became_peered":"2024-04-22T20:01:04.877209+0000","last_unstale":"2024-04-22T20:48:25.526061+0000","last_undegraded":"2024-04-22T20:48:25.526061+0000","last_fullsized":"2024-04-22T20:48:25.526061+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'8","last_scrub_stamp":"2024-04-22T20:48:25.525971+0000","last_deep_scrub":"261'8","last_deep_scrub_stamp":"2024-04-22T20:48:25.525971+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.525971+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:37.255572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":25,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.626708+0000","last_change":"2024-04-22T20:49:02.626730+0000","last_active":"2024-04-22T20:49:02.626708+0000","last_peered":"2024-04-22T20:49:02.626708+0000","last_clean":"2024-04-22T20:49:02.626708+0000","last_became_active":"2024-04-22T20:19:16.789599+0000","last_became_peered":"2024-04-22T20:19:16.789599+0000","last_unstale":"2024-04-22T20:49:02.626708+0000","last_undegraded":"2024-04-22T20:49:02.626708+0000","last_fullsized":"2024-04-22T20:49:02.626708+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.626625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.626625+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.626625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:06:20.379636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"261'2","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.611806+0000","last_change":"2024-04-22T20:48:59.611832+0000","last_active":"2024-04-22T20:48:59.611806+0000","last_peered":"2024-04-22T20:48:59.611806+0000","last_clean":"2024-04-22T20:48:59.611806+0000","last_became_active":"2024-04-22T20:05:17.890841+0000","last_became_peered":"2024-04-22T20:05:17.890841+0000","last_unstale":"2024-04-22T20:48:59.611806+0000","last_undegraded":"2024-04-22T20:48:59.611806+0000","last_fullsized":"2024-04-22T20:48:59.611806+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:59.611713+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:59.611713+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.611713+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:39:18.164914+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":48,"num_write":2,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"0'0","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.299460+0000","last_change":"2024-04-22T20:48:40.299460+0000","last_active":"2024-04-22T20:48:40.299460+0000","last_peered":"2024-04-22T20:48:40.299460+0000","last_clean":"2024-04-22T20:48:40.299460+0000","last_became_active":"2024-04-22T20:05:13.807516+0000","last_became_peered":"2024-04-22T20:05:13.807516+0000","last_unstale":"2024-04-22T20:48:40.299460+0000","last_undegraded":"2024-04-22T20:48:40.299460+0000","last_fullsized":"2024-04-22T20:48:40.299460+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.299341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.299341+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.299341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:32:01.048139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"261'77","reported_seq":249,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.378570+0000","last_change":"2024-04-22T20:48:26.378570+0000","last_active":"2024-04-22T20:48:26.378570+0000","last_peered":"2024-04-22T20:48:26.378570+0000","last_clean":"2024-04-22T20:48:26.378570+0000","last_became_active":"2024-04-22T20:01:04.873015+0000","last_became_peered":"2024-04-22T20:01:04.873015+0000","last_unstale":"2024-04-22T20:48:26.378570+0000","last_undegraded":"2024-04-22T20:48:26.378570+0000","last_fullsized":"2024-04-22T20:48:26.378570+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'77","last_scrub_stamp":"2024-04-22T20:48:26.378447+0000","last_deep_scrub":"261'77","last_deep_scrub_stamp":"2024-04-22T20:48:26.378447+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.378447+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:22.636845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":334,"num_read_kb":236,"num_write":125,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"261'2","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.451468+0000","last_change":"2024-04-22T20:48:48.451468+0000","last_active":"2024-04-22T20:48:48.451468+0000","last_peered":"2024-04-22T20:48:48.451468+0000","last_clean":"2024-04-22T20:48:48.451468+0000","last_became_active":"2024-04-22T20:19:16.794564+0000","last_became_peered":"2024-04-22T20:19:16.794564+0000","last_unstale":"2024-04-22T20:48:48.451468+0000","last_undegraded":"2024-04-22T20:48:48.451468+0000","last_fullsized":"2024-04-22T20:48:48.451468+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:48.451302+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:48.451302+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.451302+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:04:33.139707+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"261'5","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.663979+0000","last_change":"2024-04-22T20:49:01.664006+0000","last_active":"2024-04-22T20:49:01.663979+0000","last_peered":"2024-04-22T20:49:01.663979+0000","last_clean":"2024-04-22T20:49:01.663979+0000","last_became_active":"2024-04-22T20:05:17.901446+0000","last_became_peered":"2024-04-22T20:05:17.901446+0000","last_unstale":"2024-04-22T20:49:01.663979+0000","last_undegraded":"2024-04-22T20:49:01.663979+0000","last_fullsized":"2024-04-22T20:49:01.663979+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:01.663883+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:01.663883+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.663883+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:07:50.105267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":11264,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.351494+0000","last_change":"2024-04-22T20:48:28.351557+0000","last_active":"2024-04-22T20:48:28.351494+0000","last_peered":"2024-04-22T20:48:28.351494+0000","last_clean":"2024-04-22T20:48:28.351494+0000","last_became_active":"2024-04-22T20:05:13.798311+0000","last_became_peered":"2024-04-22T20:05:13.798311+0000","last_unstale":"2024-04-22T20:48:28.351494+0000","last_undegraded":"2024-04-22T20:48:28.351494+0000","last_fullsized":"2024-04-22T20:48:28.351494+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:28.351390+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:28.351390+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.351390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:49:42.653069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"261'27","reported_seq":126,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.545440+0000","last_change":"2024-04-22T20:48:28.545440+0000","last_active":"2024-04-22T20:48:28.545440+0000","last_peered":"2024-04-22T20:48:28.545440+0000","last_clean":"2024-04-22T20:48:28.545440+0000","last_became_active":"2024-04-22T20:01:04.873426+0000","last_became_peered":"2024-04-22T20:01:04.873426+0000","last_unstale":"2024-04-22T20:48:28.545440+0000","last_undegraded":"2024-04-22T20:48:28.545440+0000","last_fullsized":"2024-04-22T20:48:28.545440+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'27","last_scrub_stamp":"2024-04-22T20:48:28.545320+0000","last_deep_scrub":"261'27","last_deep_scrub_stamp":"2024-04-22T20:48:28.545320+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.545320+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:51:52.257563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":80,"num_write":54,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:21.601132+0000","last_change":"2024-04-22T20:49:21.601157+0000","last_active":"2024-04-22T20:49:21.601132+0000","last_peered":"2024-04-22T20:49:21.601132+0000","last_clean":"2024-04-22T20:49:21.601132+0000","last_became_active":"2024-04-22T20:19:16.800611+0000","last_became_peered":"2024-04-22T20:19:16.800611+0000","last_unstale":"2024-04-22T20:49:21.601132+0000","last_undegraded":"2024-04-22T20:49:21.601132+0000","last_fullsized":"2024-04-22T20:49:21.601132+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:21.601034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:21.601034+0000","last_clean_scrub_stamp":"2024-04-22T20:49:21.601034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:03.789616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"258'4","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.512936+0000","last_change":"2024-04-22T20:48:44.512963+0000","last_active":"2024-04-22T20:48:44.512936+0000","last_peered":"2024-04-22T20:48:44.512936+0000","last_clean":"2024-04-22T20:48:44.512936+0000","last_became_active":"2024-04-22T20:05:17.892624+0000","last_became_peered":"2024-04-22T20:05:17.892624+0000","last_unstale":"2024-04-22T20:48:44.512936+0000","last_undegraded":"2024-04-22T20:48:44.512936+0000","last_fullsized":"2024-04-22T20:48:44.512936+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:44.512843+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:44.512843+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.512843+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:13:48.032126+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3072,"num_write":4,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.574707+0000","last_change":"2024-04-22T20:48:41.574733+0000","last_active":"2024-04-22T20:48:41.574707+0000","last_peered":"2024-04-22T20:48:41.574707+0000","last_clean":"2024-04-22T20:48:41.574707+0000","last_became_active":"2024-04-22T20:05:13.816011+0000","last_became_peered":"2024-04-22T20:05:13.816011+0000","last_unstale":"2024-04-22T20:48:41.574707+0000","last_undegraded":"2024-04-22T20:48:41.574707+0000","last_fullsized":"2024-04-22T20:48:41.574707+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:41.574619+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:41.574619+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.574619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:10:05.304886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"261'43","reported_seq":160,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.562960+0000","last_change":"2024-04-22T20:48:25.562960+0000","last_active":"2024-04-22T20:48:25.562960+0000","last_peered":"2024-04-22T20:48:25.562960+0000","last_clean":"2024-04-22T20:48:25.562960+0000","last_became_active":"2024-04-22T20:01:04.872356+0000","last_became_peered":"2024-04-22T20:01:04.872356+0000","last_unstale":"2024-04-22T20:48:25.562960+0000","last_undegraded":"2024-04-22T20:48:25.562960+0000","last_fullsized":"2024-04-22T20:48:25.562960+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'43","last_scrub_stamp":"2024-04-22T20:48:25.562858+0000","last_deep_scrub":"261'43","last_deep_scrub_stamp":"2024-04-22T20:48:25.562858+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.562858+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:39:26.408917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":169,"num_read_kb":119,"num_write":72,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.688337+0000","last_change":"2024-04-22T20:49:04.688337+0000","last_active":"2024-04-22T20:49:04.688337+0000","last_peered":"2024-04-22T20:49:04.688337+0000","last_clean":"2024-04-22T20:49:04.688337+0000","last_became_active":"2024-04-22T20:19:16.796155+0000","last_became_peered":"2024-04-22T20:19:16.796155+0000","last_unstale":"2024-04-22T20:49:04.688337+0000","last_undegraded":"2024-04-22T20:49:04.688337+0000","last_fullsized":"2024-04-22T20:49:04.688337+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:04.688168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:04.688168+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.688168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:33:59.083750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"0'0","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.425810+0000","last_change":"2024-04-22T20:48:52.425835+0000","last_active":"2024-04-22T20:48:52.425810+0000","last_peered":"2024-04-22T20:48:52.425810+0000","last_clean":"2024-04-22T20:48:52.425810+0000","last_became_active":"2024-04-22T20:05:18.209023+0000","last_became_peered":"2024-04-22T20:05:18.209023+0000","last_unstale":"2024-04-22T20:48:52.425810+0000","last_undegraded":"2024-04-22T20:48:52.425810+0000","last_fullsized":"2024-04-22T20:48:52.425810+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.425722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.425722+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.425722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:32:32.486188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"261'7","reported_seq":71,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.910210+0000","last_change":"2024-04-22T20:48:39.910210+0000","last_active":"2024-04-22T20:48:39.910210+0000","last_peered":"2024-04-22T20:48:39.910210+0000","last_clean":"2024-04-22T20:48:39.910210+0000","last_became_active":"2024-04-22T20:05:13.809548+0000","last_became_peered":"2024-04-22T20:05:13.809548+0000","last_unstale":"2024-04-22T20:48:39.910210+0000","last_undegraded":"2024-04-22T20:48:39.910210+0000","last_fullsized":"2024-04-22T20:48:39.910210+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:39.910080+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:39.910080+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.910080+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:39:16.656357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":6146,"num_write":5,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"261'64","reported_seq":252,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.544266+0000","last_change":"2024-04-22T20:48:30.544266+0000","last_active":"2024-04-22T20:48:30.544266+0000","last_peered":"2024-04-22T20:48:30.544266+0000","last_clean":"2024-04-22T20:48:30.544266+0000","last_became_active":"2024-04-22T20:01:04.873236+0000","last_became_peered":"2024-04-22T20:01:04.873236+0000","last_unstale":"2024-04-22T20:48:30.544266+0000","last_undegraded":"2024-04-22T20:48:30.544266+0000","last_fullsized":"2024-04-22T20:48:30.544266+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'64","last_scrub_stamp":"2024-04-22T20:48:30.544111+0000","last_deep_scrub":"261'64","last_deep_scrub_stamp":"2024-04-22T20:48:30.544111+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.544111+0000","objects_scrubbed":1,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:31:51.536623+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":250,"num_write":111,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3950,"num_omap_keys":8,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"261'2","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.590731+0000","last_change":"2024-04-22T20:48:51.590731+0000","last_active":"2024-04-22T20:48:51.590731+0000","last_peered":"2024-04-22T20:48:51.590731+0000","last_clean":"2024-04-22T20:48:51.590731+0000","last_became_active":"2024-04-22T20:19:16.791597+0000","last_became_peered":"2024-04-22T20:19:16.791597+0000","last_unstale":"2024-04-22T20:48:51.590731+0000","last_undegraded":"2024-04-22T20:48:51.590731+0000","last_fullsized":"2024-04-22T20:48:51.590731+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:51.590624+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:51.590624+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.590624+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:08.152952+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"0'0","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:19.298045+0000","last_change":"2024-04-22T20:49:19.298045+0000","last_active":"2024-04-22T20:49:19.298045+0000","last_peered":"2024-04-22T20:49:19.298045+0000","last_clean":"2024-04-22T20:49:19.298045+0000","last_became_active":"2024-04-22T20:05:18.210704+0000","last_became_peered":"2024-04-22T20:05:18.210704+0000","last_unstale":"2024-04-22T20:49:19.298045+0000","last_undegraded":"2024-04-22T20:49:19.298045+0000","last_fullsized":"2024-04-22T20:49:19.298045+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:19.297858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:19.297858+0000","last_clean_scrub_stamp":"2024-04-22T20:49:19.297858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:48:21.613563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":55,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.609480+0000","last_change":"2024-04-22T20:48:27.609480+0000","last_active":"2024-04-22T20:48:27.609480+0000","last_peered":"2024-04-22T20:48:27.609480+0000","last_clean":"2024-04-22T20:48:27.609480+0000","last_became_active":"2024-04-22T20:05:13.810902+0000","last_became_peered":"2024-04-22T20:05:13.810902+0000","last_unstale":"2024-04-22T20:48:27.609480+0000","last_undegraded":"2024-04-22T20:48:27.609480+0000","last_fullsized":"2024-04-22T20:48:27.609480+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:27.609361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:27.609361+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.609361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:13:08.903641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"250'9","reported_seq":86,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.596526+0000","last_change":"2024-04-22T20:48:27.596526+0000","last_active":"2024-04-22T20:48:27.596526+0000","last_peered":"2024-04-22T20:48:27.596526+0000","last_clean":"2024-04-22T20:48:27.596526+0000","last_became_active":"2024-04-22T20:01:04.874230+0000","last_became_peered":"2024-04-22T20:01:04.874230+0000","last_unstale":"2024-04-22T20:48:27.596526+0000","last_undegraded":"2024-04-22T20:48:27.596526+0000","last_fullsized":"2024-04-22T20:48:27.596526+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"250'9","last_scrub_stamp":"2024-04-22T20:48:27.596354+0000","last_deep_scrub":"250'9","last_deep_scrub_stamp":"2024-04-22T20:48:27.596354+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.596354+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:12:20.356156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:16.548937+0000","last_change":"2024-04-22T20:49:16.548937+0000","last_active":"2024-04-22T20:49:16.548937+0000","last_peered":"2024-04-22T20:49:16.548937+0000","last_clean":"2024-04-22T20:49:16.548937+0000","last_became_active":"2024-04-22T20:19:16.795139+0000","last_became_peered":"2024-04-22T20:19:16.795139+0000","last_unstale":"2024-04-22T20:49:16.548937+0000","last_undegraded":"2024-04-22T20:49:16.548937+0000","last_fullsized":"2024-04-22T20:49:16.548937+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:16.548816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:16.548816+0000","last_clean_scrub_stamp":"2024-04-22T20:49:16.548816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:52:47.710385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"261'19","reported_seq":88,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:20.283697+0000","last_change":"2024-04-22T20:49:20.283758+0000","last_active":"2024-04-22T20:49:20.283697+0000","last_peered":"2024-04-22T20:49:20.283697+0000","last_clean":"2024-04-22T20:49:20.283697+0000","last_became_active":"2024-04-22T20:05:18.212336+0000","last_became_peered":"2024-04-22T20:05:18.212336+0000","last_unstale":"2024-04-22T20:49:20.283697+0000","last_undegraded":"2024-04-22T20:49:20.283697+0000","last_fullsized":"2024-04-22T20:49:20.283697+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'19","last_scrub_stamp":"2024-04-22T20:49:20.283608+0000","last_deep_scrub":"261'19","last_deep_scrub_stamp":"2024-04-22T20:49:20.283608+0000","last_clean_scrub_stamp":"2024-04-22T20:49:20.283608+0000","objects_scrubbed":3,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:02.878763+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437184,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":38918,"num_write":23,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"261'2","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.569695+0000","last_change":"2024-04-22T20:48:49.569733+0000","last_active":"2024-04-22T20:48:49.569695+0000","last_peered":"2024-04-22T20:48:49.569695+0000","last_clean":"2024-04-22T20:48:49.569695+0000","last_became_active":"2024-04-22T20:05:13.816036+0000","last_became_peered":"2024-04-22T20:05:13.816036+0000","last_unstale":"2024-04-22T20:48:49.569695+0000","last_undegraded":"2024-04-22T20:48:49.569695+0000","last_fullsized":"2024-04-22T20:48:49.569695+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:49.569601+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:49.569601+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.569601+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:07:18.107853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"261'53","reported_seq":186,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.516524+0000","last_change":"2024-04-22T20:48:36.516524+0000","last_active":"2024-04-22T20:48:36.516524+0000","last_peered":"2024-04-22T20:48:36.516524+0000","last_clean":"2024-04-22T20:48:36.516524+0000","last_became_active":"2024-04-22T20:01:04.878067+0000","last_became_peered":"2024-04-22T20:01:04.878067+0000","last_unstale":"2024-04-22T20:48:36.516524+0000","last_undegraded":"2024-04-22T20:48:36.516524+0000","last_fullsized":"2024-04-22T20:48:36.516524+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'53","last_scrub_stamp":"2024-04-22T20:48:36.516407+0000","last_deep_scrub":"261'53","last_deep_scrub_stamp":"2024-04-22T20:48:36.516407+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.516407+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:26.653699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":150,"num_write":99,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.849660+0000","last_change":"2024-04-22T20:49:11.849660+0000","last_active":"2024-04-22T20:49:11.849660+0000","last_peered":"2024-04-22T20:49:11.849660+0000","last_clean":"2024-04-22T20:49:11.849660+0000","last_became_active":"2024-04-22T20:19:16.795855+0000","last_became_peered":"2024-04-22T20:19:16.795855+0000","last_unstale":"2024-04-22T20:49:11.849660+0000","last_undegraded":"2024-04-22T20:49:11.849660+0000","last_fullsized":"2024-04-22T20:49:11.849660+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:11.849504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:11.849504+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.849504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:18:41.306522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"0'0","reported_seq":49,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.281165+0000","last_change":"2024-04-22T20:49:12.281192+0000","last_active":"2024-04-22T20:49:12.281165+0000","last_peered":"2024-04-22T20:49:12.281165+0000","last_clean":"2024-04-22T20:49:12.281165+0000","last_became_active":"2024-04-22T20:05:17.901954+0000","last_became_peered":"2024-04-22T20:05:17.901954+0000","last_unstale":"2024-04-22T20:49:12.281165+0000","last_undegraded":"2024-04-22T20:49:12.281165+0000","last_fullsized":"2024-04-22T20:49:12.281165+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:12.281076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:12.281076+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.281076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:21:13.094684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"261'4","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.504218+0000","last_change":"2024-04-22T20:48:33.504218+0000","last_active":"2024-04-22T20:48:33.504218+0000","last_peered":"2024-04-22T20:48:33.504218+0000","last_clean":"2024-04-22T20:48:33.504218+0000","last_became_active":"2024-04-22T20:05:13.813256+0000","last_became_peered":"2024-04-22T20:05:13.813256+0000","last_unstale":"2024-04-22T20:48:33.504218+0000","last_undegraded":"2024-04-22T20:48:33.504218+0000","last_fullsized":"2024-04-22T20:48:33.504218+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:33.504106+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:33.504106+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.504106+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:41:49.887327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":44,"num_write":4,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"261'46","reported_seq":173,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.448464+0000","last_change":"2024-04-22T20:48:33.448487+0000","last_active":"2024-04-22T20:48:33.448464+0000","last_peered":"2024-04-22T20:48:33.448464+0000","last_clean":"2024-04-22T20:48:33.448464+0000","last_became_active":"2024-04-22T20:01:04.873533+0000","last_became_peered":"2024-04-22T20:01:04.873533+0000","last_unstale":"2024-04-22T20:48:33.448464+0000","last_undegraded":"2024-04-22T20:48:33.448464+0000","last_fullsized":"2024-04-22T20:48:33.448464+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'46","last_scrub_stamp":"2024-04-22T20:48:33.448376+0000","last_deep_scrub":"261'46","last_deep_scrub_stamp":"2024-04-22T20:48:33.448376+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.448376+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:02:25.135711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":192,"num_read_kb":137,"num_write":82,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:21.272514+0000","last_change":"2024-04-22T20:49:21.272536+0000","last_active":"2024-04-22T20:49:21.272514+0000","last_peered":"2024-04-22T20:49:21.272514+0000","last_clean":"2024-04-22T20:49:21.272514+0000","last_became_active":"2024-04-22T20:19:16.792856+0000","last_became_peered":"2024-04-22T20:19:16.792856+0000","last_unstale":"2024-04-22T20:49:21.272514+0000","last_undegraded":"2024-04-22T20:49:21.272514+0000","last_fullsized":"2024-04-22T20:49:21.272514+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:21.272423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:21.272423+0000","last_clean_scrub_stamp":"2024-04-22T20:49:21.272423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:12:40.395975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"261'9","reported_seq":68,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.654866+0000","last_change":"2024-04-22T20:48:59.654866+0000","last_active":"2024-04-22T20:48:59.654866+0000","last_peered":"2024-04-22T20:48:59.654866+0000","last_clean":"2024-04-22T20:48:59.654866+0000","last_became_active":"2024-04-22T20:05:17.897040+0000","last_became_peered":"2024-04-22T20:05:17.897040+0000","last_unstale":"2024-04-22T20:48:59.654866+0000","last_undegraded":"2024-04-22T20:48:59.654866+0000","last_fullsized":"2024-04-22T20:48:59.654866+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-22T20:48:59.654751+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-22T20:48:59.654751+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.654751+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:27:11.365352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":12290,"num_write":7,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"261'4","reported_seq":64,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.219154+0000","last_change":"2024-04-22T20:49:07.219154+0000","last_active":"2024-04-22T20:49:07.219154+0000","last_peered":"2024-04-22T20:49:07.219154+0000","last_clean":"2024-04-22T20:49:07.219154+0000","last_became_active":"2024-04-22T20:05:13.814222+0000","last_became_peered":"2024-04-22T20:05:13.814222+0000","last_unstale":"2024-04-22T20:49:07.219154+0000","last_undegraded":"2024-04-22T20:49:07.219154+0000","last_fullsized":"2024-04-22T20:49:07.219154+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:49:07.219038+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:49:07.219038+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.219038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:39:41.878224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"261'54","reported_seq":199,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.410439+0000","last_change":"2024-04-22T20:48:34.410468+0000","last_active":"2024-04-22T20:48:34.410439+0000","last_peered":"2024-04-22T20:48:34.410439+0000","last_clean":"2024-04-22T20:48:34.410439+0000","last_became_active":"2024-04-22T20:01:04.873422+0000","last_became_peered":"2024-04-22T20:01:04.873422+0000","last_unstale":"2024-04-22T20:48:34.410439+0000","last_undegraded":"2024-04-22T20:48:34.410439+0000","last_fullsized":"2024-04-22T20:48:34.410439+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'54","last_scrub_stamp":"2024-04-22T20:48:34.410341+0000","last_deep_scrub":"261'54","last_deep_scrub_stamp":"2024-04-22T20:48:34.410341+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.410341+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:47:24.001212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":252,"num_read_kb":184,"num_write":82,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":33,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:22.728082+0000","last_change":"2024-04-22T20:49:22.728111+0000","last_active":"2024-04-22T20:49:22.728082+0000","last_peered":"2024-04-22T20:49:22.728082+0000","last_clean":"2024-04-22T20:49:22.728082+0000","last_became_active":"2024-04-22T20:19:16.795633+0000","last_became_peered":"2024-04-22T20:19:16.795633+0000","last_unstale":"2024-04-22T20:49:22.728082+0000","last_undegraded":"2024-04-22T20:49:22.728082+0000","last_fullsized":"2024-04-22T20:49:22.728082+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:22.727960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:22.727960+0000","last_clean_scrub_stamp":"2024-04-22T20:49:22.727960+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:23:44.276656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"261'10","reported_seq":69,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.653851+0000","last_change":"2024-04-22T20:49:00.653851+0000","last_active":"2024-04-22T20:49:00.653851+0000","last_peered":"2024-04-22T20:49:00.653851+0000","last_clean":"2024-04-22T20:49:00.653851+0000","last_became_active":"2024-04-22T20:05:17.899682+0000","last_became_peered":"2024-04-22T20:05:17.899682+0000","last_unstale":"2024-04-22T20:49:00.653851+0000","last_undegraded":"2024-04-22T20:49:00.653851+0000","last_fullsized":"2024-04-22T20:49:00.653851+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'10","last_scrub_stamp":"2024-04-22T20:49:00.653727+0000","last_deep_scrub":"261'10","last_deep_scrub_stamp":"2024-04-22T20:49:00.653727+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.653727+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:51:18.897192+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14,"num_read_kb":8210,"num_write":13,"num_write_kb":5136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"261'6","reported_seq":64,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.625974+0000","last_change":"2024-04-22T20:48:28.625974+0000","last_active":"2024-04-22T20:48:28.625974+0000","last_peered":"2024-04-22T20:48:28.625974+0000","last_clean":"2024-04-22T20:48:28.625974+0000","last_became_active":"2024-04-22T20:05:13.812893+0000","last_became_peered":"2024-04-22T20:05:13.812893+0000","last_unstale":"2024-04-22T20:48:28.625974+0000","last_undegraded":"2024-04-22T20:48:28.625974+0000","last_fullsized":"2024-04-22T20:48:28.625974+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:28.625814+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:28.625814+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.625814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:45:06.391872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4096,"num_write":16,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"261'47","reported_seq":172,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.628532+0000","last_change":"2024-04-22T20:48:23.628571+0000","last_active":"2024-04-22T20:48:23.628532+0000","last_peered":"2024-04-22T20:48:23.628532+0000","last_clean":"2024-04-22T20:48:23.628532+0000","last_became_active":"2024-04-22T20:01:04.877755+0000","last_became_peered":"2024-04-22T20:01:04.877755+0000","last_unstale":"2024-04-22T20:48:23.628532+0000","last_undegraded":"2024-04-22T20:48:23.628532+0000","last_fullsized":"2024-04-22T20:48:23.628532+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'47","last_scrub_stamp":"2024-04-22T20:48:23.628425+0000","last_deep_scrub":"261'47","last_deep_scrub_stamp":"2024-04-22T20:48:23.628425+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.628425+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:41:22.945025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":193,"num_read_kb":135,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"261'2","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:20.736240+0000","last_change":"2024-04-22T20:49:20.736240+0000","last_active":"2024-04-22T20:49:20.736240+0000","last_peered":"2024-04-22T20:49:20.736240+0000","last_clean":"2024-04-22T20:49:20.736240+0000","last_became_active":"2024-04-22T20:19:16.793555+0000","last_became_peered":"2024-04-22T20:19:16.793555+0000","last_unstale":"2024-04-22T20:49:20.736240+0000","last_undegraded":"2024-04-22T20:49:20.736240+0000","last_fullsized":"2024-04-22T20:49:20.736240+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:20.736112+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:20.736112+0000","last_clean_scrub_stamp":"2024-04-22T20:49:20.736112+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:41:57.356363+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"258'2","reported_seq":54,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.475071+0000","last_change":"2024-04-22T20:48:36.475071+0000","last_active":"2024-04-22T20:48:36.475071+0000","last_peered":"2024-04-22T20:48:36.475071+0000","last_clean":"2024-04-22T20:48:36.475071+0000","last_became_active":"2024-04-22T20:05:17.890070+0000","last_became_peered":"2024-04-22T20:05:17.890070+0000","last_unstale":"2024-04-22T20:48:36.475071+0000","last_undegraded":"2024-04-22T20:48:36.475071+0000","last_fullsized":"2024-04-22T20:48:36.475071+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'2","last_scrub_stamp":"2024-04-22T20:48:36.474953+0000","last_deep_scrub":"258'2","last_deep_scrub_stamp":"2024-04-22T20:48:36.474953+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.474953+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:22:16.222599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"261'5","reported_seq":64,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.415205+0000","last_change":"2024-04-22T20:48:30.415205+0000","last_active":"2024-04-22T20:48:30.415205+0000","last_peered":"2024-04-22T20:48:30.415205+0000","last_clean":"2024-04-22T20:48:30.415205+0000","last_became_active":"2024-04-22T20:05:13.806882+0000","last_became_peered":"2024-04-22T20:05:13.806882+0000","last_unstale":"2024-04-22T20:48:30.415205+0000","last_undegraded":"2024-04-22T20:48:30.415205+0000","last_fullsized":"2024-04-22T20:48:30.415205+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:30.415089+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:30.415089+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.415089+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:11:39.469597+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":6144,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"261'11","reported_seq":89,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.276700+0000","last_change":"2024-04-22T20:48:25.276728+0000","last_active":"2024-04-22T20:48:25.276700+0000","last_peered":"2024-04-22T20:48:25.276700+0000","last_clean":"2024-04-22T20:48:25.276700+0000","last_became_active":"2024-04-22T20:01:04.860991+0000","last_became_peered":"2024-04-22T20:01:04.860991+0000","last_unstale":"2024-04-22T20:48:25.276700+0000","last_undegraded":"2024-04-22T20:48:25.276700+0000","last_fullsized":"2024-04-22T20:48:25.276700+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'11","last_scrub_stamp":"2024-04-22T20:48:25.276600+0000","last_deep_scrub":"261'11","last_deep_scrub_stamp":"2024-04-22T20:48:25.276600+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.276600+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:39:08.585319+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":31,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"261'2","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:24.240697+0000","last_change":"2024-04-22T20:49:24.240735+0000","last_active":"2024-04-22T20:49:24.240697+0000","last_peered":"2024-04-22T20:49:24.240697+0000","last_clean":"2024-04-22T20:49:24.240697+0000","last_became_active":"2024-04-22T20:19:16.791995+0000","last_became_peered":"2024-04-22T20:19:16.791995+0000","last_unstale":"2024-04-22T20:49:24.240697+0000","last_undegraded":"2024-04-22T20:49:24.240697+0000","last_fullsized":"2024-04-22T20:49:24.240697+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:24.240601+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:24.240601+0000","last_clean_scrub_stamp":"2024-04-22T20:49:24.240601+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:53:39.725161+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"258'4","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.641997+0000","last_change":"2024-04-22T20:48:58.641997+0000","last_active":"2024-04-22T20:48:58.641997+0000","last_peered":"2024-04-22T20:48:58.641997+0000","last_clean":"2024-04-22T20:48:58.641997+0000","last_became_active":"2024-04-22T20:05:17.890408+0000","last_became_peered":"2024-04-22T20:05:17.890408+0000","last_unstale":"2024-04-22T20:48:58.641997+0000","last_undegraded":"2024-04-22T20:48:58.641997+0000","last_fullsized":"2024-04-22T20:48:58.641997+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:58.641819+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:58.641819+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.641819+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:00.757652+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1342,"num_write":4,"num_write_kb":1342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"258'4","reported_seq":64,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.956522+0000","last_change":"2024-04-22T20:48:35.956522+0000","last_active":"2024-04-22T20:48:35.956522+0000","last_peered":"2024-04-22T20:48:35.956522+0000","last_clean":"2024-04-22T20:48:35.956522+0000","last_became_active":"2024-04-22T20:05:13.808473+0000","last_became_peered":"2024-04-22T20:05:13.808473+0000","last_unstale":"2024-04-22T20:48:35.956522+0000","last_undegraded":"2024-04-22T20:48:35.956522+0000","last_fullsized":"2024-04-22T20:48:35.956522+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:35.956357+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:35.956357+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.956357+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:07:38.200109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"261'47","reported_seq":213,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.554713+0000","last_change":"2024-04-22T20:48:26.554763+0000","last_active":"2024-04-22T20:48:26.554713+0000","last_peered":"2024-04-22T20:48:26.554713+0000","last_clean":"2024-04-22T20:48:26.554713+0000","last_became_active":"2024-04-22T20:01:04.874816+0000","last_became_peered":"2024-04-22T20:01:04.874816+0000","last_unstale":"2024-04-22T20:48:26.554713+0000","last_undegraded":"2024-04-22T20:48:26.554713+0000","last_fullsized":"2024-04-22T20:48:26.554713+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'47","last_scrub_stamp":"2024-04-22T20:48:26.554614+0000","last_deep_scrub":"261'47","last_deep_scrub_stamp":"2024-04-22T20:48:26.554614+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.554614+0000","objects_scrubbed":1,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:35:39.583851+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":291,"num_read_kb":200,"num_write":62,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3253,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.555978+0000","last_change":"2024-04-22T20:48:53.555978+0000","last_active":"2024-04-22T20:48:53.555978+0000","last_peered":"2024-04-22T20:48:53.555978+0000","last_clean":"2024-04-22T20:48:53.555978+0000","last_became_active":"2024-04-22T20:19:16.794954+0000","last_became_peered":"2024-04-22T20:19:16.794954+0000","last_unstale":"2024-04-22T20:48:53.555978+0000","last_undegraded":"2024-04-22T20:48:53.555978+0000","last_fullsized":"2024-04-22T20:48:53.555978+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:53.555824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:53.555824+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.555824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:08:41.681280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"261'6","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.542755+0000","last_change":"2024-04-22T20:48:32.542810+0000","last_active":"2024-04-22T20:48:32.542755+0000","last_peered":"2024-04-22T20:48:32.542755+0000","last_clean":"2024-04-22T20:48:32.542755+0000","last_became_active":"2024-04-22T20:05:17.889370+0000","last_became_peered":"2024-04-22T20:05:17.889370+0000","last_unstale":"2024-04-22T20:48:32.542755+0000","last_undegraded":"2024-04-22T20:48:32.542755+0000","last_fullsized":"2024-04-22T20:48:32.542755+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:32.542653+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:32.542653+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.542653+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:05:37.965129+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4100,"num_write":6,"num_write_kb":5124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"261'2","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.599057+0000","last_change":"2024-04-22T20:48:25.599082+0000","last_active":"2024-04-22T20:48:25.599057+0000","last_peered":"2024-04-22T20:48:25.599057+0000","last_clean":"2024-04-22T20:48:25.599057+0000","last_became_active":"2024-04-22T20:05:13.815317+0000","last_became_peered":"2024-04-22T20:05:13.815317+0000","last_unstale":"2024-04-22T20:48:25.599057+0000","last_undegraded":"2024-04-22T20:48:25.599057+0000","last_fullsized":"2024-04-22T20:48:25.599057+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:25.598973+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:25.598973+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.598973+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:21:44.105372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"261'28","reported_seq":128,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.478567+0000","last_change":"2024-04-22T20:48:23.478567+0000","last_active":"2024-04-22T20:48:23.478567+0000","last_peered":"2024-04-22T20:48:23.478567+0000","last_clean":"2024-04-22T20:48:23.478567+0000","last_became_active":"2024-04-22T20:01:04.878599+0000","last_became_peered":"2024-04-22T20:01:04.878599+0000","last_unstale":"2024-04-22T20:48:23.478567+0000","last_undegraded":"2024-04-22T20:48:23.478567+0000","last_fullsized":"2024-04-22T20:48:23.478567+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:23.478425+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:23.478425+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.478425+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:13:33.610274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":77,"num_write":54,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:15.594051+0000","last_change":"2024-04-22T20:49:15.594073+0000","last_active":"2024-04-22T20:49:15.594051+0000","last_peered":"2024-04-22T20:49:15.594051+0000","last_clean":"2024-04-22T20:49:15.594051+0000","last_became_active":"2024-04-22T20:19:16.793678+0000","last_became_peered":"2024-04-22T20:19:16.793678+0000","last_unstale":"2024-04-22T20:49:15.594051+0000","last_undegraded":"2024-04-22T20:49:15.594051+0000","last_fullsized":"2024-04-22T20:49:15.594051+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:15.593942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:15.593942+0000","last_clean_scrub_stamp":"2024-04-22T20:49:15.593942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:08.495410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"0'0","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.495293+0000","last_change":"2024-04-22T20:48:40.495316+0000","last_active":"2024-04-22T20:48:40.495293+0000","last_peered":"2024-04-22T20:48:40.495293+0000","last_clean":"2024-04-22T20:48:40.495293+0000","last_became_active":"2024-04-22T20:05:17.892604+0000","last_became_peered":"2024-04-22T20:05:17.892604+0000","last_unstale":"2024-04-22T20:48:40.495293+0000","last_undegraded":"2024-04-22T20:48:40.495293+0000","last_fullsized":"2024-04-22T20:48:40.495293+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.495210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.495210+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.495210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:53:52.170315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.d","version":"261'11","reported_seq":78,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.602185+0000","last_change":"2024-04-22T20:48:48.602213+0000","last_active":"2024-04-22T20:48:48.602185+0000","last_peered":"2024-04-22T20:48:48.602185+0000","last_clean":"2024-04-22T20:48:48.602185+0000","last_became_active":"2024-04-22T20:05:13.813946+0000","last_became_peered":"2024-04-22T20:05:13.813946+0000","last_unstale":"2024-04-22T20:48:48.602185+0000","last_undegraded":"2024-04-22T20:48:48.602185+0000","last_fullsized":"2024-04-22T20:48:48.602185+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'11","last_scrub_stamp":"2024-04-22T20:48:48.602091+0000","last_deep_scrub":"261'11","last_deep_scrub_stamp":"2024-04-22T20:48:48.602091+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.602091+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:32.170316+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":15362,"num_write":9,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"261'28","reported_seq":136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.953873+0000","last_change":"2024-04-22T20:48:31.953898+0000","last_active":"2024-04-22T20:48:31.953873+0000","last_peered":"2024-04-22T20:48:31.953873+0000","last_clean":"2024-04-22T20:48:31.953873+0000","last_became_active":"2024-04-22T20:01:04.872834+0000","last_became_peered":"2024-04-22T20:01:04.872834+0000","last_unstale":"2024-04-22T20:48:31.953873+0000","last_undegraded":"2024-04-22T20:48:31.953873+0000","last_fullsized":"2024-04-22T20:48:31.953873+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:31.953787+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:31.953787+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.953787+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:05:33.171326+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":90,"num_write":50,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.713000+0000","last_change":"2024-04-22T20:49:02.713000+0000","last_active":"2024-04-22T20:49:02.713000+0000","last_peered":"2024-04-22T20:49:02.713000+0000","last_clean":"2024-04-22T20:49:02.713000+0000","last_became_active":"2024-04-22T20:19:16.790917+0000","last_became_peered":"2024-04-22T20:19:16.790917+0000","last_unstale":"2024-04-22T20:49:02.713000+0000","last_undegraded":"2024-04-22T20:49:02.713000+0000","last_fullsized":"2024-04-22T20:49:02.713000+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.712882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.712882+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.712882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:02.266687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":50,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.612835+0000","last_change":"2024-04-22T20:49:01.612858+0000","last_active":"2024-04-22T20:49:01.612835+0000","last_peered":"2024-04-22T20:49:01.612835+0000","last_clean":"2024-04-22T20:49:01.612835+0000","last_became_active":"2024-04-22T20:05:18.211445+0000","last_became_peered":"2024-04-22T20:05:18.211445+0000","last_unstale":"2024-04-22T20:49:01.612835+0000","last_undegraded":"2024-04-22T20:49:01.612835+0000","last_fullsized":"2024-04-22T20:49:01.612835+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:01.612734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:01.612734+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.612734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:00:33.333955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"258'2","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.661659+0000","last_change":"2024-04-22T20:48:22.661681+0000","last_active":"2024-04-22T20:48:22.661659+0000","last_peered":"2024-04-22T20:48:22.661659+0000","last_clean":"2024-04-22T20:48:22.661659+0000","last_became_active":"2024-04-22T20:05:13.806928+0000","last_became_peered":"2024-04-22T20:05:13.806928+0000","last_unstale":"2024-04-22T20:48:22.661659+0000","last_undegraded":"2024-04-22T20:48:22.661659+0000","last_fullsized":"2024-04-22T20:48:22.661659+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'2","last_scrub_stamp":"2024-04-22T20:48:22.661579+0000","last_deep_scrub":"258'2","last_deep_scrub_stamp":"2024-04-22T20:48:22.661579+0000","last_clean_scrub_stamp":"2024-04-22T20:48:22.661579+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:42.533500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"261'28","reported_seq":127,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.409573+0000","last_change":"2024-04-22T20:48:27.409602+0000","last_active":"2024-04-22T20:48:27.409573+0000","last_peered":"2024-04-22T20:48:27.409573+0000","last_clean":"2024-04-22T20:48:27.409573+0000","last_became_active":"2024-04-22T20:01:04.869226+0000","last_became_peered":"2024-04-22T20:01:04.869226+0000","last_unstale":"2024-04-22T20:48:27.409573+0000","last_undegraded":"2024-04-22T20:48:27.409573+0000","last_fullsized":"2024-04-22T20:48:27.409573+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:27.409466+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:27.409466+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.409466+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:07.279566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":80,"num_write":56,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:18.553879+0000","last_change":"2024-04-22T20:49:18.553879+0000","last_active":"2024-04-22T20:49:18.553879+0000","last_peered":"2024-04-22T20:49:18.553879+0000","last_clean":"2024-04-22T20:49:18.553879+0000","last_became_active":"2024-04-22T20:19:16.795524+0000","last_became_peered":"2024-04-22T20:19:16.795524+0000","last_unstale":"2024-04-22T20:49:18.553879+0000","last_undegraded":"2024-04-22T20:49:18.553879+0000","last_fullsized":"2024-04-22T20:49:18.553879+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:18.553763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:18.553763+0000","last_clean_scrub_stamp":"2024-04-22T20:49:18.553763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:07:42.897592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"261'4","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.484819+0000","last_change":"2024-04-22T20:48:48.484845+0000","last_active":"2024-04-22T20:48:48.484819+0000","last_peered":"2024-04-22T20:48:48.484819+0000","last_clean":"2024-04-22T20:48:48.484819+0000","last_became_active":"2024-04-22T20:05:17.898858+0000","last_became_peered":"2024-04-22T20:05:17.898858+0000","last_unstale":"2024-04-22T20:48:48.484819+0000","last_undegraded":"2024-04-22T20:48:48.484819+0000","last_fullsized":"2024-04-22T20:48:48.484819+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:48.484715+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:48.484715+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.484715+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:42:15.414201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6144,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"258'3","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.510708+0000","last_change":"2024-04-22T20:48:27.510738+0000","last_active":"2024-04-22T20:48:27.510708+0000","last_peered":"2024-04-22T20:48:27.510708+0000","last_clean":"2024-04-22T20:48:27.510708+0000","last_became_active":"2024-04-22T20:05:13.807945+0000","last_became_peered":"2024-04-22T20:05:13.807945+0000","last_unstale":"2024-04-22T20:48:27.510708+0000","last_undegraded":"2024-04-22T20:48:27.510708+0000","last_fullsized":"2024-04-22T20:48:27.510708+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'3","last_scrub_stamp":"2024-04-22T20:48:27.510574+0000","last_deep_scrub":"258'3","last_deep_scrub_stamp":"2024-04-22T20:48:27.510574+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.510574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:46.475027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":4096,"num_write":8,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"261'24","reported_seq":115,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.452597+0000","last_change":"2024-04-22T20:48:32.452634+0000","last_active":"2024-04-22T20:48:32.452597+0000","last_peered":"2024-04-22T20:48:32.452597+0000","last_clean":"2024-04-22T20:48:32.452597+0000","last_became_active":"2024-04-22T20:01:04.871829+0000","last_became_peered":"2024-04-22T20:01:04.871829+0000","last_unstale":"2024-04-22T20:48:32.452597+0000","last_undegraded":"2024-04-22T20:48:32.452597+0000","last_fullsized":"2024-04-22T20:48:32.452597+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'24","last_scrub_stamp":"2024-04-22T20:48:32.452510+0000","last_deep_scrub":"261'24","last_deep_scrub_stamp":"2024-04-22T20:48:32.452510+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.452510+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:57.685562+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":61,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:22.288257+0000","last_change":"2024-04-22T20:49:22.288282+0000","last_active":"2024-04-22T20:49:22.288257+0000","last_peered":"2024-04-22T20:49:22.288257+0000","last_clean":"2024-04-22T20:49:22.288257+0000","last_became_active":"2024-04-22T20:19:16.793727+0000","last_became_peered":"2024-04-22T20:19:16.793727+0000","last_unstale":"2024-04-22T20:49:22.288257+0000","last_undegraded":"2024-04-22T20:49:22.288257+0000","last_fullsized":"2024-04-22T20:49:22.288257+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:22.288151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:22.288151+0000","last_clean_scrub_stamp":"2024-04-22T20:49:22.288151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:45:38.361185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"261'5","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.253489+0000","last_change":"2024-04-22T20:49:08.253566+0000","last_active":"2024-04-22T20:49:08.253489+0000","last_peered":"2024-04-22T20:49:08.253489+0000","last_clean":"2024-04-22T20:49:08.253489+0000","last_became_active":"2024-04-22T20:05:17.899328+0000","last_became_peered":"2024-04-22T20:05:17.899328+0000","last_unstale":"2024-04-22T20:49:08.253489+0000","last_undegraded":"2024-04-22T20:49:08.253489+0000","last_fullsized":"2024-04-22T20:49:08.253489+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:08.253395+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:08.253395+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.253395+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:22.668162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":10,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"261'7","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.630926+0000","last_change":"2024-04-22T20:48:25.630926+0000","last_active":"2024-04-22T20:48:25.630926+0000","last_peered":"2024-04-22T20:48:25.630926+0000","last_clean":"2024-04-22T20:48:25.630926+0000","last_became_active":"2024-04-22T20:05:13.811986+0000","last_became_peered":"2024-04-22T20:05:13.811986+0000","last_unstale":"2024-04-22T20:48:25.630926+0000","last_undegraded":"2024-04-22T20:48:25.630926+0000","last_fullsized":"2024-04-22T20:48:25.630926+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:25.630769+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:25.630769+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.630769+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:31:51.985237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":8240,"num_write":12,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"261'66","reported_seq":221,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.587356+0000","last_change":"2024-04-22T20:48:26.587419+0000","last_active":"2024-04-22T20:48:26.587356+0000","last_peered":"2024-04-22T20:48:26.587356+0000","last_clean":"2024-04-22T20:48:26.587356+0000","last_became_active":"2024-04-22T20:01:04.874827+0000","last_became_peered":"2024-04-22T20:01:04.874827+0000","last_unstale":"2024-04-22T20:48:26.587356+0000","last_undegraded":"2024-04-22T20:48:26.587356+0000","last_fullsized":"2024-04-22T20:48:26.587356+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'66","last_scrub_stamp":"2024-04-22T20:48:26.587233+0000","last_deep_scrub":"261'66","last_deep_scrub_stamp":"2024-04-22T20:48:26.587233+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.587233+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:32.347942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":281,"num_read_kb":200,"num_write":112,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.423664+0000","last_change":"2024-04-22T20:48:44.423664+0000","last_active":"2024-04-22T20:48:44.423664+0000","last_peered":"2024-04-22T20:48:44.423664+0000","last_clean":"2024-04-22T20:48:44.423664+0000","last_became_active":"2024-04-22T20:19:16.794434+0000","last_became_peered":"2024-04-22T20:19:16.794434+0000","last_unstale":"2024-04-22T20:48:44.423664+0000","last_undegraded":"2024-04-22T20:48:44.423664+0000","last_fullsized":"2024-04-22T20:48:44.423664+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:44.423537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:44.423537+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.423537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:22:23.617146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"261'9","reported_seq":65,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.412462+0000","last_change":"2024-04-22T20:48:36.412462+0000","last_active":"2024-04-22T20:48:36.412462+0000","last_peered":"2024-04-22T20:48:36.412462+0000","last_clean":"2024-04-22T20:48:36.412462+0000","last_became_active":"2024-04-22T20:05:18.209025+0000","last_became_peered":"2024-04-22T20:05:18.209025+0000","last_unstale":"2024-04-22T20:48:36.412462+0000","last_undegraded":"2024-04-22T20:48:36.412462+0000","last_fullsized":"2024-04-22T20:48:36.412462+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'9","last_scrub_stamp":"2024-04-22T20:48:36.412356+0000","last_deep_scrub":"261'9","last_deep_scrub_stamp":"2024-04-22T20:48:36.412356+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.412356+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:50:02.300050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":13312,"num_write":14,"num_write_kb":13312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"261'4","reported_seq":63,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:22.619549+0000","last_change":"2024-04-22T20:48:22.619549+0000","last_active":"2024-04-22T20:48:22.619549+0000","last_peered":"2024-04-22T20:48:22.619549+0000","last_clean":"2024-04-22T20:48:22.619549+0000","last_became_active":"2024-04-22T20:05:13.807603+0000","last_became_peered":"2024-04-22T20:05:13.807603+0000","last_unstale":"2024-04-22T20:48:22.619549+0000","last_undegraded":"2024-04-22T20:48:22.619549+0000","last_fullsized":"2024-04-22T20:48:22.619549+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:22.619402+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:22.619402+0000","last_clean_scrub_stamp":"2024-04-22T20:48:22.619402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:32:09.200890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"258'23","reported_seq":122,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.404488+0000","last_change":"2024-04-22T20:48:37.404488+0000","last_active":"2024-04-22T20:48:37.404488+0000","last_peered":"2024-04-22T20:48:37.404488+0000","last_clean":"2024-04-22T20:48:37.404488+0000","last_became_active":"2024-04-22T20:01:04.868847+0000","last_became_peered":"2024-04-22T20:01:04.868847+0000","last_unstale":"2024-04-22T20:48:37.404488+0000","last_undegraded":"2024-04-22T20:48:37.404488+0000","last_fullsized":"2024-04-22T20:48:37.404488+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'23","last_scrub_stamp":"2024-04-22T20:48:37.404364+0000","last_deep_scrub":"258'23","last_deep_scrub_stamp":"2024-04-22T20:48:37.404364+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.404364+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:24:31.027402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":76,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"261'2","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:19.587547+0000","last_change":"2024-04-22T20:49:19.587578+0000","last_active":"2024-04-22T20:49:19.587547+0000","last_peered":"2024-04-22T20:49:19.587547+0000","last_clean":"2024-04-22T20:49:19.587547+0000","last_became_active":"2024-04-22T20:19:16.795613+0000","last_became_peered":"2024-04-22T20:19:16.795613+0000","last_unstale":"2024-04-22T20:49:19.587547+0000","last_undegraded":"2024-04-22T20:49:19.587547+0000","last_fullsized":"2024-04-22T20:49:19.587547+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:19.587456+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:19.587456+0000","last_clean_scrub_stamp":"2024-04-22T20:49:19.587456+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:54:03.988371+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"261'6","reported_seq":60,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.538494+0000","last_change":"2024-04-22T20:48:31.538524+0000","last_active":"2024-04-22T20:48:31.538494+0000","last_peered":"2024-04-22T20:48:31.538494+0000","last_clean":"2024-04-22T20:48:31.538494+0000","last_became_active":"2024-04-22T20:05:17.891331+0000","last_became_peered":"2024-04-22T20:05:17.891331+0000","last_unstale":"2024-04-22T20:48:31.538494+0000","last_undegraded":"2024-04-22T20:48:31.538494+0000","last_fullsized":"2024-04-22T20:48:31.538494+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:31.538340+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:31.538340+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.538340+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:36:13.794601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":10240,"num_write":16,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"0'0","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.504525+0000","last_change":"2024-04-22T20:48:37.504525+0000","last_active":"2024-04-22T20:48:37.504525+0000","last_peered":"2024-04-22T20:48:37.504525+0000","last_clean":"2024-04-22T20:48:37.504525+0000","last_became_active":"2024-04-22T20:05:13.813811+0000","last_became_peered":"2024-04-22T20:05:13.813811+0000","last_unstale":"2024-04-22T20:48:37.504525+0000","last_undegraded":"2024-04-22T20:48:37.504525+0000","last_fullsized":"2024-04-22T20:48:37.504525+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:37.504369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:37.504369+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.504369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:22.962064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"258'49","reported_seq":180,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.613719+0000","last_change":"2024-04-22T20:48:29.613719+0000","last_active":"2024-04-22T20:48:29.613719+0000","last_peered":"2024-04-22T20:48:29.613719+0000","last_clean":"2024-04-22T20:48:29.613719+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:48:29.613719+0000","last_undegraded":"2024-04-22T20:48:29.613719+0000","last_fullsized":"2024-04-22T20:48:29.613719+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'49","last_scrub_stamp":"2024-04-22T20:48:29.613592+0000","last_deep_scrub":"258'49","last_deep_scrub_stamp":"2024-04-22T20:48:29.613592+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.613592+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:50:57.860950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":205,"num_read_kb":145,"num_write":78,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":34,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:19.684330+0000","last_change":"2024-04-22T20:49:19.684353+0000","last_active":"2024-04-22T20:49:19.684330+0000","last_peered":"2024-04-22T20:49:19.684330+0000","last_clean":"2024-04-22T20:49:19.684330+0000","last_became_active":"2024-04-22T20:19:16.793486+0000","last_became_peered":"2024-04-22T20:19:16.793486+0000","last_unstale":"2024-04-22T20:49:19.684330+0000","last_undegraded":"2024-04-22T20:49:19.684330+0000","last_fullsized":"2024-04-22T20:49:19.684330+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:19.684234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:19.684234+0000","last_clean_scrub_stamp":"2024-04-22T20:49:19.684234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:13:31.944544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"0'0","reported_seq":51,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.489207+0000","last_change":"2024-04-22T20:48:32.489207+0000","last_active":"2024-04-22T20:48:32.489207+0000","last_peered":"2024-04-22T20:48:32.489207+0000","last_clean":"2024-04-22T20:48:32.489207+0000","last_became_active":"2024-04-22T20:05:17.897008+0000","last_became_peered":"2024-04-22T20:05:17.897008+0000","last_unstale":"2024-04-22T20:48:32.489207+0000","last_undegraded":"2024-04-22T20:48:32.489207+0000","last_fullsized":"2024-04-22T20:48:32.489207+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:32.489059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:32.489059+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.489059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:11.685871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"261'15","reported_seq":85,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.378030+0000","last_change":"2024-04-22T20:48:32.378065+0000","last_active":"2024-04-22T20:48:32.378030+0000","last_peered":"2024-04-22T20:48:32.378030+0000","last_clean":"2024-04-22T20:48:32.378030+0000","last_became_active":"2024-04-22T20:05:13.798319+0000","last_became_peered":"2024-04-22T20:05:13.798319+0000","last_unstale":"2024-04-22T20:48:32.378030+0000","last_undegraded":"2024-04-22T20:48:32.378030+0000","last_fullsized":"2024-04-22T20:48:32.378030+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'15","last_scrub_stamp":"2024-04-22T20:48:32.377942+0000","last_deep_scrub":"261'15","last_deep_scrub_stamp":"2024-04-22T20:48:32.377942+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.377942+0000","objects_scrubbed":2,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:17:58.634704+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":16388,"num_write":16,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"261'37","reported_seq":148,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.458538+0000","last_change":"2024-04-22T20:48:28.458538+0000","last_active":"2024-04-22T20:48:28.458538+0000","last_peered":"2024-04-22T20:48:28.458538+0000","last_clean":"2024-04-22T20:48:28.458538+0000","last_became_active":"2024-04-22T20:01:04.869079+0000","last_became_peered":"2024-04-22T20:01:04.869079+0000","last_unstale":"2024-04-22T20:48:28.458538+0000","last_undegraded":"2024-04-22T20:48:28.458538+0000","last_fullsized":"2024-04-22T20:48:28.458538+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'37","last_scrub_stamp":"2024-04-22T20:48:28.458400+0000","last_deep_scrub":"261'37","last_deep_scrub_stamp":"2024-04-22T20:48:28.458400+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.458400+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:24:11.187493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":61,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:17.574269+0000","last_change":"2024-04-22T20:49:17.574296+0000","last_active":"2024-04-22T20:49:17.574269+0000","last_peered":"2024-04-22T20:49:17.574269+0000","last_clean":"2024-04-22T20:49:17.574269+0000","last_became_active":"2024-04-22T20:19:16.795462+0000","last_became_peered":"2024-04-22T20:19:16.795462+0000","last_unstale":"2024-04-22T20:49:17.574269+0000","last_undegraded":"2024-04-22T20:49:17.574269+0000","last_fullsized":"2024-04-22T20:49:17.574269+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:17.574180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:17.574180+0000","last_clean_scrub_stamp":"2024-04-22T20:49:17.574180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:50:53.041281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"261'7","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.601169+0000","last_change":"2024-04-22T20:48:45.601169+0000","last_active":"2024-04-22T20:48:45.601169+0000","last_peered":"2024-04-22T20:48:45.601169+0000","last_clean":"2024-04-22T20:48:45.601169+0000","last_became_active":"2024-04-22T20:05:17.892044+0000","last_became_peered":"2024-04-22T20:05:17.892044+0000","last_unstale":"2024-04-22T20:48:45.601169+0000","last_undegraded":"2024-04-22T20:48:45.601169+0000","last_fullsized":"2024-04-22T20:48:45.601169+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:45.601054+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:45.601054+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.601054+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:42:15.237225+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":5,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"261'5","reported_seq":69,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.541577+0000","last_change":"2024-04-22T20:48:29.541577+0000","last_active":"2024-04-22T20:48:29.541577+0000","last_peered":"2024-04-22T20:48:29.541577+0000","last_clean":"2024-04-22T20:48:29.541577+0000","last_became_active":"2024-04-22T20:05:13.809368+0000","last_became_peered":"2024-04-22T20:05:13.809368+0000","last_unstale":"2024-04-22T20:48:29.541577+0000","last_undegraded":"2024-04-22T20:48:29.541577+0000","last_fullsized":"2024-04-22T20:48:29.541577+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:29.541434+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:29.541434+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.541434+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:38:26.695883+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":16386,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"261'35","reported_seq":148,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.558111+0000","last_change":"2024-04-22T20:48:27.558111+0000","last_active":"2024-04-22T20:48:27.558111+0000","last_peered":"2024-04-22T20:48:27.558111+0000","last_clean":"2024-04-22T20:48:27.558111+0000","last_became_active":"2024-04-22T20:01:04.873748+0000","last_became_peered":"2024-04-22T20:01:04.873748+0000","last_unstale":"2024-04-22T20:48:27.558111+0000","last_undegraded":"2024-04-22T20:48:27.558111+0000","last_fullsized":"2024-04-22T20:48:27.558111+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'35","last_scrub_stamp":"2024-04-22T20:48:27.558001+0000","last_deep_scrub":"261'35","last_deep_scrub_stamp":"2024-04-22T20:48:27.558001+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.558001+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:44:57.759608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":146,"num_read_kb":103,"num_write":59,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.823339+0000","last_change":"2024-04-22T20:49:10.823362+0000","last_active":"2024-04-22T20:49:10.823339+0000","last_peered":"2024-04-22T20:49:10.823339+0000","last_clean":"2024-04-22T20:49:10.823339+0000","last_became_active":"2024-04-22T20:19:16.791663+0000","last_became_peered":"2024-04-22T20:19:16.791663+0000","last_unstale":"2024-04-22T20:49:10.823339+0000","last_undegraded":"2024-04-22T20:49:10.823339+0000","last_fullsized":"2024-04-22T20:49:10.823339+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:10.823245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:10.823245+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.823245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:09:49.787690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"261'7","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.669783+0000","last_change":"2024-04-22T20:48:49.669783+0000","last_active":"2024-04-22T20:48:49.669783+0000","last_peered":"2024-04-22T20:48:49.669783+0000","last_clean":"2024-04-22T20:48:49.669783+0000","last_became_active":"2024-04-22T20:05:18.210736+0000","last_became_peered":"2024-04-22T20:05:18.210736+0000","last_unstale":"2024-04-22T20:48:49.669783+0000","last_undegraded":"2024-04-22T20:48:49.669783+0000","last_fullsized":"2024-04-22T20:48:49.669783+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:48:49.669616+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:48:49.669616+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.669616+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:27:27.520540+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":20482,"num_write":5,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"261'6","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.576941+0000","last_change":"2024-04-22T20:48:47.576941+0000","last_active":"2024-04-22T20:48:47.576941+0000","last_peered":"2024-04-22T20:48:47.576941+0000","last_clean":"2024-04-22T20:48:47.576941+0000","last_became_active":"2024-04-22T20:05:13.813277+0000","last_became_peered":"2024-04-22T20:05:13.813277+0000","last_unstale":"2024-04-22T20:48:47.576941+0000","last_undegraded":"2024-04-22T20:48:47.576941+0000","last_fullsized":"2024-04-22T20:48:47.576941+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:47.576814+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:47.576814+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.576814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:43:34.331220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1024,"num_write":16,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"261'61","reported_seq":204,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.363698+0000","last_change":"2024-04-22T20:48:25.363698+0000","last_active":"2024-04-22T20:48:25.363698+0000","last_peered":"2024-04-22T20:48:25.363698+0000","last_clean":"2024-04-22T20:48:25.363698+0000","last_became_active":"2024-04-22T20:01:04.871859+0000","last_became_peered":"2024-04-22T20:01:04.871859+0000","last_unstale":"2024-04-22T20:48:25.363698+0000","last_undegraded":"2024-04-22T20:48:25.363698+0000","last_fullsized":"2024-04-22T20:48:25.363698+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'61","last_scrub_stamp":"2024-04-22T20:48:25.363585+0000","last_deep_scrub":"261'61","last_deep_scrub_stamp":"2024-04-22T20:48:25.363585+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.363585+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:50:52.645407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":245,"num_read_kb":172,"num_write":109,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":36,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.659643+0000","last_change":"2024-04-22T20:49:03.659643+0000","last_active":"2024-04-22T20:49:03.659643+0000","last_peered":"2024-04-22T20:49:03.659643+0000","last_clean":"2024-04-22T20:49:03.659643+0000","last_became_active":"2024-04-22T20:19:16.789688+0000","last_became_peered":"2024-04-22T20:19:16.789688+0000","last_unstale":"2024-04-22T20:49:03.659643+0000","last_undegraded":"2024-04-22T20:49:03.659643+0000","last_fullsized":"2024-04-22T20:49:03.659643+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:03.659497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:03.659497+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.659497+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:02.426808+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"258'4","reported_seq":59,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.411626+0000","last_change":"2024-04-22T20:48:40.411671+0000","last_active":"2024-04-22T20:48:40.411626+0000","last_peered":"2024-04-22T20:48:40.411626+0000","last_clean":"2024-04-22T20:48:40.411626+0000","last_became_active":"2024-04-22T20:05:18.211478+0000","last_became_peered":"2024-04-22T20:05:18.211478+0000","last_unstale":"2024-04-22T20:48:40.411626+0000","last_undegraded":"2024-04-22T20:48:40.411626+0000","last_fullsized":"2024-04-22T20:48:40.411626+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"258'4","last_scrub_stamp":"2024-04-22T20:48:40.411479+0000","last_deep_scrub":"258'4","last_deep_scrub_stamp":"2024-04-22T20:48:40.411479+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.411479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:39:01.313336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"261'7","reported_seq":67,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.885170+0000","last_change":"2024-04-22T20:49:06.885194+0000","last_active":"2024-04-22T20:49:06.885170+0000","last_peered":"2024-04-22T20:49:06.885170+0000","last_clean":"2024-04-22T20:49:06.885170+0000","last_became_active":"2024-04-22T20:05:13.811561+0000","last_became_peered":"2024-04-22T20:05:13.811561+0000","last_unstale":"2024-04-22T20:49:06.885170+0000","last_undegraded":"2024-04-22T20:49:06.885170+0000","last_fullsized":"2024-04-22T20:49:06.885170+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'7","last_scrub_stamp":"2024-04-22T20:49:06.885081+0000","last_deep_scrub":"261'7","last_deep_scrub_stamp":"2024-04-22T20:49:06.885081+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.885081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:02.675990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":5120,"num_write":12,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"261'33","reported_seq":145,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.529020+0000","last_change":"2024-04-22T20:48:31.529020+0000","last_active":"2024-04-22T20:48:31.529020+0000","last_peered":"2024-04-22T20:48:31.529020+0000","last_clean":"2024-04-22T20:48:31.529020+0000","last_became_active":"2024-04-22T20:01:04.874790+0000","last_became_peered":"2024-04-22T20:01:04.874790+0000","last_unstale":"2024-04-22T20:48:31.529020+0000","last_undegraded":"2024-04-22T20:48:31.529020+0000","last_fullsized":"2024-04-22T20:48:31.529020+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'33","last_scrub_stamp":"2024-04-22T20:48:31.528870+0000","last_deep_scrub":"261'33","last_deep_scrub_stamp":"2024-04-22T20:48:31.528870+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.528870+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:31:43.937257+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":100,"num_write":57,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:13.233179+0000","last_change":"2024-04-22T20:49:13.233204+0000","last_active":"2024-04-22T20:49:13.233179+0000","last_peered":"2024-04-22T20:49:13.233179+0000","last_clean":"2024-04-22T20:49:13.233179+0000","last_became_active":"2024-04-22T20:19:16.793188+0000","last_became_peered":"2024-04-22T20:19:16.793188+0000","last_unstale":"2024-04-22T20:49:13.233179+0000","last_undegraded":"2024-04-22T20:49:13.233179+0000","last_fullsized":"2024-04-22T20:49:13.233179+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:13.233055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:13.233055+0000","last_clean_scrub_stamp":"2024-04-22T20:49:13.233055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:19:48.659861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"261'5","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.199142+0000","last_change":"2024-04-22T20:49:10.199194+0000","last_active":"2024-04-22T20:49:10.199142+0000","last_peered":"2024-04-22T20:49:10.199142+0000","last_clean":"2024-04-22T20:49:10.199142+0000","last_became_active":"2024-04-22T20:05:17.901128+0000","last_became_peered":"2024-04-22T20:05:17.901128+0000","last_unstale":"2024-04-22T20:49:10.199142+0000","last_undegraded":"2024-04-22T20:49:10.199142+0000","last_fullsized":"2024-04-22T20:49:10.199142+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:49:10.199016+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:49:10.199016+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.199016+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:13:10.213103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":8192,"num_write":10,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"261'5","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.568012+0000","last_change":"2024-04-22T20:48:28.568012+0000","last_active":"2024-04-22T20:48:28.568012+0000","last_peered":"2024-04-22T20:48:28.568012+0000","last_clean":"2024-04-22T20:48:28.568012+0000","last_became_active":"2024-04-22T20:05:13.815332+0000","last_became_peered":"2024-04-22T20:05:13.815332+0000","last_unstale":"2024-04-22T20:48:28.568012+0000","last_undegraded":"2024-04-22T20:48:28.568012+0000","last_fullsized":"2024-04-22T20:48:28.568012+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'5","last_scrub_stamp":"2024-04-22T20:48:28.567863+0000","last_deep_scrub":"261'5","last_deep_scrub_stamp":"2024-04-22T20:48:28.567863+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.567863+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:52:21.493631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":9216,"num_write":10,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"261'28","reported_seq":134,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.541277+0000","last_change":"2024-04-22T20:48:26.541277+0000","last_active":"2024-04-22T20:48:26.541277+0000","last_peered":"2024-04-22T20:48:26.541277+0000","last_clean":"2024-04-22T20:48:26.541277+0000","last_became_active":"2024-04-22T20:01:04.871840+0000","last_became_peered":"2024-04-22T20:01:04.871840+0000","last_unstale":"2024-04-22T20:48:26.541277+0000","last_undegraded":"2024-04-22T20:48:26.541277+0000","last_fullsized":"2024-04-22T20:48:26.541277+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'28","last_scrub_stamp":"2024-04-22T20:48:26.541161+0000","last_deep_scrub":"261'28","last_deep_scrub_stamp":"2024-04-22T20:48:26.541161+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.541161+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:03:51.347595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":89,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"261'2","reported_seq":37,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.585467+0000","last_change":"2024-04-22T20:48:52.585467+0000","last_active":"2024-04-22T20:48:52.585467+0000","last_peered":"2024-04-22T20:48:52.585467+0000","last_clean":"2024-04-22T20:48:52.585467+0000","last_became_active":"2024-04-22T20:19:16.794123+0000","last_became_peered":"2024-04-22T20:19:16.794123+0000","last_unstale":"2024-04-22T20:48:52.585467+0000","last_undegraded":"2024-04-22T20:48:52.585467+0000","last_fullsized":"2024-04-22T20:48:52.585467+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:52.585332+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:52.585332+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.585332+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:40:48.233250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"261'4","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.624822+0000","last_change":"2024-04-22T20:48:58.624822+0000","last_active":"2024-04-22T20:48:58.624822+0000","last_peered":"2024-04-22T20:48:58.624822+0000","last_clean":"2024-04-22T20:48:58.624822+0000","last_became_active":"2024-04-22T20:05:17.895177+0000","last_became_peered":"2024-04-22T20:05:17.895177+0000","last_unstale":"2024-04-22T20:48:58.624822+0000","last_undegraded":"2024-04-22T20:48:58.624822+0000","last_fullsized":"2024-04-22T20:48:58.624822+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:58.624700+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:58.624700+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.624700+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:42:11.281730+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":4,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"261'4","reported_seq":66,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.553078+0000","last_change":"2024-04-22T20:48:30.553078+0000","last_active":"2024-04-22T20:48:30.553078+0000","last_peered":"2024-04-22T20:48:30.553078+0000","last_clean":"2024-04-22T20:48:30.553078+0000","last_became_active":"2024-04-22T20:05:13.809047+0000","last_became_peered":"2024-04-22T20:05:13.809047+0000","last_unstale":"2024-04-22T20:48:30.553078+0000","last_undegraded":"2024-04-22T20:48:30.553078+0000","last_fullsized":"2024-04-22T20:48:30.553078+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:30.552959+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:30.552959+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.552959+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:44:03.327794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":12288,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"261'33","reported_seq":140,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.902332+0000","last_change":"2024-04-22T20:48:27.902332+0000","last_active":"2024-04-22T20:48:27.902332+0000","last_peered":"2024-04-22T20:48:27.902332+0000","last_clean":"2024-04-22T20:48:27.902332+0000","last_became_active":"2024-04-22T20:01:04.872370+0000","last_became_peered":"2024-04-22T20:01:04.872370+0000","last_unstale":"2024-04-22T20:48:27.902332+0000","last_undegraded":"2024-04-22T20:48:27.902332+0000","last_fullsized":"2024-04-22T20:48:27.902332+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'33","last_scrub_stamp":"2024-04-22T20:48:27.902204+0000","last_deep_scrub":"261'33","last_deep_scrub_stamp":"2024-04-22T20:48:27.902204+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.902204+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:57:41.905060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":89,"num_write":57,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:18.668649+0000","last_change":"2024-04-22T20:49:18.668649+0000","last_active":"2024-04-22T20:49:18.668649+0000","last_peered":"2024-04-22T20:49:18.668649+0000","last_clean":"2024-04-22T20:49:18.668649+0000","last_became_active":"2024-04-22T20:19:16.793429+0000","last_became_peered":"2024-04-22T20:19:16.793429+0000","last_unstale":"2024-04-22T20:49:18.668649+0000","last_undegraded":"2024-04-22T20:49:18.668649+0000","last_fullsized":"2024-04-22T20:49:18.668649+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:18.668468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:18.668468+0000","last_clean_scrub_stamp":"2024-04-22T20:49:18.668468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:12:31.861365+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"261'2","reported_seq":57,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.823962+0000","last_change":"2024-04-22T20:49:09.823962+0000","last_active":"2024-04-22T20:49:09.823962+0000","last_peered":"2024-04-22T20:49:09.823962+0000","last_clean":"2024-04-22T20:49:09.823962+0000","last_became_active":"2024-04-22T20:05:17.900874+0000","last_became_peered":"2024-04-22T20:05:17.900874+0000","last_unstale":"2024-04-22T20:49:09.823962+0000","last_undegraded":"2024-04-22T20:49:09.823962+0000","last_fullsized":"2024-04-22T20:49:09.823962+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:09.823786+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:09.823786+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.823786+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:18:49.290238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":8192,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"261'2","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.612637+0000","last_change":"2024-04-22T20:48:26.612637+0000","last_active":"2024-04-22T20:48:26.612637+0000","last_peered":"2024-04-22T20:48:26.612637+0000","last_clean":"2024-04-22T20:48:26.612637+0000","last_became_active":"2024-04-22T20:05:13.813934+0000","last_became_peered":"2024-04-22T20:05:13.813934+0000","last_unstale":"2024-04-22T20:48:26.612637+0000","last_undegraded":"2024-04-22T20:48:26.612637+0000","last_fullsized":"2024-04-22T20:48:26.612637+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:26.612508+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:26.612508+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.612508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:58:57.908355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1024,"num_write":2,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"261'37","reported_seq":156,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.240514+0000","last_change":"2024-04-22T20:48:24.240514+0000","last_active":"2024-04-22T20:48:24.240514+0000","last_peered":"2024-04-22T20:48:24.240514+0000","last_clean":"2024-04-22T20:48:24.240514+0000","last_became_active":"2024-04-22T20:01:04.869660+0000","last_became_peered":"2024-04-22T20:01:04.869660+0000","last_unstale":"2024-04-22T20:48:24.240514+0000","last_undegraded":"2024-04-22T20:48:24.240514+0000","last_fullsized":"2024-04-22T20:48:24.240514+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'37","last_scrub_stamp":"2024-04-22T20:48:24.240383+0000","last_deep_scrub":"261'37","last_deep_scrub_stamp":"2024-04-22T20:48:24.240383+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.240383+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:40:22.419745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":157,"num_read_kb":113,"num_write":62,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:21.775347+0000","last_change":"2024-04-22T20:49:21.775372+0000","last_active":"2024-04-22T20:49:21.775347+0000","last_peered":"2024-04-22T20:49:21.775347+0000","last_clean":"2024-04-22T20:49:21.775347+0000","last_became_active":"2024-04-22T20:19:16.795509+0000","last_became_peered":"2024-04-22T20:19:16.795509+0000","last_unstale":"2024-04-22T20:49:21.775347+0000","last_undegraded":"2024-04-22T20:49:21.775347+0000","last_fullsized":"2024-04-22T20:49:21.775347+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:21.775250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:21.775250+0000","last_clean_scrub_stamp":"2024-04-22T20:49:21.775250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:56:46.381539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":52,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.628465+0000","last_change":"2024-04-22T20:49:00.628465+0000","last_active":"2024-04-22T20:49:00.628465+0000","last_peered":"2024-04-22T20:49:00.628465+0000","last_clean":"2024-04-22T20:49:00.628465+0000","last_became_active":"2024-04-22T20:05:17.891523+0000","last_became_peered":"2024-04-22T20:05:17.891523+0000","last_unstale":"2024-04-22T20:49:00.628465+0000","last_undegraded":"2024-04-22T20:49:00.628465+0000","last_fullsized":"2024-04-22T20:49:00.628465+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:00.628331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:00.628331+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.628331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:34:52.692118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"261'2","reported_seq":61,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.553971+0000","last_change":"2024-04-22T20:48:24.553971+0000","last_active":"2024-04-22T20:48:24.553971+0000","last_peered":"2024-04-22T20:48:24.553971+0000","last_clean":"2024-04-22T20:48:24.553971+0000","last_became_active":"2024-04-22T20:05:13.805704+0000","last_became_peered":"2024-04-22T20:05:13.805704+0000","last_unstale":"2024-04-22T20:48:24.553971+0000","last_undegraded":"2024-04-22T20:48:24.553971+0000","last_fullsized":"2024-04-22T20:48:24.553971+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:24.553871+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:24.553871+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.553871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:02.108209+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"261'65","reported_seq":234,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.443246+0000","last_change":"2024-04-22T20:48:29.443246+0000","last_active":"2024-04-22T20:48:29.443246+0000","last_peered":"2024-04-22T20:48:29.443246+0000","last_clean":"2024-04-22T20:48:29.443246+0000","last_became_active":"2024-04-22T20:01:04.865981+0000","last_became_peered":"2024-04-22T20:01:04.865981+0000","last_unstale":"2024-04-22T20:48:29.443246+0000","last_undegraded":"2024-04-22T20:48:29.443246+0000","last_fullsized":"2024-04-22T20:48:29.443246+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'65","last_scrub_stamp":"2024-04-22T20:48:29.443124+0000","last_deep_scrub":"261'65","last_deep_scrub_stamp":"2024-04-22T20:48:29.443124+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.443124+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:12:31.962691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":313,"num_read_kb":226,"num_write":107,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":35,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:20.556690+0000","last_change":"2024-04-22T20:49:20.556729+0000","last_active":"2024-04-22T20:49:20.556690+0000","last_peered":"2024-04-22T20:49:20.556690+0000","last_clean":"2024-04-22T20:49:20.556690+0000","last_became_active":"2024-04-22T20:19:16.799063+0000","last_became_peered":"2024-04-22T20:19:16.799063+0000","last_unstale":"2024-04-22T20:49:20.556690+0000","last_undegraded":"2024-04-22T20:49:20.556690+0000","last_fullsized":"2024-04-22T20:49:20.556690+0000","mapping_epoch":259,"log_start":"0'0","ondisk_log_start":"0'0","created":259,"last_epoch_clean":260,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:20.556572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:20.556572+0000","last_clean_scrub_stamp":"2024-04-22T20:49:20.556572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:59:47.285896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"261'2","reported_seq":56,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.247215+0000","last_change":"2024-04-22T20:49:11.247260+0000","last_active":"2024-04-22T20:49:11.247215+0000","last_peered":"2024-04-22T20:49:11.247215+0000","last_clean":"2024-04-22T20:49:11.247215+0000","last_became_active":"2024-04-22T20:05:17.901680+0000","last_became_peered":"2024-04-22T20:05:17.901680+0000","last_unstale":"2024-04-22T20:49:11.247215+0000","last_undegraded":"2024-04-22T20:49:11.247215+0000","last_fullsized":"2024-04-22T20:49:11.247215+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":254,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:49:11.247100+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:49:11.247100+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.247100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:16:47.754643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":80,"num_write":2,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"0'0","reported_seq":58,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.618610+0000","last_change":"2024-04-22T20:48:57.618610+0000","last_active":"2024-04-22T20:48:57.618610+0000","last_peered":"2024-04-22T20:48:57.618610+0000","last_clean":"2024-04-22T20:48:57.618610+0000","last_became_active":"2024-04-22T20:05:13.810865+0000","last_became_peered":"2024-04-22T20:05:13.810865+0000","last_unstale":"2024-04-22T20:48:57.618610+0000","last_undegraded":"2024-04-22T20:48:57.618610+0000","last_fullsized":"2024-04-22T20:48:57.618610+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.618439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.618439+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.618439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:51:28.002175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"261'57","reported_seq":190,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.363498+0000","last_change":"2024-04-22T20:48:35.363498+0000","last_active":"2024-04-22T20:48:35.363498+0000","last_peered":"2024-04-22T20:48:35.363498+0000","last_clean":"2024-04-22T20:48:35.363498+0000","last_became_active":"2024-04-22T20:01:04.870253+0000","last_became_peered":"2024-04-22T20:01:04.870253+0000","last_unstale":"2024-04-22T20:48:35.363498+0000","last_undegraded":"2024-04-22T20:48:35.363498+0000","last_fullsized":"2024-04-22T20:48:35.363498+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'57","last_scrub_stamp":"2024-04-22T20:48:35.363335+0000","last_deep_scrub":"261'57","last_deep_scrub_stamp":"2024-04-22T20:48:35.363335+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.363335+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:44:05.650136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":208,"num_read_kb":145,"num_write":102,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"261'344","reported_seq":961,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.428623+0000","last_change":"2024-04-22T20:48:30.428623+0000","last_active":"2024-04-22T20:48:30.428623+0000","last_peered":"2024-04-22T20:48:30.428623+0000","last_clean":"2024-04-22T20:48:30.428623+0000","last_became_active":"2024-04-22T19:53:04.823351+0000","last_became_peered":"2024-04-22T19:53:04.823351+0000","last_unstale":"2024-04-22T20:48:30.428623+0000","last_undegraded":"2024-04-22T20:48:30.428623+0000","last_fullsized":"2024-04-22T20:48:30.428623+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'344","last_scrub_stamp":"2024-04-22T20:48:30.428472+0000","last_deep_scrub":"261'344","last_deep_scrub_stamp":"2024-04-22T20:48:30.428472+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.428472+0000","objects_scrubbed":5,"log_size":44,"log_dups_size":300,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:45:58.154599+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":510,"num_read_kb":495,"num_write":1298,"num_write_kb":39497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"261'169","reported_seq":773,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.947499+0000","last_change":"2024-04-22T20:48:50.947499+0000","last_active":"2024-04-22T20:48:50.947499+0000","last_peered":"2024-04-22T20:48:50.947499+0000","last_clean":"2024-04-22T20:48:50.947499+0000","last_became_active":"2024-04-22T19:51:28.511595+0000","last_became_peered":"2024-04-22T19:51:28.511595+0000","last_unstale":"2024-04-22T20:48:50.947499+0000","last_undegraded":"2024-04-22T20:48:50.947499+0000","last_fullsized":"2024-04-22T20:48:50.947499+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'169","last_scrub_stamp":"2024-04-22T20:48:50.947377+0000","last_deep_scrub":"261'169","last_deep_scrub_stamp":"2024-04-22T20:48:50.947377+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.947377+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:25:29.288345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":537,"num_write":262,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"261'316","reported_seq":976,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.575341+0000","last_change":"2024-04-22T20:48:51.575445+0000","last_active":"2024-04-22T20:48:51.575341+0000","last_peered":"2024-04-22T20:48:51.575341+0000","last_clean":"2024-04-22T20:48:51.575341+0000","last_became_active":"2024-04-22T19:51:26.496188+0000","last_became_peered":"2024-04-22T19:51:26.496188+0000","last_unstale":"2024-04-22T20:48:51.575341+0000","last_undegraded":"2024-04-22T20:48:51.575341+0000","last_fullsized":"2024-04-22T20:48:51.575341+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'316","last_scrub_stamp":"2024-04-22T20:48:51.575227+0000","last_deep_scrub":"261'316","last_deep_scrub_stamp":"2024-04-22T20:48:51.575227+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.575227+0000","objects_scrubbed":11,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:05:45.518799+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5719773,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":497,"num_read_kb":9758,"num_write":1112,"num_write_kb":30920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6],"acting":[1,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"261'315","reported_seq":897,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.386962+0000","last_change":"2024-04-22T20:48:53.386962+0000","last_active":"2024-04-22T20:48:53.386962+0000","last_peered":"2024-04-22T20:48:53.386962+0000","last_clean":"2024-04-22T20:48:53.386962+0000","last_became_active":"2024-04-22T19:51:28.519111+0000","last_became_peered":"2024-04-22T19:51:28.519111+0000","last_unstale":"2024-04-22T20:48:53.386962+0000","last_undegraded":"2024-04-22T20:48:53.386962+0000","last_fullsized":"2024-04-22T20:48:53.386962+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'315","last_scrub_stamp":"2024-04-22T20:48:53.386841+0000","last_deep_scrub":"261'315","last_deep_scrub_stamp":"2024-04-22T20:48:53.386841+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.386841+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:13.253246+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1190,"num_read_kb":913,"num_write":510,"num_write_kb":265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"261'320","reported_seq":950,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.733306+0000","last_change":"2024-04-22T20:49:10.733421+0000","last_active":"2024-04-22T20:49:10.733306+0000","last_peered":"2024-04-22T20:49:10.733306+0000","last_clean":"2024-04-22T20:49:10.733306+0000","last_became_active":"2024-04-22T19:51:26.502744+0000","last_became_peered":"2024-04-22T19:51:26.502744+0000","last_unstale":"2024-04-22T20:49:10.733306+0000","last_undegraded":"2024-04-22T20:49:10.733306+0000","last_fullsized":"2024-04-22T20:49:10.733306+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'320","last_scrub_stamp":"2024-04-22T20:49:10.733127+0000","last_deep_scrub":"261'320","last_deep_scrub_stamp":"2024-04-22T20:49:10.733127+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.733127+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:43.010904+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437211,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":443,"num_read_kb":7116,"num_write":1035,"num_write_kb":21496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"261'151","reported_seq":737,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.271385+0000","last_change":"2024-04-22T20:48:48.271385+0000","last_active":"2024-04-22T20:48:48.271385+0000","last_peered":"2024-04-22T20:48:48.271385+0000","last_clean":"2024-04-22T20:48:48.271385+0000","last_became_active":"2024-04-22T19:51:28.518650+0000","last_became_peered":"2024-04-22T19:51:28.518650+0000","last_unstale":"2024-04-22T20:48:48.271385+0000","last_undegraded":"2024-04-22T20:48:48.271385+0000","last_fullsized":"2024-04-22T20:48:48.271385+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'151","last_scrub_stamp":"2024-04-22T20:48:48.271264+0000","last_deep_scrub":"261'151","last_deep_scrub_stamp":"2024-04-22T20:48:48.271264+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.271264+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:14.794941+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":572,"num_read_kb":453,"num_write":224,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"261'324","reported_seq":949,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.858233+0000","last_change":"2024-04-22T20:49:01.858233+0000","last_active":"2024-04-22T20:49:01.858233+0000","last_peered":"2024-04-22T20:49:01.858233+0000","last_clean":"2024-04-22T20:49:01.858233+0000","last_became_active":"2024-04-22T19:51:26.504450+0000","last_became_peered":"2024-04-22T19:51:26.504450+0000","last_unstale":"2024-04-22T20:49:01.858233+0000","last_undegraded":"2024-04-22T20:49:01.858233+0000","last_fullsized":"2024-04-22T20:49:01.858233+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:49:01.858049+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:49:01.858049+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.858049+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:50:13.565120+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8563195,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":424,"num_read_kb":25508,"num_write":1190,"num_write_kb":37655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"261'81","reported_seq":686,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.546404+0000","last_change":"2024-04-22T20:48:32.546404+0000","last_active":"2024-04-22T20:48:32.546404+0000","last_peered":"2024-04-22T20:48:32.546404+0000","last_clean":"2024-04-22T20:48:32.546404+0000","last_became_active":"2024-04-22T19:52:04.619688+0000","last_became_peered":"2024-04-22T19:52:04.619688+0000","last_unstale":"2024-04-22T20:48:32.546404+0000","last_undegraded":"2024-04-22T20:48:32.546404+0000","last_fullsized":"2024-04-22T20:48:32.546404+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'81","last_scrub_stamp":"2024-04-22T20:48:32.546276+0000","last_deep_scrub":"261'81","last_deep_scrub_stamp":"2024-04-22T20:48:32.546276+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.546276+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:51:45.224120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":353,"num_read_kb":299,"num_write":120,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"261'281","reported_seq":979,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.591663+0000","last_change":"2024-04-22T20:49:03.591744+0000","last_active":"2024-04-22T20:49:03.591663+0000","last_peered":"2024-04-22T20:49:03.591663+0000","last_clean":"2024-04-22T20:49:03.591663+0000","last_became_active":"2024-04-22T19:51:26.507185+0000","last_became_peered":"2024-04-22T19:51:26.507185+0000","last_unstale":"2024-04-22T20:49:03.591663+0000","last_undegraded":"2024-04-22T20:49:03.591663+0000","last_fullsized":"2024-04-22T20:49:03.591663+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'281","last_scrub_stamp":"2024-04-22T20:49:03.591510+0000","last_deep_scrub":"261'281","last_deep_scrub_stamp":"2024-04-22T20:49:03.591510+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.591510+0000","objects_scrubbed":13,"log_size":81,"log_dups_size":200,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:35.348173+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3508486,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":442,"num_read_kb":10200,"num_write":925,"num_write_kb":30630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"261'40","reported_seq":604,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.557842+0000","last_change":"2024-04-22T20:48:45.557842+0000","last_active":"2024-04-22T20:48:45.557842+0000","last_peered":"2024-04-22T20:48:45.557842+0000","last_clean":"2024-04-22T20:48:45.557842+0000","last_became_active":"2024-04-22T19:51:28.513693+0000","last_became_peered":"2024-04-22T19:51:28.513693+0000","last_unstale":"2024-04-22T20:48:45.557842+0000","last_undegraded":"2024-04-22T20:48:45.557842+0000","last_fullsized":"2024-04-22T20:48:45.557842+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'40","last_scrub_stamp":"2024-04-22T20:48:45.557708+0000","last_deep_scrub":"261'40","last_deep_scrub_stamp":"2024-04-22T20:48:45.557708+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.557708+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:39:08.485749+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":170,"num_read_kb":146,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"261'444","reported_seq":1196,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.909652+0000","last_change":"2024-04-22T20:48:54.909652+0000","last_active":"2024-04-22T20:48:54.909652+0000","last_peered":"2024-04-22T20:48:54.909652+0000","last_clean":"2024-04-22T20:48:54.909652+0000","last_became_active":"2024-04-22T19:51:26.508716+0000","last_became_peered":"2024-04-22T19:51:26.508716+0000","last_unstale":"2024-04-22T20:48:54.909652+0000","last_undegraded":"2024-04-22T20:48:54.909652+0000","last_fullsized":"2024-04-22T20:48:54.909652+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'444","last_scrub_stamp":"2024-04-22T20:48:54.909489+0000","last_deep_scrub":"261'444","last_deep_scrub_stamp":"2024-04-22T20:48:54.909489+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.909489+0000","objects_scrubbed":4,"log_size":44,"log_dups_size":400,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:47:48.964740+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661200,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1035,"num_read_kb":17244,"num_write":1523,"num_write_kb":25883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"261'145","reported_seq":752,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.279346+0000","last_change":"2024-04-22T20:48:50.279372+0000","last_active":"2024-04-22T20:48:50.279346+0000","last_peered":"2024-04-22T20:48:50.279346+0000","last_clean":"2024-04-22T20:48:50.279346+0000","last_became_active":"2024-04-22T19:51:28.518501+0000","last_became_peered":"2024-04-22T19:51:28.518501+0000","last_unstale":"2024-04-22T20:48:50.279346+0000","last_undegraded":"2024-04-22T20:48:50.279346+0000","last_fullsized":"2024-04-22T20:48:50.279346+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'145","last_scrub_stamp":"2024-04-22T20:48:50.279249+0000","last_deep_scrub":"261'145","last_deep_scrub_stamp":"2024-04-22T20:48:50.279249+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.279249+0000","objects_scrubbed":1,"log_size":45,"log_dups_size":100,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:46:54.700622+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":597,"num_read_kb":478,"num_write":226,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"261'377","reported_seq":1048,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.546712+0000","last_change":"2024-04-22T20:49:12.546764+0000","last_active":"2024-04-22T20:49:12.546712+0000","last_peered":"2024-04-22T20:49:12.546712+0000","last_clean":"2024-04-22T20:49:12.546712+0000","last_became_active":"2024-04-22T19:51:26.507141+0000","last_became_peered":"2024-04-22T19:51:26.507141+0000","last_unstale":"2024-04-22T20:49:12.546712+0000","last_undegraded":"2024-04-22T20:49:12.546712+0000","last_fullsized":"2024-04-22T20:49:12.546712+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'377","last_scrub_stamp":"2024-04-22T20:49:12.546615+0000","last_deep_scrub":"261'377","last_deep_scrub_stamp":"2024-04-22T20:49:12.546615+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.546615+0000","objects_scrubbed":9,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:20:29.531515+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2367620,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":550,"num_read_kb":43743,"num_write":1389,"num_write_kb":32515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"261'152","reported_seq":746,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.990255+0000","last_change":"2024-04-22T20:48:43.990284+0000","last_active":"2024-04-22T20:48:43.990255+0000","last_peered":"2024-04-22T20:48:43.990255+0000","last_clean":"2024-04-22T20:48:43.990255+0000","last_became_active":"2024-04-22T19:51:28.516012+0000","last_became_peered":"2024-04-22T19:51:28.516012+0000","last_unstale":"2024-04-22T20:48:43.990255+0000","last_undegraded":"2024-04-22T20:48:43.990255+0000","last_fullsized":"2024-04-22T20:48:43.990255+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:43.990165+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:43.990165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.990165+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:52:23.460100+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":596,"num_read_kb":470,"num_write":219,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7560,"num_omap_keys":32,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"261'333","reported_seq":987,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:17.330968+0000","last_change":"2024-04-22T20:49:17.331060+0000","last_active":"2024-04-22T20:49:17.330968+0000","last_peered":"2024-04-22T20:49:17.330968+0000","last_clean":"2024-04-22T20:49:17.330968+0000","last_became_active":"2024-04-22T19:51:26.502928+0000","last_became_peered":"2024-04-22T19:51:26.502928+0000","last_unstale":"2024-04-22T20:49:17.330968+0000","last_undegraded":"2024-04-22T20:49:17.330968+0000","last_fullsized":"2024-04-22T20:49:17.330968+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'333","last_scrub_stamp":"2024-04-22T20:49:17.330844+0000","last_deep_scrub":"261'333","last_deep_scrub_stamp":"2024-04-22T20:49:17.330844+0000","last_clean_scrub_stamp":"2024-04-22T20:49:17.330844+0000","objects_scrubbed":8,"log_size":33,"log_dups_size":300,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:54.551981+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5414404,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":443,"num_read_kb":51849,"num_write":1202,"num_write_kb":39313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"261'60","reported_seq":638,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.590400+0000","last_change":"2024-04-22T20:48:40.590452+0000","last_active":"2024-04-22T20:48:40.590400+0000","last_peered":"2024-04-22T20:48:40.590400+0000","last_clean":"2024-04-22T20:48:40.590400+0000","last_became_active":"2024-04-22T19:51:28.509774+0000","last_became_peered":"2024-04-22T19:51:28.509774+0000","last_unstale":"2024-04-22T20:48:40.590400+0000","last_undegraded":"2024-04-22T20:48:40.590400+0000","last_fullsized":"2024-04-22T20:48:40.590400+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'60","last_scrub_stamp":"2024-04-22T20:48:40.590291+0000","last_deep_scrub":"261'60","last_deep_scrub_stamp":"2024-04-22T20:48:40.590291+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.590291+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:30:33.030111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":250,"num_read_kb":211,"num_write":81,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"261'126","reported_seq":744,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.246253+0000","last_change":"2024-04-22T20:49:03.246307+0000","last_active":"2024-04-22T20:49:03.246253+0000","last_peered":"2024-04-22T20:49:03.246253+0000","last_clean":"2024-04-22T20:49:03.246253+0000","last_became_active":"2024-04-22T19:51:28.505562+0000","last_became_peered":"2024-04-22T19:51:28.505562+0000","last_unstale":"2024-04-22T20:49:03.246253+0000","last_undegraded":"2024-04-22T20:49:03.246253+0000","last_fullsized":"2024-04-22T20:49:03.246253+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'126","last_scrub_stamp":"2024-04-22T20:49:03.246160+0000","last_deep_scrub":"261'126","last_deep_scrub_stamp":"2024-04-22T20:49:03.246160+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.246160+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:08.512189+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":540,"num_read_kb":443,"num_write":193,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"261'307","reported_seq":949,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.575247+0000","last_change":"2024-04-22T20:49:05.575247+0000","last_active":"2024-04-22T20:49:05.575247+0000","last_peered":"2024-04-22T20:49:05.575247+0000","last_clean":"2024-04-22T20:49:05.575247+0000","last_became_active":"2024-04-22T19:51:26.504717+0000","last_became_peered":"2024-04-22T19:51:26.504717+0000","last_unstale":"2024-04-22T20:49:05.575247+0000","last_undegraded":"2024-04-22T20:49:05.575247+0000","last_fullsized":"2024-04-22T20:49:05.575247+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'307","last_scrub_stamp":"2024-04-22T20:49:05.575076+0000","last_deep_scrub":"261'307","last_deep_scrub_stamp":"2024-04-22T20:49:05.575076+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.575076+0000","objects_scrubbed":4,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:55:01.561551+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10266,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":474,"num_read_kb":28705,"num_write":1110,"num_write_kb":24236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0],"acting":[7,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"261'105","reported_seq":707,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.395682+0000","last_change":"2024-04-22T20:48:51.395709+0000","last_active":"2024-04-22T20:48:51.395682+0000","last_peered":"2024-04-22T20:48:51.395682+0000","last_clean":"2024-04-22T20:48:51.395682+0000","last_became_active":"2024-04-22T19:51:28.519230+0000","last_became_peered":"2024-04-22T19:51:28.519230+0000","last_unstale":"2024-04-22T20:48:51.395682+0000","last_undegraded":"2024-04-22T20:48:51.395682+0000","last_fullsized":"2024-04-22T20:48:51.395682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'105","last_scrub_stamp":"2024-04-22T20:48:51.395575+0000","last_deep_scrub":"261'105","last_deep_scrub_stamp":"2024-04-22T20:48:51.395575+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.395575+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:48:07.398102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":465,"num_read_kb":383,"num_write":180,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"261'381","reported_seq":1048,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.558671+0000","last_change":"2024-04-22T20:48:50.558671+0000","last_active":"2024-04-22T20:48:50.558671+0000","last_peered":"2024-04-22T20:48:50.558671+0000","last_clean":"2024-04-22T20:48:50.558671+0000","last_became_active":"2024-04-22T19:51:26.496156+0000","last_became_peered":"2024-04-22T19:51:26.496156+0000","last_unstale":"2024-04-22T20:48:50.558671+0000","last_undegraded":"2024-04-22T20:48:50.558671+0000","last_fullsized":"2024-04-22T20:48:50.558671+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'381","last_scrub_stamp":"2024-04-22T20:48:50.558509+0000","last_deep_scrub":"261'381","last_deep_scrub_stamp":"2024-04-22T20:48:50.558509+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.558509+0000","objects_scrubbed":6,"log_size":81,"log_dups_size":300,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:54:45.615667+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":347,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":611,"num_read_kb":792,"num_write":1516,"num_write_kb":25865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"261'172","reported_seq":773,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.926538+0000","last_change":"2024-04-22T20:48:41.926573+0000","last_active":"2024-04-22T20:48:41.926538+0000","last_peered":"2024-04-22T20:48:41.926538+0000","last_clean":"2024-04-22T20:48:41.926538+0000","last_became_active":"2024-04-22T19:51:28.515673+0000","last_became_peered":"2024-04-22T19:51:28.515673+0000","last_unstale":"2024-04-22T20:48:41.926538+0000","last_undegraded":"2024-04-22T20:48:41.926538+0000","last_fullsized":"2024-04-22T20:48:41.926538+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'172","last_scrub_stamp":"2024-04-22T20:48:41.926446+0000","last_deep_scrub":"261'172","last_deep_scrub_stamp":"2024-04-22T20:48:41.926446+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.926446+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:29:59.234763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":666,"num_read_kb":529,"num_write":258,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"261'360","reported_seq":1080,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.586516+0000","last_change":"2024-04-22T20:48:54.586516+0000","last_active":"2024-04-22T20:48:54.586516+0000","last_peered":"2024-04-22T20:48:54.586516+0000","last_clean":"2024-04-22T20:48:54.586516+0000","last_became_active":"2024-04-22T19:51:26.502156+0000","last_became_peered":"2024-04-22T19:51:26.502156+0000","last_unstale":"2024-04-22T20:48:54.586516+0000","last_undegraded":"2024-04-22T20:48:54.586516+0000","last_fullsized":"2024-04-22T20:48:54.586516+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'360","last_scrub_stamp":"2024-04-22T20:48:54.586311+0000","last_deep_scrub":"261'360","last_deep_scrub_stamp":"2024-04-22T20:48:54.586311+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.586311+0000","objects_scrubbed":5,"log_size":60,"log_dups_size":300,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:14:01.200779+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1373470,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":619,"num_read_kb":68133,"num_write":1312,"num_write_kb":41095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"261'290","reported_seq":906,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.550305+0000","last_change":"2024-04-22T20:48:41.550305+0000","last_active":"2024-04-22T20:48:41.550305+0000","last_peered":"2024-04-22T20:48:41.550305+0000","last_clean":"2024-04-22T20:48:41.550305+0000","last_became_active":"2024-04-22T19:51:28.510415+0000","last_became_peered":"2024-04-22T19:51:28.510415+0000","last_unstale":"2024-04-22T20:48:41.550305+0000","last_undegraded":"2024-04-22T20:48:41.550305+0000","last_fullsized":"2024-04-22T20:48:41.550305+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'290","last_scrub_stamp":"2024-04-22T20:48:41.550169+0000","last_deep_scrub":"261'290","last_deep_scrub_stamp":"2024-04-22T20:48:41.550169+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.550169+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:30:48.444121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1119,"num_read_kb":869,"num_write":475,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"261'320","reported_seq":975,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.247714+0000","last_change":"2024-04-22T20:49:02.247714+0000","last_active":"2024-04-22T20:49:02.247714+0000","last_peered":"2024-04-22T20:49:02.247714+0000","last_clean":"2024-04-22T20:49:02.247714+0000","last_became_active":"2024-04-22T19:51:26.506671+0000","last_became_peered":"2024-04-22T19:51:26.506671+0000","last_unstale":"2024-04-22T20:49:02.247714+0000","last_undegraded":"2024-04-22T20:49:02.247714+0000","last_fullsized":"2024-04-22T20:49:02.247714+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'320","last_scrub_stamp":"2024-04-22T20:49:02.247508+0000","last_deep_scrub":"261'320","last_deep_scrub_stamp":"2024-04-22T20:49:02.247508+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.247508+0000","objects_scrubbed":6,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:52:15.451052+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194361,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":6535,"num_write":1103,"num_write_kb":36003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"261'157","reported_seq":787,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.601043+0000","last_change":"2024-04-22T20:48:42.601043+0000","last_active":"2024-04-22T20:48:42.601043+0000","last_peered":"2024-04-22T20:48:42.601043+0000","last_clean":"2024-04-22T20:48:42.601043+0000","last_became_active":"2024-04-22T19:51:28.511889+0000","last_became_peered":"2024-04-22T19:51:28.511889+0000","last_unstale":"2024-04-22T20:48:42.601043+0000","last_undegraded":"2024-04-22T20:48:42.601043+0000","last_fullsized":"2024-04-22T20:48:42.601043+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'157","last_scrub_stamp":"2024-04-22T20:48:42.600881+0000","last_deep_scrub":"261'157","last_deep_scrub_stamp":"2024-04-22T20:48:42.600881+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.600881+0000","objects_scrubbed":2,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:11:30.539100+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":679,"num_read_kb":547,"num_write":240,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3316,"num_omap_keys":19,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"261'334","reported_seq":991,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.339976+0000","last_change":"2024-04-22T20:49:12.339976+0000","last_active":"2024-04-22T20:49:12.339976+0000","last_peered":"2024-04-22T20:49:12.339976+0000","last_clean":"2024-04-22T20:49:12.339976+0000","last_became_active":"2024-04-22T19:51:26.497718+0000","last_became_peered":"2024-04-22T19:51:26.497718+0000","last_unstale":"2024-04-22T20:49:12.339976+0000","last_undegraded":"2024-04-22T20:49:12.339976+0000","last_fullsized":"2024-04-22T20:49:12.339976+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'334","last_scrub_stamp":"2024-04-22T20:49:12.339764+0000","last_deep_scrub":"261'334","last_deep_scrub_stamp":"2024-04-22T20:49:12.339764+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.339764+0000","objects_scrubbed":10,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:51.305007+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6050340,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":471,"num_read_kb":30112,"num_write":1163,"num_write_kb":27210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1],"acting":[3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"261'94","reported_seq":710,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.252334+0000","last_change":"2024-04-22T20:49:04.252334+0000","last_active":"2024-04-22T20:49:04.252334+0000","last_peered":"2024-04-22T20:49:04.252334+0000","last_clean":"2024-04-22T20:49:04.252334+0000","last_became_active":"2024-04-22T19:51:28.507673+0000","last_became_peered":"2024-04-22T19:51:28.507673+0000","last_unstale":"2024-04-22T20:49:04.252334+0000","last_undegraded":"2024-04-22T20:49:04.252334+0000","last_fullsized":"2024-04-22T20:49:04.252334+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'94","last_scrub_stamp":"2024-04-22T20:49:04.252218+0000","last_deep_scrub":"261'94","last_deep_scrub_stamp":"2024-04-22T20:49:04.252218+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.252218+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:55:27.370264+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":404,"num_read_kb":350,"num_write":133,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"261'413","reported_seq":1133,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:13.703814+0000","last_change":"2024-04-22T20:49:13.703918+0000","last_active":"2024-04-22T20:49:13.703814+0000","last_peered":"2024-04-22T20:49:13.703814+0000","last_clean":"2024-04-22T20:49:13.703814+0000","last_became_active":"2024-04-22T19:51:26.511213+0000","last_became_peered":"2024-04-22T19:51:26.511213+0000","last_unstale":"2024-04-22T20:49:13.703814+0000","last_undegraded":"2024-04-22T20:49:13.703814+0000","last_fullsized":"2024-04-22T20:49:13.703814+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'413","last_scrub_stamp":"2024-04-22T20:49:13.703681+0000","last_deep_scrub":"261'413","last_deep_scrub_stamp":"2024-04-22T20:49:13.703681+0000","last_clean_scrub_stamp":"2024-04-22T20:49:13.703681+0000","objects_scrubbed":8,"log_size":13,"log_dups_size":400,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:02:28.872480+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306877,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":730,"num_read_kb":36415,"num_write":1798,"num_write_kb":40524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"261'129","reported_seq":660,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.351716+0000","last_change":"2024-04-22T20:48:29.351716+0000","last_active":"2024-04-22T20:48:29.351716+0000","last_peered":"2024-04-22T20:48:29.351716+0000","last_clean":"2024-04-22T20:48:29.351716+0000","last_became_active":"2024-04-22T19:51:36.567561+0000","last_became_peered":"2024-04-22T19:51:36.567561+0000","last_unstale":"2024-04-22T20:48:29.351716+0000","last_undegraded":"2024-04-22T20:48:29.351716+0000","last_fullsized":"2024-04-22T20:48:29.351716+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'129","last_scrub_stamp":"2024-04-22T20:48:29.351556+0000","last_deep_scrub":"261'129","last_deep_scrub_stamp":"2024-04-22T20:48:29.351556+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.351556+0000","objects_scrubbed":4,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:20:31.079680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":192,"num_read_kb":168,"num_write":307,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"261'38","reported_seq":593,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.462844+0000","last_change":"2024-04-22T20:48:58.462869+0000","last_active":"2024-04-22T20:48:58.462844+0000","last_peered":"2024-04-22T20:48:58.462844+0000","last_clean":"2024-04-22T20:48:58.462844+0000","last_became_active":"2024-04-22T19:51:28.519994+0000","last_became_peered":"2024-04-22T19:51:28.519994+0000","last_unstale":"2024-04-22T20:48:58.462844+0000","last_undegraded":"2024-04-22T20:48:58.462844+0000","last_fullsized":"2024-04-22T20:48:58.462844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'38","last_scrub_stamp":"2024-04-22T20:48:58.462753+0000","last_deep_scrub":"261'38","last_deep_scrub_stamp":"2024-04-22T20:48:58.462753+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.462753+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:57:15.046577+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":138,"num_write":52,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"261'317","reported_seq":1059,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.402857+0000","last_change":"2024-04-22T20:48:38.402920+0000","last_active":"2024-04-22T20:48:38.402857+0000","last_peered":"2024-04-22T20:48:38.402857+0000","last_clean":"2024-04-22T20:48:38.402857+0000","last_became_active":"2024-04-22T19:53:05.051272+0000","last_became_peered":"2024-04-22T19:53:05.051272+0000","last_unstale":"2024-04-22T20:48:38.402857+0000","last_undegraded":"2024-04-22T20:48:38.402857+0000","last_fullsized":"2024-04-22T20:48:38.402857+0000","mapping_epoch":35,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:38.402757+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:38.402757+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.402757+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:31:08.678074+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263217,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":662,"num_read_kb":11069,"num_write":1270,"num_write_kb":33730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":501,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.985652+0000","last_change":"2024-04-22T20:48:45.985652+0000","last_active":"2024-04-22T20:48:45.985652+0000","last_peered":"2024-04-22T20:48:45.985652+0000","last_clean":"2024-04-22T20:48:45.985652+0000","last_became_active":"2024-04-22T19:51:30.520848+0000","last_became_peered":"2024-04-22T19:51:30.520848+0000","last_unstale":"2024-04-22T20:48:45.985652+0000","last_undegraded":"2024-04-22T20:48:45.985652+0000","last_fullsized":"2024-04-22T20:48:45.985652+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.985531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.985531+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.985531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:16:52.817893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"261'334","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.658905+0000","last_change":"2024-04-22T20:49:01.658905+0000","last_active":"2024-04-22T20:49:01.658905+0000","last_peered":"2024-04-22T20:49:01.658905+0000","last_clean":"2024-04-22T20:49:01.658905+0000","last_became_active":"2024-04-22T19:51:32.537695+0000","last_became_peered":"2024-04-22T19:51:32.537695+0000","last_unstale":"2024-04-22T20:49:01.658905+0000","last_undegraded":"2024-04-22T20:49:01.658905+0000","last_fullsized":"2024-04-22T20:49:01.658905+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'334","last_scrub_stamp":"2024-04-22T20:49:01.658785+0000","last_deep_scrub":"261'334","last_deep_scrub_stamp":"2024-04-22T20:49:01.658785+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.658785+0000","objects_scrubbed":5,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:50:34.689896+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39872,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":550,"num_read_kb":1178,"num_write":438,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.404533+0000","last_change":"2024-04-22T20:48:45.404533+0000","last_active":"2024-04-22T20:48:45.404533+0000","last_peered":"2024-04-22T20:48:45.404533+0000","last_clean":"2024-04-22T20:48:45.404533+0000","last_became_active":"2024-04-22T19:51:34.557285+0000","last_became_peered":"2024-04-22T19:51:34.557285+0000","last_unstale":"2024-04-22T20:48:45.404533+0000","last_undegraded":"2024-04-22T20:48:45.404533+0000","last_fullsized":"2024-04-22T20:48:45.404533+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.404404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.404404+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.404404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:14:28.347705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"261'2405","reported_seq":4273,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.005951+0000","last_change":"2024-04-22T20:48:34.006025+0000","last_active":"2024-04-22T20:48:34.005951+0000","last_peered":"2024-04-22T20:48:34.005951+0000","last_clean":"2024-04-22T20:48:34.005951+0000","last_became_active":"2024-04-22T19:51:36.567195+0000","last_became_peered":"2024-04-22T19:51:36.567195+0000","last_unstale":"2024-04-22T20:48:34.005951+0000","last_undegraded":"2024-04-22T20:48:34.005951+0000","last_fullsized":"2024-04-22T20:48:34.005951+0000","mapping_epoch":30,"log_start":"261'2300","ondisk_log_start":"261'2300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2405","last_scrub_stamp":"2024-04-22T20:48:34.005850+0000","last_deep_scrub":"261'2405","last_deep_scrub_stamp":"2024-04-22T20:48:34.005850+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.005850+0000","objects_scrubbed":6,"log_size":105,"log_dups_size":2300,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:06.221017+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1044,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5986,"num_read_kb":5990,"num_write":4781,"num_write_kb":1764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"261'131","reported_seq":731,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.955649+0000","last_change":"2024-04-22T20:48:48.955649+0000","last_active":"2024-04-22T20:48:48.955649+0000","last_peered":"2024-04-22T20:48:48.955649+0000","last_clean":"2024-04-22T20:48:48.955649+0000","last_became_active":"2024-04-22T19:51:28.512786+0000","last_became_peered":"2024-04-22T19:51:28.512786+0000","last_unstale":"2024-04-22T20:48:48.955649+0000","last_undegraded":"2024-04-22T20:48:48.955649+0000","last_fullsized":"2024-04-22T20:48:48.955649+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:48.955527+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:48.955527+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.955527+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:47:43.304224+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":504,"num_read_kb":408,"num_write":191,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"261'290","reported_seq":902,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.689230+0000","last_change":"2024-04-22T20:49:08.689283+0000","last_active":"2024-04-22T20:49:08.689230+0000","last_peered":"2024-04-22T20:49:08.689230+0000","last_clean":"2024-04-22T20:49:08.689230+0000","last_became_active":"2024-04-22T19:51:26.499885+0000","last_became_peered":"2024-04-22T19:51:26.499885+0000","last_unstale":"2024-04-22T20:49:08.689230+0000","last_undegraded":"2024-04-22T20:49:08.689230+0000","last_fullsized":"2024-04-22T20:49:08.689230+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'290","last_scrub_stamp":"2024-04-22T20:49:08.689129+0000","last_deep_scrub":"261'290","last_deep_scrub_stamp":"2024-04-22T20:49:08.689129+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.689129+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:09.853456+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050193,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":16305,"num_write":962,"num_write_kb":23193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.330826+0000","last_change":"2024-04-22T20:49:06.330846+0000","last_active":"2024-04-22T20:49:06.330826+0000","last_peered":"2024-04-22T20:49:06.330826+0000","last_clean":"2024-04-22T20:49:06.330826+0000","last_became_active":"2024-04-22T19:51:30.526622+0000","last_became_peered":"2024-04-22T19:51:30.526622+0000","last_unstale":"2024-04-22T20:49:06.330826+0000","last_undegraded":"2024-04-22T20:49:06.330826+0000","last_fullsized":"2024-04-22T20:49:06.330826+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:06.330744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:06.330744+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.330744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:58:15.948210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"261'89","reported_seq":624,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.323782+0000","last_change":"2024-04-22T20:48:39.323782+0000","last_active":"2024-04-22T20:48:39.323782+0000","last_peered":"2024-04-22T20:48:39.323782+0000","last_clean":"2024-04-22T20:48:39.323782+0000","last_became_active":"2024-04-22T19:51:32.536180+0000","last_became_peered":"2024-04-22T19:51:32.536180+0000","last_unstale":"2024-04-22T20:48:39.323782+0000","last_undegraded":"2024-04-22T20:48:39.323782+0000","last_fullsized":"2024-04-22T20:48:39.323782+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'89","last_scrub_stamp":"2024-04-22T20:48:39.323672+0000","last_deep_scrub":"261'89","last_deep_scrub_stamp":"2024-04-22T20:48:39.323672+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.323672+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:12:25.808631+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":95,"num_write":64,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.367530+0000","last_change":"2024-04-22T20:48:34.367530+0000","last_active":"2024-04-22T20:48:34.367530+0000","last_peered":"2024-04-22T20:48:34.367530+0000","last_clean":"2024-04-22T20:48:34.367530+0000","last_became_active":"2024-04-22T19:51:34.549551+0000","last_became_peered":"2024-04-22T19:51:34.549551+0000","last_unstale":"2024-04-22T20:48:34.367530+0000","last_undegraded":"2024-04-22T20:48:34.367530+0000","last_fullsized":"2024-04-22T20:48:34.367530+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.367401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.367401+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.367401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:49:41.545716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"261'151","reported_seq":660,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.539676+0000","last_change":"2024-04-22T20:48:38.539676+0000","last_active":"2024-04-22T20:48:38.539676+0000","last_peered":"2024-04-22T20:48:38.539676+0000","last_clean":"2024-04-22T20:48:38.539676+0000","last_became_active":"2024-04-22T19:51:36.573189+0000","last_became_peered":"2024-04-22T19:51:36.573189+0000","last_unstale":"2024-04-22T20:48:38.539676+0000","last_undegraded":"2024-04-22T20:48:38.539676+0000","last_fullsized":"2024-04-22T20:48:38.539676+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'151","last_scrub_stamp":"2024-04-22T20:48:38.539539+0000","last_deep_scrub":"261'151","last_deep_scrub_stamp":"2024-04-22T20:48:38.539539+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.539539+0000","objects_scrubbed":5,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:40:24.145792+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":674,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":134,"num_read_kb":132,"num_write":328,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"261'212","reported_seq":847,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.682785+0000","last_change":"2024-04-22T20:48:58.682785+0000","last_active":"2024-04-22T20:48:58.682785+0000","last_peered":"2024-04-22T20:48:58.682785+0000","last_clean":"2024-04-22T20:48:58.682785+0000","last_became_active":"2024-04-22T19:51:28.517737+0000","last_became_peered":"2024-04-22T19:51:28.517737+0000","last_unstale":"2024-04-22T20:48:58.682785+0000","last_undegraded":"2024-04-22T20:48:58.682785+0000","last_fullsized":"2024-04-22T20:48:58.682785+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'212","last_scrub_stamp":"2024-04-22T20:48:58.682670+0000","last_deep_scrub":"261'212","last_deep_scrub_stamp":"2024-04-22T20:48:58.682670+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.682670+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":200,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:37:06.432832+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":871,"num_read_kb":733,"num_write":338,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"261'298","reported_seq":951,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.509069+0000","last_change":"2024-04-22T20:49:01.509118+0000","last_active":"2024-04-22T20:49:01.509069+0000","last_peered":"2024-04-22T20:49:01.509069+0000","last_clean":"2024-04-22T20:49:01.509069+0000","last_became_active":"2024-04-22T19:51:26.508933+0000","last_became_peered":"2024-04-22T19:51:26.508933+0000","last_unstale":"2024-04-22T20:49:01.509069+0000","last_undegraded":"2024-04-22T20:49:01.509069+0000","last_fullsized":"2024-04-22T20:49:01.509069+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'298","last_scrub_stamp":"2024-04-22T20:49:01.508963+0000","last_deep_scrub":"261'298","last_deep_scrub_stamp":"2024-04-22T20:49:01.508963+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.508963+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:48:40.562892+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4632139,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":513,"num_read_kb":15262,"num_write":1122,"num_write_kb":15548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.319368+0000","last_change":"2024-04-22T20:49:08.319368+0000","last_active":"2024-04-22T20:49:08.319368+0000","last_peered":"2024-04-22T20:49:08.319368+0000","last_clean":"2024-04-22T20:49:08.319368+0000","last_became_active":"2024-04-22T19:51:30.527712+0000","last_became_peered":"2024-04-22T19:51:30.527712+0000","last_unstale":"2024-04-22T20:49:08.319368+0000","last_undegraded":"2024-04-22T20:49:08.319368+0000","last_fullsized":"2024-04-22T20:49:08.319368+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:08.319242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:08.319242+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.319242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:55:18.192677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"261'118","reported_seq":671,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.458436+0000","last_change":"2024-04-22T20:48:29.458484+0000","last_active":"2024-04-22T20:48:29.458436+0000","last_peered":"2024-04-22T20:48:29.458436+0000","last_clean":"2024-04-22T20:48:29.458436+0000","last_became_active":"2024-04-22T19:51:32.536177+0000","last_became_peered":"2024-04-22T19:51:32.536177+0000","last_unstale":"2024-04-22T20:48:29.458436+0000","last_undegraded":"2024-04-22T20:48:29.458436+0000","last_fullsized":"2024-04-22T20:48:29.458436+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'118","last_scrub_stamp":"2024-04-22T20:48:29.458333+0000","last_deep_scrub":"261'118","last_deep_scrub_stamp":"2024-04-22T20:48:29.458333+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.458333+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:20:42.045716+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"261'523","reported_seq":2093,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.422135+0000","last_change":"2024-04-22T20:48:43.422135+0000","last_active":"2024-04-22T20:48:43.422135+0000","last_peered":"2024-04-22T20:48:43.422135+0000","last_clean":"2024-04-22T20:48:43.422135+0000","last_became_active":"2024-04-22T19:51:34.561828+0000","last_became_peered":"2024-04-22T19:51:34.561828+0000","last_unstale":"2024-04-22T20:48:43.422135+0000","last_undegraded":"2024-04-22T20:48:43.422135+0000","last_fullsized":"2024-04-22T20:48:43.422135+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'523","last_scrub_stamp":"2024-04-22T20:48:43.422006+0000","last_deep_scrub":"261'523","last_deep_scrub_stamp":"2024-04-22T20:48:43.422006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.422006+0000","objects_scrubbed":1,"log_size":23,"log_dups_size":500,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:09:15.020119+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"261'176","reported_seq":998,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.374062+0000","last_change":"2024-04-22T20:48:30.374062+0000","last_active":"2024-04-22T20:48:30.374062+0000","last_peered":"2024-04-22T20:48:30.374062+0000","last_clean":"2024-04-22T20:48:30.374062+0000","last_became_active":"2024-04-22T19:51:36.565673+0000","last_became_peered":"2024-04-22T19:51:36.565673+0000","last_unstale":"2024-04-22T20:48:30.374062+0000","last_undegraded":"2024-04-22T20:48:30.374062+0000","last_fullsized":"2024-04-22T20:48:30.374062+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'176","last_scrub_stamp":"2024-04-22T20:48:30.373943+0000","last_deep_scrub":"261'176","last_deep_scrub_stamp":"2024-04-22T20:48:30.373943+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.373943+0000","objects_scrubbed":2,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:59:19.241756+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":579,"num_read_kb":567,"num_write":371,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"261'176","reported_seq":824,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.454777+0000","last_change":"2024-04-22T20:48:54.454777+0000","last_active":"2024-04-22T20:48:54.454777+0000","last_peered":"2024-04-22T20:48:54.454777+0000","last_clean":"2024-04-22T20:48:54.454777+0000","last_became_active":"2024-04-22T19:51:28.515670+0000","last_became_peered":"2024-04-22T19:51:28.515670+0000","last_unstale":"2024-04-22T20:48:54.454777+0000","last_undegraded":"2024-04-22T20:48:54.454777+0000","last_fullsized":"2024-04-22T20:48:54.454777+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'176","last_scrub_stamp":"2024-04-22T20:48:54.454610+0000","last_deep_scrub":"261'176","last_deep_scrub_stamp":"2024-04-22T20:48:54.454610+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.454610+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:57:21.493070+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":729,"num_read_kb":590,"num_write":271,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"261'317","reported_seq":1004,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.627131+0000","last_change":"2024-04-22T20:48:47.627177+0000","last_active":"2024-04-22T20:48:47.627131+0000","last_peered":"2024-04-22T20:48:47.627131+0000","last_clean":"2024-04-22T20:48:47.627131+0000","last_became_active":"2024-04-22T19:51:26.503438+0000","last_became_peered":"2024-04-22T19:51:26.503438+0000","last_unstale":"2024-04-22T20:48:47.627131+0000","last_undegraded":"2024-04-22T20:48:47.627131+0000","last_fullsized":"2024-04-22T20:48:47.627131+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:47.627031+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:47.627031+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.627031+0000","objects_scrubbed":7,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:03:27.258395+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3601437,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":581,"num_read_kb":32833,"num_write":1303,"num_write_kb":25399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":501,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.962021+0000","last_change":"2024-04-22T20:48:47.962021+0000","last_active":"2024-04-22T20:48:47.962021+0000","last_peered":"2024-04-22T20:48:47.962021+0000","last_clean":"2024-04-22T20:48:47.962021+0000","last_became_active":"2024-04-22T19:51:30.523125+0000","last_became_peered":"2024-04-22T19:51:30.523125+0000","last_unstale":"2024-04-22T20:48:47.962021+0000","last_undegraded":"2024-04-22T20:48:47.962021+0000","last_fullsized":"2024-04-22T20:48:47.962021+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.961820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.961820+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.961820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:36:15.603182+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"261'257","reported_seq":837,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.599129+0000","last_change":"2024-04-22T20:49:04.599129+0000","last_active":"2024-04-22T20:49:04.599129+0000","last_peered":"2024-04-22T20:49:04.599129+0000","last_clean":"2024-04-22T20:49:04.599129+0000","last_became_active":"2024-04-22T19:51:33.248160+0000","last_became_peered":"2024-04-22T19:51:33.248160+0000","last_unstale":"2024-04-22T20:49:04.599129+0000","last_undegraded":"2024-04-22T20:49:04.599129+0000","last_fullsized":"2024-04-22T20:49:04.599129+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'257","last_scrub_stamp":"2024-04-22T20:49:04.598996+0000","last_deep_scrub":"261'257","last_deep_scrub_stamp":"2024-04-22T20:49:04.598996+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.598996+0000","objects_scrubbed":5,"log_size":57,"log_dups_size":200,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:45:50.161224+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":425,"num_read_kb":974,"num_write":344,"num_write_kb":271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"261'536","reported_seq":4238,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.560508+0000","last_change":"2024-04-22T20:48:43.560537+0000","last_active":"2024-04-22T20:48:43.560508+0000","last_peered":"2024-04-22T20:48:43.560508+0000","last_clean":"2024-04-22T20:48:43.560508+0000","last_became_active":"2024-04-22T19:51:34.555720+0000","last_became_peered":"2024-04-22T19:51:34.555720+0000","last_unstale":"2024-04-22T20:48:43.560508+0000","last_undegraded":"2024-04-22T20:48:43.560508+0000","last_fullsized":"2024-04-22T20:48:43.560508+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'536","last_scrub_stamp":"2024-04-22T20:48:43.560412+0000","last_deep_scrub":"261'536","last_deep_scrub_stamp":"2024-04-22T20:48:43.560412+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.560412+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:21:28.072035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"261'385","reported_seq":1066,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.949946+0000","last_change":"2024-04-22T20:48:46.949973+0000","last_active":"2024-04-22T20:48:46.949946+0000","last_peered":"2024-04-22T20:48:46.949946+0000","last_clean":"2024-04-22T20:48:46.949946+0000","last_became_active":"2024-04-22T19:51:28.512125+0000","last_became_peered":"2024-04-22T19:51:28.512125+0000","last_unstale":"2024-04-22T20:48:46.949946+0000","last_undegraded":"2024-04-22T20:48:46.949946+0000","last_fullsized":"2024-04-22T20:48:46.949946+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'385","last_scrub_stamp":"2024-04-22T20:48:46.949859+0000","last_deep_scrub":"261'385","last_deep_scrub_stamp":"2024-04-22T20:48:46.949859+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.949859+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":300,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:40:49.034926+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1728,"num_read_kb":1430,"num_write":663,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"261'305","reported_seq":956,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:13.548151+0000","last_change":"2024-04-22T20:49:13.548245+0000","last_active":"2024-04-22T20:49:13.548151+0000","last_peered":"2024-04-22T20:49:13.548151+0000","last_clean":"2024-04-22T20:49:13.548151+0000","last_became_active":"2024-04-22T19:51:26.504654+0000","last_became_peered":"2024-04-22T19:51:26.504654+0000","last_unstale":"2024-04-22T20:49:13.548151+0000","last_undegraded":"2024-04-22T20:49:13.548151+0000","last_fullsized":"2024-04-22T20:49:13.548151+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'305","last_scrub_stamp":"2024-04-22T20:49:13.548046+0000","last_deep_scrub":"261'305","last_deep_scrub_stamp":"2024-04-22T20:49:13.548046+0000","last_clean_scrub_stamp":"2024-04-22T20:49:13.548046+0000","objects_scrubbed":10,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:49:58.481055+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10487408,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":445,"num_read_kb":80529,"num_write":1013,"num_write_kb":26179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.400985+0000","last_change":"2024-04-22T20:48:50.401013+0000","last_active":"2024-04-22T20:48:50.400985+0000","last_peered":"2024-04-22T20:48:50.400985+0000","last_clean":"2024-04-22T20:48:50.400985+0000","last_became_active":"2024-04-22T19:51:30.526306+0000","last_became_peered":"2024-04-22T19:51:30.526306+0000","last_unstale":"2024-04-22T20:48:50.400985+0000","last_undegraded":"2024-04-22T20:48:50.400985+0000","last_fullsized":"2024-04-22T20:48:50.400985+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.400891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.400891+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.400891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:05:17.361036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"261'342","reported_seq":966,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.959517+0000","last_change":"2024-04-22T20:48:38.959567+0000","last_active":"2024-04-22T20:48:38.959517+0000","last_peered":"2024-04-22T20:48:38.959517+0000","last_clean":"2024-04-22T20:48:38.959517+0000","last_became_active":"2024-04-22T19:51:33.248059+0000","last_became_peered":"2024-04-22T19:51:33.248059+0000","last_unstale":"2024-04-22T20:48:38.959517+0000","last_undegraded":"2024-04-22T20:48:38.959517+0000","last_fullsized":"2024-04-22T20:48:38.959517+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'342","last_scrub_stamp":"2024-04-22T20:48:38.959416+0000","last_deep_scrub":"261'342","last_deep_scrub_stamp":"2024-04-22T20:48:38.959416+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.959416+0000","objects_scrubbed":9,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:10:05.470719+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45321,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":537,"num_read_kb":1022,"num_write":408,"num_write_kb":248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.728684+0000","last_change":"2024-04-22T20:48:41.728710+0000","last_active":"2024-04-22T20:48:41.728684+0000","last_peered":"2024-04-22T20:48:41.728684+0000","last_clean":"2024-04-22T20:48:41.728684+0000","last_became_active":"2024-04-22T19:51:34.557662+0000","last_became_peered":"2024-04-22T19:51:34.557662+0000","last_unstale":"2024-04-22T20:48:41.728684+0000","last_undegraded":"2024-04-22T20:48:41.728684+0000","last_fullsized":"2024-04-22T20:48:41.728684+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:41.728597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:41.728597+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.728597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:59:34.359524+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"261'566","reported_seq":1383,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.924663+0000","last_change":"2024-04-22T20:48:30.924698+0000","last_active":"2024-04-22T20:48:30.924663+0000","last_peered":"2024-04-22T20:48:30.924663+0000","last_clean":"2024-04-22T20:48:30.924663+0000","last_became_active":"2024-04-22T19:51:36.570520+0000","last_became_peered":"2024-04-22T19:51:36.570520+0000","last_unstale":"2024-04-22T20:48:30.924663+0000","last_undegraded":"2024-04-22T20:48:30.924663+0000","last_fullsized":"2024-04-22T20:48:30.924663+0000","mapping_epoch":30,"log_start":"261'500","ondisk_log_start":"261'500","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'566","last_scrub_stamp":"2024-04-22T20:48:30.924566+0000","last_deep_scrub":"261'566","last_deep_scrub_stamp":"2024-04-22T20:48:30.924566+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.924566+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":500,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:46:13.753614+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1283,"num_read_kb":1280,"num_write":1166,"num_write_kb":397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"261'311","reported_seq":968,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.300802+0000","last_change":"2024-04-22T20:48:53.300849+0000","last_active":"2024-04-22T20:48:53.300802+0000","last_peered":"2024-04-22T20:48:53.300802+0000","last_clean":"2024-04-22T20:48:53.300802+0000","last_became_active":"2024-04-22T19:51:28.511527+0000","last_became_peered":"2024-04-22T19:51:28.511527+0000","last_unstale":"2024-04-22T20:48:53.300802+0000","last_undegraded":"2024-04-22T20:48:53.300802+0000","last_fullsized":"2024-04-22T20:48:53.300802+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'311","last_scrub_stamp":"2024-04-22T20:48:53.300705+0000","last_deep_scrub":"261'311","last_deep_scrub_stamp":"2024-04-22T20:48:53.300705+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.300705+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:10.014981+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1221,"num_read_kb":977,"num_write":455,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"261'330","reported_seq":980,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.583635+0000","last_change":"2024-04-22T20:49:11.583686+0000","last_active":"2024-04-22T20:49:11.583635+0000","last_peered":"2024-04-22T20:49:11.583635+0000","last_clean":"2024-04-22T20:49:11.583635+0000","last_became_active":"2024-04-22T19:51:26.504472+0000","last_became_peered":"2024-04-22T19:51:26.504472+0000","last_unstale":"2024-04-22T20:49:11.583635+0000","last_undegraded":"2024-04-22T20:49:11.583635+0000","last_fullsized":"2024-04-22T20:49:11.583635+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'330","last_scrub_stamp":"2024-04-22T20:49:11.583522+0000","last_deep_scrub":"261'330","last_deep_scrub_stamp":"2024-04-22T20:49:11.583522+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.583522+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:51:00.824173+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2158015,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":483,"num_read_kb":33343,"num_write":1137,"num_write_kb":49829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"261'6","reported_seq":838,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.511865+0000","last_change":"2024-04-22T20:48:37.511895+0000","last_active":"2024-04-22T20:48:37.511865+0000","last_peered":"2024-04-22T20:48:37.511865+0000","last_clean":"2024-04-22T20:48:37.511865+0000","last_became_active":"2024-04-22T19:51:30.524043+0000","last_became_peered":"2024-04-22T19:51:30.524043+0000","last_unstale":"2024-04-22T20:48:37.511865+0000","last_undegraded":"2024-04-22T20:48:37.511865+0000","last_fullsized":"2024-04-22T20:48:37.511865+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'6","last_scrub_stamp":"2024-04-22T20:48:37.511762+0000","last_deep_scrub":"261'6","last_deep_scrub_stamp":"2024-04-22T20:48:37.511762+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.511762+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:16:15.538566+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":492,"num_read_kb":799,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"261'270","reported_seq":858,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.457615+0000","last_change":"2024-04-22T20:48:53.457678+0000","last_active":"2024-04-22T20:48:53.457615+0000","last_peered":"2024-04-22T20:48:53.457615+0000","last_clean":"2024-04-22T20:48:53.457615+0000","last_became_active":"2024-04-22T19:51:32.537968+0000","last_became_peered":"2024-04-22T19:51:32.537968+0000","last_unstale":"2024-04-22T20:48:53.457615+0000","last_undegraded":"2024-04-22T20:48:53.457615+0000","last_fullsized":"2024-04-22T20:48:53.457615+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'270","last_scrub_stamp":"2024-04-22T20:48:53.457514+0000","last_deep_scrub":"261'270","last_deep_scrub_stamp":"2024-04-22T20:48:53.457514+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.457514+0000","objects_scrubbed":4,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:35.634800+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43364,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":998,"num_write":350,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":506,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.498235+0000","last_change":"2024-04-22T20:48:45.498260+0000","last_active":"2024-04-22T20:48:45.498235+0000","last_peered":"2024-04-22T20:48:45.498235+0000","last_clean":"2024-04-22T20:48:45.498235+0000","last_became_active":"2024-04-22T19:51:34.555889+0000","last_became_peered":"2024-04-22T19:51:34.555889+0000","last_unstale":"2024-04-22T20:48:45.498235+0000","last_undegraded":"2024-04-22T20:48:45.498235+0000","last_fullsized":"2024-04-22T20:48:45.498235+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.498142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.498142+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.498142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:09:57.323691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"261'80","reported_seq":587,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.463200+0000","last_change":"2024-04-22T20:48:31.463234+0000","last_active":"2024-04-22T20:48:31.463200+0000","last_peered":"2024-04-22T20:48:31.463200+0000","last_clean":"2024-04-22T20:48:31.463200+0000","last_became_active":"2024-04-22T19:51:36.574555+0000","last_became_peered":"2024-04-22T19:51:36.574555+0000","last_unstale":"2024-04-22T20:48:31.463200+0000","last_undegraded":"2024-04-22T20:48:31.463200+0000","last_fullsized":"2024-04-22T20:48:31.463200+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'80","last_scrub_stamp":"2024-04-22T20:48:31.463110+0000","last_deep_scrub":"261'80","last_deep_scrub_stamp":"2024-04-22T20:48:31.463110+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.463110+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:09:05.164724+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":91,"num_write":194,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"261'189","reported_seq":793,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.643992+0000","last_change":"2024-04-22T20:48:39.644036+0000","last_active":"2024-04-22T20:48:39.643992+0000","last_peered":"2024-04-22T20:48:39.643992+0000","last_clean":"2024-04-22T20:48:39.643992+0000","last_became_active":"2024-04-22T19:51:28.519424+0000","last_became_peered":"2024-04-22T19:51:28.519424+0000","last_unstale":"2024-04-22T20:48:39.643992+0000","last_undegraded":"2024-04-22T20:48:39.643992+0000","last_fullsized":"2024-04-22T20:48:39.643992+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'189","last_scrub_stamp":"2024-04-22T20:48:39.643857+0000","last_deep_scrub":"261'189","last_deep_scrub_stamp":"2024-04-22T20:48:39.643857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.643857+0000","objects_scrubbed":1,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:30:41.321492+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":758,"num_read_kb":601,"num_write":297,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"261'289","reported_seq":920,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.419906+0000","last_change":"2024-04-22T20:48:59.420014+0000","last_active":"2024-04-22T20:48:59.419906+0000","last_peered":"2024-04-22T20:48:59.419906+0000","last_clean":"2024-04-22T20:48:59.419906+0000","last_became_active":"2024-04-22T19:51:26.510273+0000","last_became_peered":"2024-04-22T19:51:26.510273+0000","last_unstale":"2024-04-22T20:48:59.419906+0000","last_undegraded":"2024-04-22T20:48:59.419906+0000","last_fullsized":"2024-04-22T20:48:59.419906+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'289","last_scrub_stamp":"2024-04-22T20:48:59.419798+0000","last_deep_scrub":"261'289","last_deep_scrub_stamp":"2024-04-22T20:48:59.419798+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.419798+0000","objects_scrubbed":9,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:16.853038+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2115,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":380,"num_read_kb":65007,"num_write":1004,"num_write_kb":29343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.472320+0000","last_change":"2024-04-22T20:48:35.472348+0000","last_active":"2024-04-22T20:48:35.472320+0000","last_peered":"2024-04-22T20:48:35.472320+0000","last_clean":"2024-04-22T20:48:35.472320+0000","last_became_active":"2024-04-22T19:51:30.525018+0000","last_became_peered":"2024-04-22T19:51:30.525018+0000","last_unstale":"2024-04-22T20:48:35.472320+0000","last_undegraded":"2024-04-22T20:48:35.472320+0000","last_fullsized":"2024-04-22T20:48:35.472320+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:35.472216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:35.472216+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.472216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:21.173339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"261'317","reported_seq":919,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.539311+0000","last_change":"2024-04-22T20:49:02.539351+0000","last_active":"2024-04-22T20:49:02.539311+0000","last_peered":"2024-04-22T20:49:02.539311+0000","last_clean":"2024-04-22T20:49:02.539311+0000","last_became_active":"2024-04-22T19:51:32.538656+0000","last_became_peered":"2024-04-22T19:51:32.538656+0000","last_unstale":"2024-04-22T20:49:02.539311+0000","last_undegraded":"2024-04-22T20:49:02.539311+0000","last_fullsized":"2024-04-22T20:49:02.539311+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:49:02.539217+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:49:02.539217+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.539217+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:34:48.236169+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47724,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":498,"num_read_kb":1086,"num_write":396,"num_write_kb":291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"261'509","reported_seq":2207,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.561625+0000","last_change":"2024-04-22T20:48:42.561655+0000","last_active":"2024-04-22T20:48:42.561625+0000","last_peered":"2024-04-22T20:48:42.561625+0000","last_clean":"2024-04-22T20:48:42.561625+0000","last_became_active":"2024-04-22T19:51:34.561488+0000","last_became_peered":"2024-04-22T19:51:34.561488+0000","last_unstale":"2024-04-22T20:48:42.561625+0000","last_undegraded":"2024-04-22T20:48:42.561625+0000","last_fullsized":"2024-04-22T20:48:42.561625+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'509","last_scrub_stamp":"2024-04-22T20:48:42.561535+0000","last_deep_scrub":"261'509","last_deep_scrub_stamp":"2024-04-22T20:48:42.561535+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.561535+0000","objects_scrubbed":1,"log_size":109,"log_dups_size":400,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:32:08.827544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"261'136","reported_seq":634,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.548887+0000","last_change":"2024-04-22T20:48:39.548938+0000","last_active":"2024-04-22T20:48:39.548887+0000","last_peered":"2024-04-22T20:48:39.548887+0000","last_clean":"2024-04-22T20:48:39.548887+0000","last_became_active":"2024-04-22T19:51:36.573167+0000","last_became_peered":"2024-04-22T19:51:36.573167+0000","last_unstale":"2024-04-22T20:48:39.548887+0000","last_undegraded":"2024-04-22T20:48:39.548887+0000","last_fullsized":"2024-04-22T20:48:39.548887+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'136","last_scrub_stamp":"2024-04-22T20:48:39.548772+0000","last_deep_scrub":"261'136","last_deep_scrub_stamp":"2024-04-22T20:48:39.548772+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.548772+0000","objects_scrubbed":2,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:36:24.248525+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":807,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":110,"num_write":341,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"261'150","reported_seq":724,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.595935+0000","last_change":"2024-04-22T20:48:54.595961+0000","last_active":"2024-04-22T20:48:54.595935+0000","last_peered":"2024-04-22T20:48:54.595935+0000","last_clean":"2024-04-22T20:48:54.595935+0000","last_became_active":"2024-04-22T19:51:28.518398+0000","last_became_peered":"2024-04-22T19:51:28.518398+0000","last_unstale":"2024-04-22T20:48:54.595935+0000","last_undegraded":"2024-04-22T20:48:54.595935+0000","last_fullsized":"2024-04-22T20:48:54.595935+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'150","last_scrub_stamp":"2024-04-22T20:48:54.595847+0000","last_deep_scrub":"261'150","last_deep_scrub_stamp":"2024-04-22T20:48:54.595847+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.595847+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":100,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:35:56.696922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":596,"num_read_kb":465,"num_write":243,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"261'313","reported_seq":991,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.857532+0000","last_change":"2024-04-22T20:48:24.857599+0000","last_active":"2024-04-22T20:48:24.857532+0000","last_peered":"2024-04-22T20:48:24.857532+0000","last_clean":"2024-04-22T20:48:24.857532+0000","last_became_active":"2024-04-22T19:52:05.112910+0000","last_became_peered":"2024-04-22T19:52:05.112910+0000","last_unstale":"2024-04-22T20:48:24.857532+0000","last_undegraded":"2024-04-22T20:48:24.857532+0000","last_fullsized":"2024-04-22T20:48:24.857532+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:24.857439+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:24.857439+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.857439+0000","objects_scrubbed":12,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:24:03.346339+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":486619,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":517,"num_read_kb":39016,"num_write":1092,"num_write_kb":34073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":513,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.406179+0000","last_change":"2024-04-22T20:48:31.406203+0000","last_active":"2024-04-22T20:48:31.406179+0000","last_peered":"2024-04-22T20:48:31.406179+0000","last_clean":"2024-04-22T20:48:31.406179+0000","last_became_active":"2024-04-22T19:51:30.529011+0000","last_became_peered":"2024-04-22T19:51:30.529011+0000","last_unstale":"2024-04-22T20:48:31.406179+0000","last_undegraded":"2024-04-22T20:48:31.406179+0000","last_fullsized":"2024-04-22T20:48:31.406179+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:31.406079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:31.406079+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.406079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:00:54.573347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"261'493","reported_seq":1136,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.712908+0000","last_change":"2024-04-22T20:48:59.712954+0000","last_active":"2024-04-22T20:48:59.712908+0000","last_peered":"2024-04-22T20:48:59.712908+0000","last_clean":"2024-04-22T20:48:59.712908+0000","last_became_active":"2024-04-22T19:51:32.537267+0000","last_became_peered":"2024-04-22T19:51:32.537267+0000","last_unstale":"2024-04-22T20:48:59.712908+0000","last_undegraded":"2024-04-22T20:48:59.712908+0000","last_fullsized":"2024-04-22T20:48:59.712908+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'493","last_scrub_stamp":"2024-04-22T20:48:59.712822+0000","last_deep_scrub":"261'493","last_deep_scrub_stamp":"2024-04-22T20:48:59.712822+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.712822+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":400,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:29:05.889509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":856,"num_read_kb":2097,"num_write":704,"num_write_kb":611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.377549+0000","last_change":"2024-04-22T20:48:25.377584+0000","last_active":"2024-04-22T20:48:25.377549+0000","last_peered":"2024-04-22T20:48:25.377549+0000","last_clean":"2024-04-22T20:48:25.377549+0000","last_became_active":"2024-04-22T19:51:34.552446+0000","last_became_peered":"2024-04-22T19:51:34.552446+0000","last_unstale":"2024-04-22T20:48:25.377549+0000","last_undegraded":"2024-04-22T20:48:25.377549+0000","last_fullsized":"2024-04-22T20:48:25.377549+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:25.377456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:25.377456+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.377456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:06:04.391078+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"261'107","reported_seq":618,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.636250+0000","last_change":"2024-04-22T20:48:57.636279+0000","last_active":"2024-04-22T20:48:57.636250+0000","last_peered":"2024-04-22T20:48:57.636250+0000","last_clean":"2024-04-22T20:48:57.636250+0000","last_became_active":"2024-04-22T19:51:36.570666+0000","last_became_peered":"2024-04-22T19:51:36.570666+0000","last_unstale":"2024-04-22T20:48:57.636250+0000","last_undegraded":"2024-04-22T20:48:57.636250+0000","last_fullsized":"2024-04-22T20:48:57.636250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'107","last_scrub_stamp":"2024-04-22T20:48:57.636158+0000","last_deep_scrub":"261'107","last_deep_scrub_stamp":"2024-04-22T20:48:57.636158+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.636158+0000","objects_scrubbed":1,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:11:43.927733+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":116,"num_read_kb":113,"num_write":242,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"261'168","reported_seq":804,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.326095+0000","last_change":"2024-04-22T20:48:57.326118+0000","last_active":"2024-04-22T20:48:57.326095+0000","last_peered":"2024-04-22T20:48:57.326095+0000","last_clean":"2024-04-22T20:48:57.326095+0000","last_became_active":"2024-04-22T19:51:28.508288+0000","last_became_peered":"2024-04-22T19:51:28.508288+0000","last_unstale":"2024-04-22T20:48:57.326095+0000","last_undegraded":"2024-04-22T20:48:57.326095+0000","last_fullsized":"2024-04-22T20:48:57.326095+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'168","last_scrub_stamp":"2024-04-22T20:48:57.326006+0000","last_deep_scrub":"261'168","last_deep_scrub_stamp":"2024-04-22T20:48:57.326006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.326006+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:40:08.291741+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":690,"num_read_kb":576,"num_write":244,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"261'317","reported_seq":940,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.310216+0000","last_change":"2024-04-22T20:49:09.310267+0000","last_active":"2024-04-22T20:49:09.310216+0000","last_peered":"2024-04-22T20:49:09.310216+0000","last_clean":"2024-04-22T20:49:09.310216+0000","last_became_active":"2024-04-22T19:51:26.503669+0000","last_became_peered":"2024-04-22T19:51:26.503669+0000","last_unstale":"2024-04-22T20:49:09.310216+0000","last_undegraded":"2024-04-22T20:49:09.310216+0000","last_fullsized":"2024-04-22T20:49:09.310216+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:49:09.310094+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:49:09.310094+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.310094+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:52:43.733235+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194320,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":456,"num_read_kb":4498,"num_write":1097,"num_write_kb":33252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.451692+0000","last_change":"2024-04-22T20:48:33.451746+0000","last_active":"2024-04-22T20:48:33.451692+0000","last_peered":"2024-04-22T20:48:33.451692+0000","last_clean":"2024-04-22T20:48:33.451692+0000","last_became_active":"2024-04-22T19:51:30.526420+0000","last_became_peered":"2024-04-22T19:51:30.526420+0000","last_unstale":"2024-04-22T20:48:33.451692+0000","last_undegraded":"2024-04-22T20:48:33.451692+0000","last_fullsized":"2024-04-22T20:48:33.451692+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.451567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.451567+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.451567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:41:35.930208+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"261'274","reported_seq":862,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.374111+0000","last_change":"2024-04-22T20:48:38.374154+0000","last_active":"2024-04-22T20:48:38.374111+0000","last_peered":"2024-04-22T20:48:38.374111+0000","last_clean":"2024-04-22T20:48:38.374111+0000","last_became_active":"2024-04-22T19:51:32.536463+0000","last_became_peered":"2024-04-22T19:51:32.536463+0000","last_unstale":"2024-04-22T20:48:38.374111+0000","last_undegraded":"2024-04-22T20:48:38.374111+0000","last_fullsized":"2024-04-22T20:48:38.374111+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'274","last_scrub_stamp":"2024-04-22T20:48:38.374022+0000","last_deep_scrub":"261'274","last_deep_scrub_stamp":"2024-04-22T20:48:38.374022+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.374022+0000","objects_scrubbed":6,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:48:58.818076+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42355,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":457,"num_read_kb":963,"num_write":402,"num_write_kb":321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.563374+0000","last_change":"2024-04-22T20:48:44.563409+0000","last_active":"2024-04-22T20:48:44.563374+0000","last_peered":"2024-04-22T20:48:44.563374+0000","last_clean":"2024-04-22T20:48:44.563374+0000","last_became_active":"2024-04-22T19:51:34.554737+0000","last_became_peered":"2024-04-22T19:51:34.554737+0000","last_unstale":"2024-04-22T20:48:44.563374+0000","last_undegraded":"2024-04-22T20:48:44.563374+0000","last_fullsized":"2024-04-22T20:48:44.563374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:44.563281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:44.563281+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.563281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:04:54.592040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"261'96","reported_seq":603,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.589814+0000","last_change":"2024-04-22T20:48:31.589814+0000","last_active":"2024-04-22T20:48:31.589814+0000","last_peered":"2024-04-22T20:48:31.589814+0000","last_clean":"2024-04-22T20:48:31.589814+0000","last_became_active":"2024-04-22T19:51:36.571086+0000","last_became_peered":"2024-04-22T19:51:36.571086+0000","last_unstale":"2024-04-22T20:48:31.589814+0000","last_undegraded":"2024-04-22T20:48:31.589814+0000","last_fullsized":"2024-04-22T20:48:31.589814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'96","last_scrub_stamp":"2024-04-22T20:48:31.589672+0000","last_deep_scrub":"261'96","last_deep_scrub_stamp":"2024-04-22T20:48:31.589672+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.589672+0000","objects_scrubbed":3,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:43:50.688646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":72,"num_write":227,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"261'254","reported_seq":847,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.244259+0000","last_change":"2024-04-22T20:48:49.244288+0000","last_active":"2024-04-22T20:48:49.244259+0000","last_peered":"2024-04-22T20:48:49.244259+0000","last_clean":"2024-04-22T20:48:49.244259+0000","last_became_active":"2024-04-22T19:51:28.514445+0000","last_became_peered":"2024-04-22T19:51:28.514445+0000","last_unstale":"2024-04-22T20:48:49.244259+0000","last_undegraded":"2024-04-22T20:48:49.244259+0000","last_fullsized":"2024-04-22T20:48:49.244259+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'254","last_scrub_stamp":"2024-04-22T20:48:49.244171+0000","last_deep_scrub":"261'254","last_deep_scrub_stamp":"2024-04-22T20:48:49.244171+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.244171+0000","objects_scrubbed":1,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:02:41.401583+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":934,"num_read_kb":723,"num_write":394,"num_write_kb":192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":474,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'395","reported_seq":1150,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.572198+0000","last_change":"2024-04-22T20:48:51.572259+0000","last_active":"2024-04-22T20:48:51.572198+0000","last_peered":"2024-04-22T20:48:51.572198+0000","last_clean":"2024-04-22T20:48:51.572198+0000","last_became_active":"2024-04-22T19:51:26.496408+0000","last_became_peered":"2024-04-22T19:51:26.496408+0000","last_unstale":"2024-04-22T20:48:51.572198+0000","last_undegraded":"2024-04-22T20:48:51.572198+0000","last_fullsized":"2024-04-22T20:48:51.572198+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'395","last_scrub_stamp":"2024-04-22T20:48:51.572069+0000","last_deep_scrub":"261'395","last_deep_scrub_stamp":"2024-04-22T20:48:51.572069+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.572069+0000","objects_scrubbed":4,"log_size":95,"log_dups_size":300,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:05:51.329345+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":840,"num_read_kb":8023,"num_write":1479,"num_write_kb":27015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.499734+0000","last_change":"2024-04-22T20:48:33.499756+0000","last_active":"2024-04-22T20:48:33.499734+0000","last_peered":"2024-04-22T20:48:33.499734+0000","last_clean":"2024-04-22T20:48:33.499734+0000","last_became_active":"2024-04-22T19:51:30.527309+0000","last_became_peered":"2024-04-22T19:51:30.527309+0000","last_unstale":"2024-04-22T20:48:33.499734+0000","last_undegraded":"2024-04-22T20:48:33.499734+0000","last_fullsized":"2024-04-22T20:48:33.499734+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.499634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.499634+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.499634+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:37:13.088297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"261'314","reported_seq":919,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.436156+0000","last_change":"2024-04-22T20:48:27.436199+0000","last_active":"2024-04-22T20:48:27.436156+0000","last_peered":"2024-04-22T20:48:27.436156+0000","last_clean":"2024-04-22T20:48:27.436156+0000","last_became_active":"2024-04-22T19:51:32.539732+0000","last_became_peered":"2024-04-22T19:51:32.539732+0000","last_unstale":"2024-04-22T20:48:27.436156+0000","last_undegraded":"2024-04-22T20:48:27.436156+0000","last_fullsized":"2024-04-22T20:48:27.436156+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'314","last_scrub_stamp":"2024-04-22T20:48:27.436075+0000","last_deep_scrub":"261'314","last_deep_scrub_stamp":"2024-04-22T20:48:27.436075+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.436075+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:26:29.043824+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52364,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":512,"num_read_kb":1133,"num_write":408,"num_write_kb":307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"261'490","reported_seq":2212,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:23.422541+0000","last_change":"2024-04-22T20:48:23.422593+0000","last_active":"2024-04-22T20:48:23.422541+0000","last_peered":"2024-04-22T20:48:23.422541+0000","last_clean":"2024-04-22T20:48:23.422541+0000","last_became_active":"2024-04-22T19:51:34.556124+0000","last_became_peered":"2024-04-22T19:51:34.556124+0000","last_unstale":"2024-04-22T20:48:23.422541+0000","last_undegraded":"2024-04-22T20:48:23.422541+0000","last_fullsized":"2024-04-22T20:48:23.422541+0000","mapping_epoch":28,"log_start":"258'400","ondisk_log_start":"258'400","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'490","last_scrub_stamp":"2024-04-22T20:48:23.422457+0000","last_deep_scrub":"261'490","last_deep_scrub_stamp":"2024-04-22T20:48:23.422457+0000","last_clean_scrub_stamp":"2024-04-22T20:48:23.422457+0000","objects_scrubbed":1,"log_size":90,"log_dups_size":400,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:19:12.183058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"261'120","reported_seq":621,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:30.517134+0000","last_change":"2024-04-22T20:48:30.517192+0000","last_active":"2024-04-22T20:48:30.517134+0000","last_peered":"2024-04-22T20:48:30.517134+0000","last_clean":"2024-04-22T20:48:30.517134+0000","last_became_active":"2024-04-22T19:51:36.572913+0000","last_became_peered":"2024-04-22T19:51:36.572913+0000","last_unstale":"2024-04-22T20:48:30.517134+0000","last_undegraded":"2024-04-22T20:48:30.517134+0000","last_fullsized":"2024-04-22T20:48:30.517134+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'120","last_scrub_stamp":"2024-04-22T20:48:30.517024+0000","last_deep_scrub":"261'120","last_deep_scrub_stamp":"2024-04-22T20:48:30.517024+0000","last_clean_scrub_stamp":"2024-04-22T20:48:30.517024+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:01:42.774241+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1666,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":105,"num_read_kb":102,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"261'338","reported_seq":1089,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.299517+0000","last_change":"2024-04-22T20:48:42.299554+0000","last_active":"2024-04-22T20:48:42.299517+0000","last_peered":"2024-04-22T20:48:42.299517+0000","last_clean":"2024-04-22T20:48:42.299517+0000","last_became_active":"2024-04-22T19:51:28.515809+0000","last_became_peered":"2024-04-22T19:51:28.515809+0000","last_unstale":"2024-04-22T20:48:42.299517+0000","last_undegraded":"2024-04-22T20:48:42.299517+0000","last_fullsized":"2024-04-22T20:48:42.299517+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'338","last_scrub_stamp":"2024-04-22T20:48:42.299436+0000","last_deep_scrub":"261'338","last_deep_scrub_stamp":"2024-04-22T20:48:42.299436+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.299436+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:24:36.610927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1504,"num_read_kb":1227,"num_write":616,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"261'336","reported_seq":980,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.290731+0000","last_change":"2024-04-22T20:48:59.290826+0000","last_active":"2024-04-22T20:48:59.290731+0000","last_peered":"2024-04-22T20:48:59.290731+0000","last_clean":"2024-04-22T20:48:59.290731+0000","last_became_active":"2024-04-22T19:51:26.499362+0000","last_became_peered":"2024-04-22T19:51:26.499362+0000","last_unstale":"2024-04-22T20:48:59.290731+0000","last_undegraded":"2024-04-22T20:48:59.290731+0000","last_fullsized":"2024-04-22T20:48:59.290731+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'336","last_scrub_stamp":"2024-04-22T20:48:59.290601+0000","last_deep_scrub":"261'336","last_deep_scrub_stamp":"2024-04-22T20:48:59.290601+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.290601+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:58:38.439661+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":26712,"num_write":1252,"num_write_kb":36357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"261'4","reported_seq":1146,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.941246+0000","last_change":"2024-04-22T20:48:53.941288+0000","last_active":"2024-04-22T20:48:53.941246+0000","last_peered":"2024-04-22T20:48:53.941246+0000","last_clean":"2024-04-22T20:48:53.941246+0000","last_became_active":"2024-04-22T19:51:30.527490+0000","last_became_peered":"2024-04-22T19:51:30.527490+0000","last_unstale":"2024-04-22T20:48:53.941246+0000","last_undegraded":"2024-04-22T20:48:53.941246+0000","last_fullsized":"2024-04-22T20:48:53.941246+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:48:53.941159+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:48:53.941159+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.941159+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:57:29.554201+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":640,"num_read_kb":640,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"261'95","reported_seq":634,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.456406+0000","last_change":"2024-04-22T20:48:46.456437+0000","last_active":"2024-04-22T20:48:46.456406+0000","last_peered":"2024-04-22T20:48:46.456406+0000","last_clean":"2024-04-22T20:48:46.456406+0000","last_became_active":"2024-04-22T19:51:33.248349+0000","last_became_peered":"2024-04-22T19:51:33.248349+0000","last_unstale":"2024-04-22T20:48:46.456406+0000","last_undegraded":"2024-04-22T20:48:46.456406+0000","last_fullsized":"2024-04-22T20:48:46.456406+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'95","last_scrub_stamp":"2024-04-22T20:48:46.456325+0000","last_deep_scrub":"261'95","last_deep_scrub_stamp":"2024-04-22T20:48:46.456325+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.456325+0000","objects_scrubbed":5,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:50:06.808730+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":85,"num_write":60,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"261'518","reported_seq":2081,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.510605+0000","last_change":"2024-04-22T20:48:44.510682+0000","last_active":"2024-04-22T20:48:44.510605+0000","last_peered":"2024-04-22T20:48:44.510605+0000","last_clean":"2024-04-22T20:48:44.510605+0000","last_became_active":"2024-04-22T19:51:34.561825+0000","last_became_peered":"2024-04-22T19:51:34.561825+0000","last_unstale":"2024-04-22T20:48:44.510605+0000","last_undegraded":"2024-04-22T20:48:44.510605+0000","last_fullsized":"2024-04-22T20:48:44.510605+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'518","last_scrub_stamp":"2024-04-22T20:48:44.510498+0000","last_deep_scrub":"261'518","last_deep_scrub_stamp":"2024-04-22T20:48:44.510498+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.510498+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:12:43.249410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"261'131","reported_seq":635,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:27.325984+0000","last_change":"2024-04-22T20:48:27.326015+0000","last_active":"2024-04-22T20:48:27.325984+0000","last_peered":"2024-04-22T20:48:27.325984+0000","last_clean":"2024-04-22T20:48:27.325984+0000","last_became_active":"2024-04-22T19:51:36.573026+0000","last_became_peered":"2024-04-22T19:51:36.573026+0000","last_unstale":"2024-04-22T20:48:27.325984+0000","last_undegraded":"2024-04-22T20:48:27.325984+0000","last_fullsized":"2024-04-22T20:48:27.325984+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:27.325897+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:27.325897+0000","last_clean_scrub_stamp":"2024-04-22T20:48:27.325897+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:56:17.870401+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":185,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":126,"num_read_kb":119,"num_write":309,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"261'207","reported_seq":827,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.659024+0000","last_change":"2024-04-22T20:48:46.659046+0000","last_active":"2024-04-22T20:48:46.659024+0000","last_peered":"2024-04-22T20:48:46.659024+0000","last_clean":"2024-04-22T20:48:46.659024+0000","last_became_active":"2024-04-22T19:51:28.512275+0000","last_became_peered":"2024-04-22T19:51:28.512275+0000","last_unstale":"2024-04-22T20:48:46.659024+0000","last_undegraded":"2024-04-22T20:48:46.659024+0000","last_fullsized":"2024-04-22T20:48:46.659024+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'207","last_scrub_stamp":"2024-04-22T20:48:46.658952+0000","last_deep_scrub":"261'207","last_deep_scrub_stamp":"2024-04-22T20:48:46.658952+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.658952+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":100,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:19:16.667922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":832,"num_read_kb":677,"num_write":302,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"261'328","reported_seq":975,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:13.328029+0000","last_change":"2024-04-22T20:49:13.328090+0000","last_active":"2024-04-22T20:49:13.328029+0000","last_peered":"2024-04-22T20:49:13.328029+0000","last_clean":"2024-04-22T20:49:13.328029+0000","last_became_active":"2024-04-22T19:51:26.498276+0000","last_became_peered":"2024-04-22T19:51:26.498276+0000","last_unstale":"2024-04-22T20:49:13.328029+0000","last_undegraded":"2024-04-22T20:49:13.328029+0000","last_fullsized":"2024-04-22T20:49:13.328029+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'328","last_scrub_stamp":"2024-04-22T20:49:13.327925+0000","last_deep_scrub":"261'328","last_deep_scrub_stamp":"2024-04-22T20:49:13.327925+0000","last_clean_scrub_stamp":"2024-04-22T20:49:13.327925+0000","objects_scrubbed":12,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:25:52.745613+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1581644,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":468,"num_read_kb":30145,"num_write":1162,"num_write_kb":23135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"261'4","reported_seq":831,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.215281+0000","last_change":"2024-04-22T20:49:06.215317+0000","last_active":"2024-04-22T20:49:06.215281+0000","last_peered":"2024-04-22T20:49:06.215281+0000","last_clean":"2024-04-22T20:49:06.215281+0000","last_became_active":"2024-04-22T19:51:30.530363+0000","last_became_peered":"2024-04-22T19:51:30.530363+0000","last_unstale":"2024-04-22T20:49:06.215281+0000","last_undegraded":"2024-04-22T20:49:06.215281+0000","last_fullsized":"2024-04-22T20:49:06.215281+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4","last_scrub_stamp":"2024-04-22T20:49:06.215191+0000","last_deep_scrub":"261'4","last_deep_scrub_stamp":"2024-04-22T20:49:06.215191+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.215191+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:09:00.109508+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":490,"num_read_kb":488,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"261'410","reported_seq":1027,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.882043+0000","last_change":"2024-04-22T20:49:05.882095+0000","last_active":"2024-04-22T20:49:05.882043+0000","last_peered":"2024-04-22T20:49:05.882043+0000","last_clean":"2024-04-22T20:49:05.882043+0000","last_became_active":"2024-04-22T19:51:32.540622+0000","last_became_peered":"2024-04-22T19:51:32.540622+0000","last_unstale":"2024-04-22T20:49:05.882043+0000","last_undegraded":"2024-04-22T20:49:05.882043+0000","last_fullsized":"2024-04-22T20:49:05.882043+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'410","last_scrub_stamp":"2024-04-22T20:49:05.881933+0000","last_deep_scrub":"261'410","last_deep_scrub_stamp":"2024-04-22T20:49:05.881933+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.881933+0000","objects_scrubbed":9,"log_size":110,"log_dups_size":300,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:14:09.519956+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56095,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1229,"num_read_kb":1850,"num_write":1762,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":478468,"num_omap_keys":1104,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":498,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.518288+0000","last_change":"2024-04-22T20:48:34.518332+0000","last_active":"2024-04-22T20:48:34.518288+0000","last_peered":"2024-04-22T20:48:34.518288+0000","last_clean":"2024-04-22T20:48:34.518288+0000","last_became_active":"2024-04-22T19:51:34.551009+0000","last_became_peered":"2024-04-22T19:51:34.551009+0000","last_unstale":"2024-04-22T20:48:34.518288+0000","last_undegraded":"2024-04-22T20:48:34.518288+0000","last_fullsized":"2024-04-22T20:48:34.518288+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.518165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.518165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.518165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:35:03.091773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"261'124","reported_seq":632,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.530433+0000","last_change":"2024-04-22T20:48:32.530468+0000","last_active":"2024-04-22T20:48:32.530433+0000","last_peered":"2024-04-22T20:48:32.530433+0000","last_clean":"2024-04-22T20:48:32.530433+0000","last_became_active":"2024-04-22T19:51:36.571517+0000","last_became_peered":"2024-04-22T19:51:36.571517+0000","last_unstale":"2024-04-22T20:48:32.530433+0000","last_undegraded":"2024-04-22T20:48:32.530433+0000","last_fullsized":"2024-04-22T20:48:32.530433+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:32.530326+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:32.530326+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.530326+0000","objects_scrubbed":3,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:41:58.642528+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":135,"num_write":272,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"261'4187","reported_seq":4791,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.592564+0000","last_change":"2024-04-22T20:48:41.592601+0000","last_active":"2024-04-22T20:48:41.592564+0000","last_peered":"2024-04-22T20:48:41.592564+0000","last_clean":"2024-04-22T20:48:41.592564+0000","last_became_active":"2024-04-22T19:51:28.514869+0000","last_became_peered":"2024-04-22T19:51:28.514869+0000","last_unstale":"2024-04-22T20:48:41.592564+0000","last_undegraded":"2024-04-22T20:48:41.592564+0000","last_fullsized":"2024-04-22T20:48:41.592564+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4187","last_scrub_stamp":"2024-04-22T20:48:41.592448+0000","last_deep_scrub":"261'4187","last_deep_scrub_stamp":"2024-04-22T20:48:41.592448+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.592448+0000","objects_scrubbed":1,"log_size":87,"log_dups_size":3000,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:17:00.234094+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14767,"num_read_kb":11981,"num_write":6295,"num_write_kb":3142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":530,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.576811+0000","last_change":"2024-04-22T20:49:07.576811+0000","last_active":"2024-04-22T20:49:07.576811+0000","last_peered":"2024-04-22T20:49:07.576811+0000","last_clean":"2024-04-22T20:49:07.576811+0000","last_became_active":"2024-04-22T19:51:12.250797+0000","last_became_peered":"2024-04-22T19:51:12.250797+0000","last_unstale":"2024-04-22T20:49:07.576811+0000","last_undegraded":"2024-04-22T20:49:07.576811+0000","last_fullsized":"2024-04-22T20:49:07.576811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:07.576689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:07.576689+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.576689+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:34:59.756037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"261'336","reported_seq":1040,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.341507+0000","last_change":"2024-04-22T20:49:07.341575+0000","last_active":"2024-04-22T20:49:07.341507+0000","last_peered":"2024-04-22T20:49:07.341507+0000","last_clean":"2024-04-22T20:49:07.341507+0000","last_became_active":"2024-04-22T19:51:26.505982+0000","last_became_peered":"2024-04-22T19:51:26.505982+0000","last_unstale":"2024-04-22T20:49:07.341507+0000","last_undegraded":"2024-04-22T20:49:07.341507+0000","last_fullsized":"2024-04-22T20:49:07.341507+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'336","last_scrub_stamp":"2024-04-22T20:49:07.341421+0000","last_deep_scrub":"261'336","last_deep_scrub_stamp":"2024-04-22T20:49:07.341421+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.341421+0000","objects_scrubbed":14,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:31:47.801140+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13893637,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":542,"num_read_kb":107293,"num_write":1359,"num_write_kb":45616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.405165+0000","last_change":"2024-04-22T20:48:56.405202+0000","last_active":"2024-04-22T20:48:56.405165+0000","last_peered":"2024-04-22T20:48:56.405165+0000","last_clean":"2024-04-22T20:48:56.405165+0000","last_became_active":"2024-04-22T19:51:30.526939+0000","last_became_peered":"2024-04-22T19:51:30.526939+0000","last_unstale":"2024-04-22T20:48:56.405165+0000","last_undegraded":"2024-04-22T20:48:56.405165+0000","last_fullsized":"2024-04-22T20:48:56.405165+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:56.405070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:56.405070+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.405070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:57:17.766049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"261'718","reported_seq":1428,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.940154+0000","last_change":"2024-04-22T20:48:40.940188+0000","last_active":"2024-04-22T20:48:40.940154+0000","last_peered":"2024-04-22T20:48:40.940154+0000","last_clean":"2024-04-22T20:48:40.940154+0000","last_became_active":"2024-04-22T19:51:32.542970+0000","last_became_peered":"2024-04-22T19:51:32.542970+0000","last_unstale":"2024-04-22T20:48:40.940154+0000","last_undegraded":"2024-04-22T20:48:40.940154+0000","last_fullsized":"2024-04-22T20:48:40.940154+0000","mapping_epoch":26,"log_start":"261'700","ondisk_log_start":"261'700","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'718","last_scrub_stamp":"2024-04-22T20:48:40.940067+0000","last_deep_scrub":"261'718","last_deep_scrub_stamp":"2024-04-22T20:48:40.940067+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.940067+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":700,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:32:36.401897+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142271,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1403,"num_read_kb":3194,"num_write":1318,"num_write_kb":1170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":89366,"num_omap_keys":214,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.618892+0000","last_change":"2024-04-22T20:48:48.618914+0000","last_active":"2024-04-22T20:48:48.618892+0000","last_peered":"2024-04-22T20:48:48.618892+0000","last_clean":"2024-04-22T20:48:48.618892+0000","last_became_active":"2024-04-22T19:51:34.553704+0000","last_became_peered":"2024-04-22T19:51:34.553704+0000","last_unstale":"2024-04-22T20:48:48.618892+0000","last_undegraded":"2024-04-22T20:48:48.618892+0000","last_fullsized":"2024-04-22T20:48:48.618892+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:48.618804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:48.618804+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.618804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:35:39.328500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"261'272","reported_seq":1016,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.596549+0000","last_change":"2024-04-22T20:48:46.596598+0000","last_active":"2024-04-22T20:48:46.596549+0000","last_peered":"2024-04-22T20:48:46.596549+0000","last_clean":"2024-04-22T20:48:46.596549+0000","last_became_active":"2024-04-22T19:51:36.570717+0000","last_became_peered":"2024-04-22T19:51:36.570717+0000","last_unstale":"2024-04-22T20:48:46.596549+0000","last_undegraded":"2024-04-22T20:48:46.596549+0000","last_fullsized":"2024-04-22T20:48:46.596549+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'272","last_scrub_stamp":"2024-04-22T20:48:46.596449+0000","last_deep_scrub":"261'272","last_deep_scrub_stamp":"2024-04-22T20:48:46.596449+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.596449+0000","objects_scrubbed":6,"log_size":72,"log_dups_size":200,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:32:49.047679+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2081,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":619,"num_read_kb":568,"num_write":495,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"261'50","reported_seq":596,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.539678+0000","last_change":"2024-04-22T20:48:55.539707+0000","last_active":"2024-04-22T20:48:55.539678+0000","last_peered":"2024-04-22T20:48:55.539678+0000","last_clean":"2024-04-22T20:48:55.539678+0000","last_became_active":"2024-04-22T19:51:28.505719+0000","last_became_peered":"2024-04-22T19:51:28.505719+0000","last_unstale":"2024-04-22T20:48:55.539678+0000","last_undegraded":"2024-04-22T20:48:55.539678+0000","last_fullsized":"2024-04-22T20:48:55.539678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'50","last_scrub_stamp":"2024-04-22T20:48:55.539583+0000","last_deep_scrub":"261'50","last_deep_scrub_stamp":"2024-04-22T20:48:55.539583+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.539583+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:48:12.051486+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":195,"num_read_kb":157,"num_write":69,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":524,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.792144+0000","last_change":"2024-04-22T20:48:57.792144+0000","last_active":"2024-04-22T20:48:57.792144+0000","last_peered":"2024-04-22T20:48:57.792144+0000","last_clean":"2024-04-22T20:48:57.792144+0000","last_became_active":"2024-04-22T19:51:12.250786+0000","last_became_peered":"2024-04-22T19:51:12.250786+0000","last_unstale":"2024-04-22T20:48:57.792144+0000","last_undegraded":"2024-04-22T20:48:57.792144+0000","last_fullsized":"2024-04-22T20:48:57.792144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.791981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.791981+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.791981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:13:35.794791+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"261'358","reported_seq":1043,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.259784+0000","last_change":"2024-04-22T20:49:00.259858+0000","last_active":"2024-04-22T20:49:00.259784+0000","last_peered":"2024-04-22T20:49:00.259784+0000","last_clean":"2024-04-22T20:49:00.259784+0000","last_became_active":"2024-04-22T19:51:26.500968+0000","last_became_peered":"2024-04-22T19:51:26.500968+0000","last_unstale":"2024-04-22T20:49:00.259784+0000","last_undegraded":"2024-04-22T20:49:00.259784+0000","last_fullsized":"2024-04-22T20:49:00.259784+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'358","last_scrub_stamp":"2024-04-22T20:49:00.259692+0000","last_deep_scrub":"261'358","last_deep_scrub_stamp":"2024-04-22T20:49:00.259692+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.259692+0000","objects_scrubbed":7,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:46:46.910363+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048617,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":597,"num_read_kb":30331,"num_write":1294,"num_write_kb":32709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.639975+0000","last_change":"2024-04-22T20:48:55.639998+0000","last_active":"2024-04-22T20:48:55.639975+0000","last_peered":"2024-04-22T20:48:55.639975+0000","last_clean":"2024-04-22T20:48:55.639975+0000","last_became_active":"2024-04-22T19:51:30.526328+0000","last_became_peered":"2024-04-22T19:51:30.526328+0000","last_unstale":"2024-04-22T20:48:55.639975+0000","last_undegraded":"2024-04-22T20:48:55.639975+0000","last_fullsized":"2024-04-22T20:48:55.639975+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:55.639878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:55.639878+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.639878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:22:06.953672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"261'131","reported_seq":693,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.513766+0000","last_change":"2024-04-22T20:48:50.513809+0000","last_active":"2024-04-22T20:48:50.513766+0000","last_peered":"2024-04-22T20:48:50.513766+0000","last_clean":"2024-04-22T20:48:50.513766+0000","last_became_active":"2024-04-22T19:51:32.540805+0000","last_became_peered":"2024-04-22T19:51:32.540805+0000","last_unstale":"2024-04-22T20:48:50.513766+0000","last_undegraded":"2024-04-22T20:48:50.513766+0000","last_fullsized":"2024-04-22T20:48:50.513766+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:50.513677+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:50.513677+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.513677+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:46.102770+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":158,"num_read_kb":152,"num_write":112,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1504,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":499,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.594752+0000","last_change":"2024-04-22T20:48:40.594815+0000","last_active":"2024-04-22T20:48:40.594752+0000","last_peered":"2024-04-22T20:48:40.594752+0000","last_clean":"2024-04-22T20:48:40.594752+0000","last_became_active":"2024-04-22T19:51:34.554307+0000","last_became_peered":"2024-04-22T19:51:34.554307+0000","last_unstale":"2024-04-22T20:48:40.594752+0000","last_undegraded":"2024-04-22T20:48:40.594752+0000","last_fullsized":"2024-04-22T20:48:40.594752+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:40.594644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:40.594644+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.594644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:32:53.799541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"261'124","reported_seq":635,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.895261+0000","last_change":"2024-04-22T20:48:29.895308+0000","last_active":"2024-04-22T20:48:29.895261+0000","last_peered":"2024-04-22T20:48:29.895261+0000","last_clean":"2024-04-22T20:48:29.895261+0000","last_became_active":"2024-04-22T19:51:36.571130+0000","last_became_peered":"2024-04-22T19:51:36.571130+0000","last_unstale":"2024-04-22T20:48:29.895261+0000","last_undegraded":"2024-04-22T20:48:29.895261+0000","last_fullsized":"2024-04-22T20:48:29.895261+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:29.895174+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:29.895174+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.895174+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:20:29.352021+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":131,"num_write":305,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"261'90","reported_seq":667,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.696575+0000","last_change":"2024-04-22T20:48:44.696603+0000","last_active":"2024-04-22T20:48:44.696575+0000","last_peered":"2024-04-22T20:48:44.696575+0000","last_clean":"2024-04-22T20:48:44.696575+0000","last_became_active":"2024-04-22T19:51:28.513451+0000","last_became_peered":"2024-04-22T19:51:28.513451+0000","last_unstale":"2024-04-22T20:48:44.696575+0000","last_undegraded":"2024-04-22T20:48:44.696575+0000","last_fullsized":"2024-04-22T20:48:44.696575+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'90","last_scrub_stamp":"2024-04-22T20:48:44.696479+0000","last_deep_scrub":"261'90","last_deep_scrub_stamp":"2024-04-22T20:48:44.696479+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.696479+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:44:49.307610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":372,"num_read_kb":305,"num_write":130,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":529,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.613875+0000","last_change":"2024-04-22T20:49:06.613875+0000","last_active":"2024-04-22T20:49:06.613875+0000","last_peered":"2024-04-22T20:49:06.613875+0000","last_clean":"2024-04-22T20:49:06.613875+0000","last_became_active":"2024-04-22T19:51:12.250717+0000","last_became_peered":"2024-04-22T19:51:12.250717+0000","last_unstale":"2024-04-22T20:49:06.613875+0000","last_undegraded":"2024-04-22T20:49:06.613875+0000","last_fullsized":"2024-04-22T20:49:06.613875+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:06.613754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:06.613754+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.613754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:21:06.767402+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"261'323","reported_seq":995,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.609839+0000","last_change":"2024-04-22T20:48:50.609839+0000","last_active":"2024-04-22T20:48:50.609839+0000","last_peered":"2024-04-22T20:48:50.609839+0000","last_clean":"2024-04-22T20:48:50.609839+0000","last_became_active":"2024-04-22T19:51:26.495405+0000","last_became_peered":"2024-04-22T19:51:26.495405+0000","last_unstale":"2024-04-22T20:48:50.609839+0000","last_undegraded":"2024-04-22T20:48:50.609839+0000","last_fullsized":"2024-04-22T20:48:50.609839+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'323","last_scrub_stamp":"2024-04-22T20:48:50.609693+0000","last_deep_scrub":"261'323","last_deep_scrub_stamp":"2024-04-22T20:48:50.609693+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.609693+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:11:03.984643+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":264208,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":523,"num_read_kb":57840,"num_write":1265,"num_write_kb":50104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"261'2","reported_seq":515,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.599212+0000","last_change":"2024-04-22T20:48:45.599259+0000","last_active":"2024-04-22T20:48:45.599212+0000","last_peered":"2024-04-22T20:48:45.599212+0000","last_clean":"2024-04-22T20:48:45.599212+0000","last_became_active":"2024-04-22T19:51:30.521134+0000","last_became_peered":"2024-04-22T19:51:30.521134+0000","last_unstale":"2024-04-22T20:48:45.599212+0000","last_undegraded":"2024-04-22T20:48:45.599212+0000","last_fullsized":"2024-04-22T20:48:45.599212+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:45.599098+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:45.599098+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.599098+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:03:06.489339+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"261'230","reported_seq":1466,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.455374+0000","last_change":"2024-04-22T20:48:49.455413+0000","last_active":"2024-04-22T20:48:49.455374+0000","last_peered":"2024-04-22T20:48:49.455374+0000","last_clean":"2024-04-22T20:48:49.455374+0000","last_became_active":"2024-04-22T19:51:33.250082+0000","last_became_peered":"2024-04-22T19:51:33.250082+0000","last_unstale":"2024-04-22T20:48:49.455374+0000","last_undegraded":"2024-04-22T20:48:49.455374+0000","last_fullsized":"2024-04-22T20:48:49.455374+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'230","last_scrub_stamp":"2024-04-22T20:48:49.455289+0000","last_deep_scrub":"261'230","last_deep_scrub_stamp":"2024-04-22T20:48:49.455289+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.455289+0000","objects_scrubbed":9,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:37:11.492021+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":188,"num_read_kb":180,"num_write":143,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5400,"num_omap_keys":12,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:50.690582+0000","last_change":"2024-04-22T20:48:50.690632+0000","last_active":"2024-04-22T20:48:50.690582+0000","last_peered":"2024-04-22T20:48:50.690582+0000","last_clean":"2024-04-22T20:48:50.690582+0000","last_became_active":"2024-04-22T19:51:34.546652+0000","last_became_peered":"2024-04-22T19:51:34.546652+0000","last_unstale":"2024-04-22T20:48:50.690582+0000","last_undegraded":"2024-04-22T20:48:50.690582+0000","last_fullsized":"2024-04-22T20:48:50.690582+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:50.690480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:50.690480+0000","last_clean_scrub_stamp":"2024-04-22T20:48:50.690480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:59:48.164650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"261'258","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.494703+0000","last_change":"2024-04-22T20:48:34.494735+0000","last_active":"2024-04-22T20:48:34.494703+0000","last_peered":"2024-04-22T20:48:34.494703+0000","last_clean":"2024-04-22T20:48:34.494703+0000","last_became_active":"2024-04-22T19:51:36.568767+0000","last_became_peered":"2024-04-22T19:51:36.568767+0000","last_unstale":"2024-04-22T20:48:34.494703+0000","last_undegraded":"2024-04-22T20:48:34.494703+0000","last_fullsized":"2024-04-22T20:48:34.494703+0000","mapping_epoch":30,"log_start":"261'200","ondisk_log_start":"261'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'258","last_scrub_stamp":"2024-04-22T20:48:34.494609+0000","last_deep_scrub":"261'258","last_deep_scrub_stamp":"2024-04-22T20:48:34.494609+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.494609+0000","objects_scrubbed":4,"log_size":58,"log_dups_size":200,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:08:51.017317+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":642,"num_read_kb":625,"num_write":569,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"261'125","reported_seq":761,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.657888+0000","last_change":"2024-04-22T20:48:56.657915+0000","last_active":"2024-04-22T20:48:56.657888+0000","last_peered":"2024-04-22T20:48:56.657888+0000","last_clean":"2024-04-22T20:48:56.657888+0000","last_became_active":"2024-04-22T19:51:28.518084+0000","last_became_peered":"2024-04-22T19:51:28.518084+0000","last_unstale":"2024-04-22T20:48:56.657888+0000","last_undegraded":"2024-04-22T20:48:56.657888+0000","last_fullsized":"2024-04-22T20:48:56.657888+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'125","last_scrub_stamp":"2024-04-22T20:48:56.657791+0000","last_deep_scrub":"261'125","last_deep_scrub_stamp":"2024-04-22T20:48:56.657791+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.657791+0000","objects_scrubbed":1,"log_size":25,"log_dups_size":100,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:27:58.131958+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":533,"num_read_kb":444,"num_write":173,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":546,"num_omap_keys":3,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":527,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.596128+0000","last_change":"2024-04-22T20:48:49.596128+0000","last_active":"2024-04-22T20:48:49.596128+0000","last_peered":"2024-04-22T20:48:49.596128+0000","last_clean":"2024-04-22T20:48:49.596128+0000","last_became_active":"2024-04-22T19:51:11.437440+0000","last_became_peered":"2024-04-22T19:51:11.437440+0000","last_unstale":"2024-04-22T20:48:49.596128+0000","last_undegraded":"2024-04-22T20:48:49.596128+0000","last_fullsized":"2024-04-22T20:48:49.596128+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.596018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.596018+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.596018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:43:28.175145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"261'119","reported_seq":680,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:08.587820+0000","last_change":"2024-04-22T20:49:08.587820+0000","last_active":"2024-04-22T20:49:08.587820+0000","last_peered":"2024-04-22T20:49:08.587820+0000","last_clean":"2024-04-22T20:49:08.587820+0000","last_became_active":"2024-04-22T19:51:12.891620+0000","last_became_peered":"2024-04-22T19:51:12.891620+0000","last_unstale":"2024-04-22T20:49:08.587820+0000","last_undegraded":"2024-04-22T20:49:08.587820+0000","last_fullsized":"2024-04-22T20:49:08.587820+0000","mapping_epoch":15,"log_start":"16'100","ondisk_log_start":"16'100","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'119","last_scrub_stamp":"2024-04-22T20:49:08.587682+0000","last_deep_scrub":"261'119","last_deep_scrub_stamp":"2024-04-22T20:49:08.587682+0000","last_clean_scrub_stamp":"2024-04-22T20:49:08.587682+0000","objects_scrubbed":2,"log_size":19,"log_dups_size":100,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:12:58.890978+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"261'340","reported_seq":992,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:07.739859+0000","last_change":"2024-04-22T20:49:07.739920+0000","last_active":"2024-04-22T20:49:07.739859+0000","last_peered":"2024-04-22T20:49:07.739859+0000","last_clean":"2024-04-22T20:49:07.739859+0000","last_became_active":"2024-04-22T19:51:26.497608+0000","last_became_peered":"2024-04-22T19:51:26.497608+0000","last_unstale":"2024-04-22T20:49:07.739859+0000","last_undegraded":"2024-04-22T20:49:07.739859+0000","last_fullsized":"2024-04-22T20:49:07.739859+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'340","last_scrub_stamp":"2024-04-22T20:49:07.739756+0000","last_deep_scrub":"261'340","last_deep_scrub_stamp":"2024-04-22T20:49:07.739756+0000","last_clean_scrub_stamp":"2024-04-22T20:49:07.739756+0000","objects_scrubbed":4,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:25.144936+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4198438,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":523,"num_read_kb":42688,"num_write":1248,"num_write_kb":37697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.993233+0000","last_change":"2024-04-22T20:48:49.993255+0000","last_active":"2024-04-22T20:48:49.993233+0000","last_peered":"2024-04-22T20:48:49.993233+0000","last_clean":"2024-04-22T20:48:49.993233+0000","last_became_active":"2024-04-22T19:51:30.524437+0000","last_became_peered":"2024-04-22T19:51:30.524437+0000","last_unstale":"2024-04-22T20:48:49.993233+0000","last_undegraded":"2024-04-22T20:48:49.993233+0000","last_fullsized":"2024-04-22T20:48:49.993233+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.993107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.993107+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.993107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:03:04.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"261'1111","reported_seq":2658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.570536+0000","last_change":"2024-04-22T20:48:35.570575+0000","last_active":"2024-04-22T20:48:35.570536+0000","last_peered":"2024-04-22T20:48:35.570536+0000","last_clean":"2024-04-22T20:48:35.570536+0000","last_became_active":"2024-04-22T19:51:32.538560+0000","last_became_peered":"2024-04-22T19:51:32.538560+0000","last_unstale":"2024-04-22T20:48:35.570536+0000","last_undegraded":"2024-04-22T20:48:35.570536+0000","last_fullsized":"2024-04-22T20:48:35.570536+0000","mapping_epoch":26,"log_start":"261'1100","ondisk_log_start":"261'1100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'1111","last_scrub_stamp":"2024-04-22T20:48:35.570456+0000","last_deep_scrub":"261'1111","last_deep_scrub_stamp":"2024-04-22T20:48:35.570456+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.570456+0000","objects_scrubbed":11,"log_size":11,"log_dups_size":1100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:51:22.759382+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148242,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1781,"num_read_kb":3545,"num_write":1100,"num_write_kb":929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5722,"num_omap_keys":16,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.453095+0000","last_change":"2024-04-22T20:48:46.453118+0000","last_active":"2024-04-22T20:48:46.453095+0000","last_peered":"2024-04-22T20:48:46.453095+0000","last_clean":"2024-04-22T20:48:46.453095+0000","last_became_active":"2024-04-22T19:51:34.557022+0000","last_became_peered":"2024-04-22T19:51:34.557022+0000","last_unstale":"2024-04-22T20:48:46.453095+0000","last_undegraded":"2024-04-22T20:48:46.453095+0000","last_fullsized":"2024-04-22T20:48:46.453095+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:46.453005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:46.453005+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.453005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:30:22.992979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"261'167","reported_seq":684,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.647984+0000","last_change":"2024-04-22T20:48:47.648022+0000","last_active":"2024-04-22T20:48:47.647984+0000","last_peered":"2024-04-22T20:48:47.647984+0000","last_clean":"2024-04-22T20:48:47.647984+0000","last_became_active":"2024-04-22T19:51:36.567603+0000","last_became_peered":"2024-04-22T19:51:36.567603+0000","last_unstale":"2024-04-22T20:48:47.647984+0000","last_undegraded":"2024-04-22T20:48:47.647984+0000","last_fullsized":"2024-04-22T20:48:47.647984+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'167","last_scrub_stamp":"2024-04-22T20:48:47.647894+0000","last_deep_scrub":"261'167","last_deep_scrub_stamp":"2024-04-22T20:48:47.647894+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.647894+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:03:44.321582+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":641,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":222,"num_read_kb":216,"num_write":373,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"261'364","reported_seq":1007,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.287319+0000","last_change":"2024-04-22T20:48:51.287344+0000","last_active":"2024-04-22T20:48:51.287319+0000","last_peered":"2024-04-22T20:48:51.287319+0000","last_clean":"2024-04-22T20:48:51.287319+0000","last_became_active":"2024-04-22T19:51:28.512561+0000","last_became_peered":"2024-04-22T19:51:28.512561+0000","last_unstale":"2024-04-22T20:48:51.287319+0000","last_undegraded":"2024-04-22T20:48:51.287319+0000","last_fullsized":"2024-04-22T20:48:51.287319+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'364","last_scrub_stamp":"2024-04-22T20:48:51.287228+0000","last_deep_scrub":"261'364","last_deep_scrub_stamp":"2024-04-22T20:48:51.287228+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.287228+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:36:20.582606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1455,"num_read_kb":1128,"num_write":627,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"261'380","reported_seq":1106,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.729429+0000","last_change":"2024-04-22T20:49:09.729518+0000","last_active":"2024-04-22T20:49:09.729429+0000","last_peered":"2024-04-22T20:49:09.729429+0000","last_clean":"2024-04-22T20:49:09.729429+0000","last_became_active":"2024-04-22T19:51:26.498178+0000","last_became_peered":"2024-04-22T19:51:26.498178+0000","last_unstale":"2024-04-22T20:49:09.729429+0000","last_undegraded":"2024-04-22T20:49:09.729429+0000","last_fullsized":"2024-04-22T20:49:09.729429+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'380","last_scrub_stamp":"2024-04-22T20:49:09.729304+0000","last_deep_scrub":"261'380","last_deep_scrub_stamp":"2024-04-22T20:49:09.729304+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.729304+0000","objects_scrubbed":9,"log_size":80,"log_dups_size":300,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:46:50.275561+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":432486,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":709,"num_read_kb":47025,"num_write":1529,"num_write_kb":22156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":536,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.275977+0000","last_change":"2024-04-22T20:49:10.275977+0000","last_active":"2024-04-22T20:49:10.275977+0000","last_peered":"2024-04-22T20:49:10.275977+0000","last_clean":"2024-04-22T20:49:10.275977+0000","last_became_active":"2024-04-22T19:51:11.434525+0000","last_became_peered":"2024-04-22T19:51:11.434525+0000","last_unstale":"2024-04-22T20:49:10.275977+0000","last_undegraded":"2024-04-22T20:49:10.275977+0000","last_fullsized":"2024-04-22T20:49:10.275977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:10.275826+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:10.275826+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.275826+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:01:41.001016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":517,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.407429+0000","last_change":"2024-04-22T20:49:02.407429+0000","last_active":"2024-04-22T20:49:02.407429+0000","last_peered":"2024-04-22T20:49:02.407429+0000","last_clean":"2024-04-22T20:49:02.407429+0000","last_became_active":"2024-04-22T19:51:30.523242+0000","last_became_peered":"2024-04-22T19:51:30.523242+0000","last_unstale":"2024-04-22T20:49:02.407429+0000","last_undegraded":"2024-04-22T20:49:02.407429+0000","last_fullsized":"2024-04-22T20:49:02.407429+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:02.407287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:02.407287+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.407287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:47:26.442506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"261'716","reported_seq":1450,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:14.566971+0000","last_change":"2024-04-22T20:49:14.567056+0000","last_active":"2024-04-22T20:49:14.566971+0000","last_peered":"2024-04-22T20:49:14.566971+0000","last_clean":"2024-04-22T20:49:14.566971+0000","last_became_active":"2024-04-22T19:51:33.248144+0000","last_became_peered":"2024-04-22T19:51:33.248144+0000","last_unstale":"2024-04-22T20:49:14.566971+0000","last_undegraded":"2024-04-22T20:49:14.566971+0000","last_fullsized":"2024-04-22T20:49:14.566971+0000","mapping_epoch":26,"log_start":"261'700","ondisk_log_start":"261'700","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'716","last_scrub_stamp":"2024-04-22T20:49:14.566857+0000","last_deep_scrub":"261'716","last_deep_scrub_stamp":"2024-04-22T20:49:14.566857+0000","last_clean_scrub_stamp":"2024-04-22T20:49:14.566857+0000","objects_scrubbed":9,"log_size":16,"log_dups_size":700,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:48:00.839694+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130335,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1263,"num_read_kb":2915,"num_write":1018,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":507,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.457004+0000","last_change":"2024-04-22T20:48:39.457028+0000","last_active":"2024-04-22T20:48:39.457004+0000","last_peered":"2024-04-22T20:48:39.457004+0000","last_clean":"2024-04-22T20:48:39.457004+0000","last_became_active":"2024-04-22T19:51:34.562001+0000","last_became_peered":"2024-04-22T19:51:34.562001+0000","last_unstale":"2024-04-22T20:48:39.457004+0000","last_undegraded":"2024-04-22T20:48:39.457004+0000","last_fullsized":"2024-04-22T20:48:39.457004+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:39.456917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:39.456917+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.456917+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:11:40.897612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"261'107","reported_seq":598,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.978305+0000","last_change":"2024-04-22T20:48:32.978335+0000","last_active":"2024-04-22T20:48:32.978305+0000","last_peered":"2024-04-22T20:48:32.978305+0000","last_clean":"2024-04-22T20:48:32.978305+0000","last_became_active":"2024-04-22T19:51:36.570134+0000","last_became_peered":"2024-04-22T19:51:36.570134+0000","last_unstale":"2024-04-22T20:48:32.978305+0000","last_undegraded":"2024-04-22T20:48:32.978305+0000","last_fullsized":"2024-04-22T20:48:32.978305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'107","last_scrub_stamp":"2024-04-22T20:48:32.978212+0000","last_deep_scrub":"261'107","last_deep_scrub_stamp":"2024-04-22T20:48:32.978212+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.978212+0000","objects_scrubbed":2,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:21.385230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":256,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"261'102","reported_seq":699,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.482598+0000","last_change":"2024-04-22T20:48:55.482598+0000","last_active":"2024-04-22T20:48:55.482598+0000","last_peered":"2024-04-22T20:48:55.482598+0000","last_clean":"2024-04-22T20:48:55.482598+0000","last_became_active":"2024-04-22T19:51:28.516178+0000","last_became_peered":"2024-04-22T19:51:28.516178+0000","last_unstale":"2024-04-22T20:48:55.482598+0000","last_undegraded":"2024-04-22T20:48:55.482598+0000","last_fullsized":"2024-04-22T20:48:55.482598+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'102","last_scrub_stamp":"2024-04-22T20:48:55.482441+0000","last_deep_scrub":"261'102","last_deep_scrub_stamp":"2024-04-22T20:48:55.482441+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.482441+0000","objects_scrubbed":1,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:57:52.451471+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":407,"num_read_kb":337,"num_write":140,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"261'359","reported_seq":1063,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:16.322689+0000","last_change":"2024-04-22T20:49:16.322826+0000","last_active":"2024-04-22T20:49:16.322689+0000","last_peered":"2024-04-22T20:49:16.322689+0000","last_clean":"2024-04-22T20:49:16.322689+0000","last_became_active":"2024-04-22T19:51:26.502765+0000","last_became_peered":"2024-04-22T19:51:26.502765+0000","last_unstale":"2024-04-22T20:49:16.322689+0000","last_undegraded":"2024-04-22T20:49:16.322689+0000","last_fullsized":"2024-04-22T20:49:16.322689+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'359","last_scrub_stamp":"2024-04-22T20:49:16.322563+0000","last_deep_scrub":"261'359","last_deep_scrub_stamp":"2024-04-22T20:49:16.322563+0000","last_clean_scrub_stamp":"2024-04-22T20:49:16.322563+0000","objects_scrubbed":7,"log_size":59,"log_dups_size":300,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:07:24.118720+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4728363,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":573,"num_read_kb":127053,"num_write":1383,"num_write_kb":34012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":528,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.304837+0000","last_change":"2024-04-22T20:48:58.304837+0000","last_active":"2024-04-22T20:48:58.304837+0000","last_peered":"2024-04-22T20:48:58.304837+0000","last_clean":"2024-04-22T20:48:58.304837+0000","last_became_active":"2024-04-22T19:51:11.437749+0000","last_became_peered":"2024-04-22T19:51:11.437749+0000","last_unstale":"2024-04-22T20:48:58.304837+0000","last_undegraded":"2024-04-22T20:48:58.304837+0000","last_fullsized":"2024-04-22T20:48:58.304837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:58.304721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:58.304721+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.304721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:12:13.781455+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.303397+0000","last_change":"2024-04-22T20:48:56.303445+0000","last_active":"2024-04-22T20:48:56.303397+0000","last_peered":"2024-04-22T20:48:56.303397+0000","last_clean":"2024-04-22T20:48:56.303397+0000","last_became_active":"2024-04-22T19:51:30.529437+0000","last_became_peered":"2024-04-22T19:51:30.529437+0000","last_unstale":"2024-04-22T20:48:56.303397+0000","last_undegraded":"2024-04-22T20:48:56.303397+0000","last_fullsized":"2024-04-22T20:48:56.303397+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:56.303278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:56.303278+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.303278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:53:40.592791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"261'116","reported_seq":666,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.284844+0000","last_change":"2024-04-22T20:48:41.284912+0000","last_active":"2024-04-22T20:48:41.284844+0000","last_peered":"2024-04-22T20:48:41.284844+0000","last_clean":"2024-04-22T20:48:41.284844+0000","last_became_active":"2024-04-22T19:51:32.534280+0000","last_became_peered":"2024-04-22T19:51:32.534280+0000","last_unstale":"2024-04-22T20:48:41.284844+0000","last_undegraded":"2024-04-22T20:48:41.284844+0000","last_fullsized":"2024-04-22T20:48:41.284844+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'116","last_scrub_stamp":"2024-04-22T20:48:41.284749+0000","last_deep_scrub":"261'116","last_deep_scrub_stamp":"2024-04-22T20:48:41.284749+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.284749+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:15:26.736573+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":132,"num_write":94,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"261'536","reported_seq":2165,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.509077+0000","last_change":"2024-04-22T20:48:41.509108+0000","last_active":"2024-04-22T20:48:41.509077+0000","last_peered":"2024-04-22T20:48:41.509077+0000","last_clean":"2024-04-22T20:48:41.509077+0000","last_became_active":"2024-04-22T19:51:34.556673+0000","last_became_peered":"2024-04-22T19:51:34.556673+0000","last_unstale":"2024-04-22T20:48:41.509077+0000","last_undegraded":"2024-04-22T20:48:41.509077+0000","last_fullsized":"2024-04-22T20:48:41.509077+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'536","last_scrub_stamp":"2024-04-22T20:48:41.508985+0000","last_deep_scrub":"261'536","last_deep_scrub_stamp":"2024-04-22T20:48:41.508985+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.508985+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":500,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:10:21.553002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"261'152","reported_seq":651,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.460819+0000","last_change":"2024-04-22T20:48:34.460850+0000","last_active":"2024-04-22T20:48:34.460819+0000","last_peered":"2024-04-22T20:48:34.460819+0000","last_clean":"2024-04-22T20:48:34.460819+0000","last_became_active":"2024-04-22T19:51:36.575351+0000","last_became_peered":"2024-04-22T19:51:36.575351+0000","last_unstale":"2024-04-22T20:48:34.460819+0000","last_undegraded":"2024-04-22T20:48:34.460819+0000","last_fullsized":"2024-04-22T20:48:34.460819+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:34.460727+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:34.460727+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.460727+0000","objects_scrubbed":3,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:55:34.883011+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":80,"num_write":319,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"261'148","reported_seq":782,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.690078+0000","last_change":"2024-04-22T20:49:00.690116+0000","last_active":"2024-04-22T20:49:00.690078+0000","last_peered":"2024-04-22T20:49:00.690078+0000","last_clean":"2024-04-22T20:49:00.690078+0000","last_became_active":"2024-04-22T19:51:28.513940+0000","last_became_peered":"2024-04-22T19:51:28.513940+0000","last_unstale":"2024-04-22T20:49:00.690078+0000","last_undegraded":"2024-04-22T20:49:00.690078+0000","last_fullsized":"2024-04-22T20:49:00.690078+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'148","last_scrub_stamp":"2024-04-22T20:49:00.689987+0000","last_deep_scrub":"261'148","last_deep_scrub_stamp":"2024-04-22T20:49:00.689987+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.689987+0000","objects_scrubbed":2,"log_size":48,"log_dups_size":100,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:17:59.928019+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":631,"num_read_kb":550,"num_write":219,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3271,"num_omap_keys":14,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"261'357","reported_seq":1004,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.580271+0000","last_change":"2024-04-22T20:48:54.580328+0000","last_active":"2024-04-22T20:48:54.580271+0000","last_peered":"2024-04-22T20:48:54.580271+0000","last_clean":"2024-04-22T20:48:54.580271+0000","last_became_active":"2024-04-22T19:51:26.505842+0000","last_became_peered":"2024-04-22T19:51:26.505842+0000","last_unstale":"2024-04-22T20:48:54.580271+0000","last_undegraded":"2024-04-22T20:48:54.580271+0000","last_fullsized":"2024-04-22T20:48:54.580271+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'357","last_scrub_stamp":"2024-04-22T20:48:54.580170+0000","last_deep_scrub":"261'357","last_deep_scrub_stamp":"2024-04-22T20:48:54.580170+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.580170+0000","objects_scrubbed":9,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:58:46.289725+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308777,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":488,"num_read_kb":22191,"num_write":1115,"num_write_kb":33016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":527,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.583695+0000","last_change":"2024-04-22T20:48:49.583695+0000","last_active":"2024-04-22T20:48:49.583695+0000","last_peered":"2024-04-22T20:48:49.583695+0000","last_clean":"2024-04-22T20:48:49.583695+0000","last_became_active":"2024-04-22T19:51:12.250811+0000","last_became_peered":"2024-04-22T19:51:12.250811+0000","last_unstale":"2024-04-22T20:48:49.583695+0000","last_undegraded":"2024-04-22T20:48:49.583695+0000","last_fullsized":"2024-04-22T20:48:49.583695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:49.583572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:49.583572+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.583572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:02:08.812143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.523764+0000","last_change":"2024-04-22T20:48:35.523791+0000","last_active":"2024-04-22T20:48:35.523764+0000","last_peered":"2024-04-22T20:48:35.523764+0000","last_clean":"2024-04-22T20:48:35.523764+0000","last_became_active":"2024-04-22T19:51:30.525777+0000","last_became_peered":"2024-04-22T19:51:30.525777+0000","last_unstale":"2024-04-22T20:48:35.523764+0000","last_undegraded":"2024-04-22T20:48:35.523764+0000","last_fullsized":"2024-04-22T20:48:35.523764+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:35.523661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:35.523661+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.523661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:53:00.959964+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"261'186","reported_seq":762,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.648519+0000","last_change":"2024-04-22T20:48:39.648562+0000","last_active":"2024-04-22T20:48:39.648519+0000","last_peered":"2024-04-22T20:48:39.648519+0000","last_clean":"2024-04-22T20:48:39.648519+0000","last_became_active":"2024-04-22T19:51:33.248354+0000","last_became_peered":"2024-04-22T19:51:33.248354+0000","last_unstale":"2024-04-22T20:48:39.648519+0000","last_undegraded":"2024-04-22T20:48:39.648519+0000","last_fullsized":"2024-04-22T20:48:39.648519+0000","mapping_epoch":26,"log_start":"258'100","ondisk_log_start":"258'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'186","last_scrub_stamp":"2024-04-22T20:48:39.648420+0000","last_deep_scrub":"261'186","last_deep_scrub_stamp":"2024-04-22T20:48:39.648420+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.648420+0000","objects_scrubbed":7,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:06:13.449126+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":222,"num_read_kb":211,"num_write":148,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"261'548","reported_seq":4433,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.948010+0000","last_change":"2024-04-22T20:48:36.948010+0000","last_active":"2024-04-22T20:48:36.948010+0000","last_peered":"2024-04-22T20:48:36.948010+0000","last_clean":"2024-04-22T20:48:36.948010+0000","last_became_active":"2024-04-22T19:51:34.561206+0000","last_became_peered":"2024-04-22T19:51:34.561206+0000","last_unstale":"2024-04-22T20:48:36.948010+0000","last_undegraded":"2024-04-22T20:48:36.948010+0000","last_fullsized":"2024-04-22T20:48:36.948010+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'548","last_scrub_stamp":"2024-04-22T20:48:36.947822+0000","last_deep_scrub":"261'548","last_deep_scrub_stamp":"2024-04-22T20:48:36.947822+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.947822+0000","objects_scrubbed":1,"log_size":48,"log_dups_size":500,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:24:06.222760+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"261'185","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.541264+0000","last_change":"2024-04-22T20:48:33.541297+0000","last_active":"2024-04-22T20:48:33.541264+0000","last_peered":"2024-04-22T20:48:33.541264+0000","last_clean":"2024-04-22T20:48:33.541264+0000","last_became_active":"2024-04-22T19:51:36.569338+0000","last_became_peered":"2024-04-22T19:51:36.569338+0000","last_unstale":"2024-04-22T20:48:33.541264+0000","last_undegraded":"2024-04-22T20:48:33.541264+0000","last_fullsized":"2024-04-22T20:48:33.541264+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'185","last_scrub_stamp":"2024-04-22T20:48:33.541172+0000","last_deep_scrub":"261'185","last_deep_scrub_stamp":"2024-04-22T20:48:33.541172+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.541172+0000","objects_scrubbed":3,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:38:36.406576+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":109,"num_write":422,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"261'109","reported_seq":713,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.629698+0000","last_change":"2024-04-22T20:48:48.629746+0000","last_active":"2024-04-22T20:48:48.629698+0000","last_peered":"2024-04-22T20:48:48.629698+0000","last_clean":"2024-04-22T20:48:48.629698+0000","last_became_active":"2024-04-22T19:51:28.512337+0000","last_became_peered":"2024-04-22T19:51:28.512337+0000","last_unstale":"2024-04-22T20:48:48.629698+0000","last_undegraded":"2024-04-22T20:48:48.629698+0000","last_fullsized":"2024-04-22T20:48:48.629698+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'109","last_scrub_stamp":"2024-04-22T20:48:48.629602+0000","last_deep_scrub":"261'109","last_deep_scrub_stamp":"2024-04-22T20:48:48.629602+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.629602+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:59:28.312973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":434,"num_read_kb":362,"num_write":158,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'319","reported_seq":977,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.835821+0000","last_change":"2024-04-22T20:48:56.835865+0000","last_active":"2024-04-22T20:48:56.835821+0000","last_peered":"2024-04-22T20:48:56.835821+0000","last_clean":"2024-04-22T20:48:56.835821+0000","last_became_active":"2024-04-22T19:51:26.507804+0000","last_became_peered":"2024-04-22T19:51:26.507804+0000","last_unstale":"2024-04-22T20:48:56.835821+0000","last_undegraded":"2024-04-22T20:48:56.835821+0000","last_fullsized":"2024-04-22T20:48:56.835821+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'319","last_scrub_stamp":"2024-04-22T20:48:56.835711+0000","last_deep_scrub":"261'319","last_deep_scrub_stamp":"2024-04-22T20:48:56.835711+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.835711+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:14:07.605731+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":511,"num_read_kb":11012,"num_write":1234,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":527,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:48.628832+0000","last_change":"2024-04-22T20:48:48.628832+0000","last_active":"2024-04-22T20:48:48.628832+0000","last_peered":"2024-04-22T20:48:48.628832+0000","last_clean":"2024-04-22T20:48:48.628832+0000","last_became_active":"2024-04-22T19:51:11.437630+0000","last_became_peered":"2024-04-22T19:51:11.437630+0000","last_unstale":"2024-04-22T20:48:48.628832+0000","last_undegraded":"2024-04-22T20:48:48.628832+0000","last_fullsized":"2024-04-22T20:48:48.628832+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:48.628712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:48.628712+0000","last_clean_scrub_stamp":"2024-04-22T20:48:48.628712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:37:25.666135+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.639531+0000","last_change":"2024-04-22T20:48:57.639557+0000","last_active":"2024-04-22T20:48:57.639531+0000","last_peered":"2024-04-22T20:48:57.639531+0000","last_clean":"2024-04-22T20:48:57.639531+0000","last_became_active":"2024-04-22T19:51:30.526837+0000","last_became_peered":"2024-04-22T19:51:30.526837+0000","last_unstale":"2024-04-22T20:48:57.639531+0000","last_undegraded":"2024-04-22T20:48:57.639531+0000","last_fullsized":"2024-04-22T20:48:57.639531+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:57.639444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:57.639444+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.639444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:33:25.769033+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"261'551","reported_seq":1231,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:15.702420+0000","last_change":"2024-04-22T20:49:15.702420+0000","last_active":"2024-04-22T20:49:15.702420+0000","last_peered":"2024-04-22T20:49:15.702420+0000","last_clean":"2024-04-22T20:49:15.702420+0000","last_became_active":"2024-04-22T19:51:32.542344+0000","last_became_peered":"2024-04-22T19:51:32.542344+0000","last_unstale":"2024-04-22T20:49:15.702420+0000","last_undegraded":"2024-04-22T20:49:15.702420+0000","last_fullsized":"2024-04-22T20:49:15.702420+0000","mapping_epoch":26,"log_start":"261'500","ondisk_log_start":"261'500","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'551","last_scrub_stamp":"2024-04-22T20:49:15.702259+0000","last_deep_scrub":"261'551","last_deep_scrub_stamp":"2024-04-22T20:49:15.702259+0000","last_clean_scrub_stamp":"2024-04-22T20:49:15.702259+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":500,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:44:17.904007+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":97214,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":946,"num_read_kb":2097,"num_write":792,"num_write_kb":634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7364,"num_omap_keys":20,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.386032+0000","last_change":"2024-04-22T20:48:33.386077+0000","last_active":"2024-04-22T20:48:33.386032+0000","last_peered":"2024-04-22T20:48:33.386032+0000","last_clean":"2024-04-22T20:48:33.386032+0000","last_became_active":"2024-04-22T19:51:34.550951+0000","last_became_peered":"2024-04-22T19:51:34.550951+0000","last_unstale":"2024-04-22T20:48:33.386032+0000","last_undegraded":"2024-04-22T20:48:33.386032+0000","last_fullsized":"2024-04-22T20:48:33.386032+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.385902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.385902+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.385902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:42.635210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"261'118","reported_seq":632,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.587158+0000","last_change":"2024-04-22T20:48:40.587201+0000","last_active":"2024-04-22T20:48:40.587158+0000","last_peered":"2024-04-22T20:48:40.587158+0000","last_clean":"2024-04-22T20:48:40.587158+0000","last_became_active":"2024-04-22T19:51:36.572976+0000","last_became_peered":"2024-04-22T19:51:36.572976+0000","last_unstale":"2024-04-22T20:48:40.587158+0000","last_undegraded":"2024-04-22T20:48:40.587158+0000","last_fullsized":"2024-04-22T20:48:40.587158+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'118","last_scrub_stamp":"2024-04-22T20:48:40.587061+0000","last_deep_scrub":"261'118","last_deep_scrub_stamp":"2024-04-22T20:48:40.587061+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.587061+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:35:06.423672+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1164,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":120,"num_write":270,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"261'131","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.497001+0000","last_change":"2024-04-22T20:48:35.497051+0000","last_active":"2024-04-22T20:48:35.497001+0000","last_peered":"2024-04-22T20:48:35.497001+0000","last_clean":"2024-04-22T20:48:35.497001+0000","last_became_active":"2024-04-22T19:51:36.574238+0000","last_became_peered":"2024-04-22T19:51:36.574238+0000","last_unstale":"2024-04-22T20:48:35.497001+0000","last_undegraded":"2024-04-22T20:48:35.497001+0000","last_fullsized":"2024-04-22T20:48:35.497001+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:35.496901+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:35.496901+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.496901+0000","objects_scrubbed":1,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:05:26.775249+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":193,"num_read_kb":192,"num_write":291,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"261'124","reported_seq":749,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.322836+0000","last_change":"2024-04-22T20:48:45.322877+0000","last_active":"2024-04-22T20:48:45.322836+0000","last_peered":"2024-04-22T20:48:45.322836+0000","last_clean":"2024-04-22T20:48:45.322836+0000","last_became_active":"2024-04-22T19:51:28.515334+0000","last_became_peered":"2024-04-22T19:51:28.515334+0000","last_unstale":"2024-04-22T20:48:45.322836+0000","last_undegraded":"2024-04-22T20:48:45.322836+0000","last_fullsized":"2024-04-22T20:48:45.322836+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'124","last_scrub_stamp":"2024-04-22T20:48:45.322739+0000","last_deep_scrub":"261'124","last_deep_scrub_stamp":"2024-04-22T20:48:45.322739+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.322739+0000","objects_scrubbed":1,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:29:20.200394+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":424,"num_write":184,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"261'324","reported_seq":1046,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.259253+0000","last_change":"2024-04-22T20:49:01.259314+0000","last_active":"2024-04-22T20:49:01.259253+0000","last_peered":"2024-04-22T20:49:01.259253+0000","last_clean":"2024-04-22T20:49:01.259253+0000","last_became_active":"2024-04-22T19:51:26.506150+0000","last_became_peered":"2024-04-22T19:51:26.506150+0000","last_unstale":"2024-04-22T20:49:01.259253+0000","last_undegraded":"2024-04-22T20:49:01.259253+0000","last_fullsized":"2024-04-22T20:49:01.259253+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:49:01.259157+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:49:01.259157+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.259157+0000","objects_scrubbed":11,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:51:26.309359+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097285,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":716,"num_read_kb":35402,"num_write":1112,"num_write_kb":35897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":503,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.602028+0000","last_change":"2024-04-22T20:48:36.602053+0000","last_active":"2024-04-22T20:48:36.602028+0000","last_peered":"2024-04-22T20:48:36.602028+0000","last_clean":"2024-04-22T20:48:36.602028+0000","last_became_active":"2024-04-22T19:51:30.522903+0000","last_became_peered":"2024-04-22T19:51:30.522903+0000","last_unstale":"2024-04-22T20:48:36.602028+0000","last_undegraded":"2024-04-22T20:48:36.602028+0000","last_fullsized":"2024-04-22T20:48:36.602028+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:36.601934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:36.601934+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.601934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:58.284091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"261'301","reported_seq":894,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.465902+0000","last_change":"2024-04-22T20:48:51.465949+0000","last_active":"2024-04-22T20:48:51.465902+0000","last_peered":"2024-04-22T20:48:51.465902+0000","last_clean":"2024-04-22T20:48:51.465902+0000","last_became_active":"2024-04-22T19:51:32.539631+0000","last_became_peered":"2024-04-22T19:51:32.539631+0000","last_unstale":"2024-04-22T20:48:51.465902+0000","last_undegraded":"2024-04-22T20:48:51.465902+0000","last_fullsized":"2024-04-22T20:48:51.465902+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'301","last_scrub_stamp":"2024-04-22T20:48:51.465803+0000","last_deep_scrub":"261'301","last_deep_scrub_stamp":"2024-04-22T20:48:51.465803+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.465803+0000","objects_scrubbed":5,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:25:40.094839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44228,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":495,"num_read_kb":1138,"num_write":404,"num_write_kb":319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.536890+0000","last_change":"2024-04-22T20:48:42.536915+0000","last_active":"2024-04-22T20:48:42.536890+0000","last_peered":"2024-04-22T20:48:42.536890+0000","last_clean":"2024-04-22T20:48:42.536890+0000","last_became_active":"2024-04-22T19:51:34.556650+0000","last_became_peered":"2024-04-22T19:51:34.556650+0000","last_unstale":"2024-04-22T20:48:42.536890+0000","last_undegraded":"2024-04-22T20:48:42.536890+0000","last_fullsized":"2024-04-22T20:48:42.536890+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:42.536759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:42.536759+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.536759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:46:57.037831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"261'99","reported_seq":609,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.450136+0000","last_change":"2024-04-22T20:48:38.450136+0000","last_active":"2024-04-22T20:48:38.450136+0000","last_peered":"2024-04-22T20:48:38.450136+0000","last_clean":"2024-04-22T20:48:38.450136+0000","last_became_active":"2024-04-22T19:51:36.566980+0000","last_became_peered":"2024-04-22T19:51:36.566980+0000","last_unstale":"2024-04-22T20:48:38.450136+0000","last_undegraded":"2024-04-22T20:48:38.450136+0000","last_fullsized":"2024-04-22T20:48:38.450136+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'99","last_scrub_stamp":"2024-04-22T20:48:38.449980+0000","last_deep_scrub":"261'99","last_deep_scrub_stamp":"2024-04-22T20:48:38.449980+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.449980+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:31:45.431617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":86,"num_write":235,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"261'130","reported_seq":743,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.716009+0000","last_change":"2024-04-22T20:48:42.716036+0000","last_active":"2024-04-22T20:48:42.716009+0000","last_peered":"2024-04-22T20:48:42.716009+0000","last_clean":"2024-04-22T20:48:42.716009+0000","last_became_active":"2024-04-22T19:51:28.517675+0000","last_became_peered":"2024-04-22T19:51:28.517675+0000","last_unstale":"2024-04-22T20:48:42.716009+0000","last_undegraded":"2024-04-22T20:48:42.716009+0000","last_fullsized":"2024-04-22T20:48:42.716009+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'130","last_scrub_stamp":"2024-04-22T20:48:42.715887+0000","last_deep_scrub":"261'130","last_deep_scrub_stamp":"2024-04-22T20:48:42.715887+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.715887+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:12:53.780947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":543,"num_read_kb":446,"num_write":195,"num_write_kb":100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"261'341","reported_seq":1051,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:10.565004+0000","last_change":"2024-04-22T20:49:10.565004+0000","last_active":"2024-04-22T20:49:10.565004+0000","last_peered":"2024-04-22T20:49:10.565004+0000","last_clean":"2024-04-22T20:49:10.565004+0000","last_became_active":"2024-04-22T19:51:26.501897+0000","last_became_peered":"2024-04-22T19:51:26.501897+0000","last_unstale":"2024-04-22T20:49:10.565004+0000","last_undegraded":"2024-04-22T20:49:10.565004+0000","last_fullsized":"2024-04-22T20:49:10.565004+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'341","last_scrub_stamp":"2024-04-22T20:49:10.564792+0000","last_deep_scrub":"261'341","last_deep_scrub_stamp":"2024-04-22T20:49:10.564792+0000","last_clean_scrub_stamp":"2024-04-22T20:49:10.564792+0000","objects_scrubbed":12,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:20:09.024556+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3673455,"num_objects":12,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":613,"num_read_kb":41436,"num_write":1146,"num_write_kb":29344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":512,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.413044+0000","last_change":"2024-04-22T20:49:04.413074+0000","last_active":"2024-04-22T20:49:04.413044+0000","last_peered":"2024-04-22T20:49:04.413044+0000","last_clean":"2024-04-22T20:49:04.413044+0000","last_became_active":"2024-04-22T19:51:30.524388+0000","last_became_peered":"2024-04-22T19:51:30.524388+0000","last_unstale":"2024-04-22T20:49:04.413044+0000","last_undegraded":"2024-04-22T20:49:04.413044+0000","last_fullsized":"2024-04-22T20:49:04.413044+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:49:04.412948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:49:04.412948+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.412948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:33:44.545552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"261'52","reported_seq":565,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.671890+0000","last_change":"2024-04-22T20:49:03.671919+0000","last_active":"2024-04-22T20:49:03.671890+0000","last_peered":"2024-04-22T20:49:03.671890+0000","last_clean":"2024-04-22T20:49:03.671890+0000","last_became_active":"2024-04-22T19:51:32.539260+0000","last_became_peered":"2024-04-22T19:51:32.539260+0000","last_unstale":"2024-04-22T20:49:03.671890+0000","last_undegraded":"2024-04-22T20:49:03.671890+0000","last_fullsized":"2024-04-22T20:49:03.671890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'52","last_scrub_stamp":"2024-04-22T20:49:03.671816+0000","last_deep_scrub":"261'52","last_deep_scrub_stamp":"2024-04-22T20:49:03.671816+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.671816+0000","objects_scrubbed":3,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:50:59.506345+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":36,"num_write":28,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":588,"num_omap_keys":4,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.488848+0000","last_change":"2024-04-22T20:48:47.488871+0000","last_active":"2024-04-22T20:48:47.488848+0000","last_peered":"2024-04-22T20:48:47.488848+0000","last_clean":"2024-04-22T20:48:47.488848+0000","last_became_active":"2024-04-22T19:51:34.551294+0000","last_became_peered":"2024-04-22T19:51:34.551294+0000","last_unstale":"2024-04-22T20:48:47.488848+0000","last_undegraded":"2024-04-22T20:48:47.488848+0000","last_fullsized":"2024-04-22T20:48:47.488848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.488766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.488766+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.488766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:13:45.930564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"261'202","reported_seq":729,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.470421+0000","last_change":"2024-04-22T20:48:36.470421+0000","last_active":"2024-04-22T20:48:36.470421+0000","last_peered":"2024-04-22T20:48:36.470421+0000","last_clean":"2024-04-22T20:48:36.470421+0000","last_became_active":"2024-04-22T19:51:36.573936+0000","last_became_peered":"2024-04-22T19:51:36.573936+0000","last_unstale":"2024-04-22T20:48:36.470421+0000","last_undegraded":"2024-04-22T20:48:36.470421+0000","last_fullsized":"2024-04-22T20:48:36.470421+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'202","last_scrub_stamp":"2024-04-22T20:48:36.470271+0000","last_deep_scrub":"261'202","last_deep_scrub_stamp":"2024-04-22T20:48:36.470271+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.470271+0000","objects_scrubbed":1,"log_size":102,"log_dups_size":100,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:05:08.510777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":284,"num_write":451,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"261'231","reported_seq":926,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:01.411158+0000","last_change":"2024-04-22T20:49:01.411188+0000","last_active":"2024-04-22T20:49:01.411158+0000","last_peered":"2024-04-22T20:49:01.411158+0000","last_clean":"2024-04-22T20:49:01.411158+0000","last_became_active":"2024-04-22T19:51:28.508307+0000","last_became_peered":"2024-04-22T19:51:28.508307+0000","last_unstale":"2024-04-22T20:49:01.411158+0000","last_undegraded":"2024-04-22T20:49:01.411158+0000","last_fullsized":"2024-04-22T20:49:01.411158+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'231","last_scrub_stamp":"2024-04-22T20:49:01.411072+0000","last_deep_scrub":"261'231","last_deep_scrub_stamp":"2024-04-22T20:49:01.411072+0000","last_clean_scrub_stamp":"2024-04-22T20:49:01.411072+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":200,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:15:33.901303+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":959,"num_read_kb":784,"num_write":348,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"261'323","reported_seq":964,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.793110+0000","last_change":"2024-04-22T20:48:59.793110+0000","last_active":"2024-04-22T20:48:59.793110+0000","last_peered":"2024-04-22T20:48:59.793110+0000","last_clean":"2024-04-22T20:48:59.793110+0000","last_became_active":"2024-04-22T19:51:26.501477+0000","last_became_peered":"2024-04-22T19:51:26.501477+0000","last_unstale":"2024-04-22T20:48:59.793110+0000","last_undegraded":"2024-04-22T20:48:59.793110+0000","last_fullsized":"2024-04-22T20:48:59.793110+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'323","last_scrub_stamp":"2024-04-22T20:48:59.792943+0000","last_deep_scrub":"261'323","last_deep_scrub_stamp":"2024-04-22T20:48:59.792943+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.792943+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:21.956398+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":477,"num_read_kb":46522,"num_write":1280,"num_write_kb":35021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.361909+0000","last_change":"2024-04-22T20:48:54.361936+0000","last_active":"2024-04-22T20:48:54.361909+0000","last_peered":"2024-04-22T20:48:54.361909+0000","last_clean":"2024-04-22T20:48:54.361909+0000","last_became_active":"2024-04-22T19:51:30.527076+0000","last_became_peered":"2024-04-22T19:51:30.527076+0000","last_unstale":"2024-04-22T20:48:54.361909+0000","last_undegraded":"2024-04-22T20:48:54.361909+0000","last_fullsized":"2024-04-22T20:48:54.361909+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:54.361806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:54.361806+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.361806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:01:55.484727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"261'307","reported_seq":916,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:49.485477+0000","last_change":"2024-04-22T20:48:49.485517+0000","last_active":"2024-04-22T20:48:49.485477+0000","last_peered":"2024-04-22T20:48:49.485477+0000","last_clean":"2024-04-22T20:48:49.485477+0000","last_became_active":"2024-04-22T19:51:32.542262+0000","last_became_peered":"2024-04-22T19:51:32.542262+0000","last_unstale":"2024-04-22T20:48:49.485477+0000","last_undegraded":"2024-04-22T20:48:49.485477+0000","last_fullsized":"2024-04-22T20:48:49.485477+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'307","last_scrub_stamp":"2024-04-22T20:48:49.485363+0000","last_deep_scrub":"261'307","last_deep_scrub_stamp":"2024-04-22T20:48:49.485363+0000","last_clean_scrub_stamp":"2024-04-22T20:48:49.485363+0000","objects_scrubbed":5,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:26:58.383965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46510,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":491,"num_read_kb":1038,"num_write":384,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.651253+0000","last_change":"2024-04-22T20:48:38.651275+0000","last_active":"2024-04-22T20:48:38.651253+0000","last_peered":"2024-04-22T20:48:38.651253+0000","last_clean":"2024-04-22T20:48:38.651253+0000","last_became_active":"2024-04-22T19:51:34.553691+0000","last_became_peered":"2024-04-22T19:51:34.553691+0000","last_unstale":"2024-04-22T20:48:38.651253+0000","last_undegraded":"2024-04-22T20:48:38.651253+0000","last_fullsized":"2024-04-22T20:48:38.651253+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:38.651145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:38.651145+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.651145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:39:15.734170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"261'137","reported_seq":628,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.859930+0000","last_change":"2024-04-22T20:48:28.859930+0000","last_active":"2024-04-22T20:48:28.859930+0000","last_peered":"2024-04-22T20:48:28.859930+0000","last_clean":"2024-04-22T20:48:28.859930+0000","last_became_active":"2024-04-22T19:51:36.571290+0000","last_became_peered":"2024-04-22T19:51:36.571290+0000","last_unstale":"2024-04-22T20:48:28.859930+0000","last_undegraded":"2024-04-22T20:48:28.859930+0000","last_fullsized":"2024-04-22T20:48:28.859930+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'137","last_scrub_stamp":"2024-04-22T20:48:28.859747+0000","last_deep_scrub":"261'137","last_deep_scrub_stamp":"2024-04-22T20:48:28.859747+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.859747+0000","objects_scrubbed":2,"log_size":37,"log_dups_size":100,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:02:28.549243+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":440,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":111,"num_write":334,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"261'185","reported_seq":863,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.384703+0000","last_change":"2024-04-22T20:49:03.384728+0000","last_active":"2024-04-22T20:49:03.384703+0000","last_peered":"2024-04-22T20:49:03.384703+0000","last_clean":"2024-04-22T20:49:03.384703+0000","last_became_active":"2024-04-22T19:51:28.506615+0000","last_became_peered":"2024-04-22T19:51:28.506615+0000","last_unstale":"2024-04-22T20:49:03.384703+0000","last_undegraded":"2024-04-22T20:49:03.384703+0000","last_fullsized":"2024-04-22T20:49:03.384703+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'185","last_scrub_stamp":"2024-04-22T20:49:03.384610+0000","last_deep_scrub":"261'185","last_deep_scrub_stamp":"2024-04-22T20:49:03.384610+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.384610+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:20:25.431536+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":757,"num_read_kb":623,"num_write":269,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'338","reported_seq":989,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.686193+0000","last_change":"2024-04-22T20:49:11.686193+0000","last_active":"2024-04-22T20:49:11.686193+0000","last_peered":"2024-04-22T20:49:11.686193+0000","last_clean":"2024-04-22T20:49:11.686193+0000","last_became_active":"2024-04-22T19:51:26.502938+0000","last_became_peered":"2024-04-22T19:51:26.502938+0000","last_unstale":"2024-04-22T20:49:11.686193+0000","last_undegraded":"2024-04-22T20:49:11.686193+0000","last_fullsized":"2024-04-22T20:49:11.686193+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'338","last_scrub_stamp":"2024-04-22T20:49:11.686003+0000","last_deep_scrub":"261'338","last_deep_scrub_stamp":"2024-04-22T20:49:11.686003+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.686003+0000","objects_scrubbed":4,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:55:46.874873+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194317,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":478,"num_read_kb":28738,"num_write":1269,"num_write_kb":45425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:32.504136+0000","last_change":"2024-04-22T20:48:32.504160+0000","last_active":"2024-04-22T20:48:32.504136+0000","last_peered":"2024-04-22T20:48:32.504136+0000","last_clean":"2024-04-22T20:48:32.504136+0000","last_became_active":"2024-04-22T19:51:30.526994+0000","last_became_peered":"2024-04-22T19:51:30.526994+0000","last_unstale":"2024-04-22T20:48:32.504136+0000","last_undegraded":"2024-04-22T20:48:32.504136+0000","last_fullsized":"2024-04-22T20:48:32.504136+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:32.504047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:32.504047+0000","last_clean_scrub_stamp":"2024-04-22T20:48:32.504047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:39.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"261'255","reported_seq":835,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.460418+0000","last_change":"2024-04-22T20:49:00.460454+0000","last_active":"2024-04-22T20:49:00.460418+0000","last_peered":"2024-04-22T20:49:00.460418+0000","last_clean":"2024-04-22T20:49:00.460418+0000","last_became_active":"2024-04-22T19:51:32.537875+0000","last_became_peered":"2024-04-22T19:51:32.537875+0000","last_unstale":"2024-04-22T20:49:00.460418+0000","last_undegraded":"2024-04-22T20:49:00.460418+0000","last_fullsized":"2024-04-22T20:49:00.460418+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'255","last_scrub_stamp":"2024-04-22T20:49:00.460334+0000","last_deep_scrub":"261'255","last_deep_scrub_stamp":"2024-04-22T20:49:00.460334+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.460334+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:07:02.585609+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45326,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":422,"num_read_kb":961,"num_write":338,"num_write_kb":268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.635537+0000","last_change":"2024-04-22T20:48:46.635559+0000","last_active":"2024-04-22T20:48:46.635537+0000","last_peered":"2024-04-22T20:48:46.635537+0000","last_clean":"2024-04-22T20:48:46.635537+0000","last_became_active":"2024-04-22T19:51:34.553954+0000","last_became_peered":"2024-04-22T19:51:34.553954+0000","last_unstale":"2024-04-22T20:48:46.635537+0000","last_undegraded":"2024-04-22T20:48:46.635537+0000","last_fullsized":"2024-04-22T20:48:46.635537+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:46.635459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:46.635459+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.635459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:54:27.052935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"261'192","reported_seq":839,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.551497+0000","last_change":"2024-04-22T20:48:43.551521+0000","last_active":"2024-04-22T20:48:43.551497+0000","last_peered":"2024-04-22T20:48:43.551497+0000","last_clean":"2024-04-22T20:48:43.551497+0000","last_became_active":"2024-04-22T19:51:28.514593+0000","last_became_peered":"2024-04-22T19:51:28.514593+0000","last_unstale":"2024-04-22T20:48:43.551497+0000","last_undegraded":"2024-04-22T20:48:43.551497+0000","last_fullsized":"2024-04-22T20:48:43.551497+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'192","last_scrub_stamp":"2024-04-22T20:48:43.551394+0000","last_deep_scrub":"261'192","last_deep_scrub_stamp":"2024-04-22T20:48:43.551394+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.551394+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:06:00.844687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":788,"num_read_kb":638,"num_write":274,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"261'342","reported_seq":1014,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:15.359341+0000","last_change":"2024-04-22T20:49:15.359341+0000","last_active":"2024-04-22T20:49:15.359341+0000","last_peered":"2024-04-22T20:49:15.359341+0000","last_clean":"2024-04-22T20:49:15.359341+0000","last_became_active":"2024-04-22T19:51:26.501812+0000","last_became_peered":"2024-04-22T19:51:26.501812+0000","last_unstale":"2024-04-22T20:49:15.359341+0000","last_undegraded":"2024-04-22T20:49:15.359341+0000","last_fullsized":"2024-04-22T20:49:15.359341+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'342","last_scrub_stamp":"2024-04-22T20:49:15.359113+0000","last_deep_scrub":"261'342","last_deep_scrub_stamp":"2024-04-22T20:49:15.359113+0000","last_clean_scrub_stamp":"2024-04-22T20:49:15.359113+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:04:28.788196+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857304,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":552,"num_read_kb":55834,"num_write":1162,"num_write_kb":35884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":510,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.630690+0000","last_change":"2024-04-22T20:48:53.630690+0000","last_active":"2024-04-22T20:48:53.630690+0000","last_peered":"2024-04-22T20:48:53.630690+0000","last_clean":"2024-04-22T20:48:53.630690+0000","last_became_active":"2024-04-22T19:51:30.525450+0000","last_became_peered":"2024-04-22T19:51:30.525450+0000","last_unstale":"2024-04-22T20:48:53.630690+0000","last_undegraded":"2024-04-22T20:48:53.630690+0000","last_fullsized":"2024-04-22T20:48:53.630690+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:53.630576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:53.630576+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.630576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:07:46.156214+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"261'489","reported_seq":1127,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.643184+0000","last_change":"2024-04-22T20:48:37.643184+0000","last_active":"2024-04-22T20:48:37.643184+0000","last_peered":"2024-04-22T20:48:37.643184+0000","last_clean":"2024-04-22T20:48:37.643184+0000","last_became_active":"2024-04-22T19:51:32.540755+0000","last_became_peered":"2024-04-22T19:51:32.540755+0000","last_unstale":"2024-04-22T20:48:37.643184+0000","last_undegraded":"2024-04-22T20:48:37.643184+0000","last_fullsized":"2024-04-22T20:48:37.643184+0000","mapping_epoch":26,"log_start":"261'400","ondisk_log_start":"261'400","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'489","last_scrub_stamp":"2024-04-22T20:48:37.643033+0000","last_deep_scrub":"261'489","last_deep_scrub_stamp":"2024-04-22T20:48:37.643033+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.643033+0000","objects_scrubbed":7,"log_size":89,"log_dups_size":400,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:20:59.082062+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93788,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":875,"num_read_kb":2058,"num_write":756,"num_write_kb":671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":23904,"num_omap_keys":44,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":492,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.952860+0000","last_change":"2024-04-22T20:48:37.952860+0000","last_active":"2024-04-22T20:48:37.952860+0000","last_peered":"2024-04-22T20:48:37.952860+0000","last_clean":"2024-04-22T20:48:37.952860+0000","last_became_active":"2024-04-22T19:51:34.553363+0000","last_became_peered":"2024-04-22T19:51:34.553363+0000","last_unstale":"2024-04-22T20:48:37.952860+0000","last_undegraded":"2024-04-22T20:48:37.952860+0000","last_fullsized":"2024-04-22T20:48:37.952860+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:37.952749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:37.952749+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.952749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:37:14.410865+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"261'162","reported_seq":685,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.484277+0000","last_change":"2024-04-22T20:48:35.484310+0000","last_active":"2024-04-22T20:48:35.484277+0000","last_peered":"2024-04-22T20:48:35.484277+0000","last_clean":"2024-04-22T20:48:35.484277+0000","last_became_active":"2024-04-22T19:51:36.568373+0000","last_became_peered":"2024-04-22T19:51:36.568373+0000","last_unstale":"2024-04-22T20:48:35.484277+0000","last_undegraded":"2024-04-22T20:48:35.484277+0000","last_fullsized":"2024-04-22T20:48:35.484277+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'162","last_scrub_stamp":"2024-04-22T20:48:35.484193+0000","last_deep_scrub":"261'162","last_deep_scrub_stamp":"2024-04-22T20:48:35.484193+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.484193+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":100,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:59:33.641598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":213,"num_write":373,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"261'100","reported_seq":695,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.619828+0000","last_change":"2024-04-22T20:49:02.619828+0000","last_active":"2024-04-22T20:49:02.619828+0000","last_peered":"2024-04-22T20:49:02.619828+0000","last_clean":"2024-04-22T20:49:02.619828+0000","last_became_active":"2024-04-22T19:51:28.510629+0000","last_became_peered":"2024-04-22T19:51:28.510629+0000","last_unstale":"2024-04-22T20:49:02.619828+0000","last_undegraded":"2024-04-22T20:49:02.619828+0000","last_fullsized":"2024-04-22T20:49:02.619828+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'100","last_scrub_stamp":"2024-04-22T20:49:02.619712+0000","last_deep_scrub":"261'100","last_deep_scrub_stamp":"2024-04-22T20:49:02.619712+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.619712+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:29:26.099759+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":435,"num_read_kb":357,"num_write":161,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"261'305","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.605390+0000","last_change":"2024-04-22T20:48:52.605390+0000","last_active":"2024-04-22T20:48:52.605390+0000","last_peered":"2024-04-22T20:48:52.605390+0000","last_clean":"2024-04-22T20:48:52.605390+0000","last_became_active":"2024-04-22T19:51:26.497165+0000","last_became_peered":"2024-04-22T19:51:26.497165+0000","last_unstale":"2024-04-22T20:48:52.605390+0000","last_undegraded":"2024-04-22T20:48:52.605390+0000","last_fullsized":"2024-04-22T20:48:52.605390+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'305","last_scrub_stamp":"2024-04-22T20:48:52.605144+0000","last_deep_scrub":"261'305","last_deep_scrub_stamp":"2024-04-22T20:48:52.605144+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.605144+0000","objects_scrubbed":9,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:49:37.401989+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":662323,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":493,"num_read_kb":39207,"num_write":1122,"num_write_kb":42297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":508,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:33.479719+0000","last_change":"2024-04-22T20:48:33.479741+0000","last_active":"2024-04-22T20:48:33.479719+0000","last_peered":"2024-04-22T20:48:33.479719+0000","last_clean":"2024-04-22T20:48:33.479719+0000","last_became_active":"2024-04-22T19:51:30.526597+0000","last_became_peered":"2024-04-22T19:51:30.526597+0000","last_unstale":"2024-04-22T20:48:33.479719+0000","last_undegraded":"2024-04-22T20:48:33.479719+0000","last_fullsized":"2024-04-22T20:48:33.479719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:33.479622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:33.479622+0000","last_clean_scrub_stamp":"2024-04-22T20:48:33.479622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:35:51.805769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"261'319","reported_seq":916,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.463525+0000","last_change":"2024-04-22T20:48:58.463525+0000","last_active":"2024-04-22T20:48:58.463525+0000","last_peered":"2024-04-22T20:48:58.463525+0000","last_clean":"2024-04-22T20:48:58.463525+0000","last_became_active":"2024-04-22T19:51:32.537416+0000","last_became_peered":"2024-04-22T19:51:32.537416+0000","last_unstale":"2024-04-22T20:48:58.463525+0000","last_undegraded":"2024-04-22T20:48:58.463525+0000","last_fullsized":"2024-04-22T20:48:58.463525+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'319","last_scrub_stamp":"2024-04-22T20:48:58.463340+0000","last_deep_scrub":"261'319","last_deep_scrub_stamp":"2024-04-22T20:48:58.463340+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.463340+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:11:45.125481+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43451,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":521,"num_read_kb":1119,"num_write":436,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":494,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:45.694176+0000","last_change":"2024-04-22T20:48:45.694176+0000","last_active":"2024-04-22T20:48:45.694176+0000","last_peered":"2024-04-22T20:48:45.694176+0000","last_clean":"2024-04-22T20:48:45.694176+0000","last_became_active":"2024-04-22T19:51:34.561096+0000","last_became_peered":"2024-04-22T19:51:34.561096+0000","last_unstale":"2024-04-22T20:48:45.694176+0000","last_undegraded":"2024-04-22T20:48:45.694176+0000","last_fullsized":"2024-04-22T20:48:45.694176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:45.694065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:45.694065+0000","last_clean_scrub_stamp":"2024-04-22T20:48:45.694065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:07:56.257157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"261'166","reported_seq":675,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.608939+0000","last_change":"2024-04-22T20:48:56.608983+0000","last_active":"2024-04-22T20:48:56.608939+0000","last_peered":"2024-04-22T20:48:56.608939+0000","last_clean":"2024-04-22T20:48:56.608939+0000","last_became_active":"2024-04-22T19:51:36.571565+0000","last_became_peered":"2024-04-22T19:51:36.571565+0000","last_unstale":"2024-04-22T20:48:56.608939+0000","last_undegraded":"2024-04-22T20:48:56.608939+0000","last_fullsized":"2024-04-22T20:48:56.608939+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'166","last_scrub_stamp":"2024-04-22T20:48:56.608839+0000","last_deep_scrub":"261'166","last_deep_scrub_stamp":"2024-04-22T20:48:56.608839+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.608839+0000","objects_scrubbed":5,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:31:59.748806+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":181,"num_read_kb":180,"num_write":473,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"261'44","reported_seq":619,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.377542+0000","last_change":"2024-04-22T20:49:05.377569+0000","last_active":"2024-04-22T20:49:05.377542+0000","last_peered":"2024-04-22T20:49:05.377542+0000","last_clean":"2024-04-22T20:49:05.377542+0000","last_became_active":"2024-04-22T19:51:28.506835+0000","last_became_peered":"2024-04-22T19:51:28.506835+0000","last_unstale":"2024-04-22T20:49:05.377542+0000","last_undegraded":"2024-04-22T20:49:05.377542+0000","last_fullsized":"2024-04-22T20:49:05.377542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'44","last_scrub_stamp":"2024-04-22T20:49:05.377442+0000","last_deep_scrub":"261'44","last_deep_scrub_stamp":"2024-04-22T20:49:05.377442+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.377442+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:54:43.348262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":202,"num_read_kb":174,"num_write":58,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"261'377","reported_seq":1014,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:24.443059+0000","last_change":"2024-04-22T20:48:24.443059+0000","last_active":"2024-04-22T20:48:24.443059+0000","last_peered":"2024-04-22T20:48:24.443059+0000","last_clean":"2024-04-22T20:48:24.443059+0000","last_became_active":"2024-04-22T19:52:04.617820+0000","last_became_peered":"2024-04-22T19:52:04.617820+0000","last_unstale":"2024-04-22T20:48:24.443059+0000","last_undegraded":"2024-04-22T20:48:24.443059+0000","last_fullsized":"2024-04-22T20:48:24.443059+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'377","last_scrub_stamp":"2024-04-22T20:48:24.442876+0000","last_deep_scrub":"261'377","last_deep_scrub_stamp":"2024-04-22T20:48:24.442876+0000","last_clean_scrub_stamp":"2024-04-22T20:48:24.442876+0000","objects_scrubbed":13,"log_size":77,"log_dups_size":300,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:42:38.642906+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14463569,"num_objects":13,"num_object_clones":0,"num_object_copies":39,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":535,"num_read_kb":31540,"num_write":1473,"num_write_kb":43247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,5],"acting":[2,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:54.299779+0000","last_change":"2024-04-22T20:48:54.299779+0000","last_active":"2024-04-22T20:48:54.299779+0000","last_peered":"2024-04-22T20:48:54.299779+0000","last_clean":"2024-04-22T20:48:54.299779+0000","last_became_active":"2024-04-22T19:51:30.529322+0000","last_became_peered":"2024-04-22T19:51:30.529322+0000","last_unstale":"2024-04-22T20:48:54.299779+0000","last_undegraded":"2024-04-22T20:48:54.299779+0000","last_fullsized":"2024-04-22T20:48:54.299779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:54.299664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:54.299664+0000","last_clean_scrub_stamp":"2024-04-22T20:48:54.299664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:25.063326+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"261'276","reported_seq":872,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.466258+0000","last_change":"2024-04-22T20:48:47.466295+0000","last_active":"2024-04-22T20:48:47.466258+0000","last_peered":"2024-04-22T20:48:47.466258+0000","last_clean":"2024-04-22T20:48:47.466258+0000","last_became_active":"2024-04-22T19:51:33.248293+0000","last_became_peered":"2024-04-22T19:51:33.248293+0000","last_unstale":"2024-04-22T20:48:47.466258+0000","last_undegraded":"2024-04-22T20:48:47.466258+0000","last_fullsized":"2024-04-22T20:48:47.466258+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'276","last_scrub_stamp":"2024-04-22T20:48:47.466165+0000","last_deep_scrub":"261'276","last_deep_scrub_stamp":"2024-04-22T20:48:47.466165+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.466165+0000","objects_scrubbed":5,"log_size":76,"log_dups_size":200,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:12:37.324192+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56069,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1056,"num_write":364,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.684299+0000","last_change":"2024-04-22T20:48:43.684299+0000","last_active":"2024-04-22T20:48:43.684299+0000","last_peered":"2024-04-22T20:48:43.684299+0000","last_clean":"2024-04-22T20:48:43.684299+0000","last_became_active":"2024-04-22T19:51:34.560955+0000","last_became_peered":"2024-04-22T19:51:34.560955+0000","last_unstale":"2024-04-22T20:48:43.684299+0000","last_undegraded":"2024-04-22T20:48:43.684299+0000","last_fullsized":"2024-04-22T20:48:43.684299+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:43.684147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:43.684147+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.684147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:05:24.733318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"261'122","reported_seq":614,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.995578+0000","last_change":"2024-04-22T20:48:34.995676+0000","last_active":"2024-04-22T20:48:34.995578+0000","last_peered":"2024-04-22T20:48:34.995578+0000","last_clean":"2024-04-22T20:48:34.995578+0000","last_became_active":"2024-04-22T19:51:36.567139+0000","last_became_peered":"2024-04-22T19:51:36.567139+0000","last_unstale":"2024-04-22T20:48:34.995578+0000","last_undegraded":"2024-04-22T20:48:34.995578+0000","last_fullsized":"2024-04-22T20:48:34.995578+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'122","last_scrub_stamp":"2024-04-22T20:48:34.995457+0000","last_deep_scrub":"261'122","last_deep_scrub_stamp":"2024-04-22T20:48:34.995457+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.995457+0000","objects_scrubbed":3,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:31:14.860340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":383,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":92,"num_write":268,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"261'106","reported_seq":698,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.592002+0000","last_change":"2024-04-22T20:48:43.592002+0000","last_active":"2024-04-22T20:48:43.592002+0000","last_peered":"2024-04-22T20:48:43.592002+0000","last_clean":"2024-04-22T20:48:43.592002+0000","last_became_active":"2024-04-22T19:51:28.512642+0000","last_became_peered":"2024-04-22T19:51:28.512642+0000","last_unstale":"2024-04-22T20:48:43.592002+0000","last_undegraded":"2024-04-22T20:48:43.592002+0000","last_fullsized":"2024-04-22T20:48:43.592002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'106","last_scrub_stamp":"2024-04-22T20:48:43.591857+0000","last_deep_scrub":"261'106","last_deep_scrub_stamp":"2024-04-22T20:48:43.591857+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.591857+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:12:33.205148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":427,"num_read_kb":349,"num_write":169,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"261'328","reported_seq":948,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:14.740039+0000","last_change":"2024-04-22T20:49:14.740117+0000","last_active":"2024-04-22T20:49:14.740039+0000","last_peered":"2024-04-22T20:49:14.740039+0000","last_clean":"2024-04-22T20:49:14.740039+0000","last_became_active":"2024-04-22T19:51:26.511966+0000","last_became_peered":"2024-04-22T19:51:26.511966+0000","last_unstale":"2024-04-22T20:49:14.740039+0000","last_undegraded":"2024-04-22T20:49:14.740039+0000","last_fullsized":"2024-04-22T20:49:14.740039+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'328","last_scrub_stamp":"2024-04-22T20:49:14.739928+0000","last_deep_scrub":"261'328","last_deep_scrub_stamp":"2024-04-22T20:49:14.739928+0000","last_clean_scrub_stamp":"2024-04-22T20:49:14.739928+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:57:07.865559+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262244,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":436,"num_read_kb":38275,"num_write":1102,"num_write_kb":51509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":502,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:34.538975+0000","last_change":"2024-04-22T20:48:34.538975+0000","last_active":"2024-04-22T20:48:34.538975+0000","last_peered":"2024-04-22T20:48:34.538975+0000","last_clean":"2024-04-22T20:48:34.538975+0000","last_became_active":"2024-04-22T19:51:30.526949+0000","last_became_peered":"2024-04-22T19:51:30.526949+0000","last_unstale":"2024-04-22T20:48:34.538975+0000","last_undegraded":"2024-04-22T20:48:34.538975+0000","last_fullsized":"2024-04-22T20:48:34.538975+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:34.538867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:34.538867+0000","last_clean_scrub_stamp":"2024-04-22T20:48:34.538867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:45:12.857278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"261'111","reported_seq":647,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:35.397789+0000","last_change":"2024-04-22T20:48:35.397789+0000","last_active":"2024-04-22T20:48:35.397789+0000","last_peered":"2024-04-22T20:48:35.397789+0000","last_clean":"2024-04-22T20:48:35.397789+0000","last_became_active":"2024-04-22T19:51:33.248113+0000","last_became_peered":"2024-04-22T19:51:33.248113+0000","last_unstale":"2024-04-22T20:48:35.397789+0000","last_undegraded":"2024-04-22T20:48:35.397789+0000","last_fullsized":"2024-04-22T20:48:35.397789+0000","mapping_epoch":26,"log_start":"261'100","ondisk_log_start":"261'100","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'111","last_scrub_stamp":"2024-04-22T20:48:35.397661+0000","last_deep_scrub":"261'111","last_deep_scrub_stamp":"2024-04-22T20:48:35.397661+0000","last_clean_scrub_stamp":"2024-04-22T20:48:35.397661+0000","objects_scrubbed":8,"log_size":11,"log_dups_size":100,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:59:22.081071+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":119,"num_read_kb":113,"num_write":116,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11222,"num_omap_keys":30,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"261'518","reported_seq":2306,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.597254+0000","last_change":"2024-04-22T20:48:42.597282+0000","last_active":"2024-04-22T20:48:42.597254+0000","last_peered":"2024-04-22T20:48:42.597254+0000","last_clean":"2024-04-22T20:48:42.597254+0000","last_became_active":"2024-04-22T19:51:34.556789+0000","last_became_peered":"2024-04-22T19:51:34.556789+0000","last_unstale":"2024-04-22T20:48:42.597254+0000","last_undegraded":"2024-04-22T20:48:42.597254+0000","last_fullsized":"2024-04-22T20:48:42.597254+0000","mapping_epoch":28,"log_start":"261'500","ondisk_log_start":"261'500","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'518","last_scrub_stamp":"2024-04-22T20:48:42.597160+0000","last_deep_scrub":"261'518","last_deep_scrub_stamp":"2024-04-22T20:48:42.597160+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.597160+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":500,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:48:25.653530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"261'98","reported_seq":605,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:41.461752+0000","last_change":"2024-04-22T20:48:41.461786+0000","last_active":"2024-04-22T20:48:41.461752+0000","last_peered":"2024-04-22T20:48:41.461752+0000","last_clean":"2024-04-22T20:48:41.461752+0000","last_became_active":"2024-04-22T19:51:36.571175+0000","last_became_peered":"2024-04-22T19:51:36.571175+0000","last_unstale":"2024-04-22T20:48:41.461752+0000","last_undegraded":"2024-04-22T20:48:41.461752+0000","last_fullsized":"2024-04-22T20:48:41.461752+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'98","last_scrub_stamp":"2024-04-22T20:48:41.461655+0000","last_deep_scrub":"261'98","last_deep_scrub_stamp":"2024-04-22T20:48:41.461655+0000","last_clean_scrub_stamp":"2024-04-22T20:48:41.461655+0000","objects_scrubbed":2,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:32:03.950310+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":221,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"261'131","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.334715+0000","last_change":"2024-04-22T20:48:55.334715+0000","last_active":"2024-04-22T20:48:55.334715+0000","last_peered":"2024-04-22T20:48:55.334715+0000","last_clean":"2024-04-22T20:48:55.334715+0000","last_became_active":"2024-04-22T19:51:28.509110+0000","last_became_peered":"2024-04-22T19:51:28.509110+0000","last_unstale":"2024-04-22T20:48:55.334715+0000","last_undegraded":"2024-04-22T20:48:55.334715+0000","last_fullsized":"2024-04-22T20:48:55.334715+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'131","last_scrub_stamp":"2024-04-22T20:48:55.334593+0000","last_deep_scrub":"261'131","last_deep_scrub_stamp":"2024-04-22T20:48:55.334593+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.334593+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T23:00:37.391655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":519,"num_read_kb":420,"num_write":188,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"261'402","reported_seq":1038,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:25.850089+0000","last_change":"2024-04-22T20:48:25.850089+0000","last_active":"2024-04-22T20:48:25.850089+0000","last_peered":"2024-04-22T20:48:25.850089+0000","last_clean":"2024-04-22T20:48:25.850089+0000","last_became_active":"2024-04-22T19:52:04.619385+0000","last_became_peered":"2024-04-22T19:52:04.619385+0000","last_unstale":"2024-04-22T20:48:25.850089+0000","last_undegraded":"2024-04-22T20:48:25.850089+0000","last_fullsized":"2024-04-22T20:48:25.850089+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'402","last_scrub_stamp":"2024-04-22T20:48:25.849900+0000","last_deep_scrub":"261'402","last_deep_scrub_stamp":"2024-04-22T20:48:25.849900+0000","last_clean_scrub_stamp":"2024-04-22T20:48:25.849900+0000","objects_scrubbed":10,"log_size":102,"log_dups_size":300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T20:54:07.464040+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682319,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":541,"num_read_kb":21001,"num_write":1452,"num_write_kb":33571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":504,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.396156+0000","last_change":"2024-04-22T20:48:52.396156+0000","last_active":"2024-04-22T20:48:52.396156+0000","last_peered":"2024-04-22T20:48:52.396156+0000","last_clean":"2024-04-22T20:48:52.396156+0000","last_became_active":"2024-04-22T19:51:30.526469+0000","last_became_peered":"2024-04-22T19:51:30.526469+0000","last_unstale":"2024-04-22T20:48:52.396156+0000","last_undegraded":"2024-04-22T20:48:52.396156+0000","last_fullsized":"2024-04-22T20:48:52.396156+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.396006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.396006+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.396006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:26:59.620393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"261'313","reported_seq":907,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.872020+0000","last_change":"2024-04-22T20:48:55.872020+0000","last_active":"2024-04-22T20:48:55.872020+0000","last_peered":"2024-04-22T20:48:55.872020+0000","last_clean":"2024-04-22T20:48:55.872020+0000","last_became_active":"2024-04-22T19:51:32.540573+0000","last_became_peered":"2024-04-22T19:51:32.540573+0000","last_unstale":"2024-04-22T20:48:55.872020+0000","last_undegraded":"2024-04-22T20:48:55.872020+0000","last_fullsized":"2024-04-22T20:48:55.872020+0000","mapping_epoch":26,"log_start":"261'300","ondisk_log_start":"261'300","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:55.871874+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:55.871874+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.871874+0000","objects_scrubbed":7,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:02:42.734441+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48433,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":496,"num_read_kb":1091,"num_write":394,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.665362+0000","last_change":"2024-04-22T20:48:47.665362+0000","last_active":"2024-04-22T20:48:47.665362+0000","last_peered":"2024-04-22T20:48:47.665362+0000","last_clean":"2024-04-22T20:48:47.665362+0000","last_became_active":"2024-04-22T19:51:34.553857+0000","last_became_peered":"2024-04-22T19:51:34.553857+0000","last_unstale":"2024-04-22T20:48:47.665362+0000","last_undegraded":"2024-04-22T20:48:47.665362+0000","last_fullsized":"2024-04-22T20:48:47.665362+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:47.665221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:47.665221+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.665221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:00:40.709377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"261'157","reported_seq":668,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.377921+0000","last_change":"2024-04-22T20:48:39.377921+0000","last_active":"2024-04-22T20:48:39.377921+0000","last_peered":"2024-04-22T20:48:39.377921+0000","last_clean":"2024-04-22T20:48:39.377921+0000","last_became_active":"2024-04-22T19:51:36.575211+0000","last_became_peered":"2024-04-22T19:51:36.575211+0000","last_unstale":"2024-04-22T20:48:39.377921+0000","last_undegraded":"2024-04-22T20:48:39.377921+0000","last_fullsized":"2024-04-22T20:48:39.377921+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'157","last_scrub_stamp":"2024-04-22T20:48:39.377810+0000","last_deep_scrub":"261'157","last_deep_scrub_stamp":"2024-04-22T20:48:39.377810+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.377810+0000","objects_scrubbed":4,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:18:41.159874+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":158,"num_write":375,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"261'86","reported_seq":691,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.240691+0000","last_change":"2024-04-22T20:49:05.240723+0000","last_active":"2024-04-22T20:49:05.240691+0000","last_peered":"2024-04-22T20:49:05.240691+0000","last_clean":"2024-04-22T20:49:05.240691+0000","last_became_active":"2024-04-22T19:51:28.508145+0000","last_became_peered":"2024-04-22T19:51:28.508145+0000","last_unstale":"2024-04-22T20:49:05.240691+0000","last_undegraded":"2024-04-22T20:49:05.240691+0000","last_fullsized":"2024-04-22T20:49:05.240691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'86","last_scrub_stamp":"2024-04-22T20:49:05.240588+0000","last_deep_scrub":"261'86","last_deep_scrub_stamp":"2024-04-22T20:49:05.240588+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.240588+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:51:41.287365+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":376,"num_read_kb":320,"num_write":115,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"261'271","reported_seq":899,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.817431+0000","last_change":"2024-04-22T20:49:00.817466+0000","last_active":"2024-04-22T20:49:00.817431+0000","last_peered":"2024-04-22T20:49:00.817431+0000","last_clean":"2024-04-22T20:49:00.817431+0000","last_became_active":"2024-04-22T19:51:26.504301+0000","last_became_peered":"2024-04-22T19:51:26.504301+0000","last_unstale":"2024-04-22T20:49:00.817431+0000","last_undegraded":"2024-04-22T20:49:00.817431+0000","last_fullsized":"2024-04-22T20:49:00.817431+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'271","last_scrub_stamp":"2024-04-22T20:49:00.817344+0000","last_deep_scrub":"261'271","last_deep_scrub_stamp":"2024-04-22T20:49:00.817344+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.817344+0000","objects_scrubbed":4,"log_size":71,"log_dups_size":200,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:03:08.734664+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":435,"num_read_kb":4555,"num_write":1089,"num_write_kb":16946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":500,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.927324+0000","last_change":"2024-04-22T20:48:51.927324+0000","last_active":"2024-04-22T20:48:51.927324+0000","last_peered":"2024-04-22T20:48:51.927324+0000","last_clean":"2024-04-22T20:48:51.927324+0000","last_became_active":"2024-04-22T19:51:30.524417+0000","last_became_peered":"2024-04-22T19:51:30.524417+0000","last_unstale":"2024-04-22T20:48:51.927324+0000","last_undegraded":"2024-04-22T20:48:51.927324+0000","last_fullsized":"2024-04-22T20:48:51.927324+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:51.927209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:51.927209+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.927209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:31:17.320634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"261'233","reported_seq":812,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:05.680783+0000","last_change":"2024-04-22T20:49:05.680783+0000","last_active":"2024-04-22T20:49:05.680783+0000","last_peered":"2024-04-22T20:49:05.680783+0000","last_clean":"2024-04-22T20:49:05.680783+0000","last_became_active":"2024-04-22T19:51:32.540659+0000","last_became_peered":"2024-04-22T19:51:32.540659+0000","last_unstale":"2024-04-22T20:49:05.680783+0000","last_undegraded":"2024-04-22T20:49:05.680783+0000","last_fullsized":"2024-04-22T20:49:05.680783+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'233","last_scrub_stamp":"2024-04-22T20:49:05.680614+0000","last_deep_scrub":"261'233","last_deep_scrub_stamp":"2024-04-22T20:49:05.680614+0000","last_clean_scrub_stamp":"2024-04-22T20:49:05.680614+0000","objects_scrubbed":4,"log_size":33,"log_dups_size":200,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:25:49.392535+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16866,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":857,"num_write":302,"num_write_kb":237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":497,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:36.574034+0000","last_change":"2024-04-22T20:48:36.574034+0000","last_active":"2024-04-22T20:48:36.574034+0000","last_peered":"2024-04-22T20:48:36.574034+0000","last_clean":"2024-04-22T20:48:36.574034+0000","last_became_active":"2024-04-22T19:51:34.552344+0000","last_became_peered":"2024-04-22T19:51:34.552344+0000","last_unstale":"2024-04-22T20:48:36.574034+0000","last_undegraded":"2024-04-22T20:48:36.574034+0000","last_fullsized":"2024-04-22T20:48:36.574034+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:36.573910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:36.573910+0000","last_clean_scrub_stamp":"2024-04-22T20:48:36.573910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:39:04.986016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"261'138","reported_seq":686,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.476415+0000","last_change":"2024-04-22T20:48:37.476415+0000","last_active":"2024-04-22T20:48:37.476415+0000","last_peered":"2024-04-22T20:48:37.476415+0000","last_clean":"2024-04-22T20:48:37.476415+0000","last_became_active":"2024-04-22T19:51:36.567594+0000","last_became_peered":"2024-04-22T19:51:36.567594+0000","last_unstale":"2024-04-22T20:48:37.476415+0000","last_undegraded":"2024-04-22T20:48:37.476415+0000","last_fullsized":"2024-04-22T20:48:37.476415+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'138","last_scrub_stamp":"2024-04-22T20:48:37.476271+0000","last_deep_scrub":"261'138","last_deep_scrub_stamp":"2024-04-22T20:48:37.476271+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.476271+0000","objects_scrubbed":1,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:27:27.543918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":539,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":231,"num_read_kb":214,"num_write":335,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"261'87","reported_seq":658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.451837+0000","last_change":"2024-04-22T20:48:56.451837+0000","last_active":"2024-04-22T20:48:56.451837+0000","last_peered":"2024-04-22T20:48:56.451837+0000","last_clean":"2024-04-22T20:48:56.451837+0000","last_became_active":"2024-04-22T19:51:28.519542+0000","last_became_peered":"2024-04-22T19:51:28.519542+0000","last_unstale":"2024-04-22T20:48:56.451837+0000","last_undegraded":"2024-04-22T20:48:56.451837+0000","last_fullsized":"2024-04-22T20:48:56.451837+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'87","last_scrub_stamp":"2024-04-22T20:48:56.451708+0000","last_deep_scrub":"261'87","last_deep_scrub_stamp":"2024-04-22T20:48:56.451708+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.451708+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:21:39.301084+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":347,"num_read_kb":277,"num_write":137,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"261'398","reported_seq":1252,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:14.337033+0000","last_change":"2024-04-22T20:49:14.337033+0000","last_active":"2024-04-22T20:49:14.337033+0000","last_peered":"2024-04-22T20:49:14.337033+0000","last_clean":"2024-04-22T20:49:14.337033+0000","last_became_active":"2024-04-22T19:51:26.499390+0000","last_became_peered":"2024-04-22T19:51:26.499390+0000","last_unstale":"2024-04-22T20:49:14.337033+0000","last_undegraded":"2024-04-22T20:49:14.337033+0000","last_fullsized":"2024-04-22T20:49:14.337033+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'398","last_scrub_stamp":"2024-04-22T20:49:14.336876+0000","last_deep_scrub":"261'398","last_deep_scrub_stamp":"2024-04-22T20:49:14.336876+0000","last_clean_scrub_stamp":"2024-04-22T20:49:14.336876+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:04:13.353440+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4204544,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1043,"num_read_kb":88369,"num_write":1413,"num_write_kb":41583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.330304+0000","last_change":"2024-04-22T20:48:52.330304+0000","last_active":"2024-04-22T20:48:52.330304+0000","last_peered":"2024-04-22T20:48:52.330304+0000","last_clean":"2024-04-22T20:48:52.330304+0000","last_became_active":"2024-04-22T19:51:30.526183+0000","last_became_peered":"2024-04-22T19:51:30.526183+0000","last_unstale":"2024-04-22T20:48:52.330304+0000","last_undegraded":"2024-04-22T20:48:52.330304+0000","last_fullsized":"2024-04-22T20:48:52.330304+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:52.330187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:52.330187+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.330187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:16:30.153067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"261'251","reported_seq":825,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:18.295560+0000","last_change":"2024-04-22T20:49:18.295597+0000","last_active":"2024-04-22T20:49:18.295560+0000","last_peered":"2024-04-22T20:49:18.295560+0000","last_clean":"2024-04-22T20:49:18.295560+0000","last_became_active":"2024-04-22T19:51:32.538644+0000","last_became_peered":"2024-04-22T19:51:32.538644+0000","last_unstale":"2024-04-22T20:49:18.295560+0000","last_undegraded":"2024-04-22T20:49:18.295560+0000","last_fullsized":"2024-04-22T20:49:18.295560+0000","mapping_epoch":26,"log_start":"261'200","ondisk_log_start":"261'200","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'251","last_scrub_stamp":"2024-04-22T20:49:18.295471+0000","last_deep_scrub":"261'251","last_deep_scrub_stamp":"2024-04-22T20:49:18.295471+0000","last_clean_scrub_stamp":"2024-04-22T20:49:18.295471+0000","objects_scrubbed":3,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:01:54.905784+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":45694,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":402,"num_read_kb":955,"num_write":332,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:31.405108+0000","last_change":"2024-04-22T20:48:31.405108+0000","last_active":"2024-04-22T20:48:31.405108+0000","last_peered":"2024-04-22T20:48:31.405108+0000","last_clean":"2024-04-22T20:48:31.405108+0000","last_became_active":"2024-04-22T19:51:34.551085+0000","last_became_peered":"2024-04-22T19:51:34.551085+0000","last_unstale":"2024-04-22T20:48:31.405108+0000","last_undegraded":"2024-04-22T20:48:31.405108+0000","last_fullsized":"2024-04-22T20:48:31.405108+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:31.404990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:31.404990+0000","last_clean_scrub_stamp":"2024-04-22T20:48:31.404990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:49:00.257782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"261'187","reported_seq":921,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.442612+0000","last_change":"2024-04-22T20:48:42.442612+0000","last_active":"2024-04-22T20:48:42.442612+0000","last_peered":"2024-04-22T20:48:42.442612+0000","last_clean":"2024-04-22T20:48:42.442612+0000","last_became_active":"2024-04-22T19:51:36.569234+0000","last_became_peered":"2024-04-22T19:51:36.569234+0000","last_unstale":"2024-04-22T20:48:42.442612+0000","last_undegraded":"2024-04-22T20:48:42.442612+0000","last_fullsized":"2024-04-22T20:48:42.442612+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'187","last_scrub_stamp":"2024-04-22T20:48:42.442437+0000","last_deep_scrub":"261'187","last_deep_scrub_stamp":"2024-04-22T20:48:42.442437+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.442437+0000","objects_scrubbed":8,"log_size":87,"log_dups_size":100,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:21:18.797025+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1258,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":419,"num_read_kb":454,"num_write":429,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"261'180","reported_seq":760,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.345608+0000","last_change":"2024-04-22T20:48:44.345608+0000","last_active":"2024-04-22T20:48:44.345608+0000","last_peered":"2024-04-22T20:48:44.345608+0000","last_clean":"2024-04-22T20:48:44.345608+0000","last_became_active":"2024-04-22T19:51:28.515506+0000","last_became_peered":"2024-04-22T19:51:28.515506+0000","last_unstale":"2024-04-22T20:48:44.345608+0000","last_undegraded":"2024-04-22T20:48:44.345608+0000","last_fullsized":"2024-04-22T20:48:44.345608+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'180","last_scrub_stamp":"2024-04-22T20:48:44.345489+0000","last_deep_scrub":"261'180","last_deep_scrub_stamp":"2024-04-22T20:48:44.345489+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.345489+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":100,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:11:48.784252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":678,"num_read_kb":529,"num_write":279,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"261'297","reported_seq":918,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:12.691195+0000","last_change":"2024-04-22T20:49:12.691256+0000","last_active":"2024-04-22T20:49:12.691195+0000","last_peered":"2024-04-22T20:49:12.691195+0000","last_clean":"2024-04-22T20:49:12.691195+0000","last_became_active":"2024-04-22T19:51:26.504604+0000","last_became_peered":"2024-04-22T19:51:26.504604+0000","last_unstale":"2024-04-22T20:49:12.691195+0000","last_undegraded":"2024-04-22T20:49:12.691195+0000","last_fullsized":"2024-04-22T20:49:12.691195+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'297","last_scrub_stamp":"2024-04-22T20:49:12.691089+0000","last_deep_scrub":"261'297","last_deep_scrub_stamp":"2024-04-22T20:49:12.691089+0000","last_clean_scrub_stamp":"2024-04-22T20:49:12.691089+0000","objects_scrubbed":9,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:01:34.115179+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242937,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":372,"num_read_kb":9007,"num_write":952,"num_write_kb":46630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"261'2","reported_seq":505,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.636081+0000","last_change":"2024-04-22T20:48:38.636081+0000","last_active":"2024-04-22T20:48:38.636081+0000","last_peered":"2024-04-22T20:48:38.636081+0000","last_clean":"2024-04-22T20:48:38.636081+0000","last_became_active":"2024-04-22T19:51:30.523338+0000","last_became_peered":"2024-04-22T19:51:30.523338+0000","last_unstale":"2024-04-22T20:48:38.636081+0000","last_undegraded":"2024-04-22T20:48:38.636081+0000","last_fullsized":"2024-04-22T20:48:38.636081+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'2","last_scrub_stamp":"2024-04-22T20:48:38.635935+0000","last_deep_scrub":"261'2","last_deep_scrub_stamp":"2024-04-22T20:48:38.635935+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.635935+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:28:44.233266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"261'313","reported_seq":908,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.337600+0000","last_change":"2024-04-22T20:48:37.337600+0000","last_active":"2024-04-22T20:48:37.337600+0000","last_peered":"2024-04-22T20:48:37.337600+0000","last_clean":"2024-04-22T20:48:37.337600+0000","last_became_active":"2024-04-22T19:51:32.537373+0000","last_became_peered":"2024-04-22T19:51:32.537373+0000","last_unstale":"2024-04-22T20:48:37.337600+0000","last_undegraded":"2024-04-22T20:48:37.337600+0000","last_fullsized":"2024-04-22T20:48:37.337600+0000","mapping_epoch":26,"log_start":"261'301","ondisk_log_start":"261'301","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:48:37.337420+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:48:37.337420+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.337420+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":301,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:16:00.481534+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49488,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":495,"num_read_kb":1080,"num_write":394,"num_write_kb":293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":496,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.571825+0000","last_change":"2024-04-22T20:48:43.571825+0000","last_active":"2024-04-22T20:48:43.571825+0000","last_peered":"2024-04-22T20:48:43.571825+0000","last_clean":"2024-04-22T20:48:43.571825+0000","last_became_active":"2024-04-22T19:51:34.555709+0000","last_became_peered":"2024-04-22T19:51:34.555709+0000","last_unstale":"2024-04-22T20:48:43.571825+0000","last_undegraded":"2024-04-22T20:48:43.571825+0000","last_fullsized":"2024-04-22T20:48:43.571825+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-22T20:48:43.571712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-22T20:48:43.571712+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.571712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:04:20.739611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"261'146","reported_seq":639,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.318417+0000","last_change":"2024-04-22T20:48:26.318450+0000","last_active":"2024-04-22T20:48:26.318417+0000","last_peered":"2024-04-22T20:48:26.318417+0000","last_clean":"2024-04-22T20:48:26.318417+0000","last_became_active":"2024-04-22T19:51:36.574012+0000","last_became_peered":"2024-04-22T19:51:36.574012+0000","last_unstale":"2024-04-22T20:48:26.318417+0000","last_undegraded":"2024-04-22T20:48:26.318417+0000","last_fullsized":"2024-04-22T20:48:26.318417+0000","mapping_epoch":30,"log_start":"261'100","ondisk_log_start":"261'100","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'146","last_scrub_stamp":"2024-04-22T20:48:26.318331+0000","last_deep_scrub":"261'146","last_deep_scrub_stamp":"2024-04-22T20:48:26.318331+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.318331+0000","objects_scrubbed":2,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:49:20.701708+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":619,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":109,"num_write":348,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"261'372","reported_seq":1046,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:40.694389+0000","last_change":"2024-04-22T20:48:40.694419+0000","last_active":"2024-04-22T20:48:40.694389+0000","last_peered":"2024-04-22T20:48:40.694389+0000","last_clean":"2024-04-22T20:48:40.694389+0000","last_became_active":"2024-04-22T19:51:28.518339+0000","last_became_peered":"2024-04-22T19:51:28.518339+0000","last_unstale":"2024-04-22T20:48:40.694389+0000","last_undegraded":"2024-04-22T20:48:40.694389+0000","last_fullsized":"2024-04-22T20:48:40.694389+0000","mapping_epoch":22,"log_start":"261'300","ondisk_log_start":"261'300","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'372","last_scrub_stamp":"2024-04-22T20:48:40.694294+0000","last_deep_scrub":"261'372","last_deep_scrub_stamp":"2024-04-22T20:48:40.694294+0000","last_clean_scrub_stamp":"2024-04-22T20:48:40.694294+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:40:07.075387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1526,"num_read_kb":1211,"num_write":619,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"261'324","reported_seq":1032,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.592130+0000","last_change":"2024-04-22T20:48:55.592179+0000","last_active":"2024-04-22T20:48:55.592130+0000","last_peered":"2024-04-22T20:48:55.592130+0000","last_clean":"2024-04-22T20:48:55.592130+0000","last_became_active":"2024-04-22T19:51:26.502773+0000","last_became_peered":"2024-04-22T19:51:26.502773+0000","last_unstale":"2024-04-22T20:48:55.592130+0000","last_undegraded":"2024-04-22T20:48:55.592130+0000","last_fullsized":"2024-04-22T20:48:55.592130+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:48:55.592027+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:48:55.592027+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.592027+0000","objects_scrubbed":7,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:38.053119+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4371014,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":594,"num_read_kb":14386,"num_write":1094,"num_write_kb":17858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,7],"acting":[6,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"261'524","reported_seq":1296,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.877481+0000","last_change":"2024-04-22T20:48:26.877525+0000","last_active":"2024-04-22T20:48:26.877481+0000","last_peered":"2024-04-22T20:48:26.877481+0000","last_clean":"2024-04-22T20:48:26.877481+0000","last_became_active":"2024-04-22T19:52:04.617036+0000","last_became_peered":"2024-04-22T19:52:04.617036+0000","last_unstale":"2024-04-22T20:48:26.877481+0000","last_undegraded":"2024-04-22T20:48:26.877481+0000","last_fullsized":"2024-04-22T20:48:26.877481+0000","mapping_epoch":33,"log_start":"261'500","ondisk_log_start":"261'500","created":22,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'524","last_scrub_stamp":"2024-04-22T20:48:26.877358+0000","last_deep_scrub":"261'524","last_deep_scrub_stamp":"2024-04-22T20:48:26.877358+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.877358+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":500,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:34:00.586382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2230,"num_read_kb":1847,"num_write":935,"num_write_kb":601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"261'286","reported_seq":934,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:38.553956+0000","last_change":"2024-04-22T20:48:38.553990+0000","last_active":"2024-04-22T20:48:38.553956+0000","last_peered":"2024-04-22T20:48:38.553956+0000","last_clean":"2024-04-22T20:48:38.553956+0000","last_became_active":"2024-04-22T19:51:26.495948+0000","last_became_peered":"2024-04-22T19:51:26.495948+0000","last_unstale":"2024-04-22T20:48:38.553956+0000","last_undegraded":"2024-04-22T20:48:38.553956+0000","last_fullsized":"2024-04-22T20:48:38.553956+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'286","last_scrub_stamp":"2024-04-22T20:48:38.553878+0000","last_deep_scrub":"261'286","last_deep_scrub_stamp":"2024-04-22T20:48:38.553878+0000","last_clean_scrub_stamp":"2024-04-22T20:48:38.553878+0000","objects_scrubbed":6,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:55:12.594083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180512,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":454,"num_read_kb":10117,"num_write":1004,"num_write_kb":22835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"261'209","reported_seq":851,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.973501+0000","last_change":"2024-04-22T20:48:44.973554+0000","last_active":"2024-04-22T20:48:44.973501+0000","last_peered":"2024-04-22T20:48:44.973501+0000","last_clean":"2024-04-22T20:48:44.973501+0000","last_became_active":"2024-04-22T19:51:28.513026+0000","last_became_peered":"2024-04-22T19:51:28.513026+0000","last_unstale":"2024-04-22T20:48:44.973501+0000","last_undegraded":"2024-04-22T20:48:44.973501+0000","last_fullsized":"2024-04-22T20:48:44.973501+0000","mapping_epoch":22,"log_start":"258'100","ondisk_log_start":"258'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'209","last_scrub_stamp":"2024-04-22T20:48:44.973375+0000","last_deep_scrub":"261'209","last_deep_scrub_stamp":"2024-04-22T20:48:44.973375+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.973375+0000","objects_scrubbed":1,"log_size":109,"log_dups_size":100,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:27:59.805690+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":827,"num_read_kb":665,"num_write":342,"num_write_kb":185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"261'313","reported_seq":941,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.835329+0000","last_change":"2024-04-22T20:49:04.835404+0000","last_active":"2024-04-22T20:49:04.835329+0000","last_peered":"2024-04-22T20:49:04.835329+0000","last_clean":"2024-04-22T20:49:04.835329+0000","last_became_active":"2024-04-22T19:51:26.506748+0000","last_became_peered":"2024-04-22T19:51:26.506748+0000","last_unstale":"2024-04-22T20:49:04.835329+0000","last_undegraded":"2024-04-22T20:49:04.835329+0000","last_fullsized":"2024-04-22T20:49:04.835329+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'313","last_scrub_stamp":"2024-04-22T20:49:04.835221+0000","last_deep_scrub":"261'313","last_deep_scrub_stamp":"2024-04-22T20:49:04.835221+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.835221+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:46.453041+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":416,"num_read_kb":12678,"num_write":1099,"num_write_kb":41347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0],"acting":[5,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"261'4175","reported_seq":4800,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:56.586185+0000","last_change":"2024-04-22T20:48:56.586215+0000","last_active":"2024-04-22T20:48:56.586185+0000","last_peered":"2024-04-22T20:48:56.586185+0000","last_clean":"2024-04-22T20:48:56.586185+0000","last_became_active":"2024-04-22T19:51:28.511312+0000","last_became_peered":"2024-04-22T19:51:28.511312+0000","last_unstale":"2024-04-22T20:48:56.586185+0000","last_undegraded":"2024-04-22T20:48:56.586185+0000","last_fullsized":"2024-04-22T20:48:56.586185+0000","mapping_epoch":22,"log_start":"261'4100","ondisk_log_start":"261'4100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4175","last_scrub_stamp":"2024-04-22T20:48:56.586085+0000","last_deep_scrub":"261'4175","last_deep_scrub_stamp":"2024-04-22T20:48:56.586085+0000","last_clean_scrub_stamp":"2024-04-22T20:48:56.586085+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":3000,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:38:15.045023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14718,"num_read_kb":11919,"num_write":6278,"num_write_kb":3133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"261'312","reported_seq":909,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.452280+0000","last_change":"2024-04-22T20:48:28.452280+0000","last_active":"2024-04-22T20:48:28.452280+0000","last_peered":"2024-04-22T20:48:28.452280+0000","last_clean":"2024-04-22T20:48:28.452280+0000","last_became_active":"2024-04-22T19:52:05.114734+0000","last_became_peered":"2024-04-22T19:52:05.114734+0000","last_unstale":"2024-04-22T20:48:28.452280+0000","last_undegraded":"2024-04-22T20:48:28.452280+0000","last_fullsized":"2024-04-22T20:48:28.452280+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'312","last_scrub_stamp":"2024-04-22T20:48:28.452109+0000","last_deep_scrub":"261'312","last_deep_scrub_stamp":"2024-04-22T20:48:28.452109+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.452109+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":300,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:41:21.599729+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437203,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":421,"num_read_kb":5518,"num_write":1088,"num_write_kb":21667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.23","version":"261'171","reported_seq":861,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.466280+0000","last_change":"2024-04-22T20:48:57.466312+0000","last_active":"2024-04-22T20:48:57.466280+0000","last_peered":"2024-04-22T20:48:57.466280+0000","last_clean":"2024-04-22T20:48:57.466280+0000","last_became_active":"2024-04-22T19:51:28.516125+0000","last_became_peered":"2024-04-22T19:51:28.516125+0000","last_unstale":"2024-04-22T20:48:57.466280+0000","last_undegraded":"2024-04-22T20:48:57.466280+0000","last_fullsized":"2024-04-22T20:48:57.466280+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'171","last_scrub_stamp":"2024-04-22T20:48:57.466192+0000","last_deep_scrub":"261'171","last_deep_scrub_stamp":"2024-04-22T20:48:57.466192+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.466192+0000","objects_scrubbed":1,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:22:48.965827+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":741,"num_read_kb":616,"num_write":243,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"261'368","reported_seq":1024,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:26.429047+0000","last_change":"2024-04-22T20:48:26.429162+0000","last_active":"2024-04-22T20:48:26.429047+0000","last_peered":"2024-04-22T20:48:26.429047+0000","last_clean":"2024-04-22T20:48:26.429047+0000","last_became_active":"2024-04-22T19:52:04.619286+0000","last_became_peered":"2024-04-22T19:52:04.619286+0000","last_unstale":"2024-04-22T20:48:26.429047+0000","last_undegraded":"2024-04-22T20:48:26.429047+0000","last_fullsized":"2024-04-22T20:48:26.429047+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'368","last_scrub_stamp":"2024-04-22T20:48:26.428925+0000","last_deep_scrub":"261'368","last_deep_scrub_stamp":"2024-04-22T20:48:26.428925+0000","last_clean_scrub_stamp":"2024-04-22T20:48:26.428925+0000","objects_scrubbed":9,"log_size":68,"log_dups_size":300,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:38:16.414530+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4719706,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":588,"num_read_kb":69897,"num_write":1236,"num_write_kb":36151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.22","version":"261'122","reported_seq":735,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:46.272617+0000","last_change":"2024-04-22T20:48:46.272648+0000","last_active":"2024-04-22T20:48:46.272617+0000","last_peered":"2024-04-22T20:48:46.272617+0000","last_clean":"2024-04-22T20:48:46.272617+0000","last_became_active":"2024-04-22T19:51:28.516437+0000","last_became_peered":"2024-04-22T19:51:28.516437+0000","last_unstale":"2024-04-22T20:48:46.272617+0000","last_undegraded":"2024-04-22T20:48:46.272617+0000","last_fullsized":"2024-04-22T20:48:46.272617+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'122","last_scrub_stamp":"2024-04-22T20:48:46.272512+0000","last_deep_scrub":"261'122","last_deep_scrub_stamp":"2024-04-22T20:48:46.272512+0000","last_clean_scrub_stamp":"2024-04-22T20:48:46.272512+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":100,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:54:33.342867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":517,"num_read_kb":423,"num_write":178,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"261'317","reported_seq":1060,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.585793+0000","last_change":"2024-04-22T20:48:53.585833+0000","last_active":"2024-04-22T20:48:53.585793+0000","last_peered":"2024-04-22T20:48:53.585793+0000","last_clean":"2024-04-22T20:48:53.585793+0000","last_became_active":"2024-04-22T19:51:26.498039+0000","last_became_peered":"2024-04-22T19:51:26.498039+0000","last_unstale":"2024-04-22T20:48:53.585793+0000","last_undegraded":"2024-04-22T20:48:53.585793+0000","last_fullsized":"2024-04-22T20:48:53.585793+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'317","last_scrub_stamp":"2024-04-22T20:48:53.585698+0000","last_deep_scrub":"261'317","last_deep_scrub_stamp":"2024-04-22T20:48:53.585698+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.585698+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":300,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:05:32.063217+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194319,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":766,"num_read_kb":22382,"num_write":1253,"num_write_kb":18139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"261'199","reported_seq":857,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:00.407111+0000","last_change":"2024-04-22T20:49:00.407135+0000","last_active":"2024-04-22T20:49:00.407111+0000","last_peered":"2024-04-22T20:49:00.407111+0000","last_clean":"2024-04-22T20:49:00.407111+0000","last_became_active":"2024-04-22T19:51:28.520612+0000","last_became_peered":"2024-04-22T19:51:28.520612+0000","last_unstale":"2024-04-22T20:49:00.407111+0000","last_undegraded":"2024-04-22T20:49:00.407111+0000","last_fullsized":"2024-04-22T20:49:00.407111+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'199","last_scrub_stamp":"2024-04-22T20:49:00.407015+0000","last_deep_scrub":"261'199","last_deep_scrub_stamp":"2024-04-22T20:49:00.407015+0000","last_clean_scrub_stamp":"2024-04-22T20:49:00.407015+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":100,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:11:38.305157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":735,"num_read_kb":593,"num_write":262,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"261'365","reported_seq":1022,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.558974+0000","last_change":"2024-04-22T20:48:52.559103+0000","last_active":"2024-04-22T20:48:52.558974+0000","last_peered":"2024-04-22T20:48:52.558974+0000","last_clean":"2024-04-22T20:48:52.558974+0000","last_became_active":"2024-04-22T19:51:26.498845+0000","last_became_peered":"2024-04-22T19:51:26.498845+0000","last_unstale":"2024-04-22T20:48:52.558974+0000","last_undegraded":"2024-04-22T20:48:52.558974+0000","last_fullsized":"2024-04-22T20:48:52.558974+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'365","last_scrub_stamp":"2024-04-22T20:48:52.558851+0000","last_deep_scrub":"261'365","last_deep_scrub_stamp":"2024-04-22T20:48:52.558851+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.558851+0000","objects_scrubbed":9,"log_size":65,"log_dups_size":300,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:38:15.742114+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050721,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":506,"num_read_kb":33263,"num_write":1241,"num_write_kb":30860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"261'133","reported_seq":775,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:37.610209+0000","last_change":"2024-04-22T20:48:37.610234+0000","last_active":"2024-04-22T20:48:37.610209+0000","last_peered":"2024-04-22T20:48:37.610209+0000","last_clean":"2024-04-22T20:48:37.610209+0000","last_became_active":"2024-04-22T19:51:28.519669+0000","last_became_peered":"2024-04-22T19:51:28.519669+0000","last_unstale":"2024-04-22T20:48:37.610209+0000","last_undegraded":"2024-04-22T20:48:37.610209+0000","last_fullsized":"2024-04-22T20:48:37.610209+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'133","last_scrub_stamp":"2024-04-22T20:48:37.610118+0000","last_deep_scrub":"261'133","last_deep_scrub_stamp":"2024-04-22T20:48:37.610118+0000","last_clean_scrub_stamp":"2024-04-22T20:48:37.610118+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:03:13.662918+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":560,"num_read_kb":466,"num_write":196,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"261'330","reported_seq":935,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:29.490447+0000","last_change":"2024-04-22T20:48:29.490501+0000","last_active":"2024-04-22T20:48:29.490447+0000","last_peered":"2024-04-22T20:48:29.490447+0000","last_clean":"2024-04-22T20:48:29.490447+0000","last_became_active":"2024-04-22T19:52:04.617649+0000","last_became_peered":"2024-04-22T19:52:04.617649+0000","last_unstale":"2024-04-22T20:48:29.490447+0000","last_undegraded":"2024-04-22T20:48:29.490447+0000","last_fullsized":"2024-04-22T20:48:29.490447+0000","mapping_epoch":33,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'330","last_scrub_stamp":"2024-04-22T20:48:29.490328+0000","last_deep_scrub":"261'330","last_deep_scrub_stamp":"2024-04-22T20:48:29.490328+0000","last_clean_scrub_stamp":"2024-04-22T20:48:29.490328+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":300,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T00:55:48.835393+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1050919,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":29074,"num_write":1030,"num_write_kb":22627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,2],"acting":[1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2f","version":"261'128","reported_seq":718,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:51.677849+0000","last_change":"2024-04-22T20:48:51.677899+0000","last_active":"2024-04-22T20:48:51.677849+0000","last_peered":"2024-04-22T20:48:51.677849+0000","last_clean":"2024-04-22T20:48:51.677849+0000","last_became_active":"2024-04-22T19:51:28.521516+0000","last_became_peered":"2024-04-22T19:51:28.521516+0000","last_unstale":"2024-04-22T20:48:51.677849+0000","last_undegraded":"2024-04-22T20:48:51.677849+0000","last_fullsized":"2024-04-22T20:48:51.677849+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'128","last_scrub_stamp":"2024-04-22T20:48:51.677734+0000","last_deep_scrub":"261'128","last_deep_scrub_stamp":"2024-04-22T20:48:51.677734+0000","last_clean_scrub_stamp":"2024-04-22T20:48:51.677734+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:24:24.647580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":542,"num_read_kb":442,"num_write":210,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"261'338","reported_seq":1027,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.972384+0000","last_change":"2024-04-22T20:48:52.972424+0000","last_active":"2024-04-22T20:48:52.972384+0000","last_peered":"2024-04-22T20:48:52.972384+0000","last_clean":"2024-04-22T20:48:52.972384+0000","last_became_active":"2024-04-22T19:51:26.507843+0000","last_became_peered":"2024-04-22T19:51:26.507843+0000","last_unstale":"2024-04-22T20:48:52.972384+0000","last_undegraded":"2024-04-22T20:48:52.972384+0000","last_fullsized":"2024-04-22T20:48:52.972384+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'338","last_scrub_stamp":"2024-04-22T20:48:52.972288+0000","last_deep_scrub":"261'338","last_deep_scrub_stamp":"2024-04-22T20:48:52.972288+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.972288+0000","objects_scrubbed":6,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:20:36.857751+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5531396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":507,"num_read_kb":136369,"num_write":1233,"num_write_kb":55788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"261'235","reported_seq":869,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:47.241423+0000","last_change":"2024-04-22T20:48:47.241450+0000","last_active":"2024-04-22T20:48:47.241423+0000","last_peered":"2024-04-22T20:48:47.241423+0000","last_clean":"2024-04-22T20:48:47.241423+0000","last_became_active":"2024-04-22T19:51:28.515164+0000","last_became_peered":"2024-04-22T19:51:28.515164+0000","last_unstale":"2024-04-22T20:48:47.241423+0000","last_undegraded":"2024-04-22T20:48:47.241423+0000","last_fullsized":"2024-04-22T20:48:47.241423+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'235","last_scrub_stamp":"2024-04-22T20:48:47.241303+0000","last_deep_scrub":"261'235","last_deep_scrub_stamp":"2024-04-22T20:48:47.241303+0000","last_clean_scrub_stamp":"2024-04-22T20:48:47.241303+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":200,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T21:06:24.721788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":950,"num_read_kb":750,"num_write":364,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"261'502","reported_seq":1211,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:11.319386+0000","last_change":"2024-04-22T20:49:11.319443+0000","last_active":"2024-04-22T20:49:11.319386+0000","last_peered":"2024-04-22T20:49:11.319386+0000","last_clean":"2024-04-22T20:49:11.319386+0000","last_became_active":"2024-04-22T19:51:26.493860+0000","last_became_peered":"2024-04-22T19:51:26.493860+0000","last_unstale":"2024-04-22T20:49:11.319386+0000","last_undegraded":"2024-04-22T20:49:11.319386+0000","last_fullsized":"2024-04-22T20:49:11.319386+0000","mapping_epoch":20,"log_start":"261'400","ondisk_log_start":"261'400","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'502","last_scrub_stamp":"2024-04-22T20:49:11.319268+0000","last_deep_scrub":"261'502","last_deep_scrub_stamp":"2024-04-22T20:49:11.319268+0000","last_clean_scrub_stamp":"2024-04-22T20:49:11.319268+0000","objects_scrubbed":8,"log_size":102,"log_dups_size":400,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:40:00.636759+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6292618,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":951,"num_read_kb":13872,"num_write":1439,"num_write_kb":37693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"261'252","reported_seq":939,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:44.584503+0000","last_change":"2024-04-22T20:48:44.584554+0000","last_active":"2024-04-22T20:48:44.584503+0000","last_peered":"2024-04-22T20:48:44.584503+0000","last_clean":"2024-04-22T20:48:44.584503+0000","last_became_active":"2024-04-22T19:51:28.515547+0000","last_became_peered":"2024-04-22T19:51:28.515547+0000","last_unstale":"2024-04-22T20:48:44.584503+0000","last_undegraded":"2024-04-22T20:48:44.584503+0000","last_fullsized":"2024-04-22T20:48:44.584503+0000","mapping_epoch":22,"log_start":"247'200","ondisk_log_start":"247'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'252","last_scrub_stamp":"2024-04-22T20:48:44.584399+0000","last_deep_scrub":"261'252","last_deep_scrub_stamp":"2024-04-22T20:48:44.584399+0000","last_clean_scrub_stamp":"2024-04-22T20:48:44.584399+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":200,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:23:23.642516+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1120,"num_read_kb":973,"num_write":365,"num_write_kb":223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25681,"num_omap_keys":101,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"261'324","reported_seq":963,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:58.831693+0000","last_change":"2024-04-22T20:48:58.831759+0000","last_active":"2024-04-22T20:48:58.831693+0000","last_peered":"2024-04-22T20:48:58.831693+0000","last_clean":"2024-04-22T20:48:58.831693+0000","last_became_active":"2024-04-22T19:51:26.500566+0000","last_became_peered":"2024-04-22T19:51:26.500566+0000","last_unstale":"2024-04-22T20:48:58.831693+0000","last_undegraded":"2024-04-22T20:48:58.831693+0000","last_fullsized":"2024-04-22T20:48:58.831693+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'324","last_scrub_stamp":"2024-04-22T20:48:58.831593+0000","last_deep_scrub":"261'324","last_deep_scrub_stamp":"2024-04-22T20:48:58.831593+0000","last_clean_scrub_stamp":"2024-04-22T20:48:58.831593+0000","objects_scrubbed":7,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T08:41:02.913836+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283749,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":424,"num_read_kb":45844,"num_write":1120,"num_write_kb":42819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"261'152","reported_seq":759,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:43.297232+0000","last_change":"2024-04-22T20:48:43.297263+0000","last_active":"2024-04-22T20:48:43.297232+0000","last_peered":"2024-04-22T20:48:43.297232+0000","last_clean":"2024-04-22T20:48:43.297232+0000","last_became_active":"2024-04-22T19:51:28.515764+0000","last_became_peered":"2024-04-22T19:51:28.515764+0000","last_unstale":"2024-04-22T20:48:43.297232+0000","last_undegraded":"2024-04-22T20:48:43.297232+0000","last_fullsized":"2024-04-22T20:48:43.297232+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'152","last_scrub_stamp":"2024-04-22T20:48:43.297146+0000","last_deep_scrub":"261'152","last_deep_scrub_stamp":"2024-04-22T20:48:43.297146+0000","last_clean_scrub_stamp":"2024-04-22T20:48:43.297146+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":100,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T04:54:48.516556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":633,"num_read_kb":515,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"261'261","reported_seq":879,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:06.735739+0000","last_change":"2024-04-22T20:49:06.735795+0000","last_active":"2024-04-22T20:49:06.735739+0000","last_peered":"2024-04-22T20:49:06.735739+0000","last_clean":"2024-04-22T20:49:06.735739+0000","last_became_active":"2024-04-22T19:51:26.512467+0000","last_became_peered":"2024-04-22T19:51:26.512467+0000","last_unstale":"2024-04-22T20:49:06.735739+0000","last_undegraded":"2024-04-22T20:49:06.735739+0000","last_fullsized":"2024-04-22T20:49:06.735739+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'261","last_scrub_stamp":"2024-04-22T20:49:06.735656+0000","last_deep_scrub":"261'261","last_deep_scrub_stamp":"2024-04-22T20:49:06.735656+0000","last_clean_scrub_stamp":"2024-04-22T20:49:06.735656+0000","objects_scrubbed":8,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:48:01.033769+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11536625,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":377,"num_read_kb":17908,"num_write":933,"num_write_kb":43703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"261'27","reported_seq":563,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:55.366682+0000","last_change":"2024-04-22T20:48:55.366707+0000","last_active":"2024-04-22T20:48:55.366682+0000","last_peered":"2024-04-22T20:48:55.366682+0000","last_clean":"2024-04-22T20:48:55.366682+0000","last_became_active":"2024-04-22T19:51:28.510091+0000","last_became_peered":"2024-04-22T19:51:28.510091+0000","last_unstale":"2024-04-22T20:48:55.366682+0000","last_undegraded":"2024-04-22T20:48:55.366682+0000","last_fullsized":"2024-04-22T20:48:55.366682+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'27","last_scrub_stamp":"2024-04-22T20:48:55.366594+0000","last_deep_scrub":"261'27","last_deep_scrub_stamp":"2024-04-22T20:48:55.366594+0000","last_clean_scrub_stamp":"2024-04-22T20:48:55.366594+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:48:05.352437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":95,"num_write":39,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"261'340","reported_seq":996,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:53.582045+0000","last_change":"2024-04-22T20:48:53.582088+0000","last_active":"2024-04-22T20:48:53.582045+0000","last_peered":"2024-04-22T20:48:53.582045+0000","last_clean":"2024-04-22T20:48:53.582045+0000","last_became_active":"2024-04-22T19:51:26.501700+0000","last_became_peered":"2024-04-22T19:51:26.501700+0000","last_unstale":"2024-04-22T20:48:53.582045+0000","last_undegraded":"2024-04-22T20:48:53.582045+0000","last_fullsized":"2024-04-22T20:48:53.582045+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'340","last_scrub_stamp":"2024-04-22T20:48:53.581939+0000","last_deep_scrub":"261'340","last_deep_scrub_stamp":"2024-04-22T20:48:53.581939+0000","last_clean_scrub_stamp":"2024-04-22T20:48:53.581939+0000","objects_scrubbed":7,"log_size":40,"log_dups_size":300,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T05:36:58.999952+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4231939,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":506,"num_read_kb":6050,"num_write":1241,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"261'54","reported_seq":612,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:04.710469+0000","last_change":"2024-04-22T20:49:04.710497+0000","last_active":"2024-04-22T20:49:04.710469+0000","last_peered":"2024-04-22T20:49:04.710469+0000","last_clean":"2024-04-22T20:49:04.710469+0000","last_became_active":"2024-04-22T19:51:28.508768+0000","last_became_peered":"2024-04-22T19:51:28.508768+0000","last_unstale":"2024-04-22T20:49:04.710469+0000","last_undegraded":"2024-04-22T20:49:04.710469+0000","last_fullsized":"2024-04-22T20:49:04.710469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'54","last_scrub_stamp":"2024-04-22T20:49:04.710365+0000","last_deep_scrub":"261'54","last_deep_scrub_stamp":"2024-04-22T20:49:04.710365+0000","last_clean_scrub_stamp":"2024-04-22T20:49:04.710365+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:28:08.981902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":238,"num_read_kb":196,"num_write":80,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"261'355","reported_seq":1003,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:09.613696+0000","last_change":"2024-04-22T20:49:09.613768+0000","last_active":"2024-04-22T20:49:09.613696+0000","last_peered":"2024-04-22T20:49:09.613696+0000","last_clean":"2024-04-22T20:49:09.613696+0000","last_became_active":"2024-04-22T19:51:26.497156+0000","last_became_peered":"2024-04-22T19:51:26.497156+0000","last_unstale":"2024-04-22T20:49:09.613696+0000","last_undegraded":"2024-04-22T20:49:09.613696+0000","last_fullsized":"2024-04-22T20:49:09.613696+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'355","last_scrub_stamp":"2024-04-22T20:49:09.613581+0000","last_deep_scrub":"261'355","last_deep_scrub_stamp":"2024-04-22T20:49:09.613581+0000","last_clean_scrub_stamp":"2024-04-22T20:49:09.613581+0000","objects_scrubbed":8,"log_size":55,"log_dups_size":300,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T01:28:49.999075+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9437245,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":536,"num_read_kb":35895,"num_write":1244,"num_write_kb":35286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"261'4050","reported_seq":4621,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:59.446481+0000","last_change":"2024-04-22T20:48:59.446504+0000","last_active":"2024-04-22T20:48:59.446481+0000","last_peered":"2024-04-22T20:48:59.446481+0000","last_clean":"2024-04-22T20:48:59.446481+0000","last_became_active":"2024-04-22T19:51:28.514780+0000","last_became_peered":"2024-04-22T19:51:28.514780+0000","last_unstale":"2024-04-22T20:48:59.446481+0000","last_undegraded":"2024-04-22T20:48:59.446481+0000","last_fullsized":"2024-04-22T20:48:59.446481+0000","mapping_epoch":22,"log_start":"261'4000","ondisk_log_start":"261'4000","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'4050","last_scrub_stamp":"2024-04-22T20:48:59.446394+0000","last_deep_scrub":"261'4050","last_deep_scrub_stamp":"2024-04-22T20:48:59.446394+0000","last_clean_scrub_stamp":"2024-04-22T20:48:59.446394+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":3000,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:24:00.775234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14224,"num_read_kb":11535,"num_write":6070,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"261'302","reported_seq":1070,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:39.563705+0000","last_change":"2024-04-22T20:48:39.563748+0000","last_active":"2024-04-22T20:48:39.563705+0000","last_peered":"2024-04-22T20:48:39.563705+0000","last_clean":"2024-04-22T20:48:39.563705+0000","last_became_active":"2024-04-22T19:51:26.496214+0000","last_became_peered":"2024-04-22T19:51:26.496214+0000","last_unstale":"2024-04-22T20:48:39.563705+0000","last_undegraded":"2024-04-22T20:48:39.563705+0000","last_fullsized":"2024-04-22T20:48:39.563705+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'302","last_scrub_stamp":"2024-04-22T20:48:39.563612+0000","last_deep_scrub":"261'302","last_deep_scrub_stamp":"2024-04-22T20:48:39.563612+0000","last_clean_scrub_stamp":"2024-04-22T20:48:39.563612+0000","objects_scrubbed":5,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:41:49.634270+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1338117,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":584,"num_read_kb":13796,"num_write":1181,"num_write_kb":14745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1],"acting":[2,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"261'298","reported_seq":988,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:57.430136+0000","last_change":"2024-04-22T20:48:57.430161+0000","last_active":"2024-04-22T20:48:57.430136+0000","last_peered":"2024-04-22T20:48:57.430136+0000","last_clean":"2024-04-22T20:48:57.430136+0000","last_became_active":"2024-04-22T19:51:28.508681+0000","last_became_peered":"2024-04-22T19:51:28.508681+0000","last_unstale":"2024-04-22T20:48:57.430136+0000","last_undegraded":"2024-04-22T20:48:57.430136+0000","last_fullsized":"2024-04-22T20:48:57.430136+0000","mapping_epoch":22,"log_start":"261'200","ondisk_log_start":"261'200","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'298","last_scrub_stamp":"2024-04-22T20:48:57.430053+0000","last_deep_scrub":"261'298","last_deep_scrub_stamp":"2024-04-22T20:48:57.430053+0000","last_clean_scrub_stamp":"2024-04-22T20:48:57.430053+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T02:16:35.213640+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1231,"num_read_kb":1000,"num_write":489,"num_write_kb":278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"261'282","reported_seq":900,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:02.830889+0000","last_change":"2024-04-22T20:49:02.831005+0000","last_active":"2024-04-22T20:49:02.830889+0000","last_peered":"2024-04-22T20:49:02.830889+0000","last_clean":"2024-04-22T20:49:02.830889+0000","last_became_active":"2024-04-22T19:51:26.502438+0000","last_became_peered":"2024-04-22T19:51:26.502438+0000","last_unstale":"2024-04-22T20:49:02.830889+0000","last_undegraded":"2024-04-22T20:49:02.830889+0000","last_fullsized":"2024-04-22T20:49:02.830889+0000","mapping_epoch":20,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'282","last_scrub_stamp":"2024-04-22T20:49:02.830773+0000","last_deep_scrub":"261'282","last_deep_scrub_stamp":"2024-04-22T20:49:02.830773+0000","last_clean_scrub_stamp":"2024-04-22T20:49:02.830773+0000","objects_scrubbed":7,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:43:57.450915+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194332,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":386,"num_read_kb":36638,"num_write":936,"num_write_kb":32973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4],"acting":[5,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"261'374","reported_seq":1262,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:49:03.841410+0000","last_change":"2024-04-22T20:49:03.841470+0000","last_active":"2024-04-22T20:49:03.841410+0000","last_peered":"2024-04-22T20:49:03.841410+0000","last_clean":"2024-04-22T20:49:03.841410+0000","last_became_active":"2024-04-22T19:51:26.504878+0000","last_became_peered":"2024-04-22T19:51:26.504878+0000","last_unstale":"2024-04-22T20:49:03.841410+0000","last_undegraded":"2024-04-22T20:49:03.841410+0000","last_fullsized":"2024-04-22T20:49:03.841410+0000","mapping_epoch":20,"log_start":"261'300","ondisk_log_start":"261'300","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'374","last_scrub_stamp":"2024-04-22T20:49:03.841311+0000","last_deep_scrub":"261'374","last_deep_scrub_stamp":"2024-04-22T20:49:03.841311+0000","last_clean_scrub_stamp":"2024-04-22T20:49:03.841311+0000","objects_scrubbed":8,"log_size":74,"log_dups_size":300,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-23T22:34:22.018993+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5353839,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1018,"num_read_kb":55238,"num_write":1407,"num_write_kb":52838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"261'127","reported_seq":738,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:52.665674+0000","last_change":"2024-04-22T20:48:52.665674+0000","last_active":"2024-04-22T20:48:52.665674+0000","last_peered":"2024-04-22T20:48:52.665674+0000","last_clean":"2024-04-22T20:48:52.665674+0000","last_became_active":"2024-04-22T19:51:28.521372+0000","last_became_peered":"2024-04-22T19:51:28.521372+0000","last_unstale":"2024-04-22T20:48:52.665674+0000","last_undegraded":"2024-04-22T20:48:52.665674+0000","last_fullsized":"2024-04-22T20:48:52.665674+0000","mapping_epoch":22,"log_start":"261'100","ondisk_log_start":"261'100","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'127","last_scrub_stamp":"2024-04-22T20:48:52.665538+0000","last_deep_scrub":"261'127","last_deep_scrub_stamp":"2024-04-22T20:48:52.665538+0000","last_clean_scrub_stamp":"2024-04-22T20:48:52.665538+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T06:58:05.220984+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":525,"num_read_kb":437,"num_write":198,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"261'287","reported_seq":950,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:28.495078+0000","last_change":"2024-04-22T20:48:28.495119+0000","last_active":"2024-04-22T20:48:28.495078+0000","last_peered":"2024-04-22T20:48:28.495078+0000","last_clean":"2024-04-22T20:48:28.495078+0000","last_became_active":"2024-04-22T19:52:04.618273+0000","last_became_peered":"2024-04-22T19:52:04.618273+0000","last_unstale":"2024-04-22T20:48:28.495078+0000","last_undegraded":"2024-04-22T20:48:28.495078+0000","last_fullsized":"2024-04-22T20:48:28.495078+0000","mapping_epoch":33,"log_start":"261'200","ondisk_log_start":"261'200","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'287","last_scrub_stamp":"2024-04-22T20:48:28.494984+0000","last_deep_scrub":"261'287","last_deep_scrub_stamp":"2024-04-22T20:48:28.494984+0000","last_clean_scrub_stamp":"2024-04-22T20:48:28.494984+0000","objects_scrubbed":7,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T07:31:35.950376+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1439002,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":73466,"num_write":989,"num_write_kb":40708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"261'95","reported_seq":658,"reported_epoch":261,"state":"active+clean","last_fresh":"2024-04-22T20:48:42.938763+0000","last_change":"2024-04-22T20:48:42.938763+0000","last_active":"2024-04-22T20:48:42.938763+0000","last_peered":"2024-04-22T20:48:42.938763+0000","last_clean":"2024-04-22T20:48:42.938763+0000","last_became_active":"2024-04-22T19:51:28.513450+0000","last_became_peered":"2024-04-22T19:51:28.513450+0000","last_unstale":"2024-04-22T20:48:42.938763+0000","last_undegraded":"2024-04-22T20:48:42.938763+0000","last_fullsized":"2024-04-22T20:48:42.938763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'95","last_scrub_stamp":"2024-04-22T20:48:42.938629+0000","last_deep_scrub":"261'95","last_deep_scrub_stamp":"2024-04-22T20:48:42.938629+0000","last_clean_scrub_stamp":"2024-04-22T20:48:42.938629+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T03:02:38.964219+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":382,"num_read_kb":302,"num_write":152,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5385,"num_read_kb":3813,"num_write":2124,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7203,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1243,"ondisk_log_size":1243,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19082,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":13196,"num_read_kb":13004,"num_write":15956,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4092,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":663552,"data_stored":38164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1949,"ondisk_log_size":1949,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":378,"ondisk_log_size":378,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1476154,"num_objects":199,"num_object_clones":0,"num_object_copies":398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":17118,"num_read_kb":35541,"num_write":14403,"num_write_kb":11173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":650962,"num_omap_keys":1510,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1591,"ondisk_log_size":1591,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1622,"num_read_kb":1927,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":85960,"num_read_kb":69763,"num_write":34673,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43052,"num_omap_keys":180,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3910,"ondisk_log_size":3910,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":231670890,"num_objects":469,"num_object_clones":0,"num_object_copies":1407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":469,"num_whiteouts":0,"num_read":35099,"num_read_kb":2217057,"num_write":77336,"num_write_kb":2138644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":349753344,"data_stored":349753344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3387,"ondisk_log_size":3387,"up":192,"acting":192,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":169,"num_read_kb":196064,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":2,"acting":2,"num_store_stats":2},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":167,"num_read_kb":185784,"num_write":179,"num_write_kb":127402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575554,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":339124,"kb_used_data":51116,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":94032716,"statfs":{"total":96636764160,"available":96289501184,"internally_reserved":0,"allocated":52342784,"data_stored":50369134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":294910410},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575556,"num_pgs":100,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316900,"kb_used_data":46940,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":94054940,"statfs":{"total":96636764160,"available":96312258560,"internally_reserved":0,"allocated":48066560,"data_stored":46141552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":276429254},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575555,"num_pgs":104,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175548,"kb_used_data":54196,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94196292,"statfs":{"total":96636764160,"available":96457003008,"internally_reserved":0,"allocated":55496704,"data_stored":53487509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124254661},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575556,"num_pgs":105,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163576,"kb_used_data":51888,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":94208264,"statfs":{"total":96636764160,"available":96469262336,"internally_reserved":0,"allocated":53133312,"data_stored":51218557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575556,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":336520,"kb_used_data":62464,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94035320,"statfs":{"total":96636764160,"available":96292167680,"internally_reserved":0,"allocated":63963136,"data_stored":62012545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":280623560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575558,"num_pgs":90,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144372,"kb_used_data":59116,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94227468,"statfs":{"total":96636764160,"available":96488927232,"internally_reserved":0,"allocated":60534784,"data_stored":58629075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575554,"num_pgs":110,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":188908,"kb_used_data":53668,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94182932,"statfs":{"total":96636764160,"available":96443322368,"internally_reserved":0,"allocated":54956032,"data_stored":53018011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575556,"num_pgs":111,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309552,"kb_used_data":61800,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":94062288,"statfs":{"total":96636764160,"available":96319782912,"internally_reserved":0,"allocated":63283200,"data_stored":61318787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":253688262},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":48562176,"data_stored":48562176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44916736,"data_stored":44916736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":37486592,"data_stored":37486592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":50307072,"data_stored":50307072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":36368384,"data_stored":36368384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":44924928,"data_stored":44924928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":43872256,"data_stored":43872256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":43315200,"data_stored":43315200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":5948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":3716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":4195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":3595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":6129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":4904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3145728,"data_stored":3145728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13631488,"data_stored":13631488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9437184,"data_stored":9437184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-22T20:49:47.276 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-22T20:49:47.590 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-22T20:49:47.590 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-22T20:49:47.591 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-22T20:49:47.591 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.640 INFO:tasks.ceph.osd.0:Stopped 2024-04-22T20:49:47.640 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-22T20:49:47.640 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.681 INFO:tasks.ceph.osd.1:Stopped 2024-04-22T20:49:47.682 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-22T20:49:47.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.726 INFO:tasks.ceph.osd.2:Stopped 2024-04-22T20:49:47.726 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-22T20:49:47.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.778 INFO:tasks.ceph.osd.3:Stopped 2024-04-22T20:49:47.779 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-22T20:49:47.779 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.824 INFO:tasks.ceph.osd.4:Stopped 2024-04-22T20:49:47.824 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-22T20:49:47.824 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.870 INFO:tasks.ceph.osd.5:Stopped 2024-04-22T20:49:47.871 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-22T20:49:47.871 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.922 INFO:tasks.ceph.osd.6:Stopped 2024-04-22T20:49:47.922 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-22T20:49:47.922 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.974 INFO:tasks.ceph.osd.7:Stopped 2024-04-22T20:49:47.974 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-22T20:49:47.975 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-22T20:49:47.975 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:47.998 INFO:tasks.ceph.mgr.y:Stopped 2024-04-22T20:49:47.999 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-22T20:49:47.999 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:48.028 INFO:tasks.ceph.mgr.x:Stopped 2024-04-22T20:49:48.028 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-22T20:49:48.028 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-22T20:49:48.028 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:48.054 INFO:tasks.ceph.mon.a:Stopped 2024-04-22T20:49:48.055 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-22T20:49:48.055 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:48.107 INFO:tasks.ceph.mon.c:Stopped 2024-04-22T20:49:48.107 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-22T20:49:48.107 INFO:teuthology.orchestra.run:waiting for 300 2024-04-22T20:49:48.121 INFO:tasks.ceph.mon.b:Stopped 2024-04-22T20:49:48.121 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-22T20:49:48.121 DEBUG:teuthology.orchestra.run.smithi136:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2024-04-22T20:49:48.189 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T20:49:48.189 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-22T20:49:48.450 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T20:49:48.451 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-22T20:49:48.568 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T20:49:48.568 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-22T20:49:48.637 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T20:49:48.637 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-22T20:49:48.740 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T20:49:48.741 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-22T20:49:48.935 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T20:49:48.936 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-22T20:49:49.034 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T20:49:49.034 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-22T20:49:49.115 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T20:49:49.115 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-22T20:49:49.198 INFO:tasks.ceph:Archiving mon data... 2024-04-22T20:49:49.200 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/data/mon.a.tgz 2024-04-22T20:49:49.200 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-04-22T20:49:49.231 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.jTKE19emMV 2024-04-22T20:49:49.231 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.jTKE19emMV 2024-04-22T20:49:49.429 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.jTKE19emMV 2024-04-22T20:49:49.537 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.jTKE19emMV is 468KB 2024-04-22T20:49:49.635 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.jTKE19emMV 2024-04-22T20:49:49.662 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/data/mon.c.tgz 2024-04-22T20:49:49.662 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-04-22T20:49:49.729 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.XEbvsYeNjm 2024-04-22T20:49:49.730 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.XEbvsYeNjm 2024-04-22T20:49:49.902 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.XEbvsYeNjm 2024-04-22T20:49:50.007 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.XEbvsYeNjm is 469KB 2024-04-22T20:49:50.112 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.XEbvsYeNjm 2024-04-22T20:49:50.139 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/data/mon.b.tgz 2024-04-22T20:49:50.139 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-04-22T20:49:50.173 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.0UA6vw6yzY 2024-04-22T20:49:50.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.0UA6vw6yzY 2024-04-22T20:49:50.365 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.0UA6vw6yzY 2024-04-22T20:49:50.470 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.0UA6vw6yzY is 469KB 2024-04-22T20:49:50.568 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.0UA6vw6yzY 2024-04-22T20:49:50.595 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-22T20:49:50.596 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T20:49:50.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-22T20:49:50.775 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-22T20:49:50.803 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-22T20:49:50.803 INFO:tasks.ceph:Archiving crash dumps... 2024-04-22T20:49:50.805 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/crash to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/remote/smithi136/crash 2024-04-22T20:49:50.806 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T20:49:50.843 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/remote/smithi164/crash 2024-04-22T20:49:50.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-22T20:49:50.879 INFO:tasks.ceph:Compressing logs... 2024-04-22T20:49:50.879 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T20:49:50.888 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-22T20:49:50.984 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31249.log 2024-04-22T20:49:50.984 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-22T20:49:50.985 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-22T20:49:50.985 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.31249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31249.log.gz 2024-04-22T20:49:50.985 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-22T20:49:50.985 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-22T20:49:50.985 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-22T20:49:50.986 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31018.log 2024-04-22T20:49:50.986 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31205.log 2024-04-22T20:49:50.986 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.31018.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-22T20:49:50.986 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31018.log.gz 2024-04-22T20:49:50.987 INFO:teuthology.orchestra.run.smithi164.stderr: 90.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-22T20:49:50.987 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.31205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31205.log.gz 2024-04-22T20:49:50.987 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-22T20:49:50.987 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31110.log 2024-04-22T20:49:50.987 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31293.log 2024-04-22T20:49:50.988 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.31110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31110.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31062.log 2024-04-22T20:49:50.988 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-22T20:49:50.988 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.31293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31293.log.gz 2024-04-22T20:49:50.989 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31337.log 2024-04-22T20:49:50.989 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-22T20:49:50.989 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.31337.log: /var/log/ceph/ceph-client.admin.31062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31062.log.gz 2024-04-22T20:49:50.989 INFO:teuthology.orchestra.run.smithi164.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31337.log.gz 2024-04-22T20:49:50.990 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31161.log 2024-04-22T20:49:50.990 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.31161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31161.log.gz 2024-04-22T20:49:50.990 INFO:teuthology.orchestra.run.smithi164.stderr: 87.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-22T20:49:51.006 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69212.log 2024-04-22T20:49:51.006 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143416.log 2024-04-22T20:49:51.006 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.69212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112323.log 2024-04-22T20:49:51.006 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31611.log 2024-04-22T20:49:51.007 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143416.log.gz 2024-04-22T20:49:51.007 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.112323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125649.log 2024-04-22T20:49:51.007 INFO:teuthology.orchestra.run.smithi136.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.112323.log.gz 2024-04-22T20:49:51.007 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.31611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31611.log.gz 2024-04-22T20:49:51.007 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37059.log 2024-04-22T20:49:51.007 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95240.log 2024-04-22T20:49:51.007 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.125649.log: /var/log/ceph/ceph-client.admin.37059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2024-04-22T20:49:51.007 INFO:teuthology.orchestra.run.smithi136.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69212.log.gz 2024-04-22T20:49:51.008 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37059.log.gz 2024-04-22T20:49:51.008 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.95240.log: gzip -5 92.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.125649.log.gz 2024-04-22T20:49:51.008 INFO:teuthology.orchestra.run.smithi136.stderr: /var/log/ceph/ceph-client.admin.101130.log 2024-04-22T20:49:51.008 INFO:teuthology.orchestra.run.smithi136.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.95240.log.gz 2024-04-22T20:49:51.008 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55765.log 2024-04-22T20:49:51.008 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2024-04-22T20:49:51.009 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.101130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143876.log 2024-04-22T20:49:51.009 INFO:teuthology.orchestra.run.smithi136.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.101130.log.gz 2024-04-22T20:49:51.009 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.55765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104664.log 2024-04-22T20:49:51.009 INFO:teuthology.orchestra.run.smithi136.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.55765.log.gz 2024-04-22T20:49:51.009 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104075.log 2024-04-22T20:49:51.009 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143876.log.gz 2024-04-22T20:49:51.010 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.104664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126865.log 2024-04-22T20:49:51.010 INFO:teuthology.orchestra.run.smithi136.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.104664.log.gz 2024-04-22T20:49:51.010 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40874.log 2024-04-22T20:49:51.010 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.104075.log: /var/log/ceph/ceph-client.admin.126865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61608.log 2024-04-22T20:49:51.010 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.40874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91706.log 2024-04-22T20:49:51.010 INFO:teuthology.orchestra.run.smithi136.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.126865.log.gz 2024-04-22T20:49:51.010 INFO:teuthology.orchestra.run.smithi136.stderr: 43.9% -- replaced with /var/log/ceph/ceph-client.0.40874.log.gz 2024-04-22T20:49:51.010 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.61608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51053.log 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.91706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144060.log 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.91706.log.gz 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.104075.log.gz 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.51053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39053.log 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48012.log 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144060.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.51053.log.gz 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144060.log.gz 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.39053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107610.log 2024-04-22T20:49:51.011 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.48012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38533.log 2024-04-22T20:49:51.012 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48012.log.gz 2024-04-22T20:49:51.012 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.107610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37318.log 2024-04-22T20:49:51.012 INFO:teuthology.orchestra.run.smithi136.stderr: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.107610.log.gz 2024-04-22T20:49:51.012 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38533.log: 95.8% -- replaced with /var/log/ceph/ceph-client.0.39053.log.gz 2024-04-22T20:49:51.012 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.38533.log.gz /var/log/ceph/ceph-client.admin.56354.log 2024-04-22T20:49:51.012 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-22T20:49:51.012 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98774.log 2024-04-22T20:49:51.012 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37318.log.gz 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.56354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113501.log 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56354.log.gz 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.98774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94062.log 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.98774.log.gz 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.113501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121276.log 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.113501.log.gz 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.94062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98185.log 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.94062.log.gz 2024-04-22T20:49:51.013 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.121276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57580.log 2024-04-22T20:49:51.014 INFO:teuthology.orchestra.run.smithi136.stderr: 81.6%/var/log/ceph/ceph-client.admin.98185.log: -- replaced with /var/log/ceph/ceph-client.admin.121276.log.gz 2024-04-22T20:49:51.014 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128229.log 2024-04-22T20:49:51.014 INFO:teuthology.orchestra.run.smithi136.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.98185.log.gz 2024-04-22T20:49:51.014 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.57580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.136059.log 2024-04-22T20:49:51.014 INFO:teuthology.orchestra.run.smithi136.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.57580.log.gz 2024-04-22T20:49:51.014 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.128229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2024-04-22T20:49:51.014 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.136059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46245.log 2024-04-22T20:49:51.014 INFO:teuthology.orchestra.run.smithi136.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.128229.log.gz 2024-04-22T20:49:51.015 INFO:teuthology.orchestra.run.smithi136.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.0.136059.log.gz/var/log/ceph/ceph-client.admin.37316.log: 2024-04-22T20:49:51.015 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz 2024-04-22T20:49:51.015 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99363.log 2024-04-22T20:49:51.015 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.46245.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-22T20:49:51.015 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.99363.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.46245.log.gz 2024-04-22T20:49:51.015 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64268.log 2024-04-22T20:49:51.015 INFO:teuthology.orchestra.run.smithi136.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.99363.log.gz 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37682.log 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.64268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37519.log 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71601.log 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37682.log.gz 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37519.log.gz 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143922.log 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.139104.log 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.71601.log: /var/log/ceph/ceph-client.admin.143922.log: gzip -5 --verbose -- 80.7% /var/log/ceph/ceph-client.admin.45656.log 2024-04-22T20:49:51.016 INFO:teuthology.orchestra.run.smithi136.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71601.log.gz 2024-04-22T20:49:51.017 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143922.log.gz 2024-04-22T20:49:51.017 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.139104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37312.log 2024-04-22T20:49:51.017 INFO:teuthology.orchestra.run.smithi136.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.0.139104.log.gz 2024-04-22T20:49:51.017 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.45656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143554.log 2024-04-22T20:49:51.017 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37312.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.45656.log.gz 2024-04-22T20:49:51.017 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37312.log.gz 2024-04-22T20:49:51.017 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101719.log 2024-04-22T20:49:51.017 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120050.log 2024-04-22T20:49:51.018 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143554.log.gz 2024-04-22T20:49:51.018 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.101719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73963.log 2024-04-22T20:49:51.018 INFO:teuthology.orchestra.run.smithi136.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.101719.log.gz 2024-04-22T20:49:51.018 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.120050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61454.log 2024-04-22T20:49:51.018 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120050.log.gz 2024-04-22T20:49:51.018 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.73963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37685.log 2024-04-22T20:49:51.018 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.61454.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.73963.log.gz 2024-04-22T20:49:51.018 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37315.log 2024-04-22T20:49:51.019 INFO:teuthology.orchestra.run.smithi136.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61454.log.gz 2024-04-22T20:49:51.019 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37935.log 2024-04-22T20:49:51.019 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37685.log.gz 2024-04-22T20:49:51.019 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-22T20:49:51.019 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37315.log.gz 2024-04-22T20:49:51.019 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37105.log 2024-04-22T20:49:51.019 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37935.log.gz 2024-04-22T20:49:51.020 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87583.log 2024-04-22T20:49:51.020 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36967.log 2024-04-22T20:49:51.020 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37105.log.gz 2024-04-22T20:49:51.020 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.87583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95829.log 2024-04-22T20:49:51.020 INFO:teuthology.orchestra.run.smithi136.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.64268.log.gz 2024-04-22T20:49:51.020 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37013.log 2024-04-22T20:49:51.020 INFO:teuthology.orchestra.run.smithi136.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.87583.log.gz 2024-04-22T20:49:51.021 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.36967.log: /var/log/ceph/ceph-client.admin.95829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36967.log.gz 2024-04-22T20:49:51.021 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.140322.log 2024-04-22T20:49:51.021 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37013.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61608.log.gz 2024-04-22T20:49:51.021 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38123.log 2024-04-22T20:49:51.021 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37013.log.gz 2024-04-22T20:49:51.021 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.140322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105843.log 2024-04-22T20:49:51.021 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38123.log.gz 2024-04-22T20:49:51.021 INFO:teuthology.orchestra.run.smithi136.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.0.140322.log.gz 2024-04-22T20:49:51.022 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.138495.log 2024-04-22T20:49:51.022 INFO:teuthology.orchestra.run.smithi136.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.95829.log.gz 2024-04-22T20:49:51.022 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129445.log 2024-04-22T20:49:51.022 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.105843.log: /var/log/ceph/ceph-client.0.138495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125041.log 2024-04-22T20:49:51.022 INFO:teuthology.orchestra.run.smithi136.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.105843.log.gz 2024-04-22T20:49:51.022 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.129445.log: 83.9% -- replaced with /var/log/ceph/ceph-client.0.138495.log.gz 2024-04-22T20:49:51.022 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81693.log 2024-04-22T20:49:51.022 INFO:teuthology.orchestra.run.smithi136.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.129445.log.gz 2024-04-22T20:49:51.023 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134841.log 2024-04-22T20:49:51.023 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.125041.log: /var/log/ceph/ceph-client.admin.81693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78747.log 2024-04-22T20:49:51.023 INFO:teuthology.orchestra.run.smithi136.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.81693.log.gz 2024-04-22T20:49:51.023 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.134841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48649.log 2024-04-22T20:49:51.023 INFO:teuthology.orchestra.run.smithi136.stderr:gzip 84.5% -5 -- replaced with /var/log/ceph/ceph-client.0.134841.log.gz --verbose 2024-04-22T20:49:51.023 INFO:teuthology.orchestra.run.smithi136.stderr: -- /var/log/ceph/ceph.audit.log 2024-04-22T20:49:51.024 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.78747.log: /var/log/ceph/ceph-client.admin.48649.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.78747.log.gz 2024-04-22T20:49:51.024 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144198.log 2024-04-22T20:49:51.024 INFO:teuthology.orchestra.run.smithi136.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.48649.log.gz 2024-04-22T20:49:51.024 INFO:teuthology.orchestra.run.smithi136.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.125041.log.gz 2024-04-22T20:49:51.024 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69830.log 2024-04-22T20:49:51.024 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.144198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79336.log 2024-04-22T20:49:51.025 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144198.log.gz 2024-04-22T20:49:51.025 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-22T20:49:51.025 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.69830.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25105.log 2024-04-22T20:49:51.025 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.79336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97596.log 2024-04-22T20:49:51.026 INFO:teuthology.orchestra.run.smithi136.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.79336.log.gz 2024-04-22T20:49:51.026 INFO:teuthology.orchestra.run.smithi136.stderr:gzip/var/log/ceph/ceph.tmp-client.admin.25105.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25105.log.gz 2024-04-22T20:49:51.026 INFO:teuthology.orchestra.run.smithi136.stderr: /var/log/ceph/ceph-client.admin.65504.log 2024-04-22T20:49:51.026 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.97596.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.97596.log.gz 2024-04-22T20:49:51.026 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58806.log 2024-04-22T20:49:51.027 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37969.log 2024-04-22T20:49:51.027 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.65504.log: /var/log/ceph/ceph-client.admin.58806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37683.log 2024-04-22T20:49:51.027 INFO:teuthology.orchestra.run.smithi136.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.58806.log.gz 2024-04-22T20:49:51.027 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86994.log 2024-04-22T20:49:51.027 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37969.log.gz 2024-04-22T20:49:51.027 INFO:teuthology.orchestra.run.smithi136.stderr: 94.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-22T20:49:51.027 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37319.log 2024-04-22T20:49:51.027 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37683.log.gz 2024-04-22T20:49:51.028 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.86994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132405.log 2024-04-22T20:49:51.028 INFO:teuthology.orchestra.run.smithi136.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.86994.log.gz 2024-04-22T20:49:51.028 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose 91.9% -- /var/log/ceph/ceph-client.admin.59984.log -- replaced with /var/log/ceph/ceph-client.admin.69830.log.gz/var/log/ceph/ceph-client.admin.37319.log: 2024-04-22T20:49:51.028 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-22T20:49:51.028 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37319.log.gz 2024-04-22T20:49:51.028 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.132405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37314.log 2024-04-22T20:49:51.028 INFO:teuthology.orchestra.run.smithi136.stderr: 80.5% -- replaced with /var/log/ceph/ceph-client.0.132405.log.gz 2024-04-22T20:49:51.028 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.59984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78158.log 2024-04-22T20:49:51.029 INFO:teuthology.orchestra.run.smithi136.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.59984.log.gz 2024-04-22T20:49:51.029 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37314.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.89350.log 2024-04-22T20:49:51.029 INFO:teuthology.orchestra.run.smithi136.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37314.log.gz 2024-04-22T20:49:51.029 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.78158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130549.log 2024-04-22T20:49:51.029 INFO:teuthology.orchestra.run.smithi136.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.78158.log.gz 2024-04-22T20:49:51.029 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.89350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133014.log 2024-04-22T20:49:51.030 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116468.log 2024-04-22T20:49:51.030 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.130549.log: /var/log/ceph/ceph-client.0.133014.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-22T20:49:51.030 INFO:teuthology.orchestra.run.smithi136.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.0.133014.log.gz 2024-04-22T20:49:51.030 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.116468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43292.log 2024-04-22T20:49:51.031 INFO:teuthology.orchestra.run.smithi136.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.116468.log.gz 2024-04-22T20:49:51.031 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109377.log 2024-04-22T20:49:51.031 INFO:teuthology.orchestra.run.smithi136.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.65504.log.gz 2024-04-22T20:49:51.031 INFO:teuthology.orchestra.run.smithi136.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.89350.log.gz 2024-04-22T20:49:51.032 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.43292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48060.log 2024-04-22T20:49:51.032 INFO:teuthology.orchestra.run.smithi136.stderr: 43.5% -- replaced with /var/log/ceph/ceph-client.0.43292.log.gz 2024-04-22T20:49:51.032 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.109377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41477.log 2024-04-22T20:49:51.033 INFO:teuthology.orchestra.run.smithi136.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.109377.log.gz 2024-04-22T20:49:51.033 INFO:teuthology.orchestra.run.smithi136.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.130549.log.gz 2024-04-22T20:49:51.033 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.48060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38441.log 2024-04-22T20:49:51.033 INFO:teuthology.orchestra.run.smithi136.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.48060.log.gz 2024-04-22T20:49:51.034 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.41477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38395.log 2024-04-22T20:49:51.034 INFO:teuthology.orchestra.run.smithi136.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.0.41477.log.gz 2024-04-22T20:49:51.034 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99952.log 2024-04-22T20:49:51.034 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38441.log.gz 2024-04-22T20:49:51.034 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53409.log 2024-04-22T20:49:51.034 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38395.log.gz 2024-04-22T20:49:51.034 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.99952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144106.log 2024-04-22T20:49:51.035 INFO:teuthology.orchestra.run.smithi136.stderr: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.99952.log.gz 2024-04-22T20:49:51.035 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.53409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38942.log 2024-04-22T20:49:51.035 INFO:teuthology.orchestra.run.smithi136.stderr: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.53409.log.gz 2024-04-22T20:49:51.035 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38487.log 2024-04-22T20:49:51.035 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144106.log.gz 2024-04-22T20:49:51.036 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.136668.log 2024-04-22T20:49:51.036 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38942.log.gz 2024-04-22T20:49:51.036 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108788.log 2024-04-22T20:49:51.036 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38487.log.gz 2024-04-22T20:49:51.036 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.136668.log: gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2024-04-22T20:49:51.036 INFO:teuthology.orchestra.run.smithi136.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.0.136668.log.gz 2024-04-22T20:49:51.036 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.108788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110555.log 2024-04-22T20:49:51.036 INFO:teuthology.orchestra.run.smithi136.stderr: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.108788.log.gz 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ops-log-ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37317.log 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.110555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96418.log 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.110555.log.gz 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38122.log 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37317.log.gz 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.96418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103486.log 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.96418.log.gz 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111734.log 2024-04-22T20:49:51.037 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38122.log.gz 2024-04-22T20:49:51.038 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.103486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58782.log 2024-04-22T20:49:51.038 INFO:teuthology.orchestra.run.smithi136.stderr: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.103486.log.gz 2024-04-22T20:49:51.038 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.111734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.140931.log 2024-04-22T20:49:51.038 INFO:teuthology.orchestra.run.smithi136.stderr: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.111734.log.gz 2024-04-22T20:49:51.038 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.58782.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 0.0% 2024-04-22T20:49:51.038 INFO:teuthology.orchestra.run.smithi136.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58782.log.gz 2024-04-22T20:49:51.038 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.140931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94651.log 2024-04-22T20:49:51.039 INFO:teuthology.orchestra.run.smithi136.stderr: 80.2% -- replaced with /var/log/ceph/ceph-client.0.140931.log.gz 2024-04-22T20:49:51.039 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143646.log 2024-04-22T20:49:51.039 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-22T20:49:51.039 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.94651.log: /var/log/ceph/ceph-client.admin.143646.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.94651.log.gz 2024-04-22T20:49:51.039 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107021.log 2024-04-22T20:49:51.039 INFO:teuthology.orchestra.run.smithi136.stderr: -- replaced with /var/log/ceph/ceph-client.admin.143646.log.gz 2024-04-22T20:49:51.040 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-22T20:49:51.040 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.107021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84049.log 2024-04-22T20:49:51.040 INFO:teuthology.orchestra.run.smithi136.stderr: 84.9%/var/log/ceph/ceph-mgr.y.log: -- replaced with /var/log/ceph/ceph-client.admin.107021.log.gz 2024-04-22T20:49:51.040 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76703.log 2024-04-22T20:49:51.040 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-22T20:49:51.040 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.84049.log: /var/log/ceph/ceph-client.0.76703.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.84049.log.gz 2024-04-22T20:49:51.040 INFO:teuthology.orchestra.run.smithi136.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.0.76703.log.gz 2024-04-22T20:49:51.041 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144336.log 2024-04-22T20:49:51.041 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108199.log 2024-04-22T20:49:51.041 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144336.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144336.log.gz 2024-04-22T20:49:51.041 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127621.log 2024-04-22T20:49:51.041 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.108199.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.108199.log.gz 2024-04-22T20:49:51.041 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133623.log 2024-04-22T20:49:51.041 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.127621.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.127621.log.gz 2024-04-22T20:49:51.042 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80515.log 2024-04-22T20:49:51.042 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.133623.log: 80.3% -- replaced with /var/log/ceph/ceph-client.0.133623.log.gz 2024-04-22T20:49:51.042 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97007.log 2024-04-22T20:49:51.042 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.80515.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.80515.log.gz 2024-04-22T20:49:51.042 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144152.log 2024-04-22T20:49:51.042 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.97007.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.97007.log.gz 2024-04-22T20:49:51.042 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58169.log 2024-04-22T20:49:51.042 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144152.log.gz 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38579.log 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.58169.log: 80.5% -- replaced with /var/log/ceph/ceph-client.admin.58169.log.gz 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143738.log 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38579.log.gz 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77569.log 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143738.log.gz 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126257.log 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.77569.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.77569.log.gz 2024-04-22T20:49:51.043 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81104.log 2024-04-22T20:49:51.044 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.126257.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.126257.log.gz 2024-04-22T20:49:51.044 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49827.log 2024-04-22T20:49:51.044 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.81104.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.81104.log.gz 2024-04-22T20:49:51.044 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144290.log 2024-04-22T20:49:51.044 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.49827.log: 80.2% -- replaced with /var/log/ceph/ceph-client.admin.49827.log.gz 2024-04-22T20:49:51.044 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61137.log 2024-04-22T20:49:51.044 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144290.log.gz 2024-04-22T20:49:51.044 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-22T20:49:51.045 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.61137.log: 57.3% -- replaced with /var/log/ceph/ceph-client.admin.61137.log.gz 2024-04-22T20:49:51.045 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88172.log 2024-04-22T20:49:51.045 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/ceph.client.0.log 2024-04-22T20:49:51.045 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.88172.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.88172.log.gz 2024-04-22T20:49:51.045 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61546.log 2024-04-22T20:49:51.045 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/valgrind/ceph.client.0.log: 62.4% -- replaced with /var/log/ceph/valgrind/ceph.client.0.log.gz 2024-04-22T20:49:51.045 INFO:teuthology.orchestra.run.smithi136.stderr: 95.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-22T20:49:51.045 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38986.log 2024-04-22T20:49:51.046 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.61546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105254.log 2024-04-22T20:49:51.046 INFO:teuthology.orchestra.run.smithi136.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61546.log.gz 2024-04-22T20:49:51.046 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38986.log.gz 2024-04-22T20:49:51.055 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38303.log 2024-04-22T20:49:51.055 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.105254.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.105254.log.gz 2024-04-22T20:49:51.055 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122639.log 2024-04-22T20:49:51.055 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38303.log.gz 2024-04-22T20:49:51.056 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46834.log 2024-04-22T20:49:51.056 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.122639.log: 84.1% -- replaced with /var/log/ceph/ceph-client.0.122639.log.gz 2024-04-22T20:49:51.056 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143462.log 2024-04-22T20:49:51.056 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.46834.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.46834.log.gz 2024-04-22T20:49:51.057 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52820.log 2024-04-22T20:49:51.057 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143462.log.gz 2024-04-22T20:49:51.057 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143508.log 2024-04-22T20:49:51.057 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.52820.log: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.52820.log.gz 2024-04-22T20:49:51.057 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114090.log 2024-04-22T20:49:51.057 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143508.log.gz 2024-04-22T20:49:51.057 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76094.log 2024-04-22T20:49:51.058 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.114090.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.114090.log.gz 2024-04-22T20:49:51.058 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-22T20:49:51.058 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.76094.log: 92.1% -- replaced with /var/log/ceph/ceph-client.0.76094.log.gz 2024-04-22T20:49:51.058 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60882.log 2024-04-22T20:49:51.058 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115290.log 2024-04-22T20:49:51.058 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.60882.log: 59.5% -- replaced with /var/log/ceph/ceph-client.admin.60882.log.gz 2024-04-22T20:49:51.058 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115879.log 2024-04-22T20:49:51.059 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.115290.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.115290.log.gz 2024-04-22T20:49:51.060 INFO:teuthology.orchestra.run.smithi136.stderr: 94.4% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-04-22T20:49:51.061 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73372.log 2024-04-22T20:49:51.061 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.115879.log: 84.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.115879.log.gz 2024-04-22T20:49:51.061 INFO:teuthology.orchestra.run.smithi136.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.143784.log 2024-04-22T20:49:51.068 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.73372.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.73372.log.gz 2024-04-22T20:49:51.068 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53998.log 2024-04-22T20:49:51.068 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143784.log.gz 2024-04-22T20:49:51.068 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82871.log 2024-04-22T20:49:51.069 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.53998.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.53998.log.gz 2024-04-22T20:49:51.069 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54587.log 2024-04-22T20:49:51.069 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.82871.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.82871.log.gz 2024-04-22T20:49:51.069 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.139713.log 2024-04-22T20:49:51.069 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.54587.log: 79.3% -- replaced with /var/log/ceph/ceph-client.admin.54587.log.gz 2024-04-22T20:49:51.069 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.admin.log 2024-04-22T20:49:51.069 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.139713.log: 83.5% -- replaced with /var/log/ceph/ceph-client.0.139713.log.gz 2024-04-22T20:49:51.070 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.131796.log 2024-04-22T20:49:51.075 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ops-log-ceph-client.admin.log: 94.3% -- replaced with /var/log/ceph/ops-log-ceph-client.admin.log.gz 2024-04-22T20:49:51.075 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92295.log 2024-04-22T20:49:51.075 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.131796.log: 83.8% -- replaced with /var/log/ceph/ceph-client.0.131796.log.gz 2024-04-22T20:49:51.076 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85816.log 2024-04-22T20:49:51.076 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.92295.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.92295.log.gz 2024-04-22T20:49:51.076 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67358.log 2024-04-22T20:49:51.076 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.85816.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.85816.log.gz 2024-04-22T20:49:51.076 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66740.log 2024-04-22T20:49:51.084 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.67358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42689.log 2024-04-22T20:49:51.098 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.66740.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.66740.log.gz 2024-04-22T20:49:51.101 INFO:teuthology.orchestra.run.smithi136.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.67358.log.gz 2024-04-22T20:49:51.113 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102308.log 2024-04-22T20:49:51.113 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.42689.log: 43.5% -- replaced with /var/log/ceph/ceph-client.0.42689.log.gz 2024-04-22T20:49:51.113 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50440.log 2024-04-22T20:49:51.114 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.102308.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.102308.log.gz 2024-04-22T20:49:51.114 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39030.log 2024-04-22T20:49:51.114 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.50440.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50440.log.gz 2024-04-22T20:49:51.114 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120687.log 2024-04-22T20:49:51.114 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.39030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39030.log.gz 2024-04-22T20:49:51.114 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38121.log 2024-04-22T20:49:51.114 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.120687.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.120687.log.gz 2024-04-22T20:49:51.115 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70493.log 2024-04-22T20:49:51.115 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38121.log.gz 2024-04-22T20:49:51.115 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90528.log 2024-04-22T20:49:51.115 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.70493.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.70493.log.gz 2024-04-22T20:49:51.115 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86405.log 2024-04-22T20:49:51.115 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.90528.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.90528.log.gz 2024-04-22T20:49:51.115 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61734.log 2024-04-22T20:49:51.115 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.86405.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.86405.log.gz 2024-04-22T20:49:51.116 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.142561.log 2024-04-22T20:49:51.116 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.61734.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61734.log.gz 2024-04-22T20:49:51.116 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82282.log 2024-04-22T20:49:51.130 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.142561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36923.log 2024-04-22T20:49:51.130 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.82282.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.82282.log.gz 2024-04-22T20:49:51.145 INFO:teuthology.orchestra.run.smithi136.stderr: 86.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67976.log 2024-04-22T20:49:51.145 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.36923.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36923.log.gz 2024-04-22T20:49:51.145 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51642.log 2024-04-22T20:49:51.167 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.67976.log: 92.8% -- replaced with /var/log/ceph/ceph-client.0.142561.log.gz 2024-04-22T20:49:51.167 INFO:teuthology.orchestra.run.smithi136.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67976.log.gz 2024-04-22T20:49:51.167 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143968.log 2024-04-22T20:49:51.168 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.51642.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.51642.log.gz 2024-04-22T20:49:51.168 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143692.log 2024-04-22T20:49:51.168 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143968.log.gz 2024-04-22T20:49:51.168 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143830.log 2024-04-22T20:49:51.168 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143692.log.gz 2024-04-22T20:49:51.168 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117646.log 2024-04-22T20:49:51.168 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143830.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143830.log.gz 2024-04-22T20:49:51.169 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118235.log 2024-04-22T20:49:51.169 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.117646.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.117646.log.gz 2024-04-22T20:49:51.169 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74723.log 2024-04-22T20:49:51.169 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.118235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118235.log.gz 2024-04-22T20:49:51.169 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120098.log 2024-04-22T20:49:51.169 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.74723.log: 84.0% -- replaced with /var/log/ceph/ceph-client.0.74723.log.gz 2024-04-22T20:49:51.169 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48036.log 2024-04-22T20:49:51.170 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.120098.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120098.log.gz 2024-04-22T20:49:51.170 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38117.log 2024-04-22T20:49:51.170 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.48036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48036.log.gz 2024-04-22T20:49:51.170 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102897.log 2024-04-22T20:49:51.170 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38117.log.gz 2024-04-22T20:49:51.170 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115269.log 2024-04-22T20:49:51.170 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.102897.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.102897.log.gz 2024-04-22T20:49:51.171 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85227.log 2024-04-22T20:49:51.171 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.115269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115269.log.gz 2024-04-22T20:49:51.171 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31175.log 2024-04-22T20:49:51.171 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.85227.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.85227.log.gz 2024-04-22T20:49:51.171 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61248.log 2024-04-22T20:49:51.171 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.31175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31175.log.gz 2024-04-22T20:49:51.172 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89939.log 2024-04-22T20:49:51.172 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.61248.log: 50.2% -- replaced with /var/log/ceph/ceph-client.admin.61248.log.gz 2024-04-22T20:49:51.172 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52231.log 2024-04-22T20:49:51.172 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.89939.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.89939.log.gz 2024-04-22T20:49:51.172 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88761.log 2024-04-22T20:49:51.172 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.52231.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.52231.log.gz 2024-04-22T20:49:51.173 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62267.log 2024-04-22T20:49:51.173 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.88761.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.88761.log.gz 2024-04-22T20:49:51.177 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75332.log 2024-04-22T20:49:51.178 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.62267.log: 21.6% -- replaced with /var/log/ceph/ceph-client.admin.62267.log.gz 2024-04-22T20:49:51.178 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72782.log 2024-04-22T20:49:51.178 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.75332.log: 84.4% -- replaced with /var/log/ceph/ceph-client.0.75332.log.gz 2024-04-22T20:49:51.178 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63032.log 2024-04-22T20:49:51.178 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.72782.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72782.log.gz 2024-04-22T20:49:51.178 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117057.log 2024-04-22T20:49:51.186 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.63032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131157.log 2024-04-22T20:49:51.186 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.117057.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.117057.log.gz 2024-04-22T20:49:51.189 INFO:teuthology.orchestra.run.smithi136.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.63032.log.gz 2024-04-22T20:49:51.189 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111145.log 2024-04-22T20:49:51.191 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.131157.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.131157.log.gz 2024-04-22T20:49:51.191 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64886.log 2024-04-22T20:49:51.192 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.111145.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.111145.log.gz 2024-04-22T20:49:51.192 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91117.log 2024-04-22T20:49:51.199 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.64886.log: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.64886.log.gz 2024-04-22T20:49:51.199 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118872.log 2024-04-22T20:49:51.199 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.91117.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.91117.log.gz 2024-04-22T20:49:51.199 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62443.log 2024-04-22T20:49:51.199 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.118872.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.118872.log.gz 2024-04-22T20:49:51.201 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109966.log 2024-04-22T20:49:51.201 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.62443.log: 84.1% -- replaced with /var/log/ceph/ceph-client.admin.62443.log.gz 2024-04-22T20:49:51.201 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59395.log 2024-04-22T20:49:51.203 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.109966.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.109966.log.gz 2024-04-22T20:49:51.203 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120074.log 2024-04-22T20:49:51.210 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.59395.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.59395.log.gz 2024-04-22T20:49:51.210 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144382.log 2024-04-22T20:49:51.210 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.120074.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120074.log.gz 2024-04-22T20:49:51.210 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137886.log 2024-04-22T20:49:51.210 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144382.log.gz 2024-04-22T20:49:51.211 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58758.log 2024-04-22T20:49:51.211 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.137886.log: 83.8% -- replaced with /var/log/ceph/ceph-client.0.137886.log.gz 2024-04-22T20:49:51.211 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56378.log 2024-04-22T20:49:51.211 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.58758.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58758.log.gz 2024-04-22T20:49:51.211 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79925.log 2024-04-22T20:49:51.211 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.56378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56378.log.gz 2024-04-22T20:49:51.211 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100541.log 2024-04-22T20:49:51.224 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.79925.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.79925.log.gz 2024-04-22T20:49:51.224 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76042.log 2024-04-22T20:49:51.224 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.100541.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.100541.log.gz 2024-04-22T20:49:51.224 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37686.log 2024-04-22T20:49:51.224 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.76042.log: 50.1% -- replaced with /var/log/ceph/ceph-client.admin.76042.log.gz 2024-04-22T20:49:51.224 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144472.log 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37686.log.gz 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47423.log 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144472.log.gz 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37681.log 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.47423.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.47423.log.gz 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144244.log 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37681.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37681.log.gz 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93473.log 2024-04-22T20:49:51.225 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144244.log.gz 2024-04-22T20:49:51.226 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43895.log 2024-04-22T20:49:51.226 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.93473.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.93473.log.gz 2024-04-22T20:49:51.226 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106432.log 2024-04-22T20:49:51.226 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.43895.log: 44.2% -- replaced with /var/log/ceph/ceph-client.0.43895.log.gz 2024-04-22T20:49:51.226 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50464.log 2024-04-22T20:49:51.226 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.106432.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.106432.log.gz 2024-04-22T20:49:51.226 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68594.log 2024-04-22T20:49:51.227 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.50464.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.50464.log.gz 2024-04-22T20:49:51.227 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119461.log 2024-04-22T20:49:51.240 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.68594.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68594.log.gz 2024-04-22T20:49:51.240 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.141868.log 2024-04-22T20:49:51.240 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.119461.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.119461.log.gz 2024-04-22T20:49:51.240 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38898.log 2024-04-22T20:49:51.240 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.141868.log: 80.4% -- replaced with /var/log/ceph/ceph-client.0.141868.log.gz 2024-04-22T20:49:51.240 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38120.log 2024-04-22T20:49:51.241 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38898.log.gz 2024-04-22T20:49:51.241 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37313.log 2024-04-22T20:49:51.241 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38120.log.gz 2024-04-22T20:49:51.241 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61672.log 2024-04-22T20:49:51.241 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37313.log.gz 2024-04-22T20:49:51.241 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49238.log 2024-04-22T20:49:51.241 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.61672.log: 59.2% -- replaced with /var/log/ceph/ceph-client.admin.61672.log.gz 2024-04-22T20:49:51.241 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55176.log 2024-04-22T20:49:51.242 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.49238.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.49238.log.gz 2024-04-22T20:49:51.242 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123899.log 2024-04-22T20:49:51.242 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.55176.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.55176.log.gz 2024-04-22T20:49:51.242 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.135450.log 2024-04-22T20:49:51.242 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.123899.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.123899.log.gz 2024-04-22T20:49:51.242 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137277.log 2024-04-22T20:49:51.242 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.135450.log: 80.3% -- replaced with /var/log/ceph/ceph-client.0.135450.log.gz 2024-04-22T20:49:51.242 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39662.log 2024-04-22T20:49:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.137277.log: 80.4% -- replaced with /var/log/ceph/ceph-client.0.137277.log.gz 2024-04-22T20:49:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143370.log 2024-04-22T20:49:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.39662.log: 43.5% -- replaced with /var/log/ceph/ceph-client.0.39662.log.gz 2024-04-22T20:49:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56402.log 2024-04-22T20:49:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143370.log.gz 2024-04-22T20:49:51.261 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74554.log 2024-04-22T20:49:51.261 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.56402.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.56402.log.gz 2024-04-22T20:49:51.261 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112912.log 2024-04-22T20:49:51.261 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.74554.log: 25.1% -- replaced with /var/log/ceph/ceph-client.admin.74554.log.gz 2024-04-22T20:49:51.262 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38116.log 2024-04-22T20:49:51.262 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.112912.log: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.112912.log.gz 2024-04-22T20:49:51.262 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31508.log 2024-04-22T20:49:51.262 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38116.log.gz 2024-04-22T20:49:51.262 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37684.log 2024-04-22T20:49:51.262 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.31508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31508.log.gz 2024-04-22T20:49:51.262 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92884.log 2024-04-22T20:49:51.263 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37684.log.gz 2024-04-22T20:49:51.263 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134232.log 2024-04-22T20:49:51.263 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.92884.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.92884.log.gz 2024-04-22T20:49:51.263 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56991.log 2024-04-22T20:49:51.263 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.134232.log: 83.1% -- replaced with /var/log/ceph/ceph-client.0.134232.log.gz 2024-04-22T20:49:51.263 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37687.log 2024-04-22T20:49:51.263 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.56991.log: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.56991.log.gz 2024-04-22T20:49:51.264 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118259.log 2024-04-22T20:49:51.264 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.37687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37687.log.gz 2024-04-22T20:49:51.264 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72190.log 2024-04-22T20:49:51.264 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.118259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118259.log.gz 2024-04-22T20:49:51.264 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40265.log 2024-04-22T20:49:51.264 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.72190.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.72190.log.gz 2024-04-22T20:49:51.264 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38854.log 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.40265.log: 80.7% -- replaced with /var/log/ceph/ceph-client.0.40265.log.gz 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118283.log 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38854.log.gz 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83460.log 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.118283.log: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.118283.log.gz 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143324.log 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.83460.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.83460.log.gz 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123291.log 2024-04-22T20:49:51.265 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143324.log.gz 2024-04-22T20:49:51.266 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38349.log 2024-04-22T20:49:51.277 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.123291.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123291.log.gz 2024-04-22T20:49:51.278 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144014.log 2024-04-22T20:49:51.278 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.38349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38349.log.gz 2024-04-22T20:49:51.278 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36859.log 2024-04-22T20:49:51.278 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144014.log.gz 2024-04-22T20:49:51.278 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128837.log 2024-04-22T20:49:51.278 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.36859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36859.log.gz 2024-04-22T20:49:51.278 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114679.log 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.128837.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.128837.log.gz 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62331.log 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.114679.log: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.114679.log.gz 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63650.log 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.62331.log: 20.3% -- replaced with /var/log/ceph/ceph-client.admin.62331.log.gz 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66122.log 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.63650.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.63650.log.gz 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42086.log 2024-04-22T20:49:51.279 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.66122.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.66122.log.gz 2024-04-22T20:49:51.280 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143600.log 2024-04-22T20:49:51.280 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.0.42086.log: 43.9% -- replaced with /var/log/ceph/ceph-client.0.42086.log.gz 2024-04-22T20:49:51.280 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144428.log 2024-04-22T20:49:51.280 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.143600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.143600.log.gz 2024-04-22T20:49:51.280 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84638.log 2024-04-22T20:49:51.280 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.144428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.144428.log.gz 2024-04-22T20:49:51.280 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.84638.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.84638.log.gz 2024-04-22T20:49:51.520 INFO:teuthology.orchestra.run.smithi164.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-22T20:49:52.014 INFO:teuthology.orchestra.run.smithi136.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-22T20:49:52.238 INFO:teuthology.orchestra.run.smithi164.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-22T20:49:53.873 INFO:teuthology.orchestra.run.smithi136.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-22T20:49:54.572 INFO:teuthology.orchestra.run.smithi136.stderr: 92.6% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-04-22T20:49:57.628 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-22T20:49:57.865 INFO:teuthology.orchestra.run.smithi164.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-22T20:49:59.385 INFO:teuthology.orchestra.run.smithi136.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-22T20:50:00.034 INFO:teuthology.orchestra.run.smithi164.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-22T20:50:00.581 INFO:teuthology.orchestra.run.smithi164.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-22T20:50:02.883 INFO:teuthology.orchestra.run.smithi164.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-22T20:50:02.885 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-04-22T20:50:02.885 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m11.933s 2024-04-22T20:50:02.885 INFO:teuthology.orchestra.run.smithi164.stderr:user 0m37.907s 2024-04-22T20:50:02.885 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m1.399s 2024-04-22T20:50:03.644 INFO:teuthology.orchestra.run.smithi136.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-22T20:50:07.873 INFO:teuthology.orchestra.run.smithi136.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-22T20:50:07.875 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-22T20:50:07.875 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m16.963s 2024-04-22T20:50:07.875 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m50.702s 2024-04-22T20:50:07.875 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m1.960s 2024-04-22T20:50:07.876 INFO:tasks.ceph:Archiving logs... 2024-04-22T20:50:07.876 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/remote/smithi136/log 2024-04-22T20:50:07.877 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T20:50:14.781 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/remote/smithi164/log 2024-04-22T20:50:14.782 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-22T20:50:20.759 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-22T20:50:20.771 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-22T20:50:20.772 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T20:50:20.774 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-22T20:50:20.878 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-22T20:50:20.890 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-22T20:50:20.890 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T20:50:20.893 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-22T20:50:20.922 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-04-22T20:50:20.923 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-04-22T20:50:20.926 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T20:50:20.926 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-04-22T20:50:20.927 INFO:teuthology.orchestra.run.smithi136.stdout:^- hv01.front.sepia.ceph.com 2 6 377 34 +2385us[+2385us] +/- 574ms 2024-04-22T20:50:20.927 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -587us[ -572us] +/- 70ms 2024-04-22T20:50:20.927 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +533us[ +533us] +/- 62ms 2024-04-22T20:50:20.927 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T20:50:20.928 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-22T20:50:20.929 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-04-22T20:50:20.929 INFO:teuthology.orchestra.run.smithi164.stdout:^- hv01.front.sepia.ceph.com 2 7 377 95 +2461us[+2461us] +/- 573ms 2024-04-22T20:50:20.929 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -542us[ -543us] +/- 70ms 2024-04-22T20:50:20.929 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 32 +628us[ +628us] +/- 62ms 2024-04-22T20:50:20.929 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-22T20:50:20.929 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-22T20:50:20.942 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-22T20:50:20.942 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-22T20:50:20.971 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T20:50:21.050 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-22T20:50:21.126 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T20:50:21.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T20:50:21.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-22T20:50:21.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T20:50:21.207 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-22T20:50:21.219 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-22T20:50:21.263 INFO:teuthology.task.internal:Duration was 4019.441272 seconds 2024-04-22T20:50:21.264 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-22T20:50:21.296 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-22T20:50:21.297 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T20:50:21.299 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-22T20:50:21.346 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T20:50:21.349 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-22T20:50:21.808 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-22T20:50:21.809 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-04-22T20:50:21.809 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T20:50:21.841 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-04-22T20:50:21.841 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-22T20:50:21.882 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-22T20:50:21.883 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T20:50:21.885 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-22T20:50:21.960 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-22T20:50:21.960 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T20:50:21.963 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-22T20:50:22.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-22T20:50:22.121 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-22T20:50:22.122 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T20:50:22.157 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-22T20:50:22.194 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-22T20:50:22.214 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T20:50:22.217 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-22T20:50:22.249 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-04-22T20:50:22.270 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-04-22T20:50:22.291 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T20:50:22.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T20:50:22.330 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-22T20:50:22.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-22T20:50:22.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-22T20:50:22.370 INFO:teuthology.task.internal:Transferring archived files... 2024-04-22T20:50:22.371 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/remote/smithi136 2024-04-22T20:50:22.371 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T20:50:22.421 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525/remote/smithi164 2024-04-22T20:50:22.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-22T20:50:22.466 INFO:teuthology.task.internal:Removing archive directory... 2024-04-22T20:50:22.466 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T20:50:22.468 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-22T20:50:22.528 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-22T20:50:22.542 INFO:teuthology.task.internal:Not uploading archives. 2024-04-22T20:50:22.542 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-22T20:50:22.588 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-22T20:50:22.589 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T20:50:22.591 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-22T20:50:22.617 INFO:teuthology.orchestra.run.smithi136.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 20:50 /home/ubuntu/cephtest 2024-04-22T20:50:22.619 INFO:teuthology.orchestra.run.smithi164.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 22 20:50 /home/ubuntu/cephtest 2024-04-22T20:50:22.620 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-22T20:50:22.633 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-22T20:50:22.689 INFO:teuthology.nuke:Checking targets against current locks 2024-04-22T20:50:22.716 DEBUG:teuthology.nuke:shortname: smithi136 2024-04-22T20:50:22.716 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T20:50:22.742 DEBUG:teuthology.nuke:shortname: smithi164 2024-04-22T20:50:22.742 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-22T20:50:22.746 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 19:32:49.616780', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJSCkFNzF6ZvVnumrKIjSS6/2qOd60SruNj/CDdUaDztzMmFgqo2+kgONtxBG4u9h14txovdnHTbzSRMJ1F78DU='} 2024-04-22T20:50:22.770 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-04-22_19:31:26-rgw-wip-65551-distro-default-smithi/7668525', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-22 19:32:49.615539', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN1p9ccY9F1b48i+zlg0avenm21Ad/5kMNXPeMFgLexvKge3oAIUxDNWtttOMQZ0FQmAJsbIkeSjsllTxCSr6vI='} 2024-04-22T20:50:22.799 INFO:teuthology.orchestra.console.smithi136:Power off 2024-04-22T20:50:22.800 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T20:50:22.825 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2024-04-22T20:50:22.825 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T20:50:22.844 INFO:teuthology.orchestra.console.smithi164:Power off 2024-04-22T20:50:22.845 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-22T20:50:22.856 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-22T20:50:22.869 DEBUG:teuthology.orchestra.console.smithi164:power off output: Chassis Power Control: Down/Off 2024-04-22T20:50:22.869 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T20:50:22.896 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-22T20:50:26.857 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T20:50:26.897 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T20:50:26.967 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-22T20:50:27.008 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is on 2024-04-22T20:50:30.969 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T20:50:31.009 DEBUG:teuthology.orchestra.console.smithi164:pexpect command: ipmitool -H smithi164.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T20:50:31.079 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-04-22T20:50:31.120 DEBUG:teuthology.orchestra.console.smithi164:check power output: Chassis Power is off 2024-04-22T20:50:31.221 INFO:teuthology.orchestra.console.smithi164:Power off completed 2024-04-22T20:50:35.082 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-22T20:50:35.192 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-04-22T20:50:35.293 INFO:teuthology.orchestra.console.smithi136:Power off completed 2024-04-22T20:50:35.399 INFO:teuthology.run:Summary data: description: rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/https rgw_pool_type/ec s3tests-branch sharding$/{single} striping$/{stripe-equals-chunk} supported-random-distro$/{centos_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} duration: 4019.441272497177 failure_reason: 'Command failed (s3 tests against rgw) on smithi136 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt tox -- -v -m ''not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3''"' flavor: default owner: scheduled_cbodley@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=174e96b77bb14192bf6d271a752eadf8 status: fail success: false 2024-04-22T20:50:35.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-22T20:50:35.523 INFO:teuthology.run:FAIL